Towards Efficacy Hypotheses for Safety Cases
Explore the source record for details and available documents.
assurance case↗
SEARCH · Search NASA
Results for “Safety cases”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Model Transformation for a System of Systems Dependability Safety Case
Software plays an increasingly larger role in all aspects of NASA's science missions. This has been extended to the identification, management and control of faults which affect safety-critical functions and by default, the overall success of the mission. Traditionally, the analysis of fault identification, management and control are hardware based. Due to the increasing complexity of system, there has been a corresponding increase in the complexity in fault management software. The NASA Independent Validation & Verification (IV&V) program is creating processes and procedures to identify, and incorporate safety-critical software requirements along with corresponding software faults so that potential hazards may be mitigated. This Specific to Generic ... A Case for Reuse paper describes the phases of a dependability and safety study which identifies a new, process to create a foundation for reusable assets. These assets support the identification and management of specific software faults and, their transformation from specific to generic software faults. This approach also has applications to other systems outside of the NASA environment. This paper addresses how a mission specific dependability and safety case is being transformed to a generic dependability and safety case which can be reused for any type of space mission with an emphasis on software fault conditions.
Murphy, Judy↗
Deep Isolation-Development of the Safety Case for Disposal of Radioactive Wastes in Horizontal Boreholes - 20028
Deep Isolation has developed a safe, secure, and permanent geological disposal method for high-level waste, including spent nuclear fuel as well as sealed sources and other highly radioactive materials. The method leverages well established directional drilling technology to create horizontal repositories deep underground. The combination of great depth (1-3 km) and the ability to precisely position repositories in a horizontal orientation provides access to geologic strata that are inaccessible for typical mined and deep vertical borehole repositories. A number of potential direct and indirect safety benefits accrue including:1) greater depth below regional freshwater aquifers; 2) increased flexibility and more geologic options for siting; 3) access to deep formations with persistent and sustained reducing conditions. In addition there are a number of safety elements related specifically to the horizontal repository geometry including: 1) passive direction of thermally driven fluid and radionuclide movement away from the vertical access hole and toward the 'dead end' portion of the repository; 2) mitigation of seismic hazards by orienting repositories parallel to local and regional fault structures. In this paper we explore and discuss some of the key technologic, geologic and hydrologic elements that support the deep horizontal borehole safety case. The stalemate seen across the globe on the disposal of spent nuclear fuel and high-level waste can be broken. Deep Isolation offers a novel option for safe, secure, and permanent deep geological disposal of nuclear waste that can be developed as centralized repositories or adapted to smaller regional or site-specific repositories located near waste sources. (authors)
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
Model Transformation for a System of Systems Dependability Safety Case
The presentation reviews the dependability and safety effort of NASA's Independent Verification and Validation Facility. Topics include: safety engineering process, applications to non-space environment, Phase I overview, process creation, sample SRM artifact, Phase I end result, Phase II model transformation, fault management, and applying Phase II to individual projects.
Murphy, Judy↗
Correlated Topics in a Scalable Multidimensional Text Cube: Algorithms and Aviation Safety Case Study
As world-wide air traffic continues to grow even at a modest pace, the overall complexity of the system will increase significantly. This increased complexity can lead to a larger number of fatalities per year even if the extremely low fatality rate that we currently enjoy is maintained. One important source of information about the safety of the aviation system is in Aviation Safety Text Reports which are written by members of the flight crew, air traffic controllers, and other parties involved with the aviation system. These anonymized narrative reports contain fixed-field contextual information about the flight but also contain free-form narratives that describe, in the author s own words, the nature of the safety incident and, in many cases, the contributing factors that led to the safety incident. Several thousand such reports are filed each month, each of which is read and analyzed by highly trained experts. However, it is possible that there are emerging safety issues due to the fact that they may be reported very infrequently and in different contexts with different descriptions. The goal of this research paper is to develop correlated topic models which uncover correlations in the subspaces defined by the intersection of numerous fixed fields and discovered correlated topics. This task requires the discovery of latent topics in the text reports and the creation of a topic cube. Furthermore, because the number of potential cells in the topic cube is very large, we discuss novel methods of pruning the search space in the topic cells, thereby making the analysis feasible. We demonstrate the new algorithms on an analysis of pilot fatigue and its contributing factors, as well as the safety incidents that are correlated with this phenomenon.
Zhao, Bo↗
Bayesian Optimized Deep Ensemble for Uncertainty Quantification of Deep Neural Networks: a System Safety Case Study on Sodium Fast Reactor Thermal Stratification Modeling
Deep neural networks (DNNs) are increasingly important to scientific computing and engineering system simulations. Accurate uncertainty quantification (UQ) for DNNs is critical in safety-sensitive engineering domains. Traditional Deep Ensemble (DE) methods, while easy to implement, frequently suffer from poorly calibrated uncertainty estimates and limited predictive accuracy due to reliance on fixed architectures with varied weight initializations. To address these issues, we introduce a workflow that combines Bayesian Optimization (BO) and DE. The workflow is modular, scalable, and integrates parallel BO initialized with Sobol sequences to individually optimize the hyperparameters of each ensemble member. This method enhances ensemble diversity, improves predictive accuracy, and provides reliable uncertainty estimates. We evaluate the proposed BODE approach in a sodium fast reactor thermal stratification modeling case study, where we used a densely connected convolutional neural network to predict turbulent viscosity during the reactor transient with consideration of data noise. We benchmark its performance against several optimization approaches, including baseline deep ensemble, evolutionary algorithm-optimized ensemble, ensemble formed via random search combined with greedy selection, and a BO ensemble using random initialization. Here, our results demonstrate superior performance of the developed BODE approach. In noise-free scenarios, BODE notably reduces incorrect aleatoric uncertainty and significantly enhances predictive accuracy. Under conditions of 5% and 10% Gaussian noise, BODE adaptively quantifies uncertainty proportional to data noise, achieving up to an 80% reduction in root mean square error compared to baseline methods and producing well-calibrated prediction intervals.
Bayesian optimization↗
Multiple Kernel Learning for Heterogeneous Anomaly Detection: Algorithm and Aviation Safety Case Study
The world-wide aviation system is one of the most complex dynamical systems ever developed and is generating data at an extremely rapid rate. Most modern commercial aircraft record several hundred flight parameters including information from the guidance, navigation, and control systems, the avionics and propulsion systems, and the pilot inputs into the aircraft. These parameters may be continuous measurements or binary or categorical measurements recorded in one second intervals for the duration of the flight. Currently, most approaches to aviation safety are reactive, meaning that they are designed to react to an aviation safety incident or accident. In this paper, we discuss a novel approach based on the theory of multiple kernel learning to detect potential safety anomalies in very large data bases of discrete and continuous data from world-wide operations of commercial fleets. We pose a general anomaly detection problem which includes both discrete and continuous data streams, where we assume that the discrete streams have a causal influence on the continuous streams. We also assume that atypical sequence of events in the discrete streams can lead to off-nominal system performance. We discuss the application domain, novel algorithms, and also discuss results on real-world data sets. Our algorithm uncovers operationally significant events in high dimensional data streams in the aviation industry which are not detectable using state of the art methods
Das, Santanu↗
A Software Safety Risk Taxonomy for Use in Retrospective Safety Cases
No abstract available
Hill, Janice↗
Goal Structured Notation in a Radiation Hardening Safety Case for COTS-Based Spacecraft
A systematic approach is presented to constructing a radiation assurance case using Goal Structured Notation (GSN) for spacecraft containing COTS parts. The GSN paradigm is applied to an SRAM single-event upset experiment board designed to fly on a CubeSat November 2016. Construction of a radiation assurance case without use of hardened parts or extensive radiation testing is discussed.
Assurance Case↗
Safety Case for Geologic Disposal of Radioactive Waste.
Abstract not provided.
Sevougian, Stephen David↗
Digitalization, Management and Synthesis of Thermal Hydraulic Legacy Data Using Dynamical System Scaling
A unique attribute of the nuclear industry is the extent by which designers, developers, operators, and regulators pay attention to demonstrating the safety case. The safety case for nuclear systems is a written, traceable expression of safety, demonstrating the system’s protective measures against uncontrolled radiological releases. The nuclear installation resiliency or safety takes overriding priority over function and performance. The preparation of the safety case heavily relies on extensive experimental campaigns that challenge the target design under a spectrum of postulated accident scenarios. For safety reasons, these experiments are typically conducted in sub-scale test apparatuses intended to reproduce the postulated accident scenarios and event sequences realistically but subject to proven similarity criteria, often referred to as scaling analyses. Since the dawn of the nuclear industry, significant resources have been committed to the construction and operation of such test facilities and a massive amount of data has been generated. Such test data have been and are still used to validate the fundamental assumptions of nuclear power plant phenomena. As we move into the digital world, capturing this multi-decades worth of information in a transparent, easily accessible, and usable manner for the public and industry stakeholders is of paramount importance. Over the last few years FPoliSolutions has been actively developing an enterprise digital data ontology platform (FPoliDON) intended to do just that. The concept was introduced in a paper at the previous NURETH edition and has been significantly enhanced since. With the vision of helping designers use the data to assess their safety case evaluation models, the platform was designed not only to ontologically organize the experimental data, but also to guide the user in the complex analytics associated with the interpretation and use of such test results. A new feature which has been recently added is the ability for the user to access a library of sophisticated mathematical procedures for performing scaling analyses. For this purpose, the Dynamical System Scaling (DSS) procedure invented by Dr. Jose’ Reyes was formulated as one of the services in the digital platform. These methods have been demonstrated to provide an excelled mathematical apparatus to identify similarity criteria or quantify distortions between measured data and the target prototypical conditions. Moreover, DSS can provide a level of synthesis across SETs and IETs which has the promise to yield efficiency in the evaluation code assessment activities. This is often a daunting and expensive task to set up evaluation models that support the safety case of current and future nuclear power plants.
Reyes, Jose↗
Piloted Well Clear Performance Evaluation of Detect and Avoid Systems with Suggestive Guidance
Regulations to establish operational and performance requirements for unmanned aircraft systems (UAS) are being developed by a consortium of government, industry and academic institutions (RTCA, 2013). Those requirements will apply to the new detect-and-avoid (DAA) systems and other equipment necessary to integrate UAS with the United States (U.S) National Airspace System (NAS) and will be determined according to their contribution to the overall safety case. That safety case requires demonstration that DAA-equipped UAS collectively operating in the NAS meet an airspace safety threshold (AST). Several key gaps must be closed in order to link equipment requirements to an airspace safety case. Foremost among these is calculation of the systems risk ratio, the degree to which a particular system mitigates violation of an aircraft separation standard (FAA, 2013). The risk ratio of a DAA system, in combination with risk ratios of other collision mitigation mechanisms, will determine the overall safety of the airspace measured in terms of the number of collisions per flight hour. It is not known what the effectiveness is of a pilot-in-the-loop DAA system or even what parameters of the DAA system most improve the pilots ability to maintain separation. The relationship between the DAA system design and the overall effectiveness of the DAA system that includes the pilot, expressed as a risk ratio, must be determined before DAA operational and performance requirements can be finalized. Much research has been devoted to integrating UAS into non-segregated airspace (Dalamagkidis, 2009, Ostwald, 2007, Gillian, 2012, Hesselink, 2011, Santiago, 2015, Rorie 2015 and 2016). Several traffic displays intended for use as part of a DAA system have gone through human-in-the-loop simulation and flight-testing. Most of these evaluations were part of development programs to produce a deployable system, so it is unclear how to generalize particular aspects of those designs to general requirements for future traffic displays (Calhoun, 2014). Other displays have undergone testing to collect data that may generalize to new displays, but have not been evaluated in the context of the development of an overall safety case for UAS equipped with DAA systems in the NAS (Bell, 2012). Other research efforts focus on DAA surveillance performance and separation standards. Together with this work, they are expected to facilitate validation of the airspace safety case (Park, 2014 and Johnson, 2015). The contribution of the present work is to quantify the effectiveness of the pilot-automation system to remain well clear as a function of display features and surveillance sensor error. This quantification will help enable selection of a minimum set of DAA design features that meets the AST, a set that may not be unique for all UAS platforms. A second objective is to collect and analyze pilot performance parameters that will improve the modeling of overall DAA system performance in non-human-in-the-loop simulations. Simulating the DAA-equipped UAS in such batch experiments will allow investigation of a much larger number of encounters than is possible in human simulations. This capability is necessary to demonstrate that a particular set of DAA requirements meets the AST under all foreseeable operational conditions.
detect and avoid↗
A Taxonomy of Fallacies in System Safety Arguments
Safety cases are gaining acceptance as assurance vehicles for safety-related systems. A safety case documents the evidence and argument that a system is safe to operate; however, logical fallacies in the underlying argument may undermine a system s safety claims. Removing these fallacies is essential to reduce the risk of safety-related system failure. We present a taxonomy of common fallacies in safety arguments that is intended to assist safety professionals in avoiding and detecting fallacious reasoning in the arguments they develop and review. The taxonomy derives from a survey of general argument fallacies and a separate survey of fallacies in real-world safety arguments. Our taxonomy is specific to safety argumentation, and it is targeted at professionals who work with safety arguments but may lack formal training in logic or argumentation. We discuss the rationale for the selection and categorization of fallacies in the taxonomy. In addition to its applications to the development and review of safety cases, our taxonomy could also support the analysis of system failures and promote the development of more robust safety case patterns.
Greenwell, William S.↗
Evaluation of AI-Enabled Digital Documented Safety Analysis: A Case Study
Safety basis documentation development and review under U.S. Department of Energy (DOE) authorization have emerged as critical constraint throttling deployment of advanced nuclear reactors, with traditional processes demanding extraordinary resource investment that delays the delivery of these technologies. Traditional Documented Safety Analysis (DSA) processes rely on static documents with limited traceability [U.S. DOE]. The regulatory review and engagement processes are similarly constrained, often requiring significant effort and extensive manual verification. The scale of this challenge is exemplified by the U.S. Nuclear Regulatory Commission (NRC) review of the NuScale application, which required over 250,000 staff hours and the evaluation of approximately two million pages of documentation [Bergman 2021]. The volume and complexity of information within nuclear licensing applications or authorization reviews demands innovative approaches to document generation and data management.
22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
Dynamic Assurance Cases: Closing the Loop Between Design and Operational Assurance
Safety cases are typically used to certify systems at design time as being approved for operation. However, systems typically undergo changes in operation. We present ongoing work extending the safety case concept to account for operational updates.
Safety cases↗
Model-Driven Development of Safety Architectures
We describe the use of model-driven development for safety assurance of a pioneering NASA flight operation involving a fleet of small unmanned aircraft systems (sUAS) flying beyond visual line of sight. The central idea is to develop a safety architecture that provides the basis for risk assessment and visualization within a safety case, the formal justification of acceptable safety required by the aviation regulatory authority. A safety architecture is composed from a collection of bow tie diagrams (BTDs), a practical approach to manage safety risk by linking the identified hazards to the appropriate mitigation measures. The safety justification for a given unmanned aircraft system (UAS) operation can have many related BTDs. In practice, however, each BTD is independently developed, which poses challenges with respect to incremental development, maintaining consistency across different safety artifacts when changes occur, and in extracting and presenting stakeholder specific information relevant for decision making. We show how a safety architecture reconciles the various BTDs of a system, and, collectively, provide an overarching picture of system safety, by considering them as views of a unified model. We also show how it enables model-driven development of BTDs, replete with validations, transformations, and a range of views. Our approach, which we have implemented in our toolset, AdvoCATE, is illustrated with a running example drawn from a real UAS safety case. The models and some of the innovations described here were instrumental in successfully obtaining regulatory flight approval.
Safety case↗