Search NASA⌕ Search

SEARCH · Search NASA

Results for “Assurance Case”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2015

Understanding and Evaluating Assurance Cases

Assurance cases are a method for providing assurance for a system by giving an argument to justify a claim about the system, based on evidence about its design, development, and tested behavior. In comparison with assurance based on guidelines or standards (which essentially specify only the evidence to be produced), the chief novelty in assurance cases is provision of an explicit argument. In principle, this can allow assurance cases to be more finely tuned to the specific circumstances of the system, and more agile than guidelines in adapting to new techniques and applications. The first part of this report (Sections 1-4) provides an introduction to assurance cases. Although this material should be accessible to all those with an interest in these topics, the examples focus on software for airborne systems, traditionally assured using the DO-178C guidelines and its predecessors. A brief survey of some existing assurance cases is provided in Section 5. The second part (Section 6) considers the criteria, methods, and tools that may be used to evaluate whether an assurance case provides sufficient confidence that a particular system or service is fit for its intended use. An assurance case cannot provide unequivocal "proof" for its claim, so much of the discussion focuses on the interpretation of such less-than-definitive arguments, and on methods to counteract confirmation bias and other fallibilities in human reasoning.

Rushby, John↗
NASA NTRSDate not supplied

Guiding Integration of Formal Verification in Assurance Cases

Assurance cases are being increasingly acknowledged as away to build trust in complex systems with autonomous capabilities. An assurance case is a comprehensive, defensible, and valid justification that a system will function as intended for a specific mission and operating environment. Formal verification is often reserved for the most critical components of such systems. However, formal verification tools are often complex, and their usage is subject to many constraints and contextual dependencies. This can raise challenges both for performing the verification as well as reflecting the verification results appropriately in the assurance case, especially for non-expert users of the verification tool. To address these challenges, we present a tool-supported methodology for integrating formal verification results in an assurance case by capturing key verification method information in a rigorously constructed assurance case. In particular, we capture the tool specification in terms of its inputs, outputs, and assurance constraints as assumptions over inputs and guarantees provided over its outputs. The tool specification is parametrized over the inputs and outputs to both guide the intended application of the tool, as well as to check that the tool has been applied following the stated assumptions and that the guarantees hold. We define a generic tool assurance argument pattern that enables integration of the verification results in the assurance case by allowing custom refinement and automated instantiation for each tool use. We demonstrate our methodology on two formal verification tools and their applications to the verification of neural network properties for the aircraft domain.

Assurance Cases↗
NASA NTRSDate not supplied

Implementing an Objectives-Driven, Risk-Informed, and Case-Assured Approach to Safety and Mission Success at NASA

NASA is developing a “Standard for Assurance of Space Flight Safety and Mission Success” that implements an objectives-driven, risk-informed, and case-assured approach to safety and mission success (S&MS) for NASA space flight programs and projects. The standard aligns with the philosophy of risk leadership that has recently been established in NASA policy to assure acceptable levels of flight crew safety and mission success risk. It is consistent with existing NASA risk management requirements and is compatible with NASA program management and systems engineering requirements. The methodology described in the standard is presented in terms of an S&MS assurance framework that is designed to allow substantial flexibility in the specific means by which programs and projects achieve acceptable mission S&MS risk. Such flexibility is necessary to accommodate the increasingly broad range of acquisition strategies employed by NASA, including commercial transportation services, as well as to accommodate the increasingly rapid evolution of space flight-related technologies and practices. A key feature of the S&MS assurance framework is the specification of S&MS success criteria for each life-cycle review (LCR). The S&MS assurance case is structured around these criteria, the satisfaction of which indicates that the program/project is adhering to the S&MS risk posture. This enables the evolving S&MS assurance case to be used as a fundamental program/project submittal at each LCR, where its inherent structure of argument, supported by evidence, directly supports the evaluation of the program/project with respect to the S&MS success criteria, and by extension, the S&MS risk posture. As such, the S&MS assurance case is integral to program/project systems engineering, risk management, and S&MS oversight activities, and provides the principal basis for S&MS risk acceptance by the Decision Authority throughout the program/project life cycle.

Risk Posture↗
DOE OSTI2024

Implementing an Objectives-Driven, Risk-Informed, and Case-Assured Approach to Safety and Mission Success at NASA

NASA is developing a “Standard for Assurance of Space Flight Safety and Mission Success” that implements an objectives-driven, risk-informed, and case-assured approach to safety and mission success (S&MS) for NASA space flight programs and projects. The standard aligns with the philosophy of risk leadership that has recently been established in NASA policy to assure acceptable levels of flight crew safety and mission success risk. It is consistent with existing NASA risk management requirements and is compatible with NASA program management and systems engineering requirements. The methodology described in the standard is presented in terms of an S&MS assurance framework that is designed to allow substantial flexibility in the specific means by which programs and projects achieve acceptable mission S&MS risk. Such flexibility is necessary to accommodate the increasingly broad range of acquisition strategies employed by NASA, including commercial transportation services, as well as to accommodate the increasingly rapid evolution of space flight-related technologies and practices. A key feature of the S&MS assurance framework is the specification of S&MS success criteria for each life-cycle review (LCR). The S&MS assurance case is structured around these criteria, the satisfaction of which indicates that the program/project is adhering to the S&MS risk posture. This enables the evolving S&MS assurance case to be used as a fundamental program/project submittal at each LCR, where its inherent structure of argument, supported by evidence, directly supports the evaluation of the program/project with respect to the S&MS success criteria, and by extension, the S&MS risk posture. As such, the S&MS assurance case is integral to program/project systems engineering, risk management, and S&MS oversight activities, and provides the principal basis for S&MS risk acceptance by the Decision Authority throughout the program/project life cycle.

42 ENGINEERING↗
NASA NTRS2021

An Assurance Case with a Model at its Core

We describe our pilot study development of an Assurance Case arguing the robustness of a spacecraft demonstration of optical communication. Our Assurance Case addresses the concern that optical communication may be interrupted by the presence of cloud cover, threatening the success of the demonstration. Central to our Assurance Case is its use of a model of atmospheric attenuation to support a key portion of the robustness argument. The conclusion for the demonstration is that its schedule slack plus ability to store data for transmission later accommodates occasional weather-caused atmospheric attenuation.We indicate how the overall structure of the Assurance Case derives from the Objectives Hierarchy set forth in the NASA Reliability and Maintainability standard. We then present the portions of the Assurance Case that argue (1) the atmospheric attenuation model is sufficiently accurate, (2) application of the model shows the desired robustness of the demonstration, and (3) all the model assumptions are satisfied in its application. Lastly, we suggest how an engineering model of the demonstration system and its use could inform the development of the Assurance Case encompassing additional plausible hazards.

DiVenti, Anthony↗
NASA NTRSDate not supplied

Adopting an Objectives-Driven Assurance Case Approach for Achieving Space Flight Mission Planetary Protection Objectives

Traditionally, NASA has utilized prescriptive technical and process requirements to ensure safety and mission assurance performance objectives for space flight missions are achieved. While prescriptive re-quirements may be easier to communicate and manage throughout the systems engineering process, the highly-constrained nature of prescriptive requirements can limit the ability to take advantage of cost-saving opportunities and offer limited ability to explore other options or alternative designs, processes, and methods. It can also be difficult to develop prescriptive requirements for objectives that are prob-abilistic in nature or that cannot be satisfied by direct verification. In contrast, the development of an assurance case allows for a compelling, comprehensible, and valid argument to be developed with support-ing evidence that shows safety and mission assurance objectives have been satisfied. Analogous to how patent applications are constructed for inventions, an assurance case has a high-level claim of meeting a safety and mission assurance objective, followed by a more specific set of sub-claims and technical evidence which supports the claims. The objectives-driven assurance case approach allows for a better understand-ing and exploration of the trade space, more flexibility to balance trades, and the ability to realize and implement technical and process innovations for resource, time, and cost savings. The assurance case is a living case that evolves over the entire program life cycle. Recently, NASA’s Office of Planetary Pro-tection (OPP) has adopted the assurance case approach as an acceptable methodology for demonstrating avoidance of contamination of target solar system bodies explored by NASA space flight missions. This methodology has been incorporated into NASA’s new technical standard for planetary protection and is currently being utilized by the Mars Sample Return campaign for safe sample containment during sample return. This presentation will explore the development and implementation of an assurance case approach in the context of planetary protection, the shift from prescriptive requirements and the ongoing culture change in the technical community, and the support and guidance from NASA’s OPP in adopting the assurance case approach for achieving planetary protection objectives on NASA’s space flight missions.

Elaine Seasly↗
NASA NTRS2015

Current Practices in Constructing and Evaluating Assurance Cases With Applications to Aviation

This report introduces and provides an overview of assurance cases including theory, practice, and evaluation. This report includes a section that introduces the principles, terminology, and history of assurance cases. The core of the report presents twelve example uses of assurance cases from a range of domains, using a novel classification scheme. The report also reviews the state of the art in assurance case evaluation methods.

David J Rinehart↗
NASA NTRS2020

IV&V Assurance Case Design for Artemis II

As human-rated missions like those in NASA’s Artemis program continue to grow in both size and complexity, and the role of software in achieving mission objectives expands dramatically, NASA’s Independent Verification and Validation (IV&V) Teams face evolving challenges in assuring the safety and performance of the safety- and mission-critical embedded software that is essential to landing astronauts on the surface of the Moon by 2024. Key among these challenges is IV&V’s desire to present a cohesive, integrated assurance statement to its stakeholders that encapsulates and summarizes our assurance positions across the integrated Artemis systems and their combined role in support of a safe and successful flight. In order to meet this challenge, the IV&V Teams have begun a transition to using formal assurance case concepts and documentation in the Goal Structuring Notation (GSN) to build an argument in support of software assurance. IV&V recognizes significant benefits to the logical argumentation structure provided by assurance cases and GSN over our current practices for documenting and managing assurance claims. In order to reap these benefits, IV&V is integrating the use of assurance case concepts with our paradigm of follow-the-risk capability based assurance. Because of this, assurance cases created and used by IV&V are distinct from the sort of assurance case created by a development project or embedded software assurance organization. IV&V’s assurance cases depend much less upon standards and regulations, and more on evidence captured by IV&V regarding the environment, requirements, design, and implementation. IV&V constructs an independent network of claims based on an independent decomposition of arguments. Based upon the risk posture of these claims and their associated software and software artifacts, IV&V then develops and executes engineering analyses and testing, which provide evidence to either support or refute the claim. This emerging risk-informed assurance case methodology is being put into practice as IV&V plans for support of the Artemis II mission, the first flight of the Orion capsule and Space Launch System with astronauts on board.

Gerek Whitman↗
NASA NTRS2020

IV&V Assurance Case Design for Artemis II

As human-rated missions like those in NASA's Artemis program continue to grow in both size and complexity, and the role of software in achieving mission objectives expands dramatically, NASA's Independent Verification and Validation (IV&V) Teams face evolving challenges in assuring the safety and performance of the safety- and mission-critical embedded software that is essential to landing astronauts on the surface of the Moon by 2024. Key among these challenges is IV&V's desire to present a cohesive, integrated assurance statement to its stakeholders that encapsulates and summarizes our assurance positions across the integrated Artemis systems and their combined role in support of a safe and successful flight. In order to meet this challenge, the IV&V Teams have begun a transition to using formal assurance case concepts and documentation in the Goal Structuring Notation (GSN) to build an argument in support of software assurance. IV&V recognizes significant benefits to the logical argumentation structure provided by assurance cases and GSN over our current practices for documenting and managing assurance claims. In order to reap these benefits, IV&V is integrating the use of assurance case concepts with our paradigm of follow-the-risk capability based assurance. Because of this, assurance cases created and used by IV&V are distinct from the sort of assurance case created by a development project or embedded software assurance organization. IV&V's assurance cases depend much less upon standards and regulations, and more on evidence captured by IV&V regarding the environment, requirements, design, and implementation. IV&V constructs an independent network of claims based on an independent decomposition of arguments. Based upon the risk posture of these claims and their associated software and software artifacts, IV&V then develops and executes engineering analyses and testing, which provide evidence to either support or refute the claim. This emerging risk-informed assurance case methodology is being put into practice as IV&V plans for support of the Artemis II mission, the first flight of the Orion capsule and Space Launch System with astronauts on board.

Whitman, Gerek↗
NASA NTRS2017

Understanding What It Means for Assurance Cases to "Work"

This report is the result of our year-long investigation into assurance case practices and effectiveness. Assurance cases are a method for working toward acceptable critical system performance. They represent a significant thread of applied assurance methods extending back many decades and being employed in a range of industries and applications. Our research presented in this report includes a literature survey of over 50 sources and interviews with nearly a dozen practitioners in the field. We have organized our results into seven major claimed assurance case benefits and their supporting mechanisms, evidence, counter-evidence, and caveats.

David J Rinehart↗
NASA NTRSDate not supplied

Adopting an Objectives-Driven Assurance Case Approach for Achieving Space Flight Mission Planetary Protection Objectives

Traditionally, the National Aeronautics and Space Administration (NASA) has utilized prescriptive technical and process requirements to ensure safety and mission assurance performance objectives for planetary protection are achieved during space flight missions. While prescriptive requirements may be easier to communicate and manage throughout the systems engineering process, the highly constrained nature of prescriptive requirements can limit the ability to take advantage of cost-saving opportunities and offer limited ability to explore other options or alternative designs, processes, and methods. It can also be difficult to develop prescriptive requirements for objectives that are probabilistic in nature or that cannot be satisfied by direct verification. In contrast, the development of an assurance case allows for a compelling, comprehensible, and valid argument to be developed with supporting evidence that shows safety and mission assurance objectives have been satisfied. Analogous to how patent applications are constructed for inventions, an assurance case has a high-level claim of meeting a safety and mission assurance objective, followed by a more specific set of sub-claims and technical evidence which supports the claims. The objectives-driven assurance case approach allows for a better understanding and exploration of the trade space, more flexibility to balance trades, and the ability to realize and implement technical and process innovations for resource, time, and cost savings. The assurance case is a living case that evolves over the entire program life cycle. Recently, NASA’s Office of Planetary Protection (OPP) has adopted the assurance case approach as an acceptable methodology for demonstrating avoidance of contamination of target solar system bodies explored by NASA space flight missions. This methodology has been incorporated into NASA’s new technical standard for planetary protection and is currently being utilized by the Mars Sample Return campaign for safe sample containment during sample return.

Assurance Case↗
NASA NTRSDate not supplied

Application of Objectives-Driven Assurance Cases to System Development in an Evolving Acquisition Model

System properties such as “safety” and “dependability” cannot, in practice, be proven, and must be argued in an “assurance case” aimed at supporting risk-acceptance decisions that have to be made by system acquirers and/or regulatory authorities. The paper is concerned with applications of the “assurance case” idea early in design and development of new systems, when (apart from dedicated testing) the only available operating experience information derives from previous (non-identical) systems. Much of the discussion is based on an evolving acquisition model at the US National Aeronautics and Space Administration; previously, most major systems were developed in-house, but some major systems will now be developed by and acquired from commercial providers. Key points discussed include the following. (1) By promoting a particular kind of focused discussion between acquirers and providers, the use of assurance cases should be particularly valuable under the new acquisition model. (2) In principle, objectives-driven (sometimes called “performance-based”) approaches to assurance of performance have significant advantages in cases where they are applicable. (3) For truly novel systems, completeness of the safety analysis is a significant issue; it is important for the assurance case to include a commitment by the provider (or applicant) to seriously pursue analysis of operating experience, so that previously unrecognized hazards can be identified and addressed. (4) Inquiries into major accidents often point to deficiencies in management oversight in all parts of the life cycle; management processes need to be addressed in the formulation and the implementation of an assurance case. Under the new acquisition model, these considerations imply a serious reconsideration of the way in which the development process is managed by both providers and acquirers.

Objectives-driven↗
NASA NTRS2016

Goal Structuring Notation in a Radiation Hardening Assurance Case for COTS-Based Spacecraft

A systematic approach is presented to constructing a radiation assurance case using Goal Structuring Notation (GSN) for spacecraft containing COTS parts. The GSN paradigm is applied to an SRAM single-event upset experiment board designed to fly on a CubeSat in January 2017. A custom software language for development of a GSN assurance case is under development at Vanderbilt. Construction of a radiation assurance case without use of hardened parts or extensive radiation testing is discussed.

Radiation Effects Modeling (REM)↗
NASA NTRS2018

A Hybrid Method of Assurance Cases and Testing for Improved Confidence in Autonomous Space Systems

Autonomous systems react intelligently to their environments, making them capable of handling many possible conditions, but challenging to test. We are investigating a new test development method that aims to maximize the confidence to be achieved by combining Assurance Cases with High Throughput Testing (HTT). Assurance Cases, developed for safety-critical systems, are a rigorous argument that the system satisfies a property (e.g., the Mars rover will not tip over during a traverse). They integrate testing, analysis, and environmental and operational assumptions, from which the set of conditions that testing must cover is determined. In our method, information from the Assurance Case is used to determine the test coverage needed, and then input to HTT to generate the minimal test suites needed to provide that coverage.

Huntsberger, Terry↗
NASA NTRS2016

Goal Structuring Notation in a Radiation Hardening Assurance Case for COTS-Based Spacecraft

A systematic approach is presented to constructing a radiation assurance case using Goal Structuring Notation (GSN) for spacecraft containing commercial-off-the-shelf (COTS) parts. The GSN paradigm is applied to an SRAM single-event upset experiment board designed to fly on a CubeSat November 2016. Construction of a radiation assurance case without use of hardened parts or extensive radiation testing is discussed.

Witulski, Arthur↗
NASA NTRS2009

Assurance Cases for Proofs as Evidence

Proof-carrying code (PCC) provides a 'gold standard' for establishing formal and objective confidence in program behavior. However, in order to extend the benefits of PCC - and other formal certification techniques - to realistic systems, we must establish the correspondence of a mathematical proof of a program's semantics and its actual behavior. In this paper, we argue that assurance cases are an effective means of establishing such a correspondence. To this end, we present an assurance case pattern for arguing that a proof is free from various proof hazards. We also instantiate this pattern for a proof-based mechanism to provide evidence about a generic medical device software.

Chaki, Sagar↗