Search NASA⌕ Search

SEARCH · Search NASA

Results for “assurance argument”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Defining Baconian Probability for Use in Assurance Argumentation

The use of assurance cases (e.g., safety cases) in certification raises questions about confidence in assurance argument claims. Some researchers propose to assess confidence in assurance cases using Baconian induction. That is, a writer or analyst (1) identifies defeaters that might rebut or undermine each proposition in the assurance argument and (2) determines whether each defeater can be dismissed or ignored and why. Some researchers also propose denoting confidence using the counts of defeaters identified and eliminated-which they call Baconian probability-and performing arithmetic on these measures. But Baconian probabilities were first defined as ordinal rankings which cannot be manipulated arithmetically. In this paper, we recount noteworthy definitions of Baconian induction, review proposals to assess confidence in assurance claims using Baconian probability, analyze how these comport with or diverge from the original definition, and make recommendations for future practice.

Graydon, Patrick J.↗
NASA NTRS2018

The Simple Assurance Argument Interchange Format (SAAIF) Manual

This document describes the Simple Assurance Argument Interchange Format, a proposed meta-model for describing structured assurance arguments. We describe the syntax and semantics of the model elements, compare the meta-model to existing argument formats, and give an example to illustrate its use.

Graydon, Patrick J.↗
NASA NTRS2016

An Investigation of Proposed Techniques for Quantifying Confidence in Assurance Arguments

The use of safety cases in certification raises the question of assurance argument sufficiency and the issue of confidence (or uncertainty) in the argument's claims. Some researchers propose to model confidence quantitatively and to calculate confidence in argument conclusions. We know of little evidence to suggest that any proposed technique would deliver trustworthy results when implemented by system safety practitioners. Proponents do not usually assess the efficacy of their techniques through controlled experiment or historical study. Instead, they present an illustrative example where the calculation delivers a plausible result. In this paper, we review current proposals, claims made about them, and evidence advanced in favor of them. We then show that proposed techniques can deliver implausible results in some cases. We conclude that quantitative confidence techniques require further validation before they should be recommended as part of the basis for deciding whether an assurance argument justifies fielding a critical system.

Graydon, Patrick J.↗
NASA NTRS2015

Formal Assurance Arguments: A Solution In Search of a Problem?

An assurance case comprises evidence and argument showing how that evidence supports assurance claims (e.g., about safety or security). It is unsurprising that some computer scientists have proposed formalizing assurance arguments: most associate formality with rigor. But while engineers can sometimes prove that source code refines a formal specification, it is not clear that formalization will improve assurance arguments or that this benefit is worth its cost. For example, formalization might reduce the benefits of argumentation by limiting the audience to people who can read formal logic. In this paper, we present (1) a systematic survey of the literature surrounding formal assurance arguments, (2) an analysis of errors that formalism can help to eliminate, (3) a discussion of existing evidence, and (4) suggestions for experimental work to definitively answer the question.

Graydon, Patrick J.↗
NASA NTRS2017

Assurance Arguments for the Non-Graphically-Inclined: Two Approaches

We introduce and discuss two approaches to presenting assurance arguments. One approach is based on a monograph structure, while the other is based on a tabular structure. In today's research and academic setting, assurance cases often use a graphical notation; however for people who are not graphically inclined, these notations can be difficult to read. This document proposes, outlines, explains, and presents examples of two non-graphical assurance argument notations that may be appropriate for non-graphically-inclined readers and also provide argument writers with freedom to add details and manipulate an argument in multiple ways.

Heavner, Emily↗
NASA NTRS2015

Argument-Based Airworthiness Assurance of Small UAS

Presently, there are three avenues by which Unmanned Aircraft System (UAS) operations are authorized in the U.S. National Airspace System (NAS): obtaining either (i) a certificate of authorization (COA), or (ii) a special airworthiness certificate (SAC) in either the experimental, or the restricted category, or (iii) an exemption from an airworthiness certificate together with a civil COA. The first is meant primarily for public entities, such as NASA; the remaining two are the only available means for civil UAS operations. Recently, the Federal Aviation Administration (FAA) has also proposed a regulatory framework targeted for certain small UAS, specifically those weighing 55 pounds or less, although final rulemaking remains pending. We have previously shown how an assurance case can aggregate heterogeneous reasoning and safety evidence, with application to UAS safety. In this paper, we describe how assurance cases can serve as a common framework to justify overall system safety, unifying both operational aspects and airworthiness, in particular system design assurance. We also show how this approach can coexist with, and augment, existing safety analysis processes and best-practices, by transforming the artifacts they produce into structured assurance arguments. To illustrate the applicability and utility of our approach, we have been applying it for the design assurance of an unmanned rotorcraft system, intended for precision agriculture operations, as part of the NASA Unmanned Aircraft System (UAS) Integration in the National Airspace System (NAS) project.

Rotocraft↗
NASA NTRSDate not supplied

Overarching Properties as Means of Compliance: An Industrial Case Study

The Overarching Properties (OPs) have been created by an inter-national working group and are being evaluated by the National Aeronautics and Space Administration (NASA), the Federal Aviation Administration (FAA), industry, and other certifying agencies in an effort to streamline certification processes. Their intent is to facilitate the use of alternative approaches and to al-low flexibility to combine the system, software, and complex hardware certification. The hope is that the FAA may eventually establish an Advisory Circular that offers the OPs as a Means of Compliance (MoC) for software approval (and eventually systems and hardware) by showing the product possesses the three OPs: Intent (specification of the intended behavior), Correctness (implementation of the intended behavior) and Innocuity (safety of unintended behavior). In the certification community, there is still a concern about the practicability of using such high level properties in certification. This paper aims to address that concern by showing possession of the OPs in an industrial case study using assurance arguments. The two main contributions of this paper are: a certification process based on OPs as Means of Compliance, and a certification argument for an on-board physical model of an UAV, as industrial example. We pro-pose a hybrid approach for the certification process that combines OPs with existing certification standards. Thus, OPs can be used for parts of a system that uses technologies that are not supported by current standards or for which existing standards require additional effort without commensurate additional safety assurance.

certification↗
NASA NTRSDate not supplied

Guiding Integration of Formal Verification in Assurance Cases

Assurance cases are being increasingly acknowledged as away to build trust in complex systems with autonomous capabilities. An assurance case is a comprehensive, defensible, and valid justification that a system will function as intended for a specific mission and operating environment. Formal verification is often reserved for the most critical components of such systems. However, formal verification tools are often complex, and their usage is subject to many constraints and contextual dependencies. This can raise challenges both for performing the verification as well as reflecting the verification results appropriately in the assurance case, especially for non-expert users of the verification tool. To address these challenges, we present a tool-supported methodology for integrating formal verification results in an assurance case by capturing key verification method information in a rigorously constructed assurance case. In particular, we capture the tool specification in terms of its inputs, outputs, and assurance constraints as assumptions over inputs and guarantees provided over its outputs. The tool specification is parametrized over the inputs and outputs to both guide the intended application of the tool, as well as to check that the tool has been applied following the stated assumptions and that the guarantees hold. We define a generic tool assurance argument pattern that enables integration of the verification results in the assurance case by allowing custom refinement and automated instantiation for each tool use. We demonstrate our methodology on two formal verification tools and their applications to the verification of neural network properties for the aircraft domain.

Assurance Cases↗
NASA NTRS2012

Comparison of Aircraft Models and Integration Schemes for Interval Management in the TRACON

Reusable models of common elements for communication, computation, decision and control in air traffic management are necessary in order to enable simulation, analysis and assurance of emergent properties, such as safety and stability, for a given operational concept. Uncertainties due to faults, such as dropped messages, along with non-linearities and sensor noise are an integral part of these models, and impact emergent system behavior. Flight control algorithms designed using a linearized version of the flight mechanics will exhibit error due to model uncertainty, and may not be stable outside a neighborhood of the given point of linearization. Moreover, the communication mechanism by which the sensed state of an aircraft is fed back to a flight control system (such as an ADS-B message) impacts the overall system behavior; both due to sensor noise as well as dropped messages (vacant samples). Additionally simulation of the flight controller system can exhibit further numerical instability, due to selection of the integration scheme and approximations made in the flight dynamics. We examine the theoretical and numerical stability of a speed controller under the Euler and Runge-Kutta schemes of integration, for the Maintain phase for a Mid-Term (2035-2045) Interval Management (IM) Operational Concept for descent and landing operations. We model uncertainties in communication due to missed ADS-B messages by vacant samples in the integration schemes, and compare the emergent behavior of the system, in terms of stability, via the boundedness of the final system state. Any bound on the errors incurred by these uncertainties will play an essential part in a composable assurance argument required for real-time, flight-deck guidance and control systems,. Thus, we believe that the creation of reusable models, which possess property guarantees, such as safety and stability, is an innovative and essential requirement to assessing the emergent properties of novel airspace concepts of operation.

Neogi, Natasha↗
NASA NTRS2014

Querying Safety Cases

Querying a safety case to show how the various stakeholders' concerns about system safety are addressed has been put forth as one of the benefits of argument-based assurance (in a recent study by the Health Foundation, UK, which reviewed the use of safety cases in safety-critical industries). However, neither the literature nor current practice offer much guidance on querying mechanisms appropriate for, or available within, a safety case paradigm. This paper presents a preliminary approach that uses a formal basis for querying safety cases, specifically Goal Structuring Notation (GSN) argument structures. Our approach semantically enriches GSN arguments with domain-specific metadata that the query language leverages, along with its inherent structure, to produce views. We have implemented the approach in our toolset AdvoCATE, and illustrate it by application to a fragment of the safety argument for an Unmanned Aircraft System (UAS) being developed at NASA Ames. We also discuss the potential practical utility of our query mechanism within the context of the existing framework for UAS safety assurance.

Safety Case↗
NASA NTRSDate not supplied

False Beliefs about the Overarching Properties and Overarching Properties Related Arguments

False beliefs about new ideas are not new. They happen all the time, for a wide variety of reasons. Although the Overarching Properties are solidly grounded in time-honored principles, they constitute a novel expression of those ideas, and newness-based false beliefs about them are to be expected. Similarly, although Overarching Properties Related Arguments are solidly grounded in thousands of years of study of the principles of argumentation, they introduce concepts and applications that are novel to many engineers and engineering managers. So, false beliefs about them are also to be expected. Indeed, false beliefs have arisen and are spreading in the wild about both the Overarching Properties and Overarching Properties Related Arguments. This paper seeks to dispel three known false beliefs about each concept.

overarching properties↗
NASA NTRSDate not supplied

Leveling Arguments: Easier Said Than Done

The aerospace industry’s current desire to rapidly adopt new technologies is inconsistent with the traditional approach to creating new aerospace certification standards. To address this, an FAA-sponsored international Overarching Properties Working Group comprising industry representatives and certifying agencies has introduced the concept of Overarching Properties (OPs). OPs provide a foundation for proposing alternative Means of Compliance (MoC) by demonstrating a product's possession of the Intent, Correctness, and Innocuity properties. Handling criticality levels with the OPs is in its infancy but still required to integrate alternative MoC into the existing aerospace certification infrastructure. This paper examines the impact of criticality levels on the creation and evaluation of OP-related arguments (OPRAs), specifically focusing on the Auxiliary Power Unit (APU) system. The study highlights the need for tailored OPRA creation based on assigned criticality levels and proposes self-assessment techniques to strengthen arguments. This research serves as a stepping stone in exploring criticality levels for OPs and bridging the gap in aerospace certification practices.

argument↗
NASA NTRS2015

AdvoCATE - User Guide

The basic vision of AdvoCATE is to automate the creation, manipulation, and management of large-scale assurance cases based on a formal theory of argument structures. Its main purposes are for creating and manipulating argument structures for safety assurance cases using the Goal Structuring Notation (GSN), and as a test bed and proof-of-concept for the formal theory of argument structures. AdvoCATE is available for Windows 7, Macintosh OSX, and Linux. Eventually, AdvoCATE will serve as a dashboard for safety related information and provide an infrastructure for safety decisions and management.

Safety Case↗
NASA NTRS2020

IV&V Assurance Case Design for Artemis II

As human-rated missions like those in NASA’s Artemis program continue to grow in both size and complexity, and the role of software in achieving mission objectives expands dramatically, NASA’s Independent Verification and Validation (IV&V) Teams face evolving challenges in assuring the safety and performance of the safety- and mission-critical embedded software that is essential to landing astronauts on the surface of the Moon by 2024. Key among these challenges is IV&V’s desire to present a cohesive, integrated assurance statement to its stakeholders that encapsulates and summarizes our assurance positions across the integrated Artemis systems and their combined role in support of a safe and successful flight. In order to meet this challenge, the IV&V Teams have begun a transition to using formal assurance case concepts and documentation in the Goal Structuring Notation (GSN) to build an argument in support of software assurance. IV&V recognizes significant benefits to the logical argumentation structure provided by assurance cases and GSN over our current practices for documenting and managing assurance claims. In order to reap these benefits, IV&V is integrating the use of assurance case concepts with our paradigm of follow-the-risk capability based assurance. Because of this, assurance cases created and used by IV&V are distinct from the sort of assurance case created by a development project or embedded software assurance organization. IV&V’s assurance cases depend much less upon standards and regulations, and more on evidence captured by IV&V regarding the environment, requirements, design, and implementation. IV&V constructs an independent network of claims based on an independent decomposition of arguments. Based upon the risk posture of these claims and their associated software and software artifacts, IV&V then develops and executes engineering analyses and testing, which provide evidence to either support or refute the claim. This emerging risk-informed assurance case methodology is being put into practice as IV&V plans for support of the Artemis II mission, the first flight of the Orion capsule and Space Launch System with astronauts on board.

Gerek Whitman↗
NASA NTRS2020

IV&V Assurance Case Design for Artemis II

As human-rated missions like those in NASA's Artemis program continue to grow in both size and complexity, and the role of software in achieving mission objectives expands dramatically, NASA's Independent Verification and Validation (IV&V) Teams face evolving challenges in assuring the safety and performance of the safety- and mission-critical embedded software that is essential to landing astronauts on the surface of the Moon by 2024. Key among these challenges is IV&V's desire to present a cohesive, integrated assurance statement to its stakeholders that encapsulates and summarizes our assurance positions across the integrated Artemis systems and their combined role in support of a safe and successful flight. In order to meet this challenge, the IV&V Teams have begun a transition to using formal assurance case concepts and documentation in the Goal Structuring Notation (GSN) to build an argument in support of software assurance. IV&V recognizes significant benefits to the logical argumentation structure provided by assurance cases and GSN over our current practices for documenting and managing assurance claims. In order to reap these benefits, IV&V is integrating the use of assurance case concepts with our paradigm of follow-the-risk capability based assurance. Because of this, assurance cases created and used by IV&V are distinct from the sort of assurance case created by a development project or embedded software assurance organization. IV&V's assurance cases depend much less upon standards and regulations, and more on evidence captured by IV&V regarding the environment, requirements, design, and implementation. IV&V constructs an independent network of claims based on an independent decomposition of arguments. Based upon the risk posture of these claims and their associated software and software artifacts, IV&V then develops and executes engineering analyses and testing, which provide evidence to either support or refute the claim. This emerging risk-informed assurance case methodology is being put into practice as IV&V plans for support of the Artemis II mission, the first flight of the Orion capsule and Space Launch System with astronauts on board.

Whitman, Gerek↗
NASA NTRS2015

Formal Foundations for Hierarchical Safety Cases

Safety cases are increasingly being required in many safety-critical domains to assure, using structured argumentation and evidence, that a system is acceptably safe. However, comprehensive system-wide safety arguments present appreciable challenges to develop, understand, evaluate, and manage, partly due to the volume of information that they aggregate, such as the results of hazard analysis, requirements analysis, testing, formal verification, and other engineering activities. Previously, we have proposed hierarchical safety cases, hicases, to aid the comprehension of safety case argument structures. In this paper, we build on a formal notion of safety case to formalise the use of hierarchy as a structuring technique, and show that hicases satisfy several desirable properties. Our aim is to provide a formal, theoretical foundation for safety cases. In particular, we believe that tools for high assurance systems should be granted similar assurance to the systems to which they are applied. To this end, we formally specify and prove the correctness of key operations for constructing and managing hicases, which gives the specification for implementing hicases in AdvoCATE, our toolset for safety case automation. We motivate and explain the theory with the help of a simple running example, extracted from a real safety case and developed using AdvoCATE.

Hierarchy↗