SEARCH · Search NASA

Results for “Assurance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

NASA NTRS2016

Technical Reference Suite Addressing Challenges of Providing Assurance for Fault Management Architectural Design

Research into complexities of software systems Fault Management (FM) and how architectural design decisions affect safety, preservation of assets, and maintenance of desired system functionality has coalesced into a technical reference (TR) suite that advances the provision of safety and mission assurance. The NASA Independent Verification and Validation (IV&V) Program, with Software Assurance Research Program support, extracted FM architectures across the IV&V portfolio to evaluate robustness, assess visibility for validation and test, and define software assurance methods applied to the architectures and designs. This investigation spanned IV&V projects with seven different primary developers, a wide range of sizes and complexities, and encompassed Deep Space Robotic, Human Spaceflight, and Earth Orbiter mission FM architectures. The initiative continues with an expansion of the TR suite to include Launch Vehicles, adding the benefit of investigating differences intrinsic to model-based FM architectures and insight into complexities of FM within an Agile software development environment, in order to improve awareness of how nontraditional processes affect FM architectural design and system health management. The identification of particular FM architectures, visibility, and associated IV&V techniques provides a TR suite that enables greater assurance that critical software systems will adequately protect against faults and respond to adverse conditions. Additionally, the role FM has with regard to strengthened security requirements, with potential to advance overall asset protection of flight software systems, is being addressed with the development of an adverse conditions database encompassing flight software vulnerabilities. Capitalizing on the established framework, this TR suite provides assurance capability for a variety of FM architectures and varied development approaches. Research results are being disseminated across NASA, other agencies, and the software community. This paper discusses the findings and TR suite informing the FM domain in best practices for FM architectural design, visibility observations, and methods employed for IV&V and mission assurance.

Fitz, Rhonda↗

NASA NTRS2016

Assurance of Fault Management: Risk-Significant Adverse Condition Awareness

Fault Management (FM) systems are ranked high in risk-based assessment of criticality within flight software, emphasizing the importance of establishing highly competent domain expertise to provide assurance for NASA projects, especially as spaceflight systems continue to increase in complexity. Insight into specific characteristics of FM architectures seen embedded within safety- and mission-critical software systems analyzed by the NASA Independent Verification Validation (IVV) Program has been enhanced with an FM Technical Reference (TR) suite. Benefits are aimed beyond the IVV community to those that seek ways to efficiently and effectively provide software assurance to reduce the FM risk posture of NASA and other space missions. The identification of particular FM architectures, visibility, and associated IVV techniques provides a TR suite that enables greater assurance that critical software systems will adequately protect against faults and respond to adverse conditions. The role FM has with regard to overall asset protection of flight software systems is being addressed with the development of an adverse condition (AC) database encompassing flight software vulnerabilities.Identification of potential off-nominal conditions and analysis to determine how a system responds to these conditions are important aspects of hazard analysis and fault management. Understanding what ACs the mission may face, and ensuring they are prevented or addressed is the responsibility of the assurance team, which necessarily should have insight into ACs beyond those defined by the project itself. Research efforts sponsored by NASAs Office of Safety and Mission Assurance defined terminology, categorized data fields, and designed a baseline repository that centralizes and compiles a comprehensive listing of ACs and correlated data relevant across many NASA missions. This prototype tool helps projects improve analysis by tracking ACs, and allowing queries based on project, mission type, domain component, causal fault, and other key characteristics. The repository has a firm structure, initial collection of data, and an interface established for informational queries, with plans for integration within the Enterprise Architecture at NASA IVV, enabling support and accessibility across the Agency. The development of an improved workflow process for adaptive, risk-informed FM assurance is currently underway.

Software Verification & Validation↗

NASA NTRS2021

An Assurance Case with a Model at its Core

We describe our pilot study development of an Assurance Case arguing the robustness of a spacecraft demonstration of optical communication. Our Assurance Case addresses the concern that optical communication may be interrupted by the presence of cloud cover, threatening the success of the demonstration. Central to our Assurance Case is its use of a model of atmospheric attenuation to support a key portion of the robustness argument. The conclusion for the demonstration is that its schedule slack plus ability to store data for transmission later accommodates occasional weather-caused atmospheric attenuation.We indicate how the overall structure of the Assurance Case derives from the Objectives Hierarchy set forth in the NASA Reliability and Maintainability standard. We then present the portions of the Assurance Case that argue (1) the atmospheric attenuation model is sufficiently accurate, (2) application of the model shows the desired robustness of the demonstration, and (3) all the model assumptions are satisfied in its application. Lastly, we suggest how an engineering model of the demonstration system and its use could inform the development of the Assurance Case encompassing additional plausible hazards.

DiVenti, Anthony↗

NASA NTRS2015

Argument-Based Airworthiness Assurance of Small UAS

Presently, there are three avenues by which Unmanned Aircraft System (UAS) operations are authorized in the U.S. National Airspace System (NAS): obtaining either (i) a certificate of authorization (COA), or (ii) a special airworthiness certificate (SAC) in either the experimental, or the restricted category, or (iii) an exemption from an airworthiness certificate together with a civil COA. The first is meant primarily for public entities, such as NASA; the remaining two are the only available means for civil UAS operations. Recently, the Federal Aviation Administration (FAA) has also proposed a regulatory framework targeted for certain small UAS, specifically those weighing 55 pounds or less, although final rulemaking remains pending. We have previously shown how an assurance case can aggregate heterogeneous reasoning and safety evidence, with application to UAS safety. In this paper, we describe how assurance cases can serve as a common framework to justify overall system safety, unifying both operational aspects and airworthiness, in particular system design assurance. We also show how this approach can coexist with, and augment, existing safety analysis processes and best-practices, by transforming the artifacts they produce into structured assurance arguments. To illustrate the applicability and utility of our approach, we have been applying it for the design assurance of an unmanned rotorcraft system, intended for precision agriculture operations, as part of the NASA Unmanned Aircraft System (UAS) Integration in the National Airspace System (NAS) project.

Rotocraft↗

NASA NTRS2017

Risk-Significant Adverse Condition Awareness Strengthens Assurance of Fault Management Systems

As spaceflight systems increase in complexity, Fault Management (FM) systems are ranked high in risk-based assessment of software criticality, emphasizing the importance of establishing highly competent domain expertise to provide assurance. Adverse conditions (ACs) and specific vulnerabilities encountered by safety- and mission-critical software systems have been identified through efforts to reduce the risk posture of software-intensive NASA missions. Acknowledgement of potential off-nominal conditions and analysis to determine software system resiliency are important aspects of hazard analysis and FM. A key component of assuring FM is an assessment of how well software addresses susceptibility to failure through consideration of ACs. Focus on significant risk predicted through experienced analysis conducted at the NASA Independent Verification & Validation (IV&V) Program enables the scoping of effective assurance strategies with regard to overall asset protection of complex spaceflight as well as ground systems. Research efforts sponsored by NASAs Office of Safety and Mission Assurance (OSMA) defined terminology, categorized data fields, and designed a baseline repository that centralizes and compiles a comprehensive listing of ACs and correlated data relevant across many NASA missions. This prototype tool helps projects improve analysis by tracking ACs and allowing queries based on project, mission type, domain/component, causal fault, and other key characteristics. Vulnerability in off-nominal situations, architectural design weaknesses, and unexpected or undesirable system behaviors in reaction to faults are curtailed with the awareness of ACs and risk-significant scenarios modeled for analysts through this database. Integration within the Enterprise Architecture at NASA IV&V enables interfacing with other tools and datasets, technical support, and accessibility across the Agency. This paper discusses the development of an improved workflow process utilizing this database for adaptive, risk-informed FM assurance that critical software systems will safely and securely protect against faults and respond to ACs in order to achieve successful missions.

IV&V↗

NASA NTRS2017

Risk-Significant Adverse Condition Awareness Strengthens Assurance of Fault Management Systems

As spaceflight systems increase in complexity, Fault Management (FM) systems are ranked high in risk-based assessment of software criticality, emphasizing the importance of establishing highly competent domain expertise to provide assurance. Adverse conditions (ACs) and specific vulnerabilities encountered by safety- and mission-critical software systems have been identified through efforts to reduce the risk posture of software-intensive NASA missions. Acknowledgement of potential off-nominal conditions and analysis to determine software system resiliency are important aspects of hazard analysis and FM. A key component of assuring FM is an assessment of how well software addresses susceptibility to failure through consideration of ACs. Focus on significant risk predicted through experienced analysis conducted at the NASA Independent Verification Validation (IVV) Program enables the scoping of effective assurance strategies with regard to overall asset protection of complex spaceflight as well as ground systems. Research efforts sponsored by NASA's Office of Safety and Mission Assurance defined terminology, categorized data fields, and designed a baseline repository that centralizes and compiles a comprehensive listing of ACs and correlated data relevant across many NASA missions. This prototype tool helps projects improve analysis by tracking ACs and allowing queries based on project, mission type, domaincomponent, causal fault, and other key characteristics. Vulnerability in off-nominal situations, architectural design weaknesses, and unexpected or undesirable system behaviors in reaction to faults are curtailed with the awareness of ACs and risk-significant scenarios modeled for analysts through this database. Integration within the Enterprise Architecture at NASA IVV enables interfacing with other tools and datasets, technical support, and accessibility across the Agency. This paper discusses the development of an improved workflow process utilizing this database for adaptive, risk-informed FM assurance that critical software systems will safely and securely protect against faults and respond to ACs in order to achieve successful missions.

Fault management↗

NASA NTRSDate not supplied

Establishing the Assurance Efficacy of Automated Risk Mitigation Strategies

Verification and validation of increasingly autonomous aviation systems is a major challenge. Traditional techniques for the assurance of high-confidence, safety-critical systems are not equipped to handle the complexity, uncertainty, and lack of predictability inherent in non-deterministic systems. Techniques such as run time monitoring, formal methods, and testing and simulation have been applied to some effect, but it is difficult to properly assess the success of such measures. The authors propose the concept of Assurance Efficacy to address this gap. Assurance Efficacy is seen as a parameter, criteria, or perspective by which to evaluate, identify and explore safety risk mitigation strategies and operational assurance architectures. Validation of the utility of this concept through flight testing is a first step in determining its potential role in assessing the overall safety of complex, increasingly autonomous systems that cannot be fully assured in the design phase.

system safety↗

NASA NTRSDate not supplied

Dynamic Assurance of Autonomous Systems through Ground Control Software∗

Assurance cases are being increasingly acknowledged as a way to build trust in complex systems with autonomous capabilities [1]. An assurance case is a comprehensive, defensible, and valid justification that a system will function as intended for the specific mission and operating environment. Such justifications for systems with autonomous capabilities are often based on various probabilistic quantifications [2]. Due to the dynamic nature of the environmental conditions in which these systems operate, as well as the changing nature of the autonomous systems themselves, these probabilistic quantifications cannot be simply estimated once during design time. Rather, they need to be continually evaluated during systems operations to ensure that the assurance case justifications are valid. We refer to the assurance case that combines both the static and dynamic elements as a Dynamic Assurance Case (DAC).

dynamic assurance case↗

DOE OSTI2025

Safety Assurance Framework for Nuclear Digital Instrumentation and Control Software

Software in digital instrumentation and control (D&C) systems poses unique challenges for the safety assurance of nuclear power plants. Through a literature survey of 21 sources, we analyzed various claims, arguments, and evidence used in safety assurance cases across industries. These were organized into a Goal Structuring Notation (GSN) safety assurance case framework to organize and visualize the various arguments for DI&C safety. We developed a classification for the levels of the framework, which can be used in other safety assurance cases to improve clarity.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

DOE OSTI2026

Watching the Watchers with Verified Formal-Assurance Tools (Abbreviated Final Report)

The “Watching the Watchers” project studied the problem of establishing assurance cases for tools that are used to assure other things. Specifically, we were interested in understanding the tools and techniques one could apply to software to build an assurance case to evaluate their applicability, difficulty, level of assurance provided, and scalability. To do so we chose a set of use cases of relevance to LLNL and our various DOE and non-DOE partners and developed demonstrators to perform this evaluation. Our key focal point was around additive manufacturing problems and assurance gaps that we identified in the additive manufacturing workflow from start to completion. We also explored other areas related to AI, data analysis, and concurrent programming. Follow-on research is planned to take our prototypes from this project and adapt and mature them to fit LLNL mission applications.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2008

Measurement Assurance for End-Item Users

The goal of a Quality Management System (QMS) as specified in ISO 9001 and AS9100 is to assure the end product meets specifications and customer requirements. Measuring devices, often called measuring and test equipments (MTE), provide the evidence of product conformity to the prescribed requirements. Therefore the processes which employ MTE can become a weak link to the overall QMS if proper attention is not given to development and execution of these processes. Traditionally, calibration of MTE is given more focus in industry standards and process control efforts than the equally important proper usage of the same equipment. It is a common complaint of calibration laboratory personnel that MTE users are only interested in "a sticker." If the QMS requires the MTE "to demonstrate conformity of the product," then the quality of the measurement process must be adequate for the task. This leads to an ad hoc definition; measurement assurance is a discipline that assures that all processes, activities, environments, standards, and procedures involved in making a measurement produce a result that can be rigorously evaluated for validity and accuracy. To evaluate that the existing measurement processes are providing an adequate level of quality to support the decisions based upon this measurement data, an understanding of measurement assurance basics is essential. This topic is complimentary to the calibration standard, ANSI/NCSL Z540.3-2006, which targets the calibration of MTE at the organizational level. This paper will discuss general measurement assurance when MTE is used to provide evidence of product conformity, therefore the target audience of this paper is end item users of MTE. A central focus of the paper will be the verification of tolerances and the associated risks, so calibration professionals may find the paper useful in communication with their customers, MTE users.

Mimbs, Scott M.↗

NASA NTRS2015

Assuring NASA's Safety and Mission Critical Software

What is IV&V? Independent Verification and Validation (IV&V) is an objective examination of safety and mission critical software processes and products. Independence: 3 Key parameters: Technical Independence; Managerial Independence; Financial Independence. NASA IV&V perspectives: Will the system's software: Do what it is supposed to do?; Not do what it is not supposed to do?; Respond as expected under adverse conditions?. Systems Engineering: Determines if the right system has been built and that it has been built correctly. IV&V Technical Approaches: Aligned with IEEE 1012; Captured in a Catalog of Methods; Spans the full project lifecycle. IV&V Assurance Strategy: The IV&V Project's strategy for providing mission assurance; Assurance Strategy is driven by the specific needs of an individual project; Implemented via an Assurance Design; Communicated via Assurance Statements.

Validation↗

NASA NTRS2015

Formal Assurance Arguments: A Solution In Search of a Problem?

An assurance case comprises evidence and argument showing how that evidence supports assurance claims (e.g., about safety or security). It is unsurprising that some computer scientists have proposed formalizing assurance arguments: most associate formality with rigor. But while engineers can sometimes prove that source code refines a formal specification, it is not clear that formalization will improve assurance arguments or that this benefit is worth its cost. For example, formalization might reduce the benefits of argumentation by limiting the audience to people who can read formal logic. In this paper, we present (1) a systematic survey of the literature surrounding formal assurance arguments, (2) an analysis of errors that formalism can help to eliminate, (3) a discussion of existing evidence, and (4) suggestions for experimental work to definitively answer the question.

Graydon, Patrick J.↗

NASA NTRS2016

Defining Baconian Probability for Use in Assurance Argumentation

The use of assurance cases (e.g., safety cases) in certification raises questions about confidence in assurance argument claims. Some researchers propose to assess confidence in assurance cases using Baconian induction. That is, a writer or analyst (1) identifies defeaters that might rebut or undermine each proposition in the assurance argument and (2) determines whether each defeater can be dismissed or ignored and why. Some researchers also propose denoting confidence using the counts of defeaters identified and eliminated-which they call Baconian probability-and performing arithmetic on these measures. But Baconian probabilities were first defined as ordinal rankings which cannot be manipulated arithmetically. In this paper, we recount noteworthy definitions of Baconian induction, review proposals to assess confidence in assurance claims using Baconian probability, analyze how these comport with or diverge from the original definition, and make recommendations for future practice.

Graydon, Patrick J.↗

NASA NTRS2018

Considerations in Assuring Safety of Increasingly Autonomous Systems

Recent technological advances have accelerated the development and application of increasingly autonomous (IA) systems in civil and military aviation. IA systems can provide automation of complex mission tasks-ranging across reduced crew operations, air-traffic management, and unmanned, autonomous aircraft-with most applications calling for collaboration and teaming among humans and IA agents. IA systems are expected to provide benefits in terms of safety, reliability, efficiency, affordability, and previously unattainable mission capability. There is also a potential for improving safety by removal of human errors. There are, however, several challenges in the safety assurance of these systems due to the highly adaptive and non-deterministic behavior of these systems, and vulnerabilities due to potential divergence of airplane state awareness between the IA system and humans. These systems must deal with external sensors and actuators, and they must respond in time commensurate with the activities of the system in its environment. One of the main challenges is that safety assurance, currently relying upon authority transfer from an autonomous function to a human to mitigate safety concerns, will need to address their mitigation by automation in a collaborative dynamic context. These challenges have a fundamental, multidimensional impact on the safety assurance methods, system architecture, and V&V capabilities to be employed. The goal of this report is to identify relevant issues to be addressed in these areas, the potential gaps in the current safety assurance techniques, and critical questions that would need to be answered to assure safety of IA systems. We focus on a scenario of reduced crew operation when an IA system is employed which reduces, changes or eliminates a human's role in transition from two-pilot operations.

Alves, Erin E.↗

NASA NTRS2016

Identifying Where Mission Assurance Can Benefit from Model Based Systems Engineering

Model Based Systems Engineering (MBSE) is being introduced and used to support many aspects of the engineering of complex systems. Our focus is on space systems, with a particular interest in Mission Assurance of those systems. We seek to identify where Mission Assurance can potentially benefit when a space system is developed using an MBSE approach. To do so we consider how MBSE can help provide the evidence that Mission Assurance needs, using the recently developed NASA Reliability and Maintainability Objective Hierarchy as the exposition of those needs. We illustrate this MBSE’s potential contributions using examples drawn from our prior survey of the MBSE literature addressing space mission assurance concerns. The overall conclusion we draw from this is that there are numerous ways in which MBSE may, and in some cases already does, support Mission Assurance.

Feather, Martin S.↗

NASA NTRS2014

Towards Run-time Assurance of Advanced Propulsion Algorithms

This paper covers the motivation and rationale for investigating the application of run-time assurance methods as a potential means of providing safety assurance for advanced propulsion control systems. Certification is becoming increasingly infeasible for such systems using current verification practices. Run-time assurance systems hold the promise of certifying these advanced systems by continuously monitoring the state of the feedback system during operation and reverting to a simpler, certified system if anomalous behavior is detected. The discussion will also cover initial efforts underway to apply a run-time assurance framework to NASA's model-based engine control approach. Preliminary experimental results are presented and discussed.

run time assurance↗

NASA NTRS2015

Fault Management Architectures and the Challenges of Providing Software Assurance

Fault Management (FM) is focused on safety, the preservation of assets, and maintaining the desired functionality of the system. How FM is implemented varies among missions. Common to most missions is system complexity due to a need to establish a multi-dimensional structure across hardware, software and spacecraft operations. FM is necessary to identify and respond to system faults, mitigate technical risks and ensure operational continuity. Generally, FM architecture, implementation, and software assurance efforts increase with mission complexity. Because FM is a systems engineering discipline with a distributed implementation, providing efficient and effective verification and validation (V&V) is challenging. A breakout session at the 2012 NASA Independent Verification & Validation (IV&V) Annual Workshop titled "V&V of Fault Management: Challenges and Successes" exposed this issue in terms of V&V for a representative set of architectures. NASA's Software Assurance Research Program (SARP) has provided funds to NASA IV&V to extend the work performed at the Workshop session in partnership with NASA's Jet Propulsion Laboratory (JPL). NASA IV&V will extract FM architectures across the IV&V portfolio and evaluate the data set, assess visibility for validation and test, and define software assurance methods that could be applied to the various architectures and designs. This SARP initiative focuses efforts on FM architectures from critical and complex projects within NASA. The identification of particular FM architectures and associated V&V/IV&V techniques provides a data set that can enable improved assurance that a system will adequately detect and respond to adverse conditions. Ultimately, results from this activity will be incorporated into the NASA Fault Management Handbook providing dissemination across NASA, other agencies and the space community. This paper discusses the approach taken to perform the evaluations and preliminary findings from the research.

Fault Management↗