Results for “assurance tool development” — Page 2

NASA NTRS2017

Assessment of the State-of-the-Art of System-Wide Safety and Assurance Technologies

Since its initiation, the System-wide Safety Assurance Technologies (SSAT) Project has been focused on developing multidisciplinary tools and techniques that are verified and validated to ensure prevention of loss of property and life in NextGen and enable proactive risk management through predictive methods. To this end, four technical challenges have been listed to help realize the goals of SSAT, namely (i) assurance of flight critical systems, (ii) discovery of precursors to safety incidents, (iii) assuring safe human-systems integration, and (iv) prognostic algorithm design for safety assurance. The objective of this report is to provide an extensive survey of SSAT-related research accomplishments by researchers within and outside NASA to get an understanding of what the state-of-the-art is for technologies enabling each of the four technical challenges. We hope that this report will serve as a good resource for anyone interested in gaining an understanding of the SSAT technical challenges, and also be useful in the future for project planning and resource allocation for related research.

Roychoudhury, Indranil↗

NASA NTRS2013

Tool Use Within NASA Software Quality Assurance

As space mission software systems become larger and more complex, it is increasingly important for the software assurance effort to have the ability to effectively assess both the artifacts produced during software system development and the development process itself. Conceptually, assurance is a straightforward idea - it is the result of activities carried out by an organization independent of the software developers to better inform project management of potential technical and programmatic risks, and thus increase management's confidence in the decisions they ultimately make. In practice, effective assurance for large, complex systems often entails assessing large, complex software artifacts (e.g., requirements specifications, architectural descriptions) as well as substantial amounts of unstructured information (e.g., anomaly reports resulting from testing activities during development). In such an environment, assurance engineers can benefit greatly from appropriate tool support. In order to do so, an assurance organization will need accurate and timely information on the tool support available for various types of assurance activities. In this paper, we investigate the current use of tool support for assurance organizations within NASA, and describe on-going work at JPL for providing assurance organizations with the information about tools they need to use them effectively.

software assurance↗

NASA NTRS1983

Design, ancillary testing, analysis and fabrication data for the advanced composite stabilizer for Boeing 737 aircraft. Volume 1: Technical summary

The horizontal stabilizer of the 737 transport was redesigned. Five shipsets were fabricated using composite materials. Weight reduction greater than the 20% goal was achieved. Parts and assemblies were readily produced on production-type tooling. Quality assurance methods were demonstrated. Repair methods were developed and demonstrated. Strength and stiffness analytical methods were substantiated by comparison with test results. Cost data was accumulated in a semiproduction environment. FAA certification was obtained.

Aniversario, R. B.↗

NASA NTRS2008

Closing the Certification Gaps in Adaptive Flight Control Software

Over the last five decades, extensive research has been performed to design and develop adaptive control systems for aerospace systems and other applications where the capability to change controller behavior at different operating conditions is highly desirable. Although adaptive flight control has been partially implemented through the use of gain-scheduled control, truly adaptive control systems using learning algorithms and on-line system identification methods have not seen commercial deployment. The reason is that the certification process for adaptive flight control software for use in national air space has not yet been decided. The purpose of this paper is to examine the gaps between the state-of-the-art methodologies used to certify conventional (i.e., non-adaptive) flight control system software and what will likely to be needed to satisfy FAA airworthiness requirements. These gaps include the lack of a certification plan or process guide, the need to develop verification and validation tools and methodologies to analyze adaptive controller stability and convergence, as well as the development of metrics to evaluate adaptive controller performance at off-nominal flight conditions. This paper presents the major certification gap areas, a description of the current state of the verification methodologies, and what further research efforts will likely be needed to close the gaps remaining in current certification practices. It is envisioned that closing the gap will require certain advances in simulation methods, comprehensive methods to determine learning algorithm stability and convergence rates, the development of performance metrics for adaptive controllers, the application of formal software assurance methods, the application of on-line software monitoring tools for adaptive controller health assessment, and the development of a certification case for adaptive system safety of flight.

Jacklin, Stephen A.↗

NASA NTRS2023

Adaptive Independent Verification and Validation (IV&V) Reduces Risk of Software Impacting Safety in Artemis Missions

The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.

Gerek A Whitman↗

NASA NTRS2023

Adaptive Independent Verification and Validation (IV&V) Reduces Risk of Software Impacting Safety in Artemis Missions

The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.

Gerek Whitman↗

NASA NTRS2003

Orbital Space Plane Cost Credibility

NASA's largest new start development program is the Orbital Space Plane (OSP) Program. The program is currently in the formulation stage. One of the critical issues to be resolved, prior to initiating full-scale development, is establishing cost credibility of NASA s budget estimates for development, production, and operations of the OSP. This paper will discuss the processes, tools, and methodologies that NASA, along with its industry partners, are implementing to assure cost credibility for the OSP program. Results of benchmarking of current tools and the development of new cost estimating capabilities and approaches will be discussed.

Creech, Steve↗

NASA NTRS2019

Model Based Engineering for Software Assurance

NASA's successful development of next generation space vehicles, habitats, and robotic systems will require reliable hardware and software systems. The aim of this initiative is to develop modeling methodology and tools to support Model-Based Systems Engineering (MBSE) for software assurance and reliability analysis. This effort expands the Unified Modeling Language (UML) software design models to include fault data for the extraction of Failure Modes and Effects Criticality Analysis (FMECA) and Fault Tree Analysis (FTA) for software. We explored different modeling approaches to integrate the UML software design models with the Systems Modeling Language (SysML) system models to generate an integrated model and reliability tools that take into account software and hardware interfaces.The benefits of this concept directly affect the safety community with quick turnarounds to produce software assurance and reliability analysis artifacts and the ability to visualize failure effects, both hardware and software. The result is enhanced system design integrity and early identification of system risks. This initiative will enable software assurance activities early in the system design lifecycle, facilitating the discovery of design weaknesses and enhancing the capability to produce safe, hazard-free systems

Wang, Lui↗

NASA NTRS2012

Systems-Wide Safety and Assurance Technologies SSAT Project

The SSAT Project goal is to develop validated multidisciplinary tools and techniques to ensure system safety in NextGen to enable proactive management of safety risk through predictive methods.

Mah, Robert W.↗

NASA NTRSDate not supplied

Certification Concepts for AI/ML Systems

This presentation goes over some of the tools developed at NASA Ames in the Robust Software Engineering group for the assurance and certification of autonomous systems. The research themes presented include improving safety and risk assessment as early as possible in the lifecycle, elicitation and formalization of requirements to facilitate traceability throughout the lifecycle, especially when formal methods are used, algorithms, tools and techniques for the V&V of ML-enabled systems, advanced testing, use of runtime monitoring to ease use of untrusted components, and contribution to draft regulatory standards and assistance in producing and presenting certification evidences.

Autonomy↗

NASA NTRS1989

Dynamic analysis of space structures including elastic, multibody, and control behavior

The problem is to develop analysis methods, modeling stategies, and simulation tools to predict with assurance the on-orbit performance and integrity of large complex space structures that cannot be verified on the ground. The problem must incorporate large reliable structural models, multi-body flexible dynamics, multi-tier controller interaction, environmental models including 1g and atmosphere, various on-board disturbances, and linkage to mission-level performance codes. All areas are in serious need of work, but the weakest link is multi-body flexible dynamics.

Pinson, Larry↗

NASA NTRS2003

Technology Transfer Challenges for High-Assurance Software Engineering Tools

In this paper, we describe our experience with the challenges thar we are currently facing in our effort to develop advanced software verification and validation tools. We categorize these challenges into several areas: cost benefits modeling, tool usability, customer application domain, and organizational issues. We provide examples of challenges in each area and identrfj, open research issues in areas which limit our ability to transfer high-assurance software engineering tools into practice.

John Penix↗

NASA NTRS2015

Integrating Safety and Mission Assurance into Systems Engineering Modeling Practices

During the early development of products, flight, or experimental hardware, emphasis is often given to the identification of technical requirements, utilizing such tools as use case and activity diagrams. Designers and project teams focus on understanding physical and performance demands and challenges. It is typically only later, during the evaluation of preliminary designs that a first pass, if performed, is made to determine the process, safety, and mission quality assurance requirements. Evaluation early in the life cycle, though, can yield requirements that force a fundamental change in design. This paper discusses an alternate paradigm for using the concepts of use case or activity diagrams to identify safety hazard and mission quality assurance risks and concerns using the same systems engineering modeling tools being used to identify technical requirements. It contains two examples of how this process might be used in the development of a space flight experiment, and the design of a Human Powered Pizza Delivery Vehicle, along with the potential benefits to decrease development time, and provide stronger budget estimates.

Requirements↗

NASA NTRS2005

Challenges of Developing New Classes of NASA Self-Managing Mission

NASA is proposing increasingly complex missions that will require a high degree of autonomy and autonomicity. These missions pose hereto unforeseen problems and raise issues that have not been well-addressed by the community. Assuring success of such missions will require new software development techniques and tools. This paper discusses some of the challenges that NASA and the rest of the software development community are facing in developing these ever-increasingly complex systems. We give an overview of a proposed NASA mission as well as techniques and tools that are being developed to address autonomic management and the complexity issues inherent in these missions.

Hinchey, M. G.↗

NASA NTRS2006

The NASA Software Research Infusion Initiative: Successful Technology Transfer for Software Assurance

New processes, methods and tools are constantly appearing in the field of software engineering. Many of these augur great potential in improving software development processes, resulting in higher quality software with greater levels of assurance. However, there are a number of obstacles that impede their infusion into software development practices. These are the recurring obstacles common to many forms of research. Practitioners cannot readily identify the emerging techniques that may most benefit them, and cannot afford to risk time and effort in evaluating and experimenting with them while there is still uncertainty about whether they will have payoff in this particular context. Similarly, researchers cannot readily identify those practitioners whose problems would be amenable to their techniques and lack the feedback from practical applications necessary to help them to evolve their techniques to make them more likely to be successful. This paper describes an ongoing effort conducted by a software engineering research infusion team, and the NASA Research Infusion Initiative, established by NASA s Software Engineering Initiative, to overcome these obstacles.

Hinchey, Michael G.↗

NASA NTRS1996

GOES-8 and -9 mission launch and operations support overview

The new series of geosynchronous operational environmental satellite (GOES) missions, GOES-8 and GOES-9, provide greater instrument coverage flexibility and greater resolution than previous GOES spinners. This added flexibility and the three-axis stabilized operating mode results in control system of increased complexity: almost 5000 real-time commands are sent daily to the satellites compared to 200 commands per day for GOES-7. These advanced technologies present new challenges to the NASA operations support personnel. In order to prepare for launching, post-launch tests and in-orbit operations, a mission planning scheme was developed in order to assure the flight system command and the systems' health monitoring. The mission operations approaches are reviewed together with the associated philosophies and the operation tools developed to assist the personnel in performing complex routine and special operation tasks.

McCuistion, D.↗

NASA NTRS1985

Space station common module power system network topology and hardware development

Candidate power system newtork topologies for the space station common module are defined and developed and the necessary hardware for test and evaluation is provided. Martin Marietta's approach to performing the proposed program is presented. Performance of the tasks described will assure systematic development and evaluation of program results, and will provide the necessary management tools, visibility, and control techniques for performance assessment. The plan is submitted in accordance with the data requirements given and includes a comprehensive task logic flow diagram, time phased manpower requirements, a program milestone schedule, and detailed descriptions of each program task.

Landis, D. M.↗

NASA NTRSDate not supplied

Summary of Technical Interchange Meetings (TIMs) Designed to Enable Earth Independent Medical Operations (EIMO)

The Exploration Medical Capability Element (ExMC) in NASA’s Human Research Program hosted a series of TIMs in 2023-2024 designed to stimulate discussion around specific topics with the goal of enabling EIMO. In context of the thematic constituent elements of EIMO, namely pre-mission planning, acute/emergent/prolonged medical decision making, supply/resource management and task load management, subject matter experts from industry, academia and government (NASA and other Agencies) provided valuable and actionable guidance and recommendations. Earth-based medical experts will remain indispensable for pre-mission planning, however, management of acute/emergent medical contingencies will require a gradual transition of medical care and decision making from terrestrial to space-based assets to enable support of astronaut health and performance and reduce overall mission risk. Key to achieving these enhancements is providing an integrated data system platform capable of utilizing multiple data streams in concert with a variety of on-board databases and passive monitoring of video and wearable sensors to enable a multi-modal, agentic AI-based clinical decision support system (CDSS) to support crew medical officer (CMO) medical decision-making. The EIMO series of TIMs (I-V) have proven to be instructive and portend a significant paradigm shift will be necessary to maintain crew health and performance on exploration class missions. Importantly, since the expected paradigm shift will be significantly different from the methods of operation that have been employed for the majority of missions from the inception of human spaceflight to date, any proposed methods must be deployed in the setting of ongoing operations early and be “tested, reviewed and practiced” while reliable back-up is available to facilitate an Enterprise-wide level of comfort and acceptance. Serious constraints on data transmission coupled with a large and expanding universe of on-board medical informatics data streams will necessitate implementation of a CDSS to supplant the current reliance on support provided by ground-based SMEs. Establishment of trust in the system by CMO/crew and the ground-based medical support team will be essential. Co-development of a CDSS with industry partners will assure that state of the art tools can be employed, and industry efficiencies can be leveraged. Training regimens, materials and tools must evolve to be responsive (just-in-time training) and facilitate autonomous execution of procedures. Proficiency metrics should be established and be based on validated competencies or milestones as opposed to a prescribed number of training hours. Training should be prioritized for broad, translatable skills that have universal application across a variety of medical conditions. Repetition was deemed to be the key to achieving proficiency and emphasis should lie in procedural training which is known to extinguish more rapidly than diagnostic skills. Advanced tools, e.g., extended reality, can provide more realistic and effective training. Use of advanced probabilistic risk assessment tools will be essential to optimize the medical system capability while carefully balancing risk relative to mass/power/volume limitations. Importance of factoring use-life of medical supplies and maintaining awareness of redundancy and opportunity to re-purpose under off nominal situations was emphasized. Consideration of adopting optimized performance standards vs. “good-enough” performance thresholds is warranted. The use of legacy systems as opposed to creating new systems may be preferable. Managing task load and associated cognitive load will be essential to maintain operational safety and behavioral health. ExMC aspires to create a shared EIMO paradigm and strategic vision for advancing medical system design through novel technologies, training, protocols, and support capabilities, built upon the spirit of successful strategies and innovations over the past six decades of space medicine operations.

Jay Lemery↗