Search NASA⌕ Search

SEARCH · Search NASA

Results for “Safety-Critical Automation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

Reliability-Based Design of a Safety-Critical Automation System: A Case Study

In 1986, NASA funded a project to modernize the NASA Ames Research Center Unitary Plan Wind Tunnels, including the replacement of obsolescent controls with a modern, automated distributed control system (DCS). The project effort on this system included an independent safety analysis (ISA) of the automation system. The purpose of the ISA was to evaluate the completeness of the hazard analyses which had already been performed on the Modernization Project. The ISA approach followed a tailoring of the risk assessment approach widely used on existing nuclear power plants. The tailoring of the nuclear industry oriented risk assessment approach to the automation system and its role in reliability-based design of the automation system is the subject of this paper.

Carroll, Carol W.↗
NASA NTRS2006

ESAS Deliverable PS 1.1.2.3: Customer Survey on Code Generations in Safety-Critical Applications

Automated code generators (ACG) are tools that convert a (higher-level) model of a software (sub-)system into executable code without the necessity for a developer to actually implement the code. Although both commercially supported and in-house tools have been used in many industrial applications, little data exists on how these tools are used in safety-critical domains (e.g., spacecraft, aircraft, automotive, nuclear). The aims of the survey, therefore, were threefold: 1) to determine if code generation is primarily used as a tool for prototyping, including design exploration and simulation, or for fiight/production code; 2) to determine the verification issues with code generators relating, in particular, to qualification and certification in safety-critical domains; and 3) to determine perceived gaps in functionality of existing tools.

Schumann, Johann↗
NASA NTRS2006

An Abstract Plan Preparation Language

This paper presents a new planning language that is more abstract than most existing planning languages such as the Planning Domain Definition Language (PDDL) or the New Domain Description Language (NDDL). The goal of this language is to simplify the formal analysis and specification of planning problems that are intended for safety-critical applications such as power management or automated rendezvous in future manned spacecraft. The new language has been named the Abstract Plan Preparation Language (APPL). A translator from APPL to NDDL has been developed in support of the Spacecraft Autonomy for Vehicles and Habitats Project (SAVH) sponsored by the Explorations Technology Development Program, which is seeking to mature autonomy technology for application to the new Crew Exploration Vehicle (CEV) that will replace the Space Shuttle.

Butler, Ricky W.↗
NASA NTRS1997

Verification and Validation in a Rapid Software Development Process

The high cost of software production is driving development organizations to adopt more automated design and analysis methods such as rapid prototyping, computer-aided software engineering (CASE) tools, and high-level code generators. Even developers of safety-critical software system have adopted many of these new methods while striving to achieve high levels Of quality and reliability. While these new methods may enhance productivity and quality in many cases, we examine some of the risks involved in the use of new methods in safety-critical contexts. We examine a case study involving the use of a CASE tool that automatically generates code from high-level system designs. We show that while high-level testing on the system structure is highly desirable, significant risks exist in the automatically generated code and in re-validating releases of the generated code after subsequent design changes. We identify these risks and suggest process improvements that retain the advantages of rapid, automated development methods within the quality and reliability contexts of safety-critical projects.

Callahan, John R.↗
DOE OSTI2026

Enhancing Operational Safety via Agentic Dialogue Hazard Identification Analysis

Operational safety in high-stakes domains such as industrial process control, autonomous, and safety-critical systems demand reliable hazard identification. While large language models (LLMs) have shown promise in automating safety analysis tasks, single-turn, monolithic inference is brittle: it lacks the self-correction, deliberation, and contextual refinement that safety engineers apply iteratively. In this paper, we introduce HAZDIAL, a framework that investigates whether structured agentic dialogue (multi-agent, multi-turn interactions) improves the quality of NLP-based hazard identification over single-pass baselines. We systematically compare two dialogue modalities: adversarial debate and constructive discussion, and propose an genetic algorithm-based agentic interaction optimization. We evaluate all configurations against a curated golden dataset using standard classification metrics (accuracy, precision, recall, F1) and a novel dialogue metrics. This work advances the intersection of dialogue systems, multi-agent reasoning, and AI safety, providing empirical evidence for dialogue-driven hazard analysis.

Das, Sanjay [ORNL] (ORCID:0009000542591915)↗
DOE OSTI2025

Exponential Backoff and Its Security Implications for Safety-Critical OT Protocols over TCP/IP Networks

The convergence of Operational Technology (OT) and Information Technology (IT) networks has become increasingly prevalent with the growth of Industrial Internet of Things (IIoT) applications. This shift, while enabling enhanced automation, remote monitoring, and data sharing, also introduces new challenges related to communication latency and cybersecurity. Oftentimes, legacy OT protocols were adapted to the TCP/IP stack without an extensive review of the ramifications to their robustness, performance, or safety objectives. To further accommodate the IT/OT convergence, protocol gateways were introduced to facilitate the migration from serial protocols to TCP/IP protocol stacks within modern IT/OT infrastructure. However, they often introduce additional vulnerabilities by exposing traditionally isolated protocols to external threats. This study investigates the security and reliability implications of migrating serial protocols to TCP/IP stacks and the impact of protocol gateways, utilizing two widely used OT protocols: Modbus TCP and DNP3. Our protocol analysis finds a significant safety-critical vulnerability resulting from this migration, and our subsequent tests clearly demonstrate its presence and impact. A multi-tiered testbed, consisting of both physical and emulated components, is used to evaluate protocol performance and the effects of device-specific implementation flaws. Through this analysis of specifications and behaviors during communication interruptions, we identify critical differences in fault handling and the impact on time-sensitive data delivery. The findings highlight how reliance on lower-level IT protocols can undermine OT system resilience, and they inform the development of mitigation strategies to enhance the robustness of industrial communication networks.

DNP3↗
NASA NTRS1997

Automated Reuse of Scientific Subroutine Libraries through Deductive Synthesis

Systematic software construction offers the potential of elevating software engineering from an art-form to an engineering discipline. The desired result is more predictable software development leading to better quality and more maintainable software. However, the overhead costs associated with the formalisms, mathematics, and methods of systematic software construction have largely precluded their adoption in real-world software development. In fact, many mainstream software development organizations, such as Microsoft, still maintain a predominantly oral culture for software development projects; which is far removed from a formalism-based culture for software development. An exception is the limited domain of safety-critical software, where the high-assuiance inherent in systematic software construction justifies the additional cost. We believe that systematic software construction will only be adopted by mainstream software development organization when the overhead costs have been greatly reduced. Two approaches to cost mitigation are reuse (amortizing costs over many applications) and automation. For the last four years, NASA Ames has funded the Amphion project, whose objective is to automate software reuse through techniques from systematic software construction. In particular, deductive program synthesis (i.e., program extraction from proofs) is used to derive a composition of software components (e.g., subroutines) that correctly implements a specification. The construction of reuse libraries of software components is the standard software engineering solution for improving software development productivity and quality.

Lowry, Michael R.↗
NASA NTRS2000

Safety Metrics for Human-Computer Controlled Systems

The rapid growth of computer technology and innovation has played a significant role in the rise of computer automation of human tasks in modem production systems across all industries. Although the rationale for automation has been to eliminate "human error" or to relieve humans from manual repetitive tasks, various computer-related hazards and accidents have emerged as a direct result of increased system complexity attributed to computer automation. The risk assessment techniques utilized for electromechanical systems are not suitable for today's software-intensive systems or complex human-computer controlled systems.This thesis will propose a new systemic model-based framework for analyzing risk in safety-critical systems where both computers and humans are controlling safety-critical functions. A new systems accident model will be developed based upon modem systems theory and human cognitive processes to better characterize system accidents, the role of human operators, and the influence of software in its direct control of significant system functions Better risk assessments will then be achievable through the application of this new framework to complex human-computer controlled systems.

Leveson, Nancy G↗
NASA NTRS2000

A Model-based Framework for Risk Assessment in Human-Computer Controlled Systems

The rapid growth of computer technology and innovation has played a significant role in the rise of computer automation of human tasks in modem production systems across all industries. Although the rationale for automation has been to eliminate "human error" or to relieve humans from manual repetitive tasks, various computer-related hazards and accidents have emerged as a direct result of increased system complexity attributed to computer automation. The risk assessment techniques utilized for electromechanical systems are not suitable for today's software-intensive systems or complex human-computer controlled systems. This thesis will propose a new systemic model-based framework for analyzing risk in safety-critical systems where both computers and humans are controlling safety-critical functions. A new systems accident model will be developed based upon modem systems theory and human cognitive processes to better characterize system accidents, the role of human operators, and the influence of software in its direct control of significant system functions. Better risk assessments will then be achievable through the application of this new framework to complex human-computer controlled systems.

Hatanaka, Iwao↗
NASA NTRS2003

Discrete Abstractions of Hybrid Systems: Verification of Safety and Application to User-Interface Design

Human interaction with a complex control system involves the user, the automation’s discrete mode logic, and the underlying continuous dynamics of the physical system. The user-interface of such systems always displays a reduced set of information about the entire system. Designing interfaces such that all the pertinent information is available and assuring that this information is correct is important for any user-interface, but especially so for safety-critical systems such as automotive systems and autopilots. Here we describe a methodology for the analysis of hybrid control systems that incorporate user interaction, with the goal of assuring that the information provided to the user is correct. That is, the user-interface must contain all information necessary to safely complete a desired procedure or task. We begin with a hybrid system model which incorporates discrete mode logic as well as nonlinear continuous dynamics. Using a hybrid computational tool for reachability, we find the largest region of the state-space in which we can guarantee the state of the system can always remain – this is the safe region of operation. By implementing a controller for safety which arises from this computation, we mathematically guarantee that this safe region is invariant, meaning that the system will always remain within the safe region if the determined controller is used on the boundary of the safe region. Verification within a hybrid framework allows us to account for the continuous dynamics underlying the discrete representations displayed to the user. Using the computed invariant regions as discrete states, we can abstract a discrete event system from this hybrid system with safety restrictions. This abstraction can be used to determine what information must be provided on the display. Furthermore, in cases in which an interface already exists, the abstraction provides the necessary input into existing interface verification methods. We provide two examples: a car traveling through a yellow light at an intersection and an aircraft autopilot in an automatic landing/go-around maneuver. The examples demonstrate the applicability of this methodology to hybrid systems that have operational constraints we can pose in terms of safety. This methodology differs from existing work in hybrid system verification in that we directly account for the user’s interactions with the system.

Meeko Oishi↗
NASA NTRS1993

Applying formal methods and object-oriented analysis to existing flight software

Correctness is paramount for safety-critical software control systems. Critical software failures in medical radiation treatment, communications, and defense are familiar to the public. The significant quantity of software malfunctions regularly reported to the software engineering community, the laws concerning liability, and a recent NRC Aeronautics and Space Engineering Board report additionally motivate the use of error-reducing and defect detection software development techniques. The benefits of formal methods in requirements driven software development ('forward engineering') is well documented. One advantage of rigorously engineering software is that formal notations are precise, verifiable, and facilitate automated processing. This paper describes the application of formal methods to reverse engineering, where formal specifications are developed for a portion of the shuttle on-orbit digital autopilot (DAP). Three objectives of the project were to: demonstrate the use of formal methods on a shuttle application, facilitate the incorporation and validation of new requirements for the system, and verify the safety-critical properties to be exhibited by the software.

Cheng, Betty H. C.↗
NASA NTRS2005

A Program Certification Assistant Based on Fully Automated Theorem Provers

We describe a certification assistant to support formal safety proofs for programs. It is based on a graphical user interface that hides the low-level details of first-order automated theorem provers while supporting limited interactivity: it allows users to customize and control the proof process on a high level, manages the auxiliary artifacts produced during this process, and provides traceability between the proof obligations and the relevant parts of the program. The certification assistant is part of a larger program synthesis system and is intended to support the deployment of automatically generated code in safety-critical applications.

Denney, Ewen↗
NASA NTRS2005

Integrated Systems Health Management for Space Exploration

Integrated Systems Health Management (ISHM) is a system engineering discipline that addresses the design, development, operation, and lifecycle management of components, subsystems, vehicles, and other operational systems with the purpose of maintaining nominal system behavior and function and assuring mission safety and effectiveness under off-nominal conditions. NASA missions are often conducted in extreme, unfamiliar environments of space, using unique experimental spacecraft. In these environments, off-nominal conditions can develop with the potential to rapidly escalate into mission- or life-threatening situations. Further, the high visibility of NASA missions means they are always characterized by extraordinary attention to safety. ISHM is a critical element of risk mitigation, mission safety, and mission assurance for exploration. ISHM enables: In-space maintenance and repair; a) Autonomous (and automated) launch abort and crew escape capability; b) Efficient testing and checkout of ground and flight systems; c) Monitoring and trending of ground and flight system operations and performance; d) Enhanced situational awareness and control for ground personnel and crew; e) Vehicle autonomy (self-sufficiency) in responding to off-nominal conditions during long-duration and distant exploration missions; f) In-space maintenance and repair; and g) Efficient ground processing of reusable systems. ISHM concepts and technologies may be applied to any complex engineered system such as transportation systems, orbital or planetary habitats, observatories, command and control systems, life support systems, safety-critical software, and even the health of flight crews. As an overarching design and operational principle implemented at the system-of-systems level, ISHM holds substantial promise in terms of affordability, safety, reliability, and effectiveness of space exploration missions.

Uckun, Serdar↗
NASA NTRS2005

Development of Advanced Verification and Validation Procedures and Tools for the Certification of Learning Systems in Aerospace Applications

Adaptive control technologies that incorporate learning algorithms have been proposed to enable automatic flight control and vehicle recovery, autonomous flight, and to maintain vehicle performance in the face of unknown, changing, or poorly defined operating environments. In order for adaptive control systems to be used in safety-critical aerospace applications, they must be proven to be highly safe and reliable. Rigorous methods for adaptive software verification and validation must be developed to ensure that control system software failures will not occur. Of central importance in this regard is the need to establish reliable methods that guarantee convergent learning, rapid convergence (learning) rate, and algorithm stability. This paper presents the major problems of adaptive control systems that use learning to improve performance. The paper then presents the major procedures and tools presently developed or currently being developed to enable the verification, validation, and ultimate certification of these adaptive control systems. These technologies include the application of automated program analysis methods, techniques to improve the learning process, analytical methods to verify stability, methods to automatically synthesize code, simulation and test methods, and tools to provide on-line software assurance.

Jacklin, Stephen↗
DOE OSTI2026

Development and Validation of the Near-Miss Safety Score (NMSS) Framework for Heavy-Duty Vehicle Safety Assessment

Heavy-duty commercial vehicles present unique safety challenges due to their size, articulation dynamics, and operational complexity. As Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS) become more common in Class 8 tractor-trailers, traditional crash-based metrics are no longer sufficient to evaluate safety performance. This study introduces the Near-Miss Safety Score (NMSS)—a quantitative, physics-informed framework developed as a leading indicator of safety for advanced commercial vehicle technologies. NMSS quantifies how close a vehicle or operator comes to a collision or safety-critical event by integrating vehicle kinematics (relative distance, velocity, and acceleration) with driver or system response latency and time-to-collision. A modifier function adjusts the base score for vehicle-specific and environmental factors such as trailer articulation, load distribution, braking condition, and roadway environment. The framework enables systematic evaluation of ADAS/ADS performance under a range of operational and degraded conditions. By capturing near-miss dynamics rather than relying on crash data, NMSS provides a proactive approach to risk assessment, accelerates technology validation, and enhances interpretability for regulators and fleet operators. The proposed NMSS was validated using data collected from a motorcoach platform, demonstrating the framework’s applicability to heavy-duty safety evaluation and performance benchmarking. Results and key insights are presented in this paper.

Siekmann, Adam [ORNL] (ORCID:0000000284653935)↗
NASA NTRS1993

WTEC monograph on instrumentation, control and safety systems of Canadian nuclear facilities

This report updates a 1989-90 survey of advanced instrumentation and controls (I&C) technologies and associated human factors issues in the U.S. and Canadian nuclear industries carried out by a team from Oak Ridge National Laboratory (Carter and Uhrig 1990). The authors found that the most advanced I&C systems are in the Canadian CANDU plants, where the newest plant (Darlington) has digital systems in almost 100 percent of its control systems and in over 70 percent of its plant protection system. Increased emphasis on human factors and cognitive science in modern control rooms has resulted in a reduced workload for the operators and the elimination of many human errors. Automation implemented through digital instrumentation and control is effectively changing the role of the operator to that of a systems manager. The hypothesis that properly introducing digital systems increases safety is supported by the Canadian experience. The performance of these digital systems has been achieved using appropriate quality assurance programs for both hardware and software development. Recent regulatory authority review of the development of safety-critical software has resulted in the creation of isolated software modules with well defined interfaces and more formal structure in the software generation. The ability of digital systems to detect impending failures and initiate a fail-safe action is a significant safety issue that should be of special interest to nuclear utilities and regulatory authorities around the world.

Uhrig, Robert E.↗
NASA NTRS2003

Fault Tree Analysis Application for Safety and Reliability

Many commercial software tools exist for fault tree analysis (FTA), an accepted method for mitigating risk in systems. The method embedded in the tools identifies a root as use in system components, but when software is identified as a root cause, it does not build trees into the software component. No commercial software tools have been built specifically for development and analysis of software fault trees. Research indicates that the methods of FTA could be applied to software, but the method is not practical without automated tool support. With appropriate automated tool support, software fault tree analysis (SFTA) may be a practical technique for identifying the underlying cause of software faults that may lead to critical system failures. We strive to demonstrate that existing commercial tools for FTA can be adapted for use with SFTA, and that applied to a safety-critical system, SFTA can be used to identify serious potential problems long before integrator and system testing.

Wallace, Dolores R.↗
NASA NTRS2005

The Integrated Safety-Critical Advanced Avionics Communication and Control (ISAACC) System Concept: Infrastructure for ISHM

Integrated System Health Management (ISHM) architectures for spacecraft will include hard real-time, critical subsystems and soft real-time monitoring subsystems. Interaction between these subsystems will be necessary and an architecture supporting multiple criticality levels will be required. Demonstration hardware for the Integrated Safety-Critical Advanced Avionics Communication & Control (ISAACC) system has been developed at NASA Marshall Space Flight Center. It is a modular system using a commercially available time-triggered protocol, ?Tp/C, that supports hard real-time distributed control systems independent of the data transmission medium. The protocol is implemented in hardware and provides guaranteed low-latency messaging with inherent fault-tolerance and fault-containment. Interoperability between modules and systems of modules using the TTP/C is guaranteed through definition of messages and the precise message schedule implemented by the master-less Time Division Multiple Access (TDMA) communications protocol. "Plug-and-play" capability for sensors and actuators provides automatically configurable modules supporting sensor recalibration and control algorithm re-tuning without software modification. Modular components of controlled physical system(s) critical to control algorithm tuning, such as pumps or valve components in an engine, can be replaced or upgraded as "plug and play" components without modification to the ISAACC module hardware or software. ISAACC modules can communicate with other vehicle subsystems through time-triggered protocols or other communications protocols implemented over Ethernet, MIL-STD- 1553 and RS-485/422. Other communication bus physical layers and protocols can be included as required. In this way, the ISAACC modules can be part of a system-of-systems in a vehicle with multi-tier subsystems of varying criticality. The goal of the ISAACC architecture development is control and monitoring of safety critical systems of a manned spacecraft. These systems include spacecraft navigation and attitude control, propulsion, automated docking, vehicle health management and life support. ISAACC can integrate local critical subsystem health management with subsystems performing long term health monitoring. The ISAACC system and its relationship to ISHM will be presented.

Gwaltney, David A.↗