Search NASA⌕ Search

SEARCH · Search NASA

Results for “Fault Injection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

New results in fault latency modelling

Studies carried out by McGough and Swern (1981, 1983) are summarized. In these studies, an avionics processor was simulated and a series of fault injection experiments was carried out to determine the degree of fault latency in a redundant flight control system that employed comparison monitoring as the exclusive means of failure detection. A determination was also made of the fault coverage of a typical self-test program. The summary presented stresses that a self-test program should be designed to capitalize on the hardware mechanization of the processor. If this is not done, subtests tend to repeatedly exercise the same hardware components while neglecting to exercise a substantial proportion of the remainder. It is also pointed out that fault latency is relatively independent of both the length and instruction mix of a program. A significant difference is found in fault coverage assessed using pin-level and gate-level fault models.

Mcgough, J. G.↗

Some methods of estimating a coverage parameter

To demonstrate the high reliability of fault-tolerant computing systems, experimental testing can be performed by injecting faults into their hardware and observing the times needed to detect and isolate failed components and reconfigure the good components. Coverage, a parameter measuring continued system success, is the probability that recovery occurs before other, perhaps catastrophic, component failures occur. Methods of estimating coverage are given for the case of randomly selecting a subset of the pins or chips for testing. Pin-level samples of times to recovery are treated by the Type I censoring model often used in life testing.

Lee, L. D.↗

Finding Fault with Faults: A Case Study

We describe our effort in extending this work beyond the initial software contruction. Our area of focus is determining the rate of fault injection over a sequence of successive builds, first observing that software faults may be seen to fall into two distinct classes some faults are incorporated during the initial coding effort, while others are added in successive software builds.

Software Systems Software faults↗

Methodology for measurement of fault latency in a digital avionic miniprocessor

Using a gate level emulation of a typical avionics miniprocessor, fault injection experiments were performed to (1) determine the time to detect a fault by comparison monitoring, (2) forecast a program's ability to detect faults and (3) validate the fault detection coverage of a typical self test program. To estimate time to detect, six programs ranging in complexity from 6 to 147 instructions, were emulated. Each program was executed repetitively in the presence of a single stuck at fault at a gate node or device pin. Detection was assumed to occur whenever the computed outputs differed from the corresponding outputs of the same program executed in a nonfaulted processor. Histograms of faults detected versus number of repetitions to detection were tabulated. Using a simple model of fault detection, which was based in an analog with the selection of balls in an urn, distributions of time to detect were computed and compared with those obtained empirically. A self test program of 2,000 executable instructions was designed expressly for the study. The only requirement imposed on the design was that it should achieve 95% coverage. The program was executed in the presence of a single stuck-at fault at a gate node on device pin. The proportion of detected faults are tabulated. In all experiments faults were selected at random over gate nodes or device pins.

Mcgough, J. C.↗

Autonomous Ocean World Exploration: Advancement of a Virtual Testbed

The search for life (extinct or extant) and potentially habitable bodies in our solar system and beyond is one of the 12 priority science questions outlined in the National Acadamies’ 2022 decadal survey [5]. Extraterrestrial destinations containing liquid water present an opportunity to search for life as we know it, and in recent years an increasing number of such locations have been discovered within our solar system. Several Jovian moons—Europa, Ganymede, and Callisto [10]—and the Saturnian moons Enceladus [8] and Titan [9] are known or suspected to harbor massive subsurface oceans. Of these "ocean worlds", Europa is the focus of at least one planned NASA orbiter mission, Europa Clipper [4], and an early lander mission concept, the Europa Lander [2, 3]. Whereas most robotic missions to the Moon and Mars (e.g. orbiters, rovers, landers) to date have had ground controllers on Earth tightly involved in mission operations, missions to more distant worlds will require a high degree of onboard autonomy due to long communication lags and blackouts, harsh environments (radiation, cold), and more limited battery and hardware life. The past decade has seen great advances in both AI technologies and computing scalability and performance that offer promising solutions for spacecraft autonomy and motivate the software system and research programs described in this paper. The Ocean Worlds Autonomy Testbed for Exploration, Research, and Simulation (OceanWATERS) [1], which has been in development at the NASA Ames Research Center since 2018, is a virtual environment for testing lander autonomy solutions. It is built on the Robot Operating System (ROS), runs on consumer-grade Linux workstations, and was released as open source in 2020. OceanWATERS provides a physical and visual simulation of a prototypical lander in a Europa-like environment (Figure 1). The lander was modeled after requirements and specifications made in JPL’s Europa Lander Study of 2016 [3]. Simulated lander systems include stereo cameras and spotlights mounted on an antenna mast that pans and tilts, a 6 degrees of freedom (DoF) robotic arm with a force-torque sensor and two interchangeable end effectors, and a battery pack power system. The environment consists of multiple terrain models including a highly detailed model sourced from the FROST dataset [11], simulation of surrounding planetary bodies based on an ephemeris model, and lighting from the sun with associated surface illumination, reflectance, and shadows. Operations supported by OceanWATERS include panoramic and directed imaging of the environment and lander workspace, Cartesian and joint-level arm commanding, grinding of the terrain surface (e.g. digging a trench), and scooping of ground material (Figure 2) which can be discarded or collected as science samples in a receptacle that can be emptied (science operations themselves are not simulated). These operations are realized as ROS Actions and are complimented by a wide selection of telemetry that is continually produced by each lander subsystem. The power system model is driven by the open-source Generic Software Architecture for Prognostics (GSAP) [11] that predicts the battery’s remaining useful life and other characteristics. As a testbed for high-level autonomy, OceanWATERS provides an execution framework based on PLEXIL [12], an open-source plan specification language and execution engine developed largely at Ames. NASA's initial development of OceanWATERS, as well the Ocean Worlds Lander Autonomy Testbed (OWLAT) [6], a complimentary physical testbed developed at JPL, was the first step in a plan for realizing candidate onboard autonomy solutions for such planetary landers. In 2020 NASA solicited applications for its Autonomous Robotics Research for Ocean Worlds (ARROW) program, and in 2021 the similar Concepts for Ocean worlds Life Detection Technology (COLDTech) program. Collectively six research teams, based in universities and companies across the United States, were awarded grants to develop and demonstrate autonomy solutions on OceanWATERS and OWLAT. These 1–2-year projects have now finished or are nearing completion, and a wide variety of autonomy challenges in ocean world surface missions were addressed. Prototyped and demonstrated solutions have included autonomous discovery, response and adaptation to system faults and unexpected environmental events, world model synthesis through perception, plan synthesis using learned models, methods to optimize sample target selection and prioritize science data transmission, extension of PLEXIL for stochastic decision-making, and an integration of a model of JPL’s mission-ready COLDArm [7]. Technologies used in these projects include many forms of machine learning, causal reasoning, automated planning, Markov decision processes, formal methods, and other advanced techniques. A more detailed summary of the ARROW and COLDTech projects is given herein. OceanWATERS has had significant enhancements since its open-source release in 2020. Many of its new features were driven or shaped by feedback from the ARROW and COLDTech teams and requirements of their projects. In support of enabling autonomous adaptation to spacecraft faults (a specific capability solicited by both programs), a fault injection and detection framework was developed that supports a wide and growing range of fault types such as locked joints, image loss, and battery failures. The power system model was completed and integrated into the simulator, starting as a single-cell battery model and later upgraded to a multi-cell model with associated faults such as cell disconnection. Arm/terrain interaction was improved by adding a force-torque sensor and associated faults, and an analytic dig force model based on the Balovnev bucket force equations. Environment fidelity was increased by modeling terrain deformation resulting from digging and scooping; visual improvements were made in textures, lighting, and shadows. To facilitate interoperation with OWLAT, a unified command and telemetry interface between the testbeds was developed at the ROS level, along with a PLEXIL interface. The number of lander operations was greatly expanded (e.g. with Cartesian-based arm and antenna movement), and a framework was designed for users to build their own lander actions. A GUI for PLEXIL plan selection was created (Figure 3), and an expansive set of plans were added, such as those that illustrate patterns for fault handling. This paper provides a self-contained high-level description of OceanWATERS, focusing on more detailed coverage of the aforementioned enhancements. It provides a high-level summary of the projects undertaken by participants in the ARROW and COLDTech programs and how these efforts have helped shape OceanWATERS. Finally, potential future work and directions for the testbed are listed, as likely informed by the recent planetary science decadal survey [5].

K Michael Dalal↗

Fault-tolerance of a neural network solving the traveling salesman problem

This study presents the results of a fault-injection experiment that stimulates a neural network solving the Traveling Salesman Problem (TSP). The network is based on a modified version of Hopfield's and Tank's original method. We define a performance characteristic for the TSP that allows an overall assessment of the solution quality for different city-distributions and problem sizes. Five different 10-, 20-, and 30- city cases are sued for the injection of up to 13 simultaneous stuck-at-0 and stuck-at-1 faults. The results of more than 4000 simulation-runs show the extreme fault-tolerance of the network, especially with respect to stuck-at-0 faults. One possible explanation for the overall surprising result is the redundancy of the problem representation.

Protzel, P.↗

VIPER: Volatiles Investigating Polar Exploration Rover

We will review the technical components of the NASA lunar rover mission, VIPER, with a focus on software components that rely on ROS 2 and Gazebo. We’ll start with an overview of the hardware and software components. Next, we’ll explore some of the custom Gazebo plugins for simulating the rover and the lunar surface. We’ll describe a fault injection framework that was built on ROS 2 parameters and how it is used to simulate hardware faults. Finally, we’ll see how ROS 2 is used as part of the rover ground software.

Jacob Michael Perron↗

Results of an electrical power system fault study (CDDF)

This report gives the results of an electrical power system fault study which has been conducted over the last 2 and one-half years. First, the results of the literature search into electrical power system faults in space and terrestrial power system applications are reported. A description of the intended implementations of the power system faults into the Large Autonomous Spacecraft Electrical Power System (LASEPS) breadboard is then presented. Then, the actual implementation of the faults into the breadboard is discussed along with a discussion describing the LASEPS breadboard. Finally, the results of the injected faults and breadboard failures are discussed.

Dugal-Whitehead, N. R.↗

Applications of an architecture design and assessment system (ADAS)

A new Architecture Design and Assessment System (ADAS) tool package is introduced, and a range of possible applications is illustrated. ADAS was used to evaluate the performance of an advanced fault-tolerant computer architecture in a modern flight control application. Bottlenecks were identified and possible solutions suggested. The tool was also used to inject faults into the architecture and evaluate the synchronization algorithm, and improvements are suggested. Finally, ADAS was used as a front end research tool to aid in the design of reconfiguration algorithms in a distributed array architecture.

Gray, F. Gail↗

A preliminary transient-fault experiment on the SIFT computer system

This paper presents the results of a preliminary experiment to study the effectiveness of a fault-tolerant system's ability to handle transient faults. The primary goal of the experiment was to develop the techniques to measure the parameters needed for a reliability analysis of the SIFT computer system which includes th effects of transient faults. A key aspect of such an analysis is the determination of the effectiveness of the operating system's ability to discriminate between transient and permanent faults. A detailed description of the preliminary transient fault experiment along with the results from 297 transient fault injections are given. Although not enough data was obtained to draw statistically significant conclusions, the foundation has been laid for a large-scale transient fault experiment.

Butler, Ricky W.↗

Design of Test Articles and Monitoring System for the Characterization of HIRF Effects on a Fault-Tolerant Computer Communication System

This report describes the design of the test articles and monitoring systems developed to characterize the response of a fault-tolerant computer communication system when stressed beyond the theoretical limits for guaranteed correct performance. A high-intensity radiated electromagnetic field (HIRF) environment was selected as the means of injecting faults, as such environments are known to have the potential to cause arbitrary and coincident common-mode fault manifestations that can overwhelm redundancy management mechanisms. The monitors generate stimuli for the systems-under-test (SUTs) and collect data in real-time on the internal state and the response at the external interfaces. A real-time health assessment capability was developed to support the automation of the test. A detailed description of the nature and structure of the collected data is included. The goal of the report is to provide insight into the design and operation of these systems, and to serve as a reference document for use in post-test analyses.

Torres-Pomales, Wilfredo↗

An Analysis of Failure Handling in Chameleon, A Framework for Supporting Cost-Effective Fault Tolerant Services

The desire for low-cost reliable computing is increasing. Most current fault tolerant computing solutions are not very flexible, i.e., they cannot adapt to reliability requirements of newly emerging applications in business, commerce, and manufacturing. It is important that users have a flexible, reliable platform to support both critical and noncritical applications. Chameleon, under development at the Center for Reliable and High-Performance Computing at the University of Illinois, is a software framework. for supporting cost-effective adaptable networked fault tolerant service. This thesis details a simulation of fault injection, detection, and recovery in Chameleon. The simulation was written in C++ using the DEPEND simulation library. The results obtained from the simulation included the amount of overhead incurred by the fault detection and recovery mechanisms supported by Chameleon. In addition, information about fault scenarios from which Chameleon cannot recover was gained. The results of the simulation showed that both critical and noncritical applications can be executed in the Chameleon environment with a fairly small amount of overhead. No single point of failure from which Chameleon could not recover was found. Chameleon was also found to be capable of recovering from several multiple failure scenarios.

Haakensen, Erik Edward↗

Fault Mitigation Schemes for Future Spaceflight Multicore Processors

Future planetary exploration missions demand significant advances in on-board computing capabilities over current avionics architectures based on a single-core processing element. The state-of-the-art multi-core processor provides much promise in meeting such challenges while introducing new fault tolerance problems when applied to space missions. Software-based schemes are being presented in this paper that can achieve system-level fault mitigation beyond that provided by radiation-hard-by-design (RHBD). For mission and time critical applications such as the Terrain Relative Navigation (TRN) for planetary or small body navigation, and landing, a range of fault tolerance methods can be adapted by the application. The software methods being investigated include Error Correction Code (ECC) for data packet routing between cores, virtual network routing, Triple Modular Redundancy (TMR), and Algorithm-Based Fault Tolerance (ABFT). A robust fault tolerance framework that provides fail-operational behavior under hard real-time constraints and graceful degradation will be demonstrated using TRN executing on a commercial Tilera(R) processor with simulated fault injections.

software based↗

Modeling of a latent fault detector in a digital system

Methods of modeling the detection time or latency period of a hardware fault in a digital system are proposed that explain how a computer detects faults in a computational mode. The objectives were to study how software reacts to a fault, to account for as many variables as possible affecting detection and to forecast a given program's detecting ability prior to computation. A series of experiments were conducted on a small emulated microprocessor with fault injection capability. Results indicate that the detecting capability of a program largely depends on the instruction subset used during computation and the frequency of its use and has little direct dependence on such variables as fault mode, number set, degree of branching and program length. A model is discussed which employs an analog with balls in an urn to explain the rate of which subsequent repetitions of an instruction or instruction set detect a given fault.

Nagel, P. M.↗

Software Testbed for Developing and Evaluating Integrated Autonomous Subsystems

To implement fault tolerant autonomy in future space systems, it will be necessary to integrate planning, adaptive control, and state estimation subsystems. However, integrating these subsystems is difficult, time-consuming, and error-prone. This paper describes Intelliface/ADAPT, a software testbed that helps researchers develop and test alternative strategies for integrating planning, execution, and diagnosis subsystems more quickly and easily. The testbed's architecture, graphical data displays, and implementations of the integrated subsystems support easy plug and play of alternate components to support research and development in fault-tolerant control of autonomous vehicles and operations support systems. Intelliface/ADAPT controls NASA's Advanced Diagnostics and Prognostics Testbed (ADAPT), which comprises batteries, electrical loads (fans, pumps, and lights), relays, circuit breakers, invertors, and sensors. During plan execution, an experimentor can inject faults into the ADAPT testbed by tripping circuit breakers, changing fan speed settings, and closing valves to restrict fluid flow. The diagnostic subsystem, based on NASA's Hybrid Diagnosis Engine (HyDE), detects and isolates these faults to determine the new state of the plant, ADAPT. Intelliface/ADAPT then updates its model of the ADAPT system's resources and determines whether the current plan can be executed using the reduced resources. If not, the planning subsystem generates a new plan that reschedules tasks, reconfigures ADAPT, and reassigns the use of ADAPT resources as needed to work around the fault. The resource model, planning domain model, and planning goals are expressed using NASA's Action Notation Modeling Language (ANML). Parts of the ANML model are generated automatically, and other parts are constructed by hand using the Planning Model Integrated Development Environment, a visual Eclipse-based IDE that accelerates ANML model development. Because native ANML planners are currently under development and not yet sufficiently capable, the ANML model is translated into the New Domain Definition Language (NDDL) and sent to NASA's EUROPA planning system for plan generation. The adaptive controller executes the new plan, using augmented, hierarchical finite state machines to select and sequence actions based on the state of the ADAPT system. Real-time sensor data, commands, and plans are displayed in information-dense arrays of timelines and graphs that zoom and scroll in unison. A dynamic schematic display uses color to show the real-time fault state and utilization of the system components and resources. An execution manager coordinates the activities of the other subsystems. The subsystems are integrated using the Internet Communications Engine (ICE). an object-oriented toolkit for building distributed applications.

Testbed↗

Measurement of fault latency in a digital avionic mini processor, part 2

The results of fault injection experiments utilizing a gate-level emulation of the central processor unit of the Bendix BDX-930 digital computer are described. Several earlier programs were reprogrammed, expanding the instruction set to capitalize on the full power of the BDX-930 computer. As a final demonstration of fault coverage an extensive, 3-axis, high performance flght control computation was added. The stages in the development of a CPU self-test program emphasizing the relationship between fault coverage, speed, and quantity of instructions were demonstrated.

Mcgough, J.↗

Fault analysis of multichannel spacecraft power systems

The NASA Marshall Space Flight Center proposes to implement computer-controlled fault injection into an electrical power system breadboard to study the reactions of the various control elements of this breadboard. Elements under study include the remote power controllers, the algorithms in the control computers, and the artificially intelligent control programs resident in this breadboard. To this end, a study of electrical power system faults is being performed to yield a list of the most common power system faults. The results of this study will be applied to a multichannel high-voltage DC spacecraft power system called the large autonomous spacecraft electrical power system (LASEPS) breadboard. The results of the power system fault study and the planned implementation of these faults into the LASEPS breadboard are described.

Dugal-Whitehead, Norma R.↗

Analysis of the Radiated Field in an Electromagnetic Reverberation Chamber as an Upset-Inducing Stimulus for Digital Systems

Preliminary data analysis for a physical fault injection experiment of a digital system exposed to High Intensity Radiated Fields (HIRF) in an electromagnetic reverberation chamber suggests a direct causal relation between the time profile of the field strength amplitude in the chamber and the severity of observed effects at the outputs of the radiated system. This report presents an analysis of the field strength modulation induced by the movement of the field stirrers in the reverberation chamber. The analysis is framed as a characterization of the discrete features of the field strength waveform responsible for the faults experienced by a radiated digital system. The results presented here will serve as a basis to refine the approach for a detailed analysis of HIRF-induced upsets observed during the radiation experiment. This work offers a novel perspective into the use of an electromagnetic reverberation chamber to generate upset-inducing stimuli for the study of fault effects in digital systems.

Torres-Pomales, Wilfredo↗