Results for “Fault Injection” — Page 5

NASA NTRS2010

Single Event Test Methodologies and System Error Rate Analysis for Triple Modular Redundant Field Programmable Gate Arrays

We present a test methodology for estimating system error rates of Field Programmable Gate Arrays (FPGAs) mitigated with Triple Modular Redundancy (TMR). The test methodology is founded in a mathematical model, which is also presented. Accelerator data from 90 nm Xilins Military/Aerospace grade FPGA are shown to fit the model. Fault injection (FI) results are discussed and related to the test data. Design implementation and the corresponding impact of multiple bit upset (MBU) are also discussed.

error rate calculations↗

NASA NTRS2011

Autonomous Dynamically Self-Organizing and Self-Healing Distributed Hardware Architecture - the eDNA Concept

This paper presents the current state of the autonomous dynamically self-organizing and self-healing electronic DNA (eDNA) hardware architecture (patent pending). In its current prototype state, the eDNA architecture is capable of responding to multiple injected faults by autonomously reconfiguring itself to accommodate the fault and keep the application running. This paper will also disclose advanced features currently available in the simulation model only. These features are future work and will soon be implemented in hardware. Finally we will describe step-by-step how an application is implemented on the eDNA architecture.

self-healing hardware↗

NASA NTRS2012

Network Connectivity for Permanent, Transient, Independent, and Correlated Faults

This paper develops a method for the quantitative analysis of network connectivity in the presence of both permanent and transient faults. Even though transient noise is considered a common occurrence in networks, a survey of the literature reveals an emphasis on permanent faults. Transient faults introduce a time element into the analysis of network reliability. With permanent faults it is sufficient to consider the faults that have accumulated by the end of the operating period. With transient faults the arrival and recovery time must be included. The number and location of faults in the system is a dynamic variable. Transient faults also introduce system recovery into the analysis. The goal is the quantitative assessment of network connectivity in the presence of both permanent and transient faults. The approach is to construct a global model that includes all classes of faults: permanent, transient, independent, and correlated. A theorem is derived about this model that give distributions for (1) the number of fault occurrences, (2) the type of fault occurrence, (3) the time of the fault occurrences, and (4) the location of the fault occurrence. These results are applied to compare and contrast the connectivity of different network architectures in the presence of permanent, transient, independent, and correlated faults. The examples below use a Monte Carlo simulation, but the theorem mentioned above could be used to guide fault-injections in a laboratory.

White, Allan L.↗

NASA NTRS2012

Cognitive Bias in the Verification and Validation of Space Flight Systems

Cognitive bias is generally recognized as playing a significant role in virtually all domains of human decision making. Insight into this role is informally built into many of the system engineering practices employed in the aerospace industry. The review process, for example, typically has features that help to counteract the effect of bias. This paper presents a discussion of how commonly recognized biases may affect the verification and validation process. Verifying and validating a system is arguably more challenging than development, both technically and cognitively. Whereas there may be a relatively limited number of options available for the design of a particular aspect of a system, there is a virtually unlimited number of potential verification scenarios that may be explored. The probability of any particular scenario occurring in operations is typically very difficult to estimate, which increases reliance on judgment that may be affected by bias. Implementing a verification activity often presents technical challenges that, if they can be overcome at all, often result in a departure from actual flight conditions (e.g., 1-g testing, simulation, time compression, artificial fault injection) that may raise additional questions about the meaningfulness of the results, and create opportunities for the introduction of additional biases. In addition to mitigating the biases it can introduce directly, the verification and validation process must also overcome the cumulative effect of biases introduced during all previous stages of development. A variety of cognitive biases will be described, with research results for illustration. A handful of case studies will be presented that show how cognitive bias may have affected the verification and validation process on recent JPL flight projects, identify areas of strength and weakness, and identify potential changes or additions to commonly used techniques that could provide a more robust verification and validation of future systems.

Larson, Steve↗

NASA NTRS2018

NASA Electronic Parts and Packaging (NEPP) Field Programmable Gate Array (FPGA) Single Event Effects (SEE) Test Guideline Update

The following are updated or new subjects added to the FPGA SEE Test Guidelines manual: academic versus mission specific device evaluation, single event latch-up (SEL) test and analysis, SEE response visibility enhancement during radiation testing, mitigation evaluation (embedded and user-implemented), unreliable design and its affects to SEE Data, testing flushable architectures versus non-flushable architectures, intellectual property core (IP Core) test and evaluation (addresses embedded and user-inserted), heavy-ion energy and linear energy transfer (LET) selection, proton versus heavy-ion testing, fault injection, mean fluence to failure analysis, and mission specific system-level single event upset (SEU) response prediction. Most sections within the guidelines manual provide information regarding best practices for test structure and test system development. The scope of this manual addresses academic versus mission specific device evaluation and visibility enhancement in IP Core testing.

Test guidelines↗

NASA NTRS2018

NASA Electronic Parts and Packaging Field Programmable Gate Array Single Event Effects Test Guideline Update

The following are updated or new subjects added to the FPGA SEE Test Guidelines manual: academic versus mission specific device evaluation, single event latch-up (SEL) test and analysis, SEE response visibility enhancement during radiation testing, mitigation evaluation (embedded and user-implemented), unreliable design and its affects to SEE Data, testing flushable architectures versus non-flushable architectures, intellectual property core (IP Core) test and evaluation (addresses embedded and user-inserted), heavy-ion energy and linear energy transfer (LET) selection, proton versus heavy-ion testing, fault injection, mean fluence to failure analysis, and mission specific system-level single event upset (SEU) response prediction. Most sections within the guidelines manual provide information regarding best practices for test structure and test system development. The scope of this manual addresses academic versus mission specific device evaluation and visibility enhancement in IP Core testing.

Field Programmable Gate Array (FPGA)↗

NASA NTRS2023

Adaptive Independent Verification and Validation (IV&V) Reduces Risk of Software Impacting Safety in Artemis Missions

The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.

Gerek A Whitman↗

NASA NTRS2022

Validation of the Mars 2020 Fault Protection Design: Navigating the Infinity of the Off-Nominal

On July 30th 2020, the Mars 2020 mission successfully launched out of Cape Canaveral, Florida, passed through the Earth’s shadow, and began its short cruise to Mars. Less than seven months later, the Perseverance rover touched down safely in Jezero Crater to begin its ambitious mission that includes looking for signs of ancient life and collecting samples for future return to Earth. Getting to the successful landing, or “Tango Delta Nominal,” could not have been achieved without also considering the off-nominal. One of the teams supporting this ambitious mission is the fault protection (FP) team. This team is tasked with assessing the various failures, or faults, that could prevent mission success and with ensuring that the autonomous behaviors built into the software and hardware can detect faults and recover the vehicle to a safe state. As part of its charter, the FP team designed a test campaign to provide confidence in the system’s robustness to off-nominal scenarios across all of Mars 2020’s mission phases. The greatest challenge associated with designing such a validation campaign was reducing the infinite number of anomalous scenarios into a finite test suite. In addition, the tests needed to be executed efficiently in order to utilize the team’s limited test venue access, but still needed to maintain a level of rigor that guaranteed confidence in the test outcomes. Given that each test scenario generated massive amounts of data, the team also developed methods for quickly ascertaining whether the autonomous fault protection behaviors maintained vehicle safety in the presence of an anomaly. This paper summarizes the processes that the Mars 2020 fault protection team employed to execute its off-nominal validation campaign. It captures both the methods of generating a suite of off-nominal tests, as well as reducing it to a subset that can be realistically executed within schedule and resource constraints. It also describes the various processes and philosophies that the team utilized to execute the tests efficiently, including creating a standardized procedure template, keeping the test cases modular so that they could be easily interchanged, and capturing common fault injections in a change-controlled database. Finally, it will describe the tools and processes for assessing the test data, focusing in particular on a tool that evaluated vehicle state using “secondary” sources of data to validate that the software had truly configured the spacecraft to the expected safe state.

Morantz, Chaz↗

NASA NTRS2023

Adaptive Independent Verification and Validation (IV&V) Reduces Risk of Software Impacting Safety in Artemis Missions

The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.

Gerek Whitman↗

DOE OSTI2024

Reusable Verification Components for High-Energy Physics readout ASICs

Verification is a critical aspect of designing front-end (FE) readout ASICs for High-Energy Physics (HEP) experiments. These ASICs share several similar functional features, resulting in similar verification objectives, which can be addressed using comparable verification strategies. This contribution presents a set of re-usable verification components for addressing common verification tasks, such as clock generation, reset handling, configuration, as well as hit and fault injections. The components were developed as part of the CHIPS initiative and they have been successfully used in the verification of multiple HEP ASICs.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2025

Factors controlling injection-induced rupture of intersecting faults during geological sequestration of CO 2

This study addresses coupled multiphase fluid flow and geomechanics effects on potential fault activation associated with subsurface CO 2 injection around intersecting faults. An enhanced fault-representation model is used to capture geomechanical responses of two intersecting faults with finite length during CO 2 injection. The faults are embedded in a strike-slip stress regime of a caprock-reservoir-basement system with the faults represented by zero-thickness interfaces with adjacent finite-thickness damage zones. A sensitivity analysis is conducted to study the effect of fault permeability, slip-weakening behavior, well location relative to the orientation of faults, and well placement (the number and location of injection wells). Five metrics (pressure, CO 2 plume, shear state on the fault, as well as shear displacement and stress path at selected fault monitoring points) are selected to assess CO 2 migration and reactivation of intersecting faults. The results show that induced ruptures are favored by low permeability faults due to high pressure buildup and by slip-weakening behavior resulting from fault strength reduction. The location of one injection well relative to fault orientation determines the magnitude of changes in effective normal stress and shear stress, affecting the location of induced ruptures. Well placement (two injection wells used in the paper) dominates pressure diffusion around the intersection and tips of faults. This redistributes changes in effective normal stress caused by each injection well, influencing the spatial distribution of ruptures along faults. A larger injection volume induces far-field ruptures that are controlled by stress transfer within the injection layer. The findings presented here can provide valuable insights into engineering operations for a long-term, safe, and reliable geologic CO 2 storage.

Fault permeability↗

NASA NTRS1994

Design for dependability: A simulation-based approach

This research addresses issues in simulation-based system level dependability analysis of fault-tolerant computer systems. The issues and difficulties of providing a general simulation-based approach for system level analysis are discussed and a methodology that address and tackle these issues is presented. The proposed methodology is designed to permit the study of a wide variety of architectures under various fault conditions. It permits detailed functional modeling of architectural features such as sparing policies, repair schemes, routing algorithms as well as other fault-tolerant mechanisms, and it allows the execution of actual application software. One key benefit of this approach is that the behavior of a system under faults does not have to be pre-defined as it is normally done. Instead, a system can be simulated in detail and injected with faults to determine its failure modes. The thesis describes how object-oriented design is used to incorporate this methodology into a general purpose design and fault injection package called DEPEND. A software model is presented that uses abstractions of application programs to study the behavior and effect of software on hardware faults in the early design stage when actual code is not available. Finally, an acceleration technique that combines hierarchical simulation, time acceleration algorithms and hybrid simulation to reduce simulation time is introduced.

Goswami, Kumar K.↗

DOE OSTI2025

Modeling supercritical CO2 injection induced rupture of a minor fault embedded in a poroelastic layered reservoir-caprock system

CO2 injection for geologic carbon sequestration involves hydromechanical processes that lead to changes in fluid pressure and stresses that can activate existing faults. This paper presents a new method and workflow of modeling fault activation considering more complex three-dimensional geometry of natural faults using the TOUGH-FLAC multiphase fluid flow and geomechanical simulator. In this method and workflow, FLAC3D mechanical interfaces and TOUGH3 finite volume elements are discretized using computer aided design and gridding software along with a tailored mesh translation routine. The method and workflow are demonstrated with a model of a curved minor fault embedded in a poro-elastic layered reservoir-caprock system. The model is used for a comprehensive sensitivity analysis of fault responses to fault length, injection mass rate, injection schedule, well-fault distance, and well locations versus fault location. Four metrics (CO2 plume, shear state of fault, pressure and stress path at fault monitoring points) are selected to assess CO2 migration, pressure change, and the reactivation of faults. The results reveal that CO2 can bypass around the tip of the minor impermeable fault, building up pressure and poro-elastic stress on both sides that tends to impede fault rupture. Our study shows the benefit of carefully designing the injection to achieve the targeted final storage volume, starting at a relatively low rate for considerable time, and then ramping up the injection rate to the full rate of injection. The initial low injection has two distinct benefits: (1) it allows for the formation of an extensive CO2 plume with a much higher mobility through a low viscosity that will result in a lower pressure for a given injection rate, and (2) it allows for gradual build-up of horizontal poro-elastic stress within the reservoir that will tend to impede activation of steeply dipping faults. The injection scenario starting at a low injection rate, denoted here as conservative injection, can significantly reduce the risk of fault activation as high fluid mobility and reservoir strengthening poro-elastic stress has been established long before reaching the peak injection rates. Moreover, simultaneous injection in two injection wells on both sides of fault can provide further reservoir strengthening through poro-elastic stress buildup acting on a fault under normal faulting stress regime. The findings presented in the paper can provide practical and effective guidance on long-term, safe, and reliable geological CO2 storage.

Cao, Meng↗

NASA NTRS1991

Load flows and faults considering dc current injections

The authors present novel methods for incorporating current injection sources into dc power flow computations and determining network fault currents when electronic devices limit fault currents. Combinations of current and voltage sources into a single network are considered in a general formulation. An example of relay coordination is presented. The present study is pertinent to the development of the Space Station Freedom electrical generation, transmission, and distribution system.

Kusic, G. L.↗

NASA NTRS1990

Measurement and analysis of workload effects on fault latency in real-time systems

The authors demonstrate the need to address fault latency in highly reliable real-time control computer systems. It is noted that the effectiveness of all known recovery mechanisms is greatly reduced in the presence of multiple latent faults. The presence of multiple latent faults increases the possibility of multiple errors, which could result in coverage failure. The authors present experimental evidence indicating that the duration of fault latency is dependent on workload. A synthetic workload generator is used to vary the workload, and a hardware fault injector is applied to inject transient faults of varying durations. This method makes it possible to derive the distribution of fault latency duration. Experimental results obtained from the fault-tolerant multiprocessor at the NASA Airlab are presented and discussed.

Woodbury, Michael H.↗

NASA NTRS1992

Viscosity determinations of some frictionally generated silicate melts: Implications for slip zone rheology during impact-induced faulting

Analytical scanning electron microscopy, using combined energy dispersive and wavelength dispersive spectrometry, was used to determine the major-element compositions of some natural and artificial glasses and their crystalline equivalents derived by the frictional melting of acid to intermediate protoliths. The major-element compositions are used to calculate the viscosities of their melt precursors using the model of Shaw at temperatures of 800-1400 C, with Fe(2+)/Fe(tot) = 0.5 and for 1-3 wt percent H2O. These results are then modified to account for suspension effects in order to determine viscosities. The results have implications for the generation of pseudotachylitic breccias as seen in the basement lithologies of the Sudbury and Vredefort structures and possibly certain dimict lunar breccias. Many of these breccias show similarities with the more commonly developed pseudotachylite fault and injection veins seen in endogenic fault zones that typically occur in thicknesses of a few centimeters or less. The main difference is one of scale: Impact-induced pseudotachylite breccias can attain several meters in thickness. This would suggest that they were generated under exceptionally high slip rates and hence high strain rates and that the friction melts generated possessed extremely low viscosities.

Spray, John G.↗

NASA NTRS1983

Development and evaluation of a Fault-Tolerant Multiprocessor (FTMP) computer. Volume 3: FTMP test and evaluation

The experimental test and evaluation of the Fault-Tolerant Multiprocessor (FTMP) is described. Major objectives of this exercise include expanding validation envelope, building confidence in the system, revealing any weaknesses in the architectural concepts and in their execution in hardware and software, and in general, stressing the hardware and software. To this end, pin-level faults were injected into one LRU of the FTMP and the FTMP response was measured in terms of fault detection, isolation, and recovery times. A total of 21,055 stuck-at-0, stuck-at-1 and invert-signal faults were injected in the CPU, memory, bus interface circuits, Bus Guardian Units, and voters and error latches. Of these, 17,418 were detected. At least 80 percent of undetected faults are estimated to be on unused pins. The multiprocessor identified all detected faults correctly and recovered successfully in each case. Total recovery time for all faults averaged a little over one second. This can be reduced to half a second by including appropriate self-tests.

Lala, J. H.↗

NASA NTRS1983

Graphics enhanced computer emulation for improved timing-race and fault tolerance control system analysis

A computer simulation system has been developed for the Space Shuttle's advanced Centaur liquid fuel booster rocket, in order to conduct systems safety verification and flight operations training. This simulation utility is designed to analyze functional system behavior by integrating control avionics with mechanical and fluid elements, and is able to emulate any system operation, from simple relay logic to complex VLSI components, with wire-by-wire detail. A novel graphics data entry system offers a pseudo-wire wrap data base that can be easily updated. Visual subsystem operations can be selected and displayed in color on a six-monitor graphics processor. System timing and fault verification analyses are conducted by injecting component fault modes and min/max timing delays, and then observing system operation through a red line monitor.

Szatkowski, G. P.↗