Search NASA⌕ Search

SEARCH · Search NASA

Results for “addressing failure”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Addressing failures in exascale computing

We present here a report produced by a workshop on “Addressing Failures in Exascale Computing” held in Park City, Utah, August 4–11, 2012. The charter of this workshop was to establish a common taxonomy about resilience across all the levels in a computing system; discuss existing knowledge on resilience across the various hardware and software layers of an exascale system; and build on those results, examining potential solutions from both a hardware and software perspective and focusing on a combined approach. The workshop brought together participants with expertise in applications, system software, and hardware; they came from industry, government, and academia; and their interests ranged from theory to implementation. The combination allowed broad and comprehensive discussions and led to this document, which summarizes and builds on those discussions.

Snir, Marc↗

Automated Addressing Failure Detection for Multiplexer Systems

Multiplexers are electronic devices which select between several input signals and deliver an output signal. Also known as data selectors or mux, multiplexers are commonly used in circuit design to provide signal switching, simplify hardware and manufacturing, and decrease cost. Systems which use a digital mux as a switch or selector are vulnerable to events called addressing failures, arising in hardware or software. An addressing failure results in the system requesting, or the mux delivering, a data signal other than the one intended. The unintended use of this erroneous signal may lead to system-level failures. This paper discusses a novel method of automatic detection for addressing failures, utilizing multiplexer channelization and a monitoring algorithm.

multiplexer↗

Addressing Failures in Molten Salt Thermal Energy Storage Tank for Central Receiver Concentrating Solar Power Plants

The thermal energy storage (TES) system is a critical component in concentrated solar power (CSP) plants that increases the plant's capacity factor and economic competitiveness by reducing the levelized cost of energy (LCOE) while simultaneously increasing the value of the delivered energy. Failures including molten-salt leaks and diverse localized cracking after several months to a few years of operation have been reported in hot tanks for CSP plants operating around the world. A model of a molten salt thermal energy storage tank was developed and validated to analyze the impact of different tank design features on the temperature and stress distributions as a function of typical plant operation conditions. Design features included the floor plate thicknesses, friction coefficients between the tank floor and the foundation, and the sparger ring location. Maximum stresses in the tank floor frequently surpassed the yield point of the material during operation, leading to a detriment of the tank's lifetime. Recommendations on design features to improve the reliability of new molten salt tanks for CSP plants are provided.

concentrating solar power↗

Optimizing Channelization for Fun and Profit

Design for reliability by using your multiplexer’s un-assigned channels. With optimal channelization, software can detect, correct, and eliminate critical outcomes due to addressing failures.

multiplexer↗

Using Rollback Avoidance to Mitigate Failures in Next-Generation Extreme-Scale Systems

High-performance computing (HPC) systems enable scientists to numerically model complex phenomena in many important physical systems. The next major milestone in the development of HPC systems is the construction of the rst supercomputer capable executing more than an exa op, 10 18 oating point operations per second. On systems of this scale, failures will occur much more frequently than on current systems. As a result, resilience is a key obstacle to building next-generation extremescale systems. Coordinated checkpointing is currently the most widely-used mechanism for handling failures on HPC systems. Although coordinated checkpointing remains e ective on current systems, increasing the scale of today's systems to build next-generation systems will increase the cost of fault tolerance as more and more time is taken away from the application to protect against or recover from failure. Rollback avoidance techniques seek to mitigate the cost of checkpoint/restart by allowing an application to continue its execution rather than rolling back to an earlier checkpoint when failures occur. These techniqes include failure prediction and preventive migration, replicated computation, fault-tolerant algorithms, and softwarebased memory fault correction. In this thesis, we examine how rollback avoidance techniques can be used to address failures on extreme-scale systems. Using a combination of analytic modeling and simulation, we evaluate the potential impact of rollback avoidance on these systems. We then present a novel rollback avoidance technique that exploits similarities in application memory. Finally, we examine the feasibility of using this technique to protect against memory faults in kernel memory.

97 MATHEMATICS AND COMPUTING↗

Modeling Freedom From Progression for Standard-Risk Medulloblastoma: A Mathematical Tumor Control Model With Multiple Modes of Failure

Purpose: As pediatric medulloblastoma (MB) is a relatively rare disease, it is important to extract the maximum information from trials and cohort studies. Here, a framework was developed for modeling tumor control with multiple modes of failure and time-to-progression for standard-risk MB, using published pattern of failure data. Methods and Materials: Outcome data for standard-risk MB published after 1990 with pattern of relapse information were used to fit a tumor control dose-response model addressing failures in both the high-dose boost volume and the elective craniospinal volume. Estimates of 5-year event-free survival from 2 large randomized MB trials were used to model the time-to-progression distribution. Uncertainty in freedom from progression (FFP) was estimated by Monte Carlo sampling over the statistical uncertainty in input data. Results: The estimated 5-year FFP (95% confidence intervals [CI]) for craniospinal doses of 15, 18, 24, and 36 Gy while maintaining 54 Gy to the posterior fossa was 77% (95% CI, 70%-81%), 78% (95% CI, 73%-81%), 79% (95% CI, 76%-82%), and 80% (95% CI, 77%-84%) respectively. The uncertainty in FFP was considerably larger for craniospinal doses below 18 Gy, reflecting the lack of data in the lower dose range. Conclusions: Estimates of tumor control and time-to-progression for standard-risk MB provides a data-driven setting for hypothesis generation or power calculations for prospective trials, taking the uncertainties into account. The presented methods can also be applied to incorporate further risk-stratification for example based on molecular biomarkers, when the necessary data become available.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

DRS: Derivational Reasoning System

The high reliability requirements for airborne systems requires fault-tolerant architectures to address failures in the presence of physical faults, and the elimination of design flaws during the specification and validation phase of the design cycle. Although much progress has been made in developing methods to address physical faults, design flaws remain a serious problem. Formal methods provides a mathematical basis for removing design flaws from digital systems. DRS (Derivational Reasoning System) is a formal design tool based on advanced research in mathematical modeling and formal synthesis. The system implements a basic design algebra for synthesizing digital circuit descriptions from high level functional specifications. DRS incorporates an executable specification language, a set of correctness preserving transformations, verification interface, and a logic synthesis interface, making it a powerful tool for realizing hardware from abstract specifications. DRS integrates recent advances in transformational reasoning, automated theorem proving and high-level CAD synthesis systems in order to provide enhanced reliability in designs with reduced time and cost.

Bose, Bhaskar↗

Modeling in the State Flow Environment to Support Launch Vehicle Verification Testing for Mission and Fault Management Algorithms in the NASA Space Launch System

Analysis methods and testing processes are essential activities in the engineering development and verification of the National Aeronautics and Space Administration's (NASA) new Space Launch System (SLS). Central to mission success is reliable verification of the Mission and Fault Management (M&FM) algorithms for the SLS launch vehicle (LV) flight software. This is particularly difficult because M&FM algorithms integrate and operate LV subsystems, which consist of diverse forms of hardware and software themselves, with equally diverse integration from the engineering disciplines of LV subsystems. M&FM operation of SLS requires a changing mix of LV automation. During pre-launch the LV is primarily operated by the Kennedy Space Center (KSC) Ground Systems Development and Operations (GSDO) organization with some LV automation of time-critical functions, and much more autonomous LV operations during ascent that have crucial interactions with the Orion crew capsule, its astronauts, and with mission controllers at the Johnson Space Center. M&FM algorithms must perform all nominal mission commanding via the flight computer to control LV states from pre-launch through disposal and also address failure conditions by initiating autonomous or commanded aborts (crew capsule escape from the failing LV), redundancy management of failing subsystems and components, and safing actions to reduce or prevent threats to ground systems and crew. To address the criticality of the verification testing of these algorithms, the NASA M&FM team has utilized the State Flow environment6 (SFE) with its existing Vehicle Management End-to-End Testbed (VMET) platform which also hosts vendor-supplied physics-based LV subsystem models. The human-derived M&FM algorithms are designed and vetted in Integrated Development Teams composed of design and development disciplines such as Systems Engineering, Flight Software (FSW), Safety and Mission Assurance (S&MA) and major subsystems and vehicle elements such as Main Propulsion Systems (MPS), boosters, avionics, Guidance, Navigation, and Control (GN&C), Thrust Vector Control (TVC), liquid engines, and the astronaut crew office. Since the algorithms are realized using model-based engineering (MBE) methods from a hybrid of the Unified Modeling Language (UML) and Systems Modeling Language (SysML), SFE methods are a natural fit to provide an in depth analysis of the interactive behavior of these algorithms with the SLS LV subsystem models. For this, the M&FM algorithms and the SLS LV subsystem models are modeled using constructs provided by Matlab which also enables modeling of the accompanying interfaces providing greater flexibility for integrated testing and analysis, which helps forecast expected behavior in forward VMET integrated testing activities. In VMET, the M&FM algorithms are prototyped and implemented using the same C++ programming language and similar state machine architectural concepts used by the FSW group. Due to the interactive complexity of the algorithms, VMET testing thus far has verified all the individual M&FM subsystem algorithms with select subsystem vendor models but is steadily progressing to assessing the interactive behavior of these algorithms with LV subsystems, as represented by subsystem models. The novel SFE applications has proven to be useful for quick look analysis into early integrated system behavior and assessment of the M&FM algorithms with the modeled LV subsystems. This early MBE analysis generates vital insight into the integrated system behaviors, algorithm sensitivities, design issues, and has aided in the debugging of the M&FM algorithms well before full testing can begin in more expensive, higher fidelity but more arduous environments such as VMET, FSW testing, and the Systems Integration Lab7 (SIL). SFE has exhibited both expected and unexpected behaviors in nominal and off nominal test cases prior to full VMET testing. In many findings, these behavioral characteristics were used to correct the M&FM algorithms, enable better test coverage, and develop more effective test cases for each of the LV subsystems. This has improved the fidelity of testing and planning for the next generation of M&FM algorithms as the SLS program evolves from non-crewed to crewed flight, impacting subsystem configurations and the M&FM algorithms that control them. SFE analysis has improved robustness and reliability of the M&FM algorithms by revealing implementation errors and documentation inconsistencies. It is also improving planning efficiency for future VMET testing of the M&FM algorithms hosted in the LV flight computers, further reducing risk for the SLS launch infrastructure, the SLS LV, and most importantly the crew.

Trevino, Luis↗

International Space Station (ISS) Environmental Control and Life Support (ECLS) System Overview of Events: February 2002 - 2004

The International Space Station continues to build up its life support equipment capability. Several ECLS equipment failures have occurred since Lab activation in February 2001. Major problems occurring between February 2001 and February 2002 were discussed in other works. Major problems occurring between February 2002 and February 2003 are discussed in this paper, as are updates from previously ongoing unresolved problems. This paper addresses failures, and root cause, with particular emphasis on likely micro-gravity causes. Impact to overall station operations and proposed and accomplished fixes will also be discussed.

Gentry, Gregory J.↗

A hierarchical approach to reliability modeling of fault-tolerant systems

A methodology for performing fault tolerant system reliability analysis is presented. The method decomposes a system into its subsystems, evaluates vent rates derived from the subsystem's conditional state probability vector and incorporates those results into a hierarchical Markov model of the system. This is done in a manner that addresses failure sequence dependence associated with the system's redundancy management strategy. The method is derived for application to a specific system definition. Results are presented that compare the hierarchical model's unreliability prediction to that of a more complicated tandard Markov model of the system. The results for the example given indicate that the hierarchical method predicts system unreliability to a desirable level of accuracy while achieving significant computational savings relative to component level Markov model of the system.

Gossman, W. E.↗

PV Module and System Reliability Research

While photovoltaic (PV) technologies have experienced widespread success and adoption, continued growth of these technologies - especially new PV technologies - requires ongoing improvements to their reliability and the testing procedures, data, and standards that underpin them. To better understand and address failure mechanisms of PV modules and systems, NREL conducts testing, models failures, analyzes performance data, helps to develop standards, and convenes expert stakeholders. This work is focused both on improving the performance of industry-dominant PV technologies and rapidly de-risking new PV technologies so they can be proven reliable without decades of field testing.

14 SOLAR ENERGY↗

Improved Re-Configurable Sliding Mode Controller for Reusable Launch Vehicle of Second Generation Addressing Aerodynamic Surface Failures and Thrust Deficiencies

In this report we present a time-varying sliding mode control (TV-SMC) technique for reusable launch vehicle (RLV) attitude control in ascent and entry flight phases. In ascent flight the guidance commands Euler roll, pitch and yaw angles, and in entry flight it commands the aerodynamic angles of bank, attack and sideslip. The controller employs a body rate inner loop and the attitude outer loop, which are separated in time-scale by the singular perturbation principle. The novelty of the TVSMC is that both the sliding surface and the boundary layer dynamics can be varied in real time using the PD-eigenvalue assignment technique. This salient feature is used to cope with control command saturation and integrator windup in the presence of severe disturbance or control effector failure, which enhances the robustness and fault tolerance of the controller. The TV-SMC is developed and tuned up for the X-33 sub-orbital technology demonstration vehicle in launch and re-entry modes. A variety of nominal, dispersion and failure scenarios have tested via high fidelity 6DOF simulations using MAVERIC/SLIM simulation software.

Shtessel, Yuri B.↗

Coupled Thermomechanical Micromechanics Modeling of the Influence of Thermally Grown Oxide Layer in an Environmental Barrier Coating System

Environmental Barrier Coatings (EBCs) have emerged as a promising means of protecting critical components for high temperature applications (e.g., aircraft engines). EBCs are often used to protect an underlying material (substrate) from extreme thermal/chemical environments. However, systems that utilize EBCs are susceptible to a number of failure modes including oxidation/delamination. Environmental Barrier Coatings (EBCs) have emerged as a promising means of protecting silicon based ceramic matrix composite (CMC) components for high temperature applications (e.g., aircraft engines). EBCs are often used to protect an underlying material (substrate) such as silicon carbide from extreme thermal/chemical environments. In a typical CMC/EBC system, an EBC may or may not be adhered to an underlying substrate with a bond coat (e.g., silicon). Irrespective, systems that utilize EBCs are susceptible to a number of failure modes including oxidation/delamination, recession, chemical attack and dissolution, thermo-mechanical degradation, erosion, and foreign object damage. Current work at NASA Glenn Research Center is aimed at addressing these failure modes in EBC systems and developing robust analysis tools to aid in the design process. The Higher-Order Theory for Functionally Graded Materials (HOTFGM), a precursor to the High-Fidelity Generalized Method of Cells micromechanics approach, was developed to investigate the coupled thermo-mechanical behavior of functionally graded composites and will be used herein to assess the development and growth of a low-stiffness thermally grown oxide (TGO) layer in EBC/CMC systems without a silicon bond coat. To accomplish this a sensitivity study is conducted to examine the influence of uniformly and non-uniformly grown oxide layer on the associated driving forces leading to mechanical failure (spallation) of EBC layer when subjected to isothermal loading, recession, chemical attack and dissolution, thermomechanical degradation, erosion, and foreign object damage. Current work at NASA Glenn Research Center is aimed at addressing these failure modes in EBC systems and developing robust analysis tools to aid in the design process. The Higher-Order Theory for Functionally Graded Materials (HOTFGM), a precursor to the High-Fidelity Generalized Method of Cells micromechanics approach, was developed to investigate the coupled thermo-mechanical behavior of functionally graded composites (Aboudi et al., 1999, Composites B). For example, HOTFGM was previously used (Arnold et al, 1995, NASA CP 10178, paper 34), to assess interlaminar stresses (including free edge effects) in a substrate with a thermal barrier coating (TBC). In this study, HOTFGM micromechanics analyses will be used to assess the development and growth of a low-stiffness thermally grown oxide (TGO) layer between a silicon carbide substrate and a ytterbium disilicate EBC. In order to realistically simulate TGO growth, an evolution law will be incorporated into HOTFGM. In addition, the effect of TGO roughness will be explored consistent with previous TBC work (Pindera et al., 2000. Material Science and Engineering, A284, pp. 158-175). This model represents a first step in developing a robust analysis tool that can ultimately be used to design durable EBC systems. Additional failure modes will be considered as part of a future work.

environmental barrier coating↗

Evaluating Online Global Recovery with Fenix Using Application-Aware In-Memory Checkpointing Techniques

Exascale systems promise the potential for computation at unprecedented scales and resolutions, but achieving exascale by the end of this decade presents significant challenges. A key challenge is due to the very large number of cores and components and the resulting mean time between failures (MTBF) in the order of hours or minutes. Since the typical run times of target scientific applications are longer than this MTBF, fault tolerance techniques will be essential. An important class of failures that must be addressed is process or node failures. While checkpoint/restart (C/R) is currently the most widely accepted technique for addressing processor failures, coordinated, stable-storage-based global C/R might be unfeasible at exascale when the time to checkpoint exceeds the expected MTBF. This paper explores transparent recovery via implicitly coordinated, diskless, application-driven checkpointing as a way to tolerate process failures in MPI applications at exascale. The discussed approach leverages User Level Failure Mitigation (ULFM), which is being proposed as an MPI extension to allow applications to create policies for tolerating process failures. Specifically, this paper demonstrates how different implementations of application-driven in-memory checkpoint storage and recovery compare in terms of performance and scalability. We also experimentally evaluate the effectiveness and scalability of the Fenix online global recovery framework on a production system -the Titan Cray XK7 at ORNL- and demonstrate the ability of Fenix to tolerate dynamically injected failures using the execution of four benchmarks and mini-applications with different behaviors.

97 MATHEMATICS AND COMPUTING↗