Search NASA⌕ Search

SEARCH · Search NASA

Results for “Failures”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

More Data Needed for Failure Rate Estimation, Validation, and Uncertainty Reduction

The currently planned schedule for advanced Environmental Control and Life Support System (ECLSS) development and test activities to support human exploration missions is unlikely to generate sufficient data to enable statistically-supportable, precise Orbital Replacement Unit (ORU) failure rate estimates to meet existing crew safety expectations. Accurate and precise failure rate estimates are critical for missions beyond Low Earth Orbit (LEO) because current risk mitigation approaches –namely regular resupply and rapid abort capabilities –will not be available. Safe operations will depend on mission planners’ ability to accurately forecast spares demand and efficiently provide the necessary resources. However, even after more than a decade of International Space Station (ISS) ECLSS operations, a significant amount of uncertainty remains in failure rate estimates. Uncertain or inaccurate failure rates result in increased risk and spares mass. A Bayesian estimation approach, such as the one currently implemented by the ISS Program, can reduce uncertainty by incorporating engineering judgement into failure rate estimates. However, experience on the ISS and with other complex systems shows that these prior failure rate estimates are often inaccurate. In addition, prior estimates are typically point values; some level of uncertainty must be added to convert these into probability distributions for Bayesian updating, and there are several potential methods for doing so. Due to the low rate of data collection, any inaccuracy in theseprior estimates currently hasa strong influence on the end result. This paper examines the challenges associated with failure rate estimation, validation, and uncertainty reduction in the context of ECLSS development for beyond-LEO missions. A variety of techniques for generating and updating Bayesian priors are discussed and evaluated using both real-world and simulated data. Potential solutions for improving failure rate estimation, including testing additional units, are analyzed and discussed, and a set of recommendations are provided for next-generation system development activities.

Reliability↗
NASA NTRS2020

Continued Discussion of Failure Mode Modeling and Overall Component Reliability: Are the Data Missing or Censored?

This paper is the continuation of a paper presented at the 13th Probabilistic Safety Assessment and Management Conference, in which a methodology of modeling failure modes of complex components was presented; see Paulos and Smith (2016). This methodology is not particularly helpful in the space industry where there is a lack of failure data, but is more helpful in industries that see a lot of component repairs and improvements, such as in the aircraft or automotive industries. The previous paper demonstrated how the typical approach of treating failure modes as being exponential in nature may yield optimistic predictions when estimating how improvements to components will perform in the future. It is more accurate to model the failure modes as a race in time; unfortunately, this does not give a closed-form solution. This paper uses simulation to solve for the model of the world, and the results compared to the standard methodology of treating the failure modes as being exponential random failures. The standard method is shown to have optimistic predictions, which will lead to prediction errors when failure modes are removed or “fixed.” The failure mode methodology presented in the first paper treated the data as being censored when the test stopped. In this paper, we will compare the results from treating the data as both censored and missing.

Smith, Curtis↗
DOE OSTI2019

Practical Challenges and Failure Modes During Fabrication of Haynes 230 Micro-Pin Solar Receivers for High Temperature Supercritical Carbon Dioxide Operation

Printed circuit heat exchangers (PCHX) fabricated from high nickel alloys have shown promise as primary heat exchangers, recuperators, and solar receivers for high temperature and pressure supercritical carbon dioxide power cycles. There are numerous challenges in fabricating these devices including forming channel features, joining via diffusion or transient liquid phase bonding, and brazing/welding of headers. Commercial entities are understandably hesitant to share propriety best practices, and both commercial and non-commercial entities working on these components tend not to publically share failures and mistakes. However, sharing of this information could prevent similar costly errors and further the understanding of the coupled manufacturing, materials and mechanics issues in creating these components. Thus, in this paper, we document the challenges, failures, and mitigation methods uncovered in fabrication of prototype micro-pin based solar thermal receivers constructed of Haynes 230 and fabricated through a combination of photochemical machining (PCM), wire electrical discharge machining (EDM), transient liquid phase (TLP) bonding, vacuum brazing, and gas tungsten arc welding (GTAW). The receiver is designed to absorb concentrated solar fluxes greater than 140 W cm-2, while heating supercritical carbon dioxide from 550 °C to 720 °C at a pressure of 20 MPa to 25 MPa. The prototype receiver consists of a thin (~ 450 µm), Haynes 230 coversheet bonded to a 15 cm × 15 cm Haynes 230 micro-pin plate. The pieces are joined using a TLP bonding process with a nickel-phosphorus interlayer. Prior to bonding, micro-pins with height ~150 µm and diameter 300 µm are fabricated using PCM in the plate, and through slot features are made using wire EDM. Finally, flow headers are joined to the microchannel plate through a combination of vacuum brazing and GTAW. During hydrostatic proof testing, the prototype device failed when the coversheet delaminated from the pin array at a pressure of 290 bar. A failure analysis including scanning electron microscopy (SEM) to view failure sites and energy-dispersive X-ray spectroscopy (EDS) to evaluate elemental analysis of the failed areas was conducted. The failure modes can be broadly categorized as (1) failures potentially relating to reliquification of the transient liquid phase bonds between the micro-pins plate and coversheet during post-processing, (2) failures related to manufacturing defects, and (3) failures attributed to design.

14 SOLAR ENERGY↗
DOE OSTI2022

Probabilistic Methods for Cyclical and Coupled Systems with Changing Failure Rates

Advancements in nuclear system designs with automated control features provide many benefits, but can lead to complex coupled systems and dynamic failure scenarios. This is especially true for microreactor designs where components are not expected to be replaced during the reactor’s lifetime. Hence, the life of the system, in addition to the safety, needs to be evaluated. Modeling these sequences of time-dependent events requires addressing cyclical processes and changing failure rates in ways that represent the actual system dynamics in contrast to a single sampling for a component’s time to failure. This research presents two distinct analytical methods for several failure distributions that evaluate a final time to failure used for different scenarios where the time to failure must be sampled multiple times. The first method is used when evaluating a component whose failure rate increases due to an outside event after the initial sampling but before the initially sampled time to failure. The second method is used when evaluating multiple identical components or a component that has been replaced with a new identical version before the second sampling. The two methods were implemented in a few representative case studies developed in the dynamic probabilistic risk assessment tool Event Modeling Risk Assessment using Linked Diagrams. Overall, this paper provides guidelines on how these approaches give a more realistic and accurate dynamic probabilistic risk assessment of complex systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

APR1400 In-Core Instrumentation Penetration Failure by Zirconia Melt

Light water reactors have many penetrations at the reactor vessel lower head which are called ICI (in-core instrumentation tubes), ICM-GT (in-core monitoring guide tubes), CRGT (control rod guide tubes) and drain tubes. For Korean PWRs (pressurized water reactors), there are dozens of bottom-up type ICI penetration tubes (45 for OPR1000 and 61 for APR1400). For the typical BWRs (boiling water reactors), hundreds of penetrations are positioned at the reactor vessel lower head due to the structural complexity at the reactor upper head. These penetrations are regarded as the most vulnerable structures during a severe accident as they could be failed by high temperature of melt (or debris) and its decay heat. That is, tube rupture and/or tube ejection could occur because there exist an empty hole inside the penetration tube which can be a melt flow path, and they are attached to the inside of the reactor vessel by a partial weld with a gap between the tubes and reactor vessel. Therefore, the research on the penetration failure is of importance, as it could lead to melt discharge into the containment and subsequent release of radioactive materials to the environment due to the containment failure. This paper focuses on the experimental investigation on the tube ejection failure for the APR1400 ICI penetration. The tube ejection experiments were performed as it was exposed to high temperature of zirconia melt and high pressure. The tube ejection failure experiments were performed for the APR1400 in-core instrumentation penetration nozzle. Zirconia melt was used as a simulant of molten corium and interacted with the penetration specimen. The specimen was pressurized up to 2.6 bar during the interaction with melt to induce the tube ejection. The penetration weld was heated up to its melting temperature, and the penetration tube and weld above the reactor vessel surface were eroded severely by the melt. However, even though it was estimated that the weld failure occurred and the reactor vessel with stainless steel cladding was ablated severely by the melt, the tube ejection did not take place, and consequently the integrity for the APR1400 ICI penetration was confirmed in the present experimental conditions. The goal of this research is to improve the penetrations failure models in MAAP5 codes based on the experimental data. To achieve this goal, it is necessary to increase the heat flux and pressure acting on the penetration tube to find the conditions for the tube ejection failure. Moreover, the penetration failure experiments will be performed by delivering melt into the penetration specimen to simulate the melt relocation process in a real severe accident situation. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRS2024

Coronado National Memorial Disasters: Investigating Geohazards & Slope Failure Susceptibility Utilizing NASA Earth Observations

The Coronado National Memorial (CORO), located in Hereford, Arizona, is situated along the United States' southern border, featuring recently established but still incomplete border barrier roads. This landscape is inherently prone to geohazards, and debris flow due to the steep mountainous topography, complex terrain, monsoonal rains, and freeze/thaw action - and the new infrastructure has exhibited these processes in the form of rockfall, embankment failure, and debris flow. NASA DEVELOP partnered with CORO to conduct a feasibility assessment of Earth observations for identification of geohazards and slope failure susceptibility. Leveraging Earth observations (United States Geological Survey 3D Elevation Program Digital Elevation Model and a locally obtained Light Detection And Ranging-derived Digital Elevation Model) from 2019 to 2023, and geospatial datasets starting from 2008, the study was able to provide tools to determine the focus for damage mitigation, identify areas most susceptible to slope failures, and prioritize at risk assets through three products: change detection maps, slope failure susceptibility maps, and a slope failure prioritization model. With an emphasis on monitoring high-risk areas and prioritizing mitigation efforts, the project addresses a critical gap in remediation strategies and aims to enhance preservation and safety of the region. Results of this study found that (i) the most identifiable areas of change were the road cuts and debris directly adjacent to the roads created for border construction, (ii) areas of highest slope failure susceptibility are located in mountainous areas with erosive geology, and (iii) roads resulting from border construction have approximately twice the risk of slope failure as roads created by the national park service.

Geohazard↗
DOE OSTI2021

Managing the techno-economic impacts of partial string failure in multistring energy storage systems

The role of energy storage systems (ESSs) is becoming increasingly important for today’s electric power systems. Unavailability of an ESS assigned to critical grid services may cause unwanted disruption of those services and hence, may have a significant techno-economic impact. Like any physical equipment, an ESS is vulnerable to various types of faults. Failure of one or more strings in a multistring ESS does not have to be the cause of shutting down the entire ESS. It can still operate with a partial number of strings and continue providing critical services to the grid, if there are no reliability or safety issues and is acceptable under applicable standards. However, it is important to make sure that the control strategies are adaptable to the changes in ESS capacity caused by failed strings. Also, depending on the previous operation and type of failure, the reallocation of duty cycle burden among available strings could be non-uniform. These complexities suggest that mitigation of the impact of partial failure in multistring ESSs is not trivial and needs careful consideration. This is the topic of this paper. The proposed work investigates the impact of partial failure of a large multistring ESS on the assigned service and develops strategies to adjust the ESS control duty cycles for reducing such impacts. In doing so, the paper proposes a novel two-stage framework that plans for the multistring failure using robust optimization theory and then adjusts in real-time using a rule-based algorithm, based on the real-time information on the power availability of the string. Illustrations provided in this work are based on frequency regulation use-case, which is a common application for many utility-scale ESSs. A 750 kilowatt (kW)/1500 kilowatt-hour (kWh) 3-string ESS is used for the demonstration in this work and the efficacy of the proposed method is demonstrated and compared against methods that do not incorporate string failure in their strategy. In particular, here we show that revenue loss of 93% can be incurred when partial string failure is not included in operation and planning. These losses in revenue are reduced by 60% with the proposed method.

25 ENERGY STORAGE↗
DOE OSTI2022

Efficient high-fidelity TRISO statistical failure analysis using Bison: Applications to AGR-2 irradiation testing

The ability of tri-structural isotropic (TRISO) fuel to contain fission products is largely dictated by the quality of the manufacturing process, since most of the fission product release is expected to occur due to coating layer failure in a small number of particles containing defects. The Bison fuel performance code has capabilities to predict failure in individual particles, accounting for the presence of defects, and to apply statistical analysis methods to compute the probability of failure in a set of fuel particles. Bison has recently undergone significant development both to improve its physical representations of fuel particle behavior and to improve the efficiency of its statistical failure calculations. Physical model improvements include new capabilities to account for the pressure generated by fission gases on inner pyrolytic carbon (IPyC) crack surfaces and to use local material coordinate orientation to accurately incorporate the anisotropy in the material properties in aspherical particles. To improve statistical modeling efficiency, a direct integration approach which involves directly integrating the failure probability function associated with statistically varying parameters has been developed. The direct integration approach is much more efficient than the Monte Carlo (MC) schemes commonly employed, and allows Bison to directly run high-dimensional fuel performance models, which improves the accuracy of failure probability calculations. Finally, a set of benchmark problems is considered here to compare the MC and direct integration approaches, and a statistical failure analysis of compacts in the Advanced Gas Reactor (AGR)-2 experiments is performed using the direct integration approach.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Resiliency-based restoration optimization for dependent network systems against cascading failures

Due to the increasing importance of large-scale and complex network systems and the potential for massive cascading failures in these real-world systems, modeling of system resiliency and optimization of restoration strategies to mitigate system performance loss caused by diverse disruptions is of significant interest among researchers and practitioners. Although society has experienced many incidents that demonstrate the influence of cascading failures aggravated by dependencies inside network systems, existing resiliency-based restoration optimization research rarely if ever jointly considers the impact of system dependencies on cascading failures. In this paper, different restoration prioritization strategies are applied to network systems subject to cascading failures that take into account system dependencies. By conducting case studies on synthetic networks and the U.S. airport network system, the effects of restoration strategies are evaluated using a system resiliency metric and two system performance measurements. Furthermore, the influence of system dependency characteristics and the interplay between them and restoration strategies on system resiliency regarding cascading failures are also investigated. This work demonstrates the distinct effects of restoration prioritization actions against cascading failures to mitigate the performance loss in network systems with different properties. It also provides insights about restoration improvement by considering dependency impacts to effectively reduce the intensity and extent of cascading failures.

42 ENGINEERING↗
DOE OSTI2017

Failure Diameter of PBX 9502: Simulations with the SURFplus model

SURFplus is a reactive burn model for high explosives aimed at modelling shock initiation and propagation of detonation waves. It utilizes the SURF model for the fast hot-spot reaction plus a slow reaction for the energy released by carbon clustering. A feature of the SURF model is that there is a partially decoupling between burn rate parameters and detonation wave properties. Previously, parameters for PBX 9502 that control shock ini- tiation had been calibrated to Pop plot data (distance-of-run to detonation as a function of shock pressure initiating the detonation). Here burn rate parameters for the high pres- sure regime are adjusted to t the failure diameter and the limiting detonation speed just above the failure diameter. Simulated results are shown for an uncon ned rate stick when the 9502 diameter is slightly above and slightly below the failure diameter. Just above the failure diameter, in the rest frame of the detonation wave, the front is sonic at the PBX/air interface. As a consequence, the lead shock in the neighborhood of the interface is supported by the detonation pressure in the interior of the explosive rather than the reaction immediately behind the front. In the interior, the sonic point occurs near the end of the fast hot-spot reaction. Consequently, the slow carbon clustering reaction can not a ect the failure diameter. Below the failure diameter, the radial extent of the detonation front decreases starting from the PBX/air interface. That is, the failure starts at the PBX boundary and propagates inward to the axis of the rate stick.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
NASA NTRS2005

Failure Assessment

Three questions to which software developers want accurate, precise answers are "How can the software system fail?", "mat bad things will happen if the software fails?t', and "How many failures will the software experience?". Numerous techniques have been devised to answer these questions; three of the best known are: 1) Software Fault Tree Analysis (SFTA) 2) Software Failure Modes, Effects, and Criticality Analysis (SFMECA 3) Software Fault/Failure Modeling. SFTA and SFMECA have been successfully used to analyze the flight software for a number of robotic planetary exploration missions, including Galileo, Cassini, and Deep Space 1. Given the increasing interest in reusing software components from mission to mission, one of us has developed techniques for reusing the corresponding portions of the SFTA and SFMECA, reducing the effort required to conduct these analyses. SFTA has also been shown to be effective in analyzing the security aspects of software systems; intrusion mechanisms and effects can easily be modeled using these techniques. The Bi- Directional Safety Analysis (BDSA) method combines a forward search (similar to SFMECA) from potential failure modes to their effects, with a backward search (similar to SFTA) from feasible hazards to the contributing causes of each hazard. BDSA offers an efficient way to identify latent failures. Recent work has extended BDSA to product-line applications such as flight-instrumentation displays and developed tool support for the reuse of the failure-analysis artifacts within a product line. BDSA has also been streamlined to support those projects having tight cost and/or schedule constraints for their failure analysis efforts. We discuss lessons learned from practice, describe available tools, and identi@ some future directions for the topic. A substantial amount of research has been devoted to estimating the number of failures that a software system will experience during test and operations, as well as the number of faults that have been inserted into that system during its development. One of us has found that the amount of structural change to a system during its development is strongly related to the number of faults inserted into it. Using techniques requiring no additional effort on the part of the development organization, the required measurements of structural evolution can be easily obtained from a development effort's configuration management system and readily transformed into an estimate of fault content. So far, structure-fault relationships have been identified for source code; current work seeks to examine artifacts available earlier in the lifecycle to determine if similar relationships between structure and fault content can be found. In particular, relationships between requirements change requests and the number of faults inserted into the implemented system would provide a significant improvement in our ability to control software quality during the early development phases.

fault tree↗
DOE OSTI2016

Using Falcon to develop an RIA PCMI failure criteria

This paper documents the development of failure criteria for both pressurized water reactor (PWR) and boiling water reactor (BWR) fuel rod failure due to pellet cladding mechanical interaction (PCMI) during a reactivity insertion accident (RIA). This study was initiated to use cladding mechanical properties using data from separate effects tests to provide a basis to translate nuclear safety research reactor (NSRR) test data to commercial light water reactor conditions. The approach employed for the revised licensing criteria combined elements of experimental results and analytical evaluations to develop a fundamental understanding of fuel behavior during RIA events. The transient behavior of intermediate and high burnup fuel rods using well characterized RIA simulation tests was established. Cladding mechanical properties using data from separate effects tests was also developed. The Zircaloy cladding mechanical properties database of separate effects tests was used to develop the cladding integrity models. The cladding integrity models were then used in combination with the application of the Falcon fuel behavior code to translate NSRR test data to commercial light water reactor conditions. Because of the complex manner in which burnup influences fuel rod failure, it was not possible to develop a fuel rod failure threshold directly from the experimental data. An alternative approach was required that combined analytical modeling and experimental data to develop the PCMI-related contribution to the failure threshold. The analytical approach computed the radial average peak fuel enthalpy required to cause cladding failure by PCMI as a function of hydrogen content. This was accomplished by applying a critical strain energy density (CSED)-based cladding integrity model using data from total and uniform elongation tests of irradiated low-tin Zircaloy-4, ZIRLO, and Zircaloy-2 cladding materials. The CSED was developed as a function of cladding hydrogen content because hydrogen induced embrittlement is a primary effect of burnup. It was found that the failure threshold decreased with hydrogen content due to the increase in PCMI loading as a result of gap closure effects and by the decrease in cladding ductility with hydrogen accumulation. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2018

Failure mechanisms of cross-ply carbon fiber reinforced polymer laminates under longitudinal compression with experimental and computational analyses

Here, this study investigates the failure mechanisms of cross-ply laminates subjected to longitudinal compressive loading. A sequence of failure initiation and propagation process is observed based on optical microscopy images of failed specimens. Specifically, the failure process in the cross-ply laminates involves a combination of four failure mechanisms: fiber kinking, delamination, matrix cracking, and fiber/matrix splitting. We find that the kink-bands in the middle 0° plies of the cross-ply laminates most often show a wedge shape and the angle of matrix cracks in the 90° plies is slightly larger than that of pure 90° plies. The reason has been attributed to the constraining effects by the adjacent plies. The next focus of this study is to propose hybrid micro-macro models to more systematically study the failure mechanisms of the cross-ply laminates. We show that these models accurately predict the combined failure modes of the cross-ply laminates and enable us to closely investigate the interactions of different failure mechanisms. More importantly, the hybrid models achieve great computational efficiency compared to full-scale microstructural models. Finally, the combined experimental and computational analyses presented here provide a new level of understanding of the kink-band morphology and damage mechanisms in the cross-ply laminates.

42 ENGINEERING↗
DOE OSTI2019

Property values associated with the failure of individual links in a system with multiple weak and strong links

Representations are developed and illustrated for the distribution of link property values at the time of link failure in the presence of aleatory uncertainty in link properties. The following topics are covered: (i) defining properties for weak links and strong links, (ii) cumulative distribution functions (CDFs) for link failure time, (iii) integral-based derivation of CDFs for link property at time of link failure, (iv) sampling-based approximation of CDFs for link property at time of link failure, (v) verification of integral-based and sampling-based determinations of CDFs for link property at time of link failure, (vi) distributions of link properties conditional on time of link failure, and (vii) equivalence of two different integral-based derivations of CDFs for link property at time of link failure.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Towards validation of combined-accelerated stress testing through failure analysis of polyamide-based photovoltaic backsheets

Abstract Novel methods for advancing reliability testing of photovoltaic (PV) modules and materials have recently been developed. Combined-accelerated stress testing (C-AST) is one such method which has demonstrated reliable reproduction of some field-failures which were not reproducible by standard certification tests. To increase confidence and assist in the development of C-AST, and other new testing protocols, it is important to validate that the failure modes observed and mechanisms induced are representative of those observed in the field, and not the product of unrealistic stress conditions. Here we outline a method using appropriate materials characterization and modelling to validate the failure mechanisms induced in C-AST such that we can increase confidence in the test protocol. The method is demonstrated by applying it to a known cracking failure of a specific polyamide (PA)-based backsheet material. We found that the failure of the PA-based backsheet was a result of a combination of stress factors. Photo-oxidation from ultra-violet (UV) radiation exposure caused a reduction in fracture toughness, which ultimately lead to the cracking failure. We show that the chemical and structural changes observed in the backsheet following C-AST aging were also observed in field-aged samples. These results increase confidence that the conditions applied in C-AST are representative of the field and demonstrates our approach to validating the failure mechanisms induced.

14 SOLAR ENERGY↗
DOE OSTI2021

Probing the properties and mechanisms of failure waves in soda-lime glass

Soda-lime glass (SLG) and other silica glasses exhibit the failure wave phenomenon under shock compression. The mechanism responsible for this peculiar behavior of glasses is still unresolved. In this study, a series of plate impact experiments was performed at three different impact stresses of 6.4, 8.3, and 10.8 GPa to better understand the mechanisms underlying the failure wave phenomenon. Specifically, spall experiments were conducted to probe the speed and existence of failure waves at different stresses in SLG. A layered glass target was used to probe the possibility of a “renucleation” of the wave at the SLG–SLG interface. When it existed, the failure wave was inferred to propagate at a speed of 1.3 km/s. However, it was observed that the failure wave phenomenon ceases to exist for impact stresses higher than 10 GPa. In experiments with a 6.4 GPa impact stress, the peak free surface velocity was significantly less than what is predicted by stress-Hugoniot calculations. This velocity deficit and other important features of the measured free surface velocity profiles were simulated using finite element analysis by incorporating an abrupt densification of SLG at a critical stress in the equation of state. Furthermore, this densification feature is similar to what would be expected of a phase transition. Although unable to unambiguously reveal the mechanism causing the failure wave phenomenon, the results of the present work clearly indicate that the failure wave causes a secondary compression and densification in SLG.

36 MATERIALS SCIENCE↗
DOE OSTI2017

Scalable Failure Masking for Stencil Computations using Ghost Region Expansion and Cell to Rank Remapping

In order to achieve exascale systems, application resilience needs to be addressed. Some programming models, such as task-DAG (directed acyclic graphs) architectures, currently embed resilience features whereas traditional SPMD (single program, multiple data) and message-passing models do not. Since a large part of the community's code base follows the latter models, it is still required to take advantage of application characteristics to minimize the overheads of fault tolerance. To that end, this paper explores how recovering from hard process/node failures in a local manner is a natural approach for certain applications to obtain resilience at lower costs in faulty environments. In particular, this paper targets enabling online, semitransparent local recovery for stencil computations on current leadership-class systems as well as presents programming support and scalable runtime mechanisms. Also described and demonstrated in this paper is the effect of failure masking, which allows the effective reduction of impact on total time to solution due to multiple failures. Furthermore, we discuss, implement, and evaluate ghost region expansion and cell-to-rank remapping to increase the probability of failure masking. To conclude, this paper shows the integration of all aforementioned mechanisms with the S3D combustion simulation through an experimental demonstration (using the Titan system) of the ability to tolerate high failure rates (i.e., node failures every five seconds) with low overhead while sustaining performance at large scales. In addition, this demonstration also displays the failure masking probability increase resulting from the combination of both ghost region expansion and cell-to-rank remapping.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Property Values Associated with the Failure of Individual Links in a System with Multiple Weak and Strong Links.

Representations are developed and illustrated for the distribution of link property values at the time of link failure in the presence of aleatory uncertainty in link properties. The following topics are considered: (i) defining properties for weak links and strong links, (ii) cumulative distribution functions (CDFs) for link failure time, (iii) integral-based derivation of CDFs for link property at time of link failure, (iv) sampling-based approximation of CDFs for link property at time of link failure, (v) verification of integral-based and sampling-based determinations of CDFs for link property at time of link failure, (vi) distributions of link properties conditional on time of link failure, and (vii) equivalence of two different integral-based derivations of CDFs for link property at time of link failure.

97 MATHEMATICS AND COMPUTING↗