Search NASA⌕ Search

SEARCH · Search NASA

Results for “hardware failure”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40

Particle Impact Simulation and Ignition Prediction

An experimentally calibrated tool is needed to predict if a system is susceptible to failure by particle impact ignition (PI) based on use conditions, materials, and flow geometry. This tool will accelerate new components, evaluating existing hardware, and help disposition anomalies. - Conduct particle impact testing with in-situ diagnostics and complementary simulations on subset of key engineering materials (IN718, M400, 316L, 6061, Ti64, Zr) to develop a proof-of-concept predictive tool for assessing the risk of PI for idealized geometries (spherical particles) in realistic environments. - Assess particle/target interactions (coefficient of restitution, ignition, kindling) using instrumented particle impact rigs while systematically varying key parameters (materials, particle size, environment, target configuration). - Determine key field variables (temperature, strain, stress) in particle impacts using Multiphysics finite element and hydrocode simulations validated through comparison with experimental measurements and observations. - Synthesize experiments and simulations into constitutive models for PI that can be integrated with existing computational fluid dynamics (CFD) and Debris Transport Analysis (DTA) tools in future efforts.

Jonathan Tylka↗

Machine learning based job status prediction in scientific clusters

Large high-performance computing systems are built with increasing number of components with more CPU cores, more memory, and more storage space. At the same time, scientific applications have been growing in complexity. Together, they are leading to more frequent unsuccessful job statuses on HPC systems. From measured job statuses, 23.4% of CPU time was spent to the unsuccessful jobs. Here, we set out to study whether these unsuccessful job statuses could be anticipated from known job characteristics. To explore this possibility, we have developed a job status prediction method for the execution of jobs on scientific clusters. The Random Forests algorithm was applied to extract and characterize the patterns of unsuccessful job statuses. Experimental results show that our method can predict the unsuccessful job statuses from the monitored ongoing job executions in 99.8% the cases with 83.6% recall and 94.8% precision. Lastly, this prediction accuracy can be sufficiently high that it can be used to mitigation procedures of predicted failures.

97 MATHEMATICS AND COMPUTING↗

System for Anomaly and Failure Detection (SAFD) system development

The System for Anomaly and Failure Detection (SAFD) algorithm was developed as an improvement over the current redline system used in the Space Shuttle Main Engine Controller (SSMEC). Simulation tests and execution against previous hot fire tests demonstrated that the SAFD algorithm can detect engine failures as much as tens of seconds before the redline system recognized the failure. Although the current algorithm only operates during steady state conditions (engine not throttling), work is underway to expand the algorithm to work during transient conditions. This task assignment originally specified developing a platform for executing the algorithm during hot fire tests at Technology Test Bed (TTB) and installing the SAFD algorithm on that platform. Two units were built and installed in the Hardware Simulation Lab and at the TTB in December 1991. Since that time, the task primarily entailed improvement and maintenance of the systems, additional testing to prove the feasibility of the algorithm, and support of hot fire testing. This document addresses the work done since the last report of June 1992. The work on the System for Anomaly and Failure Detection during this period included improving the platform and the algorithm, testing the algorithm against previous test data and in the Hardware Simulation Lab, installing other algorithms on the system, providing support for operations at the Technology Test Bed, and providing routine maintenance.

Oreilly, D.↗

Facility to Alleviate Salt Technology Risks (FASTR): Preliminary Design Report with Failure Modes and Effects Analysis

The Facility to Alleviate Salt Technology Risks (FASTR) is a versatile, high-temperature (>600°C) molten chloride salt test facility designed to enable a variety of testing to advance the Generation 3 (Gen 3) Concentrated Solar Power (CSP) molten salt technology. FASTR includes a salt preparation system and a forced flow test loop with a suite of instrumentation. The FASTR loop is capable of 725°C operation, flow rates of 3–7 kg/s, includes heated and cooled sections, and swappable components to enable testing of future vendor-supplied hardware. The salt preparation system supplies large batches (e.g., 200 kg) of clean salt for use in the FASTR forced convection loop. This report summarizes the planned design and capabilities of FASTR. In addition, a failure modes and effects analysis of the system is reported.

42 ENGINEERING↗

A spacecraft integrated power/attitude control system

A study to determine the viability and application of a system capable of performing the dual function of power storage/generation and attitude control has been conducted. Results from the study indicate that an integrated power/attitude control system (IPACS) can satisfy future mission requirements while providing significant savings in weight, volume, and cost over conventional systems. A failure-mode configuration of an IPACS was applied to a shuttle-launched RAM free-flyer and simulated using make-do hardware linked to a hybrid computer. Data from the simulation runs indicate that control interactions resulting from heavy power demands have minimal effect on system control effectiveness. The system was shown to be capable of meeting the stringent pointing requirements of 1 arc-second while operating under the influence of an orbital disturbance environment and during periods of momentum variations imposed by energy transfer requirements.

Keckler, C. R.↗

Computer aided control of a mechanical arm

A method for computer-aided remote control of a six-degree-of-freedom manipulator arm involved in the on-orbit servicing of a spacecraft is presented. The control configuration features a supervisory type of control in which each of the segments of a module exchange trajectory is controlled automatically under human supervision, with manual commands to proceed to the next step and in the event of a failure or undesirable outcome. The implementation of the supervisory system is discussed in terms of necessary onboard and ground- or Orbiter-based hardware and software, and a one-g demonstration system built to allow further investigation of system operation is described. Possible applications of the system include the construction of satellite solar power systems, environmental testing and the control of heliostat solar power stations.

Derocher, W. L., Jr.↗

Independent Orbiter Assessment (IOA): Weibull analysis report

The Auxiliary Power Unit (APU) and Hydraulic Power Unit (HPU) Space Shuttle Subsystems were reviewed as candidates for demonstrating the Weibull analysis methodology. Three hardware components were identified as analysis candidates: the turbine wheel, the gearbox, and the gas generator. Detailed review of subsystem level wearout and failure history revealed the lack of actual component failure data. In addition, component wearout data were not readily available or would require a separate data accumulation effort by the vendor. Without adequate component history data being available, the Weibull analysis methodology application to the APU and HPU subsystem group was terminated.

Raffaelli, Gary G.↗

Technical Consultation of the International Space Station (ISS) Internal Active Thermal Control System (IATCS) Cooling Water Chemistry

The Internal Active Thermal Control System (IATCS) coolant exhibited unexpected chemical changes during the first year of on-orbit operation following the launch and activation in February 2001. The coolant pH dropped from 9.3 to below the minimum specification limit of 9.0, and re-equilibrated between 8.3 and 8.5. This drop in coolant pH was shown to be the result of permeation of CO2 from the cabin into the coolant via Teflon flexible hoses which created carbonic acid in the fluid. This unexpected diffusion was the result of having a cabin CO2 partial pressure higher than the ground partial pressure (average 4.0 mmHg vs. less than 0.2 mmHg). This drop in pH was followed by a concurrent increasing coolant nickel concentration. No other metal ions were observed in the coolant and based on previous tests, the source of nickel ion was thought to be the boron nickel (BNi) braze intermetallics used in the construction of HXs and cold plates. Specifically, BNi2 braze alloy was used for the IATCS IFHX and BNi3 braze alloy was used for the IATCS Airlock Servicing and Performance Checkout Unit (SPCU) HX and cold plates. Given the failure criticality of the HXs, a Corrosion Team was established by the IATCS CWG to determine the impact of the nickel corrosion on hardware performance life.

Gentz, Steven J.↗

ISS ECLSS: 3 Years of Logistics for Maintenance

The International Space Station (ISS) Environmental Control and Life Support System (ECLSS) is designed to be maintainable. During the 3 years since the ISS US Lab became operational, there have been numerous ECLSS Orbital Replacement Units (ORUs) launched and returned to Maintain the ECLSS operation in the US segments. The maintenance logistics have provided tools for maintenance, replaced limited life ORUs and failed ORUs, upgraded ECLSS hardware to improve reliability and placed critical spares onboard prior to need. In most cases, the removed ORUs have been returned for either failure analysis and repair or refurbishment. This paper describes the ECLSS manifesting history and maintenance events and quantifies the numbers of ECLSS items, weights, and volumes.

Shkedi, Brienne↗

Sabatier Carbon Dioxide Reduction Assembly Development for Closed Loop Water Recovery

The Sabatier Carbon Dioxide Reduction System (CRA) offers water recovery on a long duration space mission to reduce water resupply. Currently, NASA Johnson Space Center (JSC), NASA Marshall Space Flight Center (MSFC), Hamilton Sundstrand Space Systems International, Inc. (HSSSI), and Southwest Research Institute (SWRI) are working together to develop a Sabatier CRA for the International Space Station (ISS). This effort is being funded by the Office of Biological and Physical Research (Code U)/Advanced Life Support program which is administered by NASA JSC. The Sabatier CRA is the next step in closing the oxygen life support loop on future space missions. The Sabatier reaction combines the waste carbon dioxide (recovered from crew metabolism) with waste hydrogen (a byproduct of electrolysis to produce oxygen) to produce water and methane (CH4). On ISS, the methane would be vented overboard, however the methane can be utilized for propulsion during a planetary exploration mission. Based on a crew size of 7-equivalent people, the Sabatier CRA can produce as much a 2000 lb/year water. Use of the Sabatier CRA will significantly reduce the amount of water that needs to be resupplied to the ISS on a yearly basis, at a tremendous cost saving to the program. Additionally, by recycling this additional water, the Sabatier CRA enables additional launch capacity for science experiments to be brought up to the ISS. The NASA/Industry team noted above has been working to reduce technical risks associated with the Sabatier CRA system. To date the technical risks have been considerably reduced, bringing the Technology Readiness Level (TRL) from TRL 4 to TRL 5/6. In doing so, the team has developed the system schematic, system models, control scheme, produced engineering development unit (EDU) hardware, performed limited integration testing of the EDU's and verified system modeling through testing. Additionally, the system schematic has been evaluated for failure modes and hazards and had a successful technical review by the NASA Safety Board. The current focus is now related to development of the water/methane phase separator, liquid sensor and CO2 compressor piston seal life. The overall goal of the current effort is to bring the system up to a TRL6 by the end of GFY04. Although the Sabatier CRA is not currently baselined for use on the ISS, its benefits are significant enough such that volume within the Oxygen Generation System rack has been reserved for future installation. The value of the water the CRA recover will allow NASA the additional crew time and payload needed to pursue its mission of scientific research.

Smith, Frederick↗

UAS-Systems Integration, Validation, and Diagnostics Simulation Capability

As part of the Phase 1 efforts of NASA's UAS-in-the-NAS Project a task was initiated to explore the merits of developing a system simulation capability for UAS to address airworthiness certification requirements. The core of the capability would be a software representation of an unmanned vehicle, including all of the relevant avionics and flight control system components. The specific system elements could be replaced with hardware representations to provide Hardware-in-the-Loop (HWITL) test and evaluation capability. The UAS Systems Integration and Validation Laboratory (UAS-SIVL) was created to provide a UAS-systems integration, validation, and diagnostics hardware-in-the-loop simulation capability. This paper discusses how SIVL provides a robust and flexible simulation framework that permits the study of failure modes, effects, propagation paths, criticality, and mitigation strategies to help develop safety, reliability, and design data that can assist with the development of certification standards, means of compliance, and design best practices for civil UAS.

Buttrill, Catherine W.↗

Impact of Advanced Synoptics and Simplified Checklists During Aircraft Systems Failures

Abstract—Natural human capacities are becoming increasingly mismatched to the enormous data volumes, processing capabilities, and decision speeds demanded in today’s aviation environment. Increasingly Autonomous Systems (IAS) are uniquely suited to solve this problem. NASA is conducting research and development of IAS - hardware and software systems, utilizing machine learning algorithms, seamlessly integrated with humans whereby task performance of the combined system is significantly greater than the individual components. IAS offer the potential for significantly improved levels of performance and safety that are superior to either human or automation alone. A human-in-the-loop test was conducted in NASA Langley’s Integration Flight Deck B-737-800 simulator to evaluate advanced synoptic pages with simplified interactive electronic checklists as an IAS for routine air carrier flight operations and in response to aircraft system failures. Twelve U.S. airline crews flew various normal and non-normal procedures and their actions and performance were recorded in response to failures. These data are fundamental to and critical for the design and development of future increasingly autonomous systems that can better support the human in the cockpit. Synoptic pages and electronic checklists significantly improved pilot responses to non-normal scenarios, but implementation of these aids and other intelligent assistants have barriers to implementation (e.g., certification cost) that must overcome.

Etherington, Timothy J.↗

Resilience Design Patterns - A Structured Approach to Resilience at Extreme Scale (version 1.0)

Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest that very high fault rates in future systems. The errors resulting from these faults will propagate and generate various kinds of failures, which may result in outcomes ranging from result corruptions to catastrophic application crashes. Practical limits on power consumption in HPC systems will require future systems to embrace innovative architectures, increasing the levels of hardware and software complexities. The resilience challenge for extreme-scale HPC systems requires management of various hardware and software technologies that are capable of handling a broad set of fault models at accelerated fault rates. These techniques must seek to improve resilience at reasonable overheads to power consumption and performance. While the HPC community has developed various solutions, application-level as well as system-based solutions, the solution space of HPC resilience techniques remains fragmented. There are no formal methods and metrics to investigate and evaluate resilience holistically in HPC systems that consider impact scope, handling coverage, and performance & power eciency across the system stack. Additionally, few of the current approaches are portable to newer architectures and software ecosystems, which are expected to be deployed on future systems. In this document, we develop a structured approach to the management of HPC resilience based on the concept of resilience-based design patterns. A design pattern is a general repeatable solution to a commonly occurring problem. We identify the commonly occurring problems and solutions used to deal with faults, errors and failures in HPC systems. The catalog of resilience design patterns provides designers with reusable design elements. We define a design framework that enhances our understanding of the important constraints and opportunities for solutions deployed at various layers of the system stack. The framework may be used to establish mechanisms and interfaces to coordinate flexible fault management across hardware and software components. The framework also enables optimization of the cost-benefit trade-os among performance, resilience, and power consumption. The overall goal of this work is to enable a systematic methodology for the design and evaluation of resilience technologies in extreme-scale HPC systems that keep scientific applications running to a correct solution in a timely and cost-ecient manner in spite of frequent faults, errors, and failures of various types.

97 MATHEMATICS AND COMPUTING↗

Holistic Measurement Driven Resilience: Combining Operational Fault and Failure Measurements and Fault Injection for Quantifying Fault Detection, Propagation and Impact. Final report

For HPC systems to date, application resilience to faults and failures has been accomplished by the brute- force method of checkpoint/restart, which allows an application to make forward progress in the face of system and application faults, errors, and failures independent of root cause or end result. It has remained the primary resilience mechanism because we lack a way to identify faults and anticipate consequences early enough to take meaningful mitigating action. However, checkpoint/restart implementations put a tremendous burden on system resources and on the applications themselves and is becoming less feasible at scale. Because we have not yet operated at scales at which checkpoint/restart fails to provide forward progress, despite increasing costs, vendors have had little motivation to provide the instrumentation necessary for early identification of faults and failures. However, as we move from petascale to exascale, component mean time to failure (MTTF) will render the existing techniques ineffectual and/or too expensive. Furthermore, fault recovery mechanisms such as failover and/or error correction introduce performance inconsistency. Instrumentation allowing early indication of problems and tools to enable use of such information by systems, operating systems, and applications offer an alternative, more scalable and less costly solution. In the HMDR project, we built on our experience and expertise developed and accumulated over years of research on design, monitoring, measurement, and assessment of resilient computing systems. Analysis of field data on the current and past generations of extreme-scale systems revealed several challenges that, if not addressed in increasingly larger and more complex systems, may hinder the effectiveness of future exascale computing systems. Specifically, i) file systems and interconnects in current-generation large-scale systems already operate at the margins of resiliency, including consistent performance, and may not scale to larger deployments; ii) automated, software-based failover mechanisms are frequently inadequate and can introduce wider failures, such that failures during recovery may lead to system/application failures, including system-wide outages; and iii) silent data corruption represents a critical fault mode and will require efficient detection mechanisms if next-generation applications are to take full advantage of exascale hardware. To address the above challenges, we assembled a team of world-renowned experts in resilient extreme- scale computing from the University of Illinois (Electrical and Computer Engineering, Computer Science, and NCSA), SNL, LANL, NERSC, and Cray. Our team includes representatives from centers that house many of the largest HPC resources in the world, both today and over the coming years. The team has a unique track record of research in i) system and application failure characterization based on the analysis of field data, ii) data-driven design of fault/error detection mechanisms, and iii) experimental characterization of system/application resiliency. The team includes system owners/operators who provide continuous data collection and access and ensure installation of appropriate analysis tools.

97 MATHEMATICS AND COMPUTING↗

A Pattern Language for High-Performance Computing Resilience

High-performance computing systems (HPC) provide powerful capabilities for modeling, simulation, and data analytics for a broad class of computational problems. They enable extreme performance of the order of quadrillion floating-point arithmetic calculations per second by aggregating the power of millions of compute, memory, networking and storage components. With the rapidly growing scale and complexity of HPC systems for achieving even greater performance, ensuring their reliable operation in the face of system degradations and failures is a critical challenge. System fault events often lead the scientific applications to produce incorrect results, or may even cause their untimely termination. The sheer number of components in modern extreme-scale HPC systems and the complex interactions and dependencies among the hardware and software components, the applications, and the physical environment makes the design of practical solutions that support fault resilience a complex undertaking. To manage this complexity, we developed a methodology for designing HPC resilience solutions using design patterns. We codified the well-known techniques for handling faults, errors and failures that have been devised, applied and improved upon over the past three decades in the form of design patterns. In this paper, we present a pattern language to enable a structured approach to the development of HPC resilience solutions. The pattern language reveals the relations among the resilience patterns and provides the means to explore alternative techniques for handling a specific fault model that may have different efficiency and complexity characteristics. Using the pattern language enables the design and implementation of comprehensive resilience solutions as a set of interconnected resilience patterns that can be instantiated across layers of the system stack.

Hukerikar, Saurabh↗

Estimating Orion Heat Shield Failure Due To Ablator Cracking During The EFT-1 Mission

The Orion EFT-1 heatshield suffered from two major certification challenges: First, the mechanical properties used in design were not evident in the flight hardware and second, the flight article itself cracked during fabrication. The combination of these events motivated the Orion Program to pursue an engineering-level Probabilistic Risk Assessment (PRA) as part of heatshield certification rationale. The PRA provided loss of Mission (LOM) likelihoods considering the probability of a crack occurring during the mission and the likelihood of subsequent structure over-temperature. The methods and input data for the PRA are presented along with a discussion of the test data used to anchor the results. The Orion program accepted an EFT-1 Loss of Vehicle (LOV) risk of 1-in-160,000 due to in-mission Avcoat cracking based on the results of this analysis. Conservatisms in the result, along with future considerations for Exploration Missions (EM) are also addressed.

Avcoat↗

A Systematic Approach to Hardware Qualification

A systematic approach for the development of a hardware qualification approach is described. This approach stems from the need to break the "Catch-22" of not being able to fly new technology because it hasn't flown. A physics of failure approach is used to identify failure modes and the impact and likelihood of these failures on the mission requirements is plotted in a Requirements Matrix. These same failure modes are plotted against the effectiveness of the available Preventions, Analyses, Control and Tests (PACTs) at screening for, or eliminating, these failure modes in a Test Effectiveness Matrix. Matrix multiplication results in a ranked set of PACTs which can be sorted according to cost and redundancy with other PACTs. This, and other information which results from the process, will allow project managers to make more informed decisions regarding the cost and risk tradeoffs inherent in any qualification program.

hardware↗

The GSFC Combined Approach of ODC Stockpiling and Tribological Testing to Mitigate the Risks of ODC Elimination

In response to the elimination of production of several Ozone Depleting Chemicals (ODCs) which have been widely used in successful space flight mechanism cleaning and lubricating procedures, GSFC developed and implemented an overall philosophy of mitigating the risks to flight hardware during the transition phase to ODC-Free cleaning procedures. One leg of that philosophy is the initiation of a several tier testing program which will deliver increasing amounts of information over the next few years, starting with original surface analysis comparisons between ODC and various ODC-Free cleaning technologies. The other leg is the stockpiling of an appropriate amount of ODC solvents such that all short term GSFC missions will be able to stay with or revert to heritage cleaning and lubricating procedures in the face of life issues. While tribological testing, mechanism life testing and space-flight experience will ultimately bring us into the 21st century with environmentally friendly means of cleaning long-life precision mechanism components, many satellites will be launched over the next few years with a number of important tribological questions unanswered. In order to prepare for this challenge, the Materials Engineering Branch in cooperation with the Electromechanical Branch launched an intensive review of all ongoing missions. The failure risk was determined for each long-life mechanism based on a number of parameters, including a comparison of flight solvents used to clean the heritage/life test hardware. Also studied was the ability of the mechanism manufacturers to stockpile ODCs based on state laws and company policies. A stockpiling strategy was constructed based on this information and subsequently implemented. This paper provides an overview of the GSFC ODC elimination risk mitigation philosophy as well as a detailed examination of the development of the ODC stockpiling plan.

Predmore, Roamer↗