Search NASA⌕ Search

SEARCH · Search NASA

Results for “hardware failure”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Flight Performance of Skylab Attitude and Pointing Control System

In 1967 a paper at the AIAA Guidance, Control and Flight Dynamics Conference in Huntsville, Ala. presented for the first time the prot)osed SKYLAB Attitude and Pointing Control System (APCS) The system requirements, Apollo Telescope Mount (ATM) configuration, control philosophy, and operational modes were presented and the APCS described. The Initial mission and system design requirements changed during the period of time before the SKYLAB was launched. This paper will review the Initial and final APCS requirements and goals and their relationship. The actual flight mission (and Its alterations during the flight) and known achieved APCS performance will then be presented. SKYLAB was a tremendous success in furthering man's scientific knowledge; but perhaps SKYLAB will be remembered more for the anomalies and the efforts undertaken to solve them. On May 14, 1973, the unmanned SKYLAB Orbital Workshop (OWS) was launched from Cape Kennedy. Serious hardware failures began to occur during ascent through the atmosphere and their spectre continued to haunt both the astronauts and their ground based support team. Nor were these the only surprises affecting the design and operation of the APCS. Mission requirements for pointing to various stellar targets and to nadir for earth resources experiments were added after the hardware was designed. The chance appearance of comet Kohoutek during the SKYLAB operational life-time caused NASA to add comet observation to the mission requirements and to adjust the time when the third crew would man the SKYLAB. The development of new procedures and software for the opportunity to observe this visitor to our solar system is described.

Chubb, W. B.↗

High Energy Yield Distributed Architecture for Large Commercial and Utility-Scale PV Systems

Photovoltaic energy has come a long way from its early days when small cells were fabricated in the labs in the early 50s. While it took a long time for PV modules to mature into what we now see in the market and in rooftops across the world, the power electronics has been lagging behind & was always the weakest point in the PV plant, being responsible for the majority of failures & robbing the PV plants of their full potential by losing precious Watts in the process of converting power from DC to AC. Since the early days of PV power, it has been recognized that the modules need to operate at their maximum power point in order to extract as much power as is available from the PV modules at a given time of day and year. Numerous maximum power point devices were developed to do just this. While effective at extracting as much power as they can, soon after there was a realization that in a PV installation where a central inverter is used, many losses occur, hence energy yield losses. This has led to thinking about alternative architectures where the maximum power point tracking (MPPT) is performed much closer to the modules rather than at a central inverter. Several architectures were proposed ranging from a microinverter, microconverter, string inverter, string optimizer, and to a multi-string inverter or optimizer. While these architectures promise to increase the energy yield by 5% to 25% (depending where MPPT is located), & recovering losses due to shading, soiling, mismatch … there were no real studies and implementations in the field to corroborate these numbers. The program that GE Global Research proposed & received funding for is aimed at a type of distributed architectures, namely the dc architectures where only the MPPT function is distributed across the PV plant. Further, the project is concerned with choosing the right architecture for large scale commercial & utility scale plants but not residential installations. Findings from Task 1- Define Requirements & Develop Architecture- showed that for large scale commercial & utility PV plants, locating the MPPT at the string level with a string size of >3kW is the most cost effective solution & that the increase in energy yield of 6% to 8% is more than enough to make up for the increased cost due to additional power electronics. Our study also shows that in addition to an increase in energy yield, having a dc/dc converter at the string level brings additional benefits such as increased array visibility, allowing for increased monitoring and diagnostics; high and constant dc bus voltage which allows for reducing losses further and optimizing the inverter; and ability to detect ground and arc faults. Task 2 focused on designing a suitable dc/dc converter that meets the requirements defined in Task 1: Efficiency, Cost, & Reliability. For this task, we surveyed and simulated all candidate topologies known by the practitioners in the field. We also generated a novel topology, simulated it, & compared it to existing topologies. Our novel topology being a partial power processing converter came out on top due to its simplicity (fewer parts), it high efficiency, its reliability, and its lower cost. In the remaining tasks of the project, we built the hardware, tested it first in the lab, then in the field and once we worked out all the bugs and issues and were confident our system was robust, we installed it on the rooftop and started collecting the data by mid-Q3 2012. We shared our work with the scientific community in the form of conference & Journal papers, hence contributing to the general understanding of this technology area. The methods, techniques, and hardware developed under this project have all been tested in the field over long periods of time, and under many conditions and have been shown to be robust and reliable. The converters built and developed under this program have been in operation in the field since mid-June 2012, and no hardware failures have been recorded. While some software issues have been seen, none have been showstoppers. The cost analysis of the hardware showed that it meets the cost target set in the requirement document developed in the trade-off study. Improving the performance of solar installations for commercial and utility scale plants contributes to reducing their cost of ownership, hence spreading them to a larger section of the society and making them affordable. Our project contributed to the understanding of the distributed architectures, showed that PV plants’ cost of electricity can be reduced through additional power electronics which allows for harvesting more energy from the plant. When the project started in 2009, the cost of PV energy was still high, hence there was a real need to increase the energy yield, hence reduce the levelized cost of electricity (LCOE). While costs have come down significantly, the need for increasing the energy yield is still relevant, hence the project results, hardware, and techniques developed are still pertinent today. By taking this project from concept to rooftop demonstration, we showed that the technology is mature enough, robust enough, and has the potential to be used by many more customers, hence benefiting the public in general through the spread of PV energy, which ultimately contributes to reducing our dependency on fossil fuel, improving the environment, and creating more jobs.

14 SOLAR ENERGY↗

Use of a Slick-Plate as a Contingency Exercise Surface for the Treadmill With Vibration Isolation System

The treadmill with vibration isolation system (TVIS) was developed to counteract cardiovascular, musculoskeletal, and neurovestibular deconditioning during long-duration missions to the International Space Station (ISS). However, recent hardware failures have necessitated the development of a short-term, temporary contingency exercise countermeasure for TVIS until nominal operations could be restored. The purpose of our evaluation was twofold: 1) to examine whether a slick-plate/contingency exercise surface (CES) could be used as a walking/running surface and could elicit a heart rate (HR) greater than or equal to 70% HR maximum and 2) to determine the optimal hardware configuration, in microgravity, to simulate running/walking in a 1-g environment. One subject (male) participated in the slick surface evaluation and two subjects (one male, one female) participated in the microgravity evaluation of the slick surface configuration. During the slick surface evaluation, the subject was suspended in a parachute harness and bungee cord configuration to offset the subject's body weight. Using another bungee cord configuration, we added a vertical load back to the subject, who was then asked to run for 20 minutes on the slick surface. The microgravity evaluation simulated the ISS TVIS, and we evaluated two different slick surfaces (Teflon surface and an aluminum surface coated with Tufram) for use as a CES. We evaluated each surface with the subject walking and running, with and without a handrail, and while wearing either socks or nylon booties over shoes. In the slick surface evaluation, the subject ran for 20 minutes and reached a maximum HR of 170 bpm. In the microgravity evaluation, the subjects chose the aluminum plate coated with Tufram as the CES, while wearing a pair of nylon booties over running shoes and using a handrail, as the optimal hardware configuration. The results indicate that the CES may provide an interim capability to counteract aerobic deconditioning until TVIS can be returned to an operational status. No indices of musculoskeletal or neurovestibular deconditioning were evaluated. Future studies are needed to validate the efficacy of CES in countering aerobic deconditioning and the effects, if any, on musculoskeletal and neurovestibular deconditioning.

James A. Loehr↗

Fault tolerant distributed systems using Ada

This paper discusses the use of Ada on distributed systems in which failure of processors has to be tolerated. It is assumed that communication between tasks on separate processors will take place using the facilities of the Ada language, primarily the rendezvous. It is shown that there are numerous aspects of the language which make its use on a distributed system very difficult. The issues are raised from the desire to be able to recover, reconfigure, and provide continued service in the presence of hardware failure. For example, if a rendezvous takes place between two tasks on different processors, failure of the processor executing the serving task will cause the calling task to be permanently suspended because the rendezvous will never end. Extensive modifications to the execution support required for Ada are proposed which provide all the necessary facilities for programs written in Ada to withstand arbitrary processor failure. Mechanisms are suggested to allow processor failure to be detected and for tasks which would be permanently suspended to be released. Provided the required program structures are used, continued processing can be provided.

Knight, J. C.↗

Automated Mixed Traffic Vehicle (AMTV) technology and safety study

Technology and safety related to the implementation of an Automated Mixed Traffic Vehicle (AMTV) system are discussed. System concepts and technology status were reviewed and areas where further development is needed are identified. Failure and hazard modes were also analyzed and methods for prevention were suggested. The results presented are intended as a guide for further efforts in AMTV system design and technology development for both near term and long term applications. The AMTV systems discussed include a low speed system, and a hybrid system consisting of low speed sections and high speed sections operating in a semi-guideway. The safety analysis identified hazards that may arise in a properly functioning AMTV system, as well as hardware failure modes. Safety related failure modes were emphasized. A risk assessment was performed in order to create a priority order and significant hazards and failure modes were summarized. Corrective measures were proposed for each hazard.

Johnston, A. R.↗

Evaluating the Performance of the NASA LaRC CMF Motion Base Safety Devices

This paper describes the initial measured performance results of the previously documented NASA Langley Research Center (LaRC) Cockpit Motion Facility (CMF) motion base hardware safety devices. These safety systems are required to prevent excessive accelerations that could injure personnel and damage simulator cockpits or the motion base structure. Excessive accelerations may be caused by erroneous commands or hardware failures driving an actuator to the end of its travel at high velocity, stepping a servo valve, or instantly reversing servo direction. Such commands may result from single order failures of electrical or hydraulic components within the control system itself, or from aggressive or improper cueing commands from the host simulation computer. The safety systems must mitigate these high acceleration events while minimizing the negative performance impacts. The system accomplishes this by controlling the rate of change of valve signals to limit excessive commanded accelerations. It also aids hydraulic cushion performance by limiting valve command authority as the actuator approaches its end of travel. The design takes advantage of inherent motion base hydraulic characteristics to implement all safety features using hardware only solutions.

Gupton, Lawrence E.↗

Evolution of safety-critical requirements post-launch

This paper reports the results of a small study of requirements changes to the onboard software of three spacecraft subsequent to launch. Only those requirement changes that resulted from post-launch anoma-lies (i.e., during operations) were of interest here, since the goal was to better understand the relation-ship between critical anomalies during operations and how safety-critical requirements evolve. The results of the study were surprising in that anomaly-driven, post-launch requirements changes were rarely due to previous requirements having been incorrect. Instead, changes involved new requirements (1) for the software to handle rare events or (2) for the software to compensate for hardware failures or limitations. The prevalence of new requirements as a result of post-launch anomalies suggests a need for increased requirements-engineering support of maintenance activities in these systems. The results also confirm both the difficulty and the benefits of pursuing requirements completeness, especially in terms of fault tolerance, during development of critical systems.

Software requirements↗

Review of Quantitative Software Reliability Methods

The current U.S. Nuclear Regulatory Commission (NRC) licensing process for digital systems rests on deterministic engineering criteria. In its 1995 probabilistic risk assessment (PRA) policy statement, the Commission encouraged the use of PRA technology in all regulatory matters to the extent supported by the state-of-the-art in PRA methods and data. Although many activities have been completed in the area of risk-informed regulation, the risk-informed analysis process for digital systems has not yet been satisfactorily developed. Since digital instrumentation and control (I&C) systems are expected to play an increasingly important role in nuclear power plant (NPP) safety, the NRC established a digital system research plan that defines a coherent set of research programs to support its regulatory needs. One of the research programs included in the NRC's digital system research plan addresses risk assessment methods and data for digital systems. Digital I&C systems have some unique characteristics, such as using software, and may have different failure causes and/or modes than analog I&C systems; hence, their incorporation into NPP PRAs entails special challenges. The objective of the NRC's digital system risk research is to identify and develop methods, analytical tools, and regulatory guidance for (1) including models of digital systems into NPP PRAs, and (2) using information on the risks of digital systems to support the NRC's risk-informed licensing and oversight activities. For several years, Brookhaven National Laboratory (BNL) has worked on NRC projects to investigate methods and tools for the probabilistic modeling of digital systems, as documented mainly in NUREG/CR-6962 and NUREG/CR-6997. However, the scope of this research principally focused on hardware failures, with limited reviews of software failure experience and software reliability methods. NRC also sponsored research at the Ohio State University investigating the modeling of digital systems using dynamic PRA methods. These efforts, documented in NUREG/CR-6901, NUREG/CR-6942, and NUREG/CR-6985, included a functional representation of the system's software but did not explicitly address failure modes caused by software defects or by inadequate design requirements. An important identified research need is to establish a commonly accepted basis for incorporating the behavior of software into digital I&C system reliability models for use in PRAs. To address this need, BNL is exploring the inclusion of software failures into the reliability models of digital I&C systems, such that their contribution to the risk of the associated NPP can be assessed.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Dynamic Modeling of Off-Nominal Operation in Advanced Life Support Systems

System failures, off-nominal operation, or unexpected interruptions in processing capability can cause unanticipated instabilities in Advanced Life Support (ALS) systems, even long after they are repaired. Much current modeling assumes ALS systems are static and linear, but ALS systems are actually dynamic and nonlinear, especially when failures and off nominal operation are considered. Modeling and simulation provide a way to study the stability and time behavior of nonlinear dynamic ALS systems under changed system configurations or operational scenarios. The dynamic behavior of a nonlinear system can be fully explored only by computer simulation over the full range of inputs and initial conditions. Previous simulations of BIO-Plex in SIMULINK, a toolbox of Matlab, were extended to model the off-nominal operation and long-term dynamics of partially closed physical/chemical and bioregenerative life support systems. System nonlinearity has many interesting potential consequences. Different equilibrium points may be reached for different initial conditions. The system stability can depend on the exact system inputs and initial conditions. The system may oscillate or even in rare cases behave chaotically. Temporary internal hardware failures or external perturbations in ALS systems can lead to dynamic instability and total ALS system failure. Appropriate control techniques can restore reliable operation and minimize the effects of dynamic instabilities due to anomalies or perturbations in a life support system.

Jones, Harry↗

Optimal Aircraft Control Upset Recovery With and Without Component Failures

This paper treats the problem of recovering sustainable nondescending (safe) flight in a transport aircraft after one or more of its control effectors fail. Such recovery can be a challenging goal for many transport aircraft currently in the operational fleet for two reasons. First, they have very little redundancy in their means of generating control forces and moments. These aircraft have, as primary control surfaces, a single rudder and pairwise elevators and aileron/spoiler units that provide yaw, pitch, and roll moments with sufficient bandwidth to be used in stabilizing and maneuvering the airframe. Beyond this, throttling the engines can provide additional moments, but on a much slower time scale. Other aerodynamic surfaces, such as leading and trailing edge flaps, are only intended to be placed in a position and left, and are, hence, very slow-moving. Because of this, loss of a primary control surface strongly degrades the controllability of the vehicle, particularly when the failed effector becomes stuck in a non-neutral position where it exerts a disturbance moment that must be countered by the remaining operating effectors. The second challenge in recovering safe flight is that these vehicles are not agile, nor can they tolerate large accelerations. This is of special importance when, at the outset of the recovery maneuver, the aircraft is flying toward the ground, as is frequently the case when there are major control hardware failures. Recovery of safe flight is examined in this paper in the context of trajectory optimization. For a particular transport aircraft, and a failure scenario inspired by an historical air disaster, recovery scenarios are calculated with and without control surface failures, to bring the aircraft to safe flight from the adverse flight condition that it had assumed, apparently as a result of contact with a vortex from a larger aircraft's wake. An effort has been made to represent relevant airframe dynamics, acceleration limits, and actuator limits faithfully, since these contribute to the lack of agility and control power that plays an important role in defining what can be achieved with the vehicle when it is in extremis.

Sparks, Dean W.↗

Asynchronous distributed-memory task-parallel algorithm for compressible flows on unstructured 3D Eulerian grids

Here, we discuss the implementation of a finite element method, used to numerically solve the Euler equations of compressible flows, using an asynchronous runtime system (RTS). The algorithm is implemented for distributed-memory machines, using stationary unstructured 3D meshes, combining data-, and task-parallelism on top of the Charm++ RTS. Charm++’s execution model is asynchronous by default, allowing arbitrary overlap of computation and communication. Task-parallelism allows scheduling parts of an algorithm independently of, or dependent on, each other. Built-in automatic load balancing enables continuous redistribution of computational load by migration of work units based on real-time CPU load measurement. The RTS also features automatic checkpointing, fault tolerance, resilience against hardware failure, and supports power-, and energy-aware computation. We demonstrate scalability up to 25 x 10 9 cells at $\mathscr{O}$10 4 compute cores and the benefits of automatic load balancing for irregular workloads. The full source code with documentation is available at https://quinoacomputing.org.

42 ENGINEERING↗

Dynamic File Striping and Data Layout Transformation on Parallel System with Fluctuating I/O Workload

As the number of compute cores on modern parallel machines increases to more than hundreds of thousands, scalable and consistent I/O performance is becoming hard to obtain due to fluctuating file system performance. This fluctuation is often caused by rebuilding RAID disk from hardware failures or concurrent jobs competing for I/O. We present a mechanism that stripes across a dynamically-selected subset of I/O servers with the lightest workload to achieve the best I/O bandwidth available from the system. We implement this mechanism into an I/O software layer that enables memory-to-file data layout transformation and allows transparent file partitioning. File partitioning is a technique that divides data among a set of files and manages file access, making data appear as a single file to users. Experimental results on NERSC’s Hopper indicate that our approach effectively isolates I/O variation on shared systems and improves overall I/O performance significantly.

Son, Seung Woo↗

Detecting Anomalies from End-to-End Internet Performance Measurements (PingER) Using Cluster Based Local Outlier Factor

PingER (Ping End-to-End Reporting) is a worldwide end-to-end Internet performance measurement framework. It was developed by the SLAC National Accelerator Laboratory, Stanford, USA and running from the last 20 years. It has more than 700 monitoring agents and remote sites which monitor the performance of Internet links around 170 countries of the world. At present, the size of the compressed PingER data set is about 60 GB comprising of 100,000 flat files. The data is publicly available for valuable Internet performance analyses. However, the data sets suffer from missing values and anomalies due to congestion, bottleneck links, queuing overflow, network software misconfiguration, hardware failure, cable cuts, and social upheavals. Therefore, the objective of this paper is to detect such performance drops or spikes labeled as anomalies or outliers for the PingER data set. In the proposed approach, the raw text files of the data set are transformed into a PingER dimensional model. The missing values are imputed using the k-NN algorithm. The data is partitioned into similar instances using the k-means clustering algorithm. Afterward, clustering is integrated with the Local Outlier Factor (LOF) using the Cluster Based Local Outlier Factor (CBLOF) algorithm to detect the anomalies or outliers from the PingER data. Lastly, anomalies are further analyzed to identify the time frame and location of the hosts generating the major percentage of the anomalies in the PingER data set ranging from 1998 to 2016.

97 MATHEMATICS AND COMPUTING↗

Mapping to Irregular Torus Topologies and Other Techniques for Petascale Biomolecular Simulation, In: SC '14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Currently deployed petascale supercomputers typically use toroidal network topologies in three or more dimensions. While these networks perform well for topology-agnostic codes on a few thousand nodes, leadership machines with 20,000 nodes require topology awareness to avoid network contention for communication-intensive codes. Topology adaptation is complicated by irregular node allocation shapes and holes due to dedicated input/output nodes or hardware failure. In the context of the popular molecular dynamics program NAMD, we present methods for mapping a periodic 3-D grid of fixed-size spatial decomposition domains to 3-D Cray Gemini and 5-D IBM Blue Gene/Q toroidal networks to enable hundred-million atom full machine simulations, and to similarly partition node allocations into compact domains for smaller simulations using multiple copy algorithms. Additional enabling techniques are discussed and performance is reported for NCSA Blue Waters, ORNL Titan, ANL Mira, TACC Stampede, and NERSC Edison.

Computer Science↗

Fault Oblivious eXascale Whitepaper

In this paper we present a software system which supports dynamic, irregular, adaptive applications. Data objects are created and structured in a hierarchical manner, with replication as needed to provide a high degree of redundancy. The data objects can contain data, code, tasks (work descriptors with references to data, code, and other tasks) and higher level structures such as work queues. The higher level structures benefit from the properties of the data objects: redundant storage to support resiliency in the face of hardware failure; hierarchical structure to optimize use of the HPC system; and a presence of object names, available in the per-user file system name space, which allows any application, not just specially written HPC applications, to make use of the data even while it is on the HPC system. Our use of hierarchy will make the runtime scalable to very large systems. Our use of redundancy will allow programs to be written in a fault-oblivious manner, eliminating the need for system-level checkpointing. Putting data object names into the file system name space allows for interactive use of the system by users. With this approach, we will be able to finally leave the batch era behind, a half-century after the invention of time sharing. We will be able to stop bounding program through- put by the checkpoint interval. Application data will be accessible at any time, not hidden behind opaque 128-bit pointers or MPI ranks, but given a name that is visible everywhere. Programmers can stop laying out data, and thinking about where the data is, and the code is, and the nodes are, and stick with the problem of what the application is supposed to be doing. This work, if it succeeds, will enable scientific computing to scale to the next generation of machines.

99 GENERAL AND MISCELLANEOUS↗

Adding Fault Tolerance to NPB Benchmarks Using ULFM

In the world of high-performance computing, fault tolerance and application resilience are becoming some of the primary concerns because of increasing hardware failures and memory corruptions. While the research community has been investigating various options, from system-level solutions to application-level solutions, standards such as the Message Passing Interface (MPI) are also starting to include such capabilities. The current proposal for MPI fault tolerant is centered around the User-Level Failure Mitigation (ULFM) concept, which provides means for fault detection and recovery of the MPI layer. This approach does not address application-level recovery, which is currently left to application developers. In this work, we present a modification of some of the benchmarks of the NAS parallel benchmark (NPB) to include support of the ULFM capabilities as well as application-level strategies and mechanisms for application-level failure recovery. As such, we present: (i) an application-level library to "checkpoint" and restore data, (ii) extensions of NPB benchmarks for fault tolerance based on different strategies, (iii) a fault injection tool, and (iv) some preliminary results that show the impact of such fault tolerant strategies on the application execution.

97 MATHEMATICS AND COMPUTING↗

HSNMon: High-Speed Monitoring Software for Omni-Path

HSNMon is a robust high speed network monitoring daemon that provides insight into network availability, hardware failure, and network performance for switches and network cards. HSNMon collects counters, forwarding tables, and error information at high resolution using publicly available Omni-Path Management APIs. The collected data can be forwarded into multiple logging infrastructures including syslog and message queuing systems. HSNMon is only the infrastructure for collecting the network data. It does not include any collected data or proprietary LANL information

Holman, Brett↗

Exploring the feasibility of lossy compression for PDE simulations

Checkpoint restart plays an important role in high-performance computing (HPC) applications, allowing simulation runtime to extend beyond a single job allocation and facilitating recovery from hardware failure. Yet, as machines grow in size and in complexity, traditional approaches to checkpoint restart are becoming prohibitive. Current methods store a subset of the application’s state and exploit the memory hierarchy in the machine. However, as the energy cost of data movement continues to dominate, further reductions in checkpoint size are needed. Lossy compression, which can significantly reduce checkpoint sizes, offers a potential to reduce computational cost in checkpoint restart. This article investigates the use of numerical properties of partial differential equation (PDE) simulations, such as bounds on the truncation error, to evaluate the feasibility of using lossy compression in checkpointing PDE simulations. Restart from a checkpoint with lossy compression is considered for a fail-stop error in two time-dependent HPC application codes: PlasComCM and Nek5000. Results show that error in application variables due to a restart from a lossy compressed checkpoint can be masked by the numerical error in the discretization, leading to increased efficiency in checkpoint restart without influencing overall accuracy in the simulation.

97 MATHEMATICS AND COMPUTING↗