Estimating and controlling software fault content more effectively
Explore the source record for details and available documents.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
As part of our on-going work in modeling software faults, we have developed a method of unambiguously identifying and counting faults.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
This experience report describes the lessons learned from the use of Software Failure Modes and.
Ares I-X was the first test flight of NASA's Constellation Program's Ares I Crew Launch Vehicle designed to provide manned access to low Earth orbit. As a one-time test flight, the Air Force's 45th Space Wing required a series of Range Safety analysis data products to be developed for the specified launch date and mission trajectory prior to granting flight approval on the Eastern Range. The range safety data package is required to ensure that the public, launch area, and launch complex personnel and resources are provided with an acceptable level of safety and that all aspects of prelaunch and launch operations adhere to applicable public laws. The analysis data products, defined in the Air Force Space Command Manual 91-710, Volume 2, consisted of a nominal trajectory, three sigma trajectory envelopes, stage impact footprints, acoustic intensity contours, trajectory turn angles resulting from potential vehicle malfunctions (including flight software failures), characterization of potential debris, and debris impact footprints. These data products were developed under the auspices of the Constellation's Program Launch Constellation Range Safety Panel and its Range Safety Trajectory Working Group with the intent of beginning the framework for the operational vehicle data products and providing programmatic review and oversight. A multi-center NASA team in conjunction with the 45th Space Wing, collaborated within the Trajectory Working Group forum to define the data product development processes, performed the analyses necessary to generate the data products, and performed independent verification and validation of the data products. This paper outlines the Range Safety data requirements and provides an overview of the processes established to develop both the data products and the individual analyses used to develop the data products, and it summarizes the results of the analyses required for the Ares I-X launch.
Ares I-X was the first test flight of NASA's Constellation Program's Ares I Crew Launch Vehicle designed to provide manned access to low Earth orbit. As a one-time test flight, the Air Force's 45th Space Wing required a series of Range Safety analysis data products to be developed for the specified launch date and mission trajectory prior to granting flight approval on the Eastern Range. The range safety data package is required to ensure that the public, launch area, and launch complex personnel and resources are provided with an acceptable level of safety and that all aspects of prelaunch and launch operations adhere to applicable public laws. The analysis data products, defined in the Air Force Space Command Manual 91-710, Volume 2, consisted of a nominal trajectory, three sigma trajectory envelopes, stage impact footprints, acoustic intensity contours, trajectory turn angles resulting from potential vehicle malfunctions (including flight software failures), characterization of potential debris, and debris impact footprints. These data products were developed under the auspices of the Constellation's Program Launch Constellation Range Safety Panel and its Range Safety Trajectory Working Group with the intent of beginning the framework for the operational vehicle data products and providing programmatic review and oversight. A multi-center NASA team in conjunction with the 45th Space Wing, collaborated within the Trajectory Working Group forum to define the data product development processes, performed the analyses necessary to generate the data products, and performed independent verification and validation of the data products. This paper outlines the Range Safety data requirements and provides an overview of the processes established to develop both the data products and the individual analyses used to develop the data products, and it summarizes the results of the analyses required for the Ares I-X launch.
The National Aeronautics and Space Administration (NASA) Safety & Mission Assurance (S&MA) directorate at the Johnson Space Center (JSC) has applied its knowledge and experience with Probabilistic Risk Assessment (PRA) to projects in industries ranging from spacecraft to nuclear power plants. PRA is a comprehensive and structured process for analyzing risk in complex engineered systems and/or processes. The PRA process enables the user to identify potential risk contributors such as, hardware and software failure, human error, and external events. Recent developments in the oil and gas industry have presented opportunities for NASA to lend their PRA expertise to both ongoing and developmental projects within the industry. This paper provides an overview of the PRA process and demonstrates how this process was applied in estimating the probability that a Mobile Offshore Drilling Unit (MODU) operating in the Gulf of Mexico and equipped with a generically configured Dynamic Positioning System (DPS) loses location and needs to initiate an emergency disconnect. The PRA described in this paper is intended to be generic such that the vessel meets the general requirements of an International Maritime Organization (IMO) Maritime Safety Committee (MSC)/Circ. 645 Class 3 dynamically positioned vessel. The results of this analysis are not intended to be applied to any specific drilling vessel, although provisions were made to allow the analysis to be configured to a specific vessel if required.
The National Aeronautics and Space Administration (NASA) Safety & Mission Assurance (S&MA) directorate at the Johnson Space Center (JSC) has applied its knowledge and experience with Probabilistic Risk Assessment (PRA) to projects in industries ranging from spacecraft to nuclear power plants. PRA is a comprehensive and structured process for analyzing risk in complex engineered systems and/or processes. The PRA process enables the user to identify potential risk contributors such as, hardware and software failure, human error, and external events. Recent developments in the oil and gas industry have presented opportunities for NASA to lend their PRA expertise to both ongoing and developmental projects within the industry. This paper provides an overview of the PRA process and demonstrates how this process was applied in estimating the probability that a Mobile Offshore Drilling Unit (MODU) operating in the Gulf of Mexico and equipped with a generically configured Dynamic Positioning System (DPS) loses location and needs to initiate an emergency disconnect. The PRA described in this paper is intended to be generic such that the vessel meets the general requirements of an International Maritime Organization (IMO) Maritime Safety Committee (MSC)/Circ. 645 Class 3 dynamically positioned vessel. The results of this analysis are not intended to be applied to any specific drilling vessel, although provisions were made to allow the analysis to be configured to a specific vessel if required.
Autonomy of aerospace systems requires the ability for flight control systems to be able to adapt to complex uncertain dynamic environment. In spite of the five decades of research in adaptive control, the fact still remains that currently no adaptive control system has ever been deployed on any safety-critical or human-rated production systems such as passenger transport aircraft. The problem lies in the difficulty with the certification of adaptive control systems since existing certification methods cannot readily be used for nonlinear adaptive control systems. Research to address the notion of metrics for adaptive control began to appear in the recent years. These metrics, if accepted, could pave a path towards certification that would potentially lead to the adoption of adaptive control as a future control technology for safety-critical and human-rated production systems. Development of certifiable adaptive control systems represents a major challenge to overcome. Adaptive control systems with learning algorithms will never become part of the future unless it can be proven that they are highly safe and reliable. Rigorous methods for adaptive control software verification and validation must therefore be developed to ensure that adaptive control system software failures will not occur, to verify that the adaptive control system functions as required, to eliminate unintended functionality, and to demonstrate that certification requirements imposed by regulatory bodies such as the Federal Aviation Administration (FAA) can be satisfied. This presentation will discuss some of the technical issues with adaptive flight control and related V&V challenges.
Recent work by Eckhardt and Lee (1985) shows that independently developed program versions fail dependently (specifically, simultaneous failure of several is greater than would be the case under true independence). The present authors show there is a precise duality between input choice and program choice in this model and consider a generalization in which different versions can be developed using diverse methodologies. The use of diverse methodologies is shown to decrease the probability of the simultaneous failure of several versions. Indeed, it is theoretically possible to obtain versions which exhibit better than independent failure behavior. The authors try to formalize the notion of methodological diversity by considering the sequence of decision outcomes that constitute a methodology. They show that diversity of decision implies likely diversity of behavior for the different verions developed under such forced diversity. For certain one-out-of-n systems the authors obtain an optimal method for allocating diversity between versions. For two-out-of-three systems there seem to be no simple optimality results which do not depend on constraints which cannot be verified in practice.
A capability consisting of software development and experimental techniques has been developed and is described. The capability is integrated into GENOA-PFA to model polymer matrix composite (PMC) structures. The capability considers the physics and mechanics of composite materials and structure by integration of a hierarchical multilevel macro-scale (lamina, laminate, and structure) and micro scale (fiber, matrix, and interface) simulation analyses. The modeling involves (1) ply layering methodology utilizing FEM elements with through-the-thickness representation, (2) simulation of effects of material defects and conditions (e.g., voids, fiber waviness, and residual stress) on global static and cyclic fatigue strengths, (3) including material nonlinearities (by updating properties periodically) and geometrical nonlinearities (by Lagrangian updating), (4) simulating crack initiation. and growth to failure under static, cyclic, creep, and impact loads. (5) progressive fracture analysis to determine durability and damage tolerance. (6) identifying the percent contribution of various possible composite failure modes involved in critical damage events. and (7) determining sensitivities of failure modes to design parameters (e.g., fiber volume fraction, ply thickness, fiber orientation. and adhesive-bond thickness). GENOA-PFA progressive failure analysis is now ready for use to investigate the effects on structural responses to PMC material degradation from damage induced by static, cyclic (fatigue). creep, and impact loading in 2D/3D PMC structures subjected to hygrothermal environments. Its use will significantly facilitate targeting design parameter changes that will be most effective in reducing the probability of a given failure mode occurring.
The PITS Modification Script has been under development. Testing, debugging, and optimization of the script are in progress. MSIS has begun population of ODC values from historic issue data. Design of the Linker Tool is also continuing. The quarterly review meeting held on December 17,2003, allowed for a venue to discuss installing the PITS Modification Script to the production version of PITS upon completion and incorporation of ODC into an IV&V project s issue tracking process. Feedback from NASA was positive although any global implementation will likely be timed to coordinate with other activities such as establishment of an IV&V Work Breakdown Structure.
Explore the source record for details and available documents.
Three questions to which software developers want accurate, precise answers are "How can the software system fail?", "mat bad things will happen if the software fails?t', and "How many failures will the software experience?". Numerous techniques have been devised to answer these questions; three of the best known are: 1) Software Fault Tree Analysis (SFTA) 2) Software Failure Modes, Effects, and Criticality Analysis (SFMECA 3) Software Fault/Failure Modeling. SFTA and SFMECA have been successfully used to analyze the flight software for a number of robotic planetary exploration missions, including Galileo, Cassini, and Deep Space 1. Given the increasing interest in reusing software components from mission to mission, one of us has developed techniques for reusing the corresponding portions of the SFTA and SFMECA, reducing the effort required to conduct these analyses. SFTA has also been shown to be effective in analyzing the security aspects of software systems; intrusion mechanisms and effects can easily be modeled using these techniques. The Bi- Directional Safety Analysis (BDSA) method combines a forward search (similar to SFMECA) from potential failure modes to their effects, with a backward search (similar to SFTA) from feasible hazards to the contributing causes of each hazard. BDSA offers an efficient way to identify latent failures. Recent work has extended BDSA to product-line applications such as flight-instrumentation displays and developed tool support for the reuse of the failure-analysis artifacts within a product line. BDSA has also been streamlined to support those projects having tight cost and/or schedule constraints for their failure analysis efforts. We discuss lessons learned from practice, describe available tools, and identi@ some future directions for the topic. A substantial amount of research has been devoted to estimating the number of failures that a software system will experience during test and operations, as well as the number of faults that have been inserted into that system during its development. One of us has found that the amount of structural change to a system during its development is strongly related to the number of faults inserted into it. Using techniques requiring no additional effort on the part of the development organization, the required measurements of structural evolution can be easily obtained from a development effort's configuration management system and readily transformed into an estimate of fault content. So far, structure-fault relationships have been identified for source code; current work seeks to examine artifacts available earlier in the lifecycle to determine if similar relationships between structure and fault content can be found. In particular, relationships between requirements change requests and the number of faults inserted into the implemented system would provide a significant improvement in our ability to control software quality during the early development phases.
This slide presentation reviews three failures of software and how the failures contributed to or caused the failure of a launch or payload insertion into orbit. In order to avoid these systematic failures in the future, failure mitigation strategies are suggested for use.
Quality of software not only is vital to the successful operation of the space station, it is also an important factor in establishing testing requirements, time needed for software verification and integration as well as launching schedules for the space station. Defense of management decisions can be greatly strengthened by combining engineering judgments with statistical analysis. Unlike hardware, software has the characteristics of no wearout and costly redundancies, thus making traditional statistical analysis not suitable in evaluating reliability of software. A statistical model was developed to provide a representation of the number as well as types of failures occur during software testing and verification. From this model, quantitative measure of software reliability based on failure history during testing are derived. Criteria to terminate testing based on reliability objectives and methods to estimate the expected number of fixings required are also presented.
Explore the source record for details and available documents.