Practical Issues in Implementing Software Reliability Measurement
Many ways of estimating software systems' reliability, or reliability-related quantities, have been developed over the past several years.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Many ways of estimating software systems' reliability, or reliability-related quantities, have been developed over the past several years.
During the six-month period from 1 April 1991 to 30 September 1991 the following research papers in statistical modeling of software reliability appeared: (1) A Nonparametric Software Reliability Growth Model; (2) On the Use and the Performance of Software Reliability Growth Models; (3) Research and Development Issues in Software Reliability Engineering; (4) Special Issues on Software; and (5) Software Reliability and Safety.
NASA policy and procedures are established which govern the selection, testing, and application of electrical, electronic, and electromechanical (EEE) parts. Recent advances in the state-of-the-art of electronic parts and associated technologies can significantly impact the electronic designs and reliability of NASA space transportation avionics. Significant issues that result from these advances are examined, including: recent advances in microelectronics technology (as applied to or considered for use in NASA projects); electron packaging technology advances (concurrent with, and as a result of, the development of the advanced microelectronic devices); availability of parts used in space avionics; and standardization and integration of parts activities between projects, centers, and contractors.
Ultra reliable systems are critical to NASA particularly as consideration is being given to extended lunar missions and manned missions to Mars. NASA has formulated a program designed to improve the reliability of NASA systems. The long term goal for the NASA ultra reliability is to ultimately improve NASA systems by an order of magnitude. The approach outlined in this presentation involves the steps used in developing a strategic plan to achieve the long term objective of ultra reliability. Consideration is given to: complex systems, hardware (including aircraft, aerospace craft and launch vehicles), software, human interactions, long life missions, infrastructure development, and cross cutting technologies. Several NASA-wide workshops have been held, identifying issues for reliability improvement and providing mitigation strategies for these issues. In addition to representation from all of the NASA centers, experts from government (NASA and non-NASA), universities and industry participated. Highlights of a strategic plan, which is being developed using the results from these workshops, will be presented.
As the design of Micro-Electro-Mechanical System (MEMS) devices matures and their application extends to critical areas, the issues of reliability and long-term survivability become increasingly important. This paper reviews some general approaches to addressing the reliability and qualification of MEMS devices for space applications.
The problem to be addressed in this paper is to explore how the use of Soft Computing Technologies (SCT) could be employed to improve overall vehicle system safety, reliability, and rocket engine performance by development of a qualitative and reliable engine control system (QRECS). Specifically, this will be addressed by enhancing rocket engine control using SCT, innovative data mining tools, and sound software engineering practices used in Marshall's Flight Software Group (FSG). The principle goals for addressing the issue of quality are to improve software management, software development time, software maintenance, processor execution, fault tolerance and mitigation, and nonlinear control in power level transitions. The intent is not to discuss any shortcomings of existing engine control methodologies, but to provide alternative design choices for control, implementation, performance, and sustaining engineering, all relative to addressing the issue of reliability. The approaches outlined in this paper will require knowledge in the fields of rocket engine propulsion (system level), software engineering for embedded flight software systems, and soft computing technologies (i.e., neural networks, fuzzy logic, data mining, and Bayesian belief networks); some of which are briefed in this paper. For this effort, the targeted demonstration rocket engine testbed is the MC-1 engine (formerly FASTRAC) which is simulated with hardware and software in the Marshall Avionics & Software Testbed (MAST) laboratory that currently resides at NASA's Marshall Space Flight Center, building 4476, and is managed by the Avionics Department. A brief plan of action for design, development, implementation, and testing a Phase One effort for QRECS is given, along with expected results. Phase One will focus on development of a Smart Start Engine Module and a Mainstage Engine Module for proper engine start and mainstage engine operations. The overall intent is to demonstrate that by employing soft computing technologies, the quality and reliability of the overall scheme to engine controller development is further improved and vehicle safety is further insured. The final product that this paper proposes is an approach to development of an alternative low cost engine controller that would be capable of performing in unique vision spacecraft vehicles requiring low cost advanced avionics architectures for autonomous operations from engine pre-start to engine shutdown.
This technical publication describes the methodology, model, software tool, input data, and analysis result that support aerospace design reliability studies. The focus of these activities is on propulsion systems mechanical design reliability. The goal of these activities is to support design from a reliability perspective. Paralleling performance analyses in schedule and method, this requires the proper use of metrics in a validated reliability model useful for design, sensitivity, and trade studies. Design reliability analysis in this view is one of several critical design functions. A design reliability method is detailed and two example analyses are provided-one qualitative and the other quantitative. The use of aerospace and commercial data sources for quantification is discussed and sources listed. A tool that was developed to support both types of analyses is presented. Finally, special topics discussed include the development of design criteria, issues of reliability quantification, quality control, and reliability verification.
This paper examines the problems encountered in assessing the reliability of dye-penetrant nondestructive inspection (NDI) techniques in preventing failures due to undetected surface flaws, as well as from flaw acceptance (Fitness-For-Purpose). A Monte Carlo simulation procedure which includes the major variables of the problem is presented as a means of quantifying reliability. Some issues associated with distribution selection are examined. A methodology for selecting the penetrant type and flaw acceptance size for the specific components analyzed using the simulation is proposed. Current methodology limitations are discussed along with possible future effort. Penetrant selection and acceptable sizes of detected flaws are based on a probabilistic assessment of the effect of component and dye-penetrant system variables on structural reliability.
A workload prediction methodology was developed in response to the need to measure workloads associated with operation of advanced aircraft. The application of the methodology will involve: (1) conducting mission/task analyses of critical mission segments and assigning estimates of workload for the sensory, cognitive, and psychomotor workload components of each task identified; (2) developing computer-based workload prediction models using the task analysis data; and (3) exercising the computer models to produce predictions of crew workload under varying automation and/or crew configurations. Critical issues include reliability and validity of workload predictors and selection of appropriate criterion measures.
Simple heating and cooling of chemisorption and physisorption compressors provide high pressure gas, which, when expanded through a Joule-Thomson (J- T) valve, can provide cooling from 0.3 to 200 K depending on the sorbent/gas combination. System life tests are under way on Pr(1-n)Ce(n)O(x)/O2 (PCO/O2) chemisorption compressors for 65-80 K cooling and on carbon/krypton (C/Kr) physisorption compressors for 120-140 K cooling. Investigations have been initiated to examine reliability physics issues related to critical compressor components such as sorbents, container materials and electrical heaters.
To leap past the limitations of existing propulsion, the NASA Breakthrough Propulsion Physics (BPP) Project seeks further advancements in physics from which new propulsion methods can eventually be derived. Three visionary breakthroughs are sought: (1) propulsion that requires no propellant, (2) propulsion that circumvents existing speed limits, and (3) breakthrough methods of energy production to power such devices. Because these propulsion goals are presumably far from fruition, a special emphasis is to identify credible research that will make measurable progress toward these goals in the near-term. The management techniques to address this challenge are presented, with a special emphasis on the process used to review, prioritize, and select research tasks. This selection process includes these key features: (a) research tasks are constrained to only address the immediate unknowns, curious effects or critical issues, (b) reliability of assertions is more important than the implications of the assertions, which includes the practice where the reviewers judge credibility rather than feasibility, and (c) total scores are obtained by multiplying the criteria scores rather than by adding. Lessons learned and revisions planned are discussed.
The term, propulsion breakthrough, refers to concepts like propellantless space drives and faster-than-light travel, the kind of breakthroughs that would make interstellar exploration practical. Although no such breakthroughs appear imminent, a variety of investigations into these goals have begun. From 1996 to 2002, NASA supported the Breakthrough Propulsion Physics Project to examine physics in the context of breakthrough spaceflight. Three facets of these assessments are now reported: (1) predicting benefits, (2) selecting research, and (3) recent technical progress. Predicting benefits is challenging since the breakthroughs are still only notional concepts, but kinetic energy can serve as a basis for comparison. In terms of kinetic energy, a hypothetical space drive could require many orders of magnitude less energy than a rocket for journeys to our nearest neighboring star. Assessing research options is challenging when the goals are beyond known physics and when the implications of success are profound. To mitigate the challenges, a selection process is described where: (a) research tasks are constrained to only address the immediate unknowns, curious effects or critical issues, (b) reliability of assertions is more important than their implications, and (c) reviewers judge credibility rather than feasibility. The recent findings of a number of tasks, some selected using this process, are discussed. Of the 14 tasks included, six reached null conclusions, four remain unresolved, and four have opportunities for sequels. A dominant theme with the sequels is research about the properties of space, inertial frames, and the quantum vacuum.
Better performing Forward Error Correction on the forward link along with adequate power in the data open an uplink operations trade space that enable missions to: Command to greater distances in deep space (increased uplink margin). Increase the size of the payload data (latency may be a factor). Provides space for the security header/trailer of the CCSDS Space Data Link Security Protocol. Note: These higher rates could be used for relief of emergency communication margins/rates and not limited to improving top-end rate performance. A higher performance uplink could also reduce the requirements on flight emergency antenna size and/or the performance required from ground stations. Use of a selective repeat ARQ protocol may increase the uplink design requirements but the resultant development is deemed acceptable, due the factor of 4 to 8 potential increase in uplink data rate.
The issues involved in the use of the programming language Ada on distributed systems are discussed. The effects of Ada programs on hardware failures such as loss of a processor are emphasized. It is shown that many Ada language elements are not well suited to this environment. Processor failure can easily lead to difficulties on those processors which remain. As an example, the calling task in a rendezvous may be suspended forever if the processor executing the serving task fails. A mechanism for detecting failure is proposed and changes to the Ada run time support system are suggested which avoid most of the difficulties. Ada program structures are defined which allow programs to reconfigure and continue to provide service following processor failure.
With the rapidly increasing insertion of photonic devices, circuits and subsystems into NASA spacecraft, a variety of issues associated with reliability and radiation tolerance have arisen. In this paper, we discuss these issues from the perspective of the work currently ongoing in the NASA Electronic Parts and Packaging (NEPP) Program. This Program is focused on evaluating the reliability and radiation response of advanced and emerging microelectronics and photonics technologies of interest to NASA spacecraft system designers.
Many vendors produce products that are not domain specific (e.g., network server) and have limited functionality (e.g., mobile phone). In contrast, many customers of COTS develop systems that am domain specific (e.g., target tracking system) and have great variability in functionality (e.g., corporate information system). This discussion takes the viewpoint of how the customer can ensure the quality of COTS components. In evaluating the benefits and costs of using COTS, we must consider the environment in which COTS will operate. Thus we must distinguish between using a non-mission critical application like a spreadsheet program to produce a budget and a mission critical application like military strategic and tactical operations. Whereas customers will tolerate an occasional bug in the former, zero tolerance is the rule in the latter. We emphasize the latter because this is the arena where there are major unresolved problems in the application of COTS. Furthermore, COTS components may be embedded in the larger customer system. We refer to these as embedded systems. These components must be reliable, maintainable, and available, and must be with the larger system in order for the customer to benefit from the advertised advantages of lower development and maintenance costs. Interestingly, when the claims of COTS advantages are closely examined, one finds that to a great extent these COTS components consist of hardware and office products, not mission critical software [1]. Obviously, COTS components are different from custom components with respect to one or more of the following attributes: source, development paradigm, safety, reliability, maintainability, availability, security, and other attributes. However, the important question is whether they should be treated differently when deciding to deploy them for operational use; we suggest the answer is no. We use reliability as an example to justify our answer. In order to demonstrate its reliability, a COTS component must pass the same reliability evaluations as the custom components, otherwise the COTS components will be the weakest link in the chain of components and will be the determinant of software system reliability. The challenge is that there will be less information available for evaluating COTS components than for custom components but this does not mean we should despair and do nothing. Actually, there is a lot we can do even in the absence of documentation on COTS components because the customer will have information about how COTS components are to be used in the larger system. To illustrate our approach, we will consider the reliability, maintainability, and availability (RMA) of COTS components as used in larger systems. Finally, COTS suppliers might consider increasing visibility into their products to assist customers in determining the components' fitness for use in a particular application. We offer ideas of information that would be useful to customers, and what vendors might do to provide it.
This discussion takes the viewpoint of how the customer can ensure the quality of COTS components.
NREL hosts an annual Photovoltaic Reliability Workshop (PVRW) so that solar technology experts can discuss current and future issues in PV reliability. Longer-lasting PV systems make lower-cost solar electricity and have less impact on the environment, human health, and natural resources. PVRW offers a combination of oral presentations, panel discussions with vigorous audience participation, and poster presentations. This document collects posters that were presented at PVRW 2024.