Practical Software Quality: a Guide in Progress
Presentation intended to define software quality objectives when performing static code analysis on critical software to maximize assurance.
Software Quality Assurance↗
SEARCH · Search NASA
Results for “Software Quality Assurance”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Quality measures and assurance for AI (Artificial Intelligence) software
This report is concerned with the application of software quality and evaluation measures to AI software and, more broadly, with the question of quality assurance for AI software. Considered are not only the metrics that attempt to measure some aspect of software quality, but also the methodologies and techniques (such as systematic testing) that attempt to improve some dimension of quality, without necessarily quantifying the extent of the improvement. The report is divided into three parts Part 1 reviews existing software quality measures, i.e., those that have been developed for, and applied to, conventional software. Part 2 considers the characteristics of AI software, the applicability and potential utility of measures and techniques identified in the first part, and reviews those few methods developed specifically for AI software. Part 3 presents an assessment and recommendations for the further exploration of this important area.
Rushby, John↗
Requirements management at NASA
Requirements have always been acknowledged as the backbone of any system. However, in many past development efforts, requirements were paid little heed. At NASA, in recent years, the hue and cry for project development has been "Faster, Better, Cheaper and Safer". This has impacted the way we develop software; it has increased the risks to quality, safety and reliability. At NASA, the Software Assurance Technology Center (SATC) is working with projects to emphasize the criticality of requirements throughout development, not just in the initial phases. This emphasis is on requirements relationship to all aspects of quality, including reliability and safety. In this presentation, we will look at some of these relationships through the eyes of quality.
Requirement management↗
NASA Tech Briefs, March 2004
Topics covered include: 1) Advanced Signal Conditioners for Data-Acquisition Systems; 2) Downlink Data Multiplexer; 3) Viewing ISS Data in Real Time via the Internet; 4) Autonomous Environment-Monitoring Networks; 5) Readout of DSN Monitor Data; 6) Parallel-Processing Equalizers for Multi-Gbps Communications; 7) AIN-Based Packaging for SiC High-Temperature Electronics; 8) Software for Optimizing Quality Assurance of Other Software; 9) The TechSat 21 Autonomous Sciencecraft Experiment; 10) Software for Analyzing Laminar-to-Turbulent Flow Transitions; 11) Elastomer Filled With Single-Wall Carbon Nanotubes; 12) Modifying Ship Air-Wake Vortices for Aircraft Operations; 13) Strain-Gauge Measurement of Weight of Fluid in a Tank; 14) Advanced Docking System With Magnetic Initial Capture; 15) Blade-Pitch Control for Quieting Tilt-Rotor Aircraft; 16) Solar Array Panels With Dust-Removal Capability; 17) Aligning Arrays of Lenses and Single-Mode Optical Fibers; 18) Automatic Control of Arc Process for Making Carbon Nanotubes; 19) Curved-Focal-Plane Arrays Using Deformed-Membrane Photodetectors; 20) Role of Meteorology in Flights of a Solar-Powered Airplane; 21) Model of Mixing Layer With Multicomponent Evaporating Drops; 22) Solution-Assisted Optical Contacting; 23) Improved Discrete Approximation of Laplacian of Gaussian; 24) Utilizing Expert Knowledge in Estimating Future STS Costs; 25) Study of Rapid-Regression Liquefying Hybrid Rocket Fuels; and 26) More About the Phase-Synchronized Enhancement Method.
Source record↗
Initial Development of Fusion Magnet Simulation Capabilities for Performance and Safety Evaluation Using the MOOSE Framework
Fusion energy holds the promise of being a transformative technology as a carbon-neutral, sustainable source of energy. Whole device modeling and the development of fusion digital twins will be increasingly important for emerging fusion device concepts at both national laboratories and within the commercial fusion industry. However, meeting the challenge of whole device modeling of fusion energy devices requires robust, multiphysics, multiscale modeling and simulation technologies capable of running on large-scale supercomputers. Detailed analysis of individual systems at-scale is also required to ensure safe and efficient operation as well as provide the safety basis for future device designs and licensing activities. In a tokamak, toroidal and poloidal magnets confine and shape the fusion plasma to promote the fusion reaction. High plasma temperatures and high magnetic field requirements in modern design concepts (leading to high amounts of energy stored within each magnet) impose electrical, thermal, and mechanical loads on the magnet components, which in turn impacts the safety considerations of the magnet and their supporting systems. Idaho National Laboratory (INL) has a history of working in this space, including development and benchmarking of the Magnetic System Circuitry Analysis Program (MSCAP) and Magnet Arcing (MAGARC) codes to study magnet quench events; notably, MAGARC was used to study quenching during the ITER Engineering Design Activity. However, these legacy codes and capabilities are not parallel and scalable, and new tools are required for future advances in this area, which leads to the INL-developed Multiphysics Object-Oriented Simulation Environment (MOOSE) framework. Developed originally for fission reactor systems under United States Department of Energy, Office of Nuclear Energy modeling and simulation programs, the MOOSE framework has been well-suited to multiscale, multiphysics modeling and simulation needs for nuclear systems. The framework is open-source, well-tested, under continuous development and deployment, and developed to a Nuclear Quality Assurance, Level 1 software quality standard. MOOSE has also been used in the fusion space previously in several projects: INL’s Tritium Migration Analysis Program, Version 8 (TMAP8) for tritium migration, UK Atomic Energy Authority’s A Unified Resource for OpenMC (fusion) Reactor Applications (AURORA) code for fusion thermo-mechanical and neutronics analysis, and Argonne National Laboratory’s Cardinal for high-fidelity computational fluid dynamics and neutronics. However, to model superconducting magnets, several MOOSE enhancements are required: additions to the current MOOSE electromagnetic capabilities, new material libraries for superconductors of interest (such as YBCO), as well as fusion-specific models for thermo-mechanics. This talk will discuss initial development activities to build these capabilities in MOOSE, focusing on initial validation and benchmarking activities. Proposed coupling workflows and future work to support the simulation of fusion magnets and magnet structural assemblies for performance and safety evaluation in MOOSE will also be discussed.
70 - PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Fully Employing Software Inspections Data
Software inspections provide a proven approach to quality assurance for software products of all kinds, including requirements, design, code, test plans, among others. Common to all inspections is the aim of finding and fixing defects as early as possible, and thereby providing cost savings by minimizing the amount of rework necessary later in the lifecycle. Measurement data, such as the number and type of found defects and the effort spent by the inspection team, provide not only direct feedback about the software product to the project team but are also valuable for process improvement activities. In this paper, we discuss NASA's use of software inspections and the rich set of data that has resulted. In particular, we present results from analysis of inspection data that illustrate the benefits of fully utilizing that data for process improvement at several levels. Examining such data across multiple inspections or projects allows team members to monitor and trigger cross project improvements. Such improvements may focus on the software development processes of the whole organization as well as improvements to the applied inspection process itself.
Shull, Forrest↗
The software product assurance metrics study: JPL's software systems quality and productivity
The findings are reported of the Jet Propulsion Laboratory (JPL)/Software Product Assurance (SPA) Metrics Study, conducted as part of a larger JPL effort to improve software quality and productivity. Until recently, no comprehensive data had been assembled on how JPL manages and develops software-intensive systems. The first objective was to collect data on software development from as many projects and for as many years as possible. Results from five projects are discussed. These results reflect 15 years of JPL software development, representing over 100 data points (systems and subsystems), over a third of a billion dollars, over four million lines of code and 28,000 person months. Analysis of this data provides a benchmark for gauging the effectiveness of past, present and future software development work. In addition, the study is meant to encourage projects to record existing metrics data and to gather future data. The SPA long term goal is to integrate the collection of historical data and ongoing project data with future project estimations.
Bush, Marilyn W.↗
Teamwork for Oversight of Processes and Systems (TOPS). Implementation guide for TOPS version 2.0, 10 August 1992
As the nation redefines priorities to deal with a rapidly changing world order, both government and industry require new approaches for oversight of management systems, particularly for high technology products. Declining defense budgets will lead to significant reductions in government contract management personnel. Concurrently, defense contractors are reducing administrative and overhead staffing to control costs. These combined pressures require bold approaches for the oversight of management systems. In the Spring of 1991, the DPRO and TRW created a Process Action Team (PAT) to jointly prepare a Performance Based Management (PBM) system titled Teamwork for Oversight of Processes and Systems (TOPS). The primary goal is implementation of a performance based management system based on objective data to review critical TRW processes with an emphasis on continuous improvement. The processes are: Finance and Business Systems, Engineering and Manufacturing Systems, Quality Assurance, and Software Systems. The team established a number of goals: delivery of quality products to contractual terms and conditions; ensure that TRW management systems meet government guidance and good business practices; use of objective data to measure critical processes; elimination of wasteful/duplicative reviews and audits; emphasis on teamwork--all efforts must be perceived to add value by both sides and decisions are made by consensus; and synergy and the creation of a strong working trust between TRW and the DPRO. TOPS permits the adjustment of oversight resources when conditions change or when TRW systems performance indicate either an increase or decrease in surveillance is appropriate. Monthly Contractor Performance Assessments (CPA) are derived from a summary of supporting system level and process-level ratings obtained from objective process-level data. Tiered, objective, data-driven metrics are highly successful in achieving a cooperative and effective method of measuring performance. The teamwork-based culture developed by TOPS proved an unequaled success in removing adversarial relationships and creating an atmosphere of continuous improvement in quality processes at TRW. The new working relationship does not decrease the responsibility or authority of the DPRO to ensure contract compliance and it permits both parties to work more effectively to improve total quality and reduce cost. By emphasizing teamwork in developing a stronger approach to efficient management of the defense industrial base TOPS is a singular success.
Strand, Albert A.↗
Production and quality assurance automation in the Goddard Space Flight Center Flight Dynamics Facility
The Flight Dynamics Facility (FDF) at the NASA Goddard Space Flight Center (GSFC) generates numerous products for NASA-supported spacecraft, including the Tracking and Data Relay Satellites (TDRS's), the Hubble Space Telescope (HST), the Extreme Ultraviolet Explorer (EUVE), and the space shuttle. These products include orbit determination data, acquisition data, event scheduling data, and attitude data. In most cases, product generation involves repetitive execution of many programs. The increasing number of missions supported by the FDF has necessitated the use of automated systems to schedule, execute, and quality assure these products. This automation allows the delivery of accurate products in a timely and cost-efficient manner. To be effective, these systems must automate as many repetitive operations as possible and must be flexible enough to meet changing support requirements. The FDF Orbit Determination Task (ODT) has implemented several systems that automate product generation and quality assurance (QA). These systems include the Orbit Production Automation System (OPAS), the New Enhanced Operations Log (NEOLOG), and the Quality Assurance Automation Software (QA Tool). Implementation of these systems has resulted in a significant reduction in required manpower, elimination of shift work and most weekend support, and improved support quality, while incurring minimal development cost. This paper will present an overview of the concepts used and experiences gained from the implementation of these automation systems.
Chapman, K. B.↗
Emerging technologies for V&V of ISHM software for space exploration
Systems1,2 required to exhibit high operational reliability often rely on some form of fault protection to recognize and respond to faults, preventing faults' escalation to catastrophic failures. Integrated System Health Management (ISHM) extends the functionality of fault protection to both scale to more complex systems (and systems of systems), and to maintain capability rather than just avert catastrophe. Forms of ISHM have been utilized to good effect in the maintenance phase of systems' total lifecycles (often referred to as 'condition-based mainte-nance'), but less so in a 'fault protection' role during actual operations. One of the impediments to such use lies in the challenges of verification, validation and certification of ISHM systems themselves. This paper makes the case that state-of-the-practice V&V and certification techniques will not suffice for emerging forms of ISHM systems; however, a number of maturing software engineering assurance technologies show particular promise for addressing these ISHM V&V challenges.
fault detection, isolation, and recovery↗
Infusing and selecting V&V activities
The evolving nature of software development poses a continuing series of challenges for V&V. In response, the V&V community selectively adapts the use of existing V&V activities, and introduces new and improved ones.
verification validation risk software quality assu↗
Software security assessment instrument: reducing security risks
Explore the source record for details and available documents.
software quality assurance↗
Safety and Mission Assurance: A NASA Perspective
Safety and Mission Assurance (S&MA) consists of the safety, reliability, maintainability, software assurance and quality disciplines, which are applied to reduce the probability of mishaps and ensure mission success. NASA uses a risk management process which requires various organizations to identify, analyze and mitigate/control risks associated with operations and decision making processes.
Higginbotham, Scott Alan↗
FORCE Regression Testing
Via programs including the Light Water Reactor Sustainability and Integrated Energy Systems, the U.S. Department of Energy has invested in the Framework for Optimization of ResourCes and Economics (FORCE) software framework (Idaho National Laboratory 2024a) for the technical and economic analysis of nuclear-integrated energy systems (IES). Nuclear IES expand the use of nuclear from traditional baseload electricity generation to a flexible and adaptive source of combined heat and power. Nuclear heat can be used in the production of a variety of energy currencies such as hydrogen and ammonia as well as other heat applications including water desalination and district heating. FORCE is designed with the intent to provide interconnected analysis tools that enable the accurate technical and economic assessment of specific nuclear IES configurations for individual energy markets. FORCE consists of three main analysis pathways: HYBRID (Idaho National Laboratory 2024b), which contains high-resolution physical models for IES; Holistic Energy Resource Optimization Network (HERON) (Idaho National Laboratory 2024c), which analyzes IES long-term economic viability; and Optimization of Real-time Capacity Allocation (ORCA) (Idaho National Laboratory 2024d), designed for real-time control of IES via digital twins and optimal decision making, including autonomous and remote operation research. Development of the FORCE ecosystem is guided by three pillars: capability, which assures that the computational requirements of IES analysis are met by the software tools; reliability, which provides for consistent code performance and expected behaviors; and accessibility, which lowers the barrier to entry for using the software and accelerates analysis by users beyond the FORCE primary developers. Reliability of the FORCE ecosystem is established according to the American Nuclear Society?s Nuclear Quality Assurance (NQA-1) program [American Society of Mechanical Engineers 1982], with specific levels of software quality assurance (SQA) within NQA-1 applied to each software tool in FORCE. As the tools within FORCE have matured, some integration algorithms to accurately connect the software tools for holistic analysis have been developed and deployed within the FORCE software repository. In accordance with NQA-1 standards, regression tests are required to guarantee the software performs consistently even when new capabilities are added to the software. In this report, we document the deployment of both unit tests, which test the consistent behavior of small pieces of the FORCE code base, as well as integration tests, which test the consistent performance of full use cases for the FORCE integration algorithms. We further document the encapsulation of these tests within a test harness, which collectively checks for each successful test completion on demand. Finally, we document the automation of the test harness using GitHub Actions [GitHub 2024], which require all tests succeed before any new capability or other changes can be added to the FORCE integration software
97 MATHEMATICS AND COMPUTING↗
Assuring Software Cost Estimates: Is it an Oxymoron?
The software industry repeatedly observes cost growth of well over 100% even after decades of cost estimation research and well-known best practices, so "What's the problem?" In this paper we will provide an overview of the current state oj software cost estimation best practice. We then explore whether applying some of the methods used in software assurance might improve the quality of software cost estimates. This paper especially focuses on issues associated with model calibration, estimate review, and the development and documentation of estimates as part alan integrated plan.
Hihn, Jarius↗
Assuring software cost estimates : is it an oxymoron?
The software industry repeatedly observes cost growth of well over 100% even after decades of cost estimation research and well-known best practices, so “What’s the problem?” In this paper we will provide an overview of the current state of software cost estimation best practice. We then explore whether applying some of the methods used in software assurance might improve the quality of software cost estimates. This paper especially focuses on issues associated with model calibration, estimate review, and the development and documentation of estimates as part of an integrated plan.
Hihn, Jairus↗
IEEE/AIAA/NASA Digital Avionics Systems Conference, 9th, Virginia Beach, VA, Oct. 15-18, 1990, Proceedings
The present conference on digital avionics discusses vehicle-management systems, spacecraft avionics, special vehicle avionics, communication/navigation/identification systems, software qualification and quality assurance, launch-vehicle avionics, Ada applications, sensor and signal processing, general aviation avionics, automated software development, design-for-testability techniques, and avionics-software engineering. Also discussed are optical technology and systems, modular avionics, fault-tolerant avionics, commercial avionics, space systems, data buses, crew-station technology, embedded processors and operating systems, AI and expert systems, data links, and pilot/vehicle interfaces.
Source record↗
Standardized development of computer software. Part 2: Standards
This monograph contains standards for software development and engineering. The book sets forth rules for design, specification, coding, testing, documentation, and quality assurance audits of software; it also contains detailed outlines for the documentation to be produced.
Tausworthe, R. C.↗