Results for “Software Testing” — Page 7

NASA NTRS2011

Mars Science Laboratory Flight Software Boot Robustness Testing Project Report

On the surface of Mars, the Mars Science Laboratory will boot up its flight computers every morning, having charged the batteries through the night. This boot process is complicated, critical, and affected by numerous hardware states that can be difficult to test. The hardware test beds do not facilitate testing a long duration of back-to-back unmanned automated tests, and although the software simulation has provided the necessary functionality and fidelity for this boot testing, there has not been support for the full flexibility necessary for this task. Therefore to perform this testing a framework has been build around the software simulation that supports running automated tests loading a variety of starting configurations for software and hardware states. This implementation has been tested against the nominal cases to validate the methodology, and support for configuring off-nominal cases is ongoing. The implication of this testing is that the introduction of input configurations that have yet proved difficult to test may reveal boot scenarios worth higher fidelity investigation, and in other cases increase confidence in the robustness of the flight software boot process.

Mars Science Laboratory (MSL)↗

NASA NTRS2014

Integration and Testing of LCS Software

Kennedy Space Center is in the midst of developing a command and control system for the launch of the next generation manned space vehicle. The Space Launch System (SLS) will launch using the new Spaceport Command and Control System (SCCS). As a member of the Software Integration and Test (SWIT) Team, command scripts, and bash scripts were written to assist in integration and testing of the Launch Control System (LCS), which is a component of SCCS. The short term and midterm tasks are for the most part completed. The long term tasks if time permits will require a presentation and demonstration.

Command Script↗

NASA NTRS2012

An Overview of the Smart Sensor Inter-Agency Reference Testbench (SSIART)

In this paper, we present an overview of a proposed collaboration between the National Aeronautics and Space Administration (NASA) and the European Space Agency (ESA), which is designed to facilitate the introduction of commercial-off-the-shelf (COTS) radios for smart-sensing applications into international spaceflight programs and projects. The proposed work will produce test hardware reference designs, test software reference architectures and example implementations, test plans in reference test environments, and test results, all of which will be shared between the agencies and documented for future use by mission planners. The proposed collaborative structure together with all of the anticipated tools and results produced under the effort is collectively referred to as the Smart Sensor Inter-agency Reference Testbench or SSIART. It is intended to provide guidance in technology selection and in increasing the related readiness levels of projects and missions as well as the space industry.

Wagner, Raymond S.↗

NASA NTRS2017

NASA's Space Launch System: Progress Report

NASA and its commercial industry team achieved significant progress in 2016 in manufacturing and testing of the Block 1 vehicle for the first launch of the Space Launch System (SLS). Test and flight article hardware for the liquid hydrogen fuel tank as well as the engine section for the core stage were completed at Michoud Assembly Facility (MAF) in New Orleans. Test stands neared completion at Marshall Space Flight Center for the propellant tanks, engine section, intertank and payload section. Stennis Space Center completed major structural renovations on the B2 test stand, where the core stage "green run" test program will be conducted. The SLS team completed a hotfire test series at Stennis to successfully demonstrate the ability of the RS-25 engine to operate under SLS environments and performance conditions. The team also test fired the second qualification five-segment solid rocket motor and cast the first six motor segments for the first SLS mission. The Interim Cryogenic Propulsion Stage (ICPS) test article was delivered to Marshall for structural tests, and work is nearly finished on the flight stage. Flight software testing completed at Marshall included power quality and command and data handling. In 2017, that work continues. SLS completed Preliminary Design Review (PDR) on the Exploration Upper Stage (EUS), a powerful, human-rated spacecraft that will propel explorers to cis-lunar space. In 2017, hardware will continue to be integrated at MAF for core stage structural test articles and the first two operational flights. RS-25 hotfire testing will continue to explore engine performance, as well as test flight-like software and four new Engine Controller Units (ECUs) for the first mission. Production of development components for a more affordable RS-25 design is underway. Core stage structural test articles have begun arriving at Marshall. While engineering challenges typical of a new development are possible, SLS is working toward launch readiness in late 2018. This paper will discuss these and other technical and programmatic successes and challenges over the past year and provide a preview of work ahead before first flight

Cook, Jerry↗

NASA NTRS2015

Firing Room Remote Application Software Development

The Engineering and Technology Directorate (NE) at National Aeronautics and Space Administration (NASA) Kennedy Space Center (KSC) is designing a new command and control system for the checkout and launch of Space Launch System (SLS) and future rockets. The purposes of the semester long internship as a remote application software developer include the design, development, integration, and verification of the software and hardware in the firing rooms, in particular with the Mobile Launcher (ML) Launch Accessories (LACC) subsystem. In addition, a software test verification procedure document was created to verify and checkout LACC software for Launch Equipment Test Facility (LETF) testing.

Pathways↗

NASA NTRS2015

Pi-Sat: A Low Cost Small Satellite and Distributed Spacecraft Mission System Test Platform

Current technology and budget trends indicate a shift in satellite architectures from large, expensive single satellite missions, to small, low cost distributed spacecraft missions. At the center of this shift is the SmallSatCubesat architecture. The primary goal of the Pi-Sat project is to create a low cost, and easy to use Distributed Spacecraft Mission (DSM) test bed to facilitate the research and development of next-generation DSM technologies and concepts. This test bed also serves as a realistic software development platform for Small Satellite and Cubesat architectures. The Pi-Sat is based on the popular $35 Raspberry Pi single board computer featuring a 700Mhz ARM processor, 512MB of RAM, a flash memory card, and a wealth of IO options. The Raspberry Pi runs the Linux operating system and can easily run Code 582s Core Flight System flight software architecture. The low cost and high availability of the Raspberry Pi make it an ideal platform for a Distributed Spacecraft Mission and Cubesat software development. The Pi-Sat models currently include a Pi-Sat 1U Cube, a Pi-Sat Wireless Node, and a Pi-Sat Cubesat processor card.The Pi-Sat project takes advantage of many popular trends in the Maker community including low cost electronics, 3d printing, and rapid prototyping in order to provide a realistic platform for flight software testing, training, and technology development. The Pi-Sat has also provided fantastic hands on training opportunities for NASA summer interns and Pathways students.

Software↗

NASA NTRS1976

Payload specialist station study. Volume 3: Program study cost estimates. Part 1: Work breakdown structure

The work breakdown structure (WBS) for the Payload Specialist Station (PSS) is presented. The WBS is divided into two elements--PSS contractor and mission unique requirements. In accordance with the study ground rules, it is assumed that a single contractor, hereafter referred to as PSS Contractor will perform the following: (1) provide C and D hardware (MFDS and elements of MMSE), except for GFE; (2) identify software requirements; (3) provide GSE and ground test software; and (4) perform systems engineering and integration in support of the Aft Flight Deck (AFD) C and D concept. The PSS Contractor WBS element encompasses a core or standardized PSS concept. Payload peculiar C and D requirements identified by users will originate as a part of the WBS element mission unique requirements; these requirements will be provided to the PSS Contractor for implementation.

Source record↗

NASA NTRS1987

A program downloader and other utility software for the DATAC bus monitor unit

A set or programs designed to facilitate software testing on the DATAC Bus Monitor is described. By providing a means to simplify program loading, firmware generation, and subsequent testing of programs, the overhead involved in software evaluation is reduced and that time is used more productively in performance, analysis and improvement of current software.

Novacki, Stanley M., III↗

DOE OSTI2024

STAT7 v1.2 Verification and Validation Report

This report documents the software testing which has been performed for the STAT7 Version 1.2 (hereafter STAT7 v1.2) software, a code for the statistical propagation of uncertainties in steady-state thermal hydraulics (TH) analysis of plate-fueled research and test reactors. The key capabilities addressed in this testing were selected based on the use of the software for analysis work supporting the conversion to low-enriched uranium fuel of U.S. High-Performance Research Reactors (USHPRR) such as the Massachusetts Institute of Technology Research Reactor (MITR-II). This testing scope includes comparison of the code’s statistical processing capabilities using statistical tests, comparison of coolant property evaluations to NIST data tables, and systematic comparison of key TH solver capabilities to the results of hand calculations and to predictions from PLTEMP/ANL Version 4.4. The good agreement achieved in this testing campaign, within the acceptance criteria of a 1% relative difference, also demonstrates that these capabilities have been implemented in STAT7 v1.2 correctly.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2025

STAT7 v2.0 Verification and Validation Report

This report documents the software testing which has been performed for STAT7 Version 2.0 (hereafter STAT7 v2.0), a code for the statistical propagation of uncertainties in steady-state thermal hydraulics (TH) analysis of plate-fueled research and test reactors. The key capabilities addressed in this testing were selected based on the use of the software for analysis work supporting the conversion to low-enriched uranium fuel of U.S. High-Performance Research Reactors (USHPRR) such as the Massachusetts Institute of Technology Research Reactor (MITR-II). This testing scope includes comparison of the code’s statistical processing capabilities using statistical tests, comparison of coolant property evaluations to NIST data tables, and systematic comparison of key TH solver capabilities to the results of hand calculations and to predictions from PLTEMP/ANL Version 4.4. The

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

NASA NTRS1993

Systems aspects of COBE science data compression

A general approach to compression of diverse data from large scientific projects has been developed and this paper addresses the appropriate system and scientific constraints together with the algorithm development and test strategy. This framework has been implemented for the COsmic Background Explorer spacecraft (COBE) by retrofitting the existing VAS-based data management system with high-performance compression software permitting random access to the data. Algorithms which incorporate scientific knowledge and consume relatively few system resources are preferred over ad hoc methods. COBE exceeded its planned storage by a large and growing factor and the retrieval of data significantly affects the processing, delaying the availability of data for scientific usage and software test. Embedded compression software is planned to make the project tractable by reducing the data storage volume to an acceptable level during normal processing.

Freedman, I.↗

NASA NTRS1990

Empirically based analysis of failures in software systems

An empirical analysis of software-system failures is used to study several specific issues in software testing, reliability analysis, and reuse. Failure data from a large software manufacturer and a NASA production environment were collected and analyzed. The systems ranged in size from 30,000 to over 100,000 lines. The results show that (1) the first 15 percent of the test cases detected 67 percent of the high-severity failures and 50 percent of all failures; (2) multiple fault-detection and testing phases may result in a significant increase in reliability or none at all; (3) composite measures of system reliability did not adequately reflect reliability at the function or component level; (4) developers were biased toward portions of systems that would be heavily tested; (5) fault-proneness of reused or modified components was 74 percent less than that of newly developed components; and (6) systems with more reused software had lower component development effort, but not lower component fault-proneness.

Selby, Richard W.↗

NASA NTRS1990

Automation software for a materials testing laboratory

The software environment in use at the NASA-Lewis Research Center's High Temperature Fatigue and Structures Laboratory is reviewed. This software environment is aimed at supporting the tasks involved in performing materials behavior research. The features and capabilities of the approach to specifying a materials test include static and dynamic control mode switching, enabling multimode test control; dynamic alteration of the control waveform based upon events occurring in the response variables; precise control over the nature of both command waveform generation and data acquisition; and the nesting of waveform/data acquisition strategies so that material history dependencies may be explored. To eliminate repetitive tasks in the coventional research process, a communications network software system is established which provides file interchange and remote console capabilities.

Mcgaw, Michael A.↗

NASA NTRS2023

Adaptive Independent Verification and Validation (IV&V) Reduces Risk of Software Impacting Safety in Artemis Missions

The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.

Gerek A Whitman↗

NASA NTRS2023

Adaptive Independent Verification and Validation (IV&V) Reduces Risk of Software Impacting Safety in Artemis Missions

The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.

Gerek Whitman↗

NASA NTRS2022

Electrical Ground Support Equipment for the Sampling Caching System of the Mars 2020 Rover

In this work we describe in detail the architecture, design, testing and operation of the Electrical Ground Support Equipment (EGSE) “Blue Box” used to test and validate the Sampling Caching System (SCS) of the Mars 2020 Perseverance rover. The Blue Box architecture is centered around COTS motor controllers and COTS input-output modules communicating over an EtherCAT bus. A custom, low-level safety subsystem ensures no harm can be done to the flight articles. The modular architecture of the EGSE reduces cost and complexity while expediting assembly time. The Blue Box drives the 19 actuators of the SCS which span the main robotic arm, the corer system, the internal sample handling arm, the sample tube sealing system and the gas dust removal tool; mimicking the Rover Motor Control Assembly (RMCA). Due to the limited availability of RMCA’s, the EGSE enabled and performed the bulk of testing activities for SCS. The majority of the SCS actuators are composed of a 3-phase DC brushless motors, hall sensors for commutation, dual resolvers for output angular measurement, brakes, heaters and platinum thermistors. Additionally, the EGSE read 12 strain gauges forming part of a force torque sensor, and switches used for external positioning references. Over the 3-year span of the V&V campaign for the SCS, over 32 EGSE systems were built, tested and deployed to test venues at JPL and externally. The EGSE tested several families of the SCS subsystem, ranging from engineering units, life test units and two flight units. Test venues that this EGSE supported included lab benches, ultra-clean cleanrooms, ATLO facilities, and thermal vacuum chambers. Together with the test software systems, SSDEV and SSDEV-ECAT, the Blue Box EGSE enabled the team to efficiently test flight hardware and flight software together. We go over the safety features and fault management techniques employed to protect flight hardware. The effects of the long, 50-feet, EGSE harnesses on motor performance, EMI, electrical noise, and motor control performance are explained. Mitigations to these unwanted effects, including shielding strategy and inductance compensation, are summarized. We go over an excerpt of notable anomalies that this EGSE suffered through its operation, along with investigations and resolutions. Lessons learned, areas of improvement as part of future work, and recommendations for future implementations for similar EGSE’s, are shared.

Levine, Dan↗

NASA NTRS1985

Automation of assertion testing - Grid and adaptive techniques

Assertions can be used to automate the process of testing software. Two methods for automating the generation of input test data are described in this paper. One method selects the input values of variables at regular intervals in a 'grid'. The other, adaptive testing, uses assertion violations as a measure of errors detected and generates new test cases based on test results. The important features of assertion testing are that: it can be used throughout the entire testing cycle; it provides automatic notification of error conditions; and it can be used with automatic input generation techniques which eliminate the subjectivity in choosing test data.

Andrews, D. M.↗

NASA NTRS1993

Advanced communications technology satellite high burst rate link evaluation terminal communication protocol software user's guide, version 1.0

The Communication Protocol Software was developed at the NASA Lewis Research Center to support the Advanced Communications Technology Satellite High Burst Rate Link Evaluation Terminal (ACTS HBR-LET). The HBR-LET is an experimenters terminal to communicate with the ACTS for various experiments by government, university, and industry agencies. The Communication Protocol Software is one segment of the Control and Performance Monitor (C&PM) Software system of the HBR-LET. The Communication Protocol Software allows users to control and configure the Intermediate Frequency Switch Matrix (IFSM) on board the ACTS to yield a desired path through the spacecraft payload. Besides IFSM control, the C&PM Software System is also responsible for instrument control during HBR-LET experiments, uplink power control of the HBR-LET to demonstrate power augmentation during signal fade events, and data display. The Communication Protocol Software User's Guide, Version 1.0 (NASA CR-189162) outlines the commands and procedures to install and operate the Communication Protocol Software. Configuration files used to control the IFSM, operator commands, and error recovery procedures are discussed. The Communication Protocol Software Maintenance Manual, Version 1.0 (NASA CR-189163, to be published) is a programmer's guide to the Communication Protocol Software. This manual details the current implementation of the software from a technical perspective. Included is an overview of the Communication Protocol Software, computer algorithms, format representations, and computer hardware configuration. The Communication Protocol Software Test Plan (NASA CR-189164, to be published) provides a step-by-step procedure to verify the operation of the software. Included in the Test Plan is command transmission, telemetry reception, error detection, and error recovery procedures.

Reinhart, Richard C.↗