Incorporating cost-benefit analyses into software assurance planning
The objective is to use cost-benefit analyses to identify, for a given project, optimal sets of software assurance activities.
investment software quality software process impro↗
SEARCH · Search NASA
Results for “Software Verification”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Verification and Validation for Flight-Critical Systems (VVFCS)
On March 31, 2009 a Request for Information (RFI) was issued by NASA s Aviation Safety Program to gather input on the subject of Verification and Validation (V & V) of Flight-Critical Systems. The responses were provided to NASA on or before April 24, 2009. The RFI asked for comments in three topic areas: Modeling and Validation of New Concepts for Vehicles and Operations; Verification of Complex Integrated and Distributed Systems; and Software Safety Assurance. There were a total of 34 responses to the RFI, representing a cross-section of academic (26%), small & large industry (47%) and government agency (27%).
Graves, Sharon S.↗
Certification of COTS Software in NASA Human Rated Flight Systems
Adoption of commercial off-the-shelf (COTS) products in safety critical systems has been seen as a promising acquisition strategy to improve mission affordability and, yet, has come with significant barriers and challenges. Attempts to integrate COTS software components into NASA human rated flight systems have been, for the most part, complicated by verification and validation (V&V) requirements necessary for flight certification per NASA s own standards. For software that is from COTS sources, and, in general from 3rd party sources, either commercial, government, modified or open source, the expectation is that it meets the same certification criteria as those used for in-house and that it does so as if it were built in-house. The latter is a critical and hidden issue. This paper examines the longstanding barriers and challenges in the use of 3rd party software in safety critical systems and cover recent efforts to use COTS software in NASA s Multi-Purpose Crew Vehicle (MPCV) project. It identifies some core artifacts that without them, the use of COTS and 3rd party software is, for all practical purposes, a nonstarter for affordable and timely insertion into flight critical systems. The paper covers the first use in a flight critical system by NASA of COTS software that has prior FAA certification heritage, which was shown to meet the RTCA-DO-178B standard, and how this certification may, in some cases, be leveraged to allow the use of analysis in lieu of testing. Finally, the paper proposes the establishment of an open source forum for development of safety critical 3rd party software.
Goforth, Andre↗
FPGA Flash Memory High Speed Data Acquisition
The purpose of this research is to design and implement a VHDL ONFI Controller module for a Modular Instrumentation System. The goal of the Modular Instrumentation System will be to have a low power device that will store data and send the data at a low speed to a processor. The benefit of such a system will give an advantage over other purchased binary IP due to the capability of allowing NASA to re-use and modify the memory controller module. To accomplish the performance criteria of a low power system, an in house auxiliary board (Flash/ADC board), FPGA development kit, debug board, and modular instrumentation board will be jointly used for the data acquisition. The Flash/ADC board contains four, 1 MSPS, input channel signals and an Open NAND Flash memory module with an analog to digital converter. The ADC, data bits, and control line signals from the board are sent to an Microsemi/Actel FPGA development kit for VHDL programming of the flash memory WRITE, READ, READ STATUS, ERASE, and RESET operation waveforms using Libero software. The debug board will be used for verification of the analog input signal and be able to communicate via serial interface with the module instrumentation. The scope of the new controller module was to find and develop an ONFI controller with the debug board layout designed and completed for manufacture. Successful flash memory operation waveform test routines were completed, simulated, and tested to work on the FPGA board. Through connection of the Flash/ADC board with the FPGA, it was found that the device specifications were not being meet with Vdd reaching half of its voltage. Further testing showed that it was the manufactured Flash/ADC board that contained a misalignment with the ONFI memory module traces. The errors proved to be too great to fix in the time limit set for the project.
Gonzalez, April↗
An Overview of Distributed Spacecraft Autonomy at NASA Ames
Autonomous decision-making significantly increases mission effectiveness by mitigating the effects of communication constraints, like latency and bandwidth, and mission complexity on multi-spacecraft operations. To advance the state of the art in autonomous Distributed Space Systems (DSS), the Distributed Spacecraft Autonomy (DSA) team at NASA's Ames Research Center is developing within five relevant technical areas: distributed resource and task management, reactive operations, system modeling and simulation, human-swarm interaction, and ad hoc network communications. DSA is maturing these technologies - critical for future large autonomous DSS - from concept to launch via simulation studies and orbital deployments. A 100-node heterogenous Processor-in-the-Loop (PiL) testbed aids distributed autonomy capability development and verification of multi-spacecraft missions. The DSA software payload deployed to the D-Orbit SCV-004 spacecraft demonstrates multi-agent reconfigurability and reliability as part of an ESA-sponsored in-orbit technology demonstration. Finally, DSA's primary flight mission showcases collaborative resource allocation for multipoint science data collection with four small spacecraft as a payload on NASA's Starling 1.0 satellites.
Caleb Ashmore Adams↗
ARRISTOTLE Simulation Demo
NASA’s Katherine Johnson Independent Verification and Validation (NASA IV&V) Jon McBride Software Testing and Research (JSTAR) laboratory has developed the Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE). ARRISTOTLE is a digital twin of NASA’s Artemis mission. ARRISTOLE executes the true SLS and Orion flight binaries integrated on a single laptop computer. This demo will showcase the execution of the full Artemis liftoff. Once airborne, an inflight anomaly will be injected, resulting in Orion detaching from the booster. The Launch Abort System (LAS) will safely bring Orion back to earth. The Unity game engine is used to visualize the flyout as flight software executes.
emulation↗
Formal Specification and Parametric Verification of the ICAROUS Distributed Merging Protocol for Autonomous Aircraft Systems
ICAROUS is a software architecture that provides highly assured core software modules for building safety-centric autonomous unmanned aircraft applications. One of its core components is the ICAROUS distributed merging (IDM) protocol, which allows for decentralized merging of autonomous aircrafts through a designated intersection. This report presents initial results on formal specification and parametric verification of the IDM protocol. We present the development of a formal, discrete-time specification of the ICAROUS distributed merging protocol in TLA+. The developed TLA+ specification includes an abstracted model of the physical aircraft dynamics, the consensus machinery for leader election and coordination, and the computation of merging schedules. In addition, we present details on a command line tool we developed verimerge, that utilizes the TLC model checker for doing bounded, parametric verification and allows for plotting of these results in 2D parameter spaces. The tool also provides functionality for visualization of concrete protocol behaviors, to aid debugging and understanding. We present preliminary, bounded time verification results for a finite number of aircraft. Limitations of the current techniques and possible future extensions of this work are also discussed.
ICAROUS↗
A specification-based approach to concurrent structure verification in multiprocessor systems
A recently initiated research project concerned with the concurrent detection of software errors and errors due to physical failures in the hardware of multiprocessor systems is described in this paper. An approach to error detection is described, which is specification based and relies on the structural verification of program control flow and data structure integrity. The techniques discussed utilize the hardware redundancy inherent in parallel processing systems to provide verification of both program structure and data concurrently with program execution.
Fuchs, W. Kent↗
Verification and Validation of Safety-Critical Aircraft Systems Operating under Off-Nominal, Contingency, and Emergency Conditions
Verification and validation (V&V) of safety-critical technologies developed for loss of control (LOC) prevention and recovery and other aviation safety concerns pose significant challenges. Aircraft LOC can result from a wide spectrum of hazards, often occurring in combination, which cannot be fully replicated during evaluation. Technologies developed for LOC prevention and recovery must therefore be effective under a wide variety of hazardous and uncertain conditions, and the verification and validation of these technologies must provide some measure of assurance that the new vehicle safety technologies do no harm (i.e., that they themselves do not introduce new safety risks). V&V technologies must also enable the identification of system limitations and constraints, as well as enable the identification of safe and unsafe operating conditions (and their boundaries). Additionally, the V&V of complex, increasingly autonomous systems is a fundamental concern. Scalable, reproducible and cost-effective techniques for the assurance of safety critical systems during their design and operation is a key barrier to fielding new systems or updating current systems. Moreover, these techniques need to provide artifacts that enable a comprehensive evidence-based approach to certification. This briefing summarizes research performed under NASA’s Aviation Safety Program and follow-on research for the V&V of safety-critical aircraft system technologies developed for LOC prevention and recovery and increasingly autonomous systems, and for a broad assurance capability in both current and emerging aviation applications. Note that, in this briefing, the term “validation” refers to a confirmation that the system implementation (e.g., algorithms etc.) is performing the intended function(s), as well as an affirmation of effectiveness in these functions. “Verification” refers to a confirmation that the system implementation in the software and hardware meets its (hopefully validated) specifications (e.g., correctly executes algorithms as designed).
Validation↗
Use of Semi-Autonomous Tools for ISS Commanding and Monitoring
As the International Space Station (ISS) has moved into a utilization phase, operations have shifted to become more ground-based with fewer mission control personnel monitoring and commanding multiple ISS systems. This shift to fewer people monitoring more systems has prompted use of semi-autonomous console tools in the ISS Mission Control Center (MCC) to help flight controllers command and monitor the ISS. These console tools perform routine operational procedures while keeping the human operator "in the loop" to monitor and intervene when off-nominal events arise. Two such tools, the Pre-positioned Load (PPL) Loader and Automatic Operators Recorder Manager (AutoORM), are used by the ISS Communications RF Onboard Networks Utilization Specialist (CRONUS) flight control position. CRONUS is responsible for simultaneously commanding and monitoring the ISS Command & Data Handling (C&DH) and Communications and Tracking (C&T) systems. PPL Loader is used to uplink small pieces of frequently changed software data tables, called PPLs, to ISS computers to support different ISS operations. In order to uplink a PPL, a data load command must be built that contains multiple user-input fields. Next, a multiple step commanding and verification procedure must be performed to enable an onboard computer for software uplink, uplink the PPL, verify the PPL has incorporated correctly, and disable the computer for software uplink. PPL Loader provides different levels of automation in both building and uplinking these commands. In its manual mode, PPL Loader automatically builds the PPL data load commands but allows the flight controller to verify and save the commands for future uplink. In its auto mode, PPL Loader automatically builds the PPL data load commands for flight controller verification, but automatically performs the PPL uplink procedure by sending commands and performing verification checks while notifying CRONUS of procedure step completion. If an off-nominal condition occurs during procedure execution, PPL Loader notifies CRONUS through popup messages, allowing CRONUS to examine the situation and choose an option of how PPL loader should proceed with the procedure. The use of PPL Loader to perform frequent, routine PPL uplinks offloads CRONUS to better monitor two ISS systems. It also reduces procedure performance time and decreases risk of command errors. AutoORM identifies ISS communication outage periods and builds commands to lock, playback, and unlock ISS Operations Recorder files. Operation Recorder files are circular buffer files of continually recorded ISS telemetry data. Sections of these files can be locked from further writing, be played back to capture telemetry data that occurred during an ISS loss of signal (LOS) period, and then be unlocked for future recording use. Downlinked Operation Recorder files are used by mission support teams for data analysis, especially if failures occur during LOS. The commands to lock, playback, and unlock Operations Recorder files are encompassed in three different operational procedures and contain multiple user-input fields. AutoORM provides different levels of automation for building and uplinking the commands to lock, playback, and unlock Operations Recorder files. In its automatic mode, AutoORM automatically detects ISS LOS periods, then generates and uplinks the commands to lock, playback, and unlock Operations Recorder files when MCC regains signal with ISS. AutoORM also features semi-autonomous and manual modes which integrate CRONUS more into the command verification and uplink process. AutoORMs ability to automatically detect ISS LOS periods and build the necessary commands to preserve, playback, and release recorded telemetry data greatly offloads CRONUS to perform more high-level cognitive tasks, such as mission planning and anomaly troubleshooting. Additionally, since Operations Recorder commands contain numerical time input fields which are tedious for a human to manually build, AutoORM's ability to automatically build commands reduces operational command errors. PPL Loader and AutoORM demonstrate principles of semi-autonomous operational tools that will benefit future space mission operations. Both tools employ different levels of automation to perform simple and routine procedures, thereby offloading human operators to perform higher-level cognitive tasks. Because both tools provide procedure execution status and highlight off-nominal indications, the flight controller is able to intervene during procedure execution if needed. Semi-autonomous tools and systems that can perform routine procedures, yet keep human operators informed of execution, will be essential in future long-duration missions where the onboard crew will be solely responsible for spacecraft monitoring and control.
Brzezinski, Amy S.↗
Comparison of a Simple Patched Conic Trajectory Code to Commercially Available Software
Often in spaceflight proposal development, mission designers must eva luate numerous trajectories as different design factors are investiga ted. Although there are numerous commercial software packages availab le to help develop and analyze trajectories, most take a significant amount of time to develop the trajectory itself, which isn't effectiv e when working on proposals. Thus a new code, PatCon, which is both q uick and easy to use, was developed to aid mission designers to condu ct trade studies on launch and arrival times for any given target pla net. The code is able to run quick analyses, due to the incorporation of the patched conic approximation, to determine the trajectory. PatCon provides a simple but accurate approximation of the four body moti on problem that would be needed to solve any planetary trajectory. P atCon has been compared to a patched conic test case for verification, with limited validation or comparison with other COTS software. This paper describes the patched conic technique and its implementation i n PatCon. A description of the results and comparison of PatCon to ot her more evolved codes such as AGI#s Satellite Tool Kit and JAQAR As trodynamics# Swingby Calculator is provided. The results will include percent differences in values such as C3 numbers, and Vinfinity at a rrival, and other more subjective results such as the time it takes to build the simulation, and actual calculation time.
AndersonPark, Brooke M.↗
Simulation verification techniques study. Task report 4: Simulation module performance parameters and performance standards
Shuttle simulation software modules in the environment, crew station, vehicle configuration and vehicle dynamics categories are discussed. For each software module covered, a description of the module functions and operational modes, its interfaces with other modules, its stored data, inputs, performance parameters and critical performance parameters is given. Reference data sources which provide standards of performance are identified for each module. Performance verification methods are also discussed briefly.
Source record↗
FAME, a microprocessor based front-end analysis and modeling environment
Higher order software (HOS) is a methodology for the specification and verification of large scale, complex, real time systems. The HOS methodology was implemented as FAME (front end analysis and modeling environment), a microprocessor based system for interactively developing, analyzing, and displaying system models in a low cost user-friendly environment. The nature of the model is such that when completed it can be the basis for projection to a variety of forms such as structured design diagrams, Petri-nets, data flow diagrams, and PSL/PSA source code. The user's interface with the analyzer is easily recognized by any current user of a structured modeling approach; therefore extensive training is unnecessary. Furthermore, when all the system capabilities are used one can check on proper usage of data types, functions, and control structures thereby adding a new dimension to the design process that will lead to better and more easily verified software designs.
Rosenbaum, J. D.↗
Full Life-Cycle Defect Management Assessment: Initial Inspection Data Collection Results and Research Questions for Further Study
It is often the case in software projects that when schedule and budget resources are limited, the Verification and Validation (V&V) activities suffer. Fewer V&V activities can be afforded and moreover, short-term challenges can result in V&V activities being scaled back or dropped altogether. As a result, too often the default solution is to save activities for improving software quality until too late in the life-cycle, relying on late-term code inspections followed by thorough testing activities to reduce defect counts to acceptable levels. As many project managers realize, however, this is a resource-intensive way of achieving the required quality for software. The Full Life-cycle Defect Management Assessment Initiative, funded by NASA s Office of Safety and Mission Assurance under the Software Assurance Research Program, aims to address these problems by: Improving the effectiveness of early life-cycle V&V activities to make their benefits more attractive to team leads. Specifically, we focus on software inspection, a proven method that can be applied to any software work product, long before executable code has been developed; Better communicating this effectiveness to software development teams, along with suggestions for parameters to improve in the future to increase effectiveness; Analyzing the impact of early life-cycle V&V on the effectiveness and cost required for late life-cycle V&V activities, such as testing, in order to make the tradeoffs more apparent. This white paper reports on an initial milestone in this work, the development of a preliminary model of inspection effectiveness across multiple NASA Centers. This model contributes toward reaching our project goals by: Allowing an examination of inspection parameters, across different types of projects and different work products, for an analysis of factors that impact defect detection effectiveness. Allowing a comparison of this NASA-specific model to existing recommendations in the literature regarding how to plan effective inspections. Forming a baseline model which can be extended to incorporate factors describing: the numbers and types of defects that are missed by inspections; how such defects flow downstream through software development phases; how effectively they can be caught by testing activities in the late stages of development. The model has been implemented in a prototype web-enabled decision-support tool which allows developers to enter their inspection data and receive feedback based on a comparison against the model. The tool also allows users to access reusable materials (such as checklists) from projects included in the baseline. Both the tool itself and the model underlying it will continue to be extended throughout the remainder of this initiative. As results of analyzing inspection effectiveness for defect containment are determined, they can be shared via the tool and also via updates to existing training courses on metrics and software inspections. Moreover, the tool will help satisfy key CMMI requirements for the NASA Centers, as it will enable NASA to take a global view across peer review results for various types of projects to identify systemic problems. This analysis can result in continuous improvements to the approach to verification.
Shull, Forrest↗
Optimum-AIV: A planning and scheduling system for spacecraft AIV
A project undertaken for the European Space Agency (ESA) is presented. The project is developing a knowledge based software system for planning and scheduling of activities for spacecraft assembly, integration, and verification (AIV). The system extends into the monitoring of plan execution and the plan repair phase. The objectives are to develop an operational kernel of a planning, scheduling, and plan repair tool, called OPTIMUM-AIV, and to provide facilities which will allow individual projects to customize the kernel to suit its specific needs. The kernel shall consist of a set of software functionalities for assistance in initial specification of the AIV plan, in verification and generation of valid plans and schedules for the AIV activities, and in interactive monitoring and execution problem recovery for the detailed AIV plans. Embedded in OPTIMUM-AIV are external interfaces which allow integration with alternative scheduling systems and project databases. The current status of the OPTIMUM-AIV project, as of Jan. 1991, is that a further analysis of the AIV domain has taken place through interviews with satellite AIV experts, a software requirement document (SRD) for the full operational tool was approved, and an architectural design document (ADD) for the kernel excluding external interfaces is ready for review.
Arentoft, M. M.↗
Software assurance standard
This standard specifies the software assurance program for the provider of software. It also delineates the assurance activities for the provider and the assurance data that are to be furnished by the provider to the acquirer. In any software development effort, the provider is the entity or individual that actually designs, develops, and implements the software product, while the acquirer is the entity or individual who specifies the requirements and accepts the resulting products. This standard specifies at a high level an overall software assurance program for software developed for and by NASA. Assurance includes the disciplines of quality assurance, quality engineering, verification and validation, nonconformance reporting and corrective action, safety assurance, and security assurance. The application of these disciplines during a software development life cycle is called software assurance. Subsequent lower-level standards will specify the specific processes within these disciplines.
Source record↗
Performance Evaluation of a Data Validation System
Online data validation is a performance-enhancing component of modern control and health management systems. It is essential that performance of the data validation system be verified prior to its use in a control and health management system. A new Data Qualification and Validation (DQV) Test-bed application was developed to provide a systematic test environment for this performance verification. The DQV Test-bed was used to evaluate a model-based data validation package known as the Data Quality Validation Studio (DQVS). DQVS was employed as the primary data validation component of a rocket engine health management (EHM) system developed under NASA's NGLT (Next Generation Launch Technology) program. In this paper, the DQVS and DQV Test-bed software applications are described, and the DQV Test-bed verification procedure for this EHM system application is presented. Test-bed results are summarized and implications for EHM system performance improvements are discussed.
Wong, Edmond↗
Using Penelope to assess the correctness of NASA Ada software: A demonstration of formal methods as a counterpart to testing
Life-critical applications warrant a higher level of software reliability than has yet been achieved. Since it is not certain that traditional methods alone can provide the required ultra reliability, new methods should be examined as supplements or replacements. This paper describes a mathematical counterpart to the traditional process of empirical testing. ORA's Penelope verification system is demonstrated as a tool for evaluating the correctness of Ada software. Grady Booch's Ada calendar utility package, obtained through NASA, was specified in the Larch/Ada language. Formal verification in the Penelope environment established that many of the package's subprograms met their specifications. In other subprograms, failed attempts at verification revealed several errors that had escaped detection by testing.
Eichenlaub, Carl T.↗