SEARCH · Search NASA

Results for “software engineering software verification validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

NASA NTRS2015

Fault Management Architectures and the Challenges of Providing Software Assurance

Fault Management (FM) is focused on safety, the preservation of assets, and maintaining the desired functionality of the system. How FM is implemented varies among missions. Common to most missions is system complexity due to a need to establish a multi-dimensional structure across hardware, software and spacecraft operations. FM is necessary to identify and respond to system faults, mitigate technical risks and ensure operational continuity. Generally, FM architecture, implementation, and software assurance efforts increase with mission complexity. Because FM is a systems engineering discipline with a distributed implementation, providing efficient and effective verification and validation (V&V) is challenging. A breakout session at the 2012 NASA Independent Verification & Validation (IV&V) Annual Workshop titled "V&V of Fault Management: Challenges and Successes" exposed this issue in terms of V&V for a representative set of architectures. NASA's Software Assurance Research Program (SARP) has provided funds to NASA IV&V to extend the work performed at the Workshop session in partnership with NASA's Jet Propulsion Laboratory (JPL). NASA IV&V will extract FM architectures across the IV&V portfolio and evaluate the data set, assess visibility for validation and test, and define software assurance methods that could be applied to the various architectures and designs. This SARP initiative focuses efforts on FM architectures from critical and complex projects within NASA. The identification of particular FM architectures and associated V&V/IV&V techniques provides a data set that can enable improved assurance that a system will adequately detect and respond to adverse conditions. Ultimately, results from this activity will be incorporated into the NASA Fault Management Handbook providing dissemination across NASA, other agencies and the space community. This paper discusses the approach taken to perform the evaluations and preliminary findings from the research.

Fault Management↗

NASA NTRS2015

Fault Management Architectures and the Challenges of Providing Software Assurance

The satellite systems Fault Management (FM) is focused on safety, the preservation of assets, and maintaining the desired functionality of the system. How FM is implemented varies among missions. Common to most is system complexity due to a need to establish a multi-dimensional structure across hardware, software and operations. This structure is necessary to identify and respond to system faults, mitigate technical risks and ensure operational continuity. These architecture, implementation and software assurance efforts increase with mission complexity. Because FM is a systems engineering discipline with a distributed implementation, providing efficient and effective verification and validation (VV) is challenging. A breakout session at the 2012 NASA Independent Verification Validation (IVV) Annual Workshop titled VV of Fault Management: Challenges and Successes exposed these issues in terms of VV for a representative set of architectures. NASA's IVV is funded by NASA's Software Assurance Research Program (SARP) in partnership with NASA's Jet Propulsion Laboratory (JPL) to extend the work performed at the Workshop session. NASA IVV will extract FM architectures across the IVV portfolio and evaluate the data set for robustness, assess visibility for validation and test, and define software assurance methods that could be applied to the various architectures and designs. This work focuses efforts on FM architectures from critical and complex projects within NASA. The identification of particular FM architectures, visibility, and associated VVIVV techniques provides a data set that can enable higher assurance that a satellite system will adequately detect and respond to adverse conditions. Ultimately, results from this activity will be incorporated into the NASA Fault Management Handbook providing dissemination across NASA, other agencies and the satellite community. This paper discusses the approach taken to perform the evaluations and preliminary findings from the research including identification of FM architectures, visibility observations, and methods utilized for VVIVV.

Fault Management↗

NASA NTRS1992

Knowledge-based system V and V in the Space Station Freedom program

Knowledge Based Systems (KBS's) are expected to be heavily used in the Space Station Freedom Program (SSFP). Although SSFP Verification and Validation (V&V) requirements are based on the latest state-of-the-practice in software engineering technology, they may be insufficient for Knowledge Based Systems (KBS's); it is widely stated that there are differences in both approach and execution between KBS V&V and conventional software V&V. In order to better understand this issue, we have surveyed and/or interviewed developers from sixty expert system projects in order to understand the differences and difficulties in KBS V&V. We have used this survey results to analyze the SSFP V&V requirements for conventional software in order to determine which specific requirements are inappropriate for KBS V&V and why they are inappropriate. Further work will result in a set of recommendations that can be used either as guidelines for applying conventional software V&V requirements to KBS's or as modifications to extend the existing SSFP conventional software V&V requirements to include KBS requirements. The results of this work are significant to many projects, in addition to SSFP, which will involve KBS's.

Kelley, Keith↗

NASA NTRS2002

Verification and Validation of Neural Networks for Aerospace Systems

The Dryden Flight Research Center V&V working group and NASA Ames Research Center Automated Software Engineering (ASE) group collaborated to prepare this report. The purpose is to describe V&V processes and methods for certification of neural networks for aerospace applications, particularly adaptive flight control systems like Intelligent Flight Control Systems (IFCS) that use neural networks. This report is divided into the following two sections: Overview of Adaptive Systems and V&V Processes/Methods.

Mackall, Dale↗

NASA NTRS2002

Verification and Validation of Neural Networks for Aerospace Systems

The Dryden Flight Research Center V&V working group and NASA Ames Research Center Automated Software Engineering (ASE) group collaborated to prepare this report. The purpose is to describe V&V processes and methods for certification of neural networks for aerospace applications, particularly adaptive flight control systems like Intelligent Flight Control Systems (IFCS) that use neural networks. This report is divided into the following two sections: 1) Overview of Adaptive Systems; and 2) V&V Processes/Methods.

Mackall, Dale↗

NASA NTRS1990

Cost and quality planning for large NASA programs

The Software Cost and Quality Engineering methodology developed over the last two decades at IBM Federal Sector Div. is used to plan the NASA Space Station Data Management System (DMS). An ongoing project to capture this methodology, which is built on a foundation of experiences and lessons learned, has resulted in the development of a PC-based tool that integrates cost and quality forecasting methodologies and data in a consistent manner. This tool, Software Cost and Quality Engineering Starter Set (SCQESS), is being used to assist in the DMS costing exercises. At the same time, DMS planning serves as a forcing function and provides a platform for the continuing, iterative development, calibration, and validation and verification of SCQESS. The data that forms the cost and quality engineering data base is derived from more than 17 years of development of NASA Space Shuttle software, ranging from low criticality, low complexity support tools to highly complex and highly critical onboard software.

Rone, Kyle Y.↗

NASA NTRS1985

Some key considerations in evolving a computer system and software engineering support environment for the space station program

The space station data management system involves networks of computing resources that must work cooperatively and reliably over an indefinite life span. This program requires a long schedule of modular growth and an even longer period of maintenance and operation. The development and operation of space station computing resources will involve a spectrum of systems and software life cycle activities distributed across a variety of hosts, an integration, verification, and validation host with test bed, and distributed targets. The requirement for the early establishment and use of an apporopriate Computer Systems and Software Engineering Support Environment is identified. This environment will support the Research and Development Productivity challenges presented by the space station computing system.

Mckay, C. W.↗

NASA NTRS2009

Connecting Research and Practice: An Experience Report on Research Infusion with SAVE

NASA systems need to be highly dependable to avoid catastrophic mission failures. This calls for rigorous engineering processes including meticulous validation and verification. However, NASA systems are often highly distributed and overwhelmingly complex, making the software portion of these systems challenging to understand, maintain, change, reuse, and test. NASA's systems are long-lived and the software maintenance process typically constitutes 60-80% of the total cost of the entire lifecycle. Thus, in addition to the technical challenges of ensuring high life-time quality of NASA's systems, the post-development phase also presents a significant financial burden. Some of NASA's software-related challenges could potentially be addressed by some of the many powerful technologies that are being developed in software research laboratories. Many of these research technologies seek to facilitate maintenance and evolution by for example architecting, designing and modeling for quality, flexibility, and reuse. Other technologies attempt to detect and remove defects and other quality issues by various forms of automated defect detection, architecture analysis, and various forms of sophisticated simulation and testing. However promising, most such research technologies nevertheless do not make the transition from the research lab to the software lab. One reason the transition from research to practice seldom occurs is that research infusion and technology transfer is difficult. For example, factors related to the technology are sometimes overshadowed by other types of factors such as reluctance to change and therefore prohibits the technology from sticking. Successful infusion might also take very long time. One famous study showed that the discrepancy between the conception of the idea and its practical use was 18 years plus or minus three. Nevertheless, infusing new technology is possible. We have found that it takes special circumstances for such research infusion to succeed: 1) there must be evidence that the technology works in the practitioner's particular domain, 2) there must be a potential for great improvements and enhanced competitive edge for the practitioner, 3) the practitioner has to have strong individual curiosity and continuous interest in trying out new technologies, 4) the practitioner has to have support on multiple levels (i.e. from the researchers, from management, from sponsors etc), and 5) to remain infused, the new technology has to be integrated into the practitioner's processes so that it becomes a natural part of the daily work. NASA IV&V's Research Infusion initiative sponsored by NASA's Office of Safety & Mission Assurance (OSMA) through the Software Assurance Research Program (SARP), strives to overcome some of the problems related to research infusion.

Lindvall, Mikael↗

NASA NTRS2018

IV&V Planning & Execution Initiative

Independent verification and validation (IV&V) is a systems engineering discipline which seeks to provide confidence that mission-critical software-dependent systems will operate safely and achieve all mission objectives. In order to optimize application of IV&V resources, it is necessary to identify specific assurance objectives that are most likely to reveal difficulty and therefore to increase confidence. Software assurance literature deals predominantly with the code itself. However, our experience is that the most significant software-related issues are discovered via analysis of other system artifacts such as operations concepts, requirements and design. Consequently, it is desirable to, when assessing risk and defining assurance objectives, consider all facets of the software system architecture. We describe in this presentation progress and lessons learned from an IV&V capability development initiative to enhance the IV&V project planning process using all available architecture information.

Risk↗

NASA NTRS2006

Emerging technologies for V&V of ISHM software for space exploration

Systems1,2 required to exhibit high operational reliability often rely on some form of fault protection to recognize and respond to faults, preventing faults' escalation to catastrophic failures. Integrated System Health Management (ISHM) extends the functionality of fault protection to both scale to more complex systems (and systems of systems), and to maintain capability rather than just avert catastrophe. Forms of ISHM have been utilized to good effect in the maintenance phase of systems' total lifecycles (often referred to as 'condition-based mainte-nance'), but less so in a 'fault protection' role during actual operations. One of the impediments to such use lies in the challenges of verification, validation and certification of ISHM systems themselves. This paper makes the case that state-of-the-practice V&V and certification techniques will not suffice for emerging forms of ISHM systems; however, a number of maturing software engineering assurance technologies show particular promise for addressing these ISHM V&V challenges.

fault detection, isolation, and recovery↗

NASA NTRS2020

Is Structured Agile an Oxymoron? Tales from Implementing and Executing Agile in a US Government Environment

To paraphrase a famous quote, "No plan survives contact with the reality." Software (SW) development is often a classic example of this: whatever the plan was for a particular development, it often does not survive contact with technical realities, budget realities, program realities and schedule realities. Traditionally, SW development has followed a waterfall methodology with requirements being rigorously specified before the design, which was completed before the coding and unit testing started, which were in turn finished before validation and verification started. This model of SW engineering derives much from the HW engineering of large systems, and has been the standard methodology used in US government software acquisitions and systems for decades, with highly variable results. US Government SW requirements are built around Waterfall concepts, which assume that the plan will survive contact with reality, or at least that modifications to the plan are relatively small, and relatively few.Because of the inefficiencies and difficulties inherent in Waterfall, the commercial SW world started using a different SW development methodology called Agile more than 20 years ago. Agile believes that a plan should evolve and learn rapidly in response to the realities encountered. At its core, there are a few key elements of Agile:- A small team of people which is highly flexible and adaptive. The team collaborates and interoperates through sophisticated development architectures and release environments- An iterative, incremental development and release approach which is based upon the concept that knowledge comes from experience within the team, and that the team makes decisions based upon what it knows- A team culture which prizes transparency, inspection and adaptation. These values are necessary so that the team experience and decision making is transparent and responsive to the realities encountered during development and testingSo, how to use Agile in a US Government environment? GMSEC (Goddard Mission Services Evolution Center) develops satellite ground system software for NASA and other US Government agencies. The SW developed by the team contains a large code base of many applications used within satellite mission operations centers. It spans the full gamut of SW development types: from SW which is in a classic maintenance and sustainment mode, to new developments with a fairly well understood scope and approach, to new developments whose scope and approach are quite unclear and which require significant research and prototyping. Team members move between all of these different types of SW development. Waterfall was inadequate to the programmatic and technical needs of the team, as well as the various types of SW development being done. The software plan was not surviving contact with the technical and programmatic realities experienced by the team. To address this, the team started a small pilot project in 2016 to test the use of Agile within a small subset of the team for a new web services application. In early 2018, the use of Agile was expanded to the whole team and all the software, but we had to fulfill the NASA SW development requirements. And we needed to do this while still remaining true to the key Agile elements of transparency, inspection and adaption. In order to do this, the team worked very closely with the Software Process Improvement (SPI) team at NASA Goddard, as well as NASA engineering manageme

Beech, Theresa W.↗

NASA NTRS2015

PHM for Ground Support Systems Case Study: From Requirements to Integration

This session will detail the experience of members of the NASA Ames Prognostic Center of Excellence (PCoE) producing PHM tools for NASA Advanced Ground Support Systems, including the challenges in applying their research in a production environment. Specifically, we will 1) go over the systems engineering and review process used; 2) Discuss the challenges and pitfalls in this process; 3) discuss software architecting, documentation, verification and validation activities and 4) discuss challenges in communicating the benefits and limitations of PHM Technologies.

Prognostics↗

NASA NTRS1997

Verification and Validation in a Rapid Software Development Process

The high cost of software production is driving development organizations to adopt more automated design and analysis methods such as rapid prototyping, computer-aided software engineering (CASE) tools, and high-level code generators. Even developers of safety-critical software system have adopted many of these new methods while striving to achieve high levels Of quality and reliability. While these new methods may enhance productivity and quality in many cases, we examine some of the risks involved in the use of new methods in safety-critical contexts. We examine a case study involving the use of a CASE tool that automatically generates code from high-level system designs. We show that while high-level testing on the system structure is highly desirable, significant risks exist in the automatically generated code and in re-validating releases of the generated code after subsequent design changes. We identify these risks and suggest process improvements that retain the advantages of rapid, automated development methods within the quality and reliability contexts of safety-critical projects.

Callahan, John R.↗

NASA NTRS2022

Actuator and Motor Control End-to-End V&V on the Mars 2020 Rover

The Mars 2020 Perseverance rover is the most advanced robotic exploration system ever sent to another planet. To support the complex scientific and mobility needs of the mission, the rover utilizes 33 actuators, three multi-degree-of-freedom force-torque sensors, fifteen single or dual-speed resolvers, two solenoid valves, and twelve contact switches. The control for these actuators and sensors is achieved by several levels of flight software, coordinated between two computers with varying bandwidth control loops. Furthermore, the actuators and sensors were integrated into multiple larger robotic mechanisms that were delivered by different organizations at various points in the Integration and Test (I&T) timeline. All of this created a very complex Verification and Validation (V&V) scenario involving multiple subsystems and teams, several hardware and software testbeds with varying levels of fidelity, and significant systems engineering to ensure the overall I&T schedule could be maintained while ensuring system hardware safety.This paper details the integrated V&V effort across multiple teams and venues to provide full coverage of all necessary functionality, performance, and fault protection. First, it provides an overview of how the V&V campaign was subdivided among teams and venues and provides descriptions of the various hardware configurations used to support the testing. The Mars 2020 implementation of the plan incorporates many of the lessons learned from Mars Science Laboratory’s test campaign, and these value-added modifications are discussed here. Also included in this section is the system-level environmental testing approach used for mechanisms. Second, the paper describes the phased approach used by the teams to support new hardware and software deliveries to testbed and Systems I&T. In this approach the test campaign was built upon higher-level mechanism needs for performance, functionality, and safety at specific times in the campaign. Finally, the paper discusses lessons learned from the V&V campaign that should be applied to future large-scale motion control testing efforts.

Borne, Davis↗

NASA NTRS2005

Performance Evaluation of a Data Validation System

Online data validation is a performance-enhancing component of modern control and health management systems. It is essential that performance of the data validation system be verified prior to its use in a control and health management system. A new Data Qualification and Validation (DQV) Test-bed application was developed to provide a systematic test environment for this performance verification. The DQV Test-bed was used to evaluate a model-based data validation package known as the Data Quality Validation Studio (DQVS). DQVS was employed as the primary data validation component of a rocket engine health management (EHM) system developed under NASA's NGLT (Next Generation Launch Technology) program. In this paper, the DQVS and DQV Test-bed software applications are described, and the DQV Test-bed verification procedure for this EHM system application is presented. Test-bed results are summarized and implications for EHM system performance improvements are discussed.

Wong, Edmond↗

NASA NTRS1987

SAGA: A project to automate the management of software production systems

The Software Automation, Generation and Administration (SAGA) project is investigating the design and construction of practical software engineering environments for developing and maintaining aerospace systems and applications software. The research includes the practical organization of the software lifecycle, configuration management, software requirements specifications, executable specifications, design methodologies, programming, verification, validation and testing, version control, maintenance, the reuse of software, software libraries, documentation, and automated management.

Campbell, Roy H.↗

NASA NTRS2017

Update: Advancement of Contact Dynamics Modeling for Human Spaceflight Simulation Applications

Pong is a new software tool developed at the NASA Johnson Space Center that advances interference-based geometric contact dynamics based on 3D graphics models. The Pong software consists of three parts: a set of scripts to extract geometric data from 3D graphics models, a contact dynamics engine that provides collision detection and force calculations based on the extracted geometric data, and a set of scripts for visualizing the dynamics response with the 3D graphics models. The contact dynamics engine can be linked with an external multibody dynamics engine to provide an integrated multibody contact dynamics simulation. This paper provides a detailed overview of Pong including the overall approach and modeling capabilities, which encompasses force generation from contact primitives and friction to computational performance. Two specific Pong-based examples of International Space Station applications are discussed, and the related verification and validation using this new tool are also addressed.

Contact dynamics↗

NASA NTRS1997

Knowledge-Based Aircraft Automation: Managers Guide on the use of Artificial Intelligence for Aircraft Automation and Verification and Validation Approach for a Neural-Based Flight Controller

The ultimate goal of this report was to integrate the powerful tools of artificial intelligence into the traditional process of software development. To maintain the US aerospace competitive advantage, traditional aerospace and software engineers need to more easily incorporate the technology of artificial intelligence into the advanced aerospace systems being designed today. The future goal was to transition artificial intelligence from an emerging technology to a standard technology that is considered early in the life cycle process to develop state-of-the-art aircraft automation systems. This report addressed the future goal in two ways. First, it provided a matrix that identified typical aircraft automation applications conducive to various artificial intelligence methods. The purpose of this matrix was to provide top-level guidance to managers contemplating the possible use of artificial intelligence in the development of aircraft automation. Second, the report provided a methodology to formally evaluate neural networks as part of the traditional process of software development. The matrix was developed by organizing the discipline of artificial intelligence into the following six methods: logical, object representation-based, distributed, uncertainty management, temporal and neurocomputing. Next, a study of existing aircraft automation applications that have been conducive to artificial intelligence implementation resulted in the following five categories: pilot-vehicle interface, system status and diagnosis, situation assessment, automatic flight planning, and aircraft flight control. The resulting matrix provided management guidance to understand artificial intelligence as it applied to aircraft automation. The approach taken to develop a methodology to formally evaluate neural networks as part of the software engineering life cycle was to start with the existing software quality assurance standards and to change these standards to include neural network development. The changes were to include evaluation tools that can be applied to neural networks at each phase of the software engineering life cycle. The result was a formal evaluation approach to increase the product quality of systems that use neural networks for their implementation.

Broderick, Ron↗