Search NASA⌕ Search

SEARCH · Search NASA

Results for “Verification and Validation (V&V)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Verification and Validation for Energy System Co-Design and Simulation

This project was only active for two months and was terminated due to a re-organization of the sponsoring initiative. The original purpose of the project was to develop and apply verification and validation (V&V) methodologies for the Energy System Co-Design with Multiple Objectives and Power Electronics (E-COMP) Initiative. This report summarizes the project and the work that was completed in the shortened time frame.

42 ENGINEERING↗

Verification and Validation for Flight-Critical Systems (VVFCS)

On March 31, 2009 a Request for Information (RFI) was issued by NASA s Aviation Safety Program to gather input on the subject of Verification and Validation (V & V) of Flight-Critical Systems. The responses were provided to NASA on or before April 24, 2009. The RFI asked for comments in three topic areas: Modeling and Validation of New Concepts for Vehicles and Operations; Verification of Complex Integrated and Distributed Systems; and Software Safety Assurance. There were a total of 34 responses to the RFI, representing a cross-section of academic (26%), small & large industry (47%) and government agency (27%).

Graves, Sharon S.↗

Verification and Validation Tests of the Gamma Library of the ARC Software Package (Rev.1)

As part of the verification and validation (V&V) efforts of the Argonne Reactor Code (ARC) software system to support the Versatile Test Reactor (VTR) project of the U.S. Department of Energy (DOE), work has been performed to verify and validate the gamma (or photon) library of the multigroup cross section generation code MC 2 -3 of the ARC system, which is the essential part of the coupled neutron and gamma heating calculations to determine the heat generation rate or power distribution in the core. This V&V study was carried out in three steps: 1) review of the procedures and utility programs for generating the MC 2 -3 gamma library, 2) verification tests of cross sections and KERMA factors by comparing the total heat generation rate of coupled neutron and gamma calculation with that obtained with reaction Q values under the assumption of local gamma energy deposition and by comparing the prompt heat generation rate with that calculated with the MCNP6.2 code, and 3) validation tests of cross sections and KERMA factors by analyzing the ZPPR-15D gamma dose data measured with thermoluminescent dosimeters (TLD).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Verification and Validation of the PLTEMP/ANL Code for Thermal-Hydraulic Analysis of Experimental and Test Reactors, Volume 1

This is Volume 1 of a two-volume document that collects the verification and validation (V&V) works done for the PLTEMP/ANL code during the years of its development and improvement. Volume 1 describes the V&V of sixteen capabilities of the PLTEMP/ANL code that were identified by research reactor analysts as frequently used in their thermal-hydraulic analysis. Volume 2 describes the V&V of developments and improvements since the release of PLTEMP/ANL Version 4.3. Each chapter of the document focuses on verifying or validating a specific part of the software that calculates a specific phenomenon, e.g., channel flow calculation, coolant property calculation, heat transfer calculation, and flow instability calculation. Software verification is performed by comparing the code with hand calculation, Microsoft spreadsheet calculation, Mathematica calculation, or MATLAB calculation. The software validation is done by comparing the code with experimental data or a widely tested code like the RELAP5 code. In addition, some PLTEMP/ANL V&V works that are available in the open literature are simply cited in Volume 1 of the document. PLTEMP/ANL has been used in the safety analysis reports of several US and foreign research reactors licensed and converted from highly enriched uranium fuel to low-enriched uranium fuel. A list of such reactors is given in Volume 1 of the document.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Verification and Validation of the PLTEMP/ANL Code for Thermal-Hydraulic Analysis of Experimental and Test Reactors, Volume 2

This is Volume 2 of a two-volume document that collects the verification and validation (V&V) works done for the PLTEMP/ANL code during the years of its development and improvement. Volume 1 describes the V&V of sixteen capabilities of the PLTEMP/ANL code that were identified by research reactor analysts as frequently used in their thermal-hydraulic analysis. Volume 2 describes the V&V of developments and improvements since the release of PLTEMP/ANL Version 4.3. Each chapter of the document focuses on verifying or validating a specific part of the software that calculates a particular phenomenon, e.g., channel flow calculation, coolant property calculation, heat transfer calculation, and flow instability calculation. Software verification is performed by comparing the code with a hand calculation, Microsoft spreadsheet calculation, Mathematica calculation, or MATLAB calculation. The software validation is achieved by comparing the code with experimental data or a widely tested code like the RELAP5 code. In addition, some PLTEMP/ANL V&V works that are available in the open literature are simply referenced in Volume 1 of the document. PLTEMP/ANL has been used in safety analysis reports of several US and foreign research reactors licensed and converted from highly enriched uranium fuel to low-enriched uranium fuel. A list of such reactors is given in Volume 1 of the document.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Development and Use of Engineering Standards for Computational Fluid Dynamics for Complex Aerospace Systems

Computational fluid dynamics (CFD) and other advanced modeling and simulation (M&S) methods are increasingly relied on for predictive performance, reliability and safety of engineering systems. Analysts, designers, decision makers, and project managers, who must depend on simulation, need practical techniques and methods for assessing simulation credibility. The AIAA Guide for Verification and Validation of Computational Fluid Dynamics Simulations (AIAA G-077-1998 (2002)), originally published in 1998, was the first engineering standards document available to the engineering community for verification and validation (V&V) of simulations. Much progress has been made in these areas since 1998. The AIAA Committee on Standards for CFD is currently updating this Guide to incorporate in it the important developments that have taken place in V&V concepts, methods, and practices, particularly with regard to the broader context of predictive capability and uncertainty quantification (UQ) methods and approaches. This paper will provide an overview of the changes and extensions currently underway to update the AIAA Guide. Specifically, a framework for predictive capability will be described for incorporating a wide range of error and uncertainty sources identified during the modeling, verification, and validation processes, with the goal of estimating the total prediction uncertainty of the simulation. The Guide's goal is to provide a foundation for understanding and addressing major issues and concepts in predictive CFD. However, this Guide will not recommend specific approaches in these areas as the field is rapidly evolving. It is hoped that the guidelines provided in this paper, and explained in more detail in the Guide, will aid in the research, development, and use of CFD in engineering decision-making.

Lee, Hyung B.↗

Formal Verification for a Next-Generation Space Shuttle

This paper discusses the verification and validation (V&2) of advanced software used for integrated vehicle health monitoring (IVHM), in the context of NASA's next-generation space shuttle. We survey the current VBCV practice and standards used in selected NASA projects, review applicable formal verification techniques, and discuss their integration info existing development practice and standards. We also describe two verification tools, JMPL2SMV and Livingstone PathFinder, that can be used to thoroughly verify diagnosis applications that use model-based reasoning, such as the Livingstone system.

Nelson, Stacy D.↗

Psyche Hardware Abstraction Simulation Environment (PHASE): Development of High-Fidelity Downlink Model for Flight Software Performance Testing

NASA’s Katherine Johnson Independent Verification and Validation (IV&V) Independent Test Capability (ITC) Team develops high fidelity software-only simulations (or “digital-twins”) of NASA’s most critical missions that are then used for software verification and validation (V&V). One such digital-twin simulator nearing maturity is the Psyche Hardware Abstraction Simulation Environment (PHASE). PHASE utilizes an instruction set simulator along with ITC-developed avionics models. PHASE includes a newly-developed model of the telecom hardware with a high-fidelity downlink interface, capable of supporting reconfigurable downlink data rates and transmission modes. The different downlink transmission modes allow more processing to be ceded from software onto the telecom hardware. Thus, we are able to monitor the performance of the downlink software module in the time-partitioned OS and perform more realistic dynamic testing based on software’s resource utilization. This presentation will cover PHASE, avionics modeling, the downlink model, and its usage for flight software performance testing.

modeling↗

Verification and Validation of Elastodynamic Simulation Software for Aerospace Research

Physics-based simulation of nondestructive evaluation (NDE) inspection can help to advance the inspectability and reliability of mechanical systems. However, NDE simulations applicable to non-idealized mechanical components often require large compute domains and long run times. This has prompted development of custom NDE simulation software tailored to high performance computing (HPC) hardware. Verification and validation (V&V) is an integral part of developing this software to ensure implementations are robust and applicable to inspection problems, producing tools and simulations suitable for computational NDE research. This presentation addresses factors common to V&V of several elastodynamic simulation codes applicable to ultrasonic NDE. Examples are drawn from in-house simulation software at NASA Langley Research Center, ranging from ensuring reliability in a 1D heterogeneous media wave equation solver to the V&V needs of 3D cluster-parallel elastodynamic software. Factors specific to a research environment are addressed, where individual simulation results can be as relevant as the software product itself. Distinct facets of V&V are discussed including testing to establish software reliability, employing systematic approaches for consistency with fundamental conservation laws, establishing the numerical stability of algorithms, and demonstrating concurrence with empirical data. This talk also addresses V&V practices for small groups of researchers. This includes establishing resources (e.g. time and personnel) for V&V during project planning to mitigate and control the risk of setbacks. Similarly, we identify ways for individual researchers to use V&V during simulation software development itself to both speed up the development process and reduce incurred technical debt.

NDE↗

Sierra/SD – Verification Test Manual – 5.22

Verification and validation (V&V) of scientific computing programs are important at Sandia National Labs due to the expanding role of computational simulation in managing the United States nuclear stockpile. The complexities of structural response calculations used to analyze physical problems, the varieties of codes applied to the calculations, and the importance of accurate predictions when assessing field conditions demand confidence in the consistency and accuracy of computer codes. Confidence in the accuracy of the predictions arising from computer simulations must ultimately be gained through verification and validation. The Sierra salinas structural dynamics analysis code, Sierra/SD, is used at the DOE Laboratories, and in several DOD projects. The roles of Sierra/SD in the qualification of weapon systems and components for normal and hostile environments throughout the Stockpile-to-Target Sequence include to, • Redesign weapon components. • Certify weapon components and systems for target environments such as hypersonic vehicles. • Certify that components will survive the thermal mechanical shock loads associated with hostile environments. • Evaluate current stockpile issues, including issues associated with uncertainty quantification. • Address many other problems that are encountered in stockpile management. The Sierra/SD verification plan is described, and an evolving set of key verification tests are described in detail. The verification tests ensure the correctness of the mathematics and numerical algorithms associated with functionality describing engineering phenomena. Development is in accordance with a set of tailored Software Quality Engineering (SQE) practices. SQE practices guide the overall verification and validation effort.

97 MATHEMATICS AND COMPUTING↗

Psyche Early Project Verification & Validation Planning Development

The Psyche mission to the asteroid (16) Psyche was selected as the fourteenth mission in the Discovery program in January 2017. The Psyche mission will determine if (16) Psyche is the core of a larger differentiated body. As part of the development of this mission a Verification and Validation (V&V) engineer was assigned early in the project’s design Phase B. This paper will discuss some of the strategies that the Psyche team is using to take full advantage of the early planning for V&V on flight projects and specifically how the Psyche mission is approaching these tasks. This paper will discuss the effects of having a V&V mindset on: 1) The Psyche requirements development process, and how focusing not only on how the team will verify these requirements but also on developing the tools necessary to track and monitor that verification feeds back into the requirement development process. 2) The Psyche testbed development, and how using a verification mindset is useful for identifying holes in the testbed development process, including the required testbed speed and how to think about testbed certification early in the process while encouraging trades and developing relationships with the testbed team. 3) Developing new V&V Tools for Psyche, and how developing tools early in the project development process means that they can influence the development of other requirement and scheduling tools. 4) Verification Activity Planning, which is typically done to a preliminary level during Phase B of the project, and is important for understanding the major testing that is needed to ensure that the system as built represents the design. On Psyche we are not only focused on bringing this planning to a preliminary level, we are also developing a V&V focused schedule to help us de-conflict V&V activities that may require similar resources early on in the program’s development. 5) Validation Planning, for which we are working with the testbed and model development teams to ensure that their models can be effectively validated and that the plans to do so are in place. In this paper we will describe how the Psyche mission is approaching each of these V&V areas and identify lessons that can be taken by other space missions trying to decide how much effort should be put into V&V early in the project lifecycle.

Solish, Benjamin↗

Verification and Performance Impact of the New Parallel MCNP6.3 Particle Track Output Capability for Subcritical Multiplication Simulations

The MCNP6® code, version 6.3, has several new features that are intended to ultimately replace legacy features that are now marked for deprecation. One of these features is the new particle track output (PTRAC) format and capability, where the legacy PTRAC capability still exists alongside the modern PTRAC capability in MCNP6.3. While the MCNP6.3 code has been extensively verified and validated for many applications, the PTRAC feature is not exercised in any of the typical verification and validation (V&V) applications studied during the course of a typical MCNP code release. The primary goal of this paper is to verify that the legacy and modern PTRAC feature produces equivalent results for subcritical multiplication benchmarks previously studied. In the process of verifying that the simulated benchmark results are equivalent, the computational performance is compared between the legacy and modern PTRAC uses. In addition to verification of the update, which is important to the community as a whole, this effort also supports advances in the simulation of recent subcritical neutron noise measurements that require higher computational effort per second of real-time measurement than that of systems typically measured.

97 MATHEMATICS AND COMPUTING↗

Strategies for Validation Testing of Ground Systems

In order to accomplish the full Vision for Space Exploration announced by former President George W. Bush in 2004, NASA will have to develop a new space transportation system and supporting infrastructure. The main portion of this supporting infrastructure will reside at the Kennedy Space Center (KSC) in Florida and will either be newly developed or a modification of existing vehicle processing and launch facilities, including Ground Support Equipment (GSE). This type of large-scale launch site development is unprecedented since the time of the Apollo Program. In order to accomplish this successfully within the limited budget and schedule constraints a combination of traditional and innovative strategies for Verification and Validation (V&V) have been developed. The core of these strategies consists of a building-block approach to V&V, starting with component V&V and ending with a comprehensive end-to-end validation test of the complete launch site, called a Ground Element Integration Test (GEIT). This paper will outline these strategies and provide the high level planning for meeting the challenges of implementing V&V on a large-scale development program. KEY WORDS: Systems, Elements, Subsystem, Integration Test, Ground Systems, Ground Support Equipment, Component, End Item, Test and Verification Requirements (TVR), Verification Requirements (VR)

Annis, Tammy↗

NASA Space Observatory Precision Pointing Benchmark Problem Development

An interagency workshop in May 2021 on guidance, navigation, and control (GNC) verification and validation (V&V) methods and techniques identified needs for the formulation and public release of relevant V&V benchmark problems as a practical way forward for the discipline. A decision was made by the NASA Technical Fellow for GNC and the NASA Engineering and Safety Center (NESC) GNC Technical Discipline Team (TDT) to produce a benchmark problem focusing on the problem of space observatory precision pointing. This report contains the results of the NESC assessment.

NASA Engineering and Safety Center↗

Validation and Verification of TEDS Facility HYBRID Modeling

The HYBRID modeling repository is an in-house developed library of models for selected integrated energy systems (IES) modelling. HYBRID models have been developed since 2015 to describe the physical operation of tightly coupled thermal systems including power generators, thermal transport systems, thermal storage, thermal-to-electric conversion systems, and other thermal applications. Here, validation and verification (V&V) capabilities are demonstrated using the Thermal Energy Distribution System (TEDS) at INL. Building upon prior work, the TEDS model has been updated and verified so that it better represents the installed system configuration and the operating control system. The model control system was changed to allow replication of actual experimental procedures. Experimental operations focusing primarily on thermocline tank performance were devised and performed. Several anomalies were found in the operation data of the experiment facility. V&V activities calibrating a selected input parameter are demonstrated on a single component as well as with a single parameter within the thermocline. Calibrating is then demonstrated on multiple components and a multi-parameter metric for the entire system. The validation methodology is successfully applied to validate the model with experimental data. It is also used to confirm a hypothesis behind one of the anomalies in experimental performance.

25 ENERGY STORAGE↗

Verification and Validation of the New MCNP6.3 Criticality Features

The MCNP6® code, version 6.3, has been extensively verified and validated for many applications. The use of the same default capabilities existing in the MCNP6.2 code are also used in the verification and validation (V&V) of the MCNP6.3 code. In this paper, selected new features, code enhancements, and bug fixes in the MCNP6.3 code that impact criticality safety applications are described and investigated. More specifically, the changes within the MCNP6.3 code that are studied in this work include both the new fission matrix and the Doppler broadening resonance correction (DBRC) features. For nuclear criticality safety applications, the V&V benchmark problems within the criticality, extended criticality, and Rossi-α suites are used to study the upgrades within the MCNP6.3 code. Some additional investigations into benchmarks at elevated temperatures are used to showcase the impacts of the temperature-specific capabilities.

97 MATHEMATICS AND COMPUTING↗

Real-Time Simulation for Verification and Validation of Diagnostic and Prognostic Algorithms

To verify that a health management system (HMS) performs as expected, a virtual system simulation capability, including interaction with the associated platform or vehicle, very likely will need to be developed. The rationale for developing this capability is discussed and includes the limited capability to seed faults into the actual target system due to the risk of potential damage to high value hardware. The capability envisioned would accurately reproduce the propagation of a fault or failure as observed by sensors located at strategic locations on and around the target system and would also accurately reproduce the control system and vehicle response. In this way, HMS operation can be exercised over a broad range of conditions to verify that it meets requirements for accurate, timely response to actual faults with adequate margin against false and missed detections. An overview is also presented of a real-time rocket propulsion health management system laboratory which is available for future rocket engine programs. The health management elements and approaches of this lab are directly applicable for future space systems. In this paper the various components are discussed and the general fault detection, diagnosis, isolation and the response (FDIR) concept is presented. Additionally, the complexities of V&V (Verification and Validation) for advanced algorithms and the simulation capabilities required to meet the changing state-of-the-art in HMS are discussed.

Aguilar, Robet↗

Methodologies for Verification and Validation of Space Launch System (SLS) Structural Dynamic Models: Appendices

Verification and validation (V&V) is a highly challenging undertaking for SLS structural dynamics models due to the magnitude and complexity of SLS subassemblies and subassemblies. Responses to challenges associated with V&V of Space Launch System (SLS) structural dynamics models are presented in Volume I of this paper. Four methodologies addressing specific requirements for V&V are discussed. (1) Residual Mode Augmentation (RMA). (2) Modified Guyan Reduction (MGR) and Harmonic Reduction (HR, introduced in 1976). (3) Mode Consolidation (MC). Finally, (4) Experimental Mode Verification (EMV). This document contains the appendices to Volume I.

Coppolino, Robert N.↗