Search NASA⌕ Search

SEARCH · Search NASA

Results for “Verification & validation (V&V)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRS1990

Knowledge-based system verification and validation

The objective of this task is to develop and evaluate a methodology for verification and validation (V&V) of knowledge-based systems (KBS) for space station applications with high reliability requirements. The approach consists of three interrelated tasks. The first task is to evaluate the effectiveness of various validation methods for space station applications. The second task is to recommend requirements for KBS V&V for Space Station Freedom (SSF). The third task is to recommend modifications to the SSF to support the development of KBS using effectiveness software engineering and validation techniques. To accomplish the first task, three complementary techniques will be evaluated: (1) Sensitivity Analysis (Worchester Polytechnic Institute); (2) Formal Verification of Safety Properties (SRI International); and (3) Consistency and Completeness Checking (Lockheed AI Center). During FY89 and FY90, each contractor will independently demonstrate the user of his technique on the fault detection, isolation, and reconfiguration (FDIR) KBS or the manned maneuvering unit (MMU), a rule-based system implemented in LISP. During FY91, the application of each of the techniques to other knowledge representations and KBS architectures will be addressed. After evaluation of the results of the first task and examination of Space Station Freedom V&V requirements for conventional software, a comprehensive KBS V&V methodology will be developed and documented. Development of highly reliable KBS's cannot be accomplished without effective software engineering methods. Using the results of current in-house research to develop and assess software engineering methods for KBS's as well as assessment of techniques being developed elsewhere, an effective software engineering methodology for space station KBS's will be developed, and modification of the SSF to support these tools and methods will be addressed.

Johnson, Sally C.↗
DOE OSTI2022

Multiscale Experiments and Multiphysics Simulation of Multiphase Flow for Transportable Small Modular Reactors

A new type of safe, small, transportable nuclear reactor would address the intense and ever-growing global demand for energy produced via a resilient, carbon-free energy source. In this regard, transportable small modular reactors (SMRs) are being designed and developed for electricity generation within small/micro-grid/off-grid isolated systems, as well as for heat generation in industrial/residential applications. These reactors feature the capability to be fully factory fabricated and then directly transported to utilities’ sites as “plug-and-play” systems. Research and development (R&D) programs are underway at Idaho National Laboratory (INL) to successfully design, develop, and demonstrate such safe-by-design mobile reactor technologies, in collaboration with partner organizations. Multiscale experimental facilities and multiphysics simulation tools are required for reactor design verification and validation (V&V), and licensing. These advanced reactors are intended to feature passive safety systems such as passive containment cooling systems (PCCS), which consist of multiphase flows and multispecies distributions. This seminar talk will focus on designing and analyzing transportable SMR PCCS by using multiscale experiments and multiphysics computational fluid dynamics (CFD) simulations to support reactor licensing and safety. The corresponding research challenges are addressed via supportive verification and validation results generated by the models and simulation tools in combination with selective parametric and uncertainty analysis. This solution approach could blaze the trail for commercial adoption of such technologies. The facilities, simulation capabilities, and research opportunities available at INL in regard to such reactors and the integrated energy systems with which they go hand in hand are also discussed briefly, and may spark interest in deeper research as well as new collaborative projects.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRS2013

Work Practice Simulation of Complex Human-Automation Systems in Safety Critical Situations: The Brahms Generalized berlingen Model

The transition from the current air traffic system to the next generation air traffic system will require the introduction of new automated systems, including transferring some functions from air traffic controllers to on­-board automation. This report describes a new design verification and validation (V&V) methodology for assessing aviation safety. The approach involves a detailed computer simulation of work practices that includes people interacting with flight-critical systems. The research is part of an effort to develop new modeling and verification methodologies that can assess the safety of flight-critical systems, system configurations, and operational concepts. The 2002 Ueberlingen mid-air collision was chosen for analysis and modeling because one of the main causes of the accident was one crew's response to a conflict between the instructions of the air traffic controller and the instructions of TCAS, an automated Traffic Alert and Collision Avoidance System on-board warning system. It thus furnishes an example of the problem of authority versus autonomy. It provides a starting point for exploring authority/autonomy conflict in the larger system of organization, tools, and practices in which the participants' moment-by-moment actions take place. We have developed a general air traffic system model (not a specific simulation of Überlingen events), called the Brahms Generalized Ueberlingen Model (Brahms-GUeM). Brahms is a multi-agent simulation system that models people, tools, facilities/vehicles, and geography to simulate the current air transportation system as a collection of distributed, interactive subsystems (e.g., airports, air-traffic control towers and personnel, aircraft, automated flight systems and air-traffic tools, instruments, crew). Brahms-GUeM can be configured in different ways, called scenarios, such that anomalous events that contributed to the Überlingen accident can be modeled as functioning according to requirements or in an anomalous condition, as occurred during the accident. Brahms-GUeM thus implicitly defines a class of scenarios, which include as an instance what occurred at Überlingen. Brahms-GUeM is a modeling framework enabling "what if" analysis of alternative work system configurations and thus facilitating design of alternative operations concepts. It enables subsequent adaption (reusing simulation components) for modeling and simulating NextGen scenarios. This project demonstrates that BRAHMS provides the capacity to model the complexity of air transportation systems, going beyond idealized and simple flights to include for example the interaction of pilots and ATCOs. The research shows clearly that verification and validation must include the entire work system, on the one hand to check that mechanisms exist to handle failures of communication and alerting subsystems and/or failures of people to notice, comprehend, or communicate problematic (unsafe) situations; but also to understand how people must use their own judgment in relating fallible systems like TCAS to other sources of information and thus to evaluate how the unreliability of automation affects system safety. The simulation shows in particular that distributed agents (people and automated systems) acting without knowledge of each others' actions can create a complex, dynamic system whose interactive behavior is unexpected and is changing too quickly to comprehend and control.

complex systems↗
DOE OSTI2023

Verification and Validation for Energy System Co-Design and Simulation

This project was only active for two months and was terminated due to a re-organization of the sponsoring initiative. The original purpose of the project was to develop and apply verification and validation (V&V) methodologies for the Energy System Co-Design with Multiple Objectives and Power Electronics (E-COMP) Initiative. This report summarizes the project and the work that was completed in the shortened time frame.

42 ENGINEERING↗
NASA NTRS2010

Verification and Validation for Flight-Critical Systems (VVFCS)

On March 31, 2009 a Request for Information (RFI) was issued by NASA s Aviation Safety Program to gather input on the subject of Verification and Validation (V & V) of Flight-Critical Systems. The responses were provided to NASA on or before April 24, 2009. The RFI asked for comments in three topic areas: Modeling and Validation of New Concepts for Vehicles and Operations; Verification of Complex Integrated and Distributed Systems; and Software Safety Assurance. There were a total of 34 responses to the RFI, representing a cross-section of academic (26%), small & large industry (47%) and government agency (27%).

Graves, Sharon S.↗
DOE OSTI2022

Verification and Validation Tests of the Gamma Library of the ARC Software Package (Rev.1)

As part of the verification and validation (V&V) efforts of the Argonne Reactor Code (ARC) software system to support the Versatile Test Reactor (VTR) project of the U.S. Department of Energy (DOE), work has been performed to verify and validate the gamma (or photon) library of the multigroup cross section generation code MC 2 -3 of the ARC system, which is the essential part of the coupled neutron and gamma heating calculations to determine the heat generation rate or power distribution in the core. This V&V study was carried out in three steps: 1) review of the procedures and utility programs for generating the MC 2 -3 gamma library, 2) verification tests of cross sections and KERMA factors by comparing the total heat generation rate of coupled neutron and gamma calculation with that obtained with reaction Q values under the assumption of local gamma energy deposition and by comparing the prompt heat generation rate with that calculated with the MCNP6.2 code, and 3) validation tests of cross sections and KERMA factors by analyzing the ZPPR-15D gamma dose data measured with thermoluminescent dosimeters (TLD).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Verification and Validation of the PLTEMP/ANL Code for Thermal-Hydraulic Analysis of Experimental and Test Reactors, Volume 1

This is Volume 1 of a two-volume document that collects the verification and validation (V&V) works done for the PLTEMP/ANL code during the years of its development and improvement. Volume 1 describes the V&V of sixteen capabilities of the PLTEMP/ANL code that were identified by research reactor analysts as frequently used in their thermal-hydraulic analysis. Volume 2 describes the V&V of developments and improvements since the release of PLTEMP/ANL Version 4.3. Each chapter of the document focuses on verifying or validating a specific part of the software that calculates a specific phenomenon, e.g., channel flow calculation, coolant property calculation, heat transfer calculation, and flow instability calculation. Software verification is performed by comparing the code with hand calculation, Microsoft spreadsheet calculation, Mathematica calculation, or MATLAB calculation. The software validation is done by comparing the code with experimental data or a widely tested code like the RELAP5 code. In addition, some PLTEMP/ANL V&V works that are available in the open literature are simply cited in Volume 1 of the document. PLTEMP/ANL has been used in the safety analysis reports of several US and foreign research reactors licensed and converted from highly enriched uranium fuel to low-enriched uranium fuel. A list of such reactors is given in Volume 1 of the document.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Verification and Validation of the PLTEMP/ANL Code for Thermal-Hydraulic Analysis of Experimental and Test Reactors, Volume 2

This is Volume 2 of a two-volume document that collects the verification and validation (V&V) works done for the PLTEMP/ANL code during the years of its development and improvement. Volume 1 describes the V&V of sixteen capabilities of the PLTEMP/ANL code that were identified by research reactor analysts as frequently used in their thermal-hydraulic analysis. Volume 2 describes the V&V of developments and improvements since the release of PLTEMP/ANL Version 4.3. Each chapter of the document focuses on verifying or validating a specific part of the software that calculates a particular phenomenon, e.g., channel flow calculation, coolant property calculation, heat transfer calculation, and flow instability calculation. Software verification is performed by comparing the code with a hand calculation, Microsoft spreadsheet calculation, Mathematica calculation, or MATLAB calculation. The software validation is achieved by comparing the code with experimental data or a widely tested code like the RELAP5 code. In addition, some PLTEMP/ANL V&V works that are available in the open literature are simply referenced in Volume 1 of the document. PLTEMP/ANL has been used in safety analysis reports of several US and foreign research reactors licensed and converted from highly enriched uranium fuel to low-enriched uranium fuel. A list of such reactors is given in Volume 1 of the document.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2016

Development and Use of Engineering Standards for Computational Fluid Dynamics for Complex Aerospace Systems

Computational fluid dynamics (CFD) and other advanced modeling and simulation (M&S) methods are increasingly relied on for predictive performance, reliability and safety of engineering systems. Analysts, designers, decision makers, and project managers, who must depend on simulation, need practical techniques and methods for assessing simulation credibility. The AIAA Guide for Verification and Validation of Computational Fluid Dynamics Simulations (AIAA G-077-1998 (2002)), originally published in 1998, was the first engineering standards document available to the engineering community for verification and validation (V&V) of simulations. Much progress has been made in these areas since 1998. The AIAA Committee on Standards for CFD is currently updating this Guide to incorporate in it the important developments that have taken place in V&V concepts, methods, and practices, particularly with regard to the broader context of predictive capability and uncertainty quantification (UQ) methods and approaches. This paper will provide an overview of the changes and extensions currently underway to update the AIAA Guide. Specifically, a framework for predictive capability will be described for incorporating a wide range of error and uncertainty sources identified during the modeling, verification, and validation processes, with the goal of estimating the total prediction uncertainty of the simulation. The Guide's goal is to provide a foundation for understanding and addressing major issues and concepts in predictive CFD. However, this Guide will not recommend specific approaches in these areas as the field is rapidly evolving. It is hoped that the guidelines provided in this paper, and explained in more detail in the Guide, will aid in the research, development, and use of CFD in engineering decision-making.

Lee, Hyung B.↗
NASA NTRS2002

Formal Verification for a Next-Generation Space Shuttle

This paper discusses the verification and validation (V&2) of advanced software used for integrated vehicle health monitoring (IVHM), in the context of NASA's next-generation space shuttle. We survey the current VBCV practice and standards used in selected NASA projects, review applicable formal verification techniques, and discuss their integration info existing development practice and standards. We also describe two verification tools, JMPL2SMV and Livingstone PathFinder, that can be used to thoroughly verify diagnosis applications that use model-based reasoning, such as the Livingstone system.

Nelson, Stacy D.↗
NASA NTRSDate not supplied

Psyche Hardware Abstraction Simulation Environment (PHASE): Development of High-Fidelity Downlink Model for Flight Software Performance Testing

NASA’s Katherine Johnson Independent Verification and Validation (IV&V) Independent Test Capability (ITC) Team develops high fidelity software-only simulations (or “digital-twins”) of NASA’s most critical missions that are then used for software verification and validation (V&V). One such digital-twin simulator nearing maturity is the Psyche Hardware Abstraction Simulation Environment (PHASE). PHASE utilizes an instruction set simulator along with ITC-developed avionics models. PHASE includes a newly-developed model of the telecom hardware with a high-fidelity downlink interface, capable of supporting reconfigurable downlink data rates and transmission modes. The different downlink transmission modes allow more processing to be ceded from software onto the telecom hardware. Thus, we are able to monitor the performance of the downlink software module in the time-partitioned OS and perform more realistic dynamic testing based on software’s resource utilization. This presentation will cover PHASE, avionics modeling, the downlink model, and its usage for flight software performance testing.

modeling↗
NASA NTRSDate not supplied

Verification and Validation of Elastodynamic Simulation Software for Aerospace Research

Physics-based simulation of nondestructive evaluation (NDE) inspection can help to advance the inspectability and reliability of mechanical systems. However, NDE simulations applicable to non-idealized mechanical components often require large compute domains and long run times. This has prompted development of custom NDE simulation software tailored to high performance computing (HPC) hardware. Verification and validation (V&V) is an integral part of developing this software to ensure implementations are robust and applicable to inspection problems, producing tools and simulations suitable for computational NDE research. This presentation addresses factors common to V&V of several elastodynamic simulation codes applicable to ultrasonic NDE. Examples are drawn from in-house simulation software at NASA Langley Research Center, ranging from ensuring reliability in a 1D heterogeneous media wave equation solver to the V&V needs of 3D cluster-parallel elastodynamic software. Factors specific to a research environment are addressed, where individual simulation results can be as relevant as the software product itself. Distinct facets of V&V are discussed including testing to establish software reliability, employing systematic approaches for consistency with fundamental conservation laws, establishing the numerical stability of algorithms, and demonstrating concurrence with empirical data. This talk also addresses V&V practices for small groups of researchers. This includes establishing resources (e.g. time and personnel) for V&V during project planning to mitigate and control the risk of setbacks. Similarly, we identify ways for individual researchers to use V&V during simulation software development itself to both speed up the development process and reduce incurred technical debt.

NDE↗
DOE OSTI2024

Sierra/SD – Verification Test Manual – 5.22

Verification and validation (V&V) of scientific computing programs are important at Sandia National Labs due to the expanding role of computational simulation in managing the United States nuclear stockpile. The complexities of structural response calculations used to analyze physical problems, the varieties of codes applied to the calculations, and the importance of accurate predictions when assessing field conditions demand confidence in the consistency and accuracy of computer codes. Confidence in the accuracy of the predictions arising from computer simulations must ultimately be gained through verification and validation. The Sierra salinas structural dynamics analysis code, Sierra/SD, is used at the DOE Laboratories, and in several DOD projects. The roles of Sierra/SD in the qualification of weapon systems and components for normal and hostile environments throughout the Stockpile-to-Target Sequence include to, • Redesign weapon components. • Certify weapon components and systems for target environments such as hypersonic vehicles. • Certify that components will survive the thermal mechanical shock loads associated with hostile environments. • Evaluate current stockpile issues, including issues associated with uncertainty quantification. • Address many other problems that are encountered in stockpile management. The Sierra/SD verification plan is described, and an evolving set of key verification tests are described in detail. The verification tests ensure the correctness of the mathematics and numerical algorithms associated with functionality describing engineering phenomena. Development is in accordance with a set of tailored Software Quality Engineering (SQE) practices. SQE practices guide the overall verification and validation effort.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2019

Psyche Early Project Verification & Validation Planning Development

The Psyche mission to the asteroid (16) Psyche was selected as the fourteenth mission in the Discovery program in January 2017. The Psyche mission will determine if (16) Psyche is the core of a larger differentiated body. As part of the development of this mission a Verification and Validation (V&V) engineer was assigned early in the project’s design Phase B. This paper will discuss some of the strategies that the Psyche team is using to take full advantage of the early planning for V&V on flight projects and specifically how the Psyche mission is approaching these tasks. This paper will discuss the effects of having a V&V mindset on: 1) The Psyche requirements development process, and how focusing not only on how the team will verify these requirements but also on developing the tools necessary to track and monitor that verification feeds back into the requirement development process. 2) The Psyche testbed development, and how using a verification mindset is useful for identifying holes in the testbed development process, including the required testbed speed and how to think about testbed certification early in the process while encouraging trades and developing relationships with the testbed team. 3) Developing new V&V Tools for Psyche, and how developing tools early in the project development process means that they can influence the development of other requirement and scheduling tools. 4) Verification Activity Planning, which is typically done to a preliminary level during Phase B of the project, and is important for understanding the major testing that is needed to ensure that the system as built represents the design. On Psyche we are not only focused on bringing this planning to a preliminary level, we are also developing a V&V focused schedule to help us de-conflict V&V activities that may require similar resources early on in the program’s development. 5) Validation Planning, for which we are working with the testbed and model development teams to ensure that their models can be effectively validated and that the plans to do so are in place. In this paper we will describe how the Psyche mission is approaching each of these V&V areas and identify lessons that can be taken by other space missions trying to decide how much effort should be put into V&V early in the project lifecycle.

Solish, Benjamin↗
DOE OSTI2023

Verification and Performance Impact of the New Parallel MCNP6.3 Particle Track Output Capability for Subcritical Multiplication Simulations

The MCNP6® code, version 6.3, has several new features that are intended to ultimately replace legacy features that are now marked for deprecation. One of these features is the new particle track output (PTRAC) format and capability, where the legacy PTRAC capability still exists alongside the modern PTRAC capability in MCNP6.3. While the MCNP6.3 code has been extensively verified and validated for many applications, the PTRAC feature is not exercised in any of the typical verification and validation (V&V) applications studied during the course of a typical MCNP code release. The primary goal of this paper is to verify that the legacy and modern PTRAC feature produces equivalent results for subcritical multiplication benchmarks previously studied. In the process of verifying that the simulated benchmark results are equivalent, the computational performance is compared between the legacy and modern PTRAC uses. In addition to verification of the update, which is important to the community as a whole, this effort also supports advances in the simulation of recent subcritical neutron noise measurements that require higher computational effort per second of real-time measurement than that of systems typically measured.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2009

Strategies for Validation Testing of Ground Systems

In order to accomplish the full Vision for Space Exploration announced by former President George W. Bush in 2004, NASA will have to develop a new space transportation system and supporting infrastructure. The main portion of this supporting infrastructure will reside at the Kennedy Space Center (KSC) in Florida and will either be newly developed or a modification of existing vehicle processing and launch facilities, including Ground Support Equipment (GSE). This type of large-scale launch site development is unprecedented since the time of the Apollo Program. In order to accomplish this successfully within the limited budget and schedule constraints a combination of traditional and innovative strategies for Verification and Validation (V&V) have been developed. The core of these strategies consists of a building-block approach to V&V, starting with component V&V and ending with a comprehensive end-to-end validation test of the complete launch site, called a Ground Element Integration Test (GEIT). This paper will outline these strategies and provide the high level planning for meeting the challenges of implementing V&V on a large-scale development program. KEY WORDS: Systems, Elements, Subsystem, Integration Test, Ground Systems, Ground Support Equipment, Component, End Item, Test and Verification Requirements (TVR), Verification Requirements (VR)

Annis, Tammy↗
NASA NTRS2024

NASA Space Observatory Precision Pointing Benchmark Problem Development

An interagency workshop in May 2021 on guidance, navigation, and control (GNC) verification and validation (V&V) methods and techniques identified needs for the formulation and public release of relevant V&V benchmark problems as a practical way forward for the discipline. A decision was made by the NASA Technical Fellow for GNC and the NASA Engineering and Safety Center (NESC) GNC Technical Discipline Team (TDT) to produce a benchmark problem focusing on the problem of space observatory precision pointing. This report contains the results of the NESC assessment.

NASA Engineering and Safety Center↗
DOE OSTI2023

Validation and Verification of TEDS Facility HYBRID Modeling

The HYBRID modeling repository is an in-house developed library of models for selected integrated energy systems (IES) modelling. HYBRID models have been developed since 2015 to describe the physical operation of tightly coupled thermal systems including power generators, thermal transport systems, thermal storage, thermal-to-electric conversion systems, and other thermal applications. Here, validation and verification (V&V) capabilities are demonstrated using the Thermal Energy Distribution System (TEDS) at INL. Building upon prior work, the TEDS model has been updated and verified so that it better represents the installed system configuration and the operating control system. The model control system was changed to allow replication of actual experimental procedures. Experimental operations focusing primarily on thermocline tank performance were devised and performed. Several anomalies were found in the operation data of the experiment facility. V&V activities calibrating a selected input parameter are demonstrated on a single component as well as with a single parameter within the thermocline. Calibrating is then demonstrated on multiple components and a multi-parameter metric for the entire system. The validation methodology is successfully applied to validate the model with experimental data. It is also used to confirm a hypothesis behind one of the anomalies in experimental performance.

25 ENERGY STORAGE↗