Search NASA⌕ Search

SEARCH · Search NASA

Results for “VERIFICATION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Implementation of a Drift Flux Model into SAM with Development of a Verification and Validation Test Suite for Modeling of Noncondensable Gas Mixtures

The advanced thermal-hydraulic system code, System Analysis Module (SAM), was originally developed for the modeling of single-phase flow in advanced reactors. It has since been expanded to include a four-equation drift flux model for the modeling of two-phase flows containing a noncondensable gas. The model was expanded to support the modeling of molten salt reactor (MSR) designs in which the fuel is directly dissolved in the circulating coolant. These designs have shown that circulating gas bubbles can play an important role in the management of fission products and the operational behavior of the reactor. A drift flux model was implemented to more accurately capture the localized behavior of the void in the core and its impact on the mass transfer of fission products. A thorough assessment of the new model was performed by developing a verification and validation test suite. Verification problems were designed to test all major terms in the new governing equations. The new model converged to the correct solution at the expected order of accuracy for all verification cases. The validation cases included a wide range of flow and void conditions in different pipe geometries. Although higher void experiments show a slight underprediction of void by the drift flux model, experiments that aim to reproduce Molten Salt Reactor Experiment (MSRE) experimental conditions show good agreement with the model. The gas transport model was activated for a SAM model of the MSRE to demonstrate that it can be used in a more complex model. Finally, this gas transport model will be used along with an interfacial area transport equation being implemented in SAM for the prediction of mass transport behavior in MSR conditions.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Program Verification for Extreme-Scale Applications (Final Scientific/Technical Report)

The project seeks to develop tools and techniques to help software developers verify the correctness and accuracy of their code. The target domain is scientific software of the kind widely used and developed in the Department of Energy research community, with a particular focus on "extreme scale" programs - those that are expected to involve possibly millions of parallel threads of execution. The report covers the University of Delaware contribution to the collaborative project. The project had a number of successful outcomes, especially regarding the development and extension of the CIVL software verification framework. CIVL is a verification tool for C or Fortran programs that use MPI, OpenMP, CUDA, and/or Pthreads for parallelization. CIVL went through vast improvements and extensions, and was successfully applied to a number of challenging codes. It found a subtle bug in the Devito PDE framework. It was able to verify the functional correctness of a conjugate gradient solver using a novel probabilistic technique with vanishingly small chance of error. CIVL was used very successfully in verification competitions, and the CIVL solutions to the competition challenges were presented and published. CIVL was also used successfully by other researchers on a computational chemistry kernel.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Supplement to M&V Guidelines: Measurement and Verification for Performance-Based Contracts Version 4.0

Measurement and verification (M&V) are key to determining and confirming the continued operation of, and savings associated with, performance-based contracts. The Federal Energy Management Program (FEMP) updates these guidelines to reflect the evolution of M&V to incorporate industry best practices and ensure that savings from these performance-based projects are properly planned, communicated, and realized. FEMP released M&V Guidelines: Measurement and Verification for Federal Energy Projects Version 3.0 in 2008. This document was developed with input from industry-government working groups, previous M&V guideline authors, and national laboratory subject matter experts. The document provided the methods and guidelines, with a focus on federal energy savings performance contracts (ESPCs). When FEMP released M&V Guidelines: Measurement and Verification for Performance-Based Contracts Version 4.0 in 2015, FEMP expanded the intended audience to include anyone (energy managers, procurement officers, and contractors) involved in implementing performance-based contracts. The streamlined M&V Guidelines Version 4.0 provided the procedures and guidelines needed to quantify savings resulting from installing energy-efficient equipment, water conservation, improved operation and maintenance (O&M), renewable energy, and cogeneration projects with a performance-based contract; however, it removed some of the associated details on topics such as developing regression models and sampling plans. The details and information provided in M&V Guidelines v. 3.0 are still pertinent to M&V for performance-based contracts. The objective is to keep FEMP M&V version 4.0 condensed, providing the reference document for specifying M&V methods and procedures and a resource for developing project-specific M&V plans; this supplement provides additional details. The content of this supplement has been updated to include pertinent revisions between M&V version 3.0 and M&V version 4.0 by providing details and updating the content to ensure relevancy.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRS2003

Trends in software verification

steady increase in computational power of general purpose computers, our ability to analyze routine software artifacts is also steadily increasing. As a result, we are witnessing a shift in emphasis from the verification of abstract hand-built models of code, towards the direct verification of implementation level code. This change in emphasis poses a new set of challenges in software verification. We explore some of them in this paper.

software verification↗
NASA NTRS2006

Validation (not just verification) of Deep Space Missions

ion & Validation (V&V) is a widely recognized and critical systems engineering function. However, the often used definition 'Verification proves the design is right; validation proves it is the right design' is rather vague. And while Verification is a reasonably well standardized systems engineering process, Validation is a far more abstract concept and the rigor and scope applied to it varies widely between organizations and individuals. This is reflected in the findings in recent Mishap Reports for several NASA missions, in which shortfalls in Validation (not just Verification) were cited as root- or contributing-factors in catastrophic mission loss. Furthermore, although there is strong agreement in the community that Test is the preferred method for V&V, many people equate 'V&V' with 'Test', such that Analysis and Modeling aren't given comparable attention. Another strong motivator is a realization that the rapid growth in complexity of deep-space missions (particularly Planetary Landers and Space Observatories given their inherent unknowns) is placing greater demands on systems engineers to 'get it right' with Validation.

verification↗
NASA NTRS2014

Toward Automatic Verification of Goal-Oriented Flow Simulations

We demonstrate the power of adaptive mesh refinement with adjoint-based error estimates in verification of simulations governed by the steady Euler equations. The flow equations are discretized using a finite volume scheme on a Cartesian mesh with cut cells at the wall boundaries. The discretization error in selected simulation outputs is estimated using the method of adjoint-weighted residuals. Practical aspects of the implementation are emphasized, particularly in the formulation of the refinement criterion and the mesh adaptation strategy. Following a thorough code verification example, we demonstrate simulation verification of two- and three-dimensional problems. These involve an airfoil performance database, a pressure signature of a body in supersonic flow and a launch abort with strong jet interactions. The results show reliable estimates and automatic control of discretization error in all simulations at an affordable computational cost. Moreover, the approach remains effective even when theoretical assumptions, e.g., steady-state and solution smoothness, are relaxed.

Simulations↗
NASA NTRS2017

Verification Testing: Meet User Needs Figure of Merit

Verification is the process through which Modeling and Simulation(M&S) software goes to ensure that it has been rigorously tested and debugged for its intended use. Validation confirms that said software accurately models and represents the real world system. Credibility gives an assessment of the development and testing effort that the software has gone through as well as how accurate and reliable test results are. Together, these three components form Verification, Validation, and Credibility(VV&C), the process by which all NASA modeling software is to be tested to ensure that it is ready for implementation. NASA created this process following the CAIB (Columbia Accident Investigation Board) report seeking to understand the reasons the Columbia space shuttle failed during reentry. The reports conclusion was that the accident was fully avoidable, however, among other issues, the necessary data to make an informed decision was not there and the result was complete loss of the shuttle and crew. In an effort to mitigate this problem, NASA put out their Standard for Models and Simulations, currently in version NASA-STD-7009A, in which they detailed their recommendations, requirements and rationale for the different components of VV&C. They did this with the intention that it would allow for people receiving MS software to clearly understand and have data from the past development effort. This in turn would allow the people who had not worked with the MS software before to move forward with greater confidence and efficiency in their work. This particular project looks to perform Verification on several MATLAB (Registered Trademark)(The MathWorks, Inc.) scripts that will be later implemented in a website interface. It seeks to take note and define the limits of operation, the units and significance, and the expected datatype and format of the inputs and outputs of each of the scripts. This is intended to prevent the code from attempting to make incorrect or impossible calculations. Additionally, this project will look at the coding generally and note inconsistencies, redundancies, and other aspects that may become problematic or slow down the codes run time. Certain scripts lacking in documentation also will be commented and cataloged.

validation↗
NASA NTRS2017

MESA: Message-Based System Analysis Using Runtime Verification

In this paper, we present a novel approach and framework for run-time verication of large, safety critical messaging systems. This work was motivated by verifying the System Wide Information Management (SWIM) project of the Federal Aviation Administration (FAA). SWIM provides live air traffic, site and weather data streams for the whole National Airspace System (NAS), which can easily amount to several hundred messages per second. Such safety critical systems cannot be instrumented, therefore, verification and monitoring has to happen using a nonintrusive approach, by connecting to a variety of network interfaces. Due to a large number of potential properties to check, the verification framework needs to support efficient formulation of properties with a suitable Domain Specific Language (DSL). Our approach is to utilize a distributed system that is geared towards connectivity and scalability and interface it at the message queue level to a powerful verification engine. We implemented our approach in the tool called MESA: Message-Based System Analysis, which leverages the open source projects RACE (Runtime for Airspace Concept Evaluation) and TraceContract. RACE is a platform for instantiating and running highly concurrent and distributed systems and enables connectivity to SWIM and scalability. TraceContract is a runtime verication tool that allows for checking traces against properties specified in a powerful DSL. We applied our approach to verify a SWIM service against several requirements.We found errors such as duplicate and out-of-order messages.

Message-based System↗
NASA NTRS2017

Thermal Protection for Mars Sample Return Earth Entry Vehicle: A Grand Challenge for Design Methodology and Reliability Verification

Mars Sample Return is our Grand Challenge for the coming decade. TPS (Thermal Protection System) nominal performance is not the key challenge. The main difficulty for designers is the need to verify unprecedented reliability for the entry system: current guidelines for prevention of backward contamination require that the probability of spores larger than 1 micron diameter escaping into the Earth environment be lower than 1 million for the entire system, and the allocation to TPS would be more stringent than that. For reference, the reliability allocation for Orion TPS is closer to 11000, and the demonstrated reliability for previous human Earth return systems was closer to 1100. Improving reliability by more than 3 orders of magnitude is a grand challenge indeed. The TPS community must embrace the possibility of new architectures that are focused on reliability above thermal performance and mass efficiency. MSR (Mars Sample Return) EEV (Earth Entry Vehicle) will be hit with MMOD (Micrometeoroid and Orbital Debris) prior to reentry. A chute-less aero-shell design which allows for self-righting shape was baselined in prior MSR studies, with the assumption that a passive system will maximize EEV robustness. Hence the aero-shell along with the TPS has to take ground impact and not break apart. System verification will require testing to establish ablative performance and thermal failure but also testing of damage from MMOD, and structural performance at ground impact. Mission requirements will demand analysis, testing and verification that are focused on establishing reliability of the design. In this proposed talk, we will focus on the grand challenge of MSR EEV TPS and the need for innovative approaches to address challenges in modeling, testing, manufacturing and verification.

Design and Verification for Reliability↗
NASA NTRSDate not supplied

Verification and Validation of Safety-Critical Aircraft Systems Operating under Off-Nominal, Contingency, and Emergency Conditions

Verification and validation (V&V) of safety-critical technologies developed for loss of control (LOC) prevention and recovery and other aviation safety concerns pose significant challenges. Aircraft LOC can result from a wide spectrum of hazards, often occurring in combination, which cannot be fully replicated during evaluation. Technologies developed for LOC prevention and recovery must therefore be effective under a wide variety of hazardous and uncertain conditions, and the verification and validation of these technologies must provide some measure of assurance that the new vehicle safety technologies do no harm (i.e., that they themselves do not introduce new safety risks). V&V technologies must also enable the identification of system limitations and constraints, as well as enable the identification of safe and unsafe operating conditions (and their boundaries). Additionally, the V&V of complex, increasingly autonomous systems is a fundamental concern. Scalable, reproducible and cost-effective techniques for the assurance of safety critical systems during their design and operation is a key barrier to fielding new systems or updating current systems. Moreover, these techniques need to provide artifacts that enable a comprehensive evidence-based approach to certification. This briefing summarizes research performed under NASA’s Aviation Safety Program and follow-on research for the V&V of safety-critical aircraft system technologies developed for LOC prevention and recovery and increasingly autonomous systems, and for a broad assurance capability in both current and emerging aviation applications. Note that, in this briefing, the term “validation” refers to a confirmation that the system implementation (e.g., algorithms etc.) is performing the intended function(s), as well as an affirmation of effectiveness in these functions. “Verification” refers to a confirmation that the system implementation in the software and hardware meets its (hopefully validated) specifications (e.g., correctly executes algorithms as designed).

Validation↗
DOE OSTI2024

VALIDATION, VERIFICATION, AND CALIBRATION THROUGH A CAUSAL LENS

This paper presents an alternative method based on causal inference to perform validation, verification, and calibration of simulation models. While classical validation and verification approaches focus on the identification of the associations between data elements using statistical and machine learning methods, the novel methods in this paper focus instead on the identification of causal relationships between data elements. Statistical and machine learning-based approaches are strictly data-driven, meaning that they provide quantitative comparison measures between datasets without explicitly considering the hypotheses behind them. This can lead to the erroneous conclusion that, if two data sets are close enough, then the models that generated them are similar. In addition, when experimental and simulated data differ to an extent that fails to meet the acceptance criteria, calibration techniques are used to tweak simulation model parameters to reduce the gap between the two types of data. This produces the false expectation that a simulation model will match reality. The methods presented in this paper move away from these strictly data-driven methods for validation and calibration toward more robust, model-driven methods based on causal inference. Causal inference aims to identify the possible mechanisms that might have generated data. Thus, this analysis targets the prediction of the effects when one (or more) of the identified mechanisms are altered. There are many approaches to identify, quantify and illustrate causal relationships. For the scope of this paper, directed graphs are employed as causal models. If the directed graph lacks cycles it is known as a directed acyclic graph (DAG). A node in such a graph represents an observed data element while a directed edge connecting two nodes represents a causal relationship between two variables. The developed causal methods are designed to extract causal models from simulation models and from experimental data. Causal models capture the causal relationships between data elements (e.g., simulated and experimental data). In this context, validation and verification are performed by comparing causal models. The proposed approach does not only inform system analysts on how a simulation model matches real-world data, but also identifies elements of the simulation model that should be revised when discrepancies between simulation and experimental data are observed. Through these causal methods, analysts have a means to identify the portion of the model equation(s) that are behind an edge connecting two variables. Hence, once the structural differences between causal models have been determined, model calibration can occur by changing only those model parameters that impact the identified causal relationships.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Summary Report Of The FY25 Computational Fluid Dynamics Verification And Validation Exercises In The Advanced Reactor Technologies - Gas-cooled Reactor Program

Verification and Validation (V&V) of numerical tools is critical for ensuring reasonable predictions during design, safety analysis, and licensing. Recent work in the Advanced Reactor Technologies - Gas-cooled Reactor (ART-GCR) program has focused on V&V of common Computational Fluid Dynamics (CFD) tools that are used within the Untied States. This report presents an update on these CFD V&V activities. These Generation IV Forum (GIF) Very High Temperature Reactor (VHTR) Computational Methods, Validation, and Benchmarking (CMVB) is an international organization dedicated to the verification and validation of High Temperature Gas-Cooled Reactor (HTGR) simulation tools. Participation in the CMVB provides additional value to the V&V activities, as it allows for access to a wider range of data, and provides valuable benchmarking exercises. Three HTGR phenomena are targeted: Reactor Cavity Cooling System (RCCS) performance, core bypass flow, and lower plenum mixing. Simulations of the University of Wisconsin-Madison (UW-Madison) RCCS facilities are performed with Reynolds Averaged Navier-Stokes (RANS) in StarCCM+. Results are compared for both forced and natural convection conditions, with both exhibiting good agreement with experimental measurements. The Idaho National Laboratory (INL) matched index of refraction (MIR) and Korean Atomic Energy Research Institute (KAERI) bypass flow expeirments are used to validation CFD predictions of bypass flow. Simulations are performed with RANS in StarCCM+ and with Large Eddy Simulation (LES) in NekRS. Finally, preliminary simulations of the Institute of Nuclear and New Energy Technology (INET) lower plenum mixing facilities are presented. Initial work has developed models with LES, RANS, and porous media models. These preliminary models are presented and compared to each other to gauge differences in predictions with each of the three methods.

and Benchmarking (CMVB)↗
DOE OSTI2022

Formal verification and validation of run-to-completion style state charts using Event-B

State chart notations with ‘run to completion’ semantics are popular with engineers for designing controllers that react to environment events with a sequence of state transitions but lack formal refinement and rigorous verification methods. State chart models are typically used to design complex control systems that respond to environmental triggers with a sequential process. The model is usually constructed at a concrete level and verified and validated using animation techniques relying on human judgement. Event-B, on the other hand, is based on refinement from an initial abstraction and is designed to make formal verification by automatic theorem provers feasible. Abstraction and formal verification provide greater assurance that critical (e.g. safety or security) properties are not violated by the control system. In this paper, we introduce a notion of refinement into a ‘run to completion’ state chart modelling notation and leverage Event-B’s tool support for theorem proving. We describe the difficulties in translating ‘run to completion’ semantics into Event-B refinements and suggest a solution. We illustrate our approach and show how models can be validated at different refinement levels using our scenario checker animation tools. We show how critical invariant properties can be verified by proof despite the reactive nature of the system and how behavioural aspects of the system can be verified by testing the expected reactions using a temporal logic, model checking approach. To verify liveness, we outline a proof that the run to completion is deadlock-free and converges to complete the run.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Correction and verification of x-ray imaging crystal spectrometer analysis on Wendelstein 7-X through x-ray ray tracing

X-ray ray tracing is used to develop ion-temperature corrections for the analysis of the X-ray Imaging Crystal Spectrometer (XICS) used at Wendelstein 7-X (W7-X) and perform verification on the analysis methods. The XICS is a powerful diagnostic able to measure ion-temperature, electron-temperature, plasma flow, and impurity charge state densities. While these systems are relatively simple in design, accurate characterization of the instrumental response and validation of analysis techniques are difficult to perform experimentally due to the requirement of extended x-ray sources. For this reason, a ray tracing model has been developed that allows characterization of the spectrometer and verification of the analysis methods while fully considering the real geometry of the XICS system and W7-X plasma. Through the use of ray tracing, several important corrections have been found that must be accounted for in order to accurately reconstruct the ion-temperature profiles. The sources of these corrections are described along with their effect on the analyzed profiles. The implemented corrections stem from three effects: (1) effect of sub-pixel intensity distribution during de-curving and spatial binning, (2) effect of sub-pixel intensity distribution during forward model evaluation and generation of residuals, and (3) effect of defocus and spherical aberrations on the instrumental response. Possible improvements to the forward model and analysis procedures are explored, along with a discussion of trade-offs in terms of computational complexity. Finally, the accuracy of the tomographic inversion technique in stellarator geometry is investigated, providing for the first time a verification exercise for inversion accuracy in stellarator geometry and a complete XICS analysis tool-chain.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Verification methods for drift–diffusion reaction models for plasma simulations

Abstract Compared to other computational physics areas such as codes for general computational fluid dynamics, the documentation of verification methods for plasma fluid codes remains under developed. Current analytical solutions for plasma are often highly limited in terms of testing highly coupled physics, due to the harsh assumptions needed to derive even simple plasma equations. This work highlights these limitations, suggesting the method of manufactured solutions (MMSs) as a potential option for future verification efforts. To demonstrate the flexibility of MMS in verifying these highly coupled systems, the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework was utilized. Thanks to the MOOSE framework’s robustness and modularity, as well as to its physics module capabilities and ecosystem applications (i.e. Zapdos and the chemical reaction network) developed for plasma physics modeling and simulation, this report lays the groundwork for a structured method of conducting plasma fluid code verification.

DeChant, Corey (ORCID:0000000293779732)↗
DOE OSTI2020

Local Truncation Error-Informed Code Verification

The method of manufactured solutions (MMS) has become increasingly popular in conducting code verification studies on predictive codes, such as nuclear power system codes and computational fluid dynamic codes. The reason for the popularity of this approach is that it can be used when an analytical solution is not available. Using MMS, code developers are able to verify that their code is free of coding errors that impact the observed order of accuracy. While MMS is still an excellent tool for code verification, it does not identify coding errors that are of the same order as the numerical method. This paper presents a method that combines MMS with modified equation analysis (MEA), which calculates the local truncation error (LTE) to identify coding error up to and including the order of the numerical method. This method is referred to as modified equation analysis methd of manufactured solutions (MEAMMS). MEAMMS is then applied to a custom-built code, which solves the shallow water equations, to test the performance of the code verification method. MEAMMS is able to detect all coding errors that impact the implementation of the numerical scheme. To show how MEAMMS is different than MMS, they are both applied to the same first-order numerical method test problem with a first-order coding error. When there are first-order coding errors, only MEAMMS is able to identify them. Finally, this shows that MEAMMS is able to identify a larger set of coding errors while still being able to identify the coding errors MMS is able to identify.

42 ENGINEERING↗
DOE OSTI2022

Verification of a specialized hydrodynamic simulation code for modeling deflagration and detonation of high explosives

A specialized hydrodynamic simulation code has been developed and verified for the simulation of one-dimensional unsteady problems involving the detonation and deflagration of high explosives. To model all the relevant physical processes in these problems, a code is required to simulate compressible hydrodynamics, unsteady thermal conduction, and chemical reactions with complex rate laws. Several verification exercises are presented which test the implementation of these capabilities. The code also requires models for physics processes such as equations of state and conductivity for pure materials and mixtures as well as rate laws for chemical reactions. Additional verification tests are required to ensure that these models are implemented correctly. Though this code is limited in the types of problems it can simulate, its computationally efficient formulation allows it to be used in calibration studies for reactive burn models for high explosives. Furthermore, this study demonstrates how a series of verification tests can be used to ensure that the various physics processes needed to simulate complex phenomenon can be tested to ensure that they are correctly implemented.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Successive Procedure for Solution Verification Based on User Needs

This paper discusses a revised solution verification procedure for computational fluid dynamics simulations to estimate the uncertainties in the quantities of interest based on discretization error models. This proposed procedure builds upon current procedures described in ASME V&V 20 but provides more guidance in determining the necessary number of mesh levels to build reliable discretization error models. Such guidance is particularly useful for practicing engineers without prior experience in solution verification. The key features of this proposed solution verification procedure are the ability to determine the need for additional mesh levels iteratively and the seamless treatment for underdetermined, exact, and overdetermined solutions of the power series approximation to the discretization error models. This study applies the proposed procedure to a set of synthetic examples to demonstrate the revised procedure’s clarity in determining the number of mesh solutions required for a reliable estimate of the discretization error in computational fluid dynamics settings. Additionally, this proposed procedure prevents a potential pathway in the current procedure in ASME V&V 20 that may lead to unreasonably small discretization errors.

Weinmeister, Justin↗