Search NASA⌕ Search

SEARCH · Search NASA

Results for “VERIFICATION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Formal verification and validation of run-to-completion style state charts using Event-B

State chart notations with ‘run to completion’ semantics are popular with engineers for designing controllers that react to environment events with a sequence of state transitions but lack formal refinement and rigorous verification methods. State chart models are typically used to design complex control systems that respond to environmental triggers with a sequential process. The model is usually constructed at a concrete level and verified and validated using animation techniques relying on human judgement. Event-B, on the other hand, is based on refinement from an initial abstraction and is designed to make formal verification by automatic theorem provers feasible. Abstraction and formal verification provide greater assurance that critical (e.g. safety or security) properties are not violated by the control system. In this paper, we introduce a notion of refinement into a ‘run to completion’ state chart modelling notation and leverage Event-B’s tool support for theorem proving. We describe the difficulties in translating ‘run to completion’ semantics into Event-B refinements and suggest a solution. We illustrate our approach and show how models can be validated at different refinement levels using our scenario checker animation tools. We show how critical invariant properties can be verified by proof despite the reactive nature of the system and how behavioural aspects of the system can be verified by testing the expected reactions using a temporal logic, model checking approach. To verify liveness, we outline a proof that the run to completion is deadlock-free and converges to complete the run.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Correction and verification of x-ray imaging crystal spectrometer analysis on Wendelstein 7-X through x-ray ray tracing

X-ray ray tracing is used to develop ion-temperature corrections for the analysis of the X-ray Imaging Crystal Spectrometer (XICS) used at Wendelstein 7-X (W7-X) and perform verification on the analysis methods. The XICS is a powerful diagnostic able to measure ion-temperature, electron-temperature, plasma flow, and impurity charge state densities. While these systems are relatively simple in design, accurate characterization of the instrumental response and validation of analysis techniques are difficult to perform experimentally due to the requirement of extended x-ray sources. For this reason, a ray tracing model has been developed that allows characterization of the spectrometer and verification of the analysis methods while fully considering the real geometry of the XICS system and W7-X plasma. Through the use of ray tracing, several important corrections have been found that must be accounted for in order to accurately reconstruct the ion-temperature profiles. The sources of these corrections are described along with their effect on the analyzed profiles. The implemented corrections stem from three effects: (1) effect of sub-pixel intensity distribution during de-curving and spatial binning, (2) effect of sub-pixel intensity distribution during forward model evaluation and generation of residuals, and (3) effect of defocus and spherical aberrations on the instrumental response. Possible improvements to the forward model and analysis procedures are explored, along with a discussion of trade-offs in terms of computational complexity. Finally, the accuracy of the tomographic inversion technique in stellarator geometry is investigated, providing for the first time a verification exercise for inversion accuracy in stellarator geometry and a complete XICS analysis tool-chain.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Verification methods for drift–diffusion reaction models for plasma simulations

Abstract Compared to other computational physics areas such as codes for general computational fluid dynamics, the documentation of verification methods for plasma fluid codes remains under developed. Current analytical solutions for plasma are often highly limited in terms of testing highly coupled physics, due to the harsh assumptions needed to derive even simple plasma equations. This work highlights these limitations, suggesting the method of manufactured solutions (MMSs) as a potential option for future verification efforts. To demonstrate the flexibility of MMS in verifying these highly coupled systems, the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework was utilized. Thanks to the MOOSE framework’s robustness and modularity, as well as to its physics module capabilities and ecosystem applications (i.e. Zapdos and the chemical reaction network) developed for plasma physics modeling and simulation, this report lays the groundwork for a structured method of conducting plasma fluid code verification.

DeChant, Corey (ORCID:0000000293779732)↗
DOE OSTI2020

Local Truncation Error-Informed Code Verification

The method of manufactured solutions (MMS) has become increasingly popular in conducting code verification studies on predictive codes, such as nuclear power system codes and computational fluid dynamic codes. The reason for the popularity of this approach is that it can be used when an analytical solution is not available. Using MMS, code developers are able to verify that their code is free of coding errors that impact the observed order of accuracy. While MMS is still an excellent tool for code verification, it does not identify coding errors that are of the same order as the numerical method. This paper presents a method that combines MMS with modified equation analysis (MEA), which calculates the local truncation error (LTE) to identify coding error up to and including the order of the numerical method. This method is referred to as modified equation analysis methd of manufactured solutions (MEAMMS). MEAMMS is then applied to a custom-built code, which solves the shallow water equations, to test the performance of the code verification method. MEAMMS is able to detect all coding errors that impact the implementation of the numerical scheme. To show how MEAMMS is different than MMS, they are both applied to the same first-order numerical method test problem with a first-order coding error. When there are first-order coding errors, only MEAMMS is able to identify them. Finally, this shows that MEAMMS is able to identify a larger set of coding errors while still being able to identify the coding errors MMS is able to identify.

42 ENGINEERING↗
DOE OSTI2022

Verification of a specialized hydrodynamic simulation code for modeling deflagration and detonation of high explosives

A specialized hydrodynamic simulation code has been developed and verified for the simulation of one-dimensional unsteady problems involving the detonation and deflagration of high explosives. To model all the relevant physical processes in these problems, a code is required to simulate compressible hydrodynamics, unsteady thermal conduction, and chemical reactions with complex rate laws. Several verification exercises are presented which test the implementation of these capabilities. The code also requires models for physics processes such as equations of state and conductivity for pure materials and mixtures as well as rate laws for chemical reactions. Additional verification tests are required to ensure that these models are implemented correctly. Though this code is limited in the types of problems it can simulate, its computationally efficient formulation allows it to be used in calibration studies for reactive burn models for high explosives. Furthermore, this study demonstrates how a series of verification tests can be used to ensure that the various physics processes needed to simulate complex phenomenon can be tested to ensure that they are correctly implemented.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Successive Procedure for Solution Verification Based on User Needs

This paper discusses a revised solution verification procedure for computational fluid dynamics simulations to estimate the uncertainties in the quantities of interest based on discretization error models. This proposed procedure builds upon current procedures described in ASME V&V 20 but provides more guidance in determining the necessary number of mesh levels to build reliable discretization error models. Such guidance is particularly useful for practicing engineers without prior experience in solution verification. The key features of this proposed solution verification procedure are the ability to determine the need for additional mesh levels iteratively and the seamless treatment for underdetermined, exact, and overdetermined solutions of the power series approximation to the discretization error models. This study applies the proposed procedure to a set of synthetic examples to demonstrate the revised procedure’s clarity in determining the number of mesh solutions required for a reliable estimate of the discretization error in computational fluid dynamics settings. Additionally, this proposed procedure prevents a potential pathway in the current procedure in ASME V&V 20 that may lead to unreasonably small discretization errors.

Weinmeister, Justin↗
DOE OSTI2025

Using a Large Language Model as a Building Block to Generate Usable Validation and Verification Suite for OpenMP

In the HPC area, both hardware and software move quickly. Often new hardware is developed and deployed, the corresponding software stack, including compilers and other tools, are under active development while leading edge software developers are working to port and tune their applications, all at the same time. While the software ecosystem is in flux, one of the key challenges for users is obtaining insight into the state of implementation of key features in the programming languages and models their applications are using – whether they have been implemented, and whether the implementation conforms to the specification, especially for newly implemented features (less tested by widespread use). OpenMP is one of the most prominent shared memory programming models used for on-node programming in HPC. With the shift towards accelerators (such as GPUs and FPGAs) and heterogeneous programming OpenMP features are getting more complex. It is natural to ask whether generative AI approaches, and large language models (LLMs) in particular, can help in producing validation and verification test suites to allow users better and faster insights into the availability and correctness of OpenMP features of interest. In this work, we explore the use of ChatGPT-4 to generate a suite of tests for OpenMP features. We have chosen a set of directives and clauses, a total of 78 combinations, which first appeared in OpenMP 3.0 (released in May 2008) but are also relevant for accelerators. We prompted ChatGPT to generate tests in the C and Fortran languages, for both host (CPU) and device (accelerator). On the Summit super-computer using the GNU implementation, we found that, of the 78 generated tests 67 C tests and 43 Fortran tests compiled successfully and fewer than those executed to completion. On further analysis we show that not all generated tests are valid. We document the process, results, and provide detailed analysis regarding the quality of tests generated. With the aim of providing input to a production quality validation and verification suite, we manually implement the corrections required to make the tests valid according to the current OpenMP specification. We quantify this effort as small, medium, or large, and record the lines of code changed to correct the invalid tests. With the corrected tests we validate recent implementations from HPE, AMD, and GNU on the Frontier supercomputer. Our experiment and subsequent analysis show that although LLMs are capable of producing HPC specific codes, they are limited by their understanding of the deeper semantics and restrictions of programming models such as OpenMP. Unsurprisingly more commonly used features have better support, while some OpenMP 3.0 directives such as sections and tasking are not universally supported on accelerators. We demonstrate that successful compilation and execution to completion are inadequate metrics for evaluating generated code and that, at this time, commodity LLMs require expert intervention for code verification. This points to gaps in the training data that is currently available for HPC. We demonstrate that with "small" effort 37% of generated invalid C tests and 63% of generated invalid Fortran tests could be corrected. This improves productivity of test generation as we circumvent writing from scratch and the common programming errors associated with it.

Pophale, Swaroop [ORNL] (ORCID:0000000185446367)↗
DOE OSTI2024

Calibration verification for stochastic agent-based disease spread models

Accurate disease spread modeling is crucial for identifying the severity of outbreaks and planning effective mitigation efforts. To be reliable when applied to new outbreaks, model calibration techniques must be robust. However, current methods frequently forgo calibration verification (a stand-alone process evaluating the calibration procedure) and instead use overall model validation (a process comparing calibrated model results to data) to check calibration processes, which may conceal errors in calibration. In this work, we develop a stochastic agent-based disease spread model to act as a testing environment as we test two calibration methods using simulation-based calibration, which is a synthetic data calibration verification method. The first calibration method is a Bayesian inference approach using an empirically-constructed likelihood and Markov chain Monte Carlo (MCMC) sampling, while the second method is a likelihood-free approach using approximate Bayesian computation (ABC). Simulation-based calibration suggests that there are challenges with the empirical likelihood calculation used in the first calibration method in this context. These issues are alleviated in the ABC approach. Despite these challenges, we note that the first calibration method performs well in a synthetic data model validation test similar to those common in disease spread modeling literature. We conclude that stand-alone calibration verification using synthetic data may benefit epidemiological researchers in identifying model calibration challenges that may be difficult to identify with other commonly used model validation techniques.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Verification and Validation of MFiX-PIC

This work is aimed at verification and validation (V&V) of particle-in-cell (PIC) approach in the National Energy Technology Laboratory’s (NETL) Multiphase Flow with Interphase eXchanges (MFiX-PIC) which was officially released in April 2019. Under verification, the results from MFiX-PIC are compared with exact or analytical solutions, while experimental results are used for benchmarking under validation. The test suite consists of the following cases: Terminal velocity. Particle advection in time-varying flow field. Particle settling in a dense medium. Evaporation. Rayleigh-Taylor instability. Minimum fluidization. Mini-circulating fluidized bed. The cases reported in a format consistent with the existing Verification and Validation Manual (Musser et al., 2018) explore a wide range of flow velocities and concentration regimes. Only one case considered in this work verifies the implementation of heat and mass transfer routines in MFiX-PIC, while a few other studies are currently underway to further expand the validation space in reacting flows. This study highlights the need for a rigorous analysis to objectively quantify the effect of model parameters. In addition, there is a scarcity of high-quality experimental data in the literature suitable for validating PIC methodology. Future efforts would include performing experiments which provide results for a valuable database. While designing such experiments, particle count should be significant to have sufficient particles per parcel while maintaining an optimal parcel to cell volume ratio. Also, datasets should have objectively assessed experimental uncertainty whereby confidence interval in those measurements become more explicit.

42 ENGINEERING↗
DOE OSTI2020

Shell Element Material Model Verification Problems for DYNA3D: Part II

A suite of supplementary shell element material model verification tests were developed for the explicit finite element program DYNA3D, in continuation of the work performed in LLNL-TR-792469 (Shell Element Material Model Verification Problems for DYNA3D). The testing procedure developed in the preceding report is extended, and used to verify all remaining untested shell model features and inputs. A collection of 78 feature-specific verification tests are proposed, some of which are applicable to multiple material models, though many are specialized to a particular model. The collective suite of tests cover all 27 currently available material models in DYNA3D. In the course of developing the proposed test suite, 24 separate shell model-related issues were identified and have subsequently been resolved.

42 ENGINEERING↗
DOE OSTI2021

24.1.3.1 - Verification of Spent Fuel Inside Dry Storage Casks by Fast/Epi-thermal Neutron Mapping

Development of a spent-fuel verification method based upon positional measurement of fast neutrons emitted from the top surface of the dry storage casks. A proof-of-principle field test on a partially loaded spent-fuel cask will be conducted by measuring the fast neutron flux on the top of the cask as a function of position using a single detector. If a useful verification signature is demonstrated, a multi-detector scanning array will be built and tested in FY2021 to demonstrate a practical rapid verification system in FY2022.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

Verification and Validation of a Modified Numerical Algorithm for Simulation of Transient Unconfined Groundwater Flow

This report extends verification and validation testing of the NUFT package of codes (Nitao 2000a,b) to the US1P module of code designed for simulation of single-phase mixed or “coupled” saturated and unsaturated (or “variably saturated”) groundwater flow. Importantly, the verification and validation testing of US1P in this report includes performance evaluation of a modified numerical algorithm that was not included in prior quality assurance (QA) of the NUFT package by Carle et al. (2014) for Underground Test Area (UGTA) project activities, which are directed at assessment of radionuclide contamination in groundwater sourced from underground nuclear test locations at the Nevada National Security Site (NNSS). An immediate purpose of this report is to provide QA for ongoing large-scale, three-dimensional (3-D) groundwater flow modeling of transient water levels associated with longterm water supply pumping and underground nuclear testing at the NNSS (Jackson and Fenelon, 2018; Jackson et al., 2021). For UGTA activities, the QA refers to the standards of ASTM (1996) for verification and validation testing of groundwater modeling codes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Verification of Spent Fuel Inside Dry Storage Casks Using Neutron Mapping (2022 Midyear Report)

Development of a spent-fuel verification method based upon positional measurement of fast neutrons emitted from the top surface of the dry storage casks. A proof-of-principle field test on a partially loaded spent-fuel cask will be conducted by measuring the fast neutron flux on the top of the cask as a function of position using a single detector. If a useful verification signature is demonstrated, a multi-detector scanning array will be built and tested to demonstrate a practical rapid verification system.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Verification of Spent Fuel Inside Dry Storage Casks by Fast/Epi-thermal Neutron Mapping (FY2022 NA-241 Annual Report)

Development of a spent-fuel verification method based upon positional measurement of fast neutrons emitted from the top surface of the dry storage casks. A proof-of-principle field test on a partially loaded spent-fuel cask will be conducted by measuring the fast neutron flux on the top of the cask as a function of position using a single detector. If a useful verification signature is demonstrated, a multidetector scanning array will be built and tested to demonstrate a practical rapid verification system.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

The Impact of Increased Requirements for Verification Measurements of Category III and IV Receipts

As part of the regular nuclear material accounting and control process, sites that have accountable nuclear material regularly perform measurements of the material in their possession. The expectation for the type of measurements and their frequency is set out in Department of Energy (DOE) Order 474.2, and further detailed in each site’s respective material control and accountability (MC&A) plan and associated controlling documents. The measurements of interest in this report are those performed on external (between DOE sites) transfer receipts of Category III or IV accountable materials. In 2020, a revision of DOE Order 474.2 was proposed that would significantly increase the requirement that measurements be performed upon receipt of material, and further, that those measurements be quantitative verification (rather than qualitative confirmatory) measurements. A survey was performed of DOE sites to discuss the impact on their MC&A measurements program if more rigorous verification measurements requirements for Category III and IV receipts were implemented. An assessment of nondestructive assay (NDA) measurement instruments to meet those needs was also performed. Across the board, sites indicated that personnel and budget would cause the largest impact. Sites which are qualified as Category I/II indicated a reduced impact as these sites already have the measurement capabilities and some personnel but would require the acquisition of additional measurement systems. Category III/IV sites indicated a much larger impact as these sites generally have smaller MC&A programs. A few smaller sites stated a measurement program would need to be created to handle the demand. Calorimetry, quantitative gamma spectroscopy and neutron coincidence counting systems were identified as common systems that could meet the verification measurement requirements. The other, and perhaps larger issue is the need for calibration standards for the calorimetry and neutron coincidence counting systems. It is proposed that a comparative study be performed for ISOCS and ISOTOPICS, qualification of a COTS calorimetry system be performed, and development of an AWCC system that uses a neutron generator.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2023

Spring 2023 Verification Presentation to Headquarters [Slides]

Testing codes is important, and more tests are needed. Verification testing is different than software-quality assurance, like regression suites. Tests should be simple, but not too simple, and tests should be code agnostic. Developing verification methods is still open research. Testing single-physics code pieces in isolation may lead to bad results when they are coupled. Good verification tests must include multiple physics models, multiple materials, and be performed in multiple dimensions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Verification Testing of Body Forces due to a Prescribed Angular Velocity in DYNA3D/Paradyn

This report documents the verification testing and regression testing done on a new feature in DYNA3D/Paradyn, which allows users to prescribe body force loads based upon an angular velocity. This feature is unique in that the direction of the angular velocity vector follows the unit vector formed by two coordinate points associated with two nodes or the average coordinates of two separate small collection of nodes. The angular velocity direction will follow the directional vector defined by these nodes while the angular velocity magnitude is defined by a load curve. A simple single element verification test was performed to determine the correct implementation of this feature, and two separate regression tests were added to the DYNA3D Software Quality Assur ance test suite. The nodal positions, velocities, and accelerations from the solution of the single element verification test compare well to analytically derived values of those nodal quantities. The regression tests serve as good examples of this new feature’s use case and were consequently added to the SQA test suite to ensure that further modifications of the DYNA3D source code do not unintentionally change the generated baseline answers.

42 ENGINEERING↗
DOE OSTI2024

Income Verification Strategies for Income-Based Solar Programs

The Inflation Reduction Act has created substantial new programs that support adoption of solar power by low-income households, including the $7 billion Solar For All program and the Low-Income Communities Bonus Credit Program, which increases the investment tax credit for certain types of deployment. In addition, a growing number of states are using solar programs to reduce energy burdens and create energy justice opportunities for low-income households and disadvantaged communities. Verifying the income of participating customers is an important component of these programs. Program managers are seeking strategies to verify a large number of subscribing customers in an accurate, timely, and cost-efficient manner. To help inform program managers, Berkeley Lab investigated how a number of energy and non-energy programs manage income verification. The most common approach is to require proof through tax documents, pay stubs, or other formal income documentation, which can pose an impediment to enrolling eligible customers and create a paperwork burden for administrators. In order to reduce the burden for both the applicant and the program manager, some programs use alternative methods. We identify three common alternative verification methods: -Categorical eligibility: Customers enrolled in other, similar income-verified assistance programs are automatically eligible for enrollment in other income-qualified programs. -Geographic eligibility: Eligibility is based on the customer’s location within a specified area, typically a low-income or disadvantaged community or census tract, and; -“Self-attestation”: The participant claims eligibility with or without further documentation. We describe these options, their pros and cons, give examples of how they are used, and explore how some low-income programs address administrative issues, audits, or other quality control measures. Finally, we explore the risk of mistaken verifications (finding a participant eligible when they are not) in the different strategies. While this memo was initiated by a request relating to income-based community solar programs, the methods are applicable to any program with income eligibility requirements in the energy or non-energy sector. Funding was provided for this research by the Solar Energy Technologies Office of the US Department of Energy, through the National Community Solar Partnership.

14 SOLAR ENERGY↗