Search NASA⌕ Search

SEARCH · Search NASA

Results for “VERIFICATION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Using a Large Language Model as a Building Block to Generate Usable Validation and Verification Suite for OpenMP

In the HPC area, both hardware and software move quickly. Often new hardware is developed and deployed, the corresponding software stack, including compilers and other tools, are under active development while leading edge software developers are working to port and tune their applications, all at the same time. While the software ecosystem is in flux, one of the key challenges for users is obtaining insight into the state of implementation of key features in the programming languages and models their applications are using – whether they have been implemented, and whether the implementation conforms to the specification, especially for newly implemented features (less tested by widespread use). OpenMP is one of the most prominent shared memory programming models used for on-node programming in HPC. With the shift towards accelerators (such as GPUs and FPGAs) and heterogeneous programming OpenMP features are getting more complex. It is natural to ask whether generative AI approaches, and large language models (LLMs) in particular, can help in producing validation and verification test suites to allow users better and faster insights into the availability and correctness of OpenMP features of interest. In this work, we explore the use of ChatGPT-4 to generate a suite of tests for OpenMP features. We have chosen a set of directives and clauses, a total of 78 combinations, which first appeared in OpenMP 3.0 (released in May 2008) but are also relevant for accelerators. We prompted ChatGPT to generate tests in the C and Fortran languages, for both host (CPU) and device (accelerator). On the Summit super-computer using the GNU implementation, we found that, of the 78 generated tests 67 C tests and 43 Fortran tests compiled successfully and fewer than those executed to completion. On further analysis we show that not all generated tests are valid. We document the process, results, and provide detailed analysis regarding the quality of tests generated. With the aim of providing input to a production quality validation and verification suite, we manually implement the corrections required to make the tests valid according to the current OpenMP specification. We quantify this effort as small, medium, or large, and record the lines of code changed to correct the invalid tests. With the corrected tests we validate recent implementations from HPE, AMD, and GNU on the Frontier supercomputer. Our experiment and subsequent analysis show that although LLMs are capable of producing HPC specific codes, they are limited by their understanding of the deeper semantics and restrictions of programming models such as OpenMP. Unsurprisingly more commonly used features have better support, while some OpenMP 3.0 directives such as sections and tasking are not universally supported on accelerators. We demonstrate that successful compilation and execution to completion are inadequate metrics for evaluating generated code and that, at this time, commodity LLMs require expert intervention for code verification. This points to gaps in the training data that is currently available for HPC. We demonstrate that with "small" effort 37% of generated invalid C tests and 63% of generated invalid Fortran tests could be corrected. This improves productivity of test generation as we circumvent writing from scratch and the common programming errors associated with it.

Pophale, Swaroop [ORNL] (ORCID:0000000185446367)↗
DOE OSTI2024

Calibration verification for stochastic agent-based disease spread models

Accurate disease spread modeling is crucial for identifying the severity of outbreaks and planning effective mitigation efforts. To be reliable when applied to new outbreaks, model calibration techniques must be robust. However, current methods frequently forgo calibration verification (a stand-alone process evaluating the calibration procedure) and instead use overall model validation (a process comparing calibrated model results to data) to check calibration processes, which may conceal errors in calibration. In this work, we develop a stochastic agent-based disease spread model to act as a testing environment as we test two calibration methods using simulation-based calibration, which is a synthetic data calibration verification method. The first calibration method is a Bayesian inference approach using an empirically-constructed likelihood and Markov chain Monte Carlo (MCMC) sampling, while the second method is a likelihood-free approach using approximate Bayesian computation (ABC). Simulation-based calibration suggests that there are challenges with the empirical likelihood calculation used in the first calibration method in this context. These issues are alleviated in the ABC approach. Despite these challenges, we note that the first calibration method performs well in a synthetic data model validation test similar to those common in disease spread modeling literature. We conclude that stand-alone calibration verification using synthetic data may benefit epidemiological researchers in identifying model calibration challenges that may be difficult to identify with other commonly used model validation techniques.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Verification and Validation of MFiX-PIC

This work is aimed at verification and validation (V&V) of particle-in-cell (PIC) approach in the National Energy Technology Laboratory’s (NETL) Multiphase Flow with Interphase eXchanges (MFiX-PIC) which was officially released in April 2019. Under verification, the results from MFiX-PIC are compared with exact or analytical solutions, while experimental results are used for benchmarking under validation. The test suite consists of the following cases: Terminal velocity. Particle advection in time-varying flow field. Particle settling in a dense medium. Evaporation. Rayleigh-Taylor instability. Minimum fluidization. Mini-circulating fluidized bed. The cases reported in a format consistent with the existing Verification and Validation Manual (Musser et al., 2018) explore a wide range of flow velocities and concentration regimes. Only one case considered in this work verifies the implementation of heat and mass transfer routines in MFiX-PIC, while a few other studies are currently underway to further expand the validation space in reacting flows. This study highlights the need for a rigorous analysis to objectively quantify the effect of model parameters. In addition, there is a scarcity of high-quality experimental data in the literature suitable for validating PIC methodology. Future efforts would include performing experiments which provide results for a valuable database. While designing such experiments, particle count should be significant to have sufficient particles per parcel while maintaining an optimal parcel to cell volume ratio. Also, datasets should have objectively assessed experimental uncertainty whereby confidence interval in those measurements become more explicit.

42 ENGINEERING↗
DOE OSTI2020

Shell Element Material Model Verification Problems for DYNA3D: Part II

A suite of supplementary shell element material model verification tests were developed for the explicit finite element program DYNA3D, in continuation of the work performed in LLNL-TR-792469 (Shell Element Material Model Verification Problems for DYNA3D). The testing procedure developed in the preceding report is extended, and used to verify all remaining untested shell model features and inputs. A collection of 78 feature-specific verification tests are proposed, some of which are applicable to multiple material models, though many are specialized to a particular model. The collective suite of tests cover all 27 currently available material models in DYNA3D. In the course of developing the proposed test suite, 24 separate shell model-related issues were identified and have subsequently been resolved.

42 ENGINEERING↗
DOE OSTI2021

24.1.3.1 - Verification of Spent Fuel Inside Dry Storage Casks by Fast/Epi-thermal Neutron Mapping

Development of a spent-fuel verification method based upon positional measurement of fast neutrons emitted from the top surface of the dry storage casks. A proof-of-principle field test on a partially loaded spent-fuel cask will be conducted by measuring the fast neutron flux on the top of the cask as a function of position using a single detector. If a useful verification signature is demonstrated, a multi-detector scanning array will be built and tested in FY2021 to demonstrate a practical rapid verification system in FY2022.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

Verification and Validation of a Modified Numerical Algorithm for Simulation of Transient Unconfined Groundwater Flow

This report extends verification and validation testing of the NUFT package of codes (Nitao 2000a,b) to the US1P module of code designed for simulation of single-phase mixed or “coupled” saturated and unsaturated (or “variably saturated”) groundwater flow. Importantly, the verification and validation testing of US1P in this report includes performance evaluation of a modified numerical algorithm that was not included in prior quality assurance (QA) of the NUFT package by Carle et al. (2014) for Underground Test Area (UGTA) project activities, which are directed at assessment of radionuclide contamination in groundwater sourced from underground nuclear test locations at the Nevada National Security Site (NNSS). An immediate purpose of this report is to provide QA for ongoing large-scale, three-dimensional (3-D) groundwater flow modeling of transient water levels associated with longterm water supply pumping and underground nuclear testing at the NNSS (Jackson and Fenelon, 2018; Jackson et al., 2021). For UGTA activities, the QA refers to the standards of ASTM (1996) for verification and validation testing of groundwater modeling codes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Verification of Spent Fuel Inside Dry Storage Casks Using Neutron Mapping (2022 Midyear Report)

Development of a spent-fuel verification method based upon positional measurement of fast neutrons emitted from the top surface of the dry storage casks. A proof-of-principle field test on a partially loaded spent-fuel cask will be conducted by measuring the fast neutron flux on the top of the cask as a function of position using a single detector. If a useful verification signature is demonstrated, a multi-detector scanning array will be built and tested to demonstrate a practical rapid verification system.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Verification of Spent Fuel Inside Dry Storage Casks by Fast/Epi-thermal Neutron Mapping (FY2022 NA-241 Annual Report)

Development of a spent-fuel verification method based upon positional measurement of fast neutrons emitted from the top surface of the dry storage casks. A proof-of-principle field test on a partially loaded spent-fuel cask will be conducted by measuring the fast neutron flux on the top of the cask as a function of position using a single detector. If a useful verification signature is demonstrated, a multidetector scanning array will be built and tested to demonstrate a practical rapid verification system.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

The Impact of Increased Requirements for Verification Measurements of Category III and IV Receipts

As part of the regular nuclear material accounting and control process, sites that have accountable nuclear material regularly perform measurements of the material in their possession. The expectation for the type of measurements and their frequency is set out in Department of Energy (DOE) Order 474.2, and further detailed in each site’s respective material control and accountability (MC&A) plan and associated controlling documents. The measurements of interest in this report are those performed on external (between DOE sites) transfer receipts of Category III or IV accountable materials. In 2020, a revision of DOE Order 474.2 was proposed that would significantly increase the requirement that measurements be performed upon receipt of material, and further, that those measurements be quantitative verification (rather than qualitative confirmatory) measurements. A survey was performed of DOE sites to discuss the impact on their MC&A measurements program if more rigorous verification measurements requirements for Category III and IV receipts were implemented. An assessment of nondestructive assay (NDA) measurement instruments to meet those needs was also performed. Across the board, sites indicated that personnel and budget would cause the largest impact. Sites which are qualified as Category I/II indicated a reduced impact as these sites already have the measurement capabilities and some personnel but would require the acquisition of additional measurement systems. Category III/IV sites indicated a much larger impact as these sites generally have smaller MC&A programs. A few smaller sites stated a measurement program would need to be created to handle the demand. Calorimetry, quantitative gamma spectroscopy and neutron coincidence counting systems were identified as common systems that could meet the verification measurement requirements. The other, and perhaps larger issue is the need for calibration standards for the calorimetry and neutron coincidence counting systems. It is proposed that a comparative study be performed for ISOCS and ISOTOPICS, qualification of a COTS calorimetry system be performed, and development of an AWCC system that uses a neutron generator.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2023

Spring 2023 Verification Presentation to Headquarters [Slides]

Testing codes is important, and more tests are needed. Verification testing is different than software-quality assurance, like regression suites. Tests should be simple, but not too simple, and tests should be code agnostic. Developing verification methods is still open research. Testing single-physics code pieces in isolation may lead to bad results when they are coupled. Good verification tests must include multiple physics models, multiple materials, and be performed in multiple dimensions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Verification Testing of Body Forces due to a Prescribed Angular Velocity in DYNA3D/Paradyn

This report documents the verification testing and regression testing done on a new feature in DYNA3D/Paradyn, which allows users to prescribe body force loads based upon an angular velocity. This feature is unique in that the direction of the angular velocity vector follows the unit vector formed by two coordinate points associated with two nodes or the average coordinates of two separate small collection of nodes. The angular velocity direction will follow the directional vector defined by these nodes while the angular velocity magnitude is defined by a load curve. A simple single element verification test was performed to determine the correct implementation of this feature, and two separate regression tests were added to the DYNA3D Software Quality Assur ance test suite. The nodal positions, velocities, and accelerations from the solution of the single element verification test compare well to analytically derived values of those nodal quantities. The regression tests serve as good examples of this new feature’s use case and were consequently added to the SQA test suite to ensure that further modifications of the DYNA3D source code do not unintentionally change the generated baseline answers.

42 ENGINEERING↗
DOE OSTI2024

Income Verification Strategies for Income-Based Solar Programs

The Inflation Reduction Act has created substantial new programs that support adoption of solar power by low-income households, including the $7 billion Solar For All program and the Low-Income Communities Bonus Credit Program, which increases the investment tax credit for certain types of deployment. In addition, a growing number of states are using solar programs to reduce energy burdens and create energy justice opportunities for low-income households and disadvantaged communities. Verifying the income of participating customers is an important component of these programs. Program managers are seeking strategies to verify a large number of subscribing customers in an accurate, timely, and cost-efficient manner. To help inform program managers, Berkeley Lab investigated how a number of energy and non-energy programs manage income verification. The most common approach is to require proof through tax documents, pay stubs, or other formal income documentation, which can pose an impediment to enrolling eligible customers and create a paperwork burden for administrators. In order to reduce the burden for both the applicant and the program manager, some programs use alternative methods. We identify three common alternative verification methods: -Categorical eligibility: Customers enrolled in other, similar income-verified assistance programs are automatically eligible for enrollment in other income-qualified programs. -Geographic eligibility: Eligibility is based on the customer’s location within a specified area, typically a low-income or disadvantaged community or census tract, and; -“Self-attestation”: The participant claims eligibility with or without further documentation. We describe these options, their pros and cons, give examples of how they are used, and explore how some low-income programs address administrative issues, audits, or other quality control measures. Finally, we explore the risk of mistaken verifications (finding a participant eligible when they are not) in the different strategies. While this memo was initiated by a request relating to income-based community solar programs, the methods are applicable to any program with income eligibility requirements in the energy or non-energy sector. Funding was provided for this research by the Solar Energy Technologies Office of the US Department of Energy, through the National Community Solar Partnership.

14 SOLAR ENERGY↗
DOE OSTI2024

The First Tri-Lab Workshop on Formal Verification: Capabilities, Challenges, Research Opportunities, and Exemplars

The First Tri-Lab Workshop on Formal Verification was held in Santa Fe, New Mexico, on December 5th, 2023. This workshop gathered staff from Sandia, Los Alamos, and Lawrence Livermore National Laboratories and NASA’s Jet Propulsion Laboratory. This report summarizes and expands on the presentations given and discussion had at this workshop. In this report, we describe the current capabilities and research needs related to formal methods at the NNSA labs. In particular, we identify medium-term and long-term research gaps in programming languages, formalization efforts of complex systems, embedded systems verification, hardware verification, cybersecurity, formal methods usability, workflows, numerical methods, the use of formal methods for artificial intelligence (and its converse, artificial intelligence for formal methods), and collaboration opportunities and considerations on these topics. We conclude with a small number of exemplar research problems related to these topics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Electropotential Verification for Nuclear Safeguards

The international safeguards regime desires methods to efficiently verify that facilities are only performing declared activities. Electropotential verification (EPV) is a newly proposed technique that was tested for its feasibility to perform facility design information verification (DIV) and verification of spent nuclear fuel while in a cooling pool. EPV works by passing a constant, low voltage current through a conductive system (facility infrastructure of nuclear fuel assembly) and measuring the resulting voltage at various places throughout the infrastructure in order to establish a baseline. Changes made to the system affect these voltage readings, which will deviate from the baseline and indicate that a change to the system was made. For facility DIV, it appears feasible that changes in configuration of the system’s grounding can be detected in real-time, and the location of the change can be inferred from the measured intensity of the change in voltage. Determination of whether or not spent fuel was present in a fuel rod, as well as the presence/absence of a fuel rod from an assembly using EPV, proved unsuccessful with the sensitivity of instrumentation used in this study.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

Verification of the ENDF/B-VII.1 Based MC 2 -3 Library Rev.1

The MC 2 -3 code, developed by Argonne National Laboratory under the DOE-NE NEAMS program, is a multigroup cross section generation code for fast reactor applications. Last year, the ENDF/B-VII.0 (E70) MC 2 -3 library, which has been extensively used, verified, and validated over a long period, was intensively reverified and updated to support the commercial grade dedication (CGD) requirement of the TerraPower Natrium project. This year, the ENDF/B-VII.1 (E71) MC 2 -3 library, the preliminary version of which was generated several years ago, was regenerated and rigorously verified to support the Natrium project as well as the completion of verification of the E71 library. The E71 library was verified using the process developed during the verification of the E70 library, including comparisons of cross sections with the NJOY-generated cross sections, comparisons of the resolved resonance cross sections with those using the PEDNF library, and comparison of total cross sections with the sum of partial cross sections. Additional verifications were conducted to ensure that the benchmark problem solutions with the E71 library are reasonable compared to the corresponding Monte Carlo solutions. Furthermore, the E71 gamma library was generated, which includes data for prompt gamma, delayed gamma, and delayed beta as well as neutron and gamma heating. The gamma library was verified at the level of individual isotopes. The EBR-II core solutions from MC 2 -3/ DIF3D and MCNP were compared, demonstrating that those solutions in terms of k-effective and assembly powers were in good agreement.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Addendum to SAND2023-09604 Xyce lumped-element transmission line model verification to support Empire-Cable cable SGEMP analyses

This report supplements the Verification of Empire-Cable SAND report by expanding on the use of Xyce to simulate the coupling to a transmission line cable model. While Empire-Cable solves its governing equations on a high-order, finite-element mesh with an an implicit-in-time formulation, Xyce must use a first order graph for the circuit and explicit-in-time approach to be compatible with non-linear electrical device models. Thus, given the different solution methodologies in Xyce as compared to Empire-Cable, the convergence rates are expected to be different but the overall quality of the solution should be the same. The original four canonical problems studied in the Empire-Cable verification report are replicated here running in Xyce using transmission line modeling parameters from the verification report. Overall, agreement between the codes is excellent with Xyce’s convergence rates being limited mostly to first order due to the circuit network approximation of a transmission line being a first order approximation.

42 ENGINEERING↗
DOE OSTI2025

SAS4A/SASSYS-1 Verification Testing for Sodium Fast Reactor Application: Acceptance Testing Report

AS4A/SASSYS-1 (SAS) is a simulation tool used to perform deterministic analyses of anticipated events as well as design basis and beyond design basis accidents for advanced liquid-metal-cooled nuclear reactors. With its origin as SAS1A in the late 1960s, the SAS series of codes has been under continuous use and development for over sixty years and represents a critical investment in safety analysis capabilities for the U.S. Department of Energy. To support the dedication effort, this report has been generated to provide a detailed description of the available verification testing. The verification testing presented in this report captures functionality testing, focusing mainly on the testing of specific functions and algorithms for accuracy and precision of output, and interface testing, focusing mainly on the testing of critical input parameters and their valid ranges. Although SAS was developed to support the analysis of any liquid-metal-cooled nuclear reactor, the testing described in this document primarily focuses on the verification of SAS capabilities as they relate to a generic pool-type Sodium Fast Reactor (SFR).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

BlueCRAB Domain Overlapping Coupling: Theory & Verification

Coupling low- and high-fidelity codes is a useful way to model complex engineering systems. Thanks to the high-fidelity code, complex phenomena can be resolved in areas of the system where this is required, yet the efficiency of the lower-fidelity code is still retained in modeling the rest of the system. This document details the theory and implementation of coupling two different thermal-hydraulic codes: the system thermal-hydraulics (STH) code System Analysis Module (SAM) and the coarse-mesh computational fluid dynamics (CFD) code Pronghorn. Both applications are included in the Comprehensive Reactor Analysis Bundle (BlueCRAB) code suite and are based on the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework. The domain overlapping (DO) coupling approach was adopted, as it offers proven advantages over more conventional domain decomposition methods. In the DO coupling, SAM provides Pronghorn with boundary conditions that depend on the system-level simulation of the entire plant. In return, the overlapping coupled SAM components–termed “surrogate components”–receive friction factors and source terms computed online based on the Pronghorn simulation. The framework is designed to be generic and enable coupling regardless of geometry and the number of inlet/outlet boundaries in the DO coupled domain. The developed method leads to consistent pressure drops, enthalpies, and scalar concentrations when comparing the coupled SAM and Pronghorn simulations. This document presents the DO coupling approach, along with two verification and two demonstration cases. The proposed problems explore different physical aspects relevant to nuclear reactor analysis, including buoyancy-driven flows, complex flow patterns, and multiple inlets and outlets. Periodically, new versions of this “BlueCRAB Domain Overlapping Coupling: Theory & Verification” report will be issued to reflect future developments and verification tests.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗