Search NASA⌕ Search

SEARCH · Search NASA

Results for “Software Verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Software Quality Assurance for the MOOSE-Based Open-Source Multiphysics Code Cardinal - An Expanded CI Testing Suite

Cardinal is a wrapping of the GPU-oriented spectral element Computational Fluid Dynamics (CFD) code NekRS and the Monte Carlo particle transport code OpenMC within the Multiphysics Object-Oriented Simulation Environment (MOOSE). Cardinal provides high-resolution thermal-hydraulics and/or radiation transport feedback to MOOSE multiphysics simulations. Multiphysics feedback is implemented in a geometry-agnostic manner which eliminates the need for rigid one-to-one mappings. A generic data transfer implementation also allows NekRS and OpenMC to couple to any MOOSE application, enabling a broad set of multiphysics capabilities. Cardinal simulations can also leverage combinations of MPI, OpenMP, and GPU resources. Cardinal continuous development and improvement efforts have led to the software being considered as a high-fidelity design and licensing tool for key areas of nuclear reactor relevant physics, including neutron transport, fluid flow, heat transfer, and mechanical processes. The fast development and expansion of the software from a pure R&D framework towards its application in the nuclear industry and regulation require a focus on developing, enhancing and, maintaining Cardinal’s software quality through strict adherence to a Software Quality Assurance (SQA) framework and SQA program. To facilitate compliance with SQA standards, the Cardinal SQA Program has been initiated during Fiscal Year 2023 (FY23). During the development of the Cardinal SQA Program, multiple gaps have been identified. These gaps are primarily related to model verification and code pedigree as they relate to the use of Cardinal as a safety analysis tool. These gaps have been captured in a report published in 2023. A second report highlighted the progress made during Fiscal Year 2024 (FY24) and described Argonne’s effort to document and integrate software verification within Cardinal’s software development process. This report documents a snapshot of the verification test cases currently available for Cardinal and NekRS in their assimilation into a Continuous Integration (CI) platform. Following the CI practice permits the integrating of source code changes frequently and ensuring that the integrated codebase clears the verification testing for the software. It should be noted that the SQA program itself, including the program plans, procedures, configuration management, and testing strategies, need to be developed in a future step of this task.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Bison Verification and Validation Activities for TRISO

Numerical modeling and simulation (M&S) tools play a key role in the research, development, and overall safety assessments of next-generation nuclear energy systems. One such tool, Bison, is a nuclear fuel performance code that is applicable to many fuel forms (e.g., light-water reactor fuel, oxide and metallic fuel for fast reactors, tri-structural isotropic (TRISO) fuel, and plate fuel), and it uses the finite element method to model the thermo- mechanical response of nuclear fuels. One fuel form widely utilized in Generation-IV high-temperature gas-cooled and fluoride- salt-cooled nuclear reactor concepts is TRISO fuel. Recently, Bison’s capabilities were significantly expanded to enable it to model the performance of TRISO particles and compacts. It is important that Bison’s computational results be reliable and predictive, since this code is used to inform high-consequence decisions. The various processes developed to address this issue generally entail two fundamental steps: verification and validation (V&V). Verification ensures that the code functions correctly and is reliable. Code/solution verification, code benchmark, and software quality assurance exercises are examples of verification activities. On the other hand, validation is the process of assessing a code’s capability to accurately model physical problems. Comparisons between code results and experiments quantify the validation level. Application of V&V procedures is crucial to the development of computational tools that are free of coding mistakes and can accurately represent reality. The current study presents an overview of Bison V&V activities relevant to the TRISO fuel concept, which include code/solution verification exercises, CRP-6 Benchmark—a Coordinated Research Program through the International Atomic Energy Agency (IAEA)—exercises, and validation exercises with the Advanced Gas Reactor (AGR)- 1/2/3/4 experiment series.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Progress Towards NQA-1 for Cardinal in FY25

Cardinal is a wrapping of the GPU-oriented spectral element Computational Fluid Dynamics (CFD) code NekRS and the Monte Carlo particle transport code OpenMC within the Multiphysics Object-Oriented Simulation Environment (MOOSE). Cardinal provides high-resolution thermal-hydraulics and/or radiation transport feedback to MOOSE multiphysics simulations. Multiphysics feedback is implemented in a geometry-agnostic manner which eliminates the need for rigid one-to-one mappings. A generic data transfer implementation also allows NekRS and OpenMC to couple to any MOOSE application, enabling a broad set of multiphysics capabilities. Cardinal simulations can also leverage combinations of MPI, OpenMP, and GPU resources. Cardinal continuous development and improvement efforts have led to the software being considered as a high-fidelity design and licensing tool for key areas of nuclear reactor relevant physics, including neutron transport, fluid flow, heat transfer, and mechanical processes. The fast development and expansion of the software from a pure R&D framework towards its application in the nuclear industry and regulation require a focus on developing, enhancing,and maintaining Cardinal’s software quality through strict adherence to a Software Quality Assurance (SQA) framework and SQA program. To facilitate compliance with SQA standards, the Cardinal SQA Program was initiated during Fiscal Year 2023 (FY23). During the development of the Cardinal SQA Program, multiple gaps have been identified. These gaps are primarily related to model verification and code pedigree as they relate to the use of Cardinal as an analysis tool. These gaps were captured in a report published in 2023. A second report highlighted the progress made during Fiscal Year 2024 (FY24) and described Argonne’s effort to document and integrate software verification within Cardinal’s software development process. This report documents the progress made towards NQA-1 for Cardinal in the Fiscal Year 2025 (FY25). All cases in the expanded Continuous Integration (CI) suite of NekRS are included in this report which test the solvers and modules available in NekRS exhaustively. The NekRS tests are integrated with the Cardinal CI suite and made available in publicly accessible Github documentation. Following the CI practice permits integrating of source code changes frequently and ensuring that the integrated codebase clears the verification testing for the software. Also in this report is a brief overview of the development of the Cardinal Software Quality Assurance Plan (SQAP) that was done in FY25, though it should be noted that the rest of the documentation for the SQA program needs to be developed in a future step of this task.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Verification Testing of OLI Systems Mixed Solvent Electrolyte Model for the Na-K-Mg-Ca-H-Cl-SO 4 -OH-HCO 3 -CO 3 -CO 2 -H 2 ) System to High Ionic Strength at 25°C

This technical report summarizes model verification results and summary statistics for 41 evaporite mineral solubility cases evaluated by Savannah River National Laboratory using OLI Systems’ aqueous electrolyte thermodynamic modeling software. The 41 verification cases containing a total of 60 solubility curves comprise mineral solubility data from low to high ionic strength at 25°C for the eight-component system Na-K-Mg-Ca-H-Cl-SO 4 -OH-HCO 3 -CO 3 -CO 2 -H 2 O as reported by Harvie et al. (1984). Thermodynamic calculations were executed using OLI Systems’ Stream Analyzer computation module within the OLI Studio software platform (Ver. 11.0, Rev. 11.0.1.9). The Mixed Solvent Electrolyte (MSE) thermodynamic framework was chosen for this investigation because of its superiority in modeling high ionic-strength inorganic salt solutions and actinide redox chemistry and solubility, both of which are relevant to the geological repository conditions at the Waste Isolation Pilot Plant in Carlsbad, New Mexico. Mineral solubility data in various inorganic salt solutions were digitized and extracted from figures generated by Harvie et al. (1984). For each of the 60 solubility curves, a case-specific chemistry model and input file were generated in OLI Studio using OLI Stream Analyzer and the MSE (H 3 O + ion) public databank provided by OLI Systems. Model simulation results were exported to Microsoft Excel to calculate summary statistics and to generate graphs comparing the OLI model predictions to the solubility data. Summary statistics include residuals (model – data) and concordance (accuracy × precision, where precision is indicated by the Pearson correlation coefficient and accuracy accounts for bias and scale differential). Private databanks were not developed, and activity coefficient model regressions were not performed to improve OLI model fits to the data. Of the 41 model verification plots, 83% have a mean of the percent residuals less than or equal to 25%. Similarly, 75% display a concordance greater than or equal to 0.75. Only seven of the 41 verification plots fail to show good agreement between the model and data. Of these seven, three are relevant to the WIPP repository because they involve the Mg-OH-Cl-SO 4 -CO 3 aqueous system. The remaining four address salt solubilities at the pH extremes (strong acid and strong base). It should be noted that in two of the three Mg-OH-Cl-SO 4 -CO 3 system cases, the regressed Harvie et al. (1984) solubility curve also deviated from the data. Lack of agreement between the OLI model-predicted solubility curves and the data is attributable to one or more of the following: specific solid species are not included in the OLI MSE databank; there is significant variation among the different solubility datasets chosen by Harvie et al. (1984); the OLI MSE model’s thermodynamic parameters were determined using different solubility datasets; and the activity coefficient parameters for certain relevant ion-ion and ion-molecule pairs have not been optimized via data regression. Two recommendations for future work are to (1) evaluate solubility data for the Mg-OH-Cl-SO 4 -CO 3 system at high ionic strength and, if necessary, develop a private OLI MSE database that includes missing species and, where necessary, regressed standard state properties and interaction parameters; (2) perform similar verification testing of the OLI model for actinide solubility data.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

Code Verification and Solution Verification framework in pin-resolved neutron transport code MPACT

Program verification in scientific computing encompasses the application of formal and mathematical techniques to a scientific computing code for its credibility, accuracy, and validity. Code Verification identifies bugs and performance issues in the software development stage. Solution Verification assesses the applicability of the code and the accuracy of the solution to problems of interest. Both activities utilize application cases and quantify the error against prescribed acceptance criteria. However, simply executing more application cases does not guarantee stronger or more comprehensive credibility. Here, we establish a verification framework that involves Code Verification and Solution Verification, both of which work together such that the overarching goal of “converge to the correct answer for the intended application” can be reasonably inferred. The application of such a verification framework is demonstrated using the pin-resolved neutron transport code MPACT, where standard unit tests and regression tests are covered, and where the Method of Exact Solutions and the Method of Manufactured Solutions are successfully used. Additionally, the applicability of Method of Manufactured Solutions is extended to the OECD/NEA C5G7 benchmark problems of practical material and geometric configurations. Solution Verification activities are demonstrated on a practical hierarchy of application models of increasing complexity ranging from 2D pin cell problems to 3D assembly problems. The convergence behavior and rate of convergence with respect to each individual variable are studied and provided. This framework can be adapted broadly to other fields involving scientific computing codes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Validation and Verification of Python based Neutron Spectrum Unfolding Software

To validate and verify the python-based code (PySL), designed to replicate the programs used by STAYSL for Beam Correction Factor (BCF) and Self-Shielding Factor (SHIELD), a series of tests were performed. To test BCF a python script was written to generate a random flux history file and both versions of the code processed the data. The test verified matching values up to at least one decimal place, approximately 10,000 tests where run and each one passed. Isotopes began to fail the tests once neutron saturation was reached. To verify this the total time of exposure was varied the isotopes that failed were compared to a list of their half-lives. The test process for SHIELD was very similar but, in this case, the code began by producing an input file with varying thickness and device type/environment for the SHIELD input. The failure condition for this test was if any of the data points for an isotope had a difference above 3%. Approximately 40 of these tests were run and there were only 3 isotopes that had reoccurring failures but only 2% of their points were above the 3% difference. A visual comparison was conducted by plotting the results from both programs. Although the test failed, the differences between their values were minuscule, and the self-shielding factor’s shape was preserved when plotted. Next steps for this project will be validating and verifying the python-based SigPhi code and then reproducing and testing the least squares unfolding performed by STAYSL.

73 - NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

SAM: A Modern System Code for Advanced Non-LWR Safety Analysis

The System Analysis Module (SAM), developed at Argonne National Laboratory and by collaborators at other organizations, is for advanced non–light water reactor safety analysis. SAM aims to provide fast-running, modest-fidelity, whole-plant transient analysis capabilities that are essential for fast-turnaround design scoping and engineering analyses of advanced reactor concepts. To facilitate code development, SAM utilizes the MOOSE object-oriented application framework, its underlying finite element library, and linear and nonlinear solvers to leverage modern advanced software environments and numerical methods. SAM aims to solve tightly coupled physical phenomena, including fission reaction, heat transfer, fluid dynamics, and thermal-mechanical responses in advanced reactor structures, systems, and components with high accuracy and efficiency. Finally, this paper gives an overview of the SAM code development, including goals and functional requirements, physical models, current capabilities, verification and validation, software quality assurance, and examples of simulations for advanced nuclear reactor applications.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

SAS4A/SASSYS-1 Lead Verification Test Suite

SAS4A/SASSYS-1 is a simulation tool used to perform deterministic analyses of anticipated events as well as design basis and beyond design basis accidents for advanced liquid-metal-cooled nuclear reactors. With its origin as SAS1A in the late 1960s, the SAS series of codes has been under continuous use and development for over forty-five years and represents a critical investment in safety analysis capabilities for the U.S. Department of Energy. Although SAS4A/SASSYS-1 was developed to support the analysis of any liquid-metal-cooled nuclear reactor, it has primarily been utilized to design and analyze Sodium Fast Reactors (SFRs). As a result, most of the verification basis for SAS4A/SASSYS-1 utilizes sodium as a coolant and geometry descriptions that are typical of a SFR facility. In this work, new verification problems are defined to extend the applicability of the SAS4A/SASSYS-1 verification basis into Lead Fast Reactor (LFR) design space. A review of the existing SAS4A/SASSYS-1 SFR verification test suite is performed to determine the number of test cases that are SFR specific. SFR specific test cases are replicated using lead as a coolant and system layouts that are representative of LFRs. This report provides definitions and reference solutions for seven new test cases that scope lead-specific features and capabilities of the software. For a verification test to be considered acceptable, comparisons of analytical solutions to SAS4A/SASSYS-1 predictions must produce negligible or justifiable errors. All LFR specific test cases are found to be acceptable.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2017

IEEE STD. 1012 and NEI 96-07, Appendix D - Strange bedfellows? - 149

The final draft of a revision to IEEE Std. 1012-2012, 'IEEE Standard for System, Software and Hardware Verification and Validation', is expected to be approved for publication in June 2017. A draft of NEI 96-07, Appendix D, 'Supplemental Guidance for Application of 10 CFR 50.59 to Digital Modifications', has been submitted to the NRC staff for review and endorsement. What, if any, relationship exists between these two documents? How can the nuclear industry utilize the first one to help ensure success when applying the second one? NRC Regulatory Guides only endorse digital system and software related industry standards for application to safety-related systems and components, but the vast majority of digital modifications are made to non-safety related systems and components, so are these industry standards of little or no use when performing digital upgrades to non-safety related equipment? This paper answers these and other related questions and shows how application of a graded approach to verification and validation, utilizing the IEEE Std. 1012 concept of 'integrity level', can increase the likelihood of a 10 CFR 50.59 evaluation outcome that does not require prior NRC review and approval for non-safety related digital modifications. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Develop and verify soil/structure interaction for pile/foundation interaction

Phase II of the Offshore Code Comparison Collaboration, Continued, with Correlation and unCertainty (OC6) project was used to verify the implementation of a new soil-structure interaction (SSI) model for use within offshore wind turbine modeling software. The REDWIN Macro-element model implemented and verified in this study enables a computationally efficient way to model the linear and nonlinear SSI problem, including hysteretic damping, of a monopile structure. The modeling approach was integrated into several modeling tools and a series of increasingly complex simulations was conducted using the IEA 10MW reference turbine mounted on a monopile support structure to verify the coupling between the tools and the REDWIN Macro-element SSI model. This campaign includes only numerical verification between various software and modeling approaches so no experimental measurements are available. The load cases (LC) considered include: LC1 – static response of the tower and substructure LC2 – frequency and mode-shape analysis of the tower and substructure LC3 – response of the tower and substructure due to wind-only loading LC4 – response of the tower and substructure due to wave-only loading LC5 – response of the tower and substructure due to wind and wave loading. Detailed properties of the modeled system are found in the following reference, “Bergua, Roger, Amy Robertson, Jason Jonkman, and Andy Platt. 2021. "Specification Document for OC6 Phase II: Verification of an Advanced Soil-Structure Interaction Model for Offshore Wind Turbines.” Golden, CO: National Renewable Energy Laboratory. NREL/TP-5000-79938. https://www.nlr.gov/docs/fy21osti/79938.pdf. Details on the results from the OC6 Phase II project can be found in the following reference, “Bergua R, Robertson A, Jonkman J, et al. OC6 Phase II: Integration and verification of a new soil–structure interaction model for offshore wind design.” Wind Energy. 2022;25(5):793-810. doi:10.1002/we.2698

17 WIND ENERGY↗
DOE OSTI2023

New Developments and Verification of the Fusion Blanket Simulation Capabilities in the MOOSE Framework

This poster contains new developments and updates in the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework related to fusion modeling and simulation, and forms the basis for an on-going effort in modeling whole fusion devices at Idaho National Laboratory using MOOSE. In this work, a slice of the midplane of a proposed ceramic breeder blanket design for the Fusion Nuclear Science Facility is simulated using a combination of neutronics, 1D system level thermal hydraulics for helium, 3D heat conduction, and tritium transport. Previous work focused on the engineering scale full assembly of a blanket sector, but the simulations here form the basis for software quality and verification efforts to increase confidence as the large scale simulations continue. This poster was presented at the Idaho National Laboratory Intern Poster Session 2023.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2018

Methodology for Building Granular Testing in Multicomponent Scientific Software

Computational science and engineering communities develop complex application software with multiple mathematical models that need to interact with one another. Partly due to complexity of verifying scientific software, and partly because of the way incentives work in science, there has been insufficient testing of these codes. With a spotlight on the results produced with scientific software, and increasing awareness of software testing and verification as a critical contributor to the reliability of these results, testing is gaining more attention by the developing teams. However, many science teams struggle to find a good solution for themselves due either to lack of training or lack of resources within the team. In this experience paper, we describe test development methodologies utilized in two different scenarios: one explains a methodology for building granular tests where none existed before, while the second demonstrates a methodology for selecting test cases that build confidence in the software through a process similar to scaffolding. The common insight from both the experiences is that testing should be a part of software design from the beginning for better software and scientific productivity.

test building, granular testing, Earth system mode↗
DOE OSTI2018

A new fuel modeling capability, CTFFuel, with a case study on the fuel thermal conductivity degradation

Here, a new fuel modeling capability, CTFFuel, is developed from the subchannel code, CTF. This code is a standalone interface to the CTF fuel rod models, allowing for fuel rod simulations to be run independently from the fluid. This paper provides an overview of the code with a case study on the thermal conductivity degradation of LWR fuels to demonstrate its capabilities. The modeling of fuel thermal conductivity degradation in the code is improved through the addition of new modeling options to account for the irradiation effects via globally defined parameters. After the initial implementation, a variety of order-of-accuracy tests and code comparisons are performed to test software quality. A controlled analysis is allowed by CTFFuel to verify the numerical scheme of CTF’s conduction solution and to benchmark its fuel temperature predictions against FRAPCON-4.0’s. Overall, the software quality and verification procedure ensures that the new model is coded correctly, that it properly interacts with the rest of the code.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

MFANS 2024 - Formally Proving Characteristics of Cyber-Physical Systems

Cyber-physical systems (CPS) are engineered systems that rely on the smooth integration of computational algorithms and physical elements. This integration presents new challenges for verifying that systems will behave as expected. The goal of this presentation is to present current challenges and potential solutions for the formal verification of cyber-physical systems. For cyber systems, formal methods refer to systematically rigorous mathematical techniques employed in the specification, development, analysis, and verification of both software and hardware systems. Recent advancements in computer science have yielded sophisticated tools specifically designed to address challenges associated with formal methods in complex systems. These tools leverage various foundational concepts such as logic, formal languages, program semantics, type systems, type theory, and automata theory. A notable achievement in the application of formal methods is the seL4 microkernel, claimed to be the first general-purpose operating-system kernel to be verified. Its proof implies the absence of bugs and guarantees that the kernel meets specifications. For physical systems, dynamic and control theory has a history of using rigorous analytic techniques to prove functional correctness. Lyapunov, optimal, classical, modern, and robust control theories all provide rigorous mathematical methods both to analyze system performance and to design controller that can be guaranteed to meet certain objectives. Recent computational techniques like level set theory and reachability analysis provide assertions that a system's state will avoid unsafe regions. Even though success has been independently achieved for cyber systems and physical systems, the integration of such systems creates new challenges. In particular, there is an obvious discrepancy between finite-state machines and infinite-state systems, resulting in different approaches for modeling and analyzing these system. While it is possible to simulate hybrid systems, this provides only a demonstration of a performance and not proof. For hybrid systems, current formal methods and system analysis approaches typically require a workarounds to work on hybrid systems like CPS. This paper will outline the state of the art and limits of current practice for formally verifying CPS and will identify possible research directions that require attention.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

SAS4A/SASSYS-1 VERIFICATION AND VALIDATION TEST SUITE

SAS4A/SASSYS-1 is a safety analysis software package already under copyright by Argonne National Laboratory. There is interest from third-party users of the software to gain access to our internal verification and validation test suite. Therefore we are seeking to establish copyright over the test suite, which includes input files and reference results for hundreds of test cases that demonstrate functionality of the software. Making the test suite available to licensed users allows them to utilize the broad range of tests for commercial grade dedication activities needed to qualify the SAS4A/SASSYS-1 software under their own Software Quality Assurance program for use in a regulatory environment (e.g. with the Nuclear Regulatory Commission).

FANNING, THOMAS↗
DOE OSTI2022

A Knowledge-based Framework for Building Energy Model Performance Verification

Building energy modeling (BEM) has been widely used by researchers, regulators, and engineers to quantify building energy performance. Quality assurance (QA) and quality control (QC) of the model's performance are essential parts of such analysis. Currently, QA/QC is done in a manual and ad-hoc manner, which is tedious, error-prone, and time-consuming when QA/QC a large number of models. To solve these challenges, we propose a a dAta-driveN buIlding perforMance verificATion framEwork (ANIMATE), which conducts automated output-based verification of building operations requirements (especially for time-series output-based verification of control requirements). While this framework was developed for verifying energy model performance, it can be extended for other applications such as BEM software testing and performance verification of real buildings in the field.

Chen, Yan↗
DOE OSTI2024

CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations

Software developers must adapt to keep up with the changing capabilities of platforms so that they can utilize the power of High-Performance Computers (HPC), including exascale systems. OpenMP, a directive-based parallel programming model, allows developers to include directives to existing C, C++, or Fortran code to allow node level parallelism without compromising performance. This paper describes our CI/CD efforts to provide easy evaluation of the support of OpenMP across different compilers using existing testsuites and benchmark suites on HPC platforms. Our main contributions include (1) the set of a Continuous Integration (CI) and Continuous Development (CD) workflow that captures bugs and provides faster feedback to compiler developers, (2) an evaluation of OpenMP (offloading) implementations supported by AMD, HPE, GNU, LLVM, and Intel, and (3) evaluation of the quality of compilers across different heterogeneous HPC platforms. With the comprehensive testing through the CI/CD workflow, we aim to provide a comprehensive understanding of the current state of OpenMP (offloading) support in different compilers and heterogeneous platforms consisting of CPUs and GPUs from NVIDIA, AMD, and Intel.

Jarmusch, Aaron↗