Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56

Benchmarking characterization methods for noisy quantum circuits

Effective methods for characterizing the noise in quantum computing devices are essential for programming and debugging circuit performance. Existing approaches vary in the information obtained as well as the amount of quantum and classical resources required, with more information generally requiring more resources. Here we benchmark the characterization methods of gate set tomography, Pauli channel noise reconstruction, and empirical direct characterization for developing models that describe noisy quantum circuit performance on a 27-qubit superconducting transmon device. We evaluate these models by comparing the accuracy of noisy circuit simulations with the corresponding experimental observations. Here, we find that the agreement of noise model to experiment does not correlate with the information gained by characterization and that the underlying circuit strongly influences the best choice of characterization approach. Empirical direct characterization scales best of the methods we tested and produced the most accurate characterizations across our benchmarks.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

IMPLEMENTATION OF THE IMPROVED QUASI-STATIC METHOD IN RATTLESNAKE/MOOSE FOR TIME-DEPENDENT RADIATION TRANSPORT MODELLING

Because of the recent interest in reactor transient modeling and the restart of the Transient Reactor (TREAT) Facility, there has been a need for more efficient, robust methods in computation frameworks. This is the impetus of implementing the Improved Quasi-Static method (IQS) in the RATTLESNAKE/MOOSE framework. IQS has implemented with CFEM diffusion by factorizing flux into time-dependent amplitude and spacial- and weakly time-dependent shape. The shape evaluation is very similar to a flux diffusion solve and is computed at large (macro) time steps. While the amplitude evaluation is a PRKE solve where the parameters are dependent on the shape and is computed at small (micro) time steps. IQS has been tested with a custom one-dimensional example and the TWIGL ramp benchmark. These examples prove it to be a viable and effective method for highly transient cases. More complex cases are intended to be applied to further test the method and its implementation.

IQS↗

Improved Verification and Validation Testing and Tools including Nuclear Criticality Safety Applications with the MCNP6.3® Code [Abstract]

A new Python-based framework has been developed to enable a more consistent layout with automatable setup, execution, and documentation of all verification and validation (V&V) test suites previously established for use with the MCNP code. In this paper, the new general framework for the V&V test suites is discussed, including information on all of the current capabilities and plans for future capabilities. For nuclear criticality safety applications, the existing V&V benchmark problems within the criticality, extended criticality, and analytic k-effective test suites have been ported into this new framework. In addition, the status and updates to the Rossi-alpha and subcritical multiplicationtest suites will be discussed. Some V&V results exercising new MCNP6.3 capabilities will be demonstrated.

97 MATHEMATICS AND COMPUTING↗

Roadmap for Molecular Benchmarks in Nonadiabatic Dynamics

Simulating the coupled electronic and nuclear response of a molecule to light excitation requires the application of nonadiabatic molecular dynamics. However, when faced with a specific photophysical or photochemical problem, selecting the most suitable theoretical approach from the wide array of available techniques is not a trivial task. The challenge is further complicated by the lack of systematic method comparisons and rigorous testing on realistic molecular systems. This absence of comprehensive molecular benchmarks remains a major obstacle to advances within the field of nonadiabatic molecular dynamics. A CECAM workshop, Standardizing Nonadiabatic Dynamics: Towards Common Benchmarks, was held in May 2024 to address this issue. This Perspective highlights the key challenges identified during the workshop in defining molecular benchmarks for nonadiabatic dynamics. Specifically, this work outlines some preliminary observations on essential components needed for simulations and proposes a roadmap aiming to establish, as an ultimate goal, a community-driven, standardized molecular benchmark set.

36 MATERIALS SCIENCE↗

Matrix-Model Simulations Using Quantum Computing, Deep Learning, and Lattice Monte Carlo

Matrix quantum mechanics plays various important roles in theoretical physics, such as a holographic description of quantum black holes, and it underpins the only practical numerical approach to the study of complex high-dimensional supergravity theories. Understanding quantum black holes and the role of entanglement in a holographic setup is of paramount importance for the realization of a quantum theory of gravity. Moreover, a complete numerical understanding of the holographic duality and the emergence of geometric space-time features from microscopic degrees of freedom could pave the way for new discoveries in quantum information science. Euclidean lattice Monte Carlo simulations are the de facto numerical tool for understanding the spectrum of large matrix models and have been used to test the holographic duality. However, they are not tailored to extract dynamical properties or even the quantum wave function of the ground state of matrix models. Quantum computing and deep learning provide potentially useful approaches to study the dynamics of matrix quantum mechanics. If successful in the context of matrix models, these rapidly improving numerical techniques could become the new Swiss army knife of quantum gravity practitioners. In this paper, we perform the first systematic survey for quantum computing and deep-learning approaches to matrix quantum mechanics, comparing them to lattice Monte Carlo simulations. These provide baseline benchmarks before addressing more complicated problems. In particular, we test the performance of each method by calculating the low-energy spectrum.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Maxine Onsite: Estimating Dose from Chronic Atmospheric Radioactive Releases

Maxine Onsite: Developed to estimate dose to individuals who are working onsite but are not site employees. Follows the methodologies discussed in US NRC Regulatory Guides 1.109 and 1.111 to determine dose from a chronic atmospheric release. Determines the dose given relative air concentrations and deposition concentrations as input. We look at four locations onsite that may be accessed by members of the public. To show compliance with the EPA and DOE public dose limit of 10 mrem/yr, determining who is a member of the public or a worker onsite is necessary. SRS has defined a member of the public for NESHAPS and DOE O 458.1. Maxine Onsite runs on Microsoft Windows Operating System through a Microsoft Excel spreadsheet. Input from the user includes parameters that are variable such as air concentrations, and fixed, such as distance to the receptor and elemental iodine fraction. Maxine Onsite was developed from Maxine which is a program meant to calculate dose to the MEI via three pathways, external, inhalation, and ingestion. Maxine Onsite is meant for looking at the exposure through the external and inhalation pathways only, no local food production onsite. The ways that Maxine Onsite is used to calculate the dose to the MEI includes the following: - Determining the respective air concentration values at each location using distances from the release stack; - Breathing rates for the MEI; - Determining the amount of time the MEI is outside; - Determining the plume travel time. The following relative air and ground deposition values for proper execution of the spreadsheet come from MAXDOSE-SR and are needed to accurately determine the dose to the MEI: relative air concentration, relative air concentration decayed by 2.26 days, relative air concentration decayed and depleted for 8 days, and relative deposition. Test cases were designed, developed, and executed to determine the accuracy of Maxine Onsite, which were verified additionally through hand calculations. Several changes were made for testing, including varying air concentrations, distances, breathing rates, and shielding factors. Each test case was then verified through hand calculations to assure the accuracy of the calculated doses. The results from the test cases, as well as the default parameters, can be used for benchmarking Maxine Onsite in the future. Maxine Onsite performs as expected and produces correct and accurate results at each of the different locations onsite (TRL, B-Area Barracks, SRELCC, and Gunsite 51) for varying test cases. The input required by the user is minimal and includes the results of the dose to the MEI on the first page for ease of access. Future plans for Maxine Onsite include adding in the ingestion pathway and adding in atmospheric dispersion estimates to the spreadsheet.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Verification of High-Fidelity Neutronics Code PROTEUS for C5G7 Benchmark Problems

Under the DOE Nuclear Energy Advanced Modeling and Simulation (NEAMS) program, the 3D whole core transport code PROTEUS has been developed at Argonne National Laboratory to perform the detailed 3D transport calculation for various reactor types including irregular or complex geometric reactors. PROTEUS includes two distinct transport solvers: SN and MOC. Among them, the MOC solver based on the extruded geometry, thus named MOCEX, can produce the faithful 3-D transport solutions by employing the rigorous and robust 2D/3D MOC formulation. This formulation overcomes the drawbacks of a full 3D MOC method requiring enormous memory and computational resources and resolves the instability issues of the 2D/1D method as well. The solution of MOCEX converges to that of a Monte Carlo code as the discretization of angle, space and energy is refined, unlike the 2D/1D method. As a verification and validation (V and V) effort of PROTEUS-MOCEX, this paper presents verification test results using the well-known 3D C5G7 benchmark problems by solving and comparing eigenvalues and pin powers with MCNP solutions.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DeepSurveySim: Simulation Software and Benchmark Challenges for Astronomical Observation Scheduling

Modern astronomical surveys have multiple competing scientific goals. Optimizing the observation schedule for these goals presents significant computational and theoretical challenges, and state-of-the-art methods rely on expensive human inspection of simulated telescope schedules. Automated methods, such as reinforcement learning, have recently been explored to accelerate scheduling. However, there do not yet exist benchmark data sets or user-friendly software frameworks for testing and comparing these methods. We present DeepSurveySim -- a high-fidelity and flexible simulation tool for use in telescope scheduling. DeepSurveySim provides methods for tracking and approximating sky conditions for a set of observations from a user-supplied telescope configuration. We envision this tool being used to produce benchmark data sets and for evaluating the efficacy of ground-based telescope scheduling algorithms, particularly for machine learning algorithms that would suffer in efficacy if limited to real data for training.We introduce three example survey configurations and related code implementations as benchmark problems that can be simulated with DeepSurveySim.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Stress Testing of Organic Light- Emitting Diode Panels and Luminaires

This report builds on previous DOE efforts with OLED technology by updating information on a previously benchmarked OLED product (the Chalina luminaire from Acuity Brands) and provides new benchmarks on the performance of Brite 2 and Brite Amber OLED panels from OLEDWorks. During the tests described here, samples of these devices were subjected to continuous operation in stress tests at elevated ambient temperature environments of 35°C or 45°C. In addition, samples were also operated continuously at room temperature in a room temperature operational life test (RTOL). One goal of this study was to investigate whether these test conditions can accelerate failure of OLED panels, either through panel shorting or an open circuit in the panel. These stress tests are shown to provide meaningful acceleration of OLED failure modes, and an acceleration factor of 2.6 was calculated at 45°C for some test conditions. In addition, changes in the photometric properties of the emitted light (e.g., luminous flux and chromaticity maintenance) was also evaluated for insights into the long-term stability of these products compared to earlier generations. Because OLEDs are a lighting system, electrical testing was also performed on the panel-driver pairs to provide insights into the impact of the driver on long-term panel performance.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Single pin BWR benchmark problem for coupled Monte Carlo - Thermal hydraulics analysis

As part of the European NURISP research project, a single pin BWR benchmark problem was defined. The aim of this initiative is to test the coupling strategies between Monte Carlo and subchannel codes developed by different project participants. In this paper the results obtained by the Delft Univ. of Technology and Karlsruhe Inst. of Technology will be presented. The benchmark problem was simulated with the following coupled codes: TRIPOLI-SUBCHANFLOW, MCNP-FLICA, MCNP-SUBCHANFLOW, and KENO-SUBCHANFLOW. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

The Electrolyte Genome project: A big data approach in battery materials discovery

We present a high-throughput infrastructure for the automated calculation of molecular properties with a focus on battery electrolytes. The infrastructure is largely open-source and handles both practical aspects (input file generation, output file parsing, and information management) as well as more complex problems (structure matching, salt complex generation, and failure recovery). Using this infrastructure, we have computed the ionization potential (IP) and electron affinities (EA) of 4830 molecules relevant to battery electrolytes (encompassing almost 55,000 quantum mechanics calculations) at the B3LYP/6-31+G(*) level. We describe automated workflows for computing redox potential, dissociation constant, and salt-molecule binding complex structure generation. We present routines for automatic recovery from calculation errors, which brings the failure rate from 9.2% to 0.8% for the QChem DFT code. Automated algorithms to check duplication between two arbitrary molecules and structures are described. We present benchmark data on basis sets and functionals on the G2-97 test set; one finding is that a IP/EA calculation method that combines PBE geometry optimization and B3LYP energy evaluation requires less computational cost and yields nearly identical results as compared to a full B3LYP calculation, and could be suitable for the calculation of large molecules. Our data indicates that among the 8 functionals tested, XYGJ-OS and B3LYP are the two best functionals to predict IP/EA with an RMSE of 0.12 and 0.27 eV, respectively. Application of our automated workflow to a large set of quinoxaline derivative molecules shows that functional group effect and substitution position effect can be separated for IP/EA of quinoxaline derivatives, and the most sensitive position is different for IP and EA. Published by Elsevier B.V

Battery↗

‘Wasteaware’ benchmark indicators for integrated sustainable waste management in cities

Highlights: • Solid waste management (SWM) is a key utility service, but data is often lacking. • Measuring their SWM performance helps a city establish priorities for action. • The Wasteaware benchmark indicators: measure both technical and governance aspects. • Have been developed over 5 years and tested in more than 50 cities on 6 continents. • Enable consistent comparison between cities and countries and monitoring progress. - Abstract: This paper addresses a major problem in international solid waste management, which is twofold: a lack of data, and a lack of consistent data to allow comparison between cities. The paper presents an indicator set for integrated sustainable waste management (ISWM) in cities both North and South, to allow benchmarking of a city’s performance, comparing cities and monitoring developments over time. It builds on pioneering work for UN-Habitat’s solid waste management in the World’s cities. The comprehensive analytical framework of a city’s solid waste management system is divided into two overlapping ‘triangles’ – one comprising the three physical components, i.e. collection, recycling, and disposal, and the other comprising three governance aspects, i.e. inclusivity; financial sustainability; and sound institutions and proactive policies. The indicator set includes essential quantitative indicators as well as qualitative composite indicators. This updated and revised ‘Wasteaware’ set of ISWM benchmark indicators is the cumulative result of testing various prototypes in more than 50 cities around the world. This experience confirms the utility of indicators in allowing comprehensive performance measurement and comparison of both ‘hard’ physical components and ‘soft’ governance aspects; and in prioritising ‘next steps’ in developing a city’s solid waste management system, by identifying both local strengths that can be built on and weak points to be addressed. The Wasteaware ISWM indicators are applicable to a broad range of cities with very different levels of income and solid waste management practices. Their wide application as a standard methodology will help to fill the historical data gap.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

RELAP-7 and PRONGHORN Initial Integration Plan

Modern nuclear reactor safety codes require the ability to solve detailed coupled neutronicthermal fluids problems. For larger cores, this implies fully coupled 3-D spatial dynamics with appropriate feedback models that can provide enough resolution to accurately compute core heat generation and removal during steady and unsteady conditions. The reactor analyis code PRONGHORN is being coupled to RELAP-7 as a first step to extend RELAP's current capabilities. This report details the mathematical models, the type of coupling, and the testing that will be used to produce an integrated system. RELAP-7 is a MOOSE-based application that solves the continuity, momentum, and energy equations in 1-D for a compressible fluid. The pipe and joint capabilities enable it to model parts of the PCU system. The PRONGHORN application, also developed on the MOOSE infrastructure, solves the coupled equations that define the neutron diffusion, fluid flow, and heat transfer in a 3-D core model. Initially, the two systems will be loosely coupled to simplify the transition towards a more complex infrastructure. The integration will be tested with the OECD/NEA MHTGR-350 Coupled Neutronics-Thermal Fluids benchmark model.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Robust validation of approximate 1-matrix functionals with few-electron harmonium atoms

A simple comparison between the exact and approximate correlation components U of the electron-electron repulsion energy of several states of few-electron harmonium atoms with varying confinement strengths provides a stringent validation tool for 1-matrix functionals. The robustness of this tool is clearly demonstrated in a survey of 14 known functionals, which reveals their substandard performance within different electron correlation regimes. Unlike spot-testing that employs dissociation curves of diatomic molecules or more extensive benchmarking against experimental atomization energies of molecules comprising some standard set, the present approach not only uncovers the flaws and patent failures of the functionals but, even more importantly, also allows for pinpointing their root causes. Since the approximate values of U are computed at exact 1-densities, the testing requires minimal programming and thus is particularly suitable for rapid screening of new functionals.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

SAS4A/SASSYS-1 Validation with EBR-II Tests Performed During the SHRT Testing Program (Rev. 2)

The EBR-II sodium-cooled fast reactor supported a wide range of fast reactor research activities. The final fifteen years of operations at EBR-II were used for experiments and tests to demonstrate the importance of passive safety in liquid metal reactors and the capability of such systems to provide strong passive responses during off-normal conditions. The Shutdown Heat Removal Test program at EBR-II provided test data supporting the validation of computer codes for design, licensing, and operation of LMRs, among other objectives. The program included nearly sixty tests, most of which were protected and unprotected loss of flow and loss of heat sink tests run at various initial powers and flow rates. This report documents the results of modeling and simulation of tests from the Shutdown Heat Removal Test program to support validation of Argonne’s SAS4A/SASSYS-1 fast reactor safety analysis code. SHRT-17 and SHRT-6 were protected loss of flow tests where a loss of electrical power to all sodium coolant pumps was simulated to demonstrate the effectiveness of natural circulation cooling characteristics. SHRT-45R, SHRT-39, and SHRT-43R were unprotected loss of flow tests where the control rod scram function of the plant protection system was disabled to demonstrate the effectiveness of EBR-II’s passive reactivity feedbacks. BOP-301 and BOP-302R were unprotected loss of heat sink tests that demonstrated how reactivity feedback effects driven by changes to the core inlet temperature can shut down the fission process. PICT-5/6 demonstrated how core power could be increased and decreased via changes in the intermediate sodium and feedwater flow rates. This validation activity leverages modeling and simulation efforts that originated under an International Atomic Energy Agency Coordinated Research Project for benchmark analysis of the EBR-II Shutdown Heat Removal Tests. The SAS4A/SASSYS-1 models originally developed for the CRP have been modernized to utilize new code features and modeling practices. Differences between the predicted and measured test data were explored and the causes of these differences were identified as being due to modeling approximations, uncertainty in transient component behavior, and instrumentation uncertainties. Overall, it was concluded that for all tests simulated there was good agreement between the flow, power, and temperature predictions and the measured data.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Hydraulic and mixing evaluations for SRS miscible liquid solutions

The hydraulic results demonstrate that pump head pressure of 20 psi recirculates about 5.6 liters/min flowrate through the existing 0.131-inch orifice when a valve connected to NT-41 is closed. In case of the valve open to NT-41, the solution flowrates to Savannah River Site (SRS) tanks, NT-21 and NT-41, are found to be about 0.5 lpm and 5.2 lpm, respectively. The modeling calculations for the mixing operations of miscible fluids contained in the SRS HB-Line tank NT-21 were performed by taking a three-dimensional Computational Fluid Dynamics (CFD) approach. The CFD modeling results were benchmarked against the literature results and the previous SRNL test results to validate the model. Final performance calculations were performed for the nominal case by using the validated model to quantify the mixing time for the HB-Line tank. The results demonstrate that when a pump recirculates a solution volume of 5.7 liters every minute out of the 72-liter tank contents containing two acid solutions of 2.7 M and 0 M concentrations (i.e., water), a minimum mixing time of 1.5 hours is adequate for the tank contents to get the tank contents adequately mixed. In addition, the sensitivity results for the tank contents of 8 M existing solution and 1.5 M incoming species show that the mixing time takes about 2 hours to get the solutions mixed. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

SU-F-T-139: Meeting the Challenges of Quality Control in the TOPAS Monte Carlo Simulation Toolkit for Proton Therapy

Purpose: Monte Carlo particle transport simulation (MC) codes have become important tools in proton therapy and biology, both for research and practice. TOPAS is an MC toolkit serving users worldwide (213 licensed users at 95 institutions in 21 countries). It provides unprecedented ease in 4D placement of geometry components, beam sources and scoring through its user-friendly and reproducible parameter file interface. Quality control (QC) of stochastic simulation software is inherently difficult, and the versatility of TOPAS introduces additional challenges. But QC is vital as the TOPAS development team implements new features, addresses user feedback and reacts to upgrades of underlying software (i.e. Geant4). Methods: Whenever code is committed to our repository, over 50 separate module tests are automatically triggered via a continuous integration service. They check that the various module options execute successfully and that their results are statistically consistent with previous reference values. Prior to each software release, longer end-to-end tests automatically validate TOPAS against experimental data and a TOPAS benchmark. These include simulating multiple properties of spread-out Bragg peaks, validating nuclear models, and searching for differences in patient simulations. Results: Continuous integration has proven effective in catching regressions at the time they are introduced, particularly when implementing new features that involve refactoring code (e.g. multithreading and ntuple output). Code coverage statistics highlight untested portions of code and guide development of new tests. The various end-to-end tests demonstrate that TOPAS accurately describes the physics of proton therapy within clinical tolerances. Conclusion: The TOPAS QC strategy of frequent short tests and pre-release long tests has led to a more reliable tool. However, the versatility of TOPAS makes it difficult to predict how users might combine different modules, and so QC ultimately remains a partnership between the developer and the user. This work was funded by National Cancer Institute grant R01 CA 140735.

60 APPLIED LIFE SCIENCES↗

Progress Towards the Reality of Low-Temperature Critical Experiments with Low-Temperature TEX

Integral benchmarks for criticality safety do not exist below room temperature. Incredible support for below room temperature benchmarks is present across the international community. Validation of low-temperature neutron cross sections is necessary to verify any operation at temperatures below room temperature which is typically observed in environments far from the equator. For example, a fissile material transportation truck may routinely observe ambient temperatures down to -40°C, which is the lower temperature bound of the normal conditions of transportation defined in the United States Title 10 Code of Federal Regulations §71.71c2. Similarly, the IAEA, which contains 176 member states, has the same requirement for subcriticality for fissile material transportation. Additionally, sub-room temperature benchmarks can validate newly produced cross sections, that include novel thermal scattering laws, from North Carolina State University. A six-case experiment designed based off the TEX-HEU baseline experiments has been proposed utilizing a chilled vacuum chamber capable of cooling the HEU stack down to -50°C. The six proposed configurations span the entire neutron fission energy range and can be tested at intermediate temperatures between room temperature and -40°C. Of the six configurations, the fastest case utilizes nearly 135 kg of HEU. Understanding the system is required prior to critical experiment operations. In order to test the functionality of the system, testing with surrogate fuel materials has been conducted. The following discusses the design of the low temperature TEX (LT-TEX) experiment as well as the progress made via surrogate testing towards a reality in which low-temperature integral benchmarks exist in the International Criticality Safety Benchmark Evaluation Project (ICSBEP).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗