Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation

As machine learning (ML) is increasingly implemented in hardware to address real-time challenges in scientific applications, the development of advanced toolchains has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as hardware synthesis, are becoming limiting factors in the rapid iteration of designs. To mitigate these emerging constraints, multipleefforts have been undertaken to develop an ML-based surrogate model that estimates resource usage of synthesized ML accelerator architectures. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of over 680 000 fully connected and convolutional neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, and the average performance across a subset of the dataset. Additionally, we introduce GNN- and transformer-based surrogate models that predict latency and resources for ML accelerators. We present the architecture and performance of the models and find that the models generally predict latency and resources for the 75% percentile within several percent of the synthesized resources on the synthetic test dataset.

Hawks, Benjamin G. [Fermilab]↗
DOE OSTI2017

VERA Core Physics Benchmark Progression Problem Results for VERA 3.6

This report contains a comparison of results obtained with VERA 3.6 Release Candidate 0 for the Core Physics Benchmark Progression Problems 1 through 9 to both computational tools and measured data from Watts Bar Unit 1 (WBN1). (Ref. 1 and Ref. 2) These problems progress from single pin cell analyses to full core depletion calculations. VERA results obtained for Problems 1 through 5 are compared against results obtained using the SCALE 6.2 Beta code KENO-VI. Results for Problem 6 and 7 are compared against those obtained using a coupled neutronic and thermal-hydraulic combination of the KAPL software MC21 and the CASL tool CTF. (Ref. 3, Ref. 4 and Ref. 5) VERA results from problems 5 and 9 are compared to data from WBN1. For all of the analyses contained in this report, VERA provided similar results to other state-of-the-art software programs and real reactor data. This indicates that the program is capable of modeling a full-scale PWR in a more discretized and computationally efficient manner than any other reactor physics program. Since the descriptions of the Core Physics Benchmark Progression Problems and the logic behind their selection is discussed in-depth elsewhere it is not covered in the scope of this document. Additionally, in-depth discussions of the software programs used in this analysis are not provided for the same rationale.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2018

VERA Benchmark Results with HELIOS Library

This report presents the comparison of results of calculations from the latest VERA 3.9R1 release with the default cross section data library along with another independent data library generated based on the HELIOS cross section data. The calculations are performed for the AMA Core Physics Benchmark Progression Problems 1 through 7, the Watts Bar Unit 1 (WB1) Cycles 1 & 2, and other actual TMI, WB1 cycles. When available, the results from continuous energy (CE) Monte Carlo transport calculations, KENO, are also used as a comparison. The Progression Problems are based on the WB1 initial startup core and span a single fuel pin up to full core depletion. The KENO results referenced in this report are extracted from a previous benchmarking report documented in Reference 7.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

TEX-Chlorine: Highly Enriched Uranium with Chloride Absorbers to Provide Validation Benchmarks for Y-12 Electrorefining Facility

This report documents the final benchmark of the TEX-Chlorine (IER-499) Thermal Epithermal eXperiments (TEX) with highly enriched uranium with chlorine absorbers and high-density polyethylene reflectors and moderators. TEX-Chlorine is a variation of the TEX-HEU baseline assembly with the addition of sodium chloride absorber plates. This evaluation contains three experimental configurations that were performed on Comet at NCERC between July and August 2024. The three configurations, which were acceptable as benchmark cases, spanned from thermal (first two configurations) to fast (third configuration). All three cases were reviewed and accepted by the ICSBEP TRG in April 2025 and was submitted to the ICSBEP in August 2025 after receiving subgroup approval.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Final Design for Thermal/Epithermal eXperiments (TEX) with Lithium Absorbers to Provide Validation Benchmarks for Y-12 Electrorefining Facility

One of the main goals of the Thermal/Epithermal eXperiments (TEX) project is to use existing Nuclear Criticality Safety Program (NCSP) assets to create critical experiment plutonium and uranium test beds for materials important to criticality safety that have insufficient benchmark evaluations. The plutonium test bed experiments were completed in 2018 and are published in the 2020 edition of the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook. The uranium test bed assemblies were completed in 2023 and accepted in the 2024 edition of the ICSBEP Handbook. The Nuclear Criticality Safety (NCS) group at Y-12 National Security Complex has identified programmatic need for validation cases for uranium electrorefining operations at Y-12. The electrorefining operation credits lithium enriched in 6 Li in addition to 35 Cl as absorbers in the design criticality safety evaluation for precluding criticality under upset conditions in the large and geometrically unfavorable electro-refiner. There is, however, inadequate experimental validation for the 6 Li absorbers. As an extension of the TEX uranium test bed, TEX-Cl critical experiments were performed with sodium chloride salt to address the 35 Cl thermal absorption as well as other validation needs at Los Alamos National Laboratory (LANL). These experiments were completed in 2024 and accepted into the 2025 ICSBEP Handbook. To continue the methodology used in TEX-Cl, TEX-Li aims to accomplish the same. The overall design of both experiments was to use commercially available, high purity, salts with polyethylene moderator and HEU plates to configure a critical assembly. Three experiments are planned for TEX-Li using encapsulated lithium carbonate (Li 2 CO 3 ), with natural 6 Li abundance. For all experiments, the highly enriched uranium (HEU) Jemima plates will be used as fissile material. Multiple layers will be stacked together with encapsulated Li 2 CO 3 alternated with polyethylene in standard configurations. Standard stacking was found to be optimal in matching the different sensitivity profiles provided by the Y-12 models. Three configurations are proposed with varying polyethylene moderation and a constant 1/4” absorber thickness. The first uses 11 layers of 5/4” polyethylene, the second uses 9 layers of 3/4” polyethylene, and the third uses 10 layers of 1/2” polyethylene. Calculations showed that some alternative forms of lithium-based materials provided slightly less-optimal sensitivity profiles when compared to lithium carbonate but come with other drawbacks. These alternatives included lithium aluminate (LiAlO 2 ), Aluminum-2050 alloy, Aluminum-8090, Aluminum-2095, lithium hydride (LiH), and lithium fluoride (LiF). Lithium aluminate and aluminum-2050 provided comparable sensitivity profiles when compared to lithium carbonate and can be used instead if lithium carbonate cannot be readily procured. After a broad material study, lithium carbonate outperformed any alternative material with a balance in affordability and workability. The assessment of experimental uncertainties of the non-absorber and absorber components was predicted to be 0.00089 and 0.00093 Δk eff , respectively. The largest uncertainties may be reduced with precision dimensional inspection of the components. Many of the parts and equipment for IER 575 have already been fabricated or procured for previous projects and therefore do not contribute significantly to the overall cost of this experiment. This includes the Jemima plates and Comet critical assembly machine, which are existing NCSP assets, as well as the aluminum platen and polyethylene reflector rings, which were fabricated and authorized for the TEX experiment involving HEU with polyethylene. Lithium carbonate containers will be procured by LANL and will be filled by LLNL. The total material costs for TEX-Li experiments are estimated to be on the order of $\$$47,400. Precision inspection, including dimensional, mass, density, and impurity, is recommended for all components for an estimated cost of $\$$12,000.

35Cl↗
DOE OSTI2025

The Risk Assessment Information System Compendium of Ecological Screening Benchmarks for Chemicals and Radionuclides (2025) (Volume VIII – Chemicals for Sediment: General)

Volume VIII presents the chemical benchmarks for sediment: general. The benchmarks here are applicable to both freshwater sediments and marine sediments or where no differentiation was made by the source. These values are not repeated in the freshwater sediment and marine sediment categories. Likewise, values listed in the freshwater sediment and marine sediment categories are not repeated in the generic categories. Consult Volume I for information on the sources.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

The Risk Assessment Information System Compendium of Ecological Screening Benchmarks for Chemicals and Radionuclides (2025) (Volume V – Chemicals for Surface Water: General)

Volume V presents the chemical benchmarks for surface water: general. The benchmarks here are applicable to both fresh water and marine or where no differentiation was made by the source. These values are not repeated in the fresh water and marine categories. Likewise, values listed in the fresh water and marine categories are not repeated in the generic categories. Consult Volume I for information on the sources.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

ORNL Testing of Multiple Graphite Benchmarks [Slides]

Integral criticality safety and reactor physics benchmark experiments from the International Handbook of Evaluated Criticality Safety Benchmark Experiments (ICSBEP Handbook) and the International Handbook of Reactor Physics Experiments (IRPhE Handbook) are essential for nuclear data validation and testing

ENDF↗
DOE OSTI2025

Benchmark Exercise Report for Experimental Study of Bubble Scrubbing in Water Coolant Pool

Mechanistic assessments of radionuclide release during postulated accidents are expected to be included in advanced reactor license applications. The mechanistic source term (MST) provides an opportunity for vendors to realistically evaluate the radiological consequences of an incident, and may aid in justifying reduced emergency planning zones and plant sites. However, the development of MSTs for advanced nuclear reactors is challenging because there are numerous phenomena that can affect the transport and retention of radionuclides. As part of a trial MST assessment for a metal-fueled, pool-type sodium cooled fast reactor (SFR), led by Argonne National Laboratory, a simplified radionuclide transport code (SRT code) was developed, which includes models to estimate the quantity of fission product aerosols scrubbed in the sodium pool during postulated accident scenarios. In a pool-type SFR, when fission products are released into the coolant pool due to failure of fuel pins, most of the radionuclides are scrubbed by the coolant pool, but some have the potential to migrate to the cover gas region through entrainment within gas bubbles. The SRT code contains a model that evaluates this scrubbing behavior and calculates the fraction of fission product aerosols that reach the cover gas. Due to a lack of available validation data for sodium pool scrubbing, the U.S. Department of Energy funded an experiment at the University of Wisconsin-Madison to measure aerosol scrubbing by injecting air bubbles containing aerosol into a coolant pool. Prior to performing an experiment with liquid sodium, a water loop experiment was performed. Their experiment evaluated the effect of changing the aerosol size, aerosol density, aerosol concentration, bubble size, and pool depth on the aerosol scrubbing efficiency of the pool. In this benchmark experiment, the base tests were conducted by repeated tests of isolated bubbles. Afterwards, more prototypic tests with bubble swarms were performed to evaluate the interactions between the bubbles. The bubble swarm test was able to confirm that a larger amount of aerosol scrubbing occurred than the single bubble test. It was also confirmed that as the bubble size, aerosol density, and pool height increase, the extent of pool scrubbing also increases and does not change with the aerosol concentration. In addition, since the degree of scrubbing is the lowest at aerosol sizes between 0.01 and 1 μm, that is, the largest amount of aerosol is emitted, it was confirmed that the analysis of this size in MST is the most important. This benchmark experiment informs the direction of future sodium experiments.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Artificial Intelligence Benchmarking

AI benchmarking is a method for evaluating the effectiveness of an AI model using a set of standardized metrics, for example, high school-level math exams. These benchmarks and their results will enable ranking various AI models based on their effectiveness in performing a specific task.

Krishnan, Anjay [Fermilab]↗
DOE OSTI2026

Final Design for Thermal/Epithermal eXperiments (TEX) with Lithium Absorbers to Provide Validation Benchmarks for Y-12 Electrorefining Facility (IER 575, CED-2 Report)

One of the main goals of the Thermal/Epithermal eXperiments (TEX) project is to use existing Nuclear Criticality Safety Program (NCSP) assets to create critical experiment plutonium and uranium test beds for materials important to criticality safety that have insufficient benchmark evaluations. The plutonium test bed experiments were completed in 2018 and are published in the 2020 edition of the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook. The uranium test bed assemblies were completed in 2023 and accepted in the 2024 edition of the ICSBEP Handbook.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Benchmarking of ENDF/B-VIII.1 Thermal Scattering Library for Hydrogen

In correctly characterizing the energy and momentum transfer at thermal and cold energies between neutrons and its interacting medium, thermal scattering libraries, which details energy states due to the intra- and inter-molecular bond effects for the medium materials, are applied in place of free-gas cross section libraries in a particle transport simulation code. They are essential to the neutron performance of a neutron facility like SNS, where thermalized neutrons from 20 K liquid hydrogen and ambient (~300 K) water are transported to the beamlines for neutron scattering experiments in studying materials. Recently, a new version of thermal scattering library for parahydrogen and orthohydrogen at 14-20 K was developed and to be released in ENDF/B-VIII.1. It is, therefore, important to benchmark its impacts on the prediction of moderator performance due to the updates in the thermal scattering library. In this study, the recent ENDF/B-VIII.1 thermal scattering library was compared to the current ENDF/B-VII.1 one in the neutron performance calculations for the decoupled and coupled hydrogen moderators at SNS under theorized and real working conditions. In addition, the predictions using both thermal scattering libraries were benchmarked to the measurements of moderator performance. The consistency between the libraries was observed mostly for parahydrogen and the difference in orthohydrogen at cold neutron energies was noted.

Lu, Wei↗
DOE OSTI2023

Unifying and benchmarking state-of-the-art quantum error mitigation techniques

Error mitigation is an essential component of achieving a practical quantum advantage in the near term, and a number of different approaches have been proposed. In this work, we recognize that many state-of-the-art error mitigation methods share a common feature: they are data-driven, employing classical data obtained from runs of different quantum circuits. For example, Zero-noise extrapolation (ZNE) uses variable noise data and Clifford-data regression (CDR) uses data from near-Clifford circuits. We show that Virtual Distillation (VD) can be viewed in a similar manner by considering classical data produced from different numbers of state preparations. Observing this fact allows us to unify these three methods under a general data-driven error mitigation framework that we call UNIfied Technique for Error mitigation with Data (UNITED). In certain situations, we find that our UNITED method can outperform the individual methods (i.e., the whole is better than the individual parts). Specifically, we employ a realistic noise model obtained from a trapped ion quantum computer to benchmark UNITED, as well as other state-of-the-art methods, in mitigating observables produced from random quantum circuits and the Quantum Alternating Operator Ansatz (QAOA) applied to Max-Cut problems with various numbers of qubits, circuit depths and total numbers of shots. We find that the performance of different techniques depends strongly on shot budgets, with more powerful methods requiring more shots for optimal performance. For our largest considered shot budget (10 10 ), we find that UNITED gives the most accurate mitigation. Hence, our work represents a benchmarking of current error mitigation methods and provides a guide for the regimes when certain methods are most useful.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

HamLib: A library of Hamiltonians for benchmarking quantum algorithms and hardware

In order to characterize and benchmark computational hardware, software, and algorithms, it is essential to have many problem instances on-hand. This is no less true for quantum computation, where a large collection of real-world problem instances would allow for benchmarking studies that in turn help to improve both algorithms and hardware designs. To this end, here we present a large dataset of qubit-based quantum Hamiltonians. The dataset, called HamLib (for Hamiltonian Library), is freely available online and contains problem sizes ranging from 2 to 1000 qubits. HamLib includes problem instances of the Heisenberg model, Fermi-Hubbard model, Bose-Hubbard model, molecular electronic structure, molecular vibrational structure, MaxCut, Max- k -SAT, Max- k -Cut, QMaxCut, and the traveling salesperson problem. The goals of this effort are (a) to save researchers time by eliminating the need to prepare problem instances and map them to qubit representations, (b) to allow for more thorough tests of new algorithms and hardware, and (c) to allow for reproducibility and standardization across research studies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Posiform planting: generating QUBO instances for benchmarking

We are interested in benchmarking both quantum annealing and classical algorithms for minimizing quadratic unconstrained binary optimization (QUBO) problems. Such problems are NP-hard in general, implying that the exact minima of randomly generated instances are hard to find and thus typically unknown. While brute forcing smaller instances is possible, such instances are typically not interesting due to being too easy for both quantum and classical algorithms. In this contribution, we propose a novel method, called posiform planting , for generating random QUBO instances of arbitrary size with known optimal solutions, and use those instances to benchmark the sampling quality of four D-Wave quantum annealers utilizing different interconnection structures (Chimera, Pegasus, and Zephyr hardware graphs) and the simulated annealing algorithm. Posiform planting differs from many existing methods in two key ways. It ensures the uniqueness of the planted optimal solution, thus avoiding groundstate degeneracy, and it enables the generation of QUBOs that are tailored to a given hardware connectivity structure, provided that the connectivity is not too sparse. Posiform planted QUBOs are a type of 2-SAT boolean satisfiability combinatorial optimization problems. Our experiments demonstrate the capability of the D-Wave quantum annealers to sample the optimal planted solution of combinatorial optimization problems with up to 5, 627 qubits.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Validation of time-dependent shift using the pulsed sphere benchmarks

The detailed behavior of neutrons in a rapidly changing time-dependent physical system is a challenging computational physics problem, particularly when using Monte Carlo methods on heterogeneous high-performance computing architectures. A small number of algorithms and code implementations have been shown to be performant for time-independent (fixed source and k-eigenvalue) Monte Carlo, and there are existing simulation tools that successfully solve the time-dependent Monte Carlo problem on smaller computing platforms. To bridge this gap, a time-dependent version of ORNL’s Shift code has been recently developed. Shift’s history-based algorithm on CPUs, and its event-based algorithm on GPUs, have both been observed to scale well to very large numbers of processors, which motivated the extension of this code to solve time-dependent problems. The validation of this new capability requires a comparison with time-dependent neutron experiments. Lawrence Livermore National Laboratory’s (LLNL) pulsed sphere benchmark experiments were simulated in Shift to validate both the time-independent as well as new time-dependent features recently incorporated into Shift. A suite of pulsed-sphere models was simulated using Shift and compared to the available experimental data and simulations with MCNP. Overall results indicate that Shift accurately simulates the pulsed sphere benchmarks, and that the new time-dependent modifications of Shift are working as intended. Validated exascale neutron transport codes are essential for a wide variety of future multiphysics applications.

Palmer, Camille J.↗
DOE OSTI2025

Benchmarking a Tunable Quantum Neural Network on Trapped-Ion and Superconducting Hardware

We implement a quantum generalization of a neural network on trapped-ion and IBM superconducting quantum computers to classify MNIST images, a common benchmark in computer vision. The network feedforward involves qubit rotations whose angles depend on the results of measurements in the previous layer. The network is trained via simulation, but inference is performed experimentally on quantum hardware. The classical-to-quantum correspondence is controlled by an interpolation parameter, $a$, which is zero in the classical limit. Increasing $a$ introduces quantum uncertainty into the measurements, which is shown to improve network performance at moderate values of the interpolation parameter. We then focus on particular images that fail to be classified by a classical neural network but are detected correctly in the quantum network. For such borderline cases, we observe strong deviations from the simulated behavior. We attribute this to physical noise, which causes the output to fluctuate between nearby minima of the classification energy landscape. Such strong sensitivity to physical noise is absent for clear images. We further benchmark physical noise by inserting additional single-qubit and two-qubit gate pairs into the neural network circuits. Our work provides a springboard toward more complex quantum neural networks on current devices: while the approach is rooted in standard classical machine learning, scaling up such networks may prove classically non-simulable and could offer a route to near-term quantum advantage.

FOS: Physical sciences↗