Search NASA⌕ Search

SEARCH · Search NASA

Results for “BENCHMARK”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

MUSiC: Critical Experiment with Bare Highly Enriched Uranium Shells Benchmark

The measurements of the Uranium Subcritical and Critical (MUSiC) experiment were performed from December 2020 through April 2021. The purpose of the experiment was to test a variety of multiplicity detection systems for a large range of multiplication values and to see how well they perform as the k eff of the MUSiC series of experiments approached the delayed critical window. Out of the ten configurations that were studied, two of them were super critical and were recently approved as benchmarks and they will be included in the International Criticality Safety Benchmark Evaluation Project (ICBESP) handbook. For each of the two experiments, uncertainties were evaluated six categories: (1) critical measurement, (2) mass, (3) composition, (4) positioning, (5) dimensions, and (6) temperature. The largest contribution to the overall uncertainty in each of the cases was due to the uncertainties in the dimensions of the HEU Rocky Flats shells. Simplified detailed models were created with an average bias of -130.5 pcm due to the removal various components surrounding the experiment. For MCNP6.2 models using ENDF-VIII.0 neutron cross section data, the average difference between the calculated and benchmark k effs was -54 pcm for the detailed model and -102 pcm for the simplified model.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

DeepSurveySim: Simulation Software and Benchmark Challenges for Astronomical Observation Scheduling

Modern astronomical surveys have multiple competing scientific goals. Optimizing the observation schedule for these goals presents significant computational and theoretical challenges, and state-of-the-art methods rely on expensive human inspection of simulated telescope schedules. Automated methods, such as reinforcement learning, have recently been explored to accelerate scheduling. However, there do not yet exist benchmark data sets or user-friendly software frameworks for testing and comparing these methods. We present DeepSurveySim -- a high-fidelity and flexible simulation tool for use in telescope scheduling. DeepSurveySim provides methods for tracking and approximating sky conditions for a set of observations from a user-supplied telescope configuration. We envision this tool being used to produce benchmark data sets and for evaluating the efficacy of ground-based telescope scheduling algorithms, particularly for machine learning algorithms that would suffer in efficacy if limited to real data for training.We introduce three example survey configurations and related code implementations as benchmark problems that can be simulated with DeepSurveySim.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

RELAP5-3D Solutions to Exercise 1 of the OECD-NEA HTTF Benchmark

Discussion of RELAP5-3D solutions to Exercise 1 of the OECD-NEA High Temperature Test Facility (HTTF) Benchmark. Results from ongoing benchmark activities supported by the Advanced Reactor Technologies - Gas-Cooled Reactor Program with use of the High Performance Computing Center at Idaho National Laboratory. Also discussion of Ultra Safe Nuclear Corporation and BWXT plans for the decade, and issues of modern thermal hydraulic tools for prismatic HTGR modeling. Open discussion of verification and validation and ART-GCR campaign to spearhead the development of an HTGR thermal hydraulics benchmark based on the HTTF.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

RELAP5-3D Modeling in the OECD-NEA HTTF Benchmark

Prismatic gas-cooled reactors are a technologically mature reactor concept of interest for near- to mid-term deployment. To accelerate the deployment of these reactors, the DOE's Advanced Reactor Technologies Gas-Cooled Reactor campaign is spearheading a thermal hydraulics code validation benchmark based on the High Temperature Test Facility at Oregon State University. This presentation provides an introduction to that benchmark and an overview of some of the RELAP5-3D validation activities stemming from that benchmark.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

A Benchmarking Framework for Evaluating Large Language Model Capabilities in Nuclear Reactor Safety Applications

Large language models (LLMs) are increasingly capable of answering technical questions, synthesizing domain knowledge, and supporting engineering workflows. For nuclear science and engineering, these capabilities require careful, domain-specific evaluation before they can be credibly incorporated into safety-related activities, regulatory review, or technical decision support. This paper presents preliminary results from benchmarking framework for evaluating LLM capabilities in nuclear contexts. The framework is organized into three evaluation categories: nuclear fundamentals, general dual-use knowledge, and plant specific knowledge. These categories are intended to distinguish general nuclear engineering competence from broader technical reasoning and more context-dependent nuclear knowledge. Initial evaluations focus on nuclear fundamentals using questions representative of the knowledge expected of a nuclear professional engineer. Results indicate that contemporary frontier models perform at a high level and substantially exceed the performance of older model generations, with some models approaching saturation of the current benchmark. These findings suggest both the rapid improvement of LLM capabilities in specialized technical domains and the need for more discriminating evaluation methods. The paper presents the benchmark structure, preliminary model-comparison results, and ongoing work. This work supports development of verifiable, responsible, and safety-conscious methods for assessing AI systems in nuclear engineering applications.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗
DOE OSTI2024

IER 480: TEX-Pu Benchmark (PU-MET-THERM-004) to Test Polyethylene and Lucite Thermal Scattering Laws [Slides]

This presentation finds that PMT-004 has four new benchmark cases highly sensitive to PE TSL (2 cases) and PMMA TSL (2 Cases). PE cases were well predicted using MCNP6.2 and ENDF/B-VIII.0. PMMA cases overpredicted by approximately 0.6-0.7% in k eff at 20°C. Accepted into 2023 version of the ICSBEP Handbook. Temperature had a large impact on reactivity of the critical configuration. Implications for validation work for thermal cases- need to adjust TSL data to correct temperature as it can have hundreds of pcm effects for a few °C. Future thermal experiments should try and measure reactivity at multiple temperatures to aid in data testing and benchmark adjustment.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Reproducible benchmark for the SNAP 8 experimental reactor at dry conditions

Here, this work provides fully reproducible benchmark models of the Systems for Nuclear Auxiliary Power (SNAP) 8 Experimental Reactor. Validation benchmarks of the criticality configuration experiments under dry conditions with no coolant are presented. In addition, relevant reactivity worth experiments, and measurement of power distributions are validated and presented here. Discrepancies between modeled and experimental results are at most 110 pcm for fuel and poison worths, and critical configurations that do not manipulate control elements are within 50 pcm. Larger discrepancies found in reflector element worth experiments are due to experimental methodology which was noted as being simplified due to limited calculational capabilities at the time, while discrepancy in the control element manipulated critical configuration is due the absence of structural components for modeling simplification. Power distributions closely follow what was observed in experiment with expected peaking in reflecting elements. All models are well documented with cited references describing and justifying assumptions, simplifications, and adjustments to reproduce the models with any nuclear safety codes; model inputs and outputs are stored in the snapReactors GitHub repository.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2026

Reproducible benchmark for the SNAP 8 experimental reactor at operating conditions

This work presents fully reproducible multiphysics benchmark models of the Systems for Nuclear Auxiliary Power (SNAP) 8 Experimental Reactor at operating conditions with coolant flow. Wet experiment (with coolant, at power) validation benchmarks are presented using both deterministic (Serpent-Griffin) and Monte-Carlo (OpenMC-Cardinal) multiphysics frameworks coupled with thermal-hydraulic solvers in MOOSE. Reactivity coefficient measurements including fuel temperature, isothermal temperature, and power coefficients show good agreement with experiments, with discrepancies within experimental uncertainty. Reactivity worth experiments for coolant, samarium, and xenon poisoning are reproduced with differences under 200 pcm. Comparison between Serpent-Griffin and OpenMC-Cardinal frameworks reveal multiphysics coupling introduces positive reactivity effects (100-200 pcm) compared to uniform temperature and density fields at nominal operating conditions. Comparison between Serpent-Griffin and reference Serpent solution shows that power distributions maintain consistent radial and axial peaking behavior. All models, assumptions, thermophysical and thermomechanical properties, and material definitions are thoroughly documented with cited references; model inputs and model generating scripts are stored in the snapReactors GitHub repository.

SNAP↗
DOE OSTI2021

Benchmarking the multiconfigurational Hartree method by the exact wavefunction of two harmonically trapped bosons with contact interaction

Highlights: • Comparison of exact and numerical single-particle density matrices. • Benchmark for multiconfigurational Hartree. • Convergence for large number of orbitals. We consider two bosons in a one-dimensional harmonic trap, interacting by a contact potential, and compare the exact solution of this problem to a self-consistent numerical solution by using the multiconfigurational time-dependent Hartree (MCTDH) method. We thereby benchmark the predictions of the MCTDH method with a few-body problem that has an analytical solution for the most commonly experimentally realized interaction potential in ultracold quantum gases. It is found that exact ground state energy and first order correlations are accurately reproduced by MCTDH up to the intermediate dimensionless coupling strengths corresponding to typical background scattering lengths of magnetically trapped ultracold dilute Bose gases. For larger couplings, established for example by (a combination of) Feshbach resonances and optical trapping, the MCTDH approach overestimates the depth of the trap-induced correlation dip of first order correlations in position space, as well as overestimates the fragmentation, defined as the average relative occupation of orbitals other than the energetically lowest one. We anticipate that qualitatively similar features in the correlation function may arise for larger particle numbers, paving the way for a quantitative assessment of the accuracy of MCTDH by experiments with ultracold atoms.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Electrocatalytic benchmarking of ruthenium-based bimetallic anodes for the electrocatalytic oxidation of biomass-derived wastewater

In this paper, we report on the synthesis, characterization, and use of ruthenium oxide (RuO 2 ) doped with a secondary metal (M2) to enhance electrochemical activity and stability for the electrocatalytic oxidation (ECO) of biomass-derived wastewaters. We used different electrochemical methods such as cyclic voltammetry (CV), electrochemical surface area (ECSA), and Tafel analysis as well as physical characterization such as grazing incidence X-ray diffraction, X-ray photoelectron spectroscopy, and scanning electron microscopy to understand how the introduction of M2 affects electrochemical performance. Our results show that including an M2 improves the ECO performance regardless of the composition of the electrolyte. Specifically, we saw increase in ECSA, which could be due to enhanced charge transfer for the pH ranges evaluated. Furthermore, the introduction of organic compounds in wastewater generated during the hydrothermal liquefaction of food waste affected the ECO performance differently, depending on M2, the electrolyte composition, and anodic half-cell potential, highlighting the need to properly control the reaction conditions when testing and characterizing the electrocatalysts under different reaction regimes. We developed an in situ electrocatalytic benchmarking protocol, Boruah – Lopez-Ruiz – Strange (BLoRS), to quickly assess if the presence of M2 improves the ECO performance; thus, saving time and resources in ex situ characterization, testing, and product analysis. This foundational work provides the basis for characterization and benchmarking of electrodes of the ECO of organic compounds.

Electrocatalysis↗
DOE OSTI2020

Phase field benchmark problems targeting fluid flow and electrochemistry

Here, we continue our development of phase field model benchmark problems with the addition of a third set, complimenting our previously developed problems for diffusion, precipitation, dendritic growth and linear elasticity. These benchmark problems are being jointly developed by the Center for Hierarchical Materials Design (CHiMaD) and the National Institute of Standards and Technology (NIST) along with input from the phase field community. The first problem in this third set targets Stokes flow, with a particular emphasis on flow around an obstruction placed asymmetrically in the domain. While Stokes flow is not traditionally in the canon of phase field problems, it is a class of problems gaining importance in areas such as filtration and water purification. The second problem deals with coupled Cahn-Hilliard diffusion and electrostatic forces, which is an important area in energy storage and battery sciences. We present our own solutions and discuss sources of numerical errors for the Stokes problem as well as simple checks to avoid fundamental issues in the coupled diffusion-electrostatics problem. The latter problem contains some subtleties that we expand on in an Appendix.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Benchmarking of massively parallel phase-field codes for directional solidification

We present a detailed benchmark comparing two state-of-the-art phase-field implementations for simulating alloy solidification under experimentally relevant conditions. The study investigates the directional solidification of Al-3wt%Cu under high-velocity solidification conditions and SCN-0.46wt% camphor under microgravity conditions from National Aeronautics and Space Administration (NASA) DECLIC-DSI-R experiments. Both codes, one employing finite-difference discretization with uniform mesh and GPU-acceleration (GPU-PF) and the other one employing finite-element discretization with adaptive-mesh and CPU-parallelization (PRISMS-PF), solve the same quantitative phase-field formulation that incorporates an anti-trapping current for the solidification of dilute alloys. We evaluate the predictions of each code for dendritic morphology, primary spacing, and tip dynamics in both 2D and 3D, as well as their numerical convergence and computational performance. While existing benchmark problems have primarily focused on simplified or small-scale simulations, they do not reflect the computational and modeling challenges posed by employing experimentally relevant time and length scales. Our results provide a practical framework for assessing phase-field code performance as well as validating and facilitating their application in integrated computational materials engineering (ICME) workflows that require integration with realistic experimental data.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Surrogate model evaluation and building energy benchmarking for commercial buildings

Building energy consumption benchmarking involves challenges associated with various energy patterns for different building types; heating, ventilating, and air-conditioning (HVAC) system types; and climates. Given significant variation in energy use patterns, accurate prediction of long-term energy use using surrogate models remains challenging. Multiple linear regression (MLR) is commonly used for building energy benchmarking because of its simple structure; however, it lacks accuracy compared to other black-box models. Although many studies have compared surrogate models and offer guidance on model selection based on metrics, they do not provide detailed analysis on improving the surrogate model accuracy. In this paper, we implement a surrogate model using polynomial ridge regression (i.e., MLR with interaction terms combined with ridge regularization) for small office and retail strip mall buildings across six HVAC system types and all climate zones, for electricity and natural gas in baseline and proposed scenarios. A simulation workflow is developed using OpenStudio TM /EnergyPlus TM to generate simulation data using measures over a wide range of efficiency inputs. Enhancements based on statistical insights are used for improving the model accuracy using filters, input transformations, and change points. Surrogate models achieved average coefficient of variation of the root mean squared error (CVRMSE) values of 2.17, 1.06, 2.05, and 3.26 for proposed electricity, proposed natural gas, baseline electricity, and baseline natural gas, respectively, with enhancements reducing CVRMSE by an average of 14.9% across all combinations. We provide model interpretation via Shapley additive explanations to determine which input variables most influence energy consumption and provide supportive arguments for enhancements.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Evaluating three evapotranspiration estimates from model of different complexity over China using the ILAMB benchmarking system

The land surface models range in complexity of terrestrial evapotranspiration, yet it is unknown how model complexity translates to accuracy of modeled evapotranspiration estimates. Here, we use the International Land Model Benchmarking system to assess ET estimates from three models of varying complexity driven by the same forcing datasets: an earth system model, a terrestrial biosphere model, and a stand-alone ET model. The performance assessment includes both temporal and spatial evaluation, and different plant functional types across China. Our results indicate that the most complex model, an earth system model, performed best against the benchmarking datasets and metrics. Terrestrial biosphere model performed best in simulating inter-annual variability of ET, while earth system model performed best in simulating the seasonal cycle. The more complex models (earth system model and terrestrial biosphere model) perform better in forest, shrub and crop ecosystems, while the simpler model (stand-alone ET model) perform better in grass ecosystems. Our study demonstrates the impact of model complexity on ET estimates and highlights directions for future ET model improvements.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Practical Introduction to Benchmarking and Characterization of Quantum Computers

Rapid progress in quantum technology has transformed quantum computing and quantum information science from theoretical possibilities into tangible engineering challenges. Breakthroughs in quantum algorithms, quantum simulations, and quantum error correction are bringing useful quantum computation closer to fruition. These remarkable achievements have been facilitated by advances in quantum characterization, verification, and validation (QCVV). QCVV methods and protocols enable scientists and engineers to scrutinize, understand, and enhance the performance of quantum information-processing devices. In this tutorial, we review the fundamental principles underpinning QCVV, and introduce a diverse array of QCVV tools used by quantum researchers. We define and explain QCVV’s core models and concepts—quantum states, measurements, and processes—and illustrate how these building blocks are leveraged to examine a target system or operation. We survey and introduce protocols ranging from simple qubit characterization to advanced benchmarking methods. Along the way, we provide illustrated examples and detailed descriptions of the protocols, highlight the advantages and disadvantages of each, and discuss their potential scalability to future large-scale quantum computers. This tutorial serves as a guidebook for researchers unfamiliar with the benchmarking and characterization of quantum computers, and also as a detailed reference for experienced practitioners.

open quantum systems & decoherence↗
DOE OSTI2022

A round-robin approach provides a detailed assessment of biomolecular small-angle scattering data reproducibility and yields consensus curves for benchmarking

Through an expansive international effort that involved data collection on 12 small-angle X-ray scattering (SAXS) and four small-angle neutron scattering (SANS) instruments, 171 SAXS and 76 SANS measurements for five proteins (ribonuclease A, lysozyme, xylanase, urate oxidase and xylose isomerase) were acquired. From these data, the solvent-subtracted protein scattering profiles were shown to be reproducible, with the caveat that an additive constant adjustment was required to account for small errors in solvent subtraction. Further, the major features of the obtained consensus SAXS data over the q measurement range 0–1 Å −1 are consistent with theoretical prediction. The inherently lower statistical precision for SANS limited the reliably measured q -range to <0.5 Å −1 , but within the limits of experimental uncertainties the major features of the consensus SANS data were also consistent with prediction for all five proteins measured in H 2 O and in D 2 O. Thus, a foundation set of consensus SAS profiles has been obtained for benchmarking scattering-profile prediction from atomic coordinates. Additionally, two sets of SAXS data measured at different facilities to q > 2.2 Å −1 showed good mutual agreement, affirming that this region has interpretable features for structural modelling. SAS measurements with inline size-exclusion chromatography (SEC) proved to be generally superior for eliminating sample heterogeneity, but with unavoidable sample dilution during column elution, while batch SAS data collected at higher concentrations and for longer times provided superior statistical precision. Careful merging of data measured using inline SEC and batch modes, or low- and high-concentration data from batch measurements, was successful in eliminating small amounts of aggregate or interparticle interference from the scattering while providing improved statistical precision overall for the benchmarking data set.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

QASMBench: A Low-Level Quantum Benchmark Suite for NISQ Evaluation and Simulation

The rapid development of quantum computing (QC) in the NISQ era urgently demands a low-level benchmark suite and insightful evaluation metrics for characterizing the properties of prototype NISQ devices, the efficiency of QC programming compilers, schedulers and assemblers, and the capability of quantum system simulators in a classical computer. In this work, we fill this gap by proposing a low-level, easy-to-use benchmark suite called QASMBench based on the OpenQASM assembly representation. It consolidates commonly used quantum routines and kernels from a variety of domains including chemistry, simulation, linear algebra, searching, optimization, arithmetic, machine learning, fault tolerance, cryptography, and so on, trading-off between generality and usability. To analyze these kernels in terms of NISQ device execution, in addition to circuit width and depth, we propose four circuit metrics including gate density, retention lifespan, measurement density, and entanglement variance, to extract more insights about the execution efficiency, the susceptibility to NISQ error, and the potential gain from machine-specific optimizations. Applications in QASMBench can be launched and verified on several NISQ platforms, including IBM-Q, Rigetti, IonQ and Quantinuum. For evaluation, we measure the execution fidelity of a subset of QASMBench applications on 12 IBM-Q machines through density matrix state tomography, comprising 25K circuit evaluations. In addition we also compare the fidelity of executions among the IBM-Q machines, the IonQ QPU and the Rigetti Aspen M-1 system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

MCS solutions for the TVA Watts Bar Unit 1 multi-physics depletion benchmark

The high-fidelity solution for exercises 1-3 of the recently developed TVA Watts Bar Unit 1 multi-physics and multi-cycle depletion benchmark using MCS was coupled with two different thermal-hydraulics (TH) codes: TH1D and COBRA-TF (CTF), and their solutions were compared. The solutions from MCS/CTF and MCS/TH1D for typical Pressurized Water Reactor (PWR) problems are quite close. And to evaluate MCS solutions for the benchmark, their results were also compared against measured data as well as publicly available high-fidelity solutions. The MCS criticality solutions at Hot Zero Power (HZP) deviate within 50 pcm against the reference solutions. While for the critical boron concentration (CBC) search calculations, the difference on the MCS calculated CBC is about 7 ppm, and the root-mean-squared (RMS) errors for the assembly power distribution and outlet coolant temperature are less than 0.5% and 1 Celsius degree respectively. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗