Search NASA⌕ Search

SEARCH · Search NASA

Results for “model benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Global evaluation of terrestrial biogeochemistry in the Energy Exascale Earth System Model (E3SM) and the role of the phosphorus cycle in the historical terrestrial carbon balance

Abstract. The importance of carbon (C)–nutrient interactions to the prediction of future C uptake has long been recognized. The Energy Exascale Earth System Model (E3SM) land model (ELM) version 1 is one of the few land surface models that include both N and P cycling and limitation (ELMv1-CNP). Here we provide a global-scale evaluation of ELMv1-CNP using the International Land Model Benchmarking (ILAMB) system. We show that ELMv1-CNP produces realistic estimates of present-day carbon pools and fluxes. Compared to simulations with optimal P availability, simulations with ELMv1-CNP produce better performance, particularly for simulated biomass, leaf area index (LAI), and global net C balance. We also show ELMv1-CNP-simulated N and P cycling is in good agreement with data-driven estimates. We compared the ELMv1-CNP-simulated response to CO2 enrichment with meta-analysis of observations from similar manipulation experiments. We show that ELMv1-CNP is able to capture the field-observed responses for photosynthesis, growth, and LAI. We investigated the role of P limitation in the historical balance and show that global C sources and sinks are significantly affected by P limitation, as the historical CO2 fertilization effect was reduced by 20 % and C emission due to land use and land cover change was 11 % lower when P limitation was considered. Our simulations suggest that the introduction of P cycle dynamics and C–N–P coupling will likely have substantial consequences for projections of future C uptake.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Shattered pellet penetration in low and high energy plasmas on DIII-D

Shattered pellet injection (SPI) has been adopted as the baseline disruption mitigation system for ITER, as the radiative payload penetration into DIII-D plasmas from SPI is superior to those using the massive gas injection (MGI) method. Because of the substantial differences in the energy content of ITER plasma and those in present experiments, reliable 3D MHD modeling, benchmarked against present experiments is needed to project to ITER plasmas. In support of these needs, the depth of SPI fragment penetration in DIII-D plasmas was investigated by injecting SPI into two discharges with vastly different energy content and pedestal height. 400 Torr-L pure Ne fragmented pellets at a velocity of about 200 m s−1 were injected into a 0.2 MJ L-mode discharge and a 2 MJ super H-mode discharge. Results show deep penetration of SPI fragments into low-energy plasmas in DIII-D. SPI fragment penetration is reduced as the plasma energy content increases, with some discharges exhibiting penetration that is confined to the outer regions of the plasma. The injected SPI fragments are also spread out over a distance of about 20 cm, which results in some fragments arriving near the end of or after the thermal quench is over.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

A Strategy for Low-Mass Dark Matter Searches with Cryogenic Detectors in the SuperCDMS SNOLAB Facility

The SuperCDMS Collaboration is currently building SuperCDMS SNOLAB, a dark matter search focused on nucleon-coupled dark matter in the 1-5 GeV mass range. Looking to the future, the Collaboration has developed a set of experience-based upgrade scenarios, as well as novel directions, to extend the search for dark matter using the SuperCDMS technology in the SNOLAB facility. The experienced-based scenarios are forecasted to probe many square decades of unexplored dark matter parameter space below 5 GeV, covering over 6 decades in mass: 1-100 eV for dark photons and axion-like particles, 1-100 MeV for dark-photon-coupled light dark matter, and 0.05-5 GeV for nucleon-coupled dark matter. They will reach the neutrino fog in the 0.5-5 GeV mass range and test a variety of benchmark models and sharp targets. The novel directions involve greater departures from current SuperCDMS technology but promise even greater reach in the long run, and their development must begin now for them to be available in a timely fashion. The experienced-based upgrade scenarios rely mainly on dramatic improvements in detector performance based on demonstrated scaling laws and reasonable extrapolations of current performance. Importantly, these improvements in detector performance obviate significant reductions in background levels beyond current expectations for the SuperCDMS SNOLAB experiment. Given that the dominant limiting backgrounds for SuperCDMS SNOLAB are cosmogenically created radioisotopes in the detectors, likely amenable only to isotopic purification and an underground detector life-cycle from before crystal growth to detector testing, the potential cost and time savings are enormous and the necessary improvements much easier to prototype.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Investigation of the Impact of TSL Data Libraries and Geometry Variations on the MSRE Benchmark

The increasing demand for green, low-carbon energy solutions has amplified the focus on advanced nuclear reactor technologies. Among these, Molten Salt Reactors (MSRs) have been spotlighted because of their special features. Historically, Oak Ridge National Laboratory (ORNL) started the Molten Salt Reactor Experiment (MSRE) in 1956. Later, ORNL and the University of California, Berkeley (UCB) jointly developed the MSRE benchmark. This was then reviewed by the International Reactor Physics Experiment Evaluation Project (IRPhEP) committee and added to their 2019 handbook. In the current study, the benchmark model was recreated via the Serpent code. The calculated effective multiplication factor was 1.02087 ± 0.00019, which gives a deviation of about 2000 pcm compared to the benchmark/experimental result. At North Carolina State University (NCSU), recent evaluations were conducted on thermal scattering cross sections for molten salt FLiBe and 20% porous nuclear graphite, anticipated for incorporation in the ENDF/B-VIII.1 library. The influences of these thermal scattering law (TSL) data and geometry changes on the multiplication factor were examined using the ENDF/B-VII.1 and ENDF/B-VIII.0 libraries. Introducing the TSL data for FLiBe and 20% porous graphite resulted in an effective multiplication factor of +270 pcm compared to the reference value obtained with ENDF/B-VII.1 in this study. The adjustment of the reactor vessel dimensions resulted in decrease of -448 pcm in the calculated effective multiplication factor.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

First Search for Dark-Trident Processes Using the MicroBooNE Detector

This thesis presents a first search for dark-trident scattering in a neutrino beam using a data set corresponding to $7.2 \times 10^{20}$ protons on target taken with the MicroBooNE detector at Fermilab. Proton interactions in the neutrino target at the Main Injector produce $\pi^0$ and $\eta$ mesons, which could decay into dark matter (DM) particles via a dark photon $A^{\prime}$. A convolutional neural network is trained to identify interactions of the DM particles in the liquid-argon time projection chamber (LArTPC) exploiting its image-like reconstruction capability. In the absence of a DM signal, limits at the $90\%$ confidence level on the squared kinematic mixing parameter $\varepsilon^2$ as a function of the dark-photon mass in the range $10\le M_{A^\prime}\le 400$~MeV are provided. The limits cover previously unconstrained parameter space for the production of fermion or scalar DM particles $\chi$ for two benchmark models with mass ratios $M_{\chi}/M_{A^\prime}=0.6$ and $2$ and for dark fine-structure constants $0.1\le\alpha_D\le 1$.Date of Award1 Aug 2024Original languageEnglishAwarding InstitutionThe University of ManchesterSupervisorStefan Soldner-Rembold (Supervisor) & Justin Evans (Supervisor)

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Neutrino-argon cross-section measurements from the MicroBooNE experiment

MicroBooNE is a liquid argon time projection chamber (LArTPC) neutrino detector located along the Fermilab Booster Neutrino Beam and 8 degrees off-axis to the Neutrinos at the Main Injector beam. MicroBooNE collected data from both beams accumulating a large neutrino-argon scattering dataset containing hundreds of thousands of events. Understanding neutrino-argon interactions is crucial for the next generation of neutrino oscillation experiments including DUNE. MicroBooNE has developed pioneering methodologies and novel reconstruction tools in order to benchmark models at very high sensitivity across the interaction phase space, including for ultra-rare channels. This proceeding presents an overview of the most recent MicroBooNE neutrino interaction results. These measurements span inclusive, CC0$π$, and rare channels including $Λ$, $K^+$ and $η$ production, providing invaluable datasets for constraining backgrounds and improving the modeling of neutrino scattering critical for the broader LArTPC neutrino physics program.

Liu, Liang [Fermilab] (ORCID:000000026753925X)↗
DOE OSTI2024

Nuclear Dependence of Charged Current Inclusive $\bar{\nu}_\mu -$A Cross Section Measurements at 6 GeV at MINERvA

The main advantage of the MINERvA detector is that it uses differently sized nuclear targets, in the same neutrino beam, allowing for a direct study of nuclear medium effects, with reduced flux and detector uncertainties. This thesis presents the first high statistics charged current antineutrino-nucleus inclusive two-dimensional cross section measurements using an antineutrino beam with $\langle E_{\bar\nu}\rangle\sim$ 6 GeV, on iron, lead, carbon and hydrocarbon targets, as a function of bjorken $x$ and four-momentum transfer squared $Q^2$, invariant mass $W$ and $Q^2$ and muon longitudinal and transverse momenta, $p_t$ and $p_z$. These nuclei of different sizes were utilized to measure the nuclear dependence of the cross section. The antineutrino interactions that were utilized for the analysis presented in this thesis, were having an antimuon in the final state with energy 2-20GeV and with an antimuon scattering angle less than $17^o$. This analysis represents one of the largest antineutrino datasets analyzed to date in the energy region of $\langle E_{\bar\nu} \rangle \sim$ 6 GeV. This direct evidence for the nuclear medium effects, can help benchmarking models, which can be used by future neutrino experiments like DUNE, by extrapolating these results to Argon target.

Gaur, Prameet Kumar↗
DOE OSTI2024

INL Poster - Juan Barrera Salazar

Generation IV nuclear reactors introduce several advantages and benefits in terms of safety and efficiency when compared with their predecessors from previous generation. This is due, among many things, to the use of innovative forms of fuel and coolant, different from the conventional ones used in the last decades. Given that these upcoming designs utilize emerging technologies, the related instrumentation is also in the process of being developed; therefore, it is necessary to establish the sensitivity requirements and the effects of uncertainty on different properties of the components and elements of the reactor designs. This report presents the results of simulations that quantify the impacts of the uncertainties of four thermophysical properties of the refrigerant salt (LiF-BeF2) for the Kairos Power benchmark model (g-FHR) for steady state making use of the Sobol’ method through polynomial chaos surrogate modeling. The properties of the salt to which uncertainty was evaluated were density, dynamic viscosity, thermal conductivity and heat capacity. This study was carried out using the Griffin/Pronghorn multiphysics model under the computational resources of the Idaho National Laboratory (INL) High Performance Computing (HPC). The results indicate a weak dependence of the uncertainty of thermal conductivity on the quantities of core pressure drop and core outlet temperature.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗
DOE OSTI2026

The AWAKEN wind farm benchmark, Part 2: Modeling results

Accurately modeling wind farm performance in complex atmospheric flows remains a challenge. This paper presents the modeling results of the American WAKE experimeNt (AWAKEN) wind farm benchmark, a collaborative effort involving 16 research groups from academia and industry within the International Energy Agency Wind Technology Collaboration Programme Task 57. The study evaluates a diverse suite of simulation tools, ranging from fast-running engineering wake models to high-fidelity large-eddy simulations, against a diurnal case study observed during the AWAKEN campaign. The benchmark utilized a three-phase structure to progressively assess model performance as observational data availability increased. Initial blind predictions showed that higher-fidelity models did not uniformly outperform simpler simulation tools. A distinct spatial bias was observed where models struggled to resolve the interplay between a low-level jet, wakes, and terrain-induced flow acceleration. In subsequent phases, leveraging additional measurements for model improvement led to a reduction in mean absolute error across the model ensemble; however, this effect was most pronounced in engineering wake models, where targeted calibration reduced error by up to 40~\%. Overall, the study demonstrates that inflow characterization remains a primary prerequisite for accuracy, particularly for models relying on coarse forcing datasets. While the limited ability to resolve local terrain-flow interactions under single-day conditions represent a recognized constraint, the overall findings on wake modeling and real-world validation still provide valuable guidance for model application and for mitigating this limitation.

Bodini, Nicola↗
DOE OSTI2021

Highly Accurate Experimental Heave Decay Tests with a Floating Sphere: A Public Benchmark Dataset for Model Validation of Fluid–Structure Interaction

Highly accurate and precise heave decay tests on a sphere with a diameter of 300 mm were completed in a meticulously designed test setup in the wave basin in the Ocean and Coastal Engineering Laboratory at Aalborg University, Denmark. The tests were dedicated to providing a rigorous benchmark dataset for numerical model validation. The sphere was ballasted to half submergence, thereby floating with the waterline at the equator when at rest in calm water. Heave decay tests were conducted, wherein the sphere was held stationary and dropped from three drop heights: a small drop height, which can be considered a linear case, a moderately nonlinear case, and a highly nonlinear case with a drop height from a position where the whole sphere was initially above the water. The precision of the heave decay time series was calculated from random and systematic standard uncertainties. At a 95% confidence level, uncertainties were found to be very low—on average only about 0.3% of the respective drop heights. Physical parameters of the test setup and associated uncertainties were quantified. A test case was formulated that closely represents the physical tests, enabling the reader to do his/her own numerical tests. The paper includes a comparison of the physical test results to the results from several independent numerical models based on linear potential flow, fully nonlinear potential flow, and the Reynolds-averaged Navier–Stokes (RANS) equations. A high correlation between physical and numerical test results is shown. The physical test results are very suitable for numerical model validation and are public as a benchmark dataset.

42 ENGINEERING↗
DOE OSTI2025

Benchmarking large language models for materials synthesis: The case of atomic layer deposition

In this work, we introduce an open-ended question benchmark, ALDbench, to evaluate the performance of large language models (LLMs) in materials synthesis, and, in particular, in the field of atomic layer deposition, a thin film growth technique used in energy applications and microelectronics. Our benchmark comprises questions with a level of difficulty ranging from the graduate level to domain expert current with the state of the art in the field. Human experts reviewed the questions along the criteria of difficulty and specificity, and the model responses along four different criteria: overall quality, specificity, relevance, and accuracy. We ran this benchmark on an instance of OpenAI’s GPT-4o. The responses from the model received a composite quality score of 3.7 on a 1–5 scale, consistent with a passing grade. However, 36% of the questions received at least one below average score. An in-depth analysis of the responses identified at least five instances of suspected hallucination. Finally, we observed statistically significant correlations between the difficulty of the question and the quality of the response, the difficulty of the question and the relevance of the response, the specificity of the question, and the accuracy of the response as graded by the human experts. Furthermore, this emphasizes the need to evaluate LLMs across multiple criteria beyond difficulty or accuracy.

Artificial intelligence↗
DOE OSTI2020

Alaskan carbon-climate feedbacks will be weaker than inferred from short-term manipulations: Alaskan Benchmark Data and Model runs

This submission aimed to assess differences in short-term step warming manipulations and long-term chronic response to climate change in Alaskan ecosystems. Briefly, climate warming is occurring fastest at high latitudes. Based on short-term field experiments, this warming is projected to stimulate soil organic matter decomposition, and promote a positive feedback to climate change. We show here that the tightly coupled, nonlinear nature of high-latitude ecosystems implies that short-term (< 10 year) warming experiments produce emergent ecosystem carbon stock temperature sensitivities inconsistent with emergent multi-decadal responses. We first demonstrate that a well-tested mechanistic ecosystem model accurately represents observed carbon cycle and active layer depth responses to short-term summer warming in four diverse Alaskan sites. We then show that short-term warming manipulations do not capture the non-linear, long-term dynamics of vegetation, and thereby soil organic matter, that occur in response to thermal, hydrological, and nutrient transformations belowground. Our results demonstrate significant spatial heterogeneity in multi-decadal Arctic carbon cycle trajectories and argue for more mechanistic models to improve predictive capabilities.The model used in the current study is available publicly (https://github.com/jinyun1tang/ECOSYS), and the current submission contains the python/ matlab codes for analyzing output from the model (includng a readme file to explain the codes). The benchmark data, also enclosed, was collected from a range of published and publicly available sources (extracted using GRABIT: https://www.mathworks.com/matlabcentral/fileexchange/7173-grabit). These sources describe warming induced changes in tundra/ boreal ecosystems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation

As machine learning (ML) is increasingly implemented in hardware to address real-time challenges in scientific applications, the development of advanced toolchains has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as hardware synthesis, are becoming limiting factors in the rapid iteration of designs. To mitigate these emerging constraints, multiple efforts have been undertaken to develop an ML-based surrogate model that estimates resource usage of ML accelerator architectures. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of over 680,000 fully connected and convolutional neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, and the average performance across a subset of the dataset. Additionally, we introduce GNN- and transformer-based surrogate models that predict latency and resources for ML accelerators. We present the architecture and performance of the models and find that the models generally predict latency and resources for the 75% percentile within several percent of the synthesized resources on the synthetic test dataset.

Hawks, Benjamin [Fermilab] (ORCID:0000000157000288↗
DOE OSTI2025

Benchmark of numerical modeling approaches on the systematic performance evaluation of wave energy converters

Different numerical modeling methods have been developed and applied to evaluate a variety of performance indicators of wave energy converters (WECs), including the power performance, structural loads, levelized cost of energy, etc. Based on the modeling fidelity, the commonly used numerical modeling approaches can be classified as linear modeling, weakly nonlinear modeling and fully nonlinear modeling approaches. Each method differs in accuracy and computational efficiency, making them suitable for different stages of WEC design. However, the selection of modeling approach could significantly impact evaluation outcomes. For instance, simplified linear models may underestimate structural loads or overestimate energy production in some operational conditions, potentially leading to less cost-effective designs. Given the widespread utilization of these models, it is essential to understand the uncertainties brought by them in performance evaluations. This work is dedicated to benchmarking different linear-potential-flow-based numerical models for evaluating the systematic performance of WECs. Three representative numerical modeling approaches are considered in this work, including linear frequency-domain modeling, statistically linearized spectral-domain modeling and Cummins equation-based nonlinear time-domain modeling. A generic point absorber WEC is considered as the research reference in this work, and different sea sites are taken into account. The numerical models are utilized to predict critical performance indicators, including power performance, the annual energy production, the capacity factor, the levelized cost of energy and the PTO fatigue loads. By comparing the results, this work identifies the uncertainties associated with different modeling approaches in evaluating WEC performance.

Fatigue↗
DOE OSTI2026

A curated benchmark for cofolding models on kinase conformational states

Abstract Protein kinases are critical drug targets, requiring therapeutics that can modulate their active and inactive conformational states. While cofolding models can generate global folds directly from kinase sequences and ligand SMILES strings, these models have not yet been tested on their ability to recover ligand-induced-fit conformational states of the kinase proteins. Here, we introduce KinConfBench, a curated benchmark of 2225 high-quality human kinase chains to evaluate the ability of four state-of-the-art cofolding models—Boltz-2, Chai-1, Protenix, and RoseTTAFold-All-Atom—to recover both canonical and rare conformational states. We show that geometric success metrics of a ligand pose in the active site do not correlate strongly with the correct kinase conformational state, motivating a new set of dynamical benchmarks for assessing cofolding models. While all four cofolding models achieve ~60–80% prediction accuracy for kinase conformational classification, they exhibit severe mode collapse when performing multiple inferences, show negligible structural diversity in sampling induced-fit motions, and display a prevalent “apo-drift” in which most cofolding models predominantly predict the kinase to be in its ligand-free state. Our results highlight that capturing ligand-induced protein conformational diversity, not just geometric fit, is critical for next-generation structure-based drug discovery.

Sun, Kunyang↗
DOE OSTI2025

wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation

As machine learning (ML) is increasingly implemented in hardware to address real-time challenges in scientific applications, the development of advanced toolchains has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as hardware synthesis, are becoming limiting factors in the rapid iteration of designs. To mitigate these emerging constraints, multipleefforts have been undertaken to develop an ML-based surrogate model that estimates resource usage of synthesized ML accelerator architectures. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of over 680 000 fully connected and convolutional neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, and the average performance across a subset of the dataset. Additionally, we introduce GNN- and transformer-based surrogate models that predict latency and resources for ML accelerators. We present the architecture and performance of the models and find that the models generally predict latency and resources for the 75% percentile within several percent of the synthesized resources on the synthetic test dataset.

Hawks, Benjamin G. [Fermilab]↗
DOE OSTI2026

Cooling Matters: Benchmarking Large Language Models and Vision-Language Models on Liquid-Cooled Versus Air-Cooled H100 GPU Systems

The unprecedented growth in artificial intelligence (AI) workloads, recently dominated by large language models (LLMs) and vision-language models (VLMs), has intensified power and cooling demands in data centers. This study benchmarks LLMs and VLMs on two HGX nodes, each with 8× NVIDIA H100 graphics processing units (GPUs), using liquid and air cooling. Leveraging GPU Burn, Weights & Biases, and IPMItool, we collect detailed thermal, power, and computation data. Results show that the liquid-cooled systems maintain GPU temperatures between 41-50$^\circ$C, while the air-cooled counterparts fluctuate between 54-72$^\circ$C under load. This thermal stability of liquid-cooled systems yields 17% higher performance (54 TFLOPs/ GPU vs. 46 TFLOPs/GPU), performance-per-watt, reduced energy overhead, and greater system efficiency than the air-cooled counterparts. These findings underscore the energy and sustainability benefits of liquid cooling, offering a compelling path forward for hyperscale data centers seeking to optimize AI infrastructure. https://github.com/iscaas/Cooling-Matters.

Latif, Imran↗
DOE OSTI2022

The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider

We describe the outcome of a data challenge conducted as part of the Dark Machines (https://www.darkmachines.org) initiative and the Les Houches 2019 workshop on Physics at TeV colliders. The challenged aims to detect signals of new physics at the Large Hadron Collider (LHC) using unsupervised machine learning algorithms. First, we propose how an anomaly score could be implemented to define model-independent signal regions in LHC searches. We define and describe a large benchmark dataset, consisting of >1 billion simulated LHC events corresponding to 10\, fb^{-1} 10 f b − 1 of proton-proton collisions at a center-of-mass energy of 13 TeV. We then review a wide range of anomaly detection and density estimation algorithms, developed in the context of the data challenge, and we measure their performance in a set of realistic analysis environments. We draw a number of useful conclusions that will aid the development of unsupervised new physics searches during the third run of the LHC, and provide our benchmark dataset for future studies at https://www.phenoMLdata.org. Code to reproduce the analysis is provided at https://github.com/bostdiek/DarkMachines-UnsupervisedChallenge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗