Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

An international benchmark for wind plant wakes from the American WAKE ExperimeNt (AWAKEN)

This article introduces the first benchmark study within the International Energy Agency Wind Task 57 framework, focusing on wind plant wakes. Leveraging data from the American WAKE ExperimeNt (AWAKEN), the benchmark aims to assess the accuracy of simulation tools in modeling wind plant wakes and their impact on the downstream flow under diverse inflow conditions. The AWAKEN field campaign, conducted in Oklahoma from 2022 to 2024, provides unprecedented observations of wind plant-atmosphere interactions, thus offering a large dataset to validate numerical models of different complexity. The benchmark will include three phases—code calibration, blind comparison, and iteration—allowing participants to refine their numerical models based on the feedback from the benchmark team. This article describes the benchmark case study selected from observations providing details on atmospheric conditions, wake evidence, and wind turbine operation. The benchmark’s structure and timeline, along with the expected publication of results, are discussed as well. This collaborative effort aims to enhance the accuracy of wind plant wake simulations, thus contributing to the improvement of wind energy production estimates.

17 WIND ENERGY↗

GlideinBenchmark: collecting resource information to optimize provisioning

Choosing the right resource can speedup jobs completion, better utilize the available hardware and visibly reduce costs, especially when renting computers on the cloud. This was demonstrated in earlier studies on HEPCloud. But the benchmarking of the resources proved to be a laborious and time-consuming process. This paper presents GlideinBenchmark, a new Web application leveraging the pilot infrastructure of GlideinWMS to benchmark resources, and shows how to use the data collected and published by GlideinBenchmark to automate the optimal selection of resources. An experiment can select the benchmark or the set of benchmarks that most closely evaluate the performance of its workflows. With GlideinBenchmark and the help of the GldieinWMS Factory it controls the benchmark execution. Finally, a scheduler like HEPCloud’s Decision Engine can use the results to optimize resource provisioning.

Mambelli, Marco↗

Studying CPU and memory utilization of applications on Fujitsu A64FX and Nvidia Grace Superchip

ARM-based manycore CPU architectures are well-positioned to provide the rising memory throughput requirements of modern data intensive scientific applications in High Performance Computing (HPC). The Fujitsu A64FX CPU platform is based on the ARM v8.2A architecture, and is the processor of the flagship Japanese supercomputer - "Fugaku", which was previously ranked as the #1 supercomputer in the world according to the Top500 list. The Nvidia Grace superchip features 144 Neoverse V2 cores based on the ARMv9 architecture with 4x128b SVE2, providing exceptional computational power. The chip supports up to 480GB of memory, making it ideal for AI, machine learning, and scientific computing workloads. In this paper, we conduct a thorough performance exploration of a variety of parallel bandwidth-sensitive benchmarks and applications compiled with the native Fujitsu compiler on a Fugaku A64FX compute node and ARM (LLVM) Compiler on an NVIDIA Grace superchip compute node, engaging all the computational cores per cluster using OpenMP multithreading (assuming the cores can drive the available bandwidth). Our ultimate goals are to study the resource utilization of scientific applications and benchmarks on A64FX and Grace superchip, considering graph application scenarios ( GAP Benchmark suite) and eleven appli- cation proxies from the Rodinia heterogeneous benchmark suite (considering domains such as Data Mining, Bioinformatics, Fluid Dynamics, Pattern Recognition, etc.). Through exhaustive performance monitoring, we quantify the resource utilization of diverse OpenMP-based HPC applications on both the Fujitsu A64FX and the Nvidia Grace Superchip platforms.

benchmarking, Performance Analysis, High performan↗

Thermochemical Nonequilibrium Modeling in a Continuous-Galerkin, Finite-Element Framework

The presented work discusses the implementation, verification, and validation of Park's two-temperature model in a scalable, computational fluid dynamics (CFD) code developed at the US Department of Energy's Oak Ridge National Laboratory (ORNL). The implementation of Park's two-temperature model was verified through 0D test cases involving an adiabatic reactor and a nitrogen thermal bath. The implementation was then validated through comparisons with other validated CFD codes and experimental data on a hypersonic cylinder and double cones. These are standard benchmark test cases for thermochemical non-equilibrium (TCNE) modeling, and all data are shared publicly. The verification and validation results showed that ORNL's in-house CFD code could model complex, high-speed flow problems with and without TCNE modeling. This work is essential for future research involving 3D shock wave/boundary layer interactions (SBLIs).

Nutter, Nicole↗

A Parameter-masked Mock Data Challenge for Beyond-two-point Galaxy Clustering Statistics

The past few years have seen the emergence of a wide array of novel techniques for analyzing high-precision data from upcoming galaxy surveys, which aim to extend the statistical analysis of galaxy clustering data beyond the linear regime and the canonical two-point (2pt) statistics. We test and benchmark some of these new techniques in a community data challenge named “Beyond-2pt,” initiated during the Aspen 2022 Summer Program “Large-Scale Structure Cosmology beyond 2-Point Statistics,” whose first round of results we present here. The challenge data set consists of high-precision mock galaxy catalogs for clustering in real space, in redshift space, and on a light cone. Participants in the challenge have developed end-to-end pipelines to analyze mock catalogs and extract unknown (“masked”) cosmological parameters of the underlying ΛCDM models with their methods. The methods represented are density-split clustering, nearest neighbor statistics, BACCO power spectrum emulator, void statistics, LEFTfield field-level inference using effective field theory (EFT), and joint power spectrum and bispectrum analyses using both EFT and simulation-based inference. In this work, we review the results of the challenge, focusing on problems solved, lessons learned, and future research needed to perfect the emerging beyond-2pt approaches. The unbiased parameter recovery demonstrated in this challenge by multiple statistics and the associated modeling and inference frameworks supports the credibility of cosmology constraints from these methods. The challenge data set is publicly available, and we welcome future submissions from methods that are not yet represented.

Krause, Elisabeth [Univ. of Arizona, Tucson, AZ (U↗

Summary Report of the FY24 DOE Contributions to the GIF VHTR CMVB

The Generation-IV Forum (GIF) Very-High-Temperature Reactor-Computational Methods Validation and Benchmark (VHTR-CMVB) initiative, involving organizations from Korea Atomic Energy Research Institute (KAERI) (South Korea), Institute of Nuclear and New Energy Technology of Tsinghua University (INET) (China), U.S. Department of Energy (DOE) (U.S.), Joint Research Centre (JRC) (Europe), and Japan Atomic Energy Agency (JAEA) (Japan), is dedicated to the verification and validation of tools for High-Temperature Gas-Cooled Reactors (HTGRs) analysis, using data shared by Computational Methods Validation and Benchmark (CMVB) signatories. For FY24, the US DOE CMVB has committed to several critical activities. Under WP1, led by the US, the integration of the High Temperature Gas Cooled Reactor - Pebble-Bed Module (HTR-PM) Phenomena Identification and Ranking Table (PIRT) into the comparative PIRT is progressing, with a new draft of the comparison tables issued earlier this year and currently being utilized by INET for their contribution. Neutronic validation efforts under WP3 include the preparation of the burnup analysis benchmark, preliminary calculations, and the development of reference models and results. In WP2, a validation exercise for hot gas mixing in the lower plenum of HTR-PM is in progress, using experimental data from INET (China) to validate modeling approaches. A model of the experimental facility has been developed using StarCCM+, with initial calculations slated for presentation at the GIF CMVB meeting this fall. Another WP2 activity focuses on validating numerical models for air-cooled Reactor Cavity Cooling System (RCCS) with experimental data from the Wisconsin Madison RCCS facility. A high-fidelity model, developed using NEK-RS, is currently being validated with available data from a low power forced convection test. These efforts are aimed at enhancing and confirming the accuracy of HTGR analysis tools, ensuring their alignment with experimental data and regulatory requirements.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Delayed Critical Highly Enriched Uranium Metal Cylinders with Thin Graphite Top and Bottom Reflectors

Nine 7, 11, and 15 in. diam highly enriched uranium (HEU, 93.15 wt % 235 U) metal cylinders were assembled on the vertical assembly machine in the Oak Ridge Critical Experiments Facility (ORCEF) and had 1, 2, or 3 in. thick HLM graphite reflectors on the top and bottom. The experiments, which were performed between April 3, 1970, and February 18, 1971, used 23 operational days at ORCEF. Before those experiments were carried out, unreflected and unmoderated, graphite- and polyethylene-reflected, and polyethylene-moderated HEU metal cylinders had been assembled to obtain delayed criticality at ORCEF in the 1960s and reported by the International Criticality Safety Benchmark Evaluation Project (ICSBEP) at the Nuclear Energy Agency (NEA)*. The data from the nine critical experiments are acceptable for use as criticality safety benchmark experiments for the NEA’s ICSBEP once the uncertainty analysis is completed. Based on previous ICSBEP benchmarks with HEU metal at ORCEF, the uncertainties in k eff are expected to be as low as ±0.0004.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Continual learning in the presence of repetition

Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world problems, the concept of repetition in the data stream is not often considered in standard benchmarks for CL. Unlike with the rehearsal mechanism in buffer-based strategies, where sample repetition is controlled by the strategy, repetition in the data stream naturally stems from the environment. This report provides a summary of the CLVision challenge at CVPR 2023, which focused on the topic of repetition in class-incremental learning. The report initially outlines the challenge objective and then describes three solutions proposed by finalist teams that aim to effectively exploit the repetition in the stream to learn continually. The experimental results from the challenge highlight the effectiveness of ensemble-based solutions that employ multiple versions of similar modules, each trained on different but overlapping subsets of classes. This report underscores the transformative potential of taking a different perspective in CL by employing repetition in the data stream to foster innovative strategy design.

Class-incremental learning↗

Computational investigation of the impact of metal–organic framework topology on hydrogen storage capacity

Metal–organic frameworks (MOFs) are promising, tunable materials for hydrogen storage. For application under cryogenic operating conditions, past work has run into a ceiling on performance due to a trade-off in the volumetric deliverable capacity (VDC) versus the gravimetric deliverable capacity (GDC). In this study, we computationally constructed and screened 105 230 MOF structures based on 529 nets to explore the effect of underlying topology on the hydrogen storage performance of the resulting materials. A machine learning model was developed based on simulated hydrogen uptake to facilitate screening of the entire dataset, and it successfully identified the top 10% of materials with a root-mean-square error of approximately 1 g L −1 as validated by subsequent grand canonical Monte Carlo simulations. We identified a promising structure based on the tsx topology that exhibits both VDC and GDC higher than the current benchmark material, MOF-5. Our data-driven analysis indicates that nets with higher net density yield MOFs with enhanced volumetric and gravimetric surface areas, thereby improving maximum VDC while shifting the capacity trade-off toward higher GDC.

36 MATERIALS SCIENCE↗

Verified, Archived, Library of Inputs and Data (VALID) Supporting Files

This dataset contains input, output, and sensitivity data files for computational simulations with the SCALE code system as part of the Verified, Archived Library of Inputs and Data (VALID). The simulations cover critical benchmark experiments from the International Criticality Safety Benchmark Evaluation Project. The files are to be housed in a public directory for distribution. The information contained in the files have been approved for release by the Organisation for Economic Co-operation and Development Nuclear Energy Agency (NEA). Users wanting to reproduce results from this dataset are required to obtain a license to the SCALE code system for which details on the distribution can be found here: https://www.ornl.gov/scale/releases.

keff↗

RUBISCO Soil Moisture Working Group (SMWG) Mini-Workshop Report

The RUBISCO Soil Moisture Working Group (SMWG), an initiative bolstered by the DOE RUBISCO Science Focus Area project, is committed to unraveling the intricacies of soil moisture and its profound impact on global hydrology, energy, and biogeochemical cycles. Confronted with the complex challenges in multi-scale SM data development, feedback analysis, and model benchmarking, the SMWG aims to catalyze advancements in the field through a synergistic approach that engages leaders in soil moisture research and encompasses diverse datasets and novel analytical techniques.

54 ENVIRONMENTAL SCIENCES↗

GlideinBenchmark: collecting resource information to optimize provisioning

Choosing the right resource can speed up job completion, better utilize the available hardware, and visibly reduce costs, especially when renting computers in the cloud. This was demonstrated in earlier studies on HEPCloud. However, the benchmarking of the resources proved to be a laborious and time-consuming process. This paper presents GlideinBenchmark, a new Web application leveraging the pilot infrastructure of GlideinWMS to benchmark resources, and it shows how to use the data collected and published by GlideinBenchmark to automate the optimal selection of resources. An experiment can select the benchmark or the set of benchmarks that most closely evaluate the performance of its workflows. GlideinBenchmark, with the help of the GlideinWMS Factory, controls the benchmark execution. Finally, a scheduler like HEPCloud's Decision Engine can use the results to optimize resource provisioning.

Mambelli, Marco [Fermilab] (ORCID:0000000294892681↗

AI Benchmark Democratization and Carpentry

Benchmarks are a cornerstone of modern machine learning, enabling reproducibility, comparison, and scientific progress. However, AI benchmarks are increasingly complex, requiring dynamic, AI-focused workflows. Rapid evolution in model architectures, scale, datasets, and deployment contexts makes evaluation a moving target. Large language models often memorize static benchmarks, causing a gap between benchmark results and real-world performance. Beyond traditional static benchmarks, continuous adaptive benchmarking frameworks are needed to align scientific assessment with deployment risks. This calls for skills and education in AI Benchmark Carpentry. From our experience with MLCommons, educational initiatives, and programs like the DOE's Trillion Parameter Consortium, key barriers include high resource demands, limited access to specialized hardware, lack of benchmark design expertise, and uncertainty in relating results to application domains. Current benchmarks often emphasize peak performance on top-tier hardware, offering limited guidance for diverse, real-world scenarios. Benchmarking must become dynamic, incorporating evolving models, updated data, and heterogeneous platforms while maintaining transparency, reproducibility, and interpretability. Democratization requires both technical innovation and systematic education across levels, building sustained expertise in benchmark design and use. Benchmarks should support application-relevant comparisons, enabling informed, context-sensitive decisions. Dynamic, inclusive benchmarking will ensure evaluation keeps pace with AI evolution and supports responsible, reproducible, and accessible AI deployment. Community efforts can provide a foundation for AI Benchmark Carpentry.

von Laszewski, Gregor [Virginia U.]↗

Updates to the HTR-PROTEUS HALEU Benchmark Using Modern Analysis Methodologies

The HTR-PROTEUS IRPhEP Handbook benchmarks represent some of the highest quality benchmarks available for systems with TRISO-HALEU fuel, graphite pebbles, graphite reflector, and high neutron leakage. Nevertheless, significant variability in computed eigenvalue results was encountered in HTR-PROTEUS configurations that are relevant for transportation configurations. The team will investigate the source of these variabilities in the original benchmark. In addition, the team will investigate the HTR-PROTEUS subcritical measurements, kinetics data, and potential criticality effects from the introduction of hydrogen in the system (water ingress). The project will generate a benchmark evaluation of additional HTR-PROTEUS measurements that can significantly increase the value of these criticality benchmarks in testing nuclear codes and data to support transportation validation needs for industry and the U.S. NRC.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

Multimetallic Layered Composites (MMLCs) for Rapid, Economical Advanced Reactor Deployment (Final Report)

This project focused on the development of multi-metallic layered composites (MMLCs) for advanced fission reactor technologies. There are many instances where one alloy or material simply cannot meet all the demands thrown at it by a reactor system, or cannot allow it to perform as strongly as one would like. Instead of focusing all our effort on developing one perfect alloy, we seek to leverage the design principle of “separation of functionality,” used in many other arenas in design, to boost performance beyond single alloys alone. One illustrative example shows the power of this approach for molten salt-cooled reactors: A three meter tall, three meter diameter reactor vessel made of Incoloy 800 was quoted at $\$$500k in 2018. A Hastelloy N vessel was quoted at $\$$5M. An MMLC vessel, in which a layer of Hastelloy N would be weld-overlaid onto Incoloy 800, was quoted at $\$$700k, and it would achieve the same performance. The potential economic gains of leveraging this approach are therefore substantial. At a minimum, each MMLC would contain one core structural layer and one coolant-facing corrosion-resistant layer. Sometimes, MMLCs required buffer layers, as the structural and corrosion-resistant layers were metallurgically incompatible. In other words, they didn’t always play nice, thus separating layers compatible with both functioned as intermediaries to keep the composite together. However, in doing so we inevitably produce new interfaces, where new issues can arise. Therefore, this project focused on what happens at these interfaces from a combination of high temperatures, irradiation, corrosion, and time. After all, a reactor makes money when it is operating, and outages of any kind erode its economic viability. First, we set out to experimentally prove that MMLCs for at least two advanced reactor systems can be made, today, in US domestic facilities. In this respect we were successful – one MMLC (a Ni-201/Incoloy 800H composite) was successfully made and drawn into two-inch coolant piping. Others were attempted, though new issues relating to cracking in vanadium layers for one and radiation damage performance of the corrosion-resistant layer in another prevented us from moving further in those specific arenas – these are engineering problems which deserve continued focus after this project. Additional experimental work focused on long-term corrosion testing of the outermost layers of the salt-cooled and liquid lead-cooled MMLC concepts, which would then be fed into predictions of how long the MMLCs could last. Next, computational (thermodynamics and atomistic) simulation studies studied how much we expect the interfaces to “blend,” due to the mixing action of neutron irradiation. This eats into both the margin for the structural layer of each MMLC, as dilution from the corrosion-resistant layer into the structural layer would decrease the total load-bearing capacity of an MMLC of finite size. On the other hand, dilution of the corrosion-resistant layer into the structural layer further reduced the margin of corrodible material, reducing the lifetime of the MMLC or necessitating extra thickness to be imparted to the MMLC to meet its functional requirements. Work here focused on irradiation-induced segregation to predict new phases which may embrittle the MMLCs, as well as quantifying irradiation-induced mixing at each interface. The results showed that mixing is expected, but it is both steady and therefore predictable, and not lifetime-limiting for most MMLC concepts – it simply has to be accounted for in calculations of reactor performance when utilizing an MMLC. Then, full-core simulations using the experimentally-derived corrosion data, the computationally discovered irradiation-induced mixing data (partially validated by experiment), and existing, benchmarked core designs for large and small sized reactor concepts (one salt-cooled, one lead-cooled) were conducted to quantify any expansion of reactor operating envelopes achieved by utilizing these MMLCs. This new framework, called REX (Reactor Envelope Expansion), incorporates a combination of core neutronics, thermal hydraulics, and the material performance data derived from this project to see how using an MMLC expands advanced fission reactor operating envelopes. It was discovered that in some cases, MMLC utilization does indeed increase the maximum operating temperatures and cycle lengths of reactor concepts, while in other cases it does not. Finally, our tech-to-market (T2M) strategy was not necessarily to create specific embodiments of MMLCs for immediate sale (because getting into the nuclear market is incredibly slow and laden with regulation, this is a long-term goal), but rather immediate stimulation of US industry using the design approach of MMLCs derived from this project. In this respect we were successful, as one of the PhD students funded on this project co-founded Allium Engineering, Inc., which created a stainless steel / low-alloy steel MMLC to function as chloride corrosion-resistant rebar for embedding into concrete structures. Allium Engineering continues to be successful, having recently opened their first factory as of this writing.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Automating Detection and Diagnosis of Faults, Failures, and Underperformance in PV Plants

The project developed hybrid physics-based and machine-learning methods for near-real-time detection of balance-of-system faults (e.g., string, combiner, and tracker outages) in utility-scale Photovoltaic plants, achieving over 50% true positive rates with under 10% false positives and significantly reducing engineering setup time. In the extended phase, the scope expanded to plant-level underperformance analysis and industry benchmarking through the SUPER.epri.com platform. SUPER standardizes data processing and performance metrics across more than 9 GWac and 120+ plants, enabling robust comparisons and insights into loss rates, inverter downtime, and capacity degradation.

14 SOLAR ENERGY↗

Measurements of Gamow-Teller transitions from 59 Co via the 59 Co ⁢(𝑡, 3 He +𝛾) charge-exchange reaction and its application to the stellar electron-capture rates

Electron-capture reactions on iron-group nuclei play a crucial role in the late stages of massive star evolution. Since stellar evolution simulations depend on accurate electron-capture rates—which are highly sensitive to the detailed Gamow-Teller (GT) strength distributions—reliable theoretical models are essential. However, experimental data on GT strength distributions are scarce. High-resolution measurements are therefore vital for benchmarking and improving these theoretical calculations. To provide high-resolution data on Gamow-Teller strength distributions of iron-group nuclei and to compare these results with theoretical calculations within this mass region. Differential cross sections for the 59 Co ⁢(𝑡, 3 He)⁢ 59 Fe charge-exchange reaction at 115 MeV/u were measured using the S800 spectrometer. Furthermore, to resolve individual levels that are not distinguishable in the S800 particle singles data, coincident 𝛾 rays from the 59 Fe residual nucleus were detected by using the Gamma-Ray Energy Tracking In-beam Nuclear Array 𝛾-ray tracking array. Here, the Gamow-Teller transition strength distribution from the ground state of 59 Co to 59 Fe was extracted up to an excitation energy of 10 MeV. Additionally, transition strengths for several low-lying states were determined from coincident 𝛾-ray measurements. Electron-capture rates calculated using the present data indicate that these low-lying states contribute significantly to the overall rates in relevant stellar environments. The experimental results show reasonable agreement with theoretical predictions based on both shell-model and projected shell-model calculations. High-resolution data on Gamow-Teller strength distributions—particularly for individual low-lying states—are essential for accurately determining electron-capture rates in iron-group nuclei. Coincident 𝛾-ray measurements provide a powerful tool for obtaining such detailed information. While the present work demonstrates that shell-model calculations successfully reproduce the experimental results, such comparisons are scarce and more experimental data are desirable.

59 ≤ A ≤ 89↗

SEED Platform for Building Performance Standards Implementation Guide (Spanish Translation)

This guide provides an overview of the Standard Energy Efficiency Data (SEED) Platform. The SEED Platform developed by the U.S. Department of Energy (DOE) to provide a low-cost, user-friendly tool for jurisdictions to launch and manage energy benchmarking and Building Performance Standard (BPS) programs. It has been translated into Spanish. This is the Spanish translation of NREL/FS-5500-90691.

benchmarking↗