Search NASA⌕ Search

SEARCH · Search NASA

Results for “performance benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Evaluating Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this work, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, shared local memory accesses, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

TEAMER Technical Support of Ramboll's for Numerical Modeling of WECs to Support OES Task 10: Cooperative Research and Development (Final Report)

National Technology & Engineering Solutions of Sandia, LLC (NTESS) in collaboration with the National Renewable Energy Laboratory (NREL), and with guidance from Ramboll, will perform fluid dynamics simulations to support of The Ocean Energy Systems (OES) Energy Technology Collaboration Program Task 10 Wave Energy Converters (WEC) Modelling Verification and Validation effort. Specific numerical simulations include the fixed device wave impingement studies to benchmark the performance of simulation techniques against physical testing results.

16 TIDAL AND WAVE POWER↗
DOE OSTI2022

Benchmarking of Solar Irradiance Nowcast Performance Derived from All-Sky Imagers

Fluctuations of the incoming solar irradiance impact the power generation from photovoltaic and concentrating solar thermal power plants. Accurate solar nowcasting becomes necessary to detect these sudden changes of generated power and to provide the desired information for optimal exploitation of solar systems. In the framework of the International Energy Agency's Photovoltaic Power Systems Program Task 16, a benchmarking exercise has been conducted relying on a bouquet of solar nowcasting methodologies by all-sky imagers (ASI). In this paper, four ASI systems nowcast the Global Horizontal Irradiance (GHI) with a time forecast ranging from 1 to 20 min during 28 days with variable cloud conditions spanning from September to November 2019 in southern Spain. All ASIs demonstrated their ability to accurately nowcast GHI, with RMSE ranging from 6.9% to 18.1%. Under cloudy conditions, all ASIs' nowcasts outperform the persistence models. Under clear skies, three ASIs are better than persistence. Discrepancies in the observed nowcasting performance become larger at increasing forecast horizons. The findings of this study highlight the feasibility of ASIs to reliably nowcast GHI at different sky conditions, time intervals and horizons. Such nowcasts can be used either to estimate solar power at distant times or detect sudden GHI fluctuations.

all-sky imagers↗
DOE OSTI2024

Benchmarking highly entangled states on a 60-atom analogue quantum simulator

Abstract Quantum systems have entered a competitive regime in which classical computers must make approximations to represent highly entangled quantum states 1,2 . However, in this beyond-classically-exact regime, fidelity comparisons between quantum and classical systems have so far been limited to digital quantum devices 2–5 , and it remains unsolved how to estimate the actual entanglement content of experiments 6 . Here, we perform fidelity benchmarking and mixed-state entanglement estimation with a 60-atom analogue Rydberg quantum simulator, reaching a high-entanglement entropy regime in which exact classical simulation becomes impractical. Our benchmarking protocol involves extrapolation from comparisons against an approximate classical algorithm, introduced here, with varying entanglement limits. We then develop and demonstrate an estimator of the experimental mixed-state entanglement 6 , finding our experiment is competitive with state-of-the-art digital quantum devices performing random circuit evolution 2–5 . Finally, we compare the experimental fidelity against that achieved by various approximate classical algorithms, and find that only the algorithm we introduce is able to keep pace with the experiment on the classical hardware we use. Our results enable a new model for evaluating the ability of both analogue and digital quantum devices to generate entanglement in the beyond-classically-exact regime, and highlight the evolving divide between quantum and classical systems.

Science & Technology - Other Topics↗
DOE OSTI2022

PDF4LHC21: Update on the benchmarking of the CT, MSHT and NNPDF global PDF fits

There have been recent updates to the three global PDF fits (CT, MSHT and NNPDF), all adding large amounts of data from the LHC, and this has resulted in significant changes to the global PDFs. Given the impact that the new PDFs will have on physics comparisons at the LHC, it is crucial to perform a benchmarking among the PDFs, similar in spirit to that which was carried out for PDF4LHC15, widely used for LHC physics. In this article we detail a benchmarking comparison of three global PDF sets - CT18, MSHT20 and NNPDF3.1 - and their similarities and differences that have been observed. The end result of this study will be a new PDF4LHC21 ensemble of combined PDFs suitable for a wide range of LHC applications.

Cridge, Thomas↗
DOE OSTI2025

Benchmarking Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this paper, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, use of local memory, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
DOE OSTI2024

Accelerating Thermochemical Equilibrium Calculations for Nuclear Reactor Applications

Thermochemical properties play a key role in modeling and simulation of several key phenomena in nuclear reactors. There has been an increasing interest in incorporating CALPHAD-based formulations in multiphysics simulations including for Molten Salt Reactors where knowledge of phase evolution of the salt and the chemical potentials of various elements are of utmost importance in source term analyses and redox control. However, the size of such simulations is often limited by the high computational cost of full thermodynamic equilibrium calculations. This work discusses the current efforts aimed at accelerating thermochemical equilibrium calculations for multiphysics simulations performed using the open-source finite element / finite volume code Multiphysics Object Oriented Simulation Environment (MOOSE) [1]. While several methods have been proposed for accelerating phase equilibrium calculations [2], most focus on relatively small systems and often rely on a- priori knowledge of the state-space of the system. Nuclear materials, however, are often multi-component systems owing to the evolution of composition under irradiation and an approach based on a-priori mapping of phase diagram is often not enough. This work is aimed at demonstrating an on-the-fly surrogate modeling framework that uses active learning to reduce the number of full equilibrium calculations that must be performed. By combining with efficient coupling approaches, the surrogate framework helps in reducing the computational cost of thermodynamic equilibrium informed multiphysics simulations of nuclear materials. The performance is benchmarked against full coupling with the thermochemistry library Thermochimica [3]. This work uses a machine learning based approach for constructing surrogate models to predict the stable phases in a multicomponent system. The surrogates were constructed using neural networks and Gaussian process classification. In this work, we compare the relative performance of the two methods. We also demonstrate the use of caching previous calculations by interpolating the values from nearest neighbors. References [1] Lindsay, A.D., et al. "2.0 – MOOSE: Enabling massively parallel multiphysics simulation", SoftwareX, 20 (2022): 101202. [2] Roos, W.A. and Zietsman J.H. "Accelerating complex chemical equilibrium calculations – A Review", Calphad, 77 (2022): 102380. [3] Piro, M.H.A., et al. "The thermochemistry library Thermochimica", Computational Materials Science, 67 (2013): 266-272.

36 MATERIALS SCIENCE↗
DOE OSTI2024

KOMPASS-II: Compaction of Crushed salt for Safe Containment – Phase 2

Long-term stable sealing elements are a basic component in the safety concept for a possible repository for heat-emitting radioactive waste in rock salt. The sealing elements will be part of the closure concept for drifts and shafts. They will be made from a welldefinied crushed salt in employ a specific manufacturing process. The use of crushed salt as geotechnical barrier as required by the German Site Selection Act from 2017 /STA 17/ represents a paradigm change in the safety function of crushed salt, since this material was formerly only considered as stabilizing backfill for the host rock. The demonstration of the long-term stability and impermeability of crushed salt is crucial for its use as a geotechnical barrier. The KOMPASS-II project, is a follow-up of the KOMPASS-I project and continues the work with focus on improving the understanding of the thermal-hydraulic-mechanical (THM) coupled processes in crushed salt compaction with the objective to enhance the scientific competence for using crushed salt for the long-term isolation of high-level nuclear waste within rock salt repositories. The project strives for an adequate characterization of the compaction process and the essential influencing parameters, as well as a robust and reliable long-term prognosis using validated constitutive models. For this purpose, experimental studies on long-term compaction tests are combined with microstructural investigations and numerical modeling. The long-term compaction tests in this project focused on the effect of mean stress, deviatoric stress and temperature on the compaction behavior of crushed salt. A laboratory benchmark was performed identifying a variability in compaction behavior. Microstructural investigations were executed with the objective to characterize the influence of pre-compaction procedure, humidity content and grain size/grain size distribution on the overall compaction process of crushed salt with respect to the deformation mechanisms. The created database was used for benchmark calculations aiming for improvement and optimization of a large number of constitutive models available for crushed salt. The models were calibrated, and the improvement process was made visible applying the virtual demonstrator.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Benchmark Simulation of the Natural Convection Shutdown Heat Removal Test Facility Using SAM

Natural convection systems are a promising method to passively remove heat from reactor cavities during loss of forced flow accident scenarios. In this work at Argonne National Laboratory (ANL), a highly instrumented Natural Convection Shutdown Heat Removal Test Facility (NSTF) was used to demonstrate the effectiveness of air-cooled natural convection systems. In previous work, RELAP5-3D simulations were performed on this facility with favorable comparisons to experiment for mass flow rate, pressure drop, air temperature increase, and air velocity. Both experimental and simulation efforts with this facility present a useful opportunity to perform a benchmark study with the System Analysis Module (SAM). SAM is an advanced thermal-hydraulic system code currently in development at ANL for advanced non-light water reactor safety analysis.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Precision-controlled ultrafast electron microscope platforms. A case study: Multiple-order coherent phonon dynamics in 1T-TaSe2 probed at 50 fs–10 fm scales

We report on the first detailed beam tests attesting the fundamental principle behind the development of high-current-efficiency ultrafast electron microscope systems where a radio frequency (RF) cavity is incorporated as a condenser lens in the beam delivery system. To allow for the experiment to be carried out with a sufficient resolution to probe the performance at the emittance floor, a new cascade loop RF controller system is developed to reduce the RF noise floor. Temporal resolution at 50 fs in full-width-at-half-maximum and detection sensitivity better than 1% are demonstrated on exfoliated 1T-TaSe2 system under a moderate repetition rate. To benchmark the performance, multi-terahertz edge-mode coherent phonon excitation is employed as the standard candle. The high temporal resolution and the significant visibility to very low dynamical contrast in diffraction signals via high-precision phase-space manipulation give strong support to the working principle for the new high-brightness femtosecond electron microscope systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Exxon Mobil-NETL Testing of DAC Materials

This joint initiative is aimed at extending our understanding of real DAC testing conditions at the NETL DAC testbed facility. Of particular focus will be: • Small Scale testing of powdered and formulated materials (supplied by ExxonMobil) to evaluate various performance metrics under DAC process cycles • Pilot scale testing of formulated materials at larger scales ExxonMobil will work with NETL to shake down equipment, validate testing methods, and define best practices for data analysis. Three tasks are proposed: 1. Validation of multi-cycle test data on powdered and formulated materials to benchmark various performance metrics of these materials. 2. Steam regeneration of powdered samples at small scale to define baseline performance under commercially relevant conditions. 3. Pilot scale testing of larger formulated materials to test commercially relevant samples under actual process cycles.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Validation of Continuous-Energy ENDF/B-VIII.0 16 O, 56 Fe, and 63,65 Cu Cross Sections for Nuclear Criticality Safety Applications

Recently completed cross-section evaluations sponsored in part by the Nuclear Criticality Safety Program were incorporated into the 2018 release of the ENDF/B-VIII.0 cross-section library. Evaluated isotopes of interest to the nuclear data and criticality safety community include 16 O, 56 Fe, and 63,65 Cu. For performance validation, benchmark models defined in the International Criticality Safety Benchmark Evaluation Project Handbook were selected based on energy-integrated k eff sensitivities to total cross sections of interest and compared with experimental values. Of the 102 benchmark configurations that were utilized, 63 are sensitive to 16 O, 32 sensitive to 63,65 Cu, and 25 sensitive to 56 Fe. Selected benchmarks were modeled in SCALE 6.2.3 Criticality Safety Analysis Sequence (CSAS) continuous-energy Monte Carlo k eff calculations with ENDF/B-VII.1, with a hybrid ENDF/B-VII.1 with ENDF/B-VIII.0 data substituted for individual isotopes of interest, and with ENDF/B-VIII.0. ENDF/B-VIII.0 showed improved agreement with experimental k eff for 56 Fe, 63 Cu, elemental copper, and full library substitution while producing lessened agreement for 16 O and 65 Cu. Furthermore, with full library and isotope-specific ENDF/B-VIII.0 performance, a best-case ENDF library was formed by excluding underperforming isotopes’ ENDF/B-VIII.0 data, reverting 16 O and 65 Cu cross sections to ENDF/B-VII.1. This resulted in the average relative deviation between calculated and experimental data improving from 1.45σ for the ENDF/B-VIII.0 library to 1.32σ for the best-case library, relative to benchmark uncertainty.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Informing Robust Functional Relationship Benchmarks: An Evaluation of the Temperature Sensitivity of Ecosystem Respiration Across the Arctic-Boreal Region

During land model development, simulated carbon dynamics are often benchmarked against observational data sets to evaluate model performance. Functional relationship benchmarks are the relationship between a driving variable (e.g., temperature) and a response variable (e.g., ecosystem respiration) and are a promising tool for assessing model performance by evaluating modeled sensitivities to changing environmental conditions. However, observed functional relationships can be influenced by choices made during data collection and throughout the benchmarking process, impacting the inferred skill of land models. To avoid misrepresenting a model's true performance, it is necessary to systematically evaluate best practices when constructing functional relationship benchmarks. We developed a set of guidelines for constructing functional relationship benchmarks, considering the choice of data set, number of daily observations, temporal extent, and temporal resolution across Alaska and Canada over a 20-year period from 2001 to 2020. The temperature sensitivity of ecosystem respiration from observations, evaluated through an apparent Q 10 , is highly variable both spatially and as a result of the data processing approach applied in the benchmark formation. When benchmarking 13 models from the Warming Permafrost Model Intercomparison Project (WrPMIP), the range in inferred model skill is substantially impacted by the choices applied in constructing functional relationship benchmarks. The inferred performance of a given model is most sensitive to the number of daily observations and temporal extent, followed by choice of benchmark data set and temporal averaging. Results from this analysis can guide the development of consistent and robust functional relationships for future model evaluation studies.

Poe, Jeralyn [Northern Arizona University, Flagsta↗
DOE OSTI2020

Building fault detection data to aid diagnostic algorithm creation and performance testing

It is estimated that approximately 4-5% of national energy consumption can be saved through corrections to existing commercial building controls infrastructure and resulting improvements to efficiency. Correspondingly, automated fault detection and diagnostics (FDD) algorithms are designed to identify the presence of operational faults and their root causes. A diversity of techniques is used for FDD spanning physical models, black box, and rule-based approaches. A persistent challenge has been the lack of common datasets and test methods to benchmark their performance accuracy. This article presents a first of its kind public dataset with ground-truth data on the presence and absence of building faults. This dataset spans a range of seasons and operational conditions and encompasses multiple building system types. It contains information on fault severity, as well as data points reflective of the measurements in building control systems that FDD algorithms typically have access to. The data were created using simulation models as well as experimental test facilities, and will be expanded over time.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Cladding Profilometry Analysis of Experimental Breeder Reactor-II Metallic Fuel Pins with HT9, D9, and SS316 Cladding

BISON finite element method fuel performance simulations were conducted using an existing automated process that couples the Fuels Irradiation & Physics Database (FIPD) and the Integral Fast Reactor Materials Information System database by writing input files and comparing the BISON output to post-irradiation fuel pin profilometry measurements contained within the databases. The importance of this work is to demonstrate the ability to benchmark fuel performance metallic fuel models within BISON using Experimental Breeder Reactor-II fuel pin data for a number of similar pins, while building off previous modeling efforts. Changes to the generic BISON input file include implementing pin specific axial power and flux profiles, pin specific fluences, frictional contact, and irradiation-induced volumetric swelling models for cladding. A statistical analysis of irradiation-induced volumetric swelling models for HT9, D9, and SS316 was performed for experiments X421/X421A, X441/X441A, and X486. Between these three experiments, there were 174 post-irradiation examination (PIE) profilometries used for validating the swelling models presented using a standard error of the estimate (SEE) method. Implementation of the volumetric swelling models for D9 and SS316 claddings was found to have a significant impact on the BISON profilometry simulated, where HT9 clad pins had an insignificant change due to low fluence values. BISON profilometry simulated for HT9, D9, and SS316 fuel pins agreed with PIE profilometry measurements, with assembly SEE values being 4.4 × 10−3 for X421A, 2.0 × 10−3 for X441A, and 2.8 × 10−3 for X486. D9 clad pins in X421/X421A had the highest SEE values, which is due to the BISON simulated profilometry being shifted axially. While this work accomplished its purpose to demonstrate the modeling of multiple fuel pins from the databases to help validate models, the results suggest that the continued development of metallic fuel models is necessary for qualifying new metallic fuel systems to better capture some physical performance phenomena, such as the hot pressing of U-Pu-Zr and the fuel cladding chemical interaction.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

The parameter-level performance of covariance matrix conditioning in cosmic microwave background data analyses

Empirical estimates of the band power covariance matrix are commonly used in cosmic microwave background (CMB) power spectrum analyses. While this approach easily captures correlations in the data, noise in the resulting covariance estimate can systematically bias the parameter fitting. Conditioning the estimated covariance matrix, by applying prior information on the shape of the eigenvectors, can reduce these biases and ensure the recovery of robust parameter constraints. In this work, we use simulations to benchmark the performance of four different conditioning schemes, motivated by contemporary CMB analyses. The simulated surveys measure the TT, TE, and EE power spectra over the angular multipole range 300 ≤ ℓ ≤ 3500 in Δℓ = 50 wide bins, for temperature map-noise levels of 10, 6.4, and $2\, \mu$K arcmin. We divide the survey data into N real = 30, 50, or 100 uniform subsets. We show the results of different conditioning schemes on the errors in the covariance estimate, and how these uncertainties on the covariance matrix propagate to the best-fitting parameters and parameter uncertainties. The most significant effect we find is an additional scatter in the best-fitting point, beyond what is expected from the data likelihood. For a minimal conditioning strategy, N real = 30, and a temperature map-noise level of 10$\, \mu$K arcmin, we find the uncertainty on the recovered best-fitting parameter to be ×1.3 larger than the apparent posterior width from the likelihood (×1.2 larger than the uncertainty when the true covariance is used). Stronger priors on the covariance matrix reduce the misestimation of parameter uncertainties to $\lt 1{{\ \rm per\ cent}}$. As expected, empirical estimates perform better with higher N real , ameliorating the adverse effects on parameter constraints.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Uncertainty-Guided Prediction Horizon of Phase-Resolved Ocean Wave Forecasting Under Data Sparsity: Experimental and Numerical Evaluation

Accurate short-term wave forecasting is critical for the safe and efficient operation of marine structures that rely on real-time, phase-resolved ocean wave information for control and monitoring purposes (e.g., digital twins). These systems often depend on environmental sensors (e.g., waverider buoys, wave-sensing LIDAR). Challenges arise when upstream sensor data are missing, sparse, or phase-shifted due to drift. This study investigates the performance of two machine learning models, time-series dense encoder (TiDE) and long short-term memory (LSTM), for forecasting phase-resolved ocean surface elevations under varying degrees of data degradation. We introduce the τ-trimming algorithm, which adapts the prediction horizon based on uncertainty thresholds derived from historical forecasts. Numerical wave tank (NWT) and wave basin experiments are used to benchmark model performance under short- and long-term data masking, spatially coarse sensor grids, and upstream phase shifts. Results show under a 50% probability of upstream data loss, the τ-trimmed TiDE model achieves a 46% reduction in error at the most upstream target, compared to 22% for LSTM. Furthermore, phase misalignment in upstream data introduces a near-linear increase in forecast error. Under moderate model settings, a ±3 s misalignment increases the mean absolute error by approximately 0.5 m, while the same error is accumulated at ±4 s using the more conservative approach. These findings inform the design of resilient, uncertainty-aware wave forecasting systems suited for realistic offshore sensing environments.

42 ENGINEERING↗
DOE OSTI2022

Simulation, Challenge Testing and Validation of Solutions for Residential and Commercial Occupancy Sensing/Counting and CO2 Sensing (Final Report)

Carbon dioxide sensors used for monitoring and control applications in buildings are known to be sensitive to long term drift and can be affected by variations in temperature, humidity, other gas interferents, pressure and other factors that cannot be corrected through the typical auto-calibration methods often embedded in the sensor electronics. This investigation was part of the Category D defined in the ARPA-E Saving Energy Nationwide in Structures With Occupancy Recognition (SENSOR) Financial Assistance Funding Opportunity Announcement No. DE-FOA-0001737, FFDA No. 81.135, January 18, 2017. The FOA defined target criteria for the evaluation of the Category C commercial CO2 sensors selected for review and development through the ARPA-E SENSOR program and charged the Iowa State Category D team with developing an evaluation methodology for Category C sensors in accordance with these criteria. As part of the work, the Iowa State team chose to engage the standards community in the development of the evaluation methodology. Standard development work in the D22.05 Indoor Air and D22.03 Ambient Atmospheres and Source Emissions subcommittees of ASTM International Committee D22 on Air Quality on the evaluation of low cost sensors and provision of guidance for using indoor carbon dioxide concentrations to evaluate indoor air quality and ventilation is of particular relevance to the Category D efforts. To date, three draft standards with specific input by the Iowa State Category D team have passed the subcommittee balloting stage and are being balloted in December 2022 and January 2023 in the main D22 committee. In addition to the aforementioned standards to which the PI was a contributing author, intellectual property developed in this project pertaining to “An Automated System for Evaluation of the Long Term Performance of Carbon Dioxide Sensors” and “An Automated Ground Truth System for Real-Time Reliability Assessment, Capture of Control Decisions and Energy Savings Measurements for Occupancy Recognition Systems and Other Applications” are the other major deliverables of the project. These standards and the developed intellectual property are expected to benefit the public in a number of ways stemming from their provision of means to benchmark the performance of carbon dioxide and other sensor systems. In particular, there is high interest in techniques to demonstrate the performance of low-cost carbon dioxide sensors which can be used for typical HVAC applications (e.g., demand controlled ventilation, outdoor and indoor air monitoring, estimation of occupancy, etc.).

42 ENGINEERING↗