A Benchmark to Understand Communication Performance in Hybrid MPI and GPU Applications.
Abstract not provided.
Haskins, Keira↗
SEARCH · Search NASA
Results for “performance benchmarking”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
h5bench: A unified benchmark suite for evaluating HDF5 I/O performance on pre‐exascale platforms
Summary Parallel I/O is a critical technique for moving data between compute and storage subsystems of supercomputers. With massive amounts of data produced or consumed by compute nodes, high‐performant parallel I/O is essential. I/O benchmarks play an important role in this process; however, there is a scarcity of I/O benchmarks representative of current workloads on HPC systems. Toward creating representative I/O kernels from real‐world applications, we have created h5bench , a set of I/O kernels that exercise hierarchical data format version 5 (HDF5) I/O on parallel file systems in numerous dimensions. Our focus on HDF5 is due to the parallel I/O library's heavy usage in various scientific applications running on supercomputing systems. The various tests benchmarked in the h5bench suite include I/O operations (read and write), data locality (arrays of basic data types and arrays of structures), array dimensionality (one‐dimensional arrays, two‐dimensional meshes, three‐dimensional cubes), I/O modes (synchronous and asynchronous). In this paper, we present the observed performance of h5bench executed along several of these dimensions on existing supercomputers (Cori and Summit) and pre‐exascale platforms (Perlmutter, Theta, and Polaris). h5bench measurements can be used to identify performance bottlenecks and their root causes and evaluate I/O optimizations. As the I/O patterns of h5bench are diverse and capture the I/O behaviors of various HPC applications, this study will be helpful to the broader supercomputing and I/O community.
97 MATHEMATICS AND COMPUTING↗
International collaboration framework for the calculation of performance loss rates: Data quality, benchmarks, and trends (towards a uniform methodology)
Abstract The IEA PVPS Task 13 group, experts who focus on photovoltaic performance, operation, and reliability from several leading R&D centers, universities, and industrial companies, is developing a framework for the calculation of performance loss rates of a large number of commercial and research photovoltaic (PV) power plants and their related weather data coming across various climatic zones. The general steps to calculate the performance loss rate are (i) input data cleaning and grading; (ii) data filtering; (iii) performance metric selection, corrections, and aggregation; and finally, (iv) application of a statistical modeling method to determine the performance loss rate value. In this study, several high‐quality power and irradiance datasets have been shared, and the participants of the study were asked to calculate the performance loss rate of each individual system using their preferred methodologies. The data are used for benchmarking activities and to define capabilities and uncertainties of all the various methods. The combination of data filtering, metrics (performance ratio or power based), and statistical modeling methods are benchmarked in terms of (i) their deviation from the average value and (ii) their uncertainty, standard error, and confidence intervals. It was observed that careful data filtering is an essential foundation for reliable performance loss rate calculations. Furthermore, the selection of the calculation steps filter/metric/statistical method is highly dependent on one another, and the steps should not be assessed individually.
14 SOLAR ENERGY↗
Measurement and Simulation of the Magnetic Fields from a 555 Timer Integrated Circuit Using a Quantum Diamond Microscope and Finite-Element Analysis
Quantum diamond microscope (QDM) magnetic field imaging is an emerging interrogation and diagnostic technique for integrated circuits (ICs). To date, the ICs measured with a QDM have been either too complex for us to predict the expected magnetic fields and benchmark the QDM performance or too simple to be relevant to the IC community. In this paper, we establish a 555 timer IC as a “model system” to optimize QDM measurement implementation, benchmark performance, and assess IC device functionality. To validate the magnetic field images taken with a QDM, we use a spice electronic circuit simulator and finite-element analysis (FEA) to model the magnetic fields from the 555 die for two functional states. Furthermore, we compare the advantages and the results of three IC-diamond measurement methods, confirm that the measured and simulated magnetic images are consistent, identify the magnetic signatures of current paths within the device, and discuss using this model system to advance QDM magnetic imaging as an IC diagnostic tool.
74 ATOMIC AND MOLECULAR PHYSICS↗
Neutronics and Thermo-Fluids Simulation of Generic Pebble-Bed Fluoride-Salt-Cooled High-Temperature Reactor
The fluoride-salt-cooled high-temperature reactor (FHR) is one of the advanced reactors that has been attracting considerable interest from both the research community and the nuclear industry. To help facilitate the nuclear community's familiarity with the FHR, Kairos Power has developed a generic FHR (gFHR) benchmark. In the research performed here, this benchmark was used to assess innovative modeling methods that combine stochastic and deterministic computer codes to perform the design and analysis of the gFHR. Further, the Monte Carlo code Serpent 2 was used to generate few-group cross sections that were then used in the neutron diffusion and thermal-fluids code AGREE to perform full-core neutronics and thermal-fluids steady-state and transient core analysis. The Argonne National Laboratory code SAM was then used to model the gFHR system and to simulate the load-follow operation of the gFHR.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
A Viable New Strategy for the Discovery of Peptide Proteolytic Cleavage Products in Plant-Microbe Interactions
Small peptides that are proteolytic cleavage products (PCPs) of less than 100 amino acids are emerging as key signaling molecules that mediate cell-to-cell communication and biological processes that occur between and within plants, fungi, and bacteria. Yet, the discovery and characterization of these molecules is largely overlooked. Today, selective enrichment and subsequent characterization by mass spectrometry–based sequencing offers the greatest potential for their comprehensive characterization, however qualitative and quantitative performance metrics are rarely captured. Herein, we addressed this need by benchmarking the performance of an enrichment strategy, optimized specifically for small PCPs, using state-of-the-art de novo–assisted peptide sequencing. As a case study, we implemented this approach to identify PCPs from different root and foliar tissues of the hybrid poplar Populus × canescens 717-1B4 in interaction with the ectomycorrhizal basidiomycete Laccaria bicolor . In total, we identified 1,660 and 2,870 Populus and L. bicolor unique PCPs, respectively. Qualitative results supported the identification of well-known PCPs, like the mature form of the photosystem II complex 5-kDa protein (approximately 3 kDa). A total of 157 PCPs were determined to be significantly more abundant in root tips with established ectomycorrhiza when compared with root tips without established ectomycorrhiza and extramatrical mycelium of L. bicolor . These PCPs mapped to 64 Populus proteins and 69 L. bicolor proteins in our database, with several of them previously implicated in biologically relevant associations between plant and fungus.
59 BASIC BIOLOGICAL SCIENCES↗
LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators
Large Language Models (LLMs) have propelled groundbreaking advancements across several domains and are commonly used for text generation applications. However, the computational demands of these complex models pose significant challenges, requiring efficient hardware acceleration. Benchmarking the performance of LLMs across diverse hardware platforms is crucial to understanding their scalability and throughput characteristics. We introduce LLM-Inference-Bench, a comprehensive benchmarking suite to evaluate the hardware inference performance of LLMs. We thoroughly analyze diverse hardware platforms, including GPUs from Nvidia and AMD and specialized AI accelerators, Intel Habana and SambaNova. Our evaluation includes several LLM inference frameworks and models from LLaMA, Mistral, and Qwen families with 7B and 70B parameters. Our benchmarking results reveal the strengths and limitations of various models, hardware platforms, and inference frameworks. We provide an interactive dashboard to help identify configurations for optimal performance for a given hardware platform.
Chitty-Venkata, Krishna Teja↗
Occupant-Centric key performance indicators to inform building design and operations
Building performance indicators are widely used to guide building design and track and benchmark operational performance. Traditional building performance indicators mostly focus on the energy efficiency perspective. As occupants are the primary building service recipients in residential and most commercial buildings, their comfort and wellbeing are crucial. As such, this study first identified significant attributes of occupant-centric key performance indicators (KPIs) and analyzed the diverse factors that should be considered in formulating an occupant-centric KPI. Then a suite of occupant-centric KPIs were synthesized from the review and enhancement of existing occupant-related performance metrics. The proposed occupant KPIs represent the occupant lens on three integrative aspects of building performance: resource use (including energy and water), indoor environmental quality, and human–building interactions. A simulation-based case study was conducted to demonstrate how occupant-centric KPIs can be used to quantify the impacts of building operation changes from the occupants’ point of view. Highlights: Occupant-centric metrics are currently ad hoc and limited, yet crucial to inform building design and operations. Literature was reviewed to reveal the state-of-the-art and gaps of occupant-centric metrics. A suite of occupant-centric key performance indicators (KPIs) covering five groups of building services were synthesized. Proposed occupant KPIs represent three aspects of performance: resource use and demand, occupant comfort and health, and human–building interactions. A case study using whole building simulation was conducted to demonstrate the use of occupant-centric KPIs in evaluating building operations during a power outage.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Understanding Portability of Automotive Workload: A Case Study with the Points-to-Image Kernel in SYCL on Heterogeneous Computing Platforms
SYCL is a promising programming model for heterogenous computing across vendors’ devices. In this paper, we study whether SYCL can be applied to an automotive workload and its portability on heterogeneous computing platforms. We explain the automotive benchmark, describe our implementations and optimizations of the benchmark, and evaluate the performance of the benchmarks using SYCL and other programming models on heterogeneous computing devices. The study also allows us to have a better understanding of portability of the benchmark across these platforms.
Jin, Zheming↗
Reassessment of PHDS Fulcrum40h High Purity Germanium (HPGe) Detector System Performance
This report presents a comprehensive evaluation of the updated PHDS Fulcrum40h High Purity Germanium (HPGe) detector system, benchmarking its performance, usability, and software capabilities against the current National Nuclear Security Administration (NNSA) Nuclear Emergency Support Team (NEST) HPGe Detector System Requirements Document. Systematic measurements were conducted using a single Fulcrum40h detector to assess key parameters including gamma efficiency and resolution, neutron detection efficiency, gamma pulse-pileup response, and gamma-to-neutron crosstalk. The Fulcrum40h system, acquired in August 2022, has undergone recent updates by PHDS to address deficiencies identified following the initial NEST requirements release. The results provide critical insights into the detector’s operational capabilities and compliance with NNSA NEST standards.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Benchmark Calculation for BEAVRS Cycles 1 and 2 Using the SCALE 6.3/Polaris–PARCS v3.4.2 Code Package
This study performed the benchmark calculation for BEAVRS to validate the SCALE 6.3/Polaris–PARCS v3.4.2 code package with the ENDF/B-VII.1 56-group library, comparing the simulated results with the measured data. The benchmark results will be used in evaluating uncertainties of the SCALE/Polaris–PARCS code package for pressurized water reactor physics analysis for key nuclear parameters such as reactivity, control bank worth, temperature coefficients, and pin and assembly power peaking factors. This report details plant and fuel design specifications and input data for SCALE/Polaris as well as GenPMAXS and PARCS. Additionally, detailed information is provided for all the input and output files produced for the benchmark calculations. The benchmark results are summarized herein so that they can be used in evaluating uncertainties for key nuclear parameters with other benchmark results.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Questionable Benchmarks [Slides]
An investigation and initial review of ‘questionable benchmarks’ was performed at LANL. Multiple methods were used, including machine learning techniques, to identify 'questionable benchmarks' and/or benchmarks with low uncertainties. Those benchmarks were then reviewed for obvious errors. This is not a recommendation to ICSBEP, but can be used as a starting point for a more comprehensive review. A journal paper is being written on this work.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
OpenEdge: A collaborative, open-source, multi-purpose direct simulation Monte Carlo for plasma simulation in magnetic fusion environments
OpenEdge is a collaborative, open-source, object-oriented Direct Simulation Monte Carlo (DSMC) code, designed specifically for plasma simulations in magnetic fusion environments. Here, the code features include advanced structures, robust capabilities, and an effective parallelization strategy, all of which significantly enhance performance. It includes specialized modules for managing complex particle interactions, including collisions, ionization/recombination, and reflection/sputtering. Benchmarks and performance analyses have confirmed its efficiency and scalability. Versatile and adaptable, OpenEdge is applied across a broad spectrum of plasma-material interaction studies and charged particle transport in various fusion research settings.
Boundary plasma↗
Neutronic Characteristics of ENDF/B-VIII.0 Compared to ENDF/B-VII.1 for Light-Water Reactor Analysis
The Evaluated Nuclear Data File (ENDF)/B-VIII.0 data library was released in 2018. To assess the new data during development and shortly after release, many validation calculations were performed with static, room-temperature benchmarks. Recently, when performing validation of ENDF/B-VIII.0 for pressurized water reactor depletion calculations, a regression in performance compared to ENDF/B-VII.1 was observed. This paper documents extensive benchmark calculations for light-water reactors performed using continuous energy Monte Carlo code with ENDF/B-VII.1 and -VIII.0 and neutronic characteristics of ENDF/B-VIII.0 are discussed and compared to those of ENDF/B-VII.1. It is our recommendation that ENDF/B data library assessment should include reactor-specific benchmark assessments, including depletion cases, such that these types of regressions may be caught earlier in the data development cycle.
21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
A Privacy Preserving Model-Free Optimization and Control Framework for Demand Response from Residential Thermal Loads
We consider the problem of optimizing the cost of procuring electricity for a large collection of homes managed by a load serving entity, by pre-cooling or pre-heating the thermal inertial loads in the homes to avoid procuring power during periods of peak electricity pricing. We would like to accomplish this objective in a completely privacy-preserving and model-free manner, that is, without direct access to the state variables (temperatures or power consumption) or the dynamical models (thermal characteristics) of individual homes, while guaranteeing personal comfort constraints of the consumers. We propose a two-stage optimization and control framework to address this problem. In the first stage, we use a long short-term memory (LSTM) network to predict hourly electricity prices, based on historical pricing data and weather forecasts. Given the hourly price forecast and thermal models of the homes, the problem of designing an optimal power consumption trajectory that minimizes the total electricity procurement cost for the collection of thermal loads can be formulated as a large-scale integer program (with millions of variables) due to the on-off cyclical dynamics of such loads. We provide a simple heuristic relaxation to make this large-scale optimization problem model-free and computationally tractable. In the second stage, we translate the results of this optimization problem into distributed open-loop control laws that can be implemented at individual homes without measuring or estimating their state variables, while simultaneously ensuring consumer comfort constraints. We demonstrate the performance of this approach on a large-scale test case comprising of 500 homes in the Houston area and benchmark its performance against a direct model-based optimization and control solution.
Sivaranjani, S.↗
Simulation of the BEAVRS benchmark using VERA
The Virtual Environment for Reactor Applications (VERA) is being developed by the Consortium for the Advanced Simulation of Light Water Reactors (CASL). Here, VERA is used to perform the Benchmark for Evaluation and Validation of Reactor Simulations (BEAVRS), which provides two cycles’ worth of operating power history, along with a full, detailed description of the geometry, and measured data. Cycle 1 and 2 are simulated with VERA and the results are compared to the measured zero power physics tests, critical boron concentration, and flux maps.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
A Comparative Study of Model Predictive Control and Optimal Causal Control for Heaving Point Absorbers
Efforts by various researchers in recent years to design simple causal control laws that can be applied to WEC devices suggest that these controllers can yield similar levels of energy output as those of more complex non-causal controllers. However, most studies were established without adequately considering device and power conversion system constraints which are relevant design drivers from a cost and economic point of view. It is therefore imperative to understand the benefits of MPC compared to causal control from a performance and constraint handling perspective. In this paper, we compare linear MPC to a casual controller that incorporates constraint handling to benchmark its performance on a one DoF heaving point absorber in a range of wave conditions. Our analysis demonstrates that MPC provides significant performance advantages compared to an optimized causal controller, particularly if significant constraints on device motion and/or forces are imposed. We further demonstrate that distinct control performance regions can be established that correlate well with classical point absorber and volumetric limits of the wave energy conversion device.
42 ENGINEERING↗
A scalable algorithm for the optimization of neural network architectures
In this work, we propose a new scalable method to optimize the architecture of an artificial neural network. The proposed algorithm, called Greedy Search for Neural Network Architecture, aims to determine a neural network with minimal number of layers that is at least as performant as neural networks of the same structure identified by other hyperparameter search algorithms in terms of accuracy and computational cost. Numerical results performed on benchmark datasets show that, for these datasets, our method outperforms state-of-the-art hyperparameter optimization algorithms in terms of attainable predictive performance by the selected neural network architecture, and time-to-solution for the hyperparameter optimization to complete.
97 MATHEMATICS AND COMPUTING↗