Search NASA⌕ Search

SEARCH · Search NASA

Results for “Performance Benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,369 records · Page 76

A large-scale benchmarking of deterministic and stochastic derivative-free optimization algorithms

This presentation summarizes our work in the PrOMMiS project on benchmarking of data-driven optimization algorithms and their applications in self-driving laboratories. This work supports the broader project goal of accelerating the identification of promising separation methods and operating conditions for critical minerals separation processes. We present a systematic benchmarking study of 42 data-driven optimization algorithms on a broad collection of 502 test problems. The results identify BAM, GLCCLUSTER, and MULTIMIN as the most effective optimization solvers, with BAM showing the highest overall performance and solving more than 80% of the benchmark problems. The study also shows that no single solver consistently outperforms the others across all problem types, indicating that our future laboratory applications may benefit from using a small set of strong solvers rather than relying on a single method. The presentation also illustrates an in-silico chemical reactor case study showing that data-driven optimization methods can guide autonomous experimentation in a self-driving laboratory and identify optimal operating conditions within a small number of experiments. Overall, the results provide a basis for selecting efficient optimization methods and demonstrate the practical use of data-driven optimization in self-driving laboratory workflows.

36 MATERIALS SCIENCE↗

Algorithm-guided experimentation for autonomous AI systems in self-driving laboratories

This presentation summarizes our work in the PrOMMiS project on benchmarking of data-driven optimization algorithms and their applications in self-driving laboratories. This work supports the broader project goal of accelerating the identification of promising separation methods and operating conditions for critical minerals separation processes. We present a systematic benchmarking study of 42 data-driven optimization algorithms on a broad collection of 502 test problems. The results identify BAM, GLCCLUSTER, and MULTIMIN as the most effective optimization solvers, with BAM showing the highest overall performance and solving more than 80% of the benchmark problems. The study also shows that no single solver consistently outperforms the others across all problem types, indicating that our future laboratory applications may benefit from using a small set of strong solvers rather than relying on a single method. The presentation also illustrates an in-silico chemical reactor case study showing that data-driven optimization methods can guide autonomous experimentation in a self-driving laboratory and identify optimal operating conditions within a small number of experiments. Overall, the results provide a basis for selecting efficient optimization methods and demonstrate the practical use of data-driven optimization in self-driving laboratory workflows.

36 MATERIALS SCIENCE↗

Performance of electron reconstruction and selection with the CMS detector in proton-proton collisions at √s = 8 TeV

The performance and strategies used in electron reconstruction and selection at CMS are presented based on data corresponding to an integrated luminosity of 19.7 fb -1 , collected in proton-proton collisions at √s = 8 TeV at the CERN LHC. The paper focuses on prompt isolated electrons with transverse momenta ranging from about 5 to a few 100 GeV. A detailed description is given of the algorithms used to cluster energy in the electromagnetic calorimeter and to reconstruct electron trajectories in the tracker. The electron momentum is estimated by combining the energy measurement in the calorimeter with the momentum measurement in the tracker. Benchmark selection criteria are presented, and their performances assessed using Z, Υ, and J/ψ decays into e + + e - pairs. The spectra of the observables relevant to electron reconstruction and selection as well as their global efficiencies are well reproduced by Monte Carlo simulations. The momentum scale is calibrated with an uncertainty smaller than 0.3%. The momentum resolution for electrons produced in Z boson decays ranges from 1.7 to 4.5%, depending on electron pseudorapidity and energy loss through bremsstrahlung in the detector material.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Simulation of Radiation Damping in Rings using Stepwise Ray-Tracing Methods

The ray-tracing code Zgoubi has been subject to benchmarking regarding synchrotron radiation damping in rings, in the past months. This work is reported here. It has been performed in view in particular of further spin diffusion simulations benchmarking, and application to electron beam polarization transmission in e-RHIC.

43 PARTICLE ACCELERATORS↗

Generation of group constants at GRS for the Rostov-2 benchmark

The OECD/NEA benchmark entitled 'Reactivity compensation of boron dilution by stepwise insertion of control rod cluster into the VVER-1000 core' aims at validating and assessing high fidelity multi-physics simulation code capabilities by comparison with the high-quality measurements performed at the nuclear power plant Rostov unit 2. The benchmark is divided into 2 phases: assembly wise and pin-by-pin resolution of the multi-physics problems. Multi-physics simulation requires the generation of an accurate parametrized few-group cross-section library, and so this benchmark offers an opportunity to validate the methodology for their generation. Based on the specifications of the benchmark, the core simulator KMACS developed at GRS was used to generate a library of cross-sections as well as the burnup distribution required to model the transient scenario of the benchmark at the assembly level. The assessment of the library is performed through comparison with measured values obtained at the initial state of the transient. Good agreement was obtained in terms of critical boron concentration, peaking factors and power profiles. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

SoK: What does it Mean to Benchmark Database Forensics?

Relational Database Management Systems are the backbone of modern enterprises and public-sector services, and are thus frequent targets of security incidents, insider threats, and thorough regulatory audits. Consequently, databases have become key sources of digital evidence, requiring investigators to reconstruct past activity from audit logs, transaction logs, and backups. Although benchmarking frameworks such as those developed by the Transaction Processing Performance Council (TPC) are widely used to evaluate database performance, they do not capture forensic requirements such as evidentiary completeness, tamper-evidence, chain of custody, or regulatory compliance under GDPR and CCPA. This survey examines the emerging domain of forensic database benchmarking. We gathered prior research on database forensics, secure logging, and tamper-evident data structures; we analyze modern forensic-ready features in commercial and open-source systems (SQL Server Ledger, Oracle Blockchain Tables, PostgreSQL pgAudit, Db2 Audit, Aurora Database Activity Streams, Oracle Real Application Security and IBM Guardium) and assess why existing benchmarks are insufficient. We propose forensic workloads, metrics, and methodologies that incorporate adversarial stressors, deleted-record recovery, and backup analysis. We also identify open research problems and call for a community-driven forensic benchmark suite. The result is an idea for evaluating not only database performance but also forensic soundness, bridging the gap between system engineering, compliance, and digital investigations.

Lenard, Ben↗

Frontier (HPE Cray EX) Exascale Supercomputer at the Oak Ridge Leadership Computing Facility

Frontier is the HPE Cray EX exascale supercomputer deployed and operated by the Oak Ridge Leadership Computing Facility (OLCF) at Oak Ridge National Laboratory (ORNL). Frontier is designed for large-scale modeling, simulation, and AI workloads and is built from HPE Cray EX system architecture with AMD CPUs and AMD Instinct GPU accelerators connected by the HPE Slingshot interconnect. System composition (representative production configuration): Frontier is composed of approximately 74 cabinets with 128 compute nodes per cabinet (~9,400 compute nodes total). Each compute node contains one 64-core AMD EPYC CPU and four AMD Instinct MI250X GPUs. Nodes are connected using HPE Slingshot (Slingshot-200 class) networking with multiple NIC ports per node providing high injection bandwidth. Frontier is connected to the Orion parallel file system (multi-tier Lustre) providing a large, center-wide high-performance storage namespace. Operational context: Frontier entered public prominence as the first system to reach No. 1 on the TOP500 list in May 2022 (HPL benchmark), establishing the first widely recognized exascale-era performance milestone. The system supports DOE Office of Science mission workloads and enables leadership-class computational science and AI for open science users.

AMD EPYC↗

Plutonium Production Reactor Progression Problems: Magnox Neutronics Benchmarks

The goal of this document is to provide a set of comprehensive benchmarking neutronics problems for gas-cooled, graphite-moderated reactors. The problems are designed to be representative problems that increase in complexity to test performance of various codes. For the simulation of the benchmarking problems, the Shift Monte Carlo code and MPACT deterministic code are chosen because of their state-of-the-art capabilities, which are optimized for various reactor designs. Because these codes have been developed primarily for light water reactor applications, their application to graphite-moderated, gas-cooled reactors has not been extensively explored. This report attempts to evaluate benchmarks for a Magnox-style reactor, one that is graphite-moderated and gas-cooled. Parameters that are calculated and presented in this benchmark include the eigenvalue, peaking factors, and isotope concentrations. Results show good agreement between Shift and MPACT, and demonstrate the capability of MPACT to model graphite-moderated systems well.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Cleanroom Energy Efficiency: Metrics and Benchmarks

Cleanrooms are among the most energy-intensive types of facilities. This is primarily due to the cleanliness requirements that result in high airflow rates and system static pressures, as well as process requirements that result in high cooling loads. Various studies have shown that there is a wide range of cleanroom energy efficiencies and that facility managers may not be aware of how energy efficient their cleanroom facility can be relative to other cleanroom facilities with the same cleanliness requirements. Metrics and benchmarks are an effective way to compare one facility to another and to track the performance of a given facility over time. This article presents the key metrics and benchmarks that facility managers can use to assess, track, and manage their cleanroom energy efficiency or to set energy efficiency targets for new construction. These include system-level metrics such as air change rates, air handling W/cfm, and filter pressure drops. Operational data are presented from over 20 different cleanrooms that were benchmarked with these metrics and that are part of the cleanroom benchmark dataset maintained by Lawrence Berkeley National Laboratory (LBNL). Overall production efficiency metrics for cleanrooms in 28 semiconductor manufacturing facilities in the United States and recorded in the Fabs21 database are also presented.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Dynamic signatures of electronically nonadiabatic coupling in sodium hydride: a rigorous test for the symmetric quasi-classical model applied to realistic, ab initio electronic states in the adiabatic representation

Sodium hydride (NaH) in the gas phase presents a seemingly simple electronic structure making it a potentially tractable system for the detailed investigation of nonadiabatic molecular dynamics from both computational and experimental standpoints. The single vibrational degree of freedom, as well as the strong nonadiabatic coupling that arises from the excited electronic states taking on considerable ionic character, provides a realistic chemical system to test the accuracy of quasi-classical methods to model population dynamics where the results are directly comparable against quantum mechanical benchmarks. Here, using a simulated pump–probe type experiment, this work presents computational predictions of population transfer through the avoided crossings of NaH via symmetric quasi-classical Meyer–Miller (SQC/MM), Ehrenfest, and exact quantum dynamics on realistic, ab initio potential energy surfaces. The main driving force for population transfer arises from the ground vibrational level of the D 1 Σ + adiabatic state that is embedded in the manifold of near-dissociation C 1 Σ + vibrational states. When coupled through a sharply localized first-order derivative coupling most of the population transfers between t = 15 and t = 30 fs depending on the initially excited vibronic wavepacket. While quantum mechanical effects are expected due to the reduced mass of NaH, predictions of the population dynamics from both the SQC/MM and Ehrenfest models perform remarkably well against the quantum dynamics benchmark. Additionally, an analysis of the vibronic structure in the nonadiabatically coupled regime is presented using a variational eigensolver methodology.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Reducing the cost of energy estimation in the variational quantum eigensolver algorithm with robust amplitude estimation

Quantum chemistry and materials is one of the most promising applications of quantum computing. Yet much work is still to be done in matching industry-relevant problems in these areas with quantum algorithms that can solve them. Most previous efforts have carried out resource estimations for quantum algorithms run on large-scale fault-tolerant architectures, which include the quantum phase estimation algorithm. In contrast, few have assessed the performance of near-term quantum algorithms, which include the variational quantum eigensolver (VQE) algorithm. Recently, a large-scale benchmark study [Gonthier et al. 2020] found evidence that the performance of the variational quantum eigensolver for a set of industry-relevant molecules may be too inefficient to be of practical use. This motivates the need for developing and assessing methods that improve the efficiency of VQE. In this work, we predict the runtime of the energy estimation subroutine of VQE when using robust amplitude estimation (RAE) to estimate Pauli expectation values. Under conservative assumptions, our resource estimation predicts that RAE can reduce the runtime over the standard estimation method in VQE by one to two orders of magnitude. Despite this improvement, we find that the runtimes are still too large to be practical. These findings motivate two complementary efforts towards quantum advantage: 1) the investigation of more efficient near-term methods for ground state energy estimation and 2) the development of problem instances that are of industrial value and classically challenging, but better suited to quantum computation.

Johnson, Peter D.↗

Memory access optimization for particle operations in computational fluid dynamics-discrete element method simulations

Computational Fluid Dynamics - Discrete Element Method is used to model gas-solid systems in several applications in energy, pharmaceutical and petrochemical industries. Computational performance bottlenecks often limit the problem sizes that can be simulated at industrial scale. The data structures used to store several millions of particles in such large-scale simulations have a large memory footprint that does not fit into the processor cache hierarchies on current high-performance-computing platforms, leading to reduced computational performance. This paper specifically addresses this aspect of memory access bottlenecks in industrial scale simulations. The use of space-filling curves to improve memory access patterns is described and their impact on computational performance is quantified in both shared and distributed memory parallelization paradigms. The Morton space filling curve applied to uniform grids and k-dimensional tree partitions are used to reorder the particle data-structure thus improving spatial and temporal locality in memory. The performance impact of these techniques when applied to two benchmark problems, namely the homogeneous-cooling-system and a fluidized-bed, are presented. We report these optimization techniques lead to approximately two-fold performance improvement in particle focused operations such as neighbor-list creation and data-exchange, with ~ 1.5 times overall improvement in a fluidization simulation with 1.27 million particles.

97 MATHEMATICS AND COMPUTING↗

Interpretation of 3D void measurements with Tripoli4.6/JEFF3.1.1 Monte Carlo code

The present work details the first analysis of the 3D void phase conducted during the EPICURE/UM17x17/7% mixed UOX/MOX configuration. This configuration is composed of a homogeneous central 17x17 MOX-7% assembly, surrounded by portions of 17x17 1102 assemblies with guide-tubes. The void bubble is modelled by a small waterproof 5x5 fuel pin parallelepiped box of 11 cm height, placed in the centre of the MOX assembly. This bubble, initially placed at the core mid-plane, is then moved in different axial positions to study the evolution in the core of the axial perturbation. Then, to simulate the growing of this bubble in order to understand the effects of increased void fraction along the fuel pin, 3 and 5 bubbles have been stacked axially, from the core mid-plane. The C/E comparison obtained with the Monte Carlo code Tripoli4 for both radial and axial fission rate distributions, and in particular the reproduction of the very important flux gradients at the void/water interfaces, changing as the bubble is displaced along the z-axis are very satisfactory. It demonstrates both the capability of the code and its library to reproduce this kind of situation, as the very good quality of the experimental results, confirming the UM-17x17 as an excellent experimental benchmark for 3D code validation. This work has been performed within the frame of the V and V program for the future APOLL03 deterministic code of CEA starting in 2012, and its V and V benchmarking database. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

US Department of Energy, Office of Science High Performance Computing Facility Operational Assessment 2019 Oak Ridge Leadership Computing Facility

Oak Ridge National Laboratory's (ORNL's) Leadership Computing Facility (OLCF) continues to surpass its operational target goals: supporting users; delivering fast, reliable computational ecosystems; creating innovative solutions for high performance computing (HPC) needs; and managing risks, safety, and security associated with operating some of the most powerful computers in the world. The results can be seen in the cutting-edge science conducted by users and the praise from the research community. Calendar year (CY) 2019 was a big year as OLCF staff ran five world-class resources (the leadershipclass computers Titan and Summit, the large analysis cluster called Eos, and the massive parallel filesystems called Atlas and Alpine)) and also began power and cooling upgrades for a 2021 exascale system called Frontier. While continuing exceptional operation of Titan, Eos, and Rhea, the OLCF released the Summit supercomputer for production on January 1, 2019. Summit debuted as the most capable and efficient system in its class and has been recognized as the most powerful system in the world for its performance on both the high performance linpack (HPL) and conjugate gradient (HPCG) benchmark applications since June 2018 according to TOP500. Summit represents the culmination of a multiyear effort between the OLCF, IBM, NVIDIA, and Mellanox to deliver a system that is unmatched for modeling, simulation, data analysis, and learning. To hit the ground running with science-ready applications on day one, application teams worked closely with the OLCF through the Center for Accelerated Application Readiness (CAAR) program for years in advance of the Summit deployment. CY 2019 was filled with outstanding results and accomplishments: a very high rating from users on overall satisfaction for the sixth year in a row; a tremendous amount of core-hours delivered to researchers from two leadership-class systems; and success in delivering on the allocation split of roughly 60%, 30%, and 10% of core-hours offered for the Innovative and Novel Computational Impact on Theory and Experiment (INCITE), Advanced Scientific Computing Research Leadership Computing Challenge (ALCC), and Director's Discretionary (DD) programs, respectively (see Operational Performance section). These accomplishments, coupled with the high utilization rates (overall and capability usage), represent the fulfillment of the promise of both leadership-class machines: efficient facilitation of leadership-class computational applications. Table ES.1 presents a summary of the 2019 OLCF metric targets and the associated results. More information can be found in the Operational Performance section for each OLCF resource. The scientific accomplishments of OLCF users are a strong indication of long-term operational success, with publications this year in such notable journals and publications as Nature, Nature Physics, Nature Plants, Physical Review X, Journal of the American Physical Society, Cell, Nano Letters, and Trends in Biotechnology. Crucial domain-specific discoveries facilitated by resources at the OLCF are described in the High Performance Computing Facility Operational Assessment 2019 Oak Ridge Leadership Computing Facility (OAR) Strategic Results section. For example, researchers used Summit to pinpoint and understand the production of proteins from genetic information, including mutations and the functional expression of disease (Section 8.2).

97 MATHEMATICS AND COMPUTING↗

Scalable Representation Learning for Long-Term Augmented Reality-Based Information Delivery in Collaborative Human-Robot Perception

Augmented reality (AR)-based information delivery has been attracting an increasing attention in the past few years to improve com- munication in human-robot teaming. In the long-term use of AR systems for collaborative human-robot perception, one of the biggest challenges is to perform place and scene matching under long-term environmental changes, such as dramatic variations in lighting, weather and vegetation across di erent times of the day, months, and seasons. To address this challenge, we introduce a novel representation learning approach that learns a scalable long-term representation model that can be used for place and scene matching in various long-term conditions. Our approach is formulated as a regularized optimization problem, which selects the most representative scene templates in di erent scenarios to construct a scalable representation of the same place that can exhibit signi cant long-term environment changes. Our approach adaptively learns to select a small subset of the templates to construct the representation model, based on a user-de ned representativeness threshold, which makes the learned model highly scalable to the long-term variations in real-world applications. To solve the formulated optimization problem, a new algo- rithmic solver is designed, which is theoretically guaranteed to converge to the global optima. Experiments are conducted using two large-scale benchmark datasets, which have demonstrated the superior performance of our approach for long-term place and scene matching.

42 ENGINEERING↗

A Roadmap to Low-Cost Hydrogen with Hydroxide Exchange Membrane Electrolyzers

Hydrogen is an ideal alternative energy carrier to generate power for all of society's energy demands including grid, industrial, and transportation sectors. Among the hydrogen production methods, water electrolysis is a promising method because of its zero greenhouse gas emission and its compatibility with all types of electricity sources. Alkaline electrolyzers (AELs) and proton exchange membrane electrolyzers (PEMELs) are currently used to produce hydrogen. AELs are commercially mature and are used in a variety of industrial applications, while PEMELs are still being developed and find limited application. In comparison with AELs, PEMELs have more compact structure and can achieve higher current densities. Recently, however, an alternative technology to PEMELs, hydroxide exchange membrane electrolyzers (HEMELs), has gained considerable attention due to the possibility to use platinum group metal (PGM)-free electrocatalysts and cheaper membranes, ionomers, and construction materials and its potential to achieve performance parity with PEMELs. Here, the state-of-the-art AELs and PEMELs along with the current status of HEMELs are discussed in terms of their positive and negative aspects. Additionally discussed are electrocatalyst, membrane, and ionomer development needs for HEMELs and benchmark electrocatalysts in terms of the cost-performance tradeoff.

08 HYDROGEN↗

Thermal loading analysis of the ring injection dump for the Spallation Neutron Source facility

The ring injection dump (RID) is the largest beam dump in the Spallation Neutron Source facility at Oak Ridge National Laboratory and accepts a fraction of the beam from the ring that is not captured in the ring during injection. Thermo-fluid modeling of the RID components was performed during the initial design by using the software ANSYS CFX to characterize the power rating for the RID. From the calculations, the power rating was lowered from 200 kW to 150 kW, at 1.3 GeV beam energy due to concerns regarding the heating of the concrete structure. The Proton Power Upgrade project scope included reevaluating the power limit to the RID in anticipation that increasing it would provide more operational flexibility. The focus was on the shielding because it was the limiting factor, and data collected from thermocouples installed on the structure were used to benchmark the analysis. A three-stage process was adopted in the latest study. First, the already existing steady-state calculations were validated and extended with the latest Monte Carlo N-Particle source term calculations. Second, transient calculations were conducted to capture the dynamic state of the system in response to the energy dumped to the RID during the following operational period of the beam from 2005–2019 (14 years), and sensitivity analyses of crucial parameters were performed to benchmark the model with reasonable accuracy. Finally, an idealized transient cycle analysis was conducted with realistic duty factors to predict the temperature distribution at higher beam powers. Overall, the results confirmed the accuracy of the original steady state model but allowed the development of a new validated transient model with higher accuracy for future analyses.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Levelized cost of energy for a Backward Bent Duct Buoy

The Reference Model Project, supported by the U.S. Department of Energy, was developed to provide publically available technical and economic benchmarks for a variety of marine energy converters. The methodology to achieve these benchmarks is to develop public domain designs that incorporate power performance estimates, structural models, anchor and mooring designs, power conversion chain designs, and estimates of the operations and maintenance, installation, and environmental permitting required. The reference model designs are intended to be conservative, robust, and experimentally verified. The Backward Bent Duct Buoy (BBDB) presented in this paper is one of three wave energy conversion devices studied within the Reference Model Project. Comprehensive modeling of the BBDB in a Northern California climate has enabled a full levelized cost of energy (LCOE) analysis to be completed on this device.

marine hydrokinetic energy development↗