SEARCH · Search NASA

Results for “performance benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,171 records · Page 65

All sources NASA NTRS DOE OSTI

DOE OSTI2022

WfBench: Automated Generation of Scientific Workflow Benchmarks

The prevalence of scientific workflows with high computational demands calls for their execution on various distributed computing platforms, including large-scale leadership-class high-performance computing (HPC) clusters. To handle the deployment, monitoring, and optimization of workflow executions, many workflow systems have been developed over the past decade. There is a need for workflow benchmarks that can be used to evaluate the performance of workflow systems on current and future software stacks and hardware platforms.We present a generator of realistic workflow benchmark specifications that can be translated into benchmark code to be executed with current workflow systems. Our approach generates workflow tasks with arbitrary performance characteristics (CPU, memory, and I/O usage) and with realistic task dependency structures based on those seen in production workflows. We present experimental results that show that our approach generates benchmarks that are representative of production workflows, and conduct a case study to demonstrate the use and usefulness of our generated benchmarks to evaluate the performance of workflow systems under different configuration scenarios.

Coleman, Taina↗

DOE OSTI2019

PPT-GPU: Scalable GPU Performance Modeling

Performance modeling is a challenging problem due to the complexities of hardware architectures. In this paper, we present PPT-GPU, a scalable and accurate simulation framework that enables GPU code developers and architects to predict the performance of applications in a fast, and accurate manner on different GPU architectures. PPT-GPU is part of the open source project, Performance Prediction Toolkit (PPT) developed at the Los Alamos National Laboratory. We extend the old GPU model in PPT that predict the runtimes of computational physics codes to offer better prediction accuracy, for which, we add models for different memory hierarchies found in GPUs and latencies for different instructions. To further show the utility of PPT-GPU, we compare our model against real GPU device(s) and the widely used cycle-accurate simulator, GPGPU-Sim using different workloads from RODINIA and Parboil benchmarks. The results indicate that the predicted performance of PPT-GPU is within a 10% error compared to the real device(s). In addition, PPT-GPU is highly scalable, where it is up to 450x faster than GPGPU-Sim with more accurate results.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Bootstrap embedding for interacting electrons in phonon coherent-state mean field

Here, we develop a Fermi–Bose bootstrap embedding framework for the ground state of interacting electrons coupled to a phonon mean field. The method combines bootstrap embedding for correlated electrons with a self-consistent coherent-state mean-field treatment for phonons. This method models the interacting electron–phonon problem as a system of correlated electrons traveling in a self-consistently specified potential landscape, allowing for efficient treatment of large lattice systems. Convergence of the methods for fragment size and total system size is demonstrated for the one-dimensional Hubbard–Holstein model for up to 350 sites. Finite-size scaling is performed to extrapolate to the infinite system size. Benchmarking against the density matrix renormalization group for a small 8-site system at half- and quarter-filling shows an orders-of-magnitude runtime advantage. The comparison further reveals that the method performs best in regimes dominated by localization, such as the Mott insulating phase and the strong-coupling tiny polaron regime, where the local embedding ansatz is still valid. However, due to the mean-field treatment for phonons, we find limitations of our methods in the weakly coupled delocalized region and at the Peierls transition, where quantum phonon fluctuations and long-range kinetic correlations become substantial.

Islam, Shariful [North Carolina State University, ↗

DOE OSTI2025

An Alternative Ensemble Streamflow Prediction Approach Using Improved Subseasonal Precipitation Forecasts from the North America Multi-Model Ensemble Phase II

In this article, streamflow forecasting at a subseasonal time scale (10–30 days into the future) is important for various human activities. The ensemble streamflow prediction (ESP) is a widely applied technique for subseasonal streamflow forecasting. However, ESP’s reliance on the randomly resampled historical precipitation limits its predictive capability. Available dynamical subseasonal precipitation forecasts provide an alternative to the randomly resampled precipitation in ESP. Prior studies found the predictive performance of raw subseasonal precipitation forecast is limited in many regions such as the central south of the United States, which raises questions about its effectiveness in assisting streamflow forecasting. To further assess the hydrologic applicability of dynamical subseasonal precipitation forecasts, we test the subseasonal precipitation forecast from North America Multi-Model Ensemble Phase II (NMME-2) at four watersheds in the central south region of the United States. The subseasonal precipitation forecasts are postprocessed with bias correction and spatial disaggregation (BCSD) to correct bias and improve spatial resolution before replacing the randomly resampled precipitation in ESP for streamflow predictions. The performance of the resulting streamflow predictions is benchmarked with ESP. Evaluation is conducted using Kling–Gupta Efficiency (KGE), continuous ranked probability score (CRPS), probability of detection (POD), false alarm ratios (FARs), as well as reliability diagrams. Our results suggest that BCSD-corrected subseasonal precipitation forecasts lead to overall improved streamflow predictions due to added skills in winter and spring. Our results also suggest that BCSD-corrected subseasonal precipitation forecasts lead to improved predictions on the occurrence of high-percentile streamflow values above 75%. Overall, BCSD-corrected subseasonal precipitation has shown promising performance, highlighting its potential broader applications for river and flood forecasting.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Biochemical Conversion of Lignocellulosic Biomass to Hydrocarbon Fuels and Products: 2022 State of Technology and Future Research

The annual State of Technology (SOT) assessment is a key activity for biochemical platform research. It allows the impact of research progress (both directly achieved in-house at the National Renewable Energy Laboratory [NREL] and indirectly extrapolated from available public data) to be quantified in terms of economic improvements in the overall cellulosic biofuel production process for a particular conversion pathway. As such, initial benchmarks can be established for currently demonstrated performance and progress can be tracked toward out-year goals to ultimately demonstrate cost-competitive biofuel technology. Building upon efforts to progress NREL's biochemical platform R&D work toward ultimate 2030 goals to demonstrate less than $2.50/gallon gasoline equivalent (GGE) fuel selling prices, experimental and techno-economic analysis (TEA) activities have primarily focused on "advanced" biochemical processing strategies to fuels and coproducts, guided by TEA modeling to highlight key barriers and priorities toward achieving this goal across a number of potential bioconversion pathways. The purpose of the present effort is to benchmark the latest experimental developments for these pathways as quantified by modeled minimum fuel selling prices (MFSPs), as a measure of current status relative to those final targets. For this SOT, TEA models were run for two separate biological conversion pathways to fuels, based on available data for integrated biomass deconstruction and hydrolysate processing; namely carboxylic acids (primarily butyric acid) and diols (2,3-butanediol [BDO]), reflecting NREL's recently published 2018 biochemical design report focused on those two pathways. The models were run across two scenarios for lignin utilization, namely combustion and conversion to value-added coproducts.

09 BIOMASS FUELS↗

DOE OSTI2021

Cumulative $\chi^2$ Metric for VALID for ENDF/B-VII.1 and ENDF/B-VIII.0 in SCALE 6.3b9

Nuclear data testing is generally performed in a manner similar to validation: a series of critical benchmark experiments is modeled, and the differences between calculated and expected results are examined to understand the performance of a code and cross section set. This process is repeated for multiple sets of experiments with different transport codes and sometimes with different datasets. Occasionally, data processing differences are also important in understanding the differences in calculated results. Relevant experts then meet periodically to compare results and set priorities for improvement in the next round of data updates. This process has been largely effective, despite the confounding factors of different benchmark sets, application areas, sponsors, nationalities, and the resulting different priorities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

Time series anomaly detection in power electronics signals with recurrent and ConvLSTM autoencoders

The anomalies in the high voltage converter modulator (HVCM) remain a major down time for the spallation neutron source facility, that delivers the most intense neutron beam in the world for scientific materials research. In this work, we propose neural network architectures based on Recurrent AutoEncoders (RAE) to detect anomalies ahead of time in the power signals coming from the HVCM. Bi-directional gated recurrent unit, bi-directional long-short term memory (LSTM), and convolutional LSTM (ConvLSTM) are developed, trained, and tested using real experimental signals from the HVCM module. The results show a good performance of the proposed RAE models, achieving precision up to 91%, recall up to 88%, false omission rate as low as 20% (i.e. 80% of the anomalies were detected), and area under the ROC curve up to 0.9. The three RAE models provide very comparable performance, with LSTM showing slightly better performance than GRU and ConvLSTM. The RAE models are benchmarked against other anomaly detection methods, including isolation forest, support vector machine, local outlier factor, feedforward and convolutional autoencoders, and others; showing a better performance. Here, the results of this study demonstrate the promising potential of RAE in anomaly detection for real-world power systems, and for increasing the reliability of the HVCM modules in the spallation neutron source.

42 ENGINEERING↗

DOE OSTI2026

A large-scale benchmarking of deterministic and stochastic derivative-free optimization algorithms

This presentation summarizes our work in the PrOMMiS project on benchmarking of data-driven optimization algorithms and their applications in self-driving laboratories. This work supports the broader project goal of accelerating the identification of promising separation methods and operating conditions for critical minerals separation processes. We present a systematic benchmarking study of 42 data-driven optimization algorithms on a broad collection of 502 test problems. The results identify BAM, GLCCLUSTER, and MULTIMIN as the most effective optimization solvers, with BAM showing the highest overall performance and solving more than 80% of the benchmark problems. The study also shows that no single solver consistently outperforms the others across all problem types, indicating that our future laboratory applications may benefit from using a small set of strong solvers rather than relying on a single method. The presentation also illustrates an in-silico chemical reactor case study showing that data-driven optimization methods can guide autonomous experimentation in a self-driving laboratory and identify optimal operating conditions within a small number of experiments. Overall, the results provide a basis for selecting efficient optimization methods and demonstrate the practical use of data-driven optimization in self-driving laboratory workflows.

36 MATERIALS SCIENCE↗

DOE OSTI2026

Algorithm-guided experimentation for autonomous AI systems in self-driving laboratories

36 MATERIALS SCIENCE↗

NASA NTRS1997

Least-Squares Spectral Element Solutions to the CAA Workshop Benchmark Problems

This paper presents computed results for some of the CAA benchmark problems via the acoustic solver developed at Rocketdyne CFD Technology Center under the corporate agreement between Boeing North American, Inc. and NASA for the Aerospace Industry Technology Program. The calculations are considered as benchmark testing of the functionality, accuracy, and performance of the solver. Results of these computations demonstrate that the solver is capable of solving the propagation of aeroacoustic signals. Testing of sound generation and on more realistic problems is now pursued for the industrial applications of this solver. Numerical calculations were performed for the second problem of Category 1 of the current workshop problems for an acoustic pulse scattered from a rigid circular cylinder, and for two of the first CAA workshop problems, i. e., the first problem of Category 1 for the propagation of a linear wave and the first problem of Category 4 for an acoustic pulse reflected from a rigid wall in a uniform flow of Mach 0.5. The aim for including the last two problems in this workshop is to test the effectiveness of some boundary conditions set up in the solver. Numerical results of the last two benchmark problems have been compared with their corresponding exact solutions and the comparisons are excellent. This demonstrates the high fidelity of the solver in handling wave propagation problems. This feature lends the method quite attractive in developing a computational acoustic solver for calculating the aero/hydrodynamic noise in a violent flow environment.

Lin, Wen H.↗

DOE OSTI2022

Generation of group constants at GRS for the Rostov-2 benchmark

The OECD/NEA benchmark entitled 'Reactivity compensation of boron dilution by stepwise insertion of control rod cluster into the VVER-1000 core' aims at validating and assessing high fidelity multi-physics simulation code capabilities by comparison with the high-quality measurements performed at the nuclear power plant Rostov unit 2. The benchmark is divided into 2 phases: assembly wise and pin-by-pin resolution of the multi-physics problems. Multi-physics simulation requires the generation of an accurate parametrized few-group cross-section library, and so this benchmark offers an opportunity to validate the methodology for their generation. Based on the specifications of the benchmark, the core simulator KMACS developed at GRS was used to generate a library of cross-sections as well as the burnup distribution required to model the transient scenario of the benchmark at the assembly level. The assessment of the library is performed through comparison with measured values obtained at the initial state of the transient. Good agreement was obtained in terms of critical boron concentration, peaking factors and power profiles. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2026

SoK: What does it Mean to Benchmark Database Forensics?

Relational Database Management Systems are the backbone of modern enterprises and public-sector services, and are thus frequent targets of security incidents, insider threats, and thorough regulatory audits. Consequently, databases have become key sources of digital evidence, requiring investigators to reconstruct past activity from audit logs, transaction logs, and backups. Although benchmarking frameworks such as those developed by the Transaction Processing Performance Council (TPC) are widely used to evaluate database performance, they do not capture forensic requirements such as evidentiary completeness, tamper-evidence, chain of custody, or regulatory compliance under GDPR and CCPA. This survey examines the emerging domain of forensic database benchmarking. We gathered prior research on database forensics, secure logging, and tamper-evident data structures; we analyze modern forensic-ready features in commercial and open-source systems (SQL Server Ledger, Oracle Blockchain Tables, PostgreSQL pgAudit, Db2 Audit, Aurora Database Activity Streams, Oracle Real Application Security and IBM Guardium) and assess why existing benchmarks are insufficient. We propose forensic workloads, metrics, and methodologies that incorporate adversarial stressors, deleted-record recovery, and backup analysis. We also identify open research problems and call for a community-driven forensic benchmark suite. The result is an idea for evaluating not only database performance but also forensic soundness, bridging the gap between system engineering, compliance, and digital investigations.

Lenard, Ben↗

DOE OSTI2022

Frontier (HPE Cray EX) Exascale Supercomputer at the Oak Ridge Leadership Computing Facility

Frontier is the HPE Cray EX exascale supercomputer deployed and operated by the Oak Ridge Leadership Computing Facility (OLCF) at Oak Ridge National Laboratory (ORNL). Frontier is designed for large-scale modeling, simulation, and AI workloads and is built from HPE Cray EX system architecture with AMD CPUs and AMD Instinct GPU accelerators connected by the HPE Slingshot interconnect. System composition (representative production configuration): Frontier is composed of approximately 74 cabinets with 128 compute nodes per cabinet (~9,400 compute nodes total). Each compute node contains one 64-core AMD EPYC CPU and four AMD Instinct MI250X GPUs. Nodes are connected using HPE Slingshot (Slingshot-200 class) networking with multiple NIC ports per node providing high injection bandwidth. Frontier is connected to the Orion parallel file system (multi-tier Lustre) providing a large, center-wide high-performance storage namespace. Operational context: Frontier entered public prominence as the first system to reach No. 1 on the TOP500 list in May 2022 (HPL benchmark), establishing the first widely recognized exascale-era performance milestone. The system supports DOE Office of Science mission workloads and enables leadership-class computational science and AI for open science users.

AMD EPYC↗

DOE OSTI2021

Plutonium Production Reactor Progression Problems: Magnox Neutronics Benchmarks

The goal of this document is to provide a set of comprehensive benchmarking neutronics problems for gas-cooled, graphite-moderated reactors. The problems are designed to be representative problems that increase in complexity to test performance of various codes. For the simulation of the benchmarking problems, the Shift Monte Carlo code and MPACT deterministic code are chosen because of their state-of-the-art capabilities, which are optimized for various reactor designs. Because these codes have been developed primarily for light water reactor applications, their application to graphite-moderated, gas-cooled reactors has not been extensively explored. This report attempts to evaluate benchmarks for a Magnox-style reactor, one that is graphite-moderated and gas-cooled. Parameters that are calculated and presented in this benchmark include the eigenvalue, peaking factors, and isotope concentrations. Results show good agreement between Shift and MPACT, and demonstrate the capability of MPACT to model graphite-moderated systems well.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2022

Dynamic signatures of electronically nonadiabatic coupling in sodium hydride: a rigorous test for the symmetric quasi-classical model applied to realistic, ab initio electronic states in the adiabatic representation

Sodium hydride (NaH) in the gas phase presents a seemingly simple electronic structure making it a potentially tractable system for the detailed investigation of nonadiabatic molecular dynamics from both computational and experimental standpoints. The single vibrational degree of freedom, as well as the strong nonadiabatic coupling that arises from the excited electronic states taking on considerable ionic character, provides a realistic chemical system to test the accuracy of quasi-classical methods to model population dynamics where the results are directly comparable against quantum mechanical benchmarks. Here, using a simulated pump–probe type experiment, this work presents computational predictions of population transfer through the avoided crossings of NaH via symmetric quasi-classical Meyer–Miller (SQC/MM), Ehrenfest, and exact quantum dynamics on realistic, ab initio potential energy surfaces. The main driving force for population transfer arises from the ground vibrational level of the D 1 Σ + adiabatic state that is embedded in the manifold of near-dissociation C 1 Σ + vibrational states. When coupled through a sharply localized first-order derivative coupling most of the population transfers between t = 15 and t = 30 fs depending on the initially excited vibronic wavepacket. While quantum mechanical effects are expected due to the reduced mass of NaH, predictions of the population dynamics from both the SQC/MM and Ehrenfest models perform remarkably well against the quantum dynamics benchmark. Additionally, an analysis of the vibronic structure in the nonadiabatically coupled regime is presented using a variational eigensolver methodology.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Reducing the cost of energy estimation in the variational quantum eigensolver algorithm with robust amplitude estimation

Quantum chemistry and materials is one of the most promising applications of quantum computing. Yet much work is still to be done in matching industry-relevant problems in these areas with quantum algorithms that can solve them. Most previous efforts have carried out resource estimations for quantum algorithms run on large-scale fault-tolerant architectures, which include the quantum phase estimation algorithm. In contrast, few have assessed the performance of near-term quantum algorithms, which include the variational quantum eigensolver (VQE) algorithm. Recently, a large-scale benchmark study [Gonthier et al. 2020] found evidence that the performance of the variational quantum eigensolver for a set of industry-relevant molecules may be too inefficient to be of practical use. This motivates the need for developing and assessing methods that improve the efficiency of VQE. In this work, we predict the runtime of the energy estimation subroutine of VQE when using robust amplitude estimation (RAE) to estimate Pauli expectation values. Under conservative assumptions, our resource estimation predicts that RAE can reduce the runtime over the standard estimation method in VQE by one to two orders of magnitude. Despite this improvement, we find that the runtimes are still too large to be practical. These findings motivate two complementary efforts towards quantum advantage: 1) the investigation of more efficient near-term methods for ground state energy estimation and 2) the development of problem instances that are of industrial value and classically challenging, but better suited to quantum computation.

Johnson, Peter D.↗

NASA NTRSDate not supplied

Least Squares Reverse Time Migration (LSRTM) for Damage Imaging in Composite Laminates

A method for adapting least squares reverse time migration (LSRTM) for ultrasonic guided wave imaging of composite laminates is proposed in this paper. As composites become more widely used in fields such as the aerospace industry, the need for high-resolution imaging in structural health monitoring (SHM) and nondestructive evaluation (NDE) is also growing. For instance, delamination is a common problem in composite laminates, which has led to a certain degree of apprehension in the use of composite materials for load-bearing structures. Although the solver-based imaging techniques using conventional reverse time migration (RTM) methods illuminate damage with a wide range of damage-scattering effects, the resulting images do not fully define the damage regions due to the limited data acquisition aperture, sensor density, frequencies/wavelengths, and incompleteness of adjoint reconstruction. Previously, the LSRTM theory was derived by incorporating a least squares migration (LSM) algorithm into RTM, and benchmarked its high-resolution damage imaging performance for isotropic plates. To improve damage imaging in composite laminates, a process to create an ultrasonic guided wave-based LSRTM method for anisotropic materials is proposed in this paper. The derivation of the forward modeling operator and the adjoint operator is presented. Numerical case studies were conducted to show the improvement of LSRTM over RTM in mapping damage in composite plates. Multiple damage sites or damage with a complex shape were created in the numerical studies based on 1) Born approximation-based modeling or 2) reduction of the plate stiffness-based modeling. LSRTM begins by using a forward modeling process. The forward modeling accuracy for the flexural wave solver used in this study for LSRTM was also compared with the results from a three-dimensional rotated staggered grid (RSG) elastodynamic finite difference solver, previously open-sourced by NASA Langley Research Center. These benchmarking studies have shown that the LSRTM method is more effective in reducing artifacts, improving resolution, and enhancing damage model reconstruction compared to RTM in composite laminates.

Jiaze He↗

DOE OSTI2022

Memory access optimization for particle operations in computational fluid dynamics-discrete element method simulations

Computational Fluid Dynamics - Discrete Element Method is used to model gas-solid systems in several applications in energy, pharmaceutical and petrochemical industries. Computational performance bottlenecks often limit the problem sizes that can be simulated at industrial scale. The data structures used to store several millions of particles in such large-scale simulations have a large memory footprint that does not fit into the processor cache hierarchies on current high-performance-computing platforms, leading to reduced computational performance. This paper specifically addresses this aspect of memory access bottlenecks in industrial scale simulations. The use of space-filling curves to improve memory access patterns is described and their impact on computational performance is quantified in both shared and distributed memory parallelization paradigms. The Morton space filling curve applied to uniform grids and k-dimensional tree partitions are used to reorder the particle data-structure thus improving spatial and temporal locality in memory. The performance impact of these techniques when applied to two benchmark problems, namely the homogeneous-cooling-system and a fluidized-bed, are presented. We report these optimization techniques lead to approximately two-fold performance improvement in particle focused operations such as neighbor-list creation and data-exchange, with ~ 1.5 times overall improvement in a fluidization simulation with 1.27 million particles.

97 MATHEMATICS AND COMPUTING↗