SEARCH · Search NASA

Results for “performance benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Time series anomaly detection in power electronics signals with recurrent and ConvLSTM autoencoders

The anomalies in the high voltage converter modulator (HVCM) remain a major down time for the spallation neutron source facility, that delivers the most intense neutron beam in the world for scientific materials research. In this work, we propose neural network architectures based on Recurrent AutoEncoders (RAE) to detect anomalies ahead of time in the power signals coming from the HVCM. Bi-directional gated recurrent unit, bi-directional long-short term memory (LSTM), and convolutional LSTM (ConvLSTM) are developed, trained, and tested using real experimental signals from the HVCM module. The results show a good performance of the proposed RAE models, achieving precision up to 91%, recall up to 88%, false omission rate as low as 20% (i.e. 80% of the anomalies were detected), and area under the ROC curve up to 0.9. The three RAE models provide very comparable performance, with LSTM showing slightly better performance than GRU and ConvLSTM. The RAE models are benchmarked against other anomaly detection methods, including isolation forest, support vector machine, local outlier factor, feedforward and convolutional autoencoders, and others; showing a better performance. Here, the results of this study demonstrate the promising potential of RAE in anomaly detection for real-world power systems, and for increasing the reliability of the HVCM modules in the spallation neutron source.

42 ENGINEERING↗

DOE OSTI2026

A large-scale benchmarking of deterministic and stochastic derivative-free optimization algorithms

This presentation summarizes our work in the PrOMMiS project on benchmarking of data-driven optimization algorithms and their applications in self-driving laboratories. This work supports the broader project goal of accelerating the identification of promising separation methods and operating conditions for critical minerals separation processes. We present a systematic benchmarking study of 42 data-driven optimization algorithms on a broad collection of 502 test problems. The results identify BAM, GLCCLUSTER, and MULTIMIN as the most effective optimization solvers, with BAM showing the highest overall performance and solving more than 80% of the benchmark problems. The study also shows that no single solver consistently outperforms the others across all problem types, indicating that our future laboratory applications may benefit from using a small set of strong solvers rather than relying on a single method. The presentation also illustrates an in-silico chemical reactor case study showing that data-driven optimization methods can guide autonomous experimentation in a self-driving laboratory and identify optimal operating conditions within a small number of experiments. Overall, the results provide a basis for selecting efficient optimization methods and demonstrate the practical use of data-driven optimization in self-driving laboratory workflows.

36 MATERIALS SCIENCE↗

DOE OSTI2026

Algorithm-guided experimentation for autonomous AI systems in self-driving laboratories

36 MATERIALS SCIENCE↗

NASA NTRS1997

Least-Squares Spectral Element Solutions to the CAA Workshop Benchmark Problems

This paper presents computed results for some of the CAA benchmark problems via the acoustic solver developed at Rocketdyne CFD Technology Center under the corporate agreement between Boeing North American, Inc. and NASA for the Aerospace Industry Technology Program. The calculations are considered as benchmark testing of the functionality, accuracy, and performance of the solver. Results of these computations demonstrate that the solver is capable of solving the propagation of aeroacoustic signals. Testing of sound generation and on more realistic problems is now pursued for the industrial applications of this solver. Numerical calculations were performed for the second problem of Category 1 of the current workshop problems for an acoustic pulse scattered from a rigid circular cylinder, and for two of the first CAA workshop problems, i. e., the first problem of Category 1 for the propagation of a linear wave and the first problem of Category 4 for an acoustic pulse reflected from a rigid wall in a uniform flow of Mach 0.5. The aim for including the last two problems in this workshop is to test the effectiveness of some boundary conditions set up in the solver. Numerical results of the last two benchmark problems have been compared with their corresponding exact solutions and the comparisons are excellent. This demonstrates the high fidelity of the solver in handling wave propagation problems. This feature lends the method quite attractive in developing a computational acoustic solver for calculating the aero/hydrodynamic noise in a violent flow environment.

Lin, Wen H.↗

DOE OSTI2022

Generation of group constants at GRS for the Rostov-2 benchmark

The OECD/NEA benchmark entitled 'Reactivity compensation of boron dilution by stepwise insertion of control rod cluster into the VVER-1000 core' aims at validating and assessing high fidelity multi-physics simulation code capabilities by comparison with the high-quality measurements performed at the nuclear power plant Rostov unit 2. The benchmark is divided into 2 phases: assembly wise and pin-by-pin resolution of the multi-physics problems. Multi-physics simulation requires the generation of an accurate parametrized few-group cross-section library, and so this benchmark offers an opportunity to validate the methodology for their generation. Based on the specifications of the benchmark, the core simulator KMACS developed at GRS was used to generate a library of cross-sections as well as the burnup distribution required to model the transient scenario of the benchmark at the assembly level. The assessment of the library is performed through comparison with measured values obtained at the initial state of the transient. Good agreement was obtained in terms of critical boron concentration, peaking factors and power profiles. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2026

SoK: What does it Mean to Benchmark Database Forensics?

Relational Database Management Systems are the backbone of modern enterprises and public-sector services, and are thus frequent targets of security incidents, insider threats, and thorough regulatory audits. Consequently, databases have become key sources of digital evidence, requiring investigators to reconstruct past activity from audit logs, transaction logs, and backups. Although benchmarking frameworks such as those developed by the Transaction Processing Performance Council (TPC) are widely used to evaluate database performance, they do not capture forensic requirements such as evidentiary completeness, tamper-evidence, chain of custody, or regulatory compliance under GDPR and CCPA. This survey examines the emerging domain of forensic database benchmarking. We gathered prior research on database forensics, secure logging, and tamper-evident data structures; we analyze modern forensic-ready features in commercial and open-source systems (SQL Server Ledger, Oracle Blockchain Tables, PostgreSQL pgAudit, Db2 Audit, Aurora Database Activity Streams, Oracle Real Application Security and IBM Guardium) and assess why existing benchmarks are insufficient. We propose forensic workloads, metrics, and methodologies that incorporate adversarial stressors, deleted-record recovery, and backup analysis. We also identify open research problems and call for a community-driven forensic benchmark suite. The result is an idea for evaluating not only database performance but also forensic soundness, bridging the gap between system engineering, compliance, and digital investigations.

Lenard, Ben↗

DOE OSTI2022

Frontier (HPE Cray EX) Exascale Supercomputer at the Oak Ridge Leadership Computing Facility

Frontier is the HPE Cray EX exascale supercomputer deployed and operated by the Oak Ridge Leadership Computing Facility (OLCF) at Oak Ridge National Laboratory (ORNL). Frontier is designed for large-scale modeling, simulation, and AI workloads and is built from HPE Cray EX system architecture with AMD CPUs and AMD Instinct GPU accelerators connected by the HPE Slingshot interconnect. System composition (representative production configuration): Frontier is composed of approximately 74 cabinets with 128 compute nodes per cabinet (~9,400 compute nodes total). Each compute node contains one 64-core AMD EPYC CPU and four AMD Instinct MI250X GPUs. Nodes are connected using HPE Slingshot (Slingshot-200 class) networking with multiple NIC ports per node providing high injection bandwidth. Frontier is connected to the Orion parallel file system (multi-tier Lustre) providing a large, center-wide high-performance storage namespace. Operational context: Frontier entered public prominence as the first system to reach No. 1 on the TOP500 list in May 2022 (HPL benchmark), establishing the first widely recognized exascale-era performance milestone. The system supports DOE Office of Science mission workloads and enables leadership-class computational science and AI for open science users.

AMD EPYC↗

DOE OSTI2021

Plutonium Production Reactor Progression Problems: Magnox Neutronics Benchmarks

The goal of this document is to provide a set of comprehensive benchmarking neutronics problems for gas-cooled, graphite-moderated reactors. The problems are designed to be representative problems that increase in complexity to test performance of various codes. For the simulation of the benchmarking problems, the Shift Monte Carlo code and MPACT deterministic code are chosen because of their state-of-the-art capabilities, which are optimized for various reactor designs. Because these codes have been developed primarily for light water reactor applications, their application to graphite-moderated, gas-cooled reactors has not been extensively explored. This report attempts to evaluate benchmarks for a Magnox-style reactor, one that is graphite-moderated and gas-cooled. Parameters that are calculated and presented in this benchmark include the eigenvalue, peaking factors, and isotope concentrations. Results show good agreement between Shift and MPACT, and demonstrate the capability of MPACT to model graphite-moderated systems well.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2022

Dynamic signatures of electronically nonadiabatic coupling in sodium hydride: a rigorous test for the symmetric quasi-classical model applied to realistic, ab initio electronic states in the adiabatic representation

Sodium hydride (NaH) in the gas phase presents a seemingly simple electronic structure making it a potentially tractable system for the detailed investigation of nonadiabatic molecular dynamics from both computational and experimental standpoints. The single vibrational degree of freedom, as well as the strong nonadiabatic coupling that arises from the excited electronic states taking on considerable ionic character, provides a realistic chemical system to test the accuracy of quasi-classical methods to model population dynamics where the results are directly comparable against quantum mechanical benchmarks. Here, using a simulated pump–probe type experiment, this work presents computational predictions of population transfer through the avoided crossings of NaH via symmetric quasi-classical Meyer–Miller (SQC/MM), Ehrenfest, and exact quantum dynamics on realistic, ab initio potential energy surfaces. The main driving force for population transfer arises from the ground vibrational level of the D 1 Σ + adiabatic state that is embedded in the manifold of near-dissociation C 1 Σ + vibrational states. When coupled through a sharply localized first-order derivative coupling most of the population transfers between t = 15 and t = 30 fs depending on the initially excited vibronic wavepacket. While quantum mechanical effects are expected due to the reduced mass of NaH, predictions of the population dynamics from both the SQC/MM and Ehrenfest models perform remarkably well against the quantum dynamics benchmark. Additionally, an analysis of the vibronic structure in the nonadiabatically coupled regime is presented using a variational eigensolver methodology.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Reducing the cost of energy estimation in the variational quantum eigensolver algorithm with robust amplitude estimation

Quantum chemistry and materials is one of the most promising applications of quantum computing. Yet much work is still to be done in matching industry-relevant problems in these areas with quantum algorithms that can solve them. Most previous efforts have carried out resource estimations for quantum algorithms run on large-scale fault-tolerant architectures, which include the quantum phase estimation algorithm. In contrast, few have assessed the performance of near-term quantum algorithms, which include the variational quantum eigensolver (VQE) algorithm. Recently, a large-scale benchmark study [Gonthier et al. 2020] found evidence that the performance of the variational quantum eigensolver for a set of industry-relevant molecules may be too inefficient to be of practical use. This motivates the need for developing and assessing methods that improve the efficiency of VQE. In this work, we predict the runtime of the energy estimation subroutine of VQE when using robust amplitude estimation (RAE) to estimate Pauli expectation values. Under conservative assumptions, our resource estimation predicts that RAE can reduce the runtime over the standard estimation method in VQE by one to two orders of magnitude. Despite this improvement, we find that the runtimes are still too large to be practical. These findings motivate two complementary efforts towards quantum advantage: 1) the investigation of more efficient near-term methods for ground state energy estimation and 2) the development of problem instances that are of industrial value and classically challenging, but better suited to quantum computation.

Johnson, Peter D.↗

NASA NTRSDate not supplied

Least Squares Reverse Time Migration (LSRTM) for Damage Imaging in Composite Laminates

A method for adapting least squares reverse time migration (LSRTM) for ultrasonic guided wave imaging of composite laminates is proposed in this paper. As composites become more widely used in fields such as the aerospace industry, the need for high-resolution imaging in structural health monitoring (SHM) and nondestructive evaluation (NDE) is also growing. For instance, delamination is a common problem in composite laminates, which has led to a certain degree of apprehension in the use of composite materials for load-bearing structures. Although the solver-based imaging techniques using conventional reverse time migration (RTM) methods illuminate damage with a wide range of damage-scattering effects, the resulting images do not fully define the damage regions due to the limited data acquisition aperture, sensor density, frequencies/wavelengths, and incompleteness of adjoint reconstruction. Previously, the LSRTM theory was derived by incorporating a least squares migration (LSM) algorithm into RTM, and benchmarked its high-resolution damage imaging performance for isotropic plates. To improve damage imaging in composite laminates, a process to create an ultrasonic guided wave-based LSRTM method for anisotropic materials is proposed in this paper. The derivation of the forward modeling operator and the adjoint operator is presented. Numerical case studies were conducted to show the improvement of LSRTM over RTM in mapping damage in composite plates. Multiple damage sites or damage with a complex shape were created in the numerical studies based on 1) Born approximation-based modeling or 2) reduction of the plate stiffness-based modeling. LSRTM begins by using a forward modeling process. The forward modeling accuracy for the flexural wave solver used in this study for LSRTM was also compared with the results from a three-dimensional rotated staggered grid (RSG) elastodynamic finite difference solver, previously open-sourced by NASA Langley Research Center. These benchmarking studies have shown that the LSRTM method is more effective in reducing artifacts, improving resolution, and enhancing damage model reconstruction compared to RTM in composite laminates.

Jiaze He↗

DOE OSTI2022

Memory access optimization for particle operations in computational fluid dynamics-discrete element method simulations

Computational Fluid Dynamics - Discrete Element Method is used to model gas-solid systems in several applications in energy, pharmaceutical and petrochemical industries. Computational performance bottlenecks often limit the problem sizes that can be simulated at industrial scale. The data structures used to store several millions of particles in such large-scale simulations have a large memory footprint that does not fit into the processor cache hierarchies on current high-performance-computing platforms, leading to reduced computational performance. This paper specifically addresses this aspect of memory access bottlenecks in industrial scale simulations. The use of space-filling curves to improve memory access patterns is described and their impact on computational performance is quantified in both shared and distributed memory parallelization paradigms. The Morton space filling curve applied to uniform grids and k-dimensional tree partitions are used to reorder the particle data-structure thus improving spatial and temporal locality in memory. The performance impact of these techniques when applied to two benchmark problems, namely the homogeneous-cooling-system and a fluidized-bed, are presented. We report these optimization techniques lead to approximately two-fold performance improvement in particle focused operations such as neighbor-list creation and data-exchange, with ~ 1.5 times overall improvement in a fluidization simulation with 1.27 million particles.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1997

Analysis of 100Mb/s Ethernet for the Whitney Commodity Computing Testbed

We evaluate the performance of a Fast Ethernet network configured with a single large switch, a single hub, and a 4x4 2D torus topology in a testbed cluster of "commodity" Pentium Pro PCs. We also evaluated a mixed network composed of ethernet hubs and switches. An MPI collective communication benchmark, and the NAS Parallel Benchmarks version 2.2 (NPB2) show that the torus network performs best for all sizes that we were able to test (up to 16 nodes). For larger networks the ethernet switch outperforms the hub, though its performance is far less than peak. The hub/switch combination tests indicate that the NAS parallel benchmarks are relatively insensitive to hub densities of less than 7 nodes per hub.

Fineberg, Samuel A.↗

DOE OSTI2020

US Department of Energy, Office of Science High Performance Computing Facility Operational Assessment 2019 Oak Ridge Leadership Computing Facility

Oak Ridge National Laboratory's (ORNL's) Leadership Computing Facility (OLCF) continues to surpass its operational target goals: supporting users; delivering fast, reliable computational ecosystems; creating innovative solutions for high performance computing (HPC) needs; and managing risks, safety, and security associated with operating some of the most powerful computers in the world. The results can be seen in the cutting-edge science conducted by users and the praise from the research community. Calendar year (CY) 2019 was a big year as OLCF staff ran five world-class resources (the leadershipclass computers Titan and Summit, the large analysis cluster called Eos, and the massive parallel filesystems called Atlas and Alpine)) and also began power and cooling upgrades for a 2021 exascale system called Frontier. While continuing exceptional operation of Titan, Eos, and Rhea, the OLCF released the Summit supercomputer for production on January 1, 2019. Summit debuted as the most capable and efficient system in its class and has been recognized as the most powerful system in the world for its performance on both the high performance linpack (HPL) and conjugate gradient (HPCG) benchmark applications since June 2018 according to TOP500. Summit represents the culmination of a multiyear effort between the OLCF, IBM, NVIDIA, and Mellanox to deliver a system that is unmatched for modeling, simulation, data analysis, and learning. To hit the ground running with science-ready applications on day one, application teams worked closely with the OLCF through the Center for Accelerated Application Readiness (CAAR) program for years in advance of the Summit deployment. CY 2019 was filled with outstanding results and accomplishments: a very high rating from users on overall satisfaction for the sixth year in a row; a tremendous amount of core-hours delivered to researchers from two leadership-class systems; and success in delivering on the allocation split of roughly 60%, 30%, and 10% of core-hours offered for the Innovative and Novel Computational Impact on Theory and Experiment (INCITE), Advanced Scientific Computing Research Leadership Computing Challenge (ALCC), and Director's Discretionary (DD) programs, respectively (see Operational Performance section). These accomplishments, coupled with the high utilization rates (overall and capability usage), represent the fulfillment of the promise of both leadership-class machines: efficient facilitation of leadership-class computational applications. Table ES.1 presents a summary of the 2019 OLCF metric targets and the associated results. More information can be found in the Operational Performance section for each OLCF resource. The scientific accomplishments of OLCF users are a strong indication of long-term operational success, with publications this year in such notable journals and publications as Nature, Nature Physics, Nature Plants, Physical Review X, Journal of the American Physical Society, Cell, Nano Letters, and Trends in Biotechnology. Crucial domain-specific discoveries facilitated by resources at the OLCF are described in the High Performance Computing Facility Operational Assessment 2019 Oak Ridge Leadership Computing Facility (OAR) Strategic Results section. For example, researchers used Summit to pinpoint and understand the production of proteins from genetic information, including mutations and the functional expression of disease (Section 8.2).

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Thermal loading analysis of the ring injection dump for the Spallation Neutron Source facility

The ring injection dump (RID) is the largest beam dump in the Spallation Neutron Source facility at Oak Ridge National Laboratory and accepts a fraction of the beam from the ring that is not captured in the ring during injection. Thermo-fluid modeling of the RID components was performed during the initial design by using the software ANSYS CFX to characterize the power rating for the RID. From the calculations, the power rating was lowered from 200 kW to 150 kW, at 1.3 GeV beam energy due to concerns regarding the heating of the concrete structure. The Proton Power Upgrade project scope included reevaluating the power limit to the RID in anticipation that increasing it would provide more operational flexibility. The focus was on the shielding because it was the limiting factor, and data collected from thermocouples installed on the structure were used to benchmark the analysis. A three-stage process was adopted in the latest study. First, the already existing steady-state calculations were validated and extended with the latest Monte Carlo N-Particle source term calculations. Second, transient calculations were conducted to capture the dynamic state of the system in response to the energy dumped to the RID during the following operational period of the beam from 2005–2019 (14 years), and sensitivity analyses of crucial parameters were performed to benchmark the model with reasonable accuracy. Finally, an idealized transient cycle analysis was conducted with realistic duty factors to predict the temperature distribution at higher beam powers. Overall, the results confirmed the accuracy of the original steady state model but allowed the development of a new validated transient model with higher accuracy for future analyses.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

NASA NTRS2001

Fluctuating Pressure Data from 2-D Nozzle Cold Flow Tests (Dual Bell)

Rocket engines nozzle performance changes as a vehicle climbs through the atmosphere. An altitude compensating nozzle, ACN, is intended to improve on a fixed geometry bell nozzle that performs at optimum at only one trajectory point. In addition to nozzle performance, nozzle transient loads are an important consideration. Any nozzle experiences large transient toads when shocks pass through the nozzle at start and shutdown. Additional transient toads will occur at transitional flow conditions. The objectives of cold flow nozzle testing at MSFC are CFD benchmark / calibration and Unsteady flow / sideloads. Initial testing performed with 2-D inserts to 14" transonic wind tunnel. Recent review of 2-D data in preparation for nozzle test facility 3-D testing. This presentation shows fluctuating pressure data and some observations from 2-D dual-bell nozzle cold flow tests.

Nesman, Tomas E.↗

DOE OSTI2023

MUSiC: Critical Experiment with Bare Highly Enriched Uranium Shells Benchmark

The measurements of the Uranium Subcritical and Critical (MUSiC) experiment were performed from December 2020 through April 2021. The purpose of the experiment was to test a variety of multiplicity detection systems for a large range of multiplication values and to see how well they perform as the k eff of the MUSiC series of experiments approached the delayed critical window. Out of the ten configurations that were studied, two of them were super critical and were recently approved as benchmarks and they will be included in the International Criticality Safety Benchmark Evaluation Project (ICBESP) handbook. For each of the two experiments, uncertainties were evaluated six categories: (1) critical measurement, (2) mass, (3) composition, (4) positioning, (5) dimensions, and (6) temperature. The largest contribution to the overall uncertainty in each of the cases was due to the uncertainties in the dimensions of the HEU Rocky Flats shells. Simplified detailed models were created with an average bias of -130.5 pcm due to the removal various components surrounding the experiment. For MCNP6.2 models using ENDF-VIII.0 neutron cross section data, the average difference between the calculated and benchmark k effs was -54 pcm for the detailed model and -102 pcm for the simplified model.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2024

Benchmarking a neutral-atom quantum computer

In this study, we simulated the algorithmic performance of a small neutral atom quantum computer and compared its performance when operating with all-to-all versus nearest-neighbor connectivity. This comparison was made using a suite of algorithmic benchmarks developed by the Quantum Economic Development Consortium. Circuits were simulated with a noise model consistent with experimental data from [Nature 604, 457 (2022)]. We find that all-to-all connectivity improves simulated circuit fidelity by [Formula: see text]–[Formula: see text], compared to nearest-neighbor connectivity.

Computer Science↗