Search NASA⌕ Search

SEARCH · Search NASA

Results for “computer system benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Scientific machine learning benchmarks

Deep learning has transformed the use of machine learning technologies for the analysis of large experimental datasets. In science, such datasets are typically generated by large-scale experimental facilities, and machine learning focuses on the identification of patterns, trends and anomalies to extract meaningful scientific insights from the data. In upcoming experimental facilities, such as the Extreme Photonics Application Centre (EPAC) in the UK or the international Square Kilometre Array (SKA), the rate of data generation and the scale of data volumes will increasingly require the use of more automated data analysis. Furthermore, at present, identifying the most appropriate machine learning algorithm for the analysis of any given scientific dataset is a challenge due to the potential applicability of many different machine learning frameworks, computer architectures and machine learning models. Historically, for modelling and simulation on high-performance computing systems, these issues have been addressed through benchmarking computer applications, algorithms and architectures. Extending such a benchmarking approach and identifying metrics for the application of machine learning methods to open, curated scientific datasets is a new challenge for both scientists and computer scientists. Here, we introduce the concept of machine learning benchmarks for science and review existing approaches. As an example, we describe the SciMLBench suite of scientific machine learning benchmarks.

42 ENGINEERING↗

Tunneling splittings in formic acid dimer: An adiabatic approximation to the Herring formula

Small symmetric molecules and low-dimensional model Hamiltonians are excellent systems for benchmarking theories to compute tunneling splittings. In this work, we investigate a three dimensional model Hamiltonian coupled to a harmonic bath that describes concerted proton transfer in the formic acid dimer. The three modes include the symmetric proton stretch, the symmetric dimer rock, and the dimer stretch. These modes provide a paradigm for the symmetric and anti-symmetric coupled tunneling pathways, these being recognized in the literature as two of the more important classes of coupling. The effects of selective vibrational excitation and coupling to a bath on the tunneling splittings are presented. The splittings for highly excited states are computed using a novel method that makes an adiabatic approximation to the Herring estimate. Results, which are in excellent agreement with the exact splittings, are compared with those obtained using the Makri-Miller approach. This latter method has been shown to provide quality results for tunneling splittings including highly excited vibrational states.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Accelerating the density-functional tight-binding method using graphical processing units

Acceleration of the density-functional tight-binding (DFTB) method on single and multiple graphical processing units (GPUs) was accomplished using the MAGMA linear algebra library. Herein two major computational bottlenecks of DFTB ground-state calculations were addressed in our implementation: the Hamiltonian matrix diagonalization and the density matrix construction. The code was implemented and benchmarked on two different computer systems: (1) the SUMMIT IBM Power9 supercomputer at the Oak Ridge National Laboratory Leadership Computing Facility with 1–6 NVIDIA Volta V100 GPUs per computer node and (2) an in-house Intel Xeon computer with 1–2 NVIDIA Tesla P100 GPUs. The performance and parallel scalability were measured for three molecular models of 1-, 2-, and 3-dimensional chemical systems, represented by carbon nanotubes, covalent organic frameworks, and water clusters.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

An Integrated High-performance Computing and Digital Real-time Simulation Testbed to Benchmark Closed-loop Load Shedding Algorithms in Power Systems

An integrated testbed using digital real-time simulator (DRTS) and a high-performance computing (HPC) cluster is presented here to compare speed and performance of computational schemes to mitigate time-critical issues in electric power systems. The first approach in this testbed validation is taken by running a set of closed-loop load shedding algorithms to compare and contrast two paradigms of arresting cascading failure propagation. Two algorithms involve solving DC and AC power flow model-based optimization problems to compute load shedding at different buses, while a model-based stochastic search using parallel computing provides a viable alternative. The algorithms are implemented in the DRTS-HPC testbed for the IEEE 14-bus benchmark transmission system. As a proof of the concept, simulation results are presented for implementation of closed-loop load-shedding algorithms for cascading failures in the DRTS-HPC testbed

24 POWER TRANSMISSION AND DISTRIBUTION↗

Preliminary Designs for Criticality Safety Benchmarks—Iron/Steel/Chromium Series

The nuclear criticality safety community uses critical benchmarks extensively when analyzing nuclear systems. Having benchmarks sensitive to important materials of interest helps to ensure safety and constrain results. Critical benchmarks are also invaluable to the nuclear data community, which uses these benchmarks as a check and measure of quality of nuclear data libraries. Additionally, integral benchmarks are often used to validate the performance of computational tools for simulating nuclear systems. Many integral benchmarks have been documented in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) handbook, the International Reactor Physics Experiment Evaluation Project (IRPhEP) handbook, and the Shielding Integral Benchmark Archive Database (SINBAD). The National Criticality Experiments Research Center (NCERC), operated by Los Alamos National Laboratory, is the only general purpose critical experiments facility in the United States of America. NCERC regularly designs and executes critical experiments and other measurements useful to a wide variety of fields including nuclear criticality safety, commercial nuclear energy, nonproliferation, and nuclear data. NCERC has recently developed tools to visualize gaps and deficiencies in existing criticality benchmarks – this work is still ongoing. Through this work and other reviews, some high priority deficiencies have been identified. This paper aims to highlight three areas where additional critical benchmarks would be useful to the nuclear criticality safety community and various other groups. Additionally, NCERC is seeking feedback in regard to these proposed measurements to make sure they are as useful to as many communities as possible.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Development of an Improved RELAP5-3D Model for the High Temperature Test Facility

High-temperature gas-cooled reactors (HTGRs) are rapidly approaching deployment. Confidence in transient analysis of these systems for design, optimization, and licensing calculations requires modeling and simulation tools that have been validated against data relevant to HTGR conditions. The High Temperature Test Facility (HTTF) is an integral effects thermal hydraulics test facility for prismatic HTGRs. In spring and summer of 2019, HTTF was used for a series of experiments that now serve as the basis for the OECD/NEA Thermal Hydraulic Code Validation Benchmark for High Temperature Gas-Cooled Reactors using HTTF Data (HTGR T/H Benchmark). This benchmark contains problems for systems code, computational fluid dynamics (CFD), and coupled systems code/CFD modeling representing lower plenum mixing and both the depressurized and pressurized conduction cooldown (DCC and PCC respectively) transients. Benchmark problems include exercises for code-to-code and code-to-data comparisons as well as an exercise for error scaling between HTTF and the Modular High Temperature Gas-Cooled Reactor, which serves as the basis for the HTTF design. Previous analysis as part of the HTGR T/H benchmark used a RELAP5-3D model developed at Idaho National Laboratory (INL) and demonstrated an ability to reproduce trends in the measured data but difficulties reproducing experimental values within their uncertainty. These difficulties were largely attributed to assumptions made during the development of the initial RELAP5-3D model, which predated the HTTF experiments. A significant cause of difficulty reproducing the measured temperatures may be the radial nodalization of the previous RELAP5-3D model. The new model provides a finer nodalization to assess the impact of radial nodalization and allows for asymmetric heating within the core, which was a feature of multiple HTTF experiments. In this paper, we present the new RELAP5-3D model of HTTF. In addition to describing the new model, this paper compares the new and old models and provides results for a full-power steady state, a DCC, and a PCC in HTTF. These analyses are based on the code-to-code comparison exercises for the DCC and PCC problems of the HTGR T/H benchmark. We present the results of these exercises from the new model and compare them to the results of the old model.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Neutron Activation Foil and Thermoluminescent Dosimeter Responses to a Lead Reflected Pulse of the CEA Valduc SILENE Critical Assembly

This benchmark experiment was conducted as a joint venture between the US Department of Energy (DOE) and the French Commissariat à l'Energie Atomique (CEA). Staff at the Oak Ridge National Laboratory (ORNL) in the US and the Centre de Valduc in France planned this experiment. The experiment was conducted on October 13, 2010 in the SILENE critical assembly facility at Valduc. Several other organizations contributed to this experiment and the subsequent evaluation, including CEA Saclay, Lawrence Livermore National Laboratory (LLNL), the Y-12 National Security Complex (NSC), Babcock International Group in the United Kingdom, and Los Alamos National Laboratory (LANL). The goal of this experiment was to measure neutron activation and thermoluminescent dosimeter (TLD) doses from a source similar to a fissile solution critical excursion. The resulting benchmark can be used for validation of computer codes and nuclear data libraries as required when performing analysis of criticality accident alarm systems (CAASs). A secondary goal of this experiment was to qualitatively test performance of two CAAS detectors similar to those currently and formerly in use in some US DOE facilities. The detectors tested were the CIDAS MkX and the Rocky Flats NCD-91. The CIDAS detects gammas with a Geiger-Muller tube, and the Rocky Flats detects neutrons via charged particles produced in a thin 6 LiF disc, depositing energy in a Si solid-state detector. These detectors were being evaluated to determine whether they would alarm, so they were not expected to generate benchmark quality data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Optimal Realization of Yang–Baxter Gate on Quantum Computers

Quantum computers provide a promising method to study the dynamics of many-body systems beyond classical simulation. On the other hand, the analytical methods developed and results obtained from the integrable systems provide deep insights on the many-body system. Quantum simulation of the integrable system not only provides a valid benchmark for quantum computers but is also the first step in studying integrable-breaking systems. The building block for the simulation of an integrable system is the Yang–Baxter gate. It is vital to know how to optimally realize the Yang–Baxter gates on quantum computers. Based on the geometric picture of the Yang–Baxter gates, the optimal realizations of two types of Yang–Baxter gates with a minimal number of controlled NOT (CNOT) or gates are presented. It is also shown how to systematically realize the Yang–Baxter gates via the pulse control. The different realizations on IBM quantum computers are tested and compared. It is found that the pulse realizations of the Yang–Baxter gates always have a higher gate fidelity compared to the optimal CNOT or realizations. On the basis of the above optimal realizations, the simulation of the Yang–Baxter equation on quantum computers is demonstrated. Finally, these results provide a guideline and standard for further experimental studies based on the Yang–Baxter gate.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Embedded Volttron Specification - Benchmarking Small Footprint Compute Devices for Volttron

An embedded system is a small footprint computing unit that typically serves a specific purpose closely associated with measurements and control of hardware devices. These units are designed for reasonable durability and operations in a wide range of operating conditions. Some embedded systems support real-time operations and can demonstrate high levels of reliability. Many have failsafe mechanisms built to handle graceful shutdown of the device in exception conditions. The available memory, processing power, and network connectivity of these devices are limited due to the nature of their specific-purpose design and intended application. Industry practice is to carefully design the software for the available hardware capability to suit desired deployment needs. Volttron is an open source agent development and deployment platform designed to enable researchers to interact with devices and appliances without having to write drivers themselves. Hosting Volttron on small footprint embeddable devices enables its demonstration for embedded use. This report details the steps required and the experience in setting up and running Volttron applications on three small footprint devices: the Intel Next Unit of Computing (NUC), the Raspberry Pi 2, and the BeagleBone Black. In addition, the report also details preliminary investigation of the execution performance of Volttron on these devices.

97 MATHEMATICS AND COMPUTING↗

An Evaluation of Molecular Dynamics Performance on the Hybrid Cray XK6 Supercomputer

For many years, the drive towards computational physics studies that match the size and time-scales of experiment has been fueled by increases in processor and interconnect performance that could be exploited with relatively little modification to existing codes. Engineering and electrical power constraints have disrupted this trend, requiring more drastic changes to both hardware and software solutions. Here, we present details of the Cray XK6 architecture that achieves increased performance with the use of GPU accelerators. We review software development efforts in the LAMMPS molecular dynamics package that have been implemented in order to utilize hybrid high performance computers. We present benchmark results for solid-state, biological, and mesoscopic systems and discuss some challenges for utilizing hybrid systems. We present some early work in improving application performance on the XK6 and performance results for the simulation of liquid copper nanostructures with the embedded atom method.

43 PARTICLE ACCELERATORS↗

Higher-order adaptive finite-element methods for Kohn–Sham density functional theory

We present an efficient computational approach to perform real-space electronic structure calculations using an adaptive higher-order finite-element discretization of Kohn–Sham density-functional theory (DFT). To this end, we develop an a priori mesh-adaption technique to construct a close to optimal finite-element discretization of the problem. We further propose an efficient solution strategy for solving the discrete eigenvalue problem by using spectral finite-elements in conjunction with Gauss–Lobatto quadrature, and a Chebyshev acceleration technique for computing the occupied eigenspace. The proposed approach has been observed to provide a staggering 100–200-fold computational advantage over the solution of a generalized eigenvalue problem. Using the proposed solution procedure, we investigate the computational efficiency afforded by higher-order finite-element discretizations of the Kohn–Sham DFT problem. Our studies suggest that staggering computational savings—of the order of 1000-fold—relative to linear finite-elements can be realized, for both all-electron and local pseudopotential calculations, by using higher-order finite-element discretizations. On all the benchmark systems studied, we observe diminishing returns in computational savings beyond the sixth-order for accuracies commensurate with chemical accuracy, suggesting that the hexic spectral-element may be an optimal choice for the finite-element discretization of the Kohn–Sham DFT problem. A comparative study of the computational efficiency of the proposed higher-order finite-element discretizations suggests that the performance of finite-element basis is competing with the plane-wave discretization for non-periodic local pseudopotential calculations, and compares to the Gaussian basis for all-electron calculations to within an order of magnitude. Further, we demonstrate the capability of the proposed approach to compute the electronic structure of a metallic system containing 1688 atoms using modest computational resources, and good scalability of the present implementation up to 192 processors.

97 MATHEMATICS AND COMPUTING↗

Random circuit block-encoded matrix and a proposal of quantum LINPACK benchmark

The LINPACK benchmark reports the performance of a computer for solving a system of linear equations with dense random matrices. Although this task was not designed with a real application directly in mind, the LINPACK benchmark has been used to define the list of TOP500 supercomputers since the debut of the list in 1993. We propose that a similar benchmark, called the quantum LINPACK benchmark, could be used to measure the whole machine performance of quantum computers. The success of the quantum LINPACK benchmark should be viewed as the minimal requirement for a quantum computer to perform a useful task of solving linear algebra problems, such as linear systems of equations. We propose an input model called the Random Circuit Block-Encoded Matrix (RACBEM), which is a proper generalization of a dense random matrix in the quantum setting. The RACBEM model is efficient to be implemented on a quantum computer and can be designed to optimally adapt to any given quantum architecture, with relying on a black-box quantum compiler. Besides solving linear systems, the RACBEM model can be used to perform a variety of linear algebra tasks relevant to many physical applications, such as computing spectral measures, time series generated by a Hamiltonian simulation, and thermal averages of the energy. We implement these linear algebra operations on IBM Q quantum devices as well as quantum virtual machines, and demonstrate their performance in solving scientific computing problems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Neutron Activation Foil and Thermoluminescent Dosimeter Responses to a Polyethylene Reflected Pulse of the CEA Valduc SILENE Critical Assembly

This benchmark experiment was conducted as a joint venture between the US Department of Energy (DOE) and the French Commissariat à l'Energie Atomique (CEA). Staff at the Oak Ridge National Laboratory (ORNL) in the US and the Centre de Valduc in France planned this experiment. The experiment was conducted on October 19, 2010 in the SILENE critical assembly facility at Valduc. Several other organizations contributed to this experiment and the subsequent evaluation, including CEA Saclay, Lawrence Livermore National Laboratory (LLNL), the Y-12 National Security Complex (NSC), Babcock International Group in the United Kingdom, and Los Alamos National Laboratory (LANL). The goal of this experiment was to measure neutron activation and thermoluminescent dosimeter (TLD) doses from a source similar to a fissile solution critical excursion. The resulting benchmark can be used for validation of computer codes and nuclear data libraries as required when performing analysis of criticality accident alarm systems (CAASs). A secondary goal of this experiment was to qualitatively test performance of two CAAS detectors similar to those currently and formerly in use in some US DOE facilities. The detectors tested were the CIDAS MkX and the Rocky Flats NCD-91. The CIDAS detects gammas with a Geiger-Muller tube and the Rocky Flats detects neutrons via charged particles produced in a thin 6 LiF disc depositing energy in a Si solid state detector. These detectors were being evaluated to determine whether they would alarm, so they were not expected to generate benchmark quality data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Sensitivity Coefficients for Diffusion Coefficients and Other Reactor Physics Parameters using CE TSUNAMI-3D

The TSUNAMI (Tools for Sensitivity and Uncertainty Analysis Methodology Implementation) capabilities within the SCALE code system leverage sensitivity coefficients for many criticality safety applications, including quantifying the data-induced uncertainty in the eigenvalue of critical systems, quantifying computational biases, and selecting applicable benchmark experiments. TSUNAMI sensitivity and uncertainty analysis tools have traditionally been limited to criticality safety application spaces, but the introduction of Generalized Perturbation Theory (GPT) sensitivity methods by Jessee in 2009 and Perfetti in 2013 has extended the scope of SCALE adjoint-weighted sensitivity and uncertainty analysis to include reactor physics applications. The continuous-energy (CE) TSUNAMI-3D GPT sensitivity analysis methods were originally developed to calculate sensitivity coefficients for a limited number of reaction-rate responses using CE Monte Carlo methods. These tools were recently applied in an Oak Ridge National Laboratory (ORNL)-directed research and development project to improve the efficiency of {sup 252}Cf production in the High Flux Isotope Reactor (HFIR) by understanding the sensitivity of key heavy actinide capture-to- fission ratios to potential system design changes. This work further extends the applicability of the CE TSUNAMI- 3D GPT methods to reactor physics applications by enabling the calculation of sensitivity coefficients for an expanded set of reaction rate responses-including a first-ever capability for conducting sensitivity analysis of neutron diffusion coefficients.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Detailed high-accuracy megavoltage transmission measurements: A sensitive experimental benchmark of EGSnrc

Purpose: There are three goals for this study: (a) to perform detailed megavoltage transmission measurements in order to identify the factors that affect the measurement accuracy, (b) to use the measured data as a benchmark for the EGSnrc system in order to identify the computational limiting factors, and (c) to provide data for others to benchmark Monte Carlo codes. Methods: Transmission measurements are performed at the National Research Council Canada on a research linac whose incident electron parameters are independently known. Automated transmission measurements are made on-axis, down to a transmission value of {approx}1.7%, for eight beams between 10 MV (the lowest stable MV beam on the linac) and 30 MV, using fully stopping Be, Al, and Pb bremsstrahlung targets and no fattening filters. To diversify energy differentiation, data are acquired for each beam using low-Z and high-Z attenuators (C and Pb) and Farmer chambers with low-Z and high-Z buildup caps. Experimental corrections are applied for beam drifts (2%), polarity (2.5% typical maximum, 6% extreme), ion recombination (0.2%), leakage (0.3%), and room scatter (0.8%)-the values in parentheses are the largest corrections applied. The experimental setup and the detectors are modeled using EGSnrc, with the newly added photonuclear attenuation included (up to a 5.6% effect). A detailed sensitivity analysis is carried out for the measured and calculated transmission data. Results: The developed experimental protocol allows for transmission measurements with 0.4% uncertainty on the smallest signals. Suggestions for accurate transmission measurements are provided. Measurements and EGSnrc calculations agree typically within 0.2% for the sensitivity of the transmission values to the detector details, to the bremsstrahlung target material, and to the incident electron energy. Direct comparison of the measured and calculated transmission data shows agreement better than 2% for C (3.4% for the 10 MV beam) and typically better than 1% for Pb. The differences can be explained by acceptable photon cross section changes of Less-Than-Or-Slanted-Equal-To 0.4%. Conclusions: Accurate transmission measurements require accounting for a number of influence quantities which, if ignored, can collectively introduce errors larger than 10%. Accurate transmission calculations require the use of the most accurate data and physics options available in EGSnrc, particularly the more accurate bremsstrahlung angular sampling option and the newly added modeling of photonuclear attenuation. Comparison between measurements and calculations implies that EGSnrc is accurate within 0.2% for relative ion chamber response calculations. Photon cross section uncertainties are the ultimate limiting factor for the accuracy of the calculated transmission data (Monte Carlo or analytical).

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A Performance Study of Quantum ESPRESSO’s PWscf Code on Multi-core and GPU Systems

We describe the porting of PWscf (Plane-Wave Self Consistent Field), a key component of the Quantum ESPRESSO open-source suite of codes for materials modeling, to GPU systems using CUDA Fortran. Kernel loop directives (CUF kernels) have been extensively used in order to have a single source code for both CPU and GPU implementations. The results of the GPU version have been carefully validated and the performance of the code on several GPU systems (both x86 and POWER8 based) has been compared with traditional Intel multi-core (CPU only) systems. This current GPU version can reduce the time-to-solution by an average factor of 2–3 running two different input cases widely used as benchmarks on small and large high performance computing systems.

CUDA Fortran↗

Immortal rays: Rethinking random ray neutron transport on GPU architectures

The Random Ray Method (TRRM) is a recently developed adaptation of the Method of Characteristics for neutral particle transport simulations. TRRM has demonstrated excellent performance on 3D nuclear reactor benchmark problems using CPU-based compute systems. When porting to GPU-based systems, however, new performance challenges arise that are unique to processors targeting massive fine-grained parallelism. For smaller problems, or for large problems that are domain decomposed across many computational nodes, the problem size per node has insufficient parallelism to saturate GPU node resources, thus greatly limiting speedup. In this study, we report on a newly developed “immortal ray” variant of TRRM. Here, the immortal ray technique exposes significantly more fine-grained parallelism by fundamentally reformulating the numerical details of ray discretization, resulting in performance tradeoffs with significant overall benefit on GPUs. For very small 2D simulation problems we found the new immortal ray variant allowed for up to a 4.4x speedup when run on a single GPU. For larger 3D simulation problems we found the new variant improved strong scaling by 3x when run on the Summit supercomputer.

97 MATHEMATICS AND COMPUTING↗