Search NASA⌕ Search

SEARCH · Search NASA

Results for “performance benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

OECD/NEA MPCMIV Benchmark - Preliminary fuel performance results

The on-going OECD/NEA Multi-physics Pellet Cladding Mechanical Interaction Validation (MPCMIV) benchmark aims to provide guidance on multi-physics validation through the modelling of two cold ramps. In this paper, the first results for the base irradiation of the father rod and fuel rodlet (refabricated from the father rod), and the first cold ramp are presented. The base irradiation consists of 3 years of irradiation in the Forsmark-2 reactor. The cold ramp encompasses a steady-state pre-ramp period of less than one hour at a low constant linear heat rate (LHR) followed by a ramp test (< 1 min) with a much higher maximal LHR. The base irradiation is modelled using the fuel performance codes FRAPCON and FAST, while the cold ramp modeling is using the fuel performance code FRAPTRAN. Several missing parameters for FRAPCON/FAST base irradiation models have been selected using multiple references such as the OECD/NEA light water reactor Uncertainty Analysis in Modelling (UAM) benchmark specifications and the FRAPCON Integral Assessment report. The obtained results for the base irradiation such as the cladding outer diameter show reasonable agreement with the experimental measurements. The results for the cold ramp show larger differences with the experimental measurements (e.g., cladding axial elongation). However, such differences have been observed as well in other studies involving pellet cladding mechanical interaction analyses and are attributed to the fuel performance modelling assumptions and to the tuned modelling parameters that were not covered by the specifications. Uncertainty and sensitivity analysis might allow a better quantification of these missing parameters. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1998

Enhancing Global Competitiveness: Benchmarking Airline Operational Performance in Highly Regulated Environments

Enhancing competitiveness in the global airline industry is at the forefront of attention with airlines, government, and the flying public. The seemingly unchecked growth of major airline alliances is heralded as an enhancement to global competition. However, like many mega-conglomerates, mega-airlines will face complications driven by size regardless of the many recitations of enhanced efficiency. Outlined herein is a conceptual model to serve as a decision tool for policy-makers, managers, and consumers of airline services. This model is developed using public data for the United States (U.S.) major airline industry available from the U/S. Department of Transportation, Federal Aviation Administration, the National Aeronautics and Space Administration, the National Transportation Safety Board, and other public and private sector sources. Data points include number of accidents, pilot deviations, operational performance indicators, flight problems, and other factors. Data from these sources provide opportunity to develop a model based on a complex dot product equation of two vectors. A row vector is weighted for importance by a key informant panel of government, industry, and consumer experts, while a column vector is established with the factor value. The resulting equation, known as the national Airline Quality Rating (AQR), where Q is quality, C is weight, and V is the value of the variables, is stated Q=C[i1-19] x V[i1-19]. Looking at historical patterns of AQR results provides the basis for establishment of an industry benchmark for the purpose of enhancing airline operational performance. A 7 year average of overall operational performance provides the resulting benchmark indicator. Applications from this example can be applied to the many competitive environments of the global industry and assist policy-makers faced with rapidly changing regulatory challenges.

Bowen, Brent D.↗
NASA NTRS1999

The OpenMP Implementation of NAS Parallel Benchmarks and its Performance

As the new ccNUMA architecture became popular in recent years, parallel programming with compiler directives on these machines has evolved to accommodate new needs. In this study, we examine the effectiveness of OpenMP directives for parallelizing the NAS Parallel Benchmarks. Implementation details will be discussed and performance will be compared with the MPI implementation. We have demonstrated that OpenMP can achieve very good results for parallelization on a shared memory system, but effective use of memory and cache is very important.

Jin, Hao-Qiang↗
DOE OSTI2020

Exploring Hilbert space on a budget: Novel benchmark set and performance metric for testing electronic structure methods in the regime of strong correlation

This work explores the ability of classical electronic structure methods to efficiently represent (compress) the information content of full configuration interaction (FCI) wave functions. We introduce a benchmark set of four hydrogen model systems of different dimensionalities and distinctive electronic structures: a 1D chain, a 1D ring, a 2D triangular lattice, and a 3D close-packed pyramid. To assess the ability of a computational method to produce accurate and compact wave functions, we introduce the accuracy volume, a metric that measures the number of variational parameters necessary to achieve a target energy error. Using this metric and the hydrogen models, we examine the performance of three classical deterministic methods: (i) selected configuration interaction (sCI) realized both via an a posteriori (ap-sCI) and variational selection of the most important determinants, (ii) an a posteriori singular value decomposition (SVD) of the FCI tensor (SVD-FCI), and (iii) the matrix product state representation obtained via the density matrix renormalization group (DMRG). We find that the DMRG generally gives the most efficient wave function representation for all systems, particularly in the 1D chain with a localized basis. For the 2D and 3D systems, all methods (except DMRG) perform best with a delocalized basis, and the efficiency of sCI and SVD-FCI is closer to that of DMRG. For larger analogs of the models, the DMRG consistently requires the fewest parameters but still scales exponentially in 2D and 3D systems, and the performance of SVD-FCI is essentially equivalent to that of ap-sCI.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets

BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. Furthermore, we observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Benchmarking Quantum Processor Performance through Quantum Distance Metrics Over An Algorithm Suite

Quantum computing is poised to solve computational paradigms that classical computing could never feasibly reach. Tasks such as prime factorization to Quantum Chemistry are examples of classically difficult problems that have analogous algorithms that are sped up on quantum computers. To attain this computational advantage, we must first traverse the noisy intermediate scale quantum (NISQ) era, in which quantum processors suffer from compounding noise factors that can lead to unreliable algorithm induction producing noisy results. We describe QASMBench, a suite of QASM-level (Quantum assembly language) benchmarks that challenge all realisable angles of quantum processor noise. We evaluate a large portion of these algorithms by performing density matrix tomography on 14 IBMQ Quantum devices.

Stein, Samuel A.↗
DOE OSTI2020

ELSI — An open infrastructure for electronic structure solvers

Routine applications of electronic structure theory to molecules and periodic systems need to compute the electron density from given Hamiltonian and, in case of non-orthogonal basis sets, overlap matrices. System sizes can range from few to thousands or, in some examples, millions of atoms. Different discretization schemes (basis sets) and different system geometries (finite non-periodic vs. infinite periodic boundary conditions) yield matrices with different structures. The ELectronic Structure Infrastructure (ELSI) project provides an open-source software interface to facilitate the implementation and optimal use of high-performance solver libraries covering cubic scaling eigensolvers, linear scaling density-matrix-based algorithms, and other reduced scaling methods in between. In this paper, we present recent improvements and developments inside ELSI, mainly covering (1) new solvers connected to the interface, (2) matrix layout and communication adapted for parallel calculations of periodic and/or spin-polarized systems, (3) routines for density matrix extrapolation in geometry optimization and molecular dynamics calculations, and (4) general utilities such as parallel matrix I/O and JSON output. The ELSI interface has been integrated into four electronic structure code projects (DFTB+, DGDFT, FHI-aims, SIESTA), allowing us to rigorously benchmark the performance of the solvers on an equal footing. Based on results of a systematic set of large-scale benchmarks performed with Kohn–Sham density-functional theory and density-functional tight-binding theory, we identify factors that strongly affect the efficiency of the solvers, and propose a decision layer that assists with the solver selection process. As a result, we describe a reverse communication interface encoding matrix-free iterative solver strategies that are amenable, e.g., for use with planewave basis sets.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1989

Tools for 3D scientific visualization in computational aerodynamics

The purpose is to describe the tools and techniques in use at the NASA Ames Research Center for performing visualization of computational aerodynamics, for example visualization of flow fields from computer simulations of fluid dynamics about vehicles such as the Space Shuttle. The hardware used for visualization is a high-performance graphics workstation connected to a super computer with a high speed channel. At present, the workstation is a Silicon Graphics IRIS 3130, the supercomputer is a CRAY2, and the high speed channel is a hyperchannel. The three techniques used for visualization are post-processing, tracking, and steering. Post-processing analysis is done after the simulation. Tracking analysis is done during a simulation but is not interactive, whereas steering analysis involves modifying the simulation interactively during the simulation. Using post-processing methods, a flow simulation is executed on a supercomputer and, after the simulation is complete, the results of the simulation are processed for viewing. The software in use and under development at NASA Ames Research Center for performing these types of tasks in computational aerodynamics is described. Workstation performance issues, benchmarking, and high-performance networks for this purpose are also discussed as well as descriptions of other hardware for digital video and film recording.

Bancroft, Gordon↗
DOE OSTI2020

Generation IV Benchmarking of TRISO Fuel Performance Models Under Accident Conditions Final Report

The Generation IV International Forum (GIF) is a co-operative international endeavor of fourteen members organized to carry out the research and development needed to establish the feasibility and performance capabilities of the next generation nuclear energy systems. GIF selected six reactor technologies, amongst which is the Very High Temperature Reactor (VHTR) that is primarily dedicated to the cogeneration of electricity and hydrogen. The technical basis for VHTR is the tristructural isotropic (TRISO)-coated particle fuel, the graphite as the core structure, helium coolant, as well as the dedicated core layout and lower power density to removal decay heat in a natural way. At the heart of safety features of the VHTR concept lie the TRISO fuel particles that are designed to keep their structural integrity and retain fission products at temperatures up to 1600°C. As part as the design and future operation of VHTRs, a key aspect is the accurate prediction of fuel performance under irradiation and accident conditions. Modeling and simulation allow prediction of TRISO fuel behavior when subject to neutron flux and in high temperature accident scenarios. The refinement of the fuel performance models and codes is performed by comparison to in-pile and out-of-pile experimental data that reproduce the expected irradiation conditions in high temperature gas-cooled reactors (HTGRs). Historically, the International Atomic Energy Agency (IAEA) developed a benchmark dedicated to the validation of predictive methods for fuel and fission product behavior through the Coordinated Research Program CRP-2 (IAEA, 1997). CRP-2 was later updated to cover fuel fabrication, quality assurance, irradiation performance, safety testing, and spent fuel. The scope of the resulting CRP-6 benchmarks focused on HTGR fuel performance and fission product release (IAEA, 2012). Taking advantage of additional TRISO fuel fabrication, irradiation, and safety testing campaigns, GIF launched a Generation IV Benchmarking of TRISO Fuel Performance Models under Accident Conditions in late 2015. This GIF benchmark is a three-year program steered by Idaho National Laboratory (INL, USA). The other participants include the Japan Atomic Energy Agency (JAEA, Japan) and the Korea Atomic Energy Research Institute (KAERI, Korea). The objectives of the benchmark are to: follow on the IAEA CRP benchmarks, (2) model fission product release under accident conditions, (3) compare results obtained by the fuel performance modeling codes of the benchmark participants, and (4) compare these code predictions to experimental data. Safety tests chosen for modeling include the first and second experiments of the Advanced Gas Reactor program (AGR-1 and AGR-2) and the High Flux Reactor (HFR) EU1bis experiment. This report presents the results obtained by the three research institutions using their respective fuel performance modeling codes. Comparisons of the corresponding fission product release predictions are made with experimental data from AGR-1, AGR-2, and HFR-EU1bis. The benchmark results show good agreements between all participants but also show a general trend of over-prediction of the experimental release data, which is mainly attributed to the use of over-estimated diffusion coefficients.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2020

Comparison of Modeled to Measured Spectra using MCNP and GADRAS to Benchmark and Contrast Modeling Limitations

The desire to improve nuclear material detection through portal monitors and other low resolution detectors has led to interest in benchmarking the performance of radiation transport codes. These codes can be used to generate a variety and quantity of spectra that may be cost prohibitive to measure directly. Particular characteristics of typical detection scenarios were isolated to compare the performance of radiation transport codes to laboratory experiments. These benchmark experiments were performed to validate simulations in a number of key areas. These experiments included high areal density configurations in both one- and three-dimensional configurations. In addition, off-angle measurements were conducted to investigate the impact of detector response assumptions on complex geometries, for example where an unknown source is not physically collocated with an apparent hot spot. Furthermore, by examining both simple shielding and backscatter configurations, isolation of the scatter emanating from the object and environmental background scatter contributions to the spectra could be elucidated. A series of experiments were performed in which the order of shielding layers and the thickness were evaluated to examine the ability of the simulations to address these variables. Experiments with increasing thickness of polyethylene around a neutron source also allows for examinations of the ability of the transport simulations to properly model n,γ reactions. Finally, depleted uranium (DU) measurements have more complicated spectra than the cobalt-60 and consequently allow for investigations into the ability of transport simulations to model self-shielding. In each case, the results of the experiments were compared to MCNP simulations to judge the performance of the code, as was done previously with GADRAS 18.7.9 simulations [7]. All of the benchmark measurements were taken with a liquid nitrogen cooled 140% high purity germanium (HPGe) detector with a bismuth collimator that had the front tin filter removed. The measurement location and detector configuration were subsequently used for all experimental configurations. The remainder of this report details the experimental measurements, Section 2 and the MCNP model, Section 3. Comparisons between the experimental results and MCNP-generated spectra are performed, Section 4. Section 5 compares the results obtained with MCNP with the previously reported results obtained with GADRAS [2]. Finally, Section 6 presents the conclusions.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2021

Benchmarking Quantum Chemistry Computations with Variational, Imaginary Time Evolution, and Krylov Space Solver Algorithms

Quantum chemistry is a key application area for noisy-intermediate scale quantum (NISQ) devices, and therefore serves as an important benchmark for current and future quantum computer performance. Previous benchmarks in this field have focused on variational methods for computing ground and excited states of various molecules, including a benchmarking suite focused on the performance of computing ground states for alkali-hydrides under an array of error mitigation methods. State-of-the-art methods to reach chemical accuracy in hybrid quantum-classical electronic structure calculations of alkali hydride molecules on NISQ devices from IBM are outlined here. Here it is demonstrated how to extend the reach of variational eigensolvers with symmetry preserving Ansätze. Next, it is outlined how to use quantum imaginary time evolution and Lanczos as a complementary method to variational techniques, highlighting the advantages of each approach. Finally, a new error mitigation method is demonstrated which uses systematic error cancellation via hidden inverse gate constructions, improving the performance of typical variational algorithms. These results show that electronic structure calculations have advanced rapidly, to routine chemical accuracy for simple molecules, from their inception on quantum computers a few short years ago, and they point to further rapid progress to larger molecules as the power of NISQ devices grows.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Performance Portability Evaluation of OpenCL Benchmarks across Intel and NVIDIA Platforms

We evaluate the capabilities of vendor-provided OpenCL implementations for performance portability across multiple computing platforms. The Rodinia benchmark suite is used for this evaluation. We apply the metric defined by Pennycook et al., and we use roofline efficiency from the Roofline performance model as the "performance efficiency" in the metric's definition. We found that the delivered performance portability is similar for several benchmarks, even if the roofline-based performance efficiencies across platforms are very different among the benchmarks. To help distinguish between these instances, we extend the metric by adding the standard deviation of the performance efficiencies for each benchmark. We argue that the standard deviation gives additional insight into performance portability assessment since it adds the performance variability across platforms. Additionally, we discuss the challenges to measure performance portability associated with algorithms and system software. In terms of algorithms, we need to carefully construct the benchmarks and appropriately use the concurrency available on a platform. In terms of system software, we depend on the vendor performance tools to support the desired programming model and runtime to be able to measure the metrics of interest.

GPU↗
NASA NTRS1991

Cray performance data from five benchmarks

The five benchmark programs discussed in TM-88956, February 1987, were run on the CRAY X-MP/24 under different operating systems and compilers. Performance data is reported for runs under early versions of UNICOS and CFT77. The most recent data includes a system of configuration for a X-MP hardware upgrade. Performance figures for the Y-MP are shown for comparison. Differences in the figures are analyzed and discussed.

Pennline, James A.↗
NASA NTRS1997

Performance Comparison of HPF and MPI Based NAS Parallel Benchmarks

Compilers supporting High Performance Form (HPF) features first appeared in late 1994 and early 1995 from Applied Parallel Research (APR), Digital Equipment Corporation, and The Portland Group (PGI). IBM introduced an HPF compiler for the IBM RS/6000 SP2 in April of 1996. Over the past two years, these implementations have shown steady improvement in terms of both features and performance. The performance of various hardware/ programming model (HPF and MPI) combinations will be compared, based on latest NAS Parallel Benchmark results, thus providing a cross-machine and cross-model comparison. Specifically, HPF based NPB results will be compared with MPI based NPB results to provide perspective on performance currently obtainable using HPF versus MPI or versus hand-tuned implementations such as those supplied by the hardware vendors. In addition, we would also present NPB, (Version 1.0) performance results for the following systems: DEC Alpha Server 8400 5/440, Fujitsu CAPP Series (VX, VPP300, and VPP700), HP/Convex Exemplar SPP2000, IBM RS/6000 SP P2SC node (120 MHz), NEC SX-4/32, SGI/CRAY T3E, and SGI Origin2000. We would also present sustained performance per dollar for Class B LU, SP and BT benchmarks.

Saini, Subhash↗
DOE OSTI2022

Oak Ridge National Laboratory Pilot Demonstration of an Attestation and Anomaly Detection Framework using Distributed Ledger Technology for Power Grid Infrastructure

This report summarizes the design and pilot demonstration of a framework called Grid Guard that was created to provide increased data and device trustworthiness to electric grid devices by leveraging distributed ledger technology (DLT), specifically blockchain. Grid Guard contains a combination of core cryptographic methods such as the secure hash algorithm (SHA), and asymmetric cryptography, private permissioned blockchain, baselining configuration data, consensus algorithm (Raft) and the Hyperledger Fabric (HLF) framework. The system implements a low energy, fast, and robust enhancement to system trustworthiness within and across electric grid systems such as substations, control centers and metering infrastructures. Blockchain is a distributed database structured that provides a practically unalterable (immutable) timeline of stored transactions. By relying on hashing and the Raft consensus algorithm, if an entity tries to illegitimately alter a record at one instance of the database the other ledger nodes are not altered. They work to cross-reference each other and easily locate any incorrectly added data and remove it. The bulk raw data is stored in an off-chain storage (outside of the blockchain ledger) and a hash of this baseline data is stored in the Blockchain ledger via hashing windows of time-series and configuration data, after aggregation and filtering. The bulk off-chain data repository is then considered to be trust-anchored using the hashes stored in the blockchain. To secure the electric grid testbed devices and data, device configuration baselines were compared to those baselines that had been previously stored in the ledger. Statistical baselines for device configurations, network communication patterns, and high-speed sensor data are calculated and then stored off-chain and hashes stored in the ledger. Measurements such as three-phase voltage and current, frequency, breaker status, protection scheme settings, network configuration settings (and other device configuration artifacts) and network traffic features (packet interarrival times) are compared every minute or other selected time windows. During phase 1 of the Grid Guard DLT project different DLT technologies were studies, and an assessment was performed on DLT technology vulnerabilities, uses, and key characteristics. DLT consensus protocols were studies (e.g., RAFT, named after Reliable, Replicated, Redundant, And Fault-Tolerant). Also, cryptography, public, private and permissioned or permissionless systems were assessed. Grid Guard implements a permissioned private DLT. Consensus algorithm selection and choice of DLT implementation depended heavily on the use-case. For this use-case, parameters were selected to measure performance and existing tools for assessment. Benchmarking was performed theoretically and practically. During phase 2 hashed transactions/blocks were inserted into the ledger every second. During phase 2 of the Grid Guard DLT project, a prototype framework was developed and demonstrated for attestation of critical substation devices and data using precision timing systems that use PTP and IRIG-B protocols) on a testbed of operational devices that emulated a distribution substation, control center, and power metering infrastructure using real Operational Technology (OT). The testbed includes OT devices such as protective relays, human machine interfaces (HMI), and power meters. To determine when to collect and compare system and network baselines, an initial examination of an anomaly detection capability to identify malicious manipulation of data streams was conducted. The resulting anomaly detection was demonstrated in a set of experiments and leveraged to trigger device artifact attestation checks. Attestation checks occur against device configuration baselines when compared with the immutable blockchain-stored baselines, which provided a cryptographically supported means by which to store baselines. The electrical substation-grid testbed was created to test the Grid Guard framework. The testbed emulates the operations of a portion of a power grid and SCADA systems as closely as possible. The testbed integrates real protocols, mainly IEC 61850 standard protocols, such as the Sampled Value (SV) and the GOOSE protocols. The testbed also supports DNP3 and other layer 2 and layer 3 protocols such as Telnet, SSH, SFTP/FTP and other proprietary protocols needed to connect to industrial control system equipment. The testbed emulates real power conditions using the OpalRT hardware-in-the-loop (HIL) device which can create fault situations that cannot be easily tested on real systems. The electrical substation-grid testbed was created using real measurement, communication, and protection devices that electrical utilities commonly use.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Benchmarking and Disclosing Resilience Performance

This fact sheet provides an overview of trends and barriers related to disclosure of climate and resilience performance. It reviews major reporting frameworks for resilience, compares the frameworks, and provides next steps for building owners.

Resilience, Resiliency, Benchmarking, Sustainabili↗
NASA NTRS2002

Performance and Scalability of the NAS Parallel Benchmarks in Java

Several features make Java an attractive choice for scientific applications. In order to gauge the applicability of Java to Computational Fluid Dynamics (CFD), we have implemented the NAS (NASA Advanced Supercomputing) Parallel Benchmarks in Java. The performance and scalability of the benchmarks point out the areas where improvement in Java compiler technology and in Java thread implementation would position Java closer to Fortran in the competition for scientific applications.

Frumkin, Michael A.↗