Search NASA⌕ Search

SEARCH · Search NASA

Results for “Performance Benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,063 records · Page 59
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

A Sparse Tensor Benchmark Suite for CPUs and GPUs

Tensor computations present significant performance chal- lenges that impact a wide spectrum of applications ranging from machine learning, healthcare analytics, social network analysis, data mining to quantum chemistry and signal processing. Efforts to improve the perfor- mance of tensor computations include exploring data layout, execution scheduling, and parallelism in common tensor kernels. This work presents a benchmark suite for arbitrary-order sparse tensor kernels using state- of-the-art tensor formats: coordinate (COO) and hierarchical coordinate (HiCOO) on CPUs and GPUs. It presents a set of reference tensor kernel implementations that are compatible with real-world tensors and power law tensors extended from synthetic graph generation techniques. We also propose Roofline performance models for these kernels to provide insights of computer platforms from sparse tensor view. This benchmark suite along with the synthetic tensor generator is publicly available.

Li, Jiajia↗
DOE OSTI2014

Evaluation of Low Power ARM Processors for Monte Carlo Particle Transport: MCNP6 on the ARM Cortex-A8 - Paper 83

In the push towards exascale computing, the limiting factors are power requirements and heat dissipation. Using current technology, the power required to drive an exascale supercomputer would near 400-MW. Clearly, future computing hardware will have to be more energy efficient in order to achieve exascale performance. Most of the current research in advanced computing architectures is in the use of standard CPUs with accelerators. However, these accelerators are not generally supported by the traditional scientific programming languages (C, C++, and Fortran) and the scientific software must be tailored specifically to take advantage of the accelerators. Large scale scientific software (hundreds-of-thousand to millions of lines of code) can not be easily ported to the computing platform du jour. The widespread use of low power ARM based processors in the mobile computing market offer a low power alternative to accelerators that are also supported by the traditional scientific programming languages and programming paradigms such as MPI and OpenMP. Computing architectures for scientific computing are often assessed using standard benchmarks such as LINPACK benchmark or using suites of mini-applications that are greatly simplified stand-ins for scientific software. Often the real applications that will deployed on the scientific supercomputers are not ported or tested until delivery of the machines. In order to evaluate the use of ARM processors for scientific computing the Monte Carlo particle transport code, MCNP version 6 was compiled on the ARM Cortex-A8 and a set of benchmark problems were used to assess performance in comparison to an Intel Xeon E5-2670 and the low power Intel Atom N270. MCNP is a continuous energy Monte Carlo particle transport code in development at Los Alamos National Laboratory since 1977. MCNP is used to transport neutrons, photon, electrons, and charged particles using continuous energy interaction data. It is commonly used in fields of nuclear power, nuclear medicine, radiation protection, and high-energy physics through-out the world. MCNP version 6 is approximately 450 thousand lines of code, written primary in Fortran90, with some C and C++. MCNP supports distributed multiprocessing using MPI and threading with OpenMP. The ARM Cortex-A8 executes MCNP6 using about twice as much power as the Intel Xeon E5-2670 and about the same amount of power as the Intel Atom N270. However, these results are encouraging for the use of low-power ARM chips in future supercomputers. The ARM Cortex-A8 is a 32-bit CPU and it's NEON general-purpose SIMD (single-instruction multiple data) engine does not support double-precision floating-point arithmetic. More recent ARM processors (ARM Cortex-A53 and Cortex-A57) support 64-bit addressing and have NEON co-processors that support double-precision floating-point arithmetic. ARM claims that the A50 series delivers a performance increase of 15 times the Cortex-A8 performance for similar power usage. (authors)

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2019

Simulation of Endovascular Aortic Repair Using 3D Printed Abdominal Aortic Aneurysm Model and Fluid Pump

Background: Abdominal aortic aneurysm (AAA) models can be manufactured with 3D printing technology. This study describes detailed methodology and validation of endovascular aortic repair (EVAR) simulation using 3D printed AAA model connected to hemodynamic pump. Method: The AAA model was printed with Objet500 Connex3 (Stratasys, Eden Prairie, MN) and connected to BDC PD-0500 fluid pump (BDC Laboratories, Wheat Ridge, CO). EVAR procedure metrics were benchmarked in two expert implanters and compared to 20 vascular surgical trainees with different levels of EVAR experience (< 20 or ≥ 20 cases). All simulations were performed using commercially available stent grafts, guidewires, catheters, fluoroscopic guidance and digital subtraction angiography. Studied outcomes included ability to complete the procedure independently, time to deploy aortic component, ability to cannulate contralateral gate and complete the repair, and total fluoroscopy and procedure times. Results: A total of 22 EVAR simulation procedures were performed with mean procedure time of 37 ± 12 min. Experienced trainees had significantly lower total procedural time (32 ± 9 vs. 44 ± 6 min, P = 0.003) and fluoroscopic time (13 ± 5 vs. 23 ± 8 min, P = 0.005). All experienced trainees completed the procedure independently in < 45 min, compared to six (46%) of those with less EVAR experience (P = 0.016). Among less experienced trainees, only two (15%) completed the entire procedure independently (P < 0.001). Benchmark implanters performed significantly better than both trainee groups in nearly all EVAR metrics. Conclusion: EVAR simulation was feasible and simulated all procedural steps with high fidelity. This model may be applicable for assessment of technical competencies and standard endovascular skill acquisition within vascular surgery training curricula.

3D PRINTING↗
DOE OSTI2016

Benchmark Generation and Simulation at Extreme Scale

The path to extreme scale high-performance computing (HPC) poses several challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Investigating the performance of parallel applications at scale on future architectures and the performance impact of different architectural choices is an important component of HPC hardware/software co-design. Simulations using models of future HPC systems and communication traces from applications running on existing HPC systems can offer an insight into the performance of future architectures. This work targets technology developed for scalable application tracing of communication events. It focuses on extreme-scale simulation of HPC applications and their communication behavior via lightweight parallel discrete event simulation for performance estimation and evaluation. Instead of simply replaying a trace within a simulator, this work promotes the generation of a benchmark from traces. This benchmark is subsequently exposed to simulation using models to reflect the performance characteristics of future-generation HPC systems. This technique provides a number of benefits, such as eliminating the data intensive trace replay and enabling simulations at different scales. The presented work features novel software co-design aspects, combining the ScalaTrace tool to generate scalable trace files, the ScalaBenchGen tool to generate the benchmark, and the xSim tool to assess the benchmark characteristics within a simulator.

Lagadapati, Mahesh↗
DOE OSTI2022

Building Envelope Campaign – Program Design and Stakeholder Engagement

Building envelope technologies impact approximately 30% of the primary energy consumed by residential and commercial buildings. The Building Envelope Campaign (BEC), which is part of the Department of Energy’s Better Buildings Program, is a market transformation effort to help building owners and managers invest in high performance building envelope technologies for both new and existing commercial buildings. The success of the Campaign has depended on constructing a compelling program design plus organizing a technical team with ability to effectively recruit Participants and Supporters from across industry and keep them engaged.The design of the campaign included developing a strategy to leverage other technology campaigns within the Better Buildings program, identifying stakeholders (including diverse member groups that may have been underserved by previous technology campaigns), recruiting Supporters and Participants, and providing technical assistance in the form of a campaign-specific Building Envelope Performance tool and metric to help benchmark various building envelope options.Engagement had to overcome three main challenges – securing the Campaign Supporters/Participants, helping participants to use the envelope tool to evaluate project options, and getting those participants to submit successful envelope projects for evaluation and recognition by the program. The concepts are simple but program design/implementation and, in particular, sustaining stakeholder engagement can be challenging. This paper will highlight the approaches taken in: program design, engaging industry members, identifying and reaching underserved stakeholders, demonstrating benefits of high performance building envelope technologies and making the case for engaging in this campaign.

McLeod, Hayley↗
DOE OSTI2022

Reactor physics benchmark of Westinghouse PWR core design suite for high burnup/high enrichment fuel - Part 2: 2D and 3D core simulations

Westinghouse has performed a comprehensive set of code-to-code benchmark comparisons to corroborate application of its state-of-the-art NEXUS-based code suite to high burnup and high enrichment (HB/HE) fueled PWR core design and reload analysis. The benchmark consisted of a progression in complexity from modeling single 2D pin cells, through 2D lattices and finally to whole core 2D and then 3D scenarios. The accompanying paper (Part 1) presented the results for a variety of pin cell and lattices representative of high energy cores and established the ability of the MPACT code to accurately match the predictions of the Monte-Carlo code SERPENT2 such that the former may be used as a surrogate reference code in spatially complex whole core modeling applications where using Monte-Carlo methods is somewhat impractical. This paper focuses on comparisons at the whole core level using a representative high power density 3-loop Westinghouse NSSS core operating for 24 month cycles employing high enriched fuel with use of combined burnable absorbers to hold down excess reactivity. The evolution of the core reactivity and power distribution characteristics throughout the cycle are calculated using the Westinghouse NEXUS-based code suite and compared to the results using MPACT. The results of this whole core benchmark confirm the conclusions that the Westinghouse code suite accurately models HB/HE fuel designs, with very good prediction of core reactivity and power distribution throughout the cycle. The industry can be confident that the NEXUS code system can be deployed to support PWR reload safety and operational analyses for cores featuring HB/HE fuel and 24-month cycle operation. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2016

Validation of Criticality Analysis Capability of SuperMC with ICSBEP

SuperMC is a general purpose, intelligent and multifunctional, continuous energy Monte Carlo program for the design and safety evaluation of nuclear systems. It is designed to perform the comprehensive neutronics calculation, taking the radiation transport as the core and including the depletion, radiation source term/dose/biohazard, material activation and transmutation, etc. The latest version of SuperMC can perform the simulation of neutron, photon and coupled neutron and photon transport, and also equipped with the functions of automatic modeling and visualization. SuperMC has been verified by more than 2000 benchmark models and experiments including the handbook of International Criticality Safety Benchmark Evaluation Project (ICSBEP), the Shielding Integral Benchmark Archive Database (SINBAD), etc. The fusion reactor (ITER benchmark model, FDS Fusion Reactors), fast reactors (BN600, CLEAR-I), PWR reactors (HM, BEAVRS) and cases from the International Reactor Physics handbook Evaluation Program (IRPhEP) were employed for validating the comprehensive capability for reactor applications. In this paper, the validation of SuperMC based on the handbook of ICSBEP will be presented. ICSBEP, released by OECD/NEA, contains criticality safety benchmark specifications derived from experiments that were performed at various nuclear critical facilities around the world, which was widely used to verify neutronic codes and associated nuclear data libraries for criticality safety analysis. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2016

Assessment of the MPACT Resonance Data Generation Procedure

Currently, heterogeneous models are being used to generate resonance self-shielded cross-section tables as a function of background cross sections for important nuclides such as 235 U and 238 U by performing the CENTRM (Continuous Energy Transport Model) slowing down calculation with the MOC (Method of Characteristics) spatial discretization and ESSM (Embedded Self-Shielding Method) calculations to obtain background cross sections. And then the resonance self-shielded cross section tables are converted into subgroup data which are to be used in estimating problem-dependent self-shielded cross sections in MPACT (Michigan Parallel Characteristics Transport Code). Although this procedure has been developed and thus resonance data have been generated and validated by benchmark calculations, assessment has never been performed to review if the resonance data are properly generated by the procedure and utilized in MPACT. This study focuses on assessing the procedure and a proper use in MPACT.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

Verification and Performance Impact of the New Parallel MCNP6.3 Particle Track Output Capability for Subcritical Multiplication Simulations

The MCNP6® code, version 6.3, has several new features that are intended to ultimately replace legacy features that are now marked for deprecation. One of these features is the new particle track output (PTRAC) format and capability, where the legacy PTRAC capability still exists alongside the modern PTRAC capability in MCNP6.3. While the MCNP6.3 code has been extensively verified and validated for many applications, the PTRAC feature is not exercised in any of the typical verification and validation (V&V) applications studied during the course of a typical MCNP code release. The primary goal of this paper is to verify that the legacy and modern PTRAC feature produces equivalent results for subcritical multiplication benchmarks previously studied. In the process of verifying that the simulated benchmark results are equivalent, the computational performance is compared between the legacy and modern PTRAC uses. In addition to verification of the update, which is important to the community as a whole, this effort also supports advances in the simulation of recent subcritical neutron noise measurements that require higher computational effort per second of real-time measurement than that of systems typically measured.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Parallel exponential time differencing methods for geophysical flow simulations

Two ocean models are considered for geophysical flow simulations: the multilayer shallow water equations and the multilayer primitive equations. For the former, we investigate the parallel performance of exponential time differencing (ETD) methods, including exponential Rosenbrock–Euler, ETD2wave, and B-ETD2wave. For the latter, we take advantage of the splitting of barotropic and baroclinic modes and propose a new two-level method in which an ETD method is applied to solve the fast barotropic mode. Furthermore, these methods could improve the computational efficiency of numerical simulations because ETD methods allow for much larger time step sizes than traditional explicit time-stepping techniques that are commonly used in existing computational ocean models. Several standard benchmark tests for ocean modeling are performed and comparison of the numerical results demonstrates a great potential of applying the parallel ETD methods for simulating real-world geophysical flows.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Logic in Memory Emulator

Logic in Memory Emulator (LiME) is a hardware/software tool specially designed for memory system evaluation and experiment. Emerging memories display a wide range of bandwidths, latencies, and capacities, making it challenging for the computer architect to navigate the design space of potential memory configurations, and for the application developer to assess performance implications of using such memories. With the LiME framework, architectural ideas can be prototyped in great detail yet with sufficient performance to support realistic evaluation on long running applications. LiME consists of two fundamental components: 1) the hardware and OS infrastructure for the emulator, and 2) a suite of benchmark applications to assist in characterizing the performance of current and future computer architectures. Some of the applications have been collected from other open source projects. Uses: Logging, replay and analysis of an application's memory behavior Evaluate impact of emerging memory technology on application performance. Emulate complex memory interactions in whole applications orders of magnitude faster than software simulation. Emulate acceleration hardware co-located with the memory subsystem. Features: Capture and log external memory accesses to a separate off-chip memory device without affecting application execution. Memory traces include the address, length, timestamp, and optionally the data for each transaction. Captured trace data can be saved to an SD card for off-line analysis. Configure a wide range of memory latencies in sub-nanosecond increments that encompass highbandwidth and storage class memories. Specify regions of interest (ROI) in applications to reduce the amount of trace data captured for analysis. Currently supports execution on Xilinx Zynq SoC which integrates an ARM processor with FPGA logic on a single device. Applications can be run under Linux or in bare metal mode on the ARM cores.

Jain, AbhishekK↗
DOE OSTI2021

Preliminary NoMAD Results of the MUSIC Experiment

The Measurement of Uranium Subcritical and Critical (MUSiC) experiment was carried out from December 2020 through April 2021 at the National Criticality Experiments Research Center (NCERC). This measurement campaign featured bare configurations of the Rocky Flats highly-enriched uranium (HEU) shells, with each configuration having different numbers of these shells. The goal of the experiment was to test multiple neutron multiplicity detectors and measurement methods for a large range of neutron multiplication values, to see when the combination of detectors and methods break down as the configurations reach the delayed supercritical window. Adding subcritical integral benchmarks gives additional validation to nuclear data. These benchmarks provide additional parameters against which to validate the data. While critical benchmarks have just a single value, $K_{eff}$ , subcritical benchmarks can be used to infer multiple parameters. As an example, recent subcritical benchmarks utilizing the Hage-Cifarelli formalism have three quantities of interest ($R_{1}$, $R_{2}$, and $M_{L}$). This gives nuclear data evaluators additional data to use when performing their evaluations, and allows for these benchmarks to be useful for additional types of nuclear data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2019

Comparing High Performance Computing Accelerator Programming Models

Accelerator devices are becoming a norm in High Performance Computing (HPC). With more systems opting for heterogeneous architectures, portable programming models like OpenMP and OpenACC are becoming increasingly important. The SPEC ACCEL 1.2 benchmark suite consists of comparable benchmarks in OpenCL, OpenMP 4.5, and OpenACC 2.5 that can be used to evaluate the performance and support for programming models and frameworks on heterogeneous platforms. In this paper we go beneath the normative metric of performance times and look at the individual kernels to study the usage, strengths, and weaknesses of the two prevalent portable heterogeneous programming models, OpenMP and OpenACC. From our analysis we identify that benchmarks like MRI-Q, SP and BT have better performance using OpenACC, while benchmarks like MiniGhost, LBM and LBDC do consistently better with the OpenMP programming model across supercomputers like Titan, and Summit.We deep dive into the kernels of select four benchmarks to answer questions like: Where does the benchmark spend most of its cycles? What is the parallelization strategy used? Why is one programming model more performant than the other? By identifying the similarities and dierences we want to contrast between the benchmark implementation strategies in the SPEC ACCEL 1.2 benchmarks and provide more insights into the OpenMP and OpenACC programming models.

Pophale, Swaroop S.↗
DOE OSTI2021

Progress on Optimizing Wind Farms and Rotor Designs Using Adjoints

Modern wind plants are increasingly tasked with multiple performance objectives. In addition to designing plants that maximize power output and minimize the levelized cost of energy (LCOE), the design and operation of wind plants is increasingly influenced by challenges regarding grid integration of variable generation renewables. This places a growing emphasis on making wind plants more controllable and predictable. WindSE is a Reynolds-averaged Navier-Stokes (RANS) model designed around analytical gradient and adjoint methods, with the ability to capture terrain-induced effects, as shown in Figure 1. The recent addition of an unsteady solver with an actuator line method (ALM) and ongoing work to enable massively parallel optimizations gives it a unique niche to explore coupled plant-level controls and design problems. This code is an open source python package built on the FEniCS framework that utilizes fast, parallel PETSc solvers to model fluid flow throughout wind-farm scale domains. Two recent studies performed using WindSE demonstrate the capability to optimize under a wide variety of flow conditions and objective functions. In the first, we present an optimization focused on modifying the layout of a wind farm with a fixed number of turbines for maximum total power output [1]. This study highlights the ability to quickly perform simulations using the steady Navier-Stokes solver combined with rotors represented as actuator disks while also stressing the importance of capturing terrain-induced effects. Gradient-based optimization using the RANS equations is viable due to the inclusion of efficiently computed adjoint derivatives. We interpret the physical results of the optimal layout and also discuss the computational cost of scaling to larger problems. In the second study, we present the capabilities of the unsteady Navier-Stokes solver, where rotor-blade profiles represented by actuator lines are optimized to enhance wake steering effects and overall power production [2]. We quantify the wind plant performance gains obtained from this type of simultaneous control co-design optimization as compared to optimizing the blade design and yaw independently. Figure 2 shows the differences between a baseline two-turbine system and an optimized system where we fine-tune the blade chord profile. Results and challenges from each study are quickly summarized and used to motivate the current development efforts within WindSE. Current and future work is focused on enabling higher-resolution studies with more degrees of freedom through parallelization of both the simulation and optimization algorithms. We present benchmarking results to show that WindSE performs well in both weak- and strong-scaling tests and further demonstrate that the optimizer obtains the same convergence rates in both shared- and distributed-memory environments. Using larger wind farms, we can study deep-array effects within an optimization context, allowing the use of objective functions that have been previously unstudied. As an example, we present ongoing work on a blockage metric which characterizes the loss of available kinetic energy due to wake effects from multiple upstream turbines.

adjoint optimization↗
DOE OSTI2017

ENDF/B-VIII.0 testing with AMPX and SCALE

As the new ENDF/B-VIII.0 data has continued development from its initial beta release into the final release version, it has subsequently undergone processing and testing at Oak Ridge National Laboratory with AMPX and SCALE. Each of the beta releases has been processed with AMPX to create both multigroup (MG) and continuous-energy (CE) libraries. The MG libraries were all processed using the SCALE 252-group structure. These libraries were then used with the VALID suite for benchmarking the new data library's performance against the currently distributed ENDF/B-VII.1 (VII.1)-based SCALE libraries. Here, we compare the MG and CE VALID results for the two current ENDF releases, versions VII.1 and the most recent beta release of ENDF/B-VIII.0 as of this writing (VIII.0.β4). Generally, both the MG and CE VALID aggregate results for VIII.0.β4 compare reasonably well with their VII.1-derived counterparts. As can be expected of any new ENDF release, some of the individual cases have substantially different computed neutron multiplicity (k{sub eff}) values between the two releases, although typically the differences are less than 100 percent-milli (pcm). However, changes to the plutonium cross sections have induced much larger changes in the ratio of the computed k{sub eff} to the experimental value (C/E) for thermal spectrum plutonium solution critical systems. This paper details the comparative performance of the VII.1 and VIII.0.β4 data for the broad range of test cases covered by the VALID suite. A companion paper at this conference provides details of the baseline validation of SCALE 6.2, including the first release of the VII.1 data. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2016

Modeling PWR systems with a unique framework for hybrid energy systems

Hybrid energy system (HES) is used to utilize the wind, solar, nuclear and other energy sources. Within the study and design of a nuclear power plant and HES, extensive system modeling is necessary to determine how the reactor is going to perform in any given situation, not only in the normal performance of the reactor but also transients including unanticipated transients without scram and hypothetical accidents. A modelling tool that is capable of solving governing equations of mass, momentum and energy for two-phase application is necessary to model the coolant systems of HES. Therefore, this study shows the development of a new coupling tool that is capable of modelling several components of HES and experimental validation of the model. For the initial development phase, the primary and secondary coolant systems (which use several components similar to the components of HES) are modelled by using this new tool. Code-to-code benchmark is also used for the performance evaluation of the new tool. It is observed that there is a good agreement between the new tool and RELAP5 predictions. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2025

BuildingQA: A Benchmark for Natural Language Question Answering over Building Knowledge Graphs

Graph-based representations of building metadata using ontologies like Brick are vital for smart building applications, but querying them remains a challenge for practitioners. Knowledge Graph Question Answering (KGQA) systems, meant to retrieve answers from natural language questions, traditionally require large-scale training data, making them ill-suited for the specialized and data-scarce building domain. The advent of Large Language Models (LLMs) offers a paradigm shift, enabling zero-shot natural language querying without building/domain-specific training. Yet, there is no standardized benchmark for building-specific KGQA which can guide and validate research in this area. To address this gap, our work makes three primary contributions. First, we introduce the BuildingQA Benchmark Dataset, constructed through a multi-stage process of collecting practitioner data, augmenting it with LLMs for linguistic diversity, and curating a final set of 188 questions across 4 buildings. Second, we characterize the benchmark's complexity and ambiguity, introducing a novel method to quantify its "lexical gap" and providing a four-stage diagnostic framework for analyzing how systems fail. Third, we benchmark zero-shot LLM-powered KGQA systems to establish baseline performance and analyze their failure modes. Our evaluation reveals that top-performing systems achieve a maximum F1 score of only 0.38. This result does not indicate a failure of these powerful systems, but rather underscores the unique challenges posed by our benchmark. It demonstrates a critical performance gap, showing that current methods successful on general KGs struggle with the specific lexical and structural nuances of the building domain. BuildingQA1 thus provides the benchmark dataset and foundational analysis needed to drive the development of novel, domain-aware methods required to unlock the use of semantic data in buildings.

Mulayim, Ozan Baris↗