Search NASA⌕ Search

SEARCH · Search NASA

Results for “Performance Benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,351 records · Page 75

How efficiently can AI recognize Wireless Devices?

This poster presents a hardware benchmarking methodology for a 3-layer CNN waveform classifier deployed using ONNX Runtime on an NVIDIA Jetson AGX Orin. The dataset consist of 9 signal types, -30 to +30 dB SNR with 5dB increments. Benchmarking on the Jetson AGX Orin gave an accuracy of 91.9% and GPU throughput of 107,120 predictions/sec (23× faster than CPU). The Jetson GPU reached approximately 27M samples/sec with stable performance but fell below the 40 MHz rate needed for real-time radio feeds. Sustained testing of 5 minutes confirmed stable performance with no memory leaks, establishing a reproducible benchmarking baseline for future edge-deployment optimization.

99 - GENERAL AND MISCELLANEOUS↗

Bench-Scale Development of a Hot Carbonate Absorption Process with Crystallization-Enabled High-Pressure Stripping for Post-Combustion CO 2 Capture (Final Scientific/Technical Report)

A novel Hot Carbonate Absorption Process with Crystallization-Enabled High-Pressure Stripping (Hot-CAP) has been developed by the University of Illinois at Urbana-Champaign and Carbon Capture Scientific, LLC in this three-year, bench-scale project. The Hot-CAP features a concentrated carbonate solution (e.g., K 2 CO 3 ) for CO 2 absorption and a bicarbonate slurry (e.g., KHCO 3 ) for high-pressure CO 2 stripping to overcome the energy use and other disadvantages associated with the benchmark monoethanolamine (MEA) process. The project was aimed at performing laboratory- and bench-scale experiments to prove its technical feasibility and generate process engineering and scale-up data, and conducting a techno-economic analysis (TEA) to demonstrate its energy use and cost competitiveness over MEA. To meet project goals and objectives, a combination of experimental, modeling, process simulation, and economic analysis studies were applied. Carefully designed and intensive experiments were conducted to measure thermodynamic and reaction engineering data relevant to four major unit operations in the Hot-CAP (i.e., CO 2 absorption, CO 2 stripping, bicarbonate crystallization, and sulfate reclamation). The rate promoters that could accelerate the CO 2 absorption rate into the potassium carbonate/bicarbonate (PCB) solution to a level greater than that into the 5 M MEA solution were identified, and the superior performance of CO 2 absorption into PCB was demonstrated in a bench-scale packed-bed column. Kinetic data on bicarbonate crystallization were developed and applied for crystallizer design and sizing. Parametric testing of high-pressure CO 2 stripping with concentrated bicarbonate-dominant slurries at high temperatures (≥140°C) in a bench-scale stripping column demonstrated lower heat use than with MEA. The feasibility of a modified process for combining SO 2 removal with CO 2 capture was preliminarily demonstrated. In addition to the experimental studies, the technical challenges pertinent to fouling of slurry-handling equipment and the design of the crystallizer and stripper were addressed through consultation with vendors and engineering analyses. A process flow diagram of the Hot-CAP was then developed and a TEA was performed to compare the energy use and cost performance of a nominal 550-MWe subcritical pulverized coal (PC)-fired power plant without CO 2 capture (DOE/NETL Case 9) with the benchmark MEA-based post-combustion CO 2 capture (PCC; DOE/NETL Case 10) and the Hot-CAP-based PCC. The results revealed that the net power produced in the PC + Hot-CAP is 609 MWe, greater than the PC + MEA (550 MWe). The 20-year levelized cost of electricity (LCOE) for the PC + Hot-CAP, including CO 2 transportation and storage, is 120.3 mills/kWh, a 60% increase over the base PC plant without CO 2 capture. The LCOE increase for the Hot-CAP is 29% lower than that for MEA. TEA results demonstrated that the Hot-CAP is energy-efficient and cost-effective compared with the benchmark MEA process.

54 ENVIRONMENTAL SCIENCES↗

Testing nuclear data libraries with burnup for reactor applications

Recently US and European nuclear data libraries have been released, namely ENDF/B-VIII.0 and JEFF-3.3 libraries, which are the result of years of evaluation and validation work in both communities. Consequently, efforts have been made to validate the evaluations in calculations of integral experiments (critical benchmarks, ..etc). Nevertheless, less stringent testing and validation efforts were performed on burnup applications before releasing the libraries. The presented work focusses on the testing of these recent nuclear data libraries for burnup calculations on two benchmarks at the pin and at the assembly level. Monte-Carlo depletion calculations were performed using the VESTA 2.2 code. The K{sub ∞} results between nuclear data libraries are compared. A strong k{sub ∞} bias is observed with burnup using both JEFF-3.3 and JEFF-4T0 compared to all other libraries, and especially ENDF/B-VIII.0, consisting in a strong k{sub ∞} over-estimation at low burnup and a high under-estimation at high burnup. JEFF-3.3 {sup 235}U and {sup 239}Pu evaluations mainly explain this result, as well as fission yields. New {sup 235}U and {sup 239}Pu evaluations were proposed for JEFF-4T0, but they do not address totally the bias issue, even if JEFF-4T0 {sup 239}Pu allows slightly reducing the k{sub ∞} underestimation at high burnup.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Performance of Linear and Nonlinear Two-Leaf Light Use Efficiency Models at Different Temporal Scales

The reliable simulation of gross primary productivity (GPP) at various spatial and temporal scales is of significance to quantifying the net exchange of carbon between terrestrial ecosystems and the atmosphere. This study aimed to verify the ability of a nonlinear two-leaf model (TL-LUEn), a linear two-leaf model (TL-LUE), and a big-leaf light use efficiency model (MOD17) to simulate GPP at half-hourly, daily and 8-day scales using GPP derived from 58 eddy-covariance flux sites in Asia, Europe and North America as benchmarks. Model evaluation showed that the overall performance of TL-LUEn was slightly but not significantly better than TL-LUE at half-hourly and daily scale, while the overall performance of both TL-LUEn and TL-LUE were significantly better (p < 0.0001) than MOD17 at the two temporal scales. The improvement of TL-LUEn over TL-LUE was relatively small in comparison with the improvement of TL-LUE over MOD17. However, the differences between TL-LUEn and MOD17, and TL-LUE and MOD17 became less distinct at the 8-day scale. As for different vegetation types, TL-LUEn and TL-LUE performed better than MOD17 for all vegetation types except crops at the half-hourly scale. At the daily and 8-day scales, both TL-LUEn and TL-LUE outperformed MOD17 for forests. However, TL-LUEn had a mixed performance for the three non-forest types while TL-LUE outperformed MOD17 slightly for all these non-forest types at daily and 8-day scales. The better performance of TL-LUEn and TL-LUE for forests was mainly achieved by the correction of the underestimation/overestimation of GPP simulated by MOD17 under low/high solar radiation and sky clearness conditions. TL-LUEn is more applicable at individual sites at the half-hourly scale while TL-LUE could be regionally used at half-hourly, daily and 8-day scales. MOD17 is also an applicable option regionally at the 8-day scale.

54 ENVIRONMENTAL SCIENCES↗

Autotuning PolyBench benchmarks with LLVM Clang/Polly loop optimization pragmas using Bayesian optimization

Here, we develop a ytopt autotuning framework that leverages Bayesian optimization to explore the parameter space search and compare four different supervised learning methods within Bayesian optimization and evaluate their effectiveness. We select six of the most complex PolyBench benchmarks and apply the newly developed LLVM Clang/Polly loop optimization pragmas to the benchmarks to optimize them. We then use the autotuning framework to optimize the pragma parameters to improve their performance. The experimental results show that our autotuning approach outperforms the other compiling methods to provide the smallest execution time for the benchmarks syr2k, 3mm, heat-3d, lu, and covariance with two large datasets in 200 code evaluations for effectively searching the parameter spaces with up to 170,368 different configurations. We find that the Floyd-Warshall benchmark did not benefit from autotuning. To cope with this issue, we provide some compiler option solutions to improve the performance. Then we present loop autotuning without a user's knowledge using a simple mctree autotuning framework to further improve the performance of the Floyd-Warshall benchmark. We also extend the ytopt autotuning framework to tune a deep learning application.

79 ASTRONOMY AND ASTROPHYSICS↗

WfBench: Automated Generation of Scientific Workflow Benchmarks

The prevalence of scientific workflows with high computational demands calls for their execution on various distributed computing platforms, including large-scale leadership-class high-performance computing (HPC) clusters. To handle the deployment, monitoring, and optimization of workflow executions, many workflow systems have been developed over the past decade. There is a need for workflow benchmarks that can be used to evaluate the performance of workflow systems on current and future software stacks and hardware platforms.We present a generator of realistic workflow benchmark specifications that can be translated into benchmark code to be executed with current workflow systems. Our approach generates workflow tasks with arbitrary performance characteristics (CPU, memory, and I/O usage) and with realistic task dependency structures based on those seen in production workflows. We present experimental results that show that our approach generates benchmarks that are representative of production workflows, and conduct a case study to demonstrate the use and usefulness of our generated benchmarks to evaluate the performance of workflow systems under different configuration scenarios.

Coleman, Taina↗

Palacios and Kitten : high performance operating systems for scalable virtualized and native supercomputing.

Palacios and Kitten are new open source tools that enable applications, whether ported or not, to achieve scalable high performance on large machines. They provide a thin layer over the hardware to support both full-featured virtualized environments and native code bases. Kitten is an OS under development at Sandia that implements a lightweight kernel architecture to provide predictable behavior and increased flexibility on large machines, while also providing Linux binary compatibility. Palacios is a VMM that is under development at Northwestern University and the University of New Mexico. Palacios, which can be embedded into Kitten and other OSes, supports existing, unmodified applications and operating systems by using virtualization that leverages hardware technologies. We describe the design and implementation of both Kitten and Palacios. Our benchmarks show that they provide near native, scalable performance. Palacios and Kitten provide an incremental path to using supercomputer resources that is not performance-compromised.

99 GENERAL AND MISCELLANEOUS↗

PPT-GPU: Scalable GPU Performance Modeling

Performance modeling is a challenging problem due to the complexities of hardware architectures. In this paper, we present PPT-GPU, a scalable and accurate simulation framework that enables GPU code developers and architects to predict the performance of applications in a fast, and accurate manner on different GPU architectures. PPT-GPU is part of the open source project, Performance Prediction Toolkit (PPT) developed at the Los Alamos National Laboratory. We extend the old GPU model in PPT that predict the runtimes of computational physics codes to offer better prediction accuracy, for which, we add models for different memory hierarchies found in GPUs and latencies for different instructions. To further show the utility of PPT-GPU, we compare our model against real GPU device(s) and the widely used cycle-accurate simulator, GPGPU-Sim using different workloads from RODINIA and Parboil benchmarks. The results indicate that the predicted performance of PPT-GPU is within a 10% error compared to the real device(s). In addition, PPT-GPU is highly scalable, where it is up to 450x faster than GPGPU-Sim with more accurate results.

97 MATHEMATICS AND COMPUTING↗

Bootstrap embedding for interacting electrons in phonon coherent-state mean field

Here, we develop a Fermi–Bose bootstrap embedding framework for the ground state of interacting electrons coupled to a phonon mean field. The method combines bootstrap embedding for correlated electrons with a self-consistent coherent-state mean-field treatment for phonons. This method models the interacting electron–phonon problem as a system of correlated electrons traveling in a self-consistently specified potential landscape, allowing for efficient treatment of large lattice systems. Convergence of the methods for fragment size and total system size is demonstrated for the one-dimensional Hubbard–Holstein model for up to 350 sites. Finite-size scaling is performed to extrapolate to the infinite system size. Benchmarking against the density matrix renormalization group for a small 8-site system at half- and quarter-filling shows an orders-of-magnitude runtime advantage. The comparison further reveals that the method performs best in regimes dominated by localization, such as the Mott insulating phase and the strong-coupling tiny polaron regime, where the local embedding ansatz is still valid. However, due to the mean-field treatment for phonons, we find limitations of our methods in the weakly coupled delocalized region and at the Peierls transition, where quantum phonon fluctuations and long-range kinetic correlations become substantial.

Islam, Shariful [North Carolina State University, ↗

An Alternative Ensemble Streamflow Prediction Approach Using Improved Subseasonal Precipitation Forecasts from the North America Multi-Model Ensemble Phase II

In this article, streamflow forecasting at a subseasonal time scale (10–30 days into the future) is important for various human activities. The ensemble streamflow prediction (ESP) is a widely applied technique for subseasonal streamflow forecasting. However, ESP’s reliance on the randomly resampled historical precipitation limits its predictive capability. Available dynamical subseasonal precipitation forecasts provide an alternative to the randomly resampled precipitation in ESP. Prior studies found the predictive performance of raw subseasonal precipitation forecast is limited in many regions such as the central south of the United States, which raises questions about its effectiveness in assisting streamflow forecasting. To further assess the hydrologic applicability of dynamical subseasonal precipitation forecasts, we test the subseasonal precipitation forecast from North America Multi-Model Ensemble Phase II (NMME-2) at four watersheds in the central south region of the United States. The subseasonal precipitation forecasts are postprocessed with bias correction and spatial disaggregation (BCSD) to correct bias and improve spatial resolution before replacing the randomly resampled precipitation in ESP for streamflow predictions. The performance of the resulting streamflow predictions is benchmarked with ESP. Evaluation is conducted using Kling–Gupta Efficiency (KGE), continuous ranked probability score (CRPS), probability of detection (POD), false alarm ratios (FARs), as well as reliability diagrams. Our results suggest that BCSD-corrected subseasonal precipitation forecasts lead to overall improved streamflow predictions due to added skills in winter and spring. Our results also suggest that BCSD-corrected subseasonal precipitation forecasts lead to improved predictions on the occurrence of high-percentile streamflow values above 75%. Overall, BCSD-corrected subseasonal precipitation has shown promising performance, highlighting its potential broader applications for river and flood forecasting.

54 ENVIRONMENTAL SCIENCES↗

Enhancing the Performance of Assisted Execution Runtime Systems through Hardware/Software Techniques

To meet the expected performance, future exascale systems will require programmers to increase the level of parallelism of their applications. Novel programming models simplify parallel programming at the cost of increasing runtime overheard. Assisted execution models have the potential of reducing this overhead but they generally also reduce processor utilization. We propose an integrated hardware/software solution that automatically partition hardware resources between application and auxiliary threads. Each system level performs well-defined tasks efficiently: 1) the runtime system is enriched with a mechanism that automatically detects computing power requirements of running threads and drives the hardware actuators; 2) the hardware enforces dynamic resource partitioning; 3) the operating system provides an efficient interface between the runtime system and the hardware resource allocation mechanism. As a test case, we apply this adaptive approach to STM2, an software transactional memory system that implements the assisted execution model. We evaluate the proposed adaptive solution on an IBMPOWER7 system using Eigenbench and STAMP benchmark suite. Results show that our approach performs equal or better than the original STM2 and achieves up to 65% and 86% performance improvement for Eigenbench and STAMP applications, respectively.

Kestor, Gokcen↗

Specification for the VERA Depletion Benchmark Suite

CASL-X-2015-1014-000 iii Consortium for Advanced Simulation of LWRs EXECUTIVE SUMMARY The CASL neutronics simulator MPACT is under development for the neutronics and T-H coupled simulation for the pressurized water reactor. MPACT includes the ORIGEN-API and internal depletion module to perform depletion calculations based upon neutron-material reaction and radioactive decay. It is a challenge to validate the depletion capability because of the insufficient measured data. One of the detoured methods to validate it is to perform a code-to-code comparison for benchmark problems. In this study a depletion benchmark suite has been developed and a detailed guideline has been provided to obtain meaningful computational outcomes which can be used in the validation of the MPACT depletion capability.

97 MATHEMATICS AND COMPUTING↗

Comparing the Performance of Blue Gene/Q with Leading Cray XE6 and InfiniBand Systems

Abstract—Three types of systems dominate the current High Performance Computing landscape: the Cray XE6, the IBM Blue Gene, and commodity clusters using InfiniBand. These systems have quite different characteristics making the choice for a particular deployment difficult. The XE6 uses Cray’s proprietary Gemini 3-D torus interconnect with two nodes at each network endpoint. The latest IBM Blue Gene/Q uses a single socket integrating processor and communication in a 5-D torus network. InfiniBand provides the flexibility of using nodes from many vendors connected in many possible topologies. The performance characteristics of each vary vastly along with their utilization model. In this work we compare the performance of these three systems using a combination of micro-benchmarks and a set of production applications. In particular we discuss the causes of variability in performance across the systems and also quantify where performance is lost using a combination of measurements and models. Our results show that significant performance can be lost in normal production operation of the Cray XT6 and InfiniBand Clusters in comparison to Blue Gene/Q.

-High Performance Computing, Performance Evaluatio↗

Biochemical Conversion of Lignocellulosic Biomass to Hydrocarbon Fuels and Products: 2022 State of Technology and Future Research

The annual State of Technology (SOT) assessment is a key activity for biochemical platform research. It allows the impact of research progress (both directly achieved in-house at the National Renewable Energy Laboratory [NREL] and indirectly extrapolated from available public data) to be quantified in terms of economic improvements in the overall cellulosic biofuel production process for a particular conversion pathway. As such, initial benchmarks can be established for currently demonstrated performance and progress can be tracked toward out-year goals to ultimately demonstrate cost-competitive biofuel technology. Building upon efforts to progress NREL's biochemical platform R&D work toward ultimate 2030 goals to demonstrate less than $2.50/gallon gasoline equivalent (GGE) fuel selling prices, experimental and techno-economic analysis (TEA) activities have primarily focused on "advanced" biochemical processing strategies to fuels and coproducts, guided by TEA modeling to highlight key barriers and priorities toward achieving this goal across a number of potential bioconversion pathways. The purpose of the present effort is to benchmark the latest experimental developments for these pathways as quantified by modeled minimum fuel selling prices (MFSPs), as a measure of current status relative to those final targets. For this SOT, TEA models were run for two separate biological conversion pathways to fuels, based on available data for integrated biomass deconstruction and hydrolysate processing; namely carboxylic acids (primarily butyric acid) and diols (2,3-butanediol [BDO]), reflecting NREL's recently published 2018 biochemical design report focused on those two pathways. The models were run across two scenarios for lignin utilization, namely combustion and conversion to value-added coproducts.

09 BIOMASS FUELS↗

Cumulative $\chi^2$ Metric for VALID for ENDF/B-VII.1 and ENDF/B-VIII.0 in SCALE 6.3b9

Nuclear data testing is generally performed in a manner similar to validation: a series of critical benchmark experiments is modeled, and the differences between calculated and expected results are examined to understand the performance of a code and cross section set. This process is repeated for multiple sets of experiments with different transport codes and sometimes with different datasets. Occasionally, data processing differences are also important in understanding the differences in calculated results. Relevant experts then meet periodically to compare results and set priorities for improvement in the next round of data updates. This process has been largely effective, despite the confounding factors of different benchmark sets, application areas, sponsors, nationalities, and the resulting different priorities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Polyethylene-Reflected Arrays of HEU(93.2) Metal Units Separated by Vermiculite

This benchmark details the results of an experiment performed in the early 1970s as part of a series testing critical configurations in three dimensional arrays. For this experiment, cylinders of 93.2% enriched uranium metal were arranged in a 2x2x2 array inside of a polyethylene reflector. Layers of vermiculite of varying heights were surrounding each cylinder to achieve criticality variations. A total of four experimental configurations were tested by D.W. Magnuson, and detailed in his experimental report “Critical Three-Dimensional Arrays of Neutron Interacting Units: Part IV. Arrays of U(93.2) Metal Reflected by Concrete and Arrays Separated by Vermiculite and Reflected by Polyethylene.” The benchmark HEU-MET-FAST054 is closely related; the results of both experiments are discussed in the same report (Ref. 1) Closely related work has been recorded in HEU-MET-FAST-053, which is a benchmark evaluation of a different series of three dimensional array experiments with four different moderator materials. HEU-MET-FAST-023 and HEU-MET-FAST-026 are also related because they utilize the same metal cylinders as these experiments.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

PSO algorithm enhanced with Lozi Chaotic Map - Tuning experiment

In this paper it is investigated the effect of tuning of control parameters of the Lozi Chaotic Map employed as a chaotic pseudo-random number generator for the particle swarm optimization algorithm. Three different benchmark functions are selected from the IEEE CEC 2013 competition benchmark set. The Lozi map is extensively tuned and the performance of PSO is evaluated.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Time series anomaly detection in power electronics signals with recurrent and ConvLSTM autoencoders

The anomalies in the high voltage converter modulator (HVCM) remain a major down time for the spallation neutron source facility, that delivers the most intense neutron beam in the world for scientific materials research. In this work, we propose neural network architectures based on Recurrent AutoEncoders (RAE) to detect anomalies ahead of time in the power signals coming from the HVCM. Bi-directional gated recurrent unit, bi-directional long-short term memory (LSTM), and convolutional LSTM (ConvLSTM) are developed, trained, and tested using real experimental signals from the HVCM module. The results show a good performance of the proposed RAE models, achieving precision up to 91%, recall up to 88%, false omission rate as low as 20% (i.e. 80% of the anomalies were detected), and area under the ROC curve up to 0.9. The three RAE models provide very comparable performance, with LSTM showing slightly better performance than GRU and ConvLSTM. The RAE models are benchmarked against other anomaly detection methods, including isolation forest, support vector machine, local outlier factor, feedforward and convolutional autoencoders, and others; showing a better performance. Here, the results of this study demonstrate the promising potential of RAE in anomaly detection for real-world power systems, and for increasing the reliability of the HVCM modules in the spallation neutron source.

42 ENGINEERING↗