Search NASA⌕ Search

SEARCH · Search NASA

Results for “Performance Benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,387 records · Page 77

Continuous Reliability Enhancement for Wind (CREW) database :

To benchmark the current U.S. wind turbine fleet reliability performance and identify the major contributors to component-level failures and other downtime events, the Department of Energy funded the development of the Continuous Reliability Enhancement for Wind (CREW) database by Sandia National Laboratories. This report is the third annual Wind Plant Reliability Benchmark, to publically report on CREW findings for the wind industry. The CREW database uses both high resolution Supervisory Control and Data Acquisition (SCADA) data from operating plants and Strategic Power Systems ORAPWindª (Operational Reliability Analysis Program for Wind) data, which consist of downtime and reserve event records and daily summaries of various time categories for each turbine. Together, these data are used as inputs into CREW s reliability modeling. The results presented here include: the primary CREW Benchmark statistics (operational availability, utilization, capacity factor, mean time between events, and mean downtime); time accounting from an availability perspective; time accounting in terms of the combination of wind speed and generation levels; power curve analysis; and top system and component contributors to unavailability.

Hines, Valerie Ann-Peters↗

MUSiC: Critical Experiment with Bare Highly Enriched Uranium Shells Benchmark

The measurements of the Uranium Subcritical and Critical (MUSiC) experiment were performed from December 2020 through April 2021. The purpose of the experiment was to test a variety of multiplicity detection systems for a large range of multiplication values and to see how well they perform as the k eff of the MUSiC series of experiments approached the delayed critical window. Out of the ten configurations that were studied, two of them were super critical and were recently approved as benchmarks and they will be included in the International Criticality Safety Benchmark Evaluation Project (ICBESP) handbook. For each of the two experiments, uncertainties were evaluated six categories: (1) critical measurement, (2) mass, (3) composition, (4) positioning, (5) dimensions, and (6) temperature. The largest contribution to the overall uncertainty in each of the cases was due to the uncertainties in the dimensions of the HEU Rocky Flats shells. Simplified detailed models were created with an average bias of -130.5 pcm due to the removal various components surrounding the experiment. For MCNP6.2 models using ENDF-VIII.0 neutron cross section data, the average difference between the calculated and benchmark k effs was -54 pcm for the detailed model and -102 pcm for the simplified model.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Benchmarking a neutral-atom quantum computer

In this study, we simulated the algorithmic performance of a small neutral atom quantum computer and compared its performance when operating with all-to-all versus nearest-neighbor connectivity. This comparison was made using a suite of algorithmic benchmarks developed by the Quantum Economic Development Consortium. Circuits were simulated with a noise model consistent with experimental data from [Nature 604, 457 (2022)]. We find that all-to-all connectivity improves simulated circuit fidelity by [Formula: see text]–[Formula: see text], compared to nearest-neighbor connectivity.

Computer Science↗

Continuous Reliability Enhancement for Wind (CREW) Database: Wind Plant Reliability Benchmark

To benchmark the current U.S. wind turbine fleet reliability performance and identify the major contributors to component-level failures and other downtime events, the Department of Energy (DOE) funded the development of the Continuous Reliability Enhancement for Wind (CREW) database by Sandia National Laboratories. This report is the second annual Wind Plant Reliability Benchmark, to publically report on CREW findings for the entire wind industry. The CREW database uses both high resolution Supervisory Control and Data Acquisition (SCADA) data from operating plants and Strategic Power Systems’ (SPS) ORAPWind® (Operational Reliability Analysis Program for Wind) data, which consists of downtime and reserve event records and daily summaries of Generating, Unavailable, and Reserve time for each turbine. Together, these data are used as inputs into CREW’s reliability modeling. The results presented here include: the primary CREW Benchmark statistics (operational availability, utilization, capacity factor, mean time between events, and mean downtime); time accounting from an availability perspective; time accounting in terms of the combination of wind speed and generation levels; power curve analysis; and top system and component contributors to unavailability.

Peters, Valerie A.↗

Genetic algorithm based task reordering to improve the performance of batch scheduled massively parallel scientific applications

The growth in size of networked high performance computers along with novel accelerator-based node architectures has further emphasized the importance of communication efficiency in high performance computing. The world's largest high performance computers are usually operated as shared user facilities due to the costs of acquisition and operation. Applications are scheduled for execution in a shared environment and are placed on nodes that are not necessarily contiguous on the interconnect. Furthermore, the placement of tasks on the nodes allocated by the scheduler is sub-optimal, leading to performance loss and variability. Here, we investigate the impact of task placement on the performance of two massively parallel application codes on the Titan supercomputer, a turbulent combustion flow solver (S3D) and a molecular dynamics code (LAMMPS). Benchmark studies show a significant deviation from ideal weak scaling and variability in performance. The inter-task communication distance was determined to be one of the significant contributors to the performance degradation and variability. A genetic algorithm-based parallel optimization technique was used to optimize the task ordering. This technique provides an improved placement of the tasks on the nodes, taking into account the application's communication topology and the system interconnect topology. As a result, application benchmarks after task reordering through genetic algorithm show a significant improvement in performance and reduction in variability, therefore enabling the applications to achieve better time to solution and scalability on Titan during production.

97 MATHEMATICS AND COMPUTING↗

Benchmarking and Evaluating Unified Memory for OpenMP GPU Offloading

Here, the latest OpenMP standard offers automatic device offloading capabilities which facilitate GPU programming. Despite this, there remain many challenges. One of these is the unified memory feature introduced in recent GPUs. GPUs in current and future HPC systems have enhanced support for unified memory space. In such systems, CPU and GPU can access each other's memory transparently, that is, the data movement is managed automatically by the underlying system software and hardware. Memory over subscription is also possible in these systems. However, there is a significant lack of knowledge about how this mechanism will perform, and how programmers should use it. We have modified several benchmarks codes, in the Rodinia benchmark suite, to study the behavior of OpenMP accelerator extensions and have used them to explore the impact of unified memory in an OpenMP context. We moreover modified the open source LLVM compiler to allow OpenMP programs to exploit unified memory. The results of our evaluation reveal that, while the performance of unified memory is comparable with that of normal GPU offloading for benchmarks with little data reuse, it suffers from significant overhead when GPU memory is over subcribed for benchmarks with large amount of data reuse. Based on these results, we provide several guidelines for programmers to achieve better performance with unified memory.

97 MATHEMATICS AND COMPUTING↗

Building Benchmark Data Platform

This project is a three-year, four-laboratory collaboration to collect and curate a handful of high-resolution building systems datasets that have broad applicability to address highest-impact use cases. We will collect and curate high-resolution, well-calibrated time series of building operational and indoor/outdoor environmental data, which are crucial to understanding and optimizing building energy efficiency performance and demand flexibility capabilities as well as benchmarking energy algorithms.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

iSSH v. Auditd: Intrusion Detection in High Performance Computing

The goal is to provide insight into intrusions in high performance computing, focusing on tracking intruders motions through the system. The current tools, such as pattern matching, do not provide sufficient tracking capabilities. We tested two tools: an instrumented version of SSH (iSSH) and Linux Auditing Framework (Auditd). First discussed is Instrumented Secure Shell (iSSH): a version of SSH developed at Lawrence Berkeley National Laboratory. The goal is to audit user activity within a computer system to increase security. Capabilities are: Keystroke logging, Records user names and authentication information, and Catching suspicious remote and local commands. Strengths for iSSH are: (1) Good for keystroke logging, making it easier to track malicious users by catching suspicious commands; (2) Works with Bro to send alerts; could be configured to send pages to systems administrators; and (3) Creates visibility into SSH sessions. Weaknesses are: (1) Relatively new, so not very well documented; and (2) No capabilities to see if files have been edited, moved, or copied within the system. Second we discuss Auditd, the user component of the Linux Auditing System. It creates logs of user behavior, and monitors systems calls and file accesses. Its goal is to improve system security by keeping track of users actions within the system. Strenghts of Auditd are: (1) Very thorough logs; (2) Wider variety of tracking abilities than iSSH; and (3) Older, so better documented. Weaknesses are: (1) Logs record everything, not just malicious behavior; (2) The size of the logs can lead to overflowing directories; and (3) This level of logging leads to a lot of false alarms. Auditd is better documented than iSSH, which would help administrators during set up and troubleshooting. iSSH has a cleaner notification system, but the logs are not as detailed as Auditd. From our performance testing: (1) File transfer speed using SCP is increased when using iSSH; and (2) Network benchmarks were roughly the same regardless of which tool was running.

99 GENERAL AND MISCELLANEOUS↗

Global Transformations for Legacy Parallel Applications via Structural Analysis and Rewriting

Performance and scalability optimization of large HPC applications is currently a labor-intensive, manual process with very low productivity. Major difficulties come from the disaggregated environment for HPC application development: the compiler is only involved in local decisions (core or multithreaded domain), while a library-based, communication-oriented programming model realizes whole-machine parallelism. Realizing any major global change in such a disaggregated environment is very difficult and involves changing large portions of the source code. We present semi-automated techniques, based on structural analysis and rewriting, for performing global transformations on an HPC application source code. We present two case studies using the Self-Consistent Field (SCF) standalone benchmark as well as the Coupled Cluster (CCSD) module (2.9 million lines of Fortran code), a key module of the NWChem computational chemistry application. We demonstrate how structural rewriting techniques can be used to automate transformations that affect multiple sections of the application's source code. We show that the transformations can be applied in a systematic fashion across the source code bases with minimal manual effort. These transformations improve the scalability of the SCF benchmark by more than two orders of magnitude and the performance of the full CCSD module by a factor of 4.0.

legacy parallel applications↗

MPACT Verification With Magnox Reactor Neutronics Progression Problems

MPACT is a state-of-the-art core simulator designed to perform high-fidelity analysis using whole-core, three-dimensional, pin-resolved neutron transport calculations on modern parallel computing hardware. MPACT was originally developed to model light water reactors, and its capabilities are being extended to simulate gas-cooled, graphite-moderated cores such as Magnox reactors. To verify MPACT’s performance in this new application, the code is being formally benchmarked using representative problems. Progression problems are a series of example models that increase in complexity designed to test a code’s performance. The progression problems include both beginning-of-cycle and depletion calculations. Reference solutions for each progression problem have been generated using Serpent 2, a continuous-energy Monte Carlo reactor physics burnup calculation code.Using the neutron multiplication eigenvalue ke as a metric, MPACT’s performance is assessed on each of the progression problems. Initial results showed that MPACT’s multi-group cross section libraries, originally developed for pressurized water reactor problems, were not sufficient to accurately solve Magnox problems. MPACT’s improved performance on the progression problems is demonstrated using this new optimized cross section library.

Luciano, Nicholas↗

Computational Offload with BlueField Smart NICs

The recent introduction of a new generation of "smart NICs" have provided new accelerator platforms that include CPU cores or reconfigurable fabric in addition to traditional networking hardware and packet offloading capabilities. While there are currently several proposals for using these smartNICs for low-latency, in-line packet processing operations, there remains a gap in knowledge as to how they might be used as computational accelerators for traditional high-performance applications. This work aims to look at benchmarks and mini-applications to evaluate possible benefits of using a smartNIC as a compute accelerator for HPC applications. We investigate NVIDIA's current-generation BlueField-2 card, which includes eight Arm CPUs along with a small amount of storage, and we test the networking and data movement performance of these cards compared to a standard Intel server host. We then detail how two different applications, YASK and miniMD can be modified to make more efficient use of the BlueField-2 device with a focus on overlapping computation and communication for operations like neighbor building and halo exchanges. Our results show that while the overall compute performance of these devices is limited, using them with a modified miniMD algorithm allows for potential speedups of 5 to 20% over the host CPU baseline with no loss in simulation accuracy.

97 MATHEMATICS AND COMPUTING↗

On the effect of scalar flux weighing of linearly anisotropic scattering matrices in few-group transport calculations

The majority of the codes available for homogenized group constant generation for deterministic transport calculations apply the approximation of scalar flux weighting during energy group condensation of higher-order anisotropic scattering matrices. In this paper, we discuss the effect of scalar flux weighting of the linearly anisotropic scattering matrices in the frame of S P{sub 3} and S{sub 12} calculations performed for a two-dimensional VVER-440 reactor benchmark. To compare group constants generated for two neutron energy groups, an infinite pin cell was homogenized with Serpent 2 and ERANOS ECCO. Serpent 2 applies scalar flux, while ERANOS ECCO performs current weighting of the linearly anisotropic scattering matrices during energy group condensation. For analyzing the effect of the various weighting options, three simple reactor models were built assuming different core sizes using standard rectangular assemblies with 15*15 fuel pins. Diffusion, SP{sub 3} and S{sub 12} calculations were performed for the 3 models using group constants generated with Serpent 2 and ERANOS ECCO. The effect of scalar flux weighting of linearly anisotropic scattering matrices is shown by comparing the decrease in reactivity due to the decreased reactor size, as well as the assembly power distribution to reference results obtained with Serpent 2 Monte Carlo calculations. Neglecting higher than linearly anisotropic scattering and indirect application of diffusion coefficients in higher-order transport calculations is advised if angular flux-moment spectra weighted higher-order scattering matrices cannot be generated. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

REAL WASTE TESTING OF SPHERICAL RESORCINOL-FORMALDEHYDE ION EXCHANGE RESIN

This report presents data on batch contact and column testing tasks for spherical resorcinol-formaldehyde (sRF) resin. The testing used a non-radioactive simulant of SRS Tank 2F dissolved salt, as well as an actual radioactive waste sample of similar composition, which are both notably high in sodium (6 M). The resin was Microbeads batch 5E-370/641 which had been made on the hundred gallon scale. Equilibrium batch contact work focused on cesium at a temperature of 25 C due to the lack of such data to better benchmark existing isotherm models. Two campaigns were performed with small-scale ion exchange columns, first with Tank 2F simulant, then with actual dissolved salt in the Shielded Cells. An extrapolation of the batch contact results with radioactive waste over-predicted the cesium loaded onto the IX sRF resin bed by approximately 11%. This difference is not unexpected considering uncertainties from measurement and extrapolation and because the ion exchange that occurs when waste flows through a resin bed probably cannot reach the same level of equilibrium as when waste and resin are joined in a long term batch contact. Resin was also characterized to better understand basic chemistry issues such as holdup of trace transition metals present in the waste feed streams. The column tests involved using two beds of sRF resin in series, with the first bed referred to as the Lead column and the second bed as the Lag column. The test matrix included two complete IX cycles for both the simulant and actual waste phases. A cycle involves cesium adsorption, until the resin in the Lead column reaches saturation, and then regenerating the sRF resin, which includes eluting the cesium. Both the simulated and the actual wastes were treated with two cycles of operation, and the resin beds that were used in the Lead and Lag columns of simulant test phase were regenerated and reused in the actual waste test phase. This task is the first to demonstrate the treatment of SRS waste with sRF resin and the tests clearly demonstrated cesium decontamination for actual waste. The results of the column tests were similar for both the simulated and the actual waste and demonstrated Cs removal with sRF from both wastes. For a flowrate of 1.4 bed volumes (BV)/hour at 25 C those results with sRF resin were: (1) Simulant and actual waste results are equivalent; (2) Cs breakthrough began between 200 and 250 BV; (3) Cs breakthrough reached 100% at around 400 BV; (4) Cs breakthrough curve from 5% to 100% is approximately linear; (5) Cs elution with 0.5 M HNO3 starts at 2 BV and ends at 6BV; (6) Most, if not all, of Cs adsorbed during treatment is released during elution; (7) At 100% breakthrough of Cs the resin bed adsorbs approximately 85% of full capacity before detection in the effluent; the remaining 15% is adsorbed at saturation; (8) Approximately 90% of resin bed changes (color and volume) are complete by 6 BV; and (9) During elution the resin shrinks to about 80% of its fully working (sodium form) BV.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Reinforcement Learning for Anomaly Detection in Nuclear Power Plant Operation and Maintenance

In nuclear power plants (NPPs), timely identification of sensor and human errors is critical to ensure safe and efficient plant operations. Anomaly detection models can be employed for this task. However, traditional anomaly detection approaches may have high dependency on labeled datasets and struggle with adaptability in complex, dynamic environments. Reinforcement learning (RL) has demonstrated significant potential in fault diagnosis and anomaly detection; however, its application to anomaly detection in NPPs remains a relatively underexplored research direction. Hence, to address this gap, in this study, we present a novel physics-informed reinforcement learning model, PIRL-AD: Physics-Informed Reinforcement Learning for Anomaly Detection, that integrates domain knowledge from calorimetric equations into the RL framework for enhanced sensor and human error anomaly detection. We evaluate the performance of PIRL-AD against a non-physics informed RL benchmark and a support vector machine (SVM) on data collected from a forced flow loop testbed. Experimental results suggest that PIRL-AD outperforms other baselines on a range of anomalous datasets that include both sensor and human-induced anomalies across key performance metrics, statistically outperforming the RL and SVM benchmarks with respect to geometric mean (respectively, 92.96% vs. 91.06% vs. 83.01%) and F1-score (respectively, 89.23% vs. 86.98% vs. 77.01%). Furthermore, the findings suggest the potential of physics-integrated reinforcement learning models for enhanced anomaly detection performance in NPPs.

Reinforcement learning↗

A Reproducible Validation of Algorithms for Estimating Array Tilt and Azimuth from Photovoltaic Power Time Series

In this research, we assess the viability of four different, publicly available algorithms for estimating the azimuth and tilt parameters of solar photovoltaic systems using only the associated AC power time series data and site latitude-longitude coordinates. In this work, we curated a benchmarking data set of 44 fixed-tilt systems, comprising 275 measured AC power inverter data streams, with known azimuth and tilt parameters. Additionally, we isolated test cases in the data set with real- world issues, including shading and clipping, to determine how algorithm performance varies based on the presence of these phenomena. Using this data set for benchmarking, we evaluated the estimated vs. actual system characteristics for each algorithm, as well as the associated algorithm execution time using a standardized benchmarking process. The two highest performing algorithms were the Solar Data Tools and the PVWatts 5- based methods, which both achieved a median absolute error of approximately 5 and 1 degrees for azimuth and tilt, respectively. During run time analysis, the SDT method was approximately 5 times faster than the PVWatts 5-based method, with the median execution time for a stream varying between 6 and 8 seconds vs. a median run time of 31 seconds for the PVWatts 5-based method.

azimuth↗

Experimental Covariance Determination for Critical Integral Experiments

Integral benchmarks for criticality safety and nuclear data validation require expensive uncertainty quantification studies. In general, uncertainty quantification techniques ignore correlations between experiments and shared parts. Experiments, such as the TEX (Thermal/Epithermal eXperiments) campaigns, consist of many shared parts, such as the ‘Jemima’ HEU fuel plates, which create a strong correlation in their uncertainties. While these correlations are known to exist, they are often not estimated due to the complexity of such calculations. This paper describes an intuitive method of determining the covariance for each of the experimental components, providing a correlation for each family of parts across the multiple cases examined within a benchmark. A proof-of-principle study using the TEX-HEU experimental campaign was performed and verified that the correlations can be calculated with information commonly found in the ICSBEP (International Criticality Safety Benchmark Evaluation Project) benchmarks. This study showed that the introduction of model and experimental covariances reduces the χ 2 per degree of freedom from 2.203 to 1.179, indicating that the omission causes overly pessimistic bias quantifications. This technique can be seamlessly integrated to current benchmark evaluations as well as reevaluations of legacy benchmarks.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗