Search NASA⌕ Search

SEARCH · Search NASA

Results for “Performance Benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,621 records · Page 90

Issues in Benchmarking Human Reliability Analysis Methods: A Literature Review

There is a diversity of human reliability analysis (HRA) methods available for use in assessing human performance within probabilistic risk assessments (PRA). Due to the significant differences in the methods, including the scope, approach, and underlying models, there is a need for an empirical comparison investigating the validity and reliability of the methods. To accomplish this empirical comparison, a benchmarking study comparing and evaluating HRA methods in assessing operator performance in simulator experiments is currently underway. In order to account for as many effects as possible in the construction of this benchmarking study, a literature review was conducted, reviewing past benchmarking studies in the areas of psychology and risk assessment. A number of lessons learned through these studies are presented in order to aid in the design of future HRA benchmarking endeavors.

99 GENERAL AND MISCELLANEOUS↗

Efficient DNA sequence compression with neural networks

Abstract Background The increasing production of genomic data has led to an intensified need for models that can cope efficiently with the lossless compression of DNA sequences. Important applications include long-term storage and compression-based data analysis. In the literature, only a few recent articles propose the use of neural networks for DNA sequence compression. However, they fall short when compared with specific DNA compression tools, such as GeCo2. This limitation is due to the absence of models specifically designed for DNA sequences. In this work, we combine the power of neural networks with specific DNA models. For this purpose, we created GeCo3, a new genomic sequence compressor that uses neural networks for mixing multiple context and substitution-tolerant context models. Findings We benchmark GeCo3 as a reference-free DNA compressor in 5 datasets, including a balanced and comprehensive dataset of DNA sequences, the Y-chromosome and human mitogenome, 2 compilations of archaeal and virus genomes, 4 whole genomes, and 2 collections of FASTQ data of a human virome and ancient DNA. GeCo3 achieves a solid improvement in compression over the previous version (GeCo2) of $2.4\%$, $7.1\%$, $6.1\%$, $5.8\%$, and $6.0\%$, respectively. To test its performance as a reference-based DNA compressor, we benchmark GeCo3 in 4 datasets constituted by the pairwise compression of the chromosomes of the genomes of several primates. GeCo3 improves the compression in $12.4\%$, $11.7\%$, $10.8\%$, and $10.1\%$ over the state of the art. The cost of this compression improvement is some additional computational time (1.7–3 times slower than GeCo2). The RAM use is constant, and the tool scales efficiently, independently of the sequence size. Overall, these values outperform the state of the art. Conclusions GeCo3 is a genomic sequence compressor with a neural network mixing approach that provides additional gains over top specific genomic compressors. The proposed mixing method is portable, requiring only the probabilities of the models as inputs, providing easy adaptation to other data compressors or compression-based data analysis tools. GeCo3 is released under GPLv3 and is available for free download at https://github.com/cobilab/geco3.

Silva, Milton↗

Full electromagnetic Vlasov code simulation of the Kelvin-Helmholtz instability

Recent advancement in numerical techniques for Vlasov simulations and their application to cross-scale coupling in the plasma universe are discussed. Magnetohydrodynamic (MHD) simulations are now widely used for numerical modeling of global and macroscopic phenomena. In the framework of the MHD approximation, however, diffusion coefficients such as resistivity and adiabatic index are given from empirical models. Thus there are recent attempts to understand first-principle kinetic processes in macroscopic phenomena, such as magnetic reconnection and the Kelvin-Helmholtz (KH) instability via full kinetic particle-in-cell and Vlasov codes. In the present study, a benchmark test for a new four-dimensional full electromagnetic Vlasov code is performed. First, the computational speed of the Vlasov code is measured and a linear performance scaling is obtained on a massively parallel supercomputer with more than 12 000 cores. Second, a first-principle Vlasov simulation of the KH instability is performed in order to evaluate current status of numerical techniques for Vlasov simulations. The KH instability is usually adopted as a benchmark test problem for guiding-center Vlasov codes, in which a cyclotron motion of charged particles is neglected. There is not any full electromagnetic Vlasov simulation of the KH instability; this is because it is difficult to follow E-vectorxB-vector drift motion accurately without approximations. The present first-principle Vlasov simulation has successfully represented the formation of KH vortices and its secondary instability. These results suggest that Vlasov code simulations would be a powerful approach for studies of cross-scale coupling on future Peta-scale supercomputers.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Status Report on the MSRE Transform Model for Thermal-Hydraulic Benchmarking

Over the last several years, there has been renewed interest in the development of molten salt reactor concepts for the next generation of nuclear power plants. The modeling tools used to design these reactors must be validated in order to accurately predict the thermal hydraulic behavior expected during operation. The majority of the practical experience with molten salt reactors comes from the Molten Salt Reactor Experiment (MSRE) that operated at Oak Ridge National Laboratory (ORNL) from 1964 until 1969. The work discussed in this paper uses the design information and experimental data from that period to create and validate a detailed model of the MSRE. Once completed, this model’s performance will be compared to other system analysis codes for thermal hydraulic benchmarking, and their ability to predict the thermal hydraulic performance of molten salt reactor systems will also be compared.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Joint LLNL, LANL, SNL, and IRSN High Multiplication Subcritical (Multiplicity) Benchmark Integral Experiment Execution (CED-3b Report)

This report documents the experimental configurations and measurements for IER-518: Joint LLNL, LANL, SNL, and IRSN High Multiplication Subcritical (Multiplicity) Benchmark Experiments. These measurements involved a series of subcritical configurations at the Sandia Critical Experiments (SCX) facility at Sandia National Laboratories (SNL). The purpose of these measurements was to produce time tagged neutron count data of configurations that exceed a subcritical multiplication of 20, which is the high end of the fundamental physics benchmarks currently available in the International Criticality Safety Benchmark Evaluation Project Handbook (ICSBEP). These measurements leverage experimental configurations 1 and detector systems 2 from previously accepted ICSBEP benchmark evaluations, allowing evaluations of these measurements to be performed at greatly reduced cost.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

MSRE TRANSFORM Model for Thermal-Hydraulic Benchmarking (Status Report)

Over the last several years, there has been renewed interest in the development of molten salt reactor concepts for the next generation of nuclear power plants. The modeling tools used to design these reactors must be validated in order to accurately predict the thermal hydraulic behavior expected during operation. The majority of the practical experience with molten salt reactors comes from the Molten Salt Reactor Experiment (MSRE) that operated at Oak Ridge National Laboratory (ORNL) from 1964 until 1969. The work discussed in this paper uses the design information and experimental data from that period to create and validate a detailed model of the MSRE. Once completed, this model’s performance will be compared to other system analysis codes for thermal hydraulic benchmarking, and their ability to predict the thermal hydraulic performance of molten salt reactor systems will also be compared.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Conservative, special-relativistic smoothed particle hydrodynamics

We present and test a new, special-relativistic formulation of smoothed particle hydrodynamics (SPH). Our approach benefits from several improvements with respect to earlier relativistic SPH formulations. It is self-consistently derived from the Lagrangian of an ideal fluid and accounts for the terms that stem from non-constant smoothing lengths, usually called 'grad-h terms'. In our approach, we evolve the canonical momentum and the canonical energy per baryon and thus circumvent some of the problems that have plagued earlier formulations of relativistic SPH. We further use a much improved artificial viscosity prescription which uses the extreme local eigenvalues of the Euler equations and triggers selectively on (a) shocks and (b) velocity noise. The shock trigger accurately monitors the relative density slope and uses it to fine-tune the amount of artificial viscosity that is applied. This procedure substantially sharpens shock fronts while still avoiding post-shock noise. If not triggered, the viscosity parameter of each particle decays to zero. None of these viscosity triggers is specific to special relativity, both could also be applied in Newtonian SPH. The performance of the new scheme is explored in a large variety of benchmark tests where it delivers excellent results. Generally, the grad-h terms deliver minor, though worthwhile, improvements. As expected for a Lagrangian method, it performs close to perfect in supersonic advection tests, but also in strong relativistic shocks, usually considered a particular challenge for SPH, the method yields convincing results. For example, due to its perfect conservation properties, it is able to handle Lorentz factors as large as {gamma} = 50,000 in the so-called wall shock test. Moreover, we find convincing results in a rarely shown, but challenging test that involves so-called relativistic simple waves and also in multi-dimensional shock tube tests.

97 MATHEMATICS AND COMPUTING↗

Benchmark Calculation for the Watts Bar Unit 1 Cycles 1-3 Using the SCALE 6.3/Polaris–PARCS v3.4.2 Code Package

Benchmark calculations for the Watts Bar Unit 1 cycles 1–3 were performed to validate the SCALE 6.3/Polaris–PARCS v3.4.2 with the ENDF/B-VII.1 56-group library by comparing the simulated results with the measured data. The benchmark results are used in evaluating uncertainties of the SCALE/Polaris–PARCS code package for pressurized water reactor physics analysis for key nuclear parameters such as reactivity, control bank worth, temperature coefficients, and pin and assembly power peaking factors. This report details plant and fuel design specifications and input data for SCALE/Polaris, GenPMAXS, and PARCS. Additional details are provided for the input and output files produced for the benchmark calculations. The benchmark results were summarized such that they can be used in evaluating uncertainties for key nuclear parameters with other benchmark results.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Entity Resolution at Large Scale: Benchmarking and Algorithmics

We seek scalable benchmarks for entity resolution problems. Solutions to these problems range from trivial approaches such as string sorting to sophisticated methods such as statistical relational learning. The theoretical and practical complexity of these approaches varies widely, so one of the primary purposes of a benchmark will be to quantify the trade-off between solution quality and runtime. We are motivated by the ubiquitous nature of entity resolution as a fundamental problem faced by any organization that ingests large amounts of noisy text data. A benchmark is typically a rigid specification that provides an objective measure usable for ranking implementations of an algorithm. For example the Top500 and HPCG500 bench- marks rank supercomputers based on their performance of dense and sparse linear algebra problems (respectively). These two benchmarks require participants to report FLOPS counts attainable on various machines. Our purpose is slightly different. Whereas the supercomputing benchmarks mentioned above hold algorithms constant and aim to rank machines, we are primarily interested in ranking algorithms. As mentioned above, entity resolution problems can be approached in completely different ways. We believe that users of our benchmarks must decide what sort of procedure to run before comparing implementations and architectures. Eventually, we also wish to provide a mechanism for ranking machines while holding algorithmic approach constant . Our primary contributions are parallel algorithms for computing solution quality mea- sures per entity. We find in some real datasets that many entities are quite easy to resolve while others are difficult, with a heavy skew toward the former case. Therefore, measures such as global confusion matrices, F measures, etc. do not meet our benchmarking needs. We design methods for computing solution quality at the granularity of a single entity in order to know when proposed solutions do well in difficult situations (perhaps justifying extra computational), or struggling in easy situations. We report on progress toward a viable benchmark for comparing entity resolution algo- rithms. Our work is incomplete, but we have designed and prototyped several algorithms to help evalute the solution quality of competing approaches to these problems. We envision a benchmark in which the objective measure is a ratio of solution quality to runtime.

97 MATHEMATICS AND COMPUTING↗

Modeling Enhancements, Cross-Section Generation Updates, and Benchmarking with Shift

This technical report documents the modeling enhancements, cross-section generation updates, and bench marking with the Shift Monte Carlo code performed under the US Department of Energy Nuclear Energy Advanced Modeling and Simulation Program in FY 2024. The work performed included several modeling enhancements, such as integration of cross-section generation in Titan and the ability to produce microscopic multigroup cross sections with Shift. Benchmarking of the cross sections produced by Shift and the two-step workflow with Griffin was performed for three problems: the Advanced Breeder Test Reactor, a generic pebble bed reactor, and a TRISO heat pipe microreactor. Comparisons of results from these benchmark problems were done with Serpent, OpenMC, and Griffin. These enhancements provide a robust foundation for applying Shift for both reference and two-step neutronics analysis for advanced reactor simulation.

97 MATHEMATICS AND COMPUTING↗

Building Energy Information Systems: User Case Studies

Measured energy performance data are essential to national efforts to improve building efficiency, as evidenced in recent benchmarking mandates, and in a growing body of work that indicates the value of permanent monitoring and energy information feedback. This paper presents case studies of energy information systems (EIS) at four enterprises and university campuses, focusing on the attained energy savings, and successes and challenges in technology use and integration. EIS are broadly defined as performance monitoring software, data acquisition hardware, and communication systems to store, analyze and display building energy information. Case investigations showed that the most common energy savings and instances of waste concerned scheduling errors, measurement and verification, and inefficient operations. Data quality is critical to effective EIS use, and is most challenging at the subsystem or component level, and with non-electric energy sources. Sophisticated prediction algorithms may not be well understood but can be applied quite effectively, and sites with custom benchmark models or metrics are more likely to perform analyses external to the EIS. Finally, resources and staffing were identified as a universal challenge, indicating a need to identify additional models of EIS use that extend beyond exclusive in-house use, to analysis services.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Optimization with the OpenACC-to-FPGA framework on the Arria 10 and Stratix 10 FPGAs

The reconfigurable computing paradigm with field programmable gate arrays (FPGAs) has received renewed interest in the high-performance computing field due to FPGAs’ unique combination of performance and energy efficiency. However, difficulties in programming and optimizing FPGAs have prevented them from being widely accepted as general-purpose computing devices. In accelerator-based heterogeneous computing, portability across diverse heterogeneous devices is also an important issue, but the unique architectural features in FPGAs make this difficult to achieve. To address these issues, a directive-based, high-level FPGA programming and optimization framework was previously developed. In this work, developed optimizations were combined holistically using the directive-based approach to show that each individual benchmark requires a unique set of optimizations to maximize performance. We perform this exploration on Intel Arria 10 and Stratix 10 FPGAs. We also explored the relationships between performance, resource usages, and compilation times, and investigated implications for performance portability. Finally, we present an initial evaluation of a real-world proxy application, LULESH.

97 MATHEMATICS AND COMPUTING↗

In-Depth Optimization with the OpenACC-to-FPGA Framework on an Arria 10 FPGA

The reconfigurable computing paradigm that uses field programmable gate arrays (FPGAs) has received renewed interest in the high-performance computing field due to FPGAs' unique combination of performance and energy efficiency. However, difficulties in programming and optimizing FPGAs have prevented them from being widely accepted as general-purpose computing devices. In accelerator-based heterogeneous computing, portability across diverse heterogeneous devices is also an important issue, but the unique architectural features in FPGAs make this difficult to achieve. To address these issues, a directive-based, high-level FPGA programming and optimization framework was previously developed. In this work, developed optimizations were combined holistically using the directive-based approach to show that each individual benchmark requires a unique set of optimizations to maximize performance. The relationships between FPGA resource usages and runtime performance were also explored.

Lambert, Jacob↗

ASC ATDM Level 2 Milestone #6015: Asynchronous Many-Task Software Stack Demonstration

This report is an outcome of the ASC ATDM Level 2 Milestone 6015: Asynchronous Many-Task Software Stack Demonstration. It comprises a summary and in depth analysis of DARMA and a DARMA-compliant Asynchronous Many-Task (AMT) runtime software stack. Herein performance and productivity of the over- all approach are assessed on benchmarks and proxy applications representative of the Sandia ATDM applications. As part of the effort to assess the perceived strengths and weaknesses of AMT models compared to more traditional methods, experiments were performed on ATS-1 (Advanced Technology Systems) test bed machines and Trinity. In addition to productivity and performance assessments, this report includes findings on the generality of DARMAs backend API as well as findings on interoperability with node- level and network-level system libraries. Together, this information provides a clear understanding of the strengths and limitations of the DARMA approach in the context of Sandias ATDM codes, to guide our future research and development in this area.

97 MATHEMATICS AND COMPUTING↗

Thermal tensor renormalization group simulations of square-lattice quantum spin models

In this work, we benchmark the well-controlled and numerically accurate exponential thermal tensor renormalization group (XTRG) in the simulation of interacting spin models in two dimensions. Finite temperature introduces a finite thermal correlation length ξ, such that for system sizes L >> ξ finite-size calculations actually simulate the thermodynamic limit. Here in this paper, we focus on the square lattice Heisenberg antiferromagnet (SLH) and quantum Ising models (QIM) on open and cylindrical geometries up to width W = 10 . We explore various one-dimensional mapping paths in the matrix product operator (MPO) representation, whose performance is clearly shown to be geometry dependent. We benchmark against quantum Monte Carlo (QMC) data, yet also the series-expansion thermal tensor network results. Thermal properties including the internal energy, specific heat, and spin structure factors, etc. are computed with high precision, obtaining excellent agreement with QMC results. XTRG also allows us to reach remarkably low temperatures. For SLH, we obtain an energy per site u$^*_g$ ≃ -0.6694 (4) and a spontaneous magnetization m$^*_S$ ≃ 0.30 ( 1 ) already consistent with the ground-state properties, which is obtained from extrapolated low-T thermal data on W ≤ 8 cylinders and W ≤ 10 open strips, respectively. We extract an exponential divergence versus T of the structure factor S (M), as well as the correlation length ξ, at the ordering wave vector M = ($π,π$) , which represents the renormalized classical behavior and can be observed over a narrow but appreciable temperature window, by analyzing the finite-size data by XTRG simulations. For the QIM with a finite-temperature phase transition, we employ several thermal quantities, including the specific heat, Binder ratio, as well as the MPO entanglement to determine the critical temperature $T_c$.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Quadratic integrand double-hybrid made spin-component-scaled

We propose two analytical expressions aiming to rationalize the spin-component-scaled (SCS) and spin-opposite-scaled (SOS) schemes for double-hybrid exchange-correlation density-functionals. Their performances are extensively tested within the framework of the nonempirical quadratic integrand double-hybrid (QIDH) model on energetic properties included into the very large GMTKN30 benchmark database, and on structural properties of semirigid medium-sized organic compounds. The SOS variant is revealed as a less computationally demanding alternative to reach the accuracy of the original QIDH model without losing any theoretical background.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Mixing Study for JT-71/72 Tanks

All modeling calculations for the mixing operations of miscible fluids contained in HB-Line tanks, JT-71/72, were performed by taking a three-dimensional Computational Fluid Dynamics (CFD) approach. The CFD modeling results were benchmarked against the literature results and the previous SRNL test results to validate the model. Final performance calculations were performed by using the validated model to quantify the mixing time for the HB-Line tanks. The mixing study results for the JT-71/72 tanks show that, for the cases modeled, the mixing time required for blending of the tank contents is no more than 35 minutes, which is well below 2.5 hours of recirculation pump operation. Therefore, the results demonstrate the adequacy of 2.5 hours’ mixing time of the tank contents by one recirculation pump to get well mixed.

42 ENGINEERING↗

HYDRAULICS AND MIXING EVALUATIONS FOR NT-21/41 TANKS

The hydraulic results demonstrate that pump head pressure of 20 psi recirculates about 5.6 liters/min flowrate through the existing 0.131-inch orifice when a valve connected to NT-41 is closed. In case of the valve open to NT-41, the solution flowrates to HB-Line tanks, NT-21 and NT-41, are found to be about 0.5 lpm and 5.2 lpm, respectively. The modeling calculations for the mixing operations of miscible fluids contained in the HB-Line tank NT-21 were performed by taking a three-dimensional Computational Fluid Dynamics (CFD) approach. The CFD modeling results were benchmarked against the literature results and the previous SRNL test results to validate the model. Final performance calculations were performed for the nominal case by using the validated model to quantify the mixing time for the HB-Line tank. The results demonstrate that when a pump recirculates a solution volume of 5.7 liters every minute out of the 72-liter tank contents containing two acid solutions of 2.7 M and 0 M concentrations (i.e., water), a minimum mixing time of 1.5 hours is adequate for the tank contents to get the tank contents adequately mixed. In addition, the sensitivity results for the tank contents of 8 M existing solution and 1.5 M incoming species show that the mixing time takes about 2 hours to get the solutions mixed.

42 ENGINEERING↗