Search NASA⌕ Search

SEARCH · Search NASA

Results for “application benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Pebble Tanker Model for Nuclear Criticality Safety Needs

This report documents a study performed to investigate the requirements for criticality safety benchmark experiments for high-assay, low-enriched uranium (HALEU) fuel in transportation applications. In this work, an exploratory application model, the “Pebble Tanker,” was developed to represent TRISO fuel in a transportation scenario for an analysis of the validation basis in industrial quantities. An aspect of the criticality validation process involves assessing the “similarity” between application and experimental benchmark systems through an integral index parameter evaluation. Here, this includes propagating nuclear data uncertainties and calculating a correlation coefficient (hereinafter referred to as “c k ”) to evaluate the similarity of benchmark experiments compared with the application Pebble Tanker model. Finding sufficient critical benchmark experiments allows for the evaluation of bias and bias uncertainty, thus determining the upper subcritical limit (USL) of the transportation package. A target k eff of ~0.94 was used in this work to establish appropriate modeling conditions, reflecting a reasonable estimate for a USL. Two container models were investigated: one with the Hermes-type pebble and one with the Pebble Bed Modular Reactor (PBMR)–type pebble. The models were simplified, considering only fuel, containment structure, and either water or air. This allows a focus on the underlying physics of applications involving TRISO fuel pebbles using the Pebble Tanker model. A crucial consideration is the transport package's ability to safely hold pebbles while flooded, maintaining subcritical conditions. Tools available in the SCALE 6.3.1 suite—the CSAS6-Shift, TSUNAMI-3D-Shift, and TSUNAMI-IP sequences—were employed for neutronics and sensitivity and uncertainty (S/U) analysis of the Pebble Tanker. Findings demonstrated sufficient available critical experiment benchmarks to perform a validation of the Pebble Tanker in the most reactive state, i.e., when the Tanker is flooded.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

MLCommons Science Benchmarks

Benchmarks are a cornerstone of modern machine learning practice, providing standardized eval- uations that enable reproducibility, comparison, and scientific progress. Yet, as AI systems particularly deep learning models become increasingly dynamic, traditional static benchmarking approaches are losing their relevance. Models rapidly evolve in architecture, scale, and capability; datasets shift; and deployment contexts continuously change, creating a moving target for evaluation. Without adaptive benchmarking frame- works, both scientific assessment and real-world de- ployment risk becoming misaligned with actual system behavior. Drawing on our experience from MLCommons, educa- tional initiatives, and government programs such as the DOE s Million Parameter Consortium, we identify key barriers that hinder the broader adoption and utility of benchmarking in AI. These include substantial resource demands, limited access to specialized hardware, lack of expertise in benchmark design, and uncertainty among practitioners about how to relate benchmark results to their own application domains. Moreover, current benchmarks often emphasize peak performance on leadership-class hardware, offering limited guidance for more diverse, real-world deployment scenarios. We argue that benchmarking itself must become dy- namic in order to incorporate evolving models, updated data, and heterogeneous computational platforms while maintaining transparency, reproducibility, and inter- pretability. Democratizing this process requires not only technical innovation, but also systematic educational efforts spanning undergraduate to professional levels to develop sustained expertise in benchmark design and use. Finally, benchmarks should be framed and com- municated to support application-relevant comparisons, enabling both developers and users to make informed, context-sensitive decisions. Advancing dynamic and inclusive benchmarking practices will be essential to ensure that evaluation keeps pace with the evolving AI landscape and supports responsible, reproducible, and accessible AI deployment.

Hawks, Benjamin G. [Fermilab]↗
DOE OSTI2022

Survey of Relevant Data from the MSRP to Guide Development of MSR Chemistry Modeling Benchmarks

The Multiphysics Applications technical area of the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program is tasked with assessing, demonstrating, and applying NEAMS tools in solving multiphysics problems of nuclear reactors, such as liquid-fueled molten salt reactors (MSRs), which are the focus of this report. MSRs are actively being pursued as a potential candidate for power and heat generation by the nuclear industry. However, the inherent multiphysics nature of liquid-fueled MSRs, stemming from the strong interrelationship of neutronics, thermal fluids, and chemistry phenomena, provides unique challenges in modeling and simulation (mod/sim). Therefore, it will be important to develop mod/sim tools with varying types of multiphysics coupling that depend on the problem. The current work is focused on the initiation and development of MSR chemistry modeling benchmarks useful for validating current and potential future NEAMS tools. Development of such benchmarks include the following actions: 1) Summarize available chemistry data from the Oak Ridge National Laboratory (ORNL) MSR Program (MSRP) including operation of the Molten Salt Reactor Experiment (MSRE) and design studies for the Molten Salt Breeder Reactor (MSBR) concept; 2) Recommend simulation problems in MSR chemistry mod/sim based on (1); 3) Assess the current state of NEAMS tools that may support (2); 4) Demonstrate and validate the capabilities of the NEAMS tools in (3) while providing iterative feedback on future code development activities. The objective of this report is to initiate this effort by completing actions (1) and (2), which are discussed in Section 2. An overview of the relevant available MSRE experimental data is provided with examples of how this data may be useful in chemistry mod/sim problems, with the caveat that most of this data is over 50 years old, therefore some problem details as well as uncertainty estimates are often not provided. In Section 3, the current state of NEAMS tools is assessed for potential use in these mod/sim problems, with considerations for future code development activities, supporting action (3). Future work supporting this project under NEAMS may include a deeper dive into the specific phenomena discussed here including tasks such as the compilation of additional available data, updates in code development activities, and ultimately the demonstration and validation of these tools, supporting action (4).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2002

Multirate Flutter Suppression System Design for the Benchmark Active Controls Technology Wing: Methodology Application Software Toolbox - Part 2

To study the effectiveness of various control system design methodologies, the NASA Langley Research Center initiated the Benchmark Active Controls Project. In this project, the various methodologies were applied to design a flutter suppression system for the Benchmark Active Controls Technology (BACT) Wing. This report describes the user's manual and software toolbox developed at the University of Washington to design a multirate flutter suppression control law for the BACT wing.

Mason, Gregory S.↗
DOE OSTI2013

Strategies for Energy-Efficient Resource Management of Hybrid Programming Models

Many scientific applications are programmed using hybrid programming models that use both message passing and shared memory, due to the increasing prevalence of large-scale systems with multicore, multisocket nodes. Previous work has shown that energy efficiency can be improved using software-controlled execution schemes that consider both the programming model and the power-aware execution capabilities of the system. However, such approaches have focused on identifying optimal resource utilization for one programming model, either shared memory or message passing, in isolation. The potential solution space, thus the challenge, increases substantially when optimizing hybrid models since the possible resource configurations increase exponentially. Nonetheless, with the accelerating adoption of hybrid programming models, we increasingly need improved energy efficiency in hybrid parallel applications on large-scale systems. In this work, we present new software-controlled execution schemes that consider the effects of dynamic concurrency throttling (DCT) and dynamic voltage and frequency scaling (DVFS) in the context of hybrid programming models. Specifically, we present predictive models and novel algorithms based on statistical analysis that anticipate application power and time requirements under different concurrency and frequency configurations. Furthermore, we apply our models and methods to the NPB MZ benchmarks and selected applications from the ASC Sequoia codes. Overall, we achieve substantial energy savings (8.74 percent on average and up to 13.8 percent) with some performance gain (up to 7.5 percent) or negligible performance loss.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Benchmarking Variables for Checkpointing in HPC Applications

Checkpoint/Restart (C/R) is a widely used fault tolerance mechanism in converged systems of cloud, edge, and HPC. However, users often rely on their experience to determine which variables to checkpoint, as there is currently no benchmark that can provide a reference. This can result in checkpointing redundant or even incorrect variables. To address this issue, we propose a benchmark suite that includes critical variables for checkpointing, which have been manually identified, and a method for identifying those critical variables, with 20 representative HPC applications. Our method involves analyzing data dependency between variables to identify critical variables analytically. We verify the identified variables' correctness with a widely used C/R library FTI by an ablation study. With our benchmark suite and data dependency analysis, HPC practitioners now have a reference for identifying checkpointing variables and better knowledge of what kind of variables to checkpoint.

Fu, Xiang↗
NASA NTRS2003

Using Grid Benchmarks for Dynamic Scheduling of Grid Applications

Navigation or dynamic scheduling of applications on computational grids can be improved through the use of an application-specific characterization of grid resources. Current grid information systems provide a description of the resources, but do not contain any application-specific information. We define a GridScape as dynamic state of the grid resources. We measure the dynamic performance of these resources using the grid benchmarks. Then we use the GridScape for automatic assignment of the tasks of a grid application to grid resources. The scalability of the system is achieved by limiting the navigation overhead to a few percent of the application resource requirements. Our task submission and assignment protocol guarantees that the navigation system does not cause grid congestion. On a synthetic data mining application we demonstrate that Gridscape-based task assignment reduces the application tunaround time.

Frumkin, Michael↗
DOE OSTI2022

Benchmarking of ENDF/B-VIII.0 cross sections for LWR ex-core transport applications

ENDF/B-VIII.0 cross section data is benchmarked on a simple Monte-Carlo model representative of the ex-core geometry of a PWR used for reactor vessel neutron fluence calculations. The benchmarking indicates that the ENDF/B-VIII.0 results show large differences as compared to previous evaluations, which have been extensively compared to measured data. This exercise suggests that ENDF/B-VIII.0 nuclear data is not suitable for ex-core LWR radiation transport applications. Evaluators are aware of the problems with the ENDF/B-VIII.0 evaluation of iron and work is underway to improve the evaluations for an upcoming release.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2014

Benchmarking of the FENDL-3 Neutron Cross-section Data Starter Library for Fusion Applications

This paper summarizes the benchmark analyses performed in a joint effort of ENEA (Italy), JAEA (Japan), KIT (Germany), and the University of Wisconsin (USA) on a computational ITER benchmark and a series of 14 MeV neutron benchmark experiments. The computational benchmark revealed a modest increase of the neutron flux levels in the deep penetration regions and a substantial increase of the gas production in steel components. The comparison to experimental results showed good agreement with no substantial differences between FENDL-3.0 and FENDL-2.1 for most of the responses. In general, FENDL-3 shows an improved performance for fusion neutronics applications.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Availability of Critical Benchmark Experiments for the Pebble Tanker Transportation Model for Nuclear Criticality Safety Validation of TRISO Pebbles

This study addresses the need for comprehensive investigations into TRi-structural ISOtropic (TRISO) fuel pebble transportation validation. In this work, an exploratory model, the pebble tanker(PT), was developed with the aim of facilitating the validation of nuclear criticality safety calculations in the context of industrial-scale transportation of TRISO fuel. The PT model was designed to investigate the availability and applicability of critical benchmark experiments crucial for assessing the transportation of these pebbles. This work incorporated sensitivity/uncertainty (S/U) similarity studies to quantify the applicability of critical benchmark experiments and to address nuclear data uncertainties in the context of TRISO transportation. Two container models were investigated: one for the Hermes-type pebble and one for the Pebble Bed Modular Reactor (PBMR)–type pebble. The models were simplified, considering fuel, containment, and either water or air, to enable a focus on the underlying physics of applications involving TRISO fuel pebbles using the PT model. A crucial aspect under consideration was the capacity of the transport package to hold pebbles while ensuring subcriticality in the flooded state. An approach in the criticality validation process involves assessing the similarity between systems through an integral index parameter evaluation. This involves calculating a correlation coefficient (referred to as c k ) based on shared nuclear data–induced uncertainty between a benchmark experiment and the application of the PT model. To facilitate this analysis, the SCALE tools, particularly the CSAS6-Shift, TSUNAMI-3D-Shift, and TSUNAMI-IP sequences, were employed for comprehensive studies in neutronics and S/U analysis. Our findings showed that there are sufficient critical experimental benchmarks to perform this validation of the PT model in the most reactive state, i.e. when the tanker is flooded. This paper provides valuable insights into validating a transport package for Generation IV TRISO fuel pebbles.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2016

Plutonium Critical Mass Curve Comparison to Mass at Upper Subcritical Limit (USL) Using Whisper

Whisper is computational software designed to assist the nuclear criticality safety analyst with validation studies with the MCNP ® Monte Carlo radiation transport package. Standard approaches to validation rely on the selection of benchmarks based upon expert judgment. Whisper uses sensitivity/uncertainty (S/U) methods to select relevant benchmarks to a particular application or set of applications being analyzed. Using these benchmarks, Whisper computes a calculational margin. Whisper attempts to quantify the margin of subcriticality (MOS) from errors in software and uncertainties in nuclear data. The combination of the Whisper-derived calculational margin and MOS comprise the baseline upper subcritical limit (USL), to which an additional margin may be applied by the nuclear criticality safety analyst as appropriate to ensure subcriticality. A series of critical mass curves for plutonium, similar to those found in Figure 31 of LA-10860-MS, have been generated using MCNP6.1.1 and the iterative parameter study software, WORM_Solver. The baseline USL for each of the data points of the curves was then computed using Whisper 1.1. The USL was then used to determine the equivalent mass for plutonium metal-water system. ANSI/ANS-8.1 states that it is acceptable to use handbook data, such as the data directly from the LA-10860-MS, as it is already considered validated (Section 4.3 4) “Use of subcritical limit data provided in ANSI/ANS standards or accepted reference publications does not require further validation.”). This paper attempts to take a novel approach to visualize traditional critical mass curves and allows comparison with the amount of mass for which the k eff is equal to the USL (calculational margin + margin of subcriticality). However, the intent is to plot the critical mass data along with USL, not to suggest that already accepted handbook data should have new and more rigorous requirements for validation.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRS2001

Code Parallelization with CAPO: A User Manual

A software tool has been developed to assist the parallelization of scientific codes. This tool, CAPO, extends an existing parallelization toolkit, CAPTools developed at the University of Greenwich, to generate OpenMP parallel codes for shared memory architectures. This is an interactive toolkit to transform a serial Fortran application code to an equivalent parallel version of the software - in a small fraction of the time normally required for a manual parallelization. We first discuss the way in which loop types are categorized and how efficient OpenMP directives can be defined and inserted into the existing code using the in-depth interprocedural analysis. The use of the toolkit on a number of application codes ranging from benchmark to real-world application codes is presented. This will demonstrate the great potential of using the toolkit to quickly parallelize serial programs as well as the good performance achievable on a large number of toolkit to quickly parallelize serial programs as well as the good performance achievable on a large number of processors. The second part of the document gives references to the parameters and the graphic user interface implemented in the toolkit. Finally a set of tutorials is included for hands-on experiences with this toolkit.

Jin, Hao-Qiang↗
NASA NTRS1988

A system for evaluating Ada implementations using synthesized benchmarks

A system is described that allows project designers to evaluate the suitability of an Ada implementation for a particular application using tailored benchmarks. Specification of the load that an application is expected to generate is written in an Ada-like notation. An Ada program is then synthesized that generates the specified load. Studies of the way in which performance varies with various parameters of the application can be performed by the system automatically.

Knight, John C.↗
DOE OSTI2019

Oak Ridge OpenSHMEM Benchmark Suite

The assessment of application performance is a fundamental task in high-performance computing (HPC). The OpenSHMEM Benchmark (OSB) suite is a collection of micro-benchmarks and mini-applications/compute kernels that have been ported to use OpenSHMEM. Some, like the NPB OpenSHMEM benchmarks, have been published before while most others have been used for evaluations but never formally introduced or discussed. This suite puts them together and is useful for assessing the performance of different use cases of OpenSHMEM. This offers system implementers a useful means of measuring performance and assessing the effects of new features as well as implementation strategies. The suite is also useful for application developers to assess the performance of the growing number of OpenSHMEM implementations that are emerging. In this paper, we describe the current set of codes available within the OSB suite, how they are intended to be used, and, where possible, a snapshot of their behavior on one of the OpenSHMEM implementations available to us. We also include detailed descriptions of every benchmark and kernel, focusing on how OpenSHMEM was used. This includes details on the enhancements we made to the benchmarks to support multithreaded variants. We encourage the OpenSHMEM community to use, review, and provide feedback on the benchmarks.

Naughton III, Thomas↗
DOE OSTI2018

Benchmark Analysis of the HTR-10 with the MAMMOTH Reactor Physics Application

This report documents a set of benchmarks used to validate the Serpent and MAMMOTH neutron transport models of the HTR-10 reactor developed at Idaho National Laboratory. The high-fidelity Serpent Monte Carlo models of the HTR-10 critical and full core configurations include both random, discrete distributions of TRISO particles in the pebbles and random, discrete pebble distributions in the pebble bed core. The Serpent results agree very well with the critical and control rod worth measurements provided in the International Re- actor Physics Experiment Evaluation Project (IRPhEP) report if the ENDF/B-VII.r1 data- set is used. These Serpent models are subsequently used to prepare cross sections and flux tallies in a 10 coarse energy group structure for the MAMMOTH Reactor Physics MOOSE- based application. We find that MAMMOTH can reproduce the Monte Carlo solution for the HTR-10 reactor by using the PJFNK-SPH equivalence method. In all cases studied, the MAMMOTH results are within 120 pcm of the Monte Carlo reference calculation and the maximum errors in neutron absorption and generation rates are within 0.536% and 0.215% from the reference calculation, respectively. A set of benchmark exercises from the International Atomic Energy Agency (IAEA) HTR-10 benchmark in IAEA-TECDOC-1382 were conducted including parts B1 (critical), B2 (temperature coefficient) and B3 (control rod worth). The results we obtain for the critical core are in excellent agreement with the expected value and we confirm that with ENDF/B-VII.r1 data we match the critical eigen- value keff=1 to within 100 pcm. Our temperature coefficient calculation is consistent with the VSOP results generated by the German participants. The Chinese and South African VSOP models produce lower temperature coefficients, a fact that was noted in the original IAEA benchmark, but for which no explanation was provided. Both a full and single control rod worth calculations with Serpent and MAMMOTH agree well with the results from the IAEA benchmark. Finally, and most importantly, this work demonstrates that MAM- MOTH can recover the Monte Carlo high-fidelity solutions to an excellent accuracy using the PJFNK-SPH method. This sets the stage for the ultimate goal of using MAMMOTH in transient, multi-physics scenarios where correction of the cross sections is obtained for steady-state conditions and then used in the transient scenario for which no Monte-Carlo solution is available.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Accelerating Parallel Applications in Cloud Platforms via Adaptive Time-Slice Control

Cloud platforms can provide flexible and cost-effective environments for parallel applications. However, the resource over-commitment issues, i.e., cloud providers often provide much more executable virtual CPUs than available physical CPUs, still impede the synchronization operations of parallel applications, causing severe performance degradation. Existing methods optimize parallel applications by promoting the priorities of involved VMs. They cannot fully explore the performance of parallel applications, because they ignore the time-slice requirements of different phases of parallel applications. Furthermore, non-parallel applications experience unsatisfied performance because of low scheduling priorities. Given empirical analysis on time-slices of virtual machines (VMs), we find that shortening time-slices can mitigate synchronization overhead which incurs during communication phases, while over-short time-slices cause frequent cache misses in computation phases. Accordingly, we propose an Adaptive Time-slice Control (ATC) mechanism. ATC first detects the phases of parallel applications based on lock latency or cache misses. Then, ATC shortens time-slices during communication phases and prolongs time-slices during computation phases for parallel applications, and sets a uniform time-slice for non-parallel applications. Finally, we evaluate ATC using seven well-known benchmarks with 25+ applications. Experiments show that ATC obtains 1.5-75x performance gain for running parallel applications than state-of-the-art solutions, with nearly unaffected impact on non-parallel applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

An iron evaluation story: from TALYS model parameters to validation on the ASPIS benchmark with the Monte-Carlo code TRIPOLI-4

Iron is a widespread structural material in nuclear applications. However some benchmark studies show that for many libraries, including TENDL-2019, producing good iron evaluations is still an open challenge. This work aims at improving TENDL-2019 {sup 56}Fe and {sup 54}Fe evaluations through feedbacks from experimental data and Monte Carlo results of the iron benchmark ASPIS. Part of the T6 code infrastructure was used to produce the evaluations. In the new evaluations, particular attention to resonance parameters and formalism was paid to address the importance of elastic scattering. The total inelastic scattering and inelastic continuum cross sections were shaped by modifying the input parameters of TALYS, one of the T6 codes. Additionally, it was necessary to normalise some reaction channels to those of JENDL-4.0. With the new files ASPIS results, obtained with the Monte Carlo code TRIPOLI-4, improved noticeably compared to TENDL 2019. The evaluations and parameter setting modifications are to be included into TENDL-2021. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2017

Sensitivity Analysis of Neutron Multiplicity Counting Statistics Using First-Order Perturbation Theory and Application to a Subcritical Plutonium Metal Benchmark

It is frequently important to estimate the uncertainty and sensitivity of measured and computed detector responses in subcritical experiments and simulations. These uncertainties arise from the physical construction of the experiment, uncertainties in the transport parameters, and counting uncertainties. Perturbation theory enables sensitivity analysis (SA) and uncertainty quantification on integral quantities like detector responses. Here the aim of our work is to apply SA to the statistics of subcritical neutron multiplicity counting distributions. Current SA methods have only been applied to mean detector responses and the keff eigenvalue. For multiplicity counting experiments, knowledge of the higher-order counting moments and their uncertainties are essential for a complete SA. We apply perturbation theory to compute the sensitivity of neutron multiplicity counting moments to arbitrarily high order. Each moment is determined by solving an adjoint transport equation with a source term that is a function of the adjoint solutions for lower-order moments. This enables moments of arbitrarily high order to be sequentially determined, and it shows that each moment is sensitive to the uncertainties of all lower-order moments. To close our SA of the moments, we derive forward transport equations that are functions of the forward flux and lower-order moment adjoint fluxes. We verify our calculations for the first three moments by comparison with multiplicity counting measurements of a subcritical plutonium metal sphere. For the first three moments, the most influential parameters are ranked, and the validity of first-order perturbation theory is demonstrated by examining the series truncation error. This enables a detailed SA of subcritical multiplicity counting measurements of fissionable material based on transport theory.

42 ENGINEERING↗