Search NASA⌕ Search

SEARCH · Search NASA

Results for “performance benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

Are Terrestrial Biosphere Models Fit for Simulating the Global Land Carbon Sink?

The Global Carbon Project estimates that the terrestrial biosphere has absorbed about one-third of anthropogenic CO 2 emissions during the 1959–2019 period. This sink-estimate is produced by an ensemble of terrestrial biosphere models and is consistent with the land uptake inferred from the residual of emissions and ocean uptake. The purpose of our study is to understand how well terrestrial biosphere models reproduce the processes that drive the terrestrial carbon sink. One challenge is to decide what level of agreement between model output and observation-based reference data is adequate considering that reference data are prone to uncertainties. To define such a level of agreement, we compute benchmark scores that quantify the similarity between independently derived reference data sets using multiple statistical metrics. Models are considered to perform well if their model scores reach benchmark scores. Our results show that reference data can differ considerably, causing benchmark scores to be low. Model scores are often of similar magnitude as benchmark scores, implying that model performance is reasonable given how different reference data are. While model performance is encouraging, ample potential for improvements remains, including a reduction in a positive leaf area index bias, improved representations of processes that govern soil organic carbon in high latitudes, and an assessment of causes that drive the inter-model spread of gross primary productivity in boreal regions and humid tropics. The success of future model development will increasingly depend on our capacity to reduce and account for observational uncertainties.

58 GEOSCIENCES↗

Implementation of NAS Parallel Benchmarks in Java

A number of features make Java an attractive but a debatable choice for High Performance Computing (HPC). In order to gauge the applicability of Java to the Computational Fluid Dynamics (CFD) we have implemented NAS Parallel Benchmarks in Java. The performance and scalability of the benchmarks point out the areas where improvement in Java compiler technology and in Java thread implementation would move Java closer to Fortran in the competition for CFD applications.

Frumkin, Michael↗

On-Orbit Radiometric Characterization of OLI (Landsat 8) for Applications in Aquatic Remote Sensing

Landsat-8 carries two separate sensors, namely the Operational Land Imager (OLI) and the Thermal Infrared Radiometer Suite (TIRS), that image the earth surface throughout the visible and thermal portions of the spectrum. Compared to Landsat heritage sensors, the OLI has enhanced features, which include its 12-bit radiometric resolution and the addition of a band centered at 443 nm. The dramatically improved data quality/quantity expands existing applications of Landsat imagery in aquatic sciences from the retrieval of bio-geochemical properties, such as near-surface concentrations of chlorophyll-a (CHL) and total suspended solids (TSS), to benthic mapping. This study offers analysis of OLI's absolute radiometric performance over bodies ofwater using benchmark observations, namely the top-of-atmosphere (TOA) ocean color observations and marine in situ radiometric measurements. Sensor-to-sensor comparisons are performed to derive gain factors (g1) from near-concurrent observations in TOA radiance and reflectance domains. The gains in the radiance domain were further validated/adjusted by determining a second set of gains (g2) via analysis of OLI-derived water-leaving radiance, i.e., L(sub w) (gamma), against in situ measurements made either at the Ocean Color AErosol RObotic NETwork (AERONETOC) sites or during field campaigns. The analyses yield the OLI calibration uncertainties that need to be accounted forwhen studying aquatic environments. Itwas found that, for the visible and near-infrared channels, the OLI radiometric responses, on average, are well in agreement (b 2 % discrepancies) with the TOA radiances estimated by ocean color satellites or those predicted by models based onmeasurements of aquatic and atmospheric properties. However, the TOA radiance at the new 443-nm band is found to be, on average, 3.4 % larger than the reference observations. The inter-sensor comparisons in the reflectance domain, however, indicated slightly different results with the OLI responses being low in the blue bands. To enhance the retrieval accuracy of aquatic-science products from OLI datasets, sets of temporally averaged gains (radiance and reflectance) are derived and recommended for use prior to the retrieval of in-water products.

Remote Sensing↗

Selected results from full-core hydrogen redistribution asymptotic analysis in YH-moderated heat-pipe cooled microreactor

Yttrium hydride is the main candidate for moderation of high temperature nuclear microreactors . This is due to its high thermal stability and relatively high hydrogen retention at temperatures exceeding 870 C. One of the main issues associated with the use of yttrium hydride is that, when exposed to temperature, stress, or concentration gradients, the hydrogen contained in the metallic matrix tends to redistribute and leak from the moderating elements, potentially leading to reactivity losses and power swings. This paper is focused on presenting selected results from the asymptotic hydrogen redistribution analysis performed on the Simplified Microreactor Benchmark Assessment (SiMBA) problem, a microreactor core conceptual design developed at INL. The analysis is based on full core analysis, relies uniquely on Bison to perform hydrogen redistribution calculations, and discusses the physical causes of the reactivity feedback. It was found that the feedback is negative, but it is one order of magnitude lower than the one found for the empire reactor unit-cell. This is due to the lower axial temperature gradient together with the effect of the reflector.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

GlideinBenchmark: collecting resource information to optimize provisioning

Choosing the right resource can speed up job completion, better utilize the available hardware, and visibly reduce costs, especially when renting computers in the cloud. This was demonstrated in earlier studies on HEPCloud. However, the benchmarking of the resources proved to be a laborious and time-consuming process. This paper presents GlideinBenchmark, a new Web application leveraging the pilot infrastructure of GlideinWMS to benchmark resources, and it shows how to use the data collected and published by GlideinBenchmark to automate the optimal selection of resources. An experiment can select the benchmark or the set of benchmarks that most closely evaluate the performance of its workflows. GlideinBenchmark, with the help of the GlideinWMS Factory, controls the benchmark execution. Finally, a scheduler like HEPCloud's Decision Engine can use the results to optimize resource provisioning.

Mambelli, Marco [Fermilab] (ORCID:0000000294892681↗

Effect of Computational Schemes on Coupled Flow and Geo-Mechanical Modeling of CO 2 Leakage through a Compromised Well

Carbon capture, utilization, and storage (CCUS) describes a set of technically viable processes to separate carbon dioxide (CO 2 ) from industrial byproduct streams and inject it into deep geologic formations for long-term storage. Legacy wells located within the spatial domain of new injection and production activities represent potential pathways for fluids (i.e., CO 2 and aqueous phase) to leak through compromised components (e.g., through fractures or micro-annulus pathways). The finite element (FE) method is a well-established numerical approach to simulate the coupling between multi-phase fluid flow and solid phase deformation interactions that occur in a compromised well system. We assumed the spatial domain consists of a three-phases system: a solid, liquid, and gas phase. For flow in the two fluids phases, we considered two sets of primary variables: the first considering capillary pressure and gas pressure (PP) scheme, and the second considering liquid pressure and gas saturation (PS) scheme. Fluid phases were coupled with the solid phase using the full coupling (i.e., monolithic coupling) and iterative coupling (i.e., sequential coupling) approaches. The challenge of achieving numerical stability in the coupled formulation in heterogeneous media was addressed using the mass lumping and the upwinding techniques. Numerical results were compared with three benchmark problems to assess the performance of coupled FE solutions: 1D Terzaghi’s consolidation, Liakopoulos experiments, and the Kueper and Frind experiments. We found good agreement between our results and the three benchmark problems. For the Kueper and Frind test, the PP scheme successfully captured the observed experimental response of the non-aqueous phase infiltration, in contrast to the PS scheme. These exercises demonstrate the importance of fluid phase primary variable selection for heterogeneous porous media. We then applied the developed model to the hypothetical case of leakage along a compromised well representing a heterogeneous media. Considering the mass lumping and the upwinding techniques, both the monotonic and the sequential coupling provided identical results, but mass lumping was needed to avoid numerical instabilities in the sequential coupling. Additionally, in the monolithic coupling, the magnitude of primary variables in the coupled solution without mass lumping and the upwinding is higher, which is essential for the risk-based analyses.

deformation flow↗

Benchmark Specifications for TREAT Tests M5, M6, and M7

Detailed information describing three TREAT tests performed on metallic fuels has been collected and organized for use as a benchmark in evaluating the performance of metallic fuel models and codes. The tests, designated M5, M6, and M7, subjected EBR-II-irradiated fuel pins to a single type of overpower transient (at prototypical conditions with full coolant flow and an exponential power rise on an 8 second period) in flowing sodium loops. Six fuel pins were tested; five were ternary (U-19Pu-10Zr) alloy fuel clad in D9 with burnup ranging from 0.8 to 9.8 at. %, and one was binary alloy fuel (U-10Zr) clad in HT9 with 2.9 at. % burnup. The information gathered from the test records is expected to be useful for pre-transient characterization of the irradiated fuel pins as well as the transient analysis of the metallic fuel when subjected to severe accident conditions. This report presents benchmark specifications for the M5, M6, and M7 TREAT tests, identifies where primary sources of benchmark-related information can be found, and includes background information to help a user of the data understand their applicability and limitations.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

A Benchmark Suite for Evaluating Scientific AI Workloads on GPUs

AI applications have been steadily increasing in the allocation portfolio among leadership computing facilities. These applications depend on deep learning frameworks with hardware acceleration and underlying software systems. With the rapid development of applications, software stacks, and hardware devices, it is essential to evaluate the performance of core operations in AI workloads for direction of optimizations and procurement of next-generation high-performance computing (HPC) infrastructures. Currently, most benchmarks lack scientific AI workloads. So, we present DeepKernelBench and the experimental results of evaluating the benchmark suite for early observations and performance comparisons on datacenter GPUs using representative workloads for scientific AI, including Attentions, General matrix multiplications, Geometrics and Fourier neural operations.

Jin, Zheming [Advanced Micro Devices (AMD)]↗

Fast-Spectrum Critical Assemblies with a Pb-HEU Core Surrounded by a Copper Reflector

The Department of Energy invests tens of millions of dollars each year to develop the next generation of nuclear engineering modeling & simulation (M&S) tools. These M&S tools are used to analyze advanced reactor designs and the safety of current nuclear operations. As computers become more powerful, we are able to enhance resolution in our calculations. This improved resolution is taking us to a point where the limitations of simulation capability are in the quality of data, including our ability to quantify the uncertainty and sensitivity of the data. In order to accurately model systems of interest, the industry must improve key nuclear data measurements and our confidence of how well we understand the data. Thus, M&S tools need evaluated and quality-assured experimental data for validation purposes. The International Criticality Safety Benchmark Evaluation Project (ICSBEP) compiles benchmark experiment data in a handbook that can be used by criticality safety engineers to validate computer codes and cross-section libraries at nuclear facilities. Both critical and subcritical experiments are included in the handbook. Figure 1 organizes all the benchmark evaluations that have been performed by the isotope of interest, in this case Pb, and the neutron energy within the system. Compared to other isotopes of interest for nuclear applications, there are few benchmark evaluations for Pb systems. This has caused the latest nuclear cross-section libraries to over/underestimate changes in the neutron population compared to experimental results. Therefore, this evaluation fills an important knowledge gap in benchmark evaluations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Status of HEU-Pb in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook

The Department of Energy invests tens of millions of dollars each year to develop the next generation of nuclear engineering modeling & simulation (M&S) tools. These tools are used to analyze advanced reactor designs and the safety of current nuclear operations. As computers become more powerful, we are able to enhance resolution in our calculations. This improved resolution is taking us to a point where the limitations of simulation capability are in the quality of data, including our ability to quantify the uncertainty and sensitivity of the data. In order to model systems of interest with increasing accuracy, the industry must improve key nuclear data measurements. Thus, M&S tools need evaluated and quality-assured experimental data for validation purposes. The International Criticality Safety Benchmark Evaluation Project (ICSBEP) compiles and evaluates experiment data in a handbook that can be used by criticality safety engineers and others to validate computer codes and cross-section libraries at nuclear facilities. Both critical and subcritical experiments are included in the handbook. Figure 1 organizes all the benchmark evaluations that have been performed by the isotope of interest, in this case Pb, and the average neutron energy the system. Compared to other isotopes of interest for nuclear applications, there are few benchmark evaluations for Pb systems. The lack of integral measurements to determine errors in Pb cross-section data has caused the latest nuclear cross-section libraries to over/underestimate k eff compared to experimental results. Therefore, this evaluation fills an important knowledge gap in benchmark evaluations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

LCIO

Performance of file systems shift during their life cycles. Evaluating this performance change over time is not trivial. Complexity arises in the interplay between external (i.e. application I/O workloads) and internal (i.e. the filesystem state) factors. Many benchmarks can test how a filesystem performs at the current snapshot state, but to observe the change over time necessitates that the filesystem state mutate (age) between benchmark runs. For a large-scale HPC parallel filesystem, the sheer scale and amount of interacting components during I/O operations magnify these challenges. LCIO addresses the question - how will the filesystem perform at different stages of its life cycle? LCIO is a synthetic benchmark, which provides the file system aging process to increase the amount of information that existing benchmarks like IOR and MDTest yield, as well as provide additional points of data that will be useful to system architects and engineers.

Bachstein, Matthew↗

Benchmarking large language models for materials synthesis: The case of atomic layer deposition

In this work, we introduce an open-ended question benchmark, ALDbench, to evaluate the performance of large language models (LLMs) in materials synthesis, and, in particular, in the field of atomic layer deposition, a thin film growth technique used in energy applications and microelectronics. Our benchmark comprises questions with a level of difficulty ranging from the graduate level to domain expert current with the state of the art in the field. Human experts reviewed the questions along the criteria of difficulty and specificity, and the model responses along four different criteria: overall quality, specificity, relevance, and accuracy. We ran this benchmark on an instance of OpenAI’s GPT-4o. The responses from the model received a composite quality score of 3.7 on a 1–5 scale, consistent with a passing grade. However, 36% of the questions received at least one below average score. An in-depth analysis of the responses identified at least five instances of suspected hallucination. Finally, we observed statistically significant correlations between the difficulty of the question and the quality of the response, the difficulty of the question and the relevance of the response, the specificity of the question, and the accuracy of the response as graded by the human experts. Furthermore, this emphasizes the need to evaluate LLMs across multiple criteria beyond difficulty or accuracy.

Artificial intelligence↗

A Comparison of GPU-Accelerated Multiphase CFD Solvers on the Polaris Supercomputer: Part 1

This report is in support of the Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program sponsored by the U.S. Department of Energy (USDOE). With INCITE-level resources, one project, titled BubblyFlow, was granted computational resources for the 2025 calendar year on the Polaris supercomputer at the Argonne Leadership Computing Facility (ALCF). The project aims to conduct simulations to understand the fundamental characteristics of turbulent bubbly flow phenomena in nature. Staff at the ALCF and Argonne’s Computational Science division, along with collaborators at the City College of New York and University of Illinois at Chicago, helped a summer student to assess the accuracy and performance of two high performance computing (HPC) codes. Both codes, ImExLBM and FluTAS, are fundamentally different in their mathematical and numerical modeling. However, both may be used to solve the same physical problem. The collaboration sought to better understand the differences between both codes in terms of accuracy and efficiency. This would ultimately help the BubblyFlow project better utilize resources and establish a knowledge-base of code capabilities in future simulation campaigns. We compare ImExLBM and FluTAS, two high-performance multiphase computational fluid dynamics (CFD) solvers, in terms of physical fidelity, time-to-solution, and parallel efficiency. We validate ImExLBM (Implicit-Explicit Lattice Boltzmann Method) against a canonical benchmark and assess it’s performance relative to FluTAS (Fluid Transport Accelerated Solver), a well-established open-source CFD code.

97 MATHEMATICS AND COMPUTING↗

Chimbuko: A Workflow-Level Scalable Performance Trace Analysis Tool

ABSTRACT Due to the sheer volume of data it is typically impractical to analyze the detailed performance of an HPC application running at-scale. While conventional small-scale benchmarking and scaling studies are often sufficient for simple applications, many modern workflow-based applications couple multiple elements with competing resource demands and complex inter-communication patterns for which performance cannot easily be studied in isolation and at small scale. This work discusses Chimbuko, a performance analysis framework that provides real-time, in situ anomaly detection. By focusing specifically on performance anomalies and their origin (aka provenance), data volumes are dramatically reduced without losing necessary details. To the best of our knowledge, Chimbuko is the first online, distributed, and scalable workflow-level performance trace analysis framework. We demonstrate the tool's usefulness on Oak Ridge National Laboratory's Summit system.

97 MATHEMATICS AND COMPUTING↗

All-electron APW+ lo calculation of magnetic molecules with the SIRIUS domain-specific package

We report APW+lo (augmented plane wave plus local orbital) density functional theory (DFT) calculations of large molecular systems using the domain specific SIRIUS multi-functional DFT package. The APW and FLAPW (full potential linearized APW) task and data parallelism options and the advanced eigen-system solver provided by SIRIUS can be exploited for performance gains in ground state Kohn–Sham calculations on large systems. This approach is distinct from our prior use of SIRIUS as a library backend to another APW+lo or FLAPW code. We benchmark the code and demonstrate performance on several magnetic molecule and metal organic framework systems. We show that the SIRIUS package in itself is capable of handling systems as large as a several hundred atoms in the unit cell without having to make technical choices that result in the loss of accuracy with respect to that needed for the study of magnetic systems.

Chemistry↗

Transient MOC with frequency transform and DSA on unstructured mesh

We present an implementation of the transient method of characteristics (MOC) with isotropic time derivatives, accelerated by diffusion synthetic acceleration (DSA). The fully implicit frequency transform method is used to solve the transient problem with analytic precursor integration. The code works on meshes composed of almost any of the commonly used non-curvilinear finite element types, and can handle the deformation of geometry in time-dependent transport calculations. We present results of a continuous Fourier analysis for the transient multigroup DSA problem, and representative benchmarking results are presented for the C5G7-TD benchmark in 2D showing reasonable performance and agreement compared to other codes. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

GlideinBenchmark: collecting resource information to optimize provisioning

Choosing the right resource can speedup jobs completion, better utilize the available hardware and visibly reduce costs, especially when renting computers on the cloud. This was demonstrated in earlier studies on HEPCloud. But the benchmarking of the resources proved to be a laborious and time-consuming process. This paper presents GlideinBenchmark, a new Web application leveraging the pilot infrastructure of GlideinWMS to benchmark resources, and shows how to use the data collected and published by GlideinBenchmark to automate the optimal selection of resources. An experiment can select the benchmark or the set of benchmarks that most closely evaluate the performance of its workflows. With GlideinBenchmark and the help of the GldieinWMS Factory it controls the benchmark execution. Finally, a scheduler like HEPCloud’s Decision Engine can use the results to optimize resource provisioning.

Mambelli, Marco↗