Search NASA⌕ Search

SEARCH · Search NASA

Results for “benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Current Status of Problem 2 in the HTGR T/H Benchmark

Problem 2 of the HTGR T/H Benchmark is for modeling the depressurized conduction cooldown (DCC) transient. This presentation shares the status of code-to-code and code-to-date comparisons (Exercises 1 and 2) of Problem 2 with results from several participants. A key factor in the discrepancy between results and data in Exercise 2 is the assumed power distribution that is used to model PG-29. This talk highlights both the similarities and differences in solutions and discusses areas where further investigation is merited.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations

Software developers must adapt to keep up with the changing capabilities of platforms so that they can utilize the power of High-Performance Computers (HPC), including exascale systems. OpenMP, a directive-based parallel programming model, allows developers to include directives to existing C, C++, or Fortran code to allow node level parallelism without compromising performance. This paper describes our CI/CD efforts to provide easy evaluation of the support of OpenMP across different compilers using existing testsuites and benchmark suites on HPC platforms. Our main contributions include (1) the set of a Continuous Integration (CI) and Continuous Development (CD) workflow that captures bugs and provides faster feedback to compiler developers, (2) an evaluation of OpenMP (offloading) implementations supported by AMD, HPE, GNU, LLVM, and Intel, and (3) evaluation of the quality of compilers across different heterogeneous HPC platforms. With the comprehensive testing through the CI/CD workflow, we aim to provide a comprehensive understanding of the current state of OpenMP (offloading) support in different compilers and heterogeneous platforms consisting of CPUs and GPUs from NVIDIA, AMD, and Intel.

Jarmusch, Aaron↗

The Surface-Topography Challenge: A Multi-Laboratory Benchmark Study to Advance the Characterization of Topography

Surface performance is critically influenced by topography in virtually all real-world applications. The current standard practice is to describe topography using one of a few industry-standard parameters. The most commonly reported number is Ra, the average absolute deviation of the height from the mean line (at some, not necessarily known or specified, lateral length scale). However, other parameters, particularly those that are scale-dependent, influence surface and interfacial properties; for example the local surface slope is critical for visual appearance, friction, and wear. The present Surface-Topography Challenge was launched to raise awareness for the need of a multi-scale description, but also to assess the reliability of different metrology techniques. In the resulting international collaborative effort, 153 scientists and engineers from 64 research groups and companies across 20 countries characterized statistically equivalent samples from two different surfaces: a “rough” and a “smooth” surface. The results of the 2088 measurements constitute the most comprehensive surface description ever compiled. We find wide disagreement across measurements and techniques when the lateral scale of the measurement is ignored. Consensus is established through scale-dependent parameters while removing data that violates an established resolution criterion and deviates from the majority measurements at each length scale. Our findings suggest best practices for characterizing and specifying topography. The public release of the accumulated data and presented analyses enables global reuse for further scientific investigation and benchmarking.

42 ENGINEERING↗

Validation Data for Benchmarking Wire Arc Additive Manufacturing Process Simulations

Residual stresses cause geometric distortion and affect mechanical performance of additively manufactured structures, yet they are notoriously difficult to assess and predict. Distortion (warpage) can drive parts outside dimensional tolerance limits, leading to part rejection or rework. For parts that meet tolerance, locked-in residual stress fields can affect structural integrity during operation, particularly subcritical cracking by fatigue, creep, or corrosion. This work develops benchmark data for a common additive manufacturing process (Wire Arc Additive Manufacturing) that can be applied for calibration and validation of physical process models that predict residual stress fields. The work includes design of two different samples of differing geometry, detailed manufacturing records for a set of physical samples, and an extensive set of residual stress measurement data developed using two diverse techniques (the contour method and neutron diffraction). An initial application of the work is also reported, where a modeling challenge was issued to secure residual stress model predictions from two independent laboratories that were blind to residual stress measurement data. These initial blind residual stress predictions show significant discrepancies relative to the measurement data, illustrating the potential value of the underlying validation data. An open repository for this work, including the sample designs, manufacturing process records, and the residual stress data, is also provided for future application in non-blind validation efforts.

36 MATERIALS SCIENCE↗

Benchmark microgravity experiments and computations for 3D dendritic-array stability in directional solidification

In this study, we present a comprehensive quantitative analysis of stability bands for dendritic arrays during directional solidification of a transparent succinonitrile-0.46 wt % camphor alloy, spanning a broad range of pulling velocities. Taking advantage of the microgravity environment aboard the International Space Station where most convection effects are suppressed, we obtain unique measurements that quantify the stable primary spacing range of spatially extended three-dimensional dendritic array structures under purely diffusive growth conditions. Through carefully designed velocity jump experiments and detailed examination of sub-grain boundary dynamics, we characterize key instabilities, including elimination and tertiary branching, shedding new light on the mechanisms governing dynamic dendritic spacing selection in extended 3D arrays. Phase field simulations are performed to characterize the stability limits of dendritic array structures for quantitative comparison with the flight experiments. Although the simulations capture general trends, significant deviations are noted at the upper stability boundary, indicating the influence of additional, unexplored factors. These findings contribute to a deeper understanding of dendritic growth dynamics and offer valuable benchmark data that could aid in refining predictive models and improving control of dendritic microstructures in metallurgical applications.

36 MATERIALS SCIENCE↗

Benchmark of numerical modeling approaches on the systematic performance evaluation of wave energy converters

Different numerical modeling methods have been developed and applied to evaluate a variety of performance indicators of wave energy converters (WECs), including the power performance, structural loads, levelized cost of energy, etc. Based on the modeling fidelity, the commonly used numerical modeling approaches can be classified as linear modeling, weakly nonlinear modeling and fully nonlinear modeling approaches. Each method differs in accuracy and computational efficiency, making them suitable for different stages of WEC design. However, the selection of modeling approach could significantly impact evaluation outcomes. For instance, simplified linear models may underestimate structural loads or overestimate energy production in some operational conditions, potentially leading to less cost-effective designs. Given the widespread utilization of these models, it is essential to understand the uncertainties brought by them in performance evaluations. This work is dedicated to benchmarking different linear-potential-flow-based numerical models for evaluating the systematic performance of WECs. Three representative numerical modeling approaches are considered in this work, including linear frequency-domain modeling, statistically linearized spectral-domain modeling and Cummins equation-based nonlinear time-domain modeling. A generic point absorber WEC is considered as the research reference in this work, and different sea sites are taken into account. The numerical models are utilized to predict critical performance indicators, including power performance, the annual energy production, the capacity factor, the levelized cost of energy and the PTO fatigue loads. By comparing the results, this work identifies the uncertainties associated with different modeling approaches in evaluating WEC performance.

Fatigue↗

Benchmarking image processing techniques for porosity measurement in polymer additive manufacturing: Review and experimental analysis

An image processing workflow is proposed for porosity measurement in polymer additive manufacturing. Various techniques, including global and local thresholding, region growing, and K-means clustering, were applied to microscopic images of carbon fiber reinforced acrylonitrile butadiene styrene (CF-ABS) and benchmarked for their ability to accurately measure porosity. Global methods included Otsu, minimum error, iterative, and entropy-based thresholding, while local methods included Niblack, Bernsen, Sauvola, and Bradley-Roth algorithms. Artificial uneven illumination was introduced to test local adaptive thresholds. Results showed significant differences in porosity values across methods. Otsu, region growing, and K-means clustering excelled under uniform illumination, while Sauvola and Bradley-Roth performed better with uneven illumination. Comparison with X-ray computed tomography (XCT) revealed slightly lower porosity values (2.55 %) than optimized methods (2.73–2.79 %) due to XCT's lower resolution excluding smaller pores. While XCT offers finer pore detection, it limits sample volume and underestimates porosity due to spatial variation. Validation using artificial grayscale images with 5 % porosity confirmed that Otsu, Bradley-Roth, region growing, and Sauvola algorithms produced accurate results. Although tested on a single material system, these methods can be adapted to others with optimization. In conclusion, given XCT's high computational and time costs, this study highlights suitable image processing techniques as cost-effective alternatives for porosity analysis in polymer composites.

Additive manufacturing↗

Physics vs structure: A systematic benchmark of learning strategies for multi-zone building thermal dynamics

Recent advances in physics-informed and data-driven machine learning promise improved thermal models for advanced building control, yet there is limited quantitative evidence on when added physics structure and architectural complexity are beneficial. Here, this work presents a systematic benchmark of five representative system identification methods for modeling multi-zone building thermal dynamics: linear state-space models, multi-layer perceptrons, neural state-space models, neural ordinary differential equations, and physically-consistent neural networks. The methods are evaluated across multiple data regimes and zone coupling strategies. Using a high-fidelity multi-zone commercial building emulator, we examine short-term and long-term prediction accuracy, computational efficiency, and ease of development. Our results reveal critical trade-offs between prediction performance, model complexity, and physical consistency. We demonstrate that decoupled, nonlinear black-box models consistently outperform coupled physics-constrained architectures in both predictive accuracy and out-of-distribution robustness in majority of the test cases for the building type considered in the study. Our findings quantify the cost of complexity in building thermal modeling and provide concrete, actionable, scenario-based guidelines for selecting model classes for control-oriented applications.

Building thermal modeling↗

Benchmarking CO₂ storage simulations: Results from the 11 th Society of Petroleum Engineers Comparative Solution Project

The 11 th Society of Petroleum Engineers Comparative Solution Project (shortened SPE11 herein) benchmarked simulation tools for geological carbon dioxide (CO 2 ) storage. A total of 45 groups from leading research institutions and industry across the globe signed up to participate, with 18 ultimately contributing valid results that were included in the comparative study reported here. This paper summarizes the SPE11 results. A comprehensive introduction and qualitative discussion of the submitted data are provided, together with an overview of online resources for accessing the full depth of data. A global metric for analyzing the relative distance between submissions is proposed and used to conduct a quantitative analysis of the submissions. This analysis attempts to statistically resolve the key aspects influencing the variability between submissions. The study shows that the major qualitative variation between the submitted results is related to thermal effects, dissolution-driven convective mixing, and resolution of facies discontinuities. Moreover, a strong dependence on grid resolution is observed across all three versions of the SPE11. However, our quantitative analysis suggests that the observed variations are predominantly influenced by factors not documented in the technical responses provided by the participants. We therefore identify that unreported variations due to human choices within the process of setting up, conducting, and reporting on the simulations underlying each SPE11 submission are at least as impactful as the computational choices reported.

Nordbotten, Jan M. [Univ. of Bergen (Norway); Norw↗

Uncertainty quantification and sensitivity analysis for SPERT III E-core reactivity measurement benchmarking

The Special Power Excursion Reactor Test (SPERT) III E-core experiment is important because it provides critical data on reactor behavior under significant reactivity insertions, which is essential for validating computer simulations and ensuring the safety of modern light water reactors. Its design similarities to contemporary reactors make it a valuable resource for understanding and mitigating extreme hazards in nuclear operations. The current study details the application of formal parametric uncertainty quantification and sensitivity analysis to a model of the SPERT-III E-core model for zero power reactivity benchmarking. Additionally, the reactivity impact from various modeling assumptions is quantified. Overall, an conservative estimate for an uncertainty in k$_{\text{eff}}$ of $\pm$1257 was observed. A less conservative, more realistic, uncertainty estimate of $\pm$1096 pcm can be justified by the potential for various parametric uncertainties to become negligible when sampled independently across the ~1400 pins in the core. The experimental results fall within both of these uncertainty bounds. Standardized regression coefficient as well as Sobol indices are used to identify the guide tube thicknesses as the primary contributors to the uncertainty in k$_{\text{eff}}$. Overall, this study provides information on how the uncertainties in input parameters and modeling methods impact simulated k$_{\text{eff}}$ values and can be used to aid model building efforts for future code validation with the SPERT-III E-core experiment.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

Benchmarking fast-neutron response of 7 Li-enriched C 7 LYC scintillators: Simulations and experiment

New insights into the fast neutron response of 7 Li-enriched C 7 LYC scintillators have been obtained through detailed Geant4 simulations of 35 Cl(n,p) and 35 Cl(n, α) reactions that are primarily responsible for the fast neutron response of C 7 LYC. The light output of a 1” x1” C 7 LYC detector was simulated using the Birks equation for light quenching in scintillators. The simulations were benchmarked against neutron scattering data from a 12 C target obtained with a white neutron beam at the Los Alamos LANSCE facility and fourteen 1”x1” C 7 LYC detectors. Incident neutron energies extracted from time-of-flight measurements, and appropriate recoil corrections for the different detector angles were used to map the response of C 7 LYC to mono-energetic neutrons. Resonances in the 35 Cl(n,p) reaction cross-section below 3 MeV incident neutron energy were directly observed in C 7 LYC. The experimental data and simulations are compared with ENDF databases, as well as recent direct cross-section measurements of neutrons scattered from a 35 Cl target. In conclusion, the results suggest a scrutiny of the 35 Cl(n,p) and 35 Cl(n, α) cross sections at low energy in the ENDF library.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Importance of Electron Correlation on the Geometry and Electronic Structure of [2Fe–2S] Systems: A Benchmark Study of the [Fe 2 S 2 (SCH 3 ) 4 ] 2–,3–,4– , [Fe 2 S 2 (SCys) 4 ] 2– , [Fe 2 S 2 (S- p -tol) 4 ] 2– , and [Fe 2 S 2 (S- o -xyl) 4 ] 2– Complexes

Iron–sulfur clusters are crucial for biological electron transport and catalysis. Obtaining accurate geometries, energetics, manifolds of their excited electronic states, and reduction energies is important to understand their role in these processes. Here, using a [2Fe–2S] model complex with Fe II and Fe III oxidation states, which leads to different charges, i.e., [Fe 2 S 2 (SMe) 4 ] 2–,3–,4– , we benchmarked a variety of computational methodologies ranging from density functional theory (DFT) to post-Hartree–Fock methods, including complete active space self-consistent field (CASSCF), multireference configuration interaction, the second-order N-electron valence state perturbation theory (NEVPT2), and the linearized integrand approximation of adiabatic connection (AC0) approaches. Additionally, we studied three experimentally well-characterized complexes, [Fe 2 S 2 (SCys) 4 ] 2– , [Fe 2 S 2 (S-o-tol) 4 ] 2– , and [Fe 2 S 2 (S-o-xyl) 4 ] 2– , via DFT methods. We conclude that the dynamic electron correlation is important for accurately predicting the geometry of these complexes. Broken symmetry (BS) DFT correctly predicts experimental geometries of low-spin multiplicity, while CASSCF does not. However, BS-DFT significantly overestimates the difference between the low- and high-spin electronic states for a given oxidation state. At the same time, CASSCF underestimates it but provides relative energies closer to the reference NEVPT2 results. Finally, AC0 provides energetics of NEVPT2 quality with the additional advantage of being able to use large CASSCF sizes. NEVPT2 gives the best estimates of the Fe III /Fe III → Fe II /Fe III (4.27 eV) and Fe II /F III → Fe II /F II (7.72 eV) reduction energies. The results provide insight into the electronic structure of these complexes and assist in the understanding of their physical properties.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Quantum Simulation of Molecular Dynamics Processes─A Benchmark Study Using a Classical Simulator and Present-Day Quantum Hardware

Here, we explore how the fundamental problems in quantum molecular dynamics can be modeled using classical simulators (emulators) of quantum computers and the actual quantum hardware available to us today. The list of problems we tackle includes propagation of a free wave packet, vibration of a harmonic oscillator, and tunneling through a barrier. Each of these problems starts with the initial wave packet setup. Although Qiskit provides a general method for initializing wave functions, in most cases it generates deep quantum circuits. While these circuits perform well on noiseless simulators, they suffer from excessive noise on quantum hardware. To overcome this issue, we designed a shallower quantum circuit for preparing a Gaussian-like initial wave packet, which improves the performance of real hardware. Next, quantum circuits are implemented to apply the kinetic and potential energy operators for the evolution of a wave function over time. The results of our modeling on classical emulators of quantum hardware agree perfectly with the results obtained using the traditional (classical) methods. This serves as a benchmark and demonstrates that the quantum algorithms and Qiskit codes we developed are accurate. However, the results obtained on the actual quantum hardware available today, such as IBM’s superconducting qubits and IonQ’s trapped ions, indicate large discrepancies due to hardware limitations. This work highlights both the potential and challenges of using quantum computers to solve fundamental quantum molecular dynamics problems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Benchmarking DFT Accuracy in Predicting O 1s Binding Energies on Metals

X-ray photoelectron spectroscopy (XPS) is a powerful tool for probing the electronic structure and composition of materials, particularly metals and metal oxides of relevance to solar cells and catalysis. Density functional theory (DFT) is often used to support XPS peak assignments, but its reliability for predicting oxygen species is not well established. Here, we compile a large data set of experimental oxygen binding energies and evaluate corresponding DFT predictions. We find that as the binding energies of metal-bound atomic oxygen species increase, especially above ≈530 eV, there is a general decrease in the accuracy of DFTpredicted values. Thus, high-binding-energy atomic oxygen species, such as those proposed as active for selective Ag-catalyzed epoxidation, are less well represented. The chemical nature of the oxygen species also influences accuracy, with molecularly bound species more reliably captured across the entire range of energies. These findings illustrate the limitations of DFT for interpreting XPS spectra and provide a benchmark for improving computational methods.

Adsorption↗

Quantum Monte Carlo Benchmarking of Molecular Adsorption on Graphene-Supported Single Pt Atom

The precise understanding of adsorption energetics and molecular geometry at catalytic sites is fundamental for advancing catalysis, particularly under the constraints of resource efficiency and environmental sustainability. Here, this study benchmarks the performance of density functional theory (DFT) calculations against diffusion Monte Carlo (DMC) calculations for adsorption properties of small gas molecules relevant to CO oxidation—namely O 2 , CO, CO 2 , and atomic oxygen—on a single Pt atom supported by pristine graphene. Our findings reveal that DMC calculations provide a significantly different landscape of adsorption energetics compared to DFT results. Notably, DFT predicts different lowest-energy configurations and spin states, particularly for O 2 , which suggests potential discrepancies in predicting the catalytic behavior. Furthermore, this study identifies the critical issue of CO poisoning, highlighted by the large disparity between the DMC adsorption energies of O 2 (−1.23(2) eV) and CO (−3.37(1) eV), which can inhibit the catalytic process. These results emphasize the necessity for more sophisticated computational approaches in catalysis research, aiming to refine the prediction accuracy of reaction mechanisms and to enhance the design of more effective catalysts.

Ahn, Jeonghwan [University of Illinois at Urbana-C↗

Benchmarking Density Functional Theory Methods for Efficient Calculations of a Strongly Correlated Li 1– x Ni 1– y O 2−δ System

Transition metal oxides (TMOs), such as LiNiO 2 , are promising candidates for energy storage and electronic devices due to their unique electronic properties, exceptional physical and chemical characteristics, and ability to adopt multiple oxidation states. However, accurately predicting their properties using mean-field density functional theory (DFT) is challenging due to the presence of strongly correlated d-electrons and the complex interplay between their structural, electronic, and magnetic responses. These challenges are further exacerbated by the need to model defects, surfaces, and interfaces, which require computationally efficient, large-scale simulations. To address these issues, we carry out a benchmark study on the Li 1–x NiO 2 system, evaluating the performance of several popular functionals. Our findings demonstrate that combining SCAN functional relaxation with single-step HSE calculations provides a practical and scalable computational strategy. This approach balances accuracy and efficiency, enabling high-throughput simulations of strongly correlated TMOs and improved predictive modeling capability of TMOs for practical applications.

25 ENERGY STORAGE↗

Rates of Sea‐Level Rise Are Highly Sensitive to Ice Viscosity Parameters in Model Benchmarks

Glacier flow plays a major role in current and future rates of globally averaged sea-level rise. The viscosity of glacial ice, controlling the rate of flow, decreases as stress increases and is highly sensitive to the value of the stress exponent, $n$, in the constitutive equation for viscous flow. Glaciologists and climate modelers almost exclusively assume $n=3$ when modeling ice flow and projecting sea-level rise through forward modeling. However, recent work suggests that $n\approx 4$ better fits observations, prompting the question: How sensitive are projections of sea-level rise to the value of $n$? We use an established community ice flow model and standard benchmark experiments designed as an idealized representation of Pine Island Glacier, West Antarctica. While initializing an $n=3$ model to match observations of an $n=4$ ice sheet is possible, we find that incorrectly assuming $n=3$ when in fact $n=4$ dramatically underestimates rates of sea-level rise. The scale of this error grows nonlinearly with the magnitude of the climate forcing, acting to increase projection uncertainties. Additionally, we find that models often account for this stress-dependent rheology mismatch during model initialization in a way that masks this rheological effect in the short term while leaving model outputs vulnerable to larger biases in longer-term projections. Initializations to observations of Pine Island Glacier display similar rheology-mismatch fingerprints to our idealized example.

climate sensitivity↗