Search NASA⌕ Search

SEARCH · Search NASA

Results for “model benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

RELAP5-3D Modeling of the OECD-NEA HTTF Benchmark

HTTF at Oregon State University (OSU) Reference: General Atomics’ modular high-temperature gas-cooled reactor Helium cooled, electrically heated (2.2 MW) Prismatic graphite blocks in the core and reflectors Alumina ceramic blocks are used to simulate the core and top and bottom reflectors One-fourth scale in length and diameter Most of the coolant channels in the core are full scale Lower pressure compared to the prototype reactor (0.7 MPa) Over 500 instruments Designed primarily to investigate depressurized (DCC) and pressurized (PCC) conduction cooldown transients

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

Outcomes and Conclusions from the 2022 AM Bench Measurements, Challenge Problems, Modeling Submissions, and Conference

The Additive Manufacturing Benchmark Test Series (AM Bench) provides rigorous measurement data for validating additive manufacturing (AM) simulations for a broad range of AM technologies and material systems. AM Bench includes extensive in situ and ex situ measurements, simulation challenges for the AM modeling community, and a corresponding conference series. In 2022, the second round of AM Bench measurements, challenge problems, and conference were completed, focusing primarily upon laser powder bed fusion (LPBF) processing of metals, and both material extrusion processing and vat photopolymerization of polymers. In all, more than 100 people from 10 National Institute of Standards and Technology (NIST) divisions and 21 additional organizations were directly involved in the AM Bench 2022 measurements, data management, and conference organization. The international AM community submitted 138 sets of blind modeling simulations for comparison with the in situ and ex situ measurements, up from 46 submissions for the first round of AM Bench in 2018. Analysis of these submissions provides valuable insight into current AM modeling capabilities. The AM Bench data are permanently archived and freely accessible online. The AM Bench conference also hosted an embedded workshop on qualification and certification of AM materials and components.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Leveraging Prior Concept Learning Improves Generalization From Few Examples in Computational Models of Human Object Recognition

Humans quickly and accurately learn new visual concepts from sparse data, sometimes just a single example. The impressive performance of artificial neural networks which hierarchically pool afferents across scales and positions suggests that the hierarchical organization of the human visual system is critical to its accuracy. These approaches, however, require magnitudes of order more examples than human learners. We used a benchmark deep learning model to show that the hierarchy can also be leveraged to vastly improve the speed of learning. We specifically show how previously learned but broadly tuned conceptual representations can be used to learn visual concepts from as few as two positive examples; reusing visual representations from earlier in the visual hierarchy, as in prior approaches, requires significantly more examples to perform comparably. These results suggest techniques for learning even more efficiently and provide a biologically plausible way to learn new visual concepts from few examples.

Rule, Joshua S.↗
DOE OSTI2022

Whole core analysis of OECD/NEA TVA-WB1 benchmark exercise 1 using continuous-energy Monte Carlo code MVP with JENDL-4.0u and ENDF/B-VII.0

The TVA WB1 benchmark problem was precisely modeled in accordance with the geometry and material specifications, and benchmark exercise 1 was solved with the MVP code and the nuclear data libraries based on JENDL-4.0u (JENDL-4.0 updated file) and ENDF/B-VII.0. The criticality, the control rod bank worth, the isothermal temperature coefficient (ITC), the integral rod worth of bank D, and differential soluble boron worth (DBW) were obtained. The criticality and the control rod bank worth were compared with the measured data and published calculation results with other codes, Serpent and KENO. The criticality and the control rod bank worth calculated by MVP and the libraries agree with the measured and the published calculation results within the criteria specified in the ANSI/ANS standard. From these results, the calculation model for the MVP and the libraries was verified. The other parameters such as ITC, the integral rod worth, and DBW were also compared between JENDL-4.0u and ENDF/B-VII.0. The comparison of the results shows JENDL-4.0u and ENDF/B-VII.0 give comparable results within the standard deviation, the difference between libraries is small. (author)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2025

Review of Grey Box/Black Box Data Contamination Metrics on Open and Commercial Models

Dataset contamination is a problem where benchmarks and tasks used to evaluate the capabilities of Large Language Models (LLMs) have been incorporated into the training dataset of the models. This gives a false sense of performance that can overestimate how these models will function on truly unseen data. This problem becomes worse with commercial LLMs with larger and non-accessible training data, so techniques have been developed to try to measure the degree to which a model is contaminated with a benchmark’s data. To understand the effectiveness of these techniques, particularly when evaluating contamination on coding tasks, we review trends and categorize techniques by the degree of access to the model that is required. The research literature on this topic has reported mixed effectiveness of these techniques, so we select a set of black box (text access only) and grey box (access to model loss/probabilities required) techniques and apply them to both commercial and non-commercial models. We implement these metrics as part of a framework to test the contamination of Python code in LLMs to see to what extent we can replicate the effectiveness (or ineffectiveness) of these contamination detection techniques. Though we find mixed results in the capabilities of these metrics to identify contamination, we do observe evidence that they can identify contamination (broadly) in fine-tuned models when both a baseline and fine-tuned model is present. Additionally, similarity metrics were able to identify between contaminated and uncontaminated data even in situations where the data is distributionally similar (e.g., drawn from the same set of code projects).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Current Status of Problem 2 in the HTGR T/H Benchmark

Problem 2 of the HTGR T/H Benchmark is for modeling the depressurized conduction cooldown (DCC) transient. This presentation shares the status of code-to-code and code-to-date comparisons (Exercises 1 and 2) of Problem 2 with results from several participants. A key factor in the discrepancy between results and data in Exercise 2 is the assumed power distribution that is used to model PG-29. This talk highlights both the similarities and differences in solutions and discusses areas where further investigation is merited.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions

The National Ecological Observatory Network (NEON) is a continental-scale observatory with sites across the US collecting standardized ecological observations that will operate for multiple decades. To maximize the utility of NEON data, we envision edge computing systems that gather, calibrate, aggregate, and ingest measurements in an integrated fashion. Edge systems will employ machine learning methods to cross-calibrate, gap-fill and provision data in near-real time to the NEON Data Portal and to High Performance Computing (HPC) systems, running ensembles of Earth system models (ESMs) that assimilate the data. For the first time gridded EC data products and response functions promise to offset pervasive observational biases through evaluating, benchmarking, optimizing parameters, and training new machine learning parameterizations within ESMs all at the same model-grid scale. Leveraging open-source software for EC data analysis, we are already building software infrastructure for integration of near-real time data streams into the International Land Model Benchmarking (ILAMB) package for use by the wider research community. We will present a perspective on the design and integration of end-to-end infrastructure for data acquisition, edge computing, HPC simulation, analysis, and validation, where Artificial Intelligence (AI) approaches are used throughout the distributed workflow to improve accuracy and computational performance.

Durden, David J.↗
DOE OSTI2026

Search for dark matter production in association with bottom quarks and a lepton pair in proton-proton collisions at $\sqrt{s}=13$ TeV

A search is performed for dark matter produced in association with bottom quarks and a pair of electrons or muons in data collected with the CMS detector at the LHC, corresponding to 138 fb −1 of integrated luminosity of proton-proton collisions at a center-of-mass energy of 13 TeV. For the first time at the LHC, the associated production of a bottom quark-antiquark pair and a new heavy neutral Higgs boson (H) that subsequently decays into a leptonically decaying Z boson and a pseudoscalar (a) is explored. The latter acts as a dark matter mediator in the context of the two Higgs doublet model plus a pseudoscalar (2HDM+a). Multivariate techniques that target a wide range of mass configurations for the H and a particles are used. The observations are consistent with the expectations from standard model processes. Upper limits at 95% confidence level are set on the product of cross section and branching fraction of the new particles, ranging from 10 −2 pb for an H mass of 400 GeV to 10 −3 pb for an H mass of 2000 GeV. Constraints on the parameter space of a benchmark 2HDM+a model are derived and compared with expectations in the context of cosmological predictions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Surrogate modeling of the leaching process in a rare earth elements recovery plant

The data are summarized in the slides for a poster as presented at the 2024 AIChE annual meeting in San Diego, California, USA. The slides summarize our work on the PrOMMiS project over the past year. The slides describe the systematic design and optimization of a flowsheet for the recovery of critical minerals and rare earth elements. Then, it summarizes the results of our benchmarking of surrogate modeling methodologies for the optimization of these flowsheets.

Fardis, Dimitrios↗
DOE OSTI2024

An international benchmark for wind plant wakes from the American WAKE ExperimeNt (AWAKEN)

This article introduces the first benchmark study within the International Energy Agency Wind Task 57 framework, focusing on wind plant wakes. Leveraging data from the American WAKE ExperimeNt (AWAKEN), the benchmark aims to assess the accuracy of simulation tools in modeling wind plant wakes and their impact on the downstream flow under diverse inflow conditions. The AWAKEN field campaign, conducted in Oklahoma from 2022 to 2024, provides unprecedented observations of wind plant-atmosphere interactions, thus offering a large dataset to validate numerical models of different complexity. The benchmark will include three phases—code calibration, blind comparison, and iteration—allowing participants to refine their numerical models based on the feedback from the benchmark team. This article describes the benchmark case study selected from observations providing details on atmospheric conditions, wake evidence, and wind turbine operation. The benchmark’s structure and timeline, along with the expected publication of results, are discussed as well. This collaborative effort aims to enhance the accuracy of wind plant wake simulations, thus contributing to the improvement of wind energy production estimates.

17 WIND ENERGY↗
DOE OSTI2024

Models implemented in the methodological approach to design the initial STEP first wall contour

The official Spherical Tokamak for Energy Production mission aims to demonstrate the ability to generate net electricity from fusion with the STEP Prototype Power plant. One of the key technological and engineering challenges in fusion power plants is managing the loads on the first wall within acceptable limits. Therefore, the conceptual design development of the STEP Prototype Power plant needs to be based on load estimates derived using legitimate plasma physics assumptions through dynamic and flexible tools. The current design foresees the STEP main chamber first wall to withstand steady-state heat loads of up to ~1 MW/m 2 , excluding critical regions expected to receive higher heat loads such as the baffle regions approaching the divertors. These critical areas will require ad hoc assessments and will be designed with the presence of limiters. This article focuses on the models and methodology adopted for designing the 2-D poloidal contour of the STEP first wall, based on the anticipated charged particle and radiation heat loads during normal operation. Firstly, the models adopted for calculating the charged particle and radiation heat loads are introduced. The first model is validated through benchmarking against the particle tracing code SMARDDA, while the second model is verified by comparing it with data from the MAST-U experiment. Secondly, the model used to design the 2-D first wall contour according to the heat loads is explained. We acknowledge that this preliminary design stage assumes certain simplifications, notably an axisymmetric geometry, for computational efficiency and clarity in presentation. It is understood that subsequent design phases will address the complexities of real-world engineering, including non-axisymmetric effects, transient plasma scenarios, and the impact of disruptions on the first wall design. Finally, an automatic procedure based on these models is presented for defining the 2-D poloidal contour of the STEP first wall to minimize heat loads, taking into account the need to radiate most of the alpha-particle and auxiliary heating power. Here, by providing an overview of the models, methodology, and an automatic procedure, this paper contributes to the design process of the STEP first wall, addressing the engineering challenges associated with fusion power plant development.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

End-to-end AI framework for interpretable prediction of molecular and crystal properties

We introduce an end-to-end computational framework that allows for hyperparameter optimization using the DeepHyper library, accelerated model training, and interpretable AI inference. The framework is based on state-of-the-art AI models including CGCNN, PhysNet, SchNet, MPNN, MPNN-transformer, and TorchMD-NET. We employ these AI models along with the benchmark QM9, hMOF, and MD17 datasets to showcase how the models can predict user-specified material properties within modern computing environments. We demonstrate transferable applications in the modeling of small molecules, inorganic crystals and nanoporous metal organic frameworks with a unified, standalone framework. We have deployed and tested this framework in the ThetaGPU supercomputer at the Argonne Leadership Computing Facility, and in the Delta supercomputer at the National Center for Supercomputing Applications to provide researchers with modern tools to conduct accelerated AI-driven discovery in leadership-class computing environments. We release these digital assets as open source scientific software in GitLab, and ready-to-use Jupyter notebooks in Google Colab.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Evaluating integration and performance of containerized climate applications on a Hewlett Packard Enterprise Cray system

Containers have taken over large swaths of cloud computing as the most convenient way of packaging and deploying applications. The features that containers offer for packaging and deploying applications translate to high performance computing (HPC) as well. At The National Oceanic and Atmospheric Administration, containers provide an easy way to build and distribute complex HPC applications, allowing faster collaboration, portability, and experiment computer environment reproducibility amongst the scientific community. The challenge arises when applications rely on message passing interface (MPI). This necessitates investigation into how to properly run these applications with their own unique requirements and produce performance on par with native runs. We investigate the MPI performance for benchmarks and containerized climate models for various containers covering selection of compiler and MPI library combinations from the Cray provided programming environments on the Cray XC supercomputer GAEA. Performance from the benchmarks and the climate models shows that for the most part containerized applications perform on par with the natively built applications when the system optimized Cray MPICH libraries are bound into the container, and the hybrid model containers have poor performance in comparison. We also describe several challenges and our solutions in running these containers, particularly challenges with heterogeneous jobs for the containerized model runs.

Abraham, Subil↗
DOE OSTI2020

Development of a Comprehensive Two-Phase Flow Database for the Validation of NEK-2P

Three-dimensional (3-D) two-phase Computational Fluid Dynamics (CFD) codes are emerging as a powerful and potentially practical tool for applications in which detailed local flow information is needed. However, two-phase flow models and associated closure relations are not well established for CFD applications, which is partly due to the lack of high-quality validation data. The main objective of this work is to develop a comprehensive database of two-phase flows that can be used to validate two-phase CFD codes such as NEK-2P. In this project, four advanced local measurement systems, including Particle Image Velocimetry and Planar Laser-Induced Fluorescence (PIV-PLIF), high-speed imaging, x-ray densitometry, and multi-sensor conductivity probe are employed to measure the local two-phase flow parameters of both gas and liquid phases. By combining these techniques, the local void fraction, bubble velocity, interfacial area concentration, bubble frequency, liquid velocity, turbulence intensity, etc., in various two-phase flow regimes can be obtained. These local measurement techniques are first used in a 25.4 mm circular pipe test section. Seven air-water two-phase flow conditions spanning the bubbly, slug, churn-turbulent, and annular flow regimes are measured in this facility. The obtained database contains the radial profiles of both gas- and liquid-phase parameters at three axial locations along the test section. The second facility used in this work contains a 30 mm × 10 mm rectangular test section, in which three two-phase flow and two single-phase flow conditions are measured. Two-dimensional distributions of local two-phase flow parameters in the cross-sectional plane are measured in this facility at three axial locations as well. A facility featuring a 3×3 electrically heated rod bundle is also designed and being constructed in this project. This facility is specially designed for optical measurements and is expected to provide high-quality boiling data in the future. Preliminary analyses have been performed for the data taken in the 25.4 mm circular pipe. Both center-peaked and wall-peaked void fraction profiles have been observed in the data depending on the two-phase flow conditions and/or developing lengths. The 1-D drift-flux model was evaluated with the newly obtained datasets, in which both gas- and liquid-phases data were directly measured. The distribution parameter model has been optimized based on a new void-profile classification method proposed in this study. The optimized drift-flux model shows a significant improvement in predicting both gas velocity and void fraction. The measured liquid-phase turbulence was used to benchmark Sato’s turbulence model considering the bubble-induced shear stress for the three tested bubbly flows. The benchmark results showed good agreement between the PIV measurements and model predictions. In the bubbly flows tested that have low void fractions less than 3%, the effect of the bubble-induced turbulence was found not significant. However, the bubble-induced shear stress becomes important with the increase of the void fraction. The Conjugate Heat Transfer (CHT) model was developed and implemented in NEK-2P. This model allows the coupled simulation of the solid domain and two-phase fluid domain, allowing the specification of realistic boundary conditions. The CHT implementation was verified first with non-boiling simulations. The predicted temperatures in the fluid domain were shown to be identical in simulations with or without the CHT model. The CHT model was then validated through simulations of three Becker benchmark CHF tests performed under both Dryout (DO) and Departure from Nucleate Boiling (DNB) conditions. Reasonably good agreement was obtained between calculated wall temperatures and corresponding experimental data.

42 ENGINEERING↗
DOE OSTI2023

Model-Free Approach for Profiling of Polydisperse Soft Matter Using Small Angle Scattering

A strategy for determining the size polydispersity of systems from their small angle coherent scattering is outlined. Here, using the method of moment expansion, we show that the various central moments representing the average particle size, variance of particle size, and skewness of size distribution function (SDF) for polydisperse systems can be extracted from spectral analysis without bias. When the degree of polydispersity is moderate, SDF can be further reconstructed based on the maximum entropy principle. Numerical benchmarking of a model study over a wide range of size nonuniformity demonstrates the validity of this analytical approach for quantifying the size distribution of general soft matter systems in a model-free manner. Furthermore, the efficacy of this method was validated by successfully applying it to the fitting of small-angle neutron scattering data obtained from L64 Pluronic micelles using various form factor models. The numerical and experimental verification underscores the reliability and versatility of this method in accurately characterizing the size distribution of complex soft matter systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Unbiased particle conformation extraction from scattering spectra using orthonormal basis expansions

A strategy is outlined for quantitatively evaluating the particle density profiles from small-angle scattering spectra of dilute solutions. The approach employs an orthonormal basis function expansion method, enabling the determination of characteristic mass distributions in self-assembled structures without the need for a specific structural model. Through computational benchmarking, the efficacy of this approach is validated by effectively reconstructing the density profile of soft-ball systems with varying fuzziness from their scattering signatures. Further, the feasibility of the method is demonstrated by fitting small-angle neutron scattering data obtained from Pluronic L64 micelles at different temperatures. This proposed approach is both simple and analytical, eliminating the requirement for a presumptive structural model in scattering analysis. The new method could therefore facilitate quantitative descriptions of complex nanoscopic structures inherent to numerous soft-matter systems using small-angle scattering techniques.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Time-dependent variational principle for mixed matrix product states in the thermodynamic limit

Here, we describe a time evolution algorithm for quantum spin chains whose Hamiltonians are composed of an infinite uniform left and right bulk part, and an arbitrary finite region in between. The left and right bulk parts are allowed to be different from each other. The algorithm is based on the time-dependent variational principle (TDVP) of matrix product states. It is inversion-free and very simple to adapt from an existing TDVP code for finite systems. The importance of working in the projective Hilbert space is highlighted. We study the quantum Ising model as a benchmark and an illustrative example. The spread of information after a local quench is studied in both the ballistic and the diffusive case. We also offer a derivation of TDVP directly from symplectic geometry.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗