Search NASA⌕ Search

SEARCH · Search NASA

Results for “model benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

The Case for and Against a Gadolinium Bias in SCALE: Round 2

The “Opening Arguments” for and against a gadolinium bias in SCALE were presented at the American Nuclear Society Annual Meeting in Philadelphia, Pennsylvania, in June, 2018. Some critical experiments included in the Oak Ridge National Laboratory Verified, Archived Library of Inputs and Data (VALID) indicate a significant bias as a function of gadolinium concentration. Other experiments indicate that no significant bias exists. The work presented here develops a larger suite of gadolinium-bearing benchmark models to further examine code, data, and benchmark performance. The new benchmark models have been reviewed for accuracy, but documentation and review for addition to the VALID library have not been completed. The problematic benchmarks included in VALID are HEU-SOL-THERM-014 and -016. These are two evaluations from a series of experiments from the Institute for Physics and Power Engineering (IPPE), Russia, documented in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook. The entire set of evaluations also includes HEU-SOL-THERM-015, -017, -018, -019, and -025. Each evaluation contains a different uranium concentration, and different configurations within each evaluation include different gadolinium concentrations. These 7 evaluations contain a total of 52 configurations and form the largest subset of experiments considered, and they allow for a more complete assessment of the performance of these benchmarks than has historically been possible using just the HEU-SOL-THERM-014 and -016 results. Additional solution experiments are considered, including MIX-SOL-THERM-006 and -007 and PU-SOL-THERM-034. MIX-SOL-THERM-007 is in the VALID library, whereas MIX-SOL-THERM-006 and PU-SOL-THERM-034 are not. The results for MIX-SOL THERM-007 have not shown a gadolinium trend. These mixed- and plutonium-fueled solutions include 28 configurations. Some experiments including solid fuel and solid gadolinium are also included. These experiments include highly enriched uranium (HEU) foils moderated with polyethylene in HEU-MET-THERM-010, -016, and -034, and low enriched uranium (LEU) pin arrays with gadolinia absorber rods in LEU-COMP THERM-036 and -043. A total 32 cases with solid fuel are included. The IPPE solution benchmarks show a fairly high degree of variability, but no clear trend as a function of gadolinium concentration can be observed. The mixed- and plutonium-fueled solutions show less variability than the HEU solutions and also no trend relative to gadolinium concentration. The solid-fueled experiments also show no trend as a function of gadolinium content. The entire set of benchmarks shows no clear trend on gadolinium content or the energy of the average neutron lethargy causing fission.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

Beyond Energy Efficiency: A clustering approach to embed demand flexibility into building energy benchmarking

The intermittency of carbon-free renewables and the demand changes associated with the widespread push for electrifying the transportation and building sectors provides an opportunity for buildings to go beyond energy efficiency and push towards providing demand flexibility to the electricity grid. The duality of energy efficiency and demand flexibility is necessary for success in a sustainable and reliable energy transition. Current building energy benchmarking models are limited in their ability to integrate concepts of demand flexibility and/or utilize granular smart meter data. Thus, current benchmarking methods are focused annual energy usage and fail to incorporate how the time of use of energy consumption impacts emissions in a quickly changing energy grid. Without a more comprehensive view of energy usage and associated real-time emissions, current benchmarking methods are unlikely to realize the full decarbonization potential of buildings. New emerging data streams provide an opportunity to develop a new generation of benchmarking energy models that embed dimensions of energy efficiency, grid interactivity, and demand flexibility into their analysis. In this paper, we propose a four-step method for embedding grid interactivity and demand flexibility into building benchmarking models that utilizes emerging building and time-series electricity data streams. We first engineer features to produce a mix-type dataset that encompasses many attributes of grid-interactive and efficient buildings, and then we apply K-medoids using Gower's Distance to produce peer-group clusters. We apply the method to a case study of 306 primary and secondary schools in southern California, USA. The results show that the method effectively clusters buildings by attributes of demand flexibility and energy efficiency. The clustering results reveal patterns in inefficient building operations and demand inflexibility at the building peer group level. In conclusion, the interpretation of clusters can serve as an integrated energy efficiency and demand flexibility benchmarking model and inform performance-specific policy targeting for buildings that go beyond traditional efficiency measures.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Quantifying structural errors in cloud condensation nuclei activity from reduced representation of aerosol size distributions

Aerosol effects on clouds and radiation are the dominant contribution to uncertainty in radiative forcing relative to the pre-industrial atmosphere. While previous studies have assessed the impact of parametric uncertainty on modeled forcing, structural errors from the numerical representation of particle distributions have not been well quantified. Here we present a framework for quantifying error in aerosol size distributions and cloud condensation nuclei activity, which we apply to the widely used 4-mode version of the Modal Aerosol Module (MAM4). Box model predictions from the MAM4 are evaluated against the Particle Monte Carlo Model for Simulating Aerosol Interactions and Chemistry (PartMC-MOSAIC), a benchmark model that tracks the evolution of individual particles. We show that size distributions simulated by MAM4 diverge from those simulated by PartMC-MOSAIC after only a few hours of aging by condensation and coagulation in polluted conditions, which leads to large errors in modeled cloud condensation nuclei concentrations. We find that differences between MAM4 and PartMC-MOSAIC are largest under polluted conditions, where the size distribution evolves rapidly though aging by condensation of semi-volatile substances and coagulation among particles. These findings suggest that structural error in modeled aerosol properties contributes to the large inter-model variability in aerosol radiative forcing.

Fierce, Laura M.↗
DOE OSTI2020

Land Model Testbed: Accelerating Development, Benchmarking and Analysis of Land Surface Models

A Land Model Testbed (LMT), designed to provide a computational framework for systematically assessing model fidelity and supporting rapid development of complex multiscale models, offers a general-purpose workflow for conducting large ensemble simulations of multiple land surface models, post-processing large volumes of model output, and evaluating model results. It leverages existing tools for launching model simulations and the International Land Model Benchmarking (ILAMB) package for assessing model fidelity through comparison with best-available observational datasets. Increased complexity and proliferation of uncertain parameters in process representations in land surface models has driven the need for frequent and intensive testing and evaluating of models to quantify uncertainties and optimize parameters such that results are consistent with observations. The LMT described here meets these needs by providing tools to run thousands of ensemble simulations simultaneously and post-process their output files, by automating execution of an enhanced version of ILAMB with site-specific benchmarks and multivariate functional relationships, and by offering ensemble diagnostics and a customizable dashboard for displaying model performance metrics and associated graphics. We envision the LMT capabilities will serve as a foundational computational resource for a proposed user facility focused on terrestrial multiscale model--data integration.

Sreepathi, Sarat↗
DOE OSTI2022

1D Heat Loss Models Validation Experiment

Contains data from the model validation in the 1D Heat Loss Models to Predict the Aquifer Temperature Profile during Hot/Cold Water Injection Project. The data include two COMSOL models (2D axisymmetric benchmark model and 2D Vinsome model), one python code (1D Vinsome based FEM numerical simulation), one matlab main code (1D Newton analytical solution and all results comparison visualization), and output files generated from the above models.

1D↗
DOE OSTI2025

Forecasting Multi-Step-Ahead Street-Scale Nuisance Flooding using a seq2seq LSTM Surrogate Model for Real-Time Application in a Coastal-Urban City

In coastal-urban cities facing an elevated risk of nuisance flooding (by rain and tide) due to increased heavy rainfall, sea level rise, urbanization, and aging drainage systems, real-time flood forecasting at the street-scale can provide useful information to transportation decision-makers. Physics-Based Models (PBMs) that offer high accuracy come with high computational runtimes and costs that limit their application for real-time flood forecasting. To address this challenge, Machine Learning (ML) surrogate models trained from PBMs have been proposed to provide street-scale flood forecasts. Previous related studies have focused on using Long Short-Term Memory (LSTM) architectures to model hourly flood depth on streets. While LSTM models can capture input sequences effectively, they fall short in accurately preserving output sequences, limiting their suitability for multi-step-ahead forecasts. The seq2seq LSTM architecture offers a key advantage here by capturing the full sequence of input–output, making it potentially more suitable for multi-step-ahead flood forecasts compared to traditional LSTM models. However, seq2seq LSTM has not been tested for street-scale flood forecasting, particularly for rapidly fluctuating nuisance flooding events which require special attention to its temporal sequences. Hence, in this study, we applied the seq2seq LSTM model to explore multi-step-ahead street-scale nuisance flooding and compared its results to the traditional LSTM model as a benchmark model. LSTM and seq2seq LSTM surrogate models were applied to 22 flood-prone streets in Norfolk, Virginia, as a case study with a 4-hr (short-term) and 8-hr (long-term) lead time. The models were trained with environmental (rainfall and tide) and topographic (elevation, Topographic Wetness Index, and Depth-To-Water) features along with PBM-derived water depths for different storm events. The results demonstrated satisfactory performance of both LSTM and seq2seq LSTM surrogate models throughout the forecast period compared to the PBM. However, the seq2seq LSTM showed lower Mean Absolute Error (MAE)/ Root Mean Square Error (RMSE) and higher Nash–Sutcliffe Efficiency (NSE)/ correlation than the LSTM across most lead times, particularly for long-term forecasting due to its supremacy in handling both input–output sequences together, which is missing in the traditional LSTM. For example, in the long-term, the average RMSE ranges were 0.0268–0.0373 m for LSTM and 0.0226–0.0319 m for seq2seq LSTM, while in the short-term, they were 0.0263–0.0293 m and 0.0261–0.0283 m, respectively. Additionally, while both models exhibited similar performance in distinguishing flooded and non-flooded streets for flood depth ≥ 0.1 m, the seq2seq LSTM model demonstrated superior performance for higher flood depths (such as ≥ 0.2 m and ≥ 0.3 m). Once trained, inference took only 0.09 to 0.11 s (short-term) and 0.30 to 0.35 s (long-term) per storm event for the 22 streets, making the application highly suitable for real-time decision-making during nuisance flood events.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

bmdrc: Python package for quantifying phenotypes from chemical exposures with benchmark dose modeling

Though chemical exposures are known to potentially have negative impacts on health, including contributing to chronic diseases such as cancer, the quantitative contribution of risk is not fully understood for every chemical. A commonly used approach to quantify levels of risk is to measure the proportion of organisms (such as a total number of zebrafish on a plate or mice in a cage) with abnormal behavioral responses or morphology at increasing concentrations of chemical exposure. A particular challenge with processing the proportional data from these assays is the appropriate estimation of chemical concentration levels that result in malformations or acute toxicity, as these values typically vary between experimental measurements. The recommended approach by the Environmental Protection Agency (EPA) is to fit benchmark dose curves with specific filters and model fitting steps, which are crucial to properly processing the proportional data. Several tools exist for the fitting of benchmark dose response curves, but none are standalone Python libraries built to process both morphological and behavioral data as proportions with all the EPA recommended filters, filter parameters, models, and model parameters. Thus, here we present the benchmark dose response curve (bmdrc) Python library, which was built to closely follow these EPA guidelines with helpful visualizations of filters and fitted model curves, and reports for reproducibility purposes. bmdrc is open-source and has demonstrated utility as a support package to an existing web portal for information on chemicals (https://srp.pnnl.gov). Our package will support any toxicology analysis where the response is a proportional value at increasing levels of a concentration of a chemical or chemical mixture.

Superfund↗
DOE OSTI2022

Analytical benchmark solution for 1-D neutron transport coupled with thermal conduction and material expansion

In this work we present fully analytical solutions for a class of finite, homogeneous, 1-D slab benchmark problems with nonlinear temperature feedback effects. The proposed class of benchmarks include multiplicative 1-D neutron transport (limited to quasistatic S{sub 2} with μ = ±1) coupled with thermal conduction, convection, Doppler broadening, and expansion effects along the length of the slab. This class of benchmark models, along with the corresponding analytical solutions, are valuable for validating multiphysics analysis frameworks that support coupled neutronics/thermal/structural calculations. Analytical solutions for the benchmarks are obtained by introducing an ansatz that the equilibrium flux and temperature distributions in the slab have the same shape. Specific values for the total microscopic cross section, σ{sub t,0}, and conductive heat transfer coefficient, h, that satisfy the assumed ansatz are then determined. A discussion of the procedure for generating benchmark models and analytical solutions is provided, along with numerical results for an example set of model parameters and thoughts on practical applications of the benchmarks for multiphysics code validation. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Integrating State Data Assimilation and Innovative Model Parameterization Reduces Simulated Carbon Uptake in the Arctic and Boreal Region

Model representation of carbon uptake and storage is essential for accurate projection of the response of the arctic-boreal zone to a rapidly changing climate. Land model estimates of LAI and aboveground biomass that can have a marked influence on model projections of carbon uptake and storage vary substantially in the arctic and boreal zone, making it challenging to correctly evaluate model estimates of Gross Primary Productivity (GPP). To understand and correct bias of LAI and aboveground biomass in the Community Land Model (CLM), we assimilated the 8-day Moderate Resolution Imaging Spectroradiometer (MODIS) LAI observation and a machine learning product of annual aboveground biomass into CLM using an Ensemble Adjustment Kalman Filter (EAKF) in an experimental region including Alaska and Western Canada. Assimilating LAI and aboveground biomass reduced these model estimates by 58% and 72%, respectively. The change of aboveground biomass was consistent with independent estimates of canopy top height at both regional and site levels. The International Land Model Benchmarking system assessment showed that data assimilation significantly improved CLM's performance in simulating the carbon and hydrological cycles, as well as in representing the functional relationships between LAI and other variables. Here, to further reduce the remaining bias in GPP after LAI bias correction, we re-parameterized CLM to account for low temperature suppression of photosynthesis. The LAI bias corrected model that included the new parameterization showed the best agreement with model benchmarks. Combining data assimilation with model parameterization provides a useful framework to assess photosynthetic processes in LSMs.

58 GEOSCIENCES↗
DOE OSTI2023

Computational fluid dynamic analysis of a novel particle-to-air fluidized-bed heat exchanger for particle-based thermal energy storage applications

Long-duration energy storage technologies are being targeted to enable cost-effective, decarbonized energy systems. Particle-based thermal energy storage systems are one promising technology by storing excess electricity or heat as sensible thermal energy in inexpensive, solid, inert particles. These systems are only possible if an effective and economical particle-to-working fluid heat exchanger exists. This study predicts the performance of a proposed, direct-contact, particle-to-air, pressurized fluidized-bed heat exchanger using computational fluid dynamics. The common Eulerian-Eulerian framework for modeling fluidized beds is first benchmarked to experimental results at a previously untested operating condition and application. Then, the benchmarked model evaluates the performance of a proposed design for a commercial-scale version of the novel particle-to-air heat exchanger. The results show pressure drop and gas-phase approach temperatures are advantageous compared to other proposed designs for particle-to-air heat exchangers in the literature; approach temperatures were less than 5 °C and gas-phase pressure drop across the fluidized bed was 32 kPa. The model also highlights the importance of gas distributor design and representation in the Eulerian-Eulerian framework to control fluidization behavior. In conclusion, the model built and benchmarked in this study can be leveraged to advance the design and analysis of these heat exchangers critical to the deployment of a promising long-duration energy storage technology.

25 ENERGY STORAGE↗
DOE OSTI2026

Systematic Benchmarking of Climate Models: Methodologies, Applications, and New Directions

As climate models become increasingly complex, there is a growing need to comprehensively and systematically assess model performance with respect to observations. Given the increasing number and diversity of climate model simulations in use, the community has moved beyond simple model intercomparison and toward developing methods capable of benchmarking a large number of simulations against a suite of climate metrics. Here, we present a detailed review of evaluation and benchmarking methods and approaches developed in the last decade, focusing primarily on scientific implications for Coupled Model Intercomparison Project (CMIP) simulations and CMIP6 results that contributed to the Intergovernmental Panel on Climate Change (IPCC) Sixth Assessment Report (AR6). Based on this review, we explain the resulting contemporary philosophy of model benchmarking, and provide clear distinctions and definitions of the terms model verification, process validation, evaluation, and benchmarking. While significant progress has been made in model development based on systematic evaluation and benchmarking efforts, some climate system biases still remain. The development of open‐source community software packages has played a fundamental role in identifying areas of significant model improvement and bias reduction. We review the key features of several software packages that have been commonly used over the past decade to evaluate and benchmark global and regional climate models. Additionally, we discuss best practices for the selection of evaluation and benchmarking metrics and for interpreting the obtained results, the importance of selecting suitable sources of reference data and accurate uncertainty quantification.

Environmental sciences↗
DOE OSTI2021

Dark scalars and heavy neutral leptons at DarkQuest

The proposed DarkQuest beam dump experiment, a modest upgrade to the existing SeaQuest/SpinQuest experiment, has great potential for uncovering new physics within a dark sector. We explore both the near-term and long-term prospects for observing two distinct, highly-motivated hidden sector benchmark models: heavy neutral leptons and Higgs-mixed scalars. We comprehensively examine the particle production and detector acceptance at DarkQuest, including an updated treatment of meson production, and light scalar production through both bremsstrahlung and gluon-gluon fusion. In both benchmark models, DarkQuest will provide an opportunity to probe previously inaccessible interesting regions of parameter space on a fairly short timescale when compared to other proposed experiments.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Validation of Continuous-Energy ENDF/B-VIII.0 16 O, 56 Fe, and 63,65 Cu Cross Sections for Nuclear Criticality Safety Applications

Recently completed cross-section evaluations sponsored in part by the Nuclear Criticality Safety Program were incorporated into the 2018 release of the ENDF/B-VIII.0 cross-section library. Evaluated isotopes of interest to the nuclear data and criticality safety community include 16 O, 56 Fe, and 63,65 Cu. For performance validation, benchmark models defined in the International Criticality Safety Benchmark Evaluation Project Handbook were selected based on energy-integrated k eff sensitivities to total cross sections of interest and compared with experimental values. Of the 102 benchmark configurations that were utilized, 63 are sensitive to 16 O, 32 sensitive to 63,65 Cu, and 25 sensitive to 56 Fe. Selected benchmarks were modeled in SCALE 6.2.3 Criticality Safety Analysis Sequence (CSAS) continuous-energy Monte Carlo k eff calculations with ENDF/B-VII.1, with a hybrid ENDF/B-VII.1 with ENDF/B-VIII.0 data substituted for individual isotopes of interest, and with ENDF/B-VIII.0. ENDF/B-VIII.0 showed improved agreement with experimental k eff for 56 Fe, 63 Cu, elemental copper, and full library substitution while producing lessened agreement for 16 O and 65 Cu. Furthermore, with full library and isotope-specific ENDF/B-VIII.0 performance, a best-case ENDF library was formed by excluding underperforming isotopes’ ENDF/B-VIII.0 data, reverting 16 O and 65 Cu cross sections to ENDF/B-VII.1. This resulted in the average relative deviation between calculated and experimental data improving from 1.45σ for the ENDF/B-VIII.0 library to 1.32σ for the best-case library, relative to benchmark uncertainty.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

ILAMBv2.7 benchmarking results comparing E3SMv2.1 land-atmosphere coupled (BGCv2LNDATM) and stand alone land (ELM) simulations with CMIP6 emission driven historical simulations

This dataset contains land model benchmarking results for the Energy Exascale Earth System Model version 2.1 (E3SMv2.1), including outputs from both coupled biogeochemistry simulations and stand-alone land model simulations. These results are compared against several emission-driven historical simulations from the Coupled Model Intercomparison Project Phase 6 (CMIP6). Benchmarking was conducted using the International Land Model Benchmarking (ILAMB) package, version 2.7 (ILAMBv2.7). CMIP6 model outputs were sourced from the Earth System Grid Federation (ESGF), while the E3SMv2.1 results were derived from raw model outputs. These outputs underwent processing steps such as time serialization, conservative regridding, and data standardization to ensure comparability. For spatial interpolation, the Earth System Modeling Framework (ESMF) tool, ESMF_RegridWeightGen, was employed to generate regridding weights, enabling the transformation of E3SM’s native cubed-sphere grid to a regular latitude-longitude grid.

Feng, Sha [PNNL]↗
DOE OSTI2022

SBbadger: biochemical reaction networks with definable degree distributions

Abstract Motivation An essential step in developing computational tools for the inference, optimization and simulation of biochemical reaction networks is gauging tool performance against earlier efforts using an appropriate set of benchmarks. General strategies for the assembly of benchmark models include collection from the literature, creation via subnetwork extraction and de novo generation. However, with respect to biochemical reaction networks, these approaches and their associated tools are either poorly suited to generate models that reflect the wide range of properties found in natural biochemical networks or to do so in numbers that enable rigorous statistical analysis. Results In this work, we present SBbadger, a python-based software tool for the generation of synthetic biochemical reaction or metabolic networks with user-defined degree distributions, multiple available kinetic formalisms and a host of other definable properties. SBbadger thus enables the creation of benchmark model sets that reflect properties of biological systems and generate the kinetics and model structures typically targeted by computational analysis and inference software. Here, we detail the computational and algorithmic workflow of SBbadger, demonstrate its performance under various settings, provide sample outputs and compare it to currently available biochemical reaction network generation software. Availability and implementation SBbadger is implemented in Python and is freely available at https://github.com/sys-bio/SBbadger and via PyPI at https://pypi.org/project/SBbadger/. Documentation can be found at https://SBbadger.readthedocs.io. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Performance analysis and comparison of data-driven models for predicting indoor temperature in multi-zone commercial buildings

Building thermal models, which characterize the properties of a building’s envelope and thermal mass, are essential for accurate indoor temperature and cooling/heating demand prediction. Because of their flexibility and ease of use, data-driven models are increasingly used. Here, this study compared and analyzed the performance of gray-box (resistance-capacitance) and black-box (recurrent neural network) models for predicting indoor air temperature in a real multi-zone commercial building. The developed resistance-capacitance model served as a benchmark model for which full sets of temporal data and building information were used as inputs. The recurrent neural network models were trained and tested assuming various available types and amounts of temporal data and known building physical information to investigate the effects of data and information availability. Feature importance analysis was conducted to select the key variables for different prediction targets under different scenarios. This research provides guidance in selecting an appropriate building thermal response modeling method based on the measured data availability, building physical information, and application.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Primordial black holes from first-order phase transition in the singlet-extended SM

Supercooled first-order phase transition (FOPT) can lead to the formation of primordial black holes (PBHs). This scenario imposes stringent requirements on the profile of the effective potential. In this work, we use the singlet extended Standard Model (xSM) as a benchmark model to investigate this possibility at the electroweak scale. The PBHs formed during a supercooled FOPT have a narrow mass distribution around the mass of Earth. This distribution is closely tied to the temperature at which the PBHs form, corresponding to the FOPT at the electroweak scale. This scenario can be probed with microlensing experiments, space-based gravitational wave detectors, and collider experiments. Remarkably, the future space-based gravitational wave detector LISA will hold the potential to either confirm this scenario that leads to PBH formation across the observable Universe in the xSM or completely rule it out. Interestingly, our findings suggest that PBHs within the xSM framework may align with observations of the six ultrashort timescale events reported by the OGLE microlensing experiment. Published by the American Physical Society 2025

Gonçalves, Dorival (ORCID:0000000342234238)↗