Search NASA⌕ Search

SEARCH · Search NASA

Results for “model benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Antarctic ice sheet model comparison with uncurated geological constraints shows that higher spatial resolution improves deglacial reconstructions

Accurately reconstructing past changes to the shape and volume of the Antarctic ice sheet relies on the use of physically based and thus internally consistent ice sheet modeling, benchmarked against spatially limited geologic data. The challenge in model benchmarking against geologic data is diagnosing whether model-data misfits are the result of an inadequate model, inherently noisy or biased geologic data, and/or incorrect association between modeled quantities and geologic observations. In this work we address this challenge by (i) the development and use of a new model-data evaluation framework applied to an uncurated data set of geologic constraints, and (ii) nested high-spatial-resolution modeling designed to test the hypothesis that model resolution is an important limitation in matching geologic data. While previous approaches to model benchmarking employed highly curated datasets, our approach applies an automated screening and quality control algorithm to an uncurated public dataset of geochronological observations (specifically, cosmogenic-nuclide exposure-age measurements from glacial deposits in ice-free areas). This optimizes data utilization by including more geological constraints, reduces potential interpretive bias, and allows unsupervised assimilation of new data as they are collected. We also incorporate a nested model framework in which high-resolution domains are downscaled from a continent-wide ice sheet model. We highlight the application of this framework by applying these methods to a small ensemble of deglacial ice-sheet model simulations, and demonstrate that the nested approach improves the ability of model simulations to match exposure age data collected from areas of complex topography and ice flow. We develop a range of diagnostic model-data comparison metrics to provide more insight into model performance than possible from a single-valued misfit statistic, showing that different metrics capture different aspects of ice sheet deflation.

Geosciences↗
DOE OSTI2022

Development of the TREAT M2 experiment as a transient benchmark

A transient benchmark based on the Transient Reactor Test Facility (TREAT) M2 Calibration experiment (M2-CAL) is under development. TREAT, at Idaho National Laboratory, is a graphite moderated air-cooled research reactor which has been used extensively for fuel material testing under extreme and accident conditions. Accurate benchmark models are a beneficial component in the operations, experimental planning, and development of TREAT. In this work, we present a transient benchmark model for the M2-CAL experiment core loading. The benchmark model incorporates the coupling between the Monte Carlo Code SERPENT and the computational fluid dynamics code OpenFOAM to capture the temperature feedback mechanism. The M2-CAL transient 2580 was simulated in this work. A pre-transient analysis was performed to determine the optimum core composition and conditions before the beginning of the transient. The analysis was validated against historic TREAT kinetic measurements and the worth of the transient rod T-2. The axial power distribution in the flux wire for the M2-CAL experiment was determined and contrasted with the experiment. The model was then used to simulate the M2-CAL transient 2580 experiment based on the reported pre-transient and transient conditions. Several transient observables were calculated and compared to the experiment. The model shows good agreement with the experimental power traces, period, and average increase of power at the power ramp (time > 7.8 s). Inverse point kinetics analysis was performed during the period of the power ramp for the model and the experiment based on that, the temperature feedback component was isolated and contrasted with the experimental feedback. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRS1985

Comparison of GSFC and JPL VLBI modeling software: Benchmark

The Very Long Base Interferometry. The VLBI modeling software packages CALC 6.0 (GSFC) and MASTERFIT (JPL) are compared in some detail. Theoretical model delays are calculated for a set of 120 fictitious observations which involve a variety of sources, baselines, and antennas. Discrepancies between the total delays given by the two programs are of the order of 2 cm (RMS). The modeling of antenna offsets appears to account for approximately half of this difference. Relativistic bending contributions to the delay differ by 3 cm (RMS), and the there appears to be some mutual cancellation of errors involving antenna offsets, bending, and the effects of the two different Solar System ephemerides employed by CALC and MASTERFIT. This cancellation has not been completely characterized here.

Sovers, O. J.↗
DOE OSTI2026

SMR safety through HTTF modeling and benchmark efforts for code validation for gas-cooled reactor applications

Accurate modeling and simulation tools for thermal-hydraulics calculations are a key element needed to design and license new advanced reactors including Small Modular Reactors (SMR) and Microreactors. Uncertainties in modeling and simulation can have significant safety and economic implications. The High Temperature Test Facility (HTTF) at Oregon State University (OSU) is a scaled integral effects experiment designed to investigate transient behavior in high-temperature gas-cooled prismatic-block nuclear reactors. High-quality measurement data is available from the HTTF that is suitable for a thermal-hydraulics code validation benchmark for gas-cooled reactor simulations. Here, this paper summarizes individual HTTF modeling efforts to date for tool validation at Idaho National Laboratory (INL), Argonne National Laboratory (ANL), Oregon State University (OSU) and Canadian Nuclear Laboratories (CNL) using system thermal-hydraulics codes, Computational Fluid Dynamics (CFD) codes and system-CFD code couplings. Also, the paper introduces the ongoing OECD Nuclear Energy Agency (NEA) High Temperature Gas Reactor Thermal-Hydraulics (HTGR T/H) benchmark that allows for better comparisons of results between different international modeling teams. The benchmark provides well defined computational problems that include code-to-code comparisons and comparisons to measured data. These problems provide an avenue for quantifying accuracy and identifying sources of uncertainty in thermal-hydraulics calculations, including in measured thermophysical properties, as part of validation for gas-cooled reactor simulation tools.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗
DOE OSTI2026

Benchmarking soil moisture and its relationship to ecohydrologic variables in Earth System Models

Soil moisture (SM) is a key regulator of ecosystem biogeophysics, influencing plant water relations and land-atmosphere energy exchanges. We evaluate the representation of SM in 16 Earth System Models from the Coupled Model Intercomparison Project Phase 6 (CMIP6) using the International Land Model Benchmarking (ILAMB) framework, focusing on surface (0–5, 0–10 cm) and rootzone (0–100 cm) depths, as well as key ecohydrological variables like gross primary productivity (GPP), leaf area index (LAI), and evapotranspiration (ET), and their coupling. Models are benchmarked against multiple observational and assimilated datasets to assess both state variables and cross-variable relationships. Surface SM is generally well represented (r > 0.87), while rootzone SM variability is systematically overestimated (normalized standard deviation > 1). ET shows strong agreement with observations (r > 0.9), whereas GPP and LAI exhibit larger inter-model spread. Skill in individual variables does not guarantee realistic SM–ecohydrology coupling, which varies strongly across models and depends on the reference dataset. Köppen-based regional analyses reveal strong regime dependence, with several models performing well in Tropical and Temperate regions but degrading in Continental (high-latitude) zones. Across both global and regional benchmarks, models cluster by land surface framework, indicating that structural choices in soil hydrology and soil–plant coupling exert a first-order control on performance. These results provide process-relevant benchmarks and suggest that improving the representation of vertical soil structure, rooting depth distributions, and soil–plant hydraulic coupling will be central to advancing soil moisture realism in next-generation Earth system models.

CMIP6↗
DOE OSTI2023

The Case for and Against a Gadolinium Bias in SCALE: Round 2

The “Opening Arguments” for and against a gadolinium bias in SCALE were presented at the American Nuclear Society Annual Meeting in Philadelphia, Pennsylvania, in June, 2018. Some critical experiments included in the Oak Ridge National Laboratory Verified, Archived Library of Inputs and Data (VALID) indicate a significant bias as a function of gadolinium concentration. Other experiments indicate that no significant bias exists. The work presented here develops a larger suite of gadolinium-bearing benchmark models to further examine code, data, and benchmark performance. The new benchmark models have been reviewed for accuracy, but documentation and review for addition to the VALID library have not been completed. The problematic benchmarks included in VALID are HEU-SOL-THERM-014 and -016. These are two evaluations from a series of experiments from the Institute for Physics and Power Engineering (IPPE), Russia, documented in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook. The entire set of evaluations also includes HEU-SOL-THERM-015, -017, -018, -019, and -025. Each evaluation contains a different uranium concentration, and different configurations within each evaluation include different gadolinium concentrations. These 7 evaluations contain a total of 52 configurations and form the largest subset of experiments considered, and they allow for a more complete assessment of the performance of these benchmarks than has historically been possible using just the HEU-SOL-THERM-014 and -016 results. Additional solution experiments are considered, including MIX-SOL-THERM-006 and -007 and PU-SOL-THERM-034. MIX-SOL-THERM-007 is in the VALID library, whereas MIX-SOL-THERM-006 and PU-SOL-THERM-034 are not. The results for MIX-SOL THERM-007 have not shown a gadolinium trend. These mixed- and plutonium-fueled solutions include 28 configurations. Some experiments including solid fuel and solid gadolinium are also included. These experiments include highly enriched uranium (HEU) foils moderated with polyethylene in HEU-MET-THERM-010, -016, and -034, and low enriched uranium (LEU) pin arrays with gadolinia absorber rods in LEU-COMP THERM-036 and -043. A total 32 cases with solid fuel are included. The IPPE solution benchmarks show a fairly high degree of variability, but no clear trend as a function of gadolinium concentration can be observed. The mixed- and plutonium-fueled solutions show less variability than the HEU solutions and also no trend relative to gadolinium concentration. The solid-fueled experiments also show no trend as a function of gadolinium content. The entire set of benchmarks shows no clear trend on gadolinium content or the energy of the average neutron lethargy causing fission.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRS2013

Comparative Benchmark Dose Modeling as a Tool to Make the First Estimate of Safe Human Exposure Levels to Lunar Dust

Brief exposures of Apollo Astronauts to lunar dust occasionally elicited upper respiratory irritation; however, no limits were ever set for prolonged exposure ot lunar dust. Habitats for exploration, whether mobile of fixed must be designed to limit human exposure to lunar dust to safe levels. We have used a new technique we call Comparative Benchmark Dose Modeling to estimate safe exposure limits for lunar dust collected during the Apollo 14 mission.

James, John T.↗
DOE OSTI2022

Beyond Energy Efficiency: A clustering approach to embed demand flexibility into building energy benchmarking

The intermittency of carbon-free renewables and the demand changes associated with the widespread push for electrifying the transportation and building sectors provides an opportunity for buildings to go beyond energy efficiency and push towards providing demand flexibility to the electricity grid. The duality of energy efficiency and demand flexibility is necessary for success in a sustainable and reliable energy transition. Current building energy benchmarking models are limited in their ability to integrate concepts of demand flexibility and/or utilize granular smart meter data. Thus, current benchmarking methods are focused annual energy usage and fail to incorporate how the time of use of energy consumption impacts emissions in a quickly changing energy grid. Without a more comprehensive view of energy usage and associated real-time emissions, current benchmarking methods are unlikely to realize the full decarbonization potential of buildings. New emerging data streams provide an opportunity to develop a new generation of benchmarking energy models that embed dimensions of energy efficiency, grid interactivity, and demand flexibility into their analysis. In this paper, we propose a four-step method for embedding grid interactivity and demand flexibility into building benchmarking models that utilizes emerging building and time-series electricity data streams. We first engineer features to produce a mix-type dataset that encompasses many attributes of grid-interactive and efficient buildings, and then we apply K-medoids using Gower's Distance to produce peer-group clusters. We apply the method to a case study of 306 primary and secondary schools in southern California, USA. The results show that the method effectively clusters buildings by attributes of demand flexibility and energy efficiency. The clustering results reveal patterns in inefficient building operations and demand inflexibility at the building peer group level. In conclusion, the interpretation of clusters can serve as an integrated energy efficiency and demand flexibility benchmarking model and inform performance-specific policy targeting for buildings that go beyond traditional efficiency measures.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Quantifying structural errors in cloud condensation nuclei activity from reduced representation of aerosol size distributions

Aerosol effects on clouds and radiation are the dominant contribution to uncertainty in radiative forcing relative to the pre-industrial atmosphere. While previous studies have assessed the impact of parametric uncertainty on modeled forcing, structural errors from the numerical representation of particle distributions have not been well quantified. Here we present a framework for quantifying error in aerosol size distributions and cloud condensation nuclei activity, which we apply to the widely used 4-mode version of the Modal Aerosol Module (MAM4). Box model predictions from the MAM4 are evaluated against the Particle Monte Carlo Model for Simulating Aerosol Interactions and Chemistry (PartMC-MOSAIC), a benchmark model that tracks the evolution of individual particles. We show that size distributions simulated by MAM4 diverge from those simulated by PartMC-MOSAIC after only a few hours of aging by condensation and coagulation in polluted conditions, which leads to large errors in modeled cloud condensation nuclei concentrations. We find that differences between MAM4 and PartMC-MOSAIC are largest under polluted conditions, where the size distribution evolves rapidly though aging by condensation of semi-volatile substances and coagulation among particles. These findings suggest that structural error in modeled aerosol properties contributes to the large inter-model variability in aerosol radiative forcing.

Fierce, Laura M.↗
DOE OSTI2020

Land Model Testbed: Accelerating Development, Benchmarking and Analysis of Land Surface Models

A Land Model Testbed (LMT), designed to provide a computational framework for systematically assessing model fidelity and supporting rapid development of complex multiscale models, offers a general-purpose workflow for conducting large ensemble simulations of multiple land surface models, post-processing large volumes of model output, and evaluating model results. It leverages existing tools for launching model simulations and the International Land Model Benchmarking (ILAMB) package for assessing model fidelity through comparison with best-available observational datasets. Increased complexity and proliferation of uncertain parameters in process representations in land surface models has driven the need for frequent and intensive testing and evaluating of models to quantify uncertainties and optimize parameters such that results are consistent with observations. The LMT described here meets these needs by providing tools to run thousands of ensemble simulations simultaneously and post-process their output files, by automating execution of an enhanced version of ILAMB with site-specific benchmarks and multivariate functional relationships, and by offering ensemble diagnostics and a customizable dashboard for displaying model performance metrics and associated graphics. We envision the LMT capabilities will serve as a foundational computational resource for a proposed user facility focused on terrestrial multiscale model--data integration.

Sreepathi, Sarat↗
DOE OSTI2022

1D Heat Loss Models Validation Experiment

Contains data from the model validation in the 1D Heat Loss Models to Predict the Aquifer Temperature Profile during Hot/Cold Water Injection Project. The data include two COMSOL models (2D axisymmetric benchmark model and 2D Vinsome model), one python code (1D Vinsome based FEM numerical simulation), one matlab main code (1D Newton analytical solution and all results comparison visualization), and output files generated from the above models.

1D↗
DOE OSTI2025

Forecasting Multi-Step-Ahead Street-Scale Nuisance Flooding using a seq2seq LSTM Surrogate Model for Real-Time Application in a Coastal-Urban City

In coastal-urban cities facing an elevated risk of nuisance flooding (by rain and tide) due to increased heavy rainfall, sea level rise, urbanization, and aging drainage systems, real-time flood forecasting at the street-scale can provide useful information to transportation decision-makers. Physics-Based Models (PBMs) that offer high accuracy come with high computational runtimes and costs that limit their application for real-time flood forecasting. To address this challenge, Machine Learning (ML) surrogate models trained from PBMs have been proposed to provide street-scale flood forecasts. Previous related studies have focused on using Long Short-Term Memory (LSTM) architectures to model hourly flood depth on streets. While LSTM models can capture input sequences effectively, they fall short in accurately preserving output sequences, limiting their suitability for multi-step-ahead forecasts. The seq2seq LSTM architecture offers a key advantage here by capturing the full sequence of input–output, making it potentially more suitable for multi-step-ahead flood forecasts compared to traditional LSTM models. However, seq2seq LSTM has not been tested for street-scale flood forecasting, particularly for rapidly fluctuating nuisance flooding events which require special attention to its temporal sequences. Hence, in this study, we applied the seq2seq LSTM model to explore multi-step-ahead street-scale nuisance flooding and compared its results to the traditional LSTM model as a benchmark model. LSTM and seq2seq LSTM surrogate models were applied to 22 flood-prone streets in Norfolk, Virginia, as a case study with a 4-hr (short-term) and 8-hr (long-term) lead time. The models were trained with environmental (rainfall and tide) and topographic (elevation, Topographic Wetness Index, and Depth-To-Water) features along with PBM-derived water depths for different storm events. The results demonstrated satisfactory performance of both LSTM and seq2seq LSTM surrogate models throughout the forecast period compared to the PBM. However, the seq2seq LSTM showed lower Mean Absolute Error (MAE)/ Root Mean Square Error (RMSE) and higher Nash–Sutcliffe Efficiency (NSE)/ correlation than the LSTM across most lead times, particularly for long-term forecasting due to its supremacy in handling both input–output sequences together, which is missing in the traditional LSTM. For example, in the long-term, the average RMSE ranges were 0.0268–0.0373 m for LSTM and 0.0226–0.0319 m for seq2seq LSTM, while in the short-term, they were 0.0263–0.0293 m and 0.0261–0.0283 m, respectively. Additionally, while both models exhibited similar performance in distinguishing flooded and non-flooded streets for flood depth ≥ 0.1 m, the seq2seq LSTM model demonstrated superior performance for higher flood depths (such as ≥ 0.2 m and ≥ 0.3 m). Once trained, inference took only 0.09 to 0.11 s (short-term) and 0.30 to 0.35 s (long-term) per storm event for the 22 streets, making the application highly suitable for real-time decision-making during nuisance flood events.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

bmdrc: Python package for quantifying phenotypes from chemical exposures with benchmark dose modeling

Though chemical exposures are known to potentially have negative impacts on health, including contributing to chronic diseases such as cancer, the quantitative contribution of risk is not fully understood for every chemical. A commonly used approach to quantify levels of risk is to measure the proportion of organisms (such as a total number of zebrafish on a plate or mice in a cage) with abnormal behavioral responses or morphology at increasing concentrations of chemical exposure. A particular challenge with processing the proportional data from these assays is the appropriate estimation of chemical concentration levels that result in malformations or acute toxicity, as these values typically vary between experimental measurements. The recommended approach by the Environmental Protection Agency (EPA) is to fit benchmark dose curves with specific filters and model fitting steps, which are crucial to properly processing the proportional data. Several tools exist for the fitting of benchmark dose response curves, but none are standalone Python libraries built to process both morphological and behavioral data as proportions with all the EPA recommended filters, filter parameters, models, and model parameters. Thus, here we present the benchmark dose response curve (bmdrc) Python library, which was built to closely follow these EPA guidelines with helpful visualizations of filters and fitted model curves, and reports for reproducibility purposes. bmdrc is open-source and has demonstrated utility as a support package to an existing web portal for information on chemicals (https://srp.pnnl.gov). Our package will support any toxicology analysis where the response is a proportional value at increasing levels of a concentration of a chemical or chemical mixture.

Superfund↗
NASA NTRS1997

Communication Improvement for the LU NAS Parallel Benchmark: A Model for Efficient Parallel Relaxation Schemes

The first release of the MPI version of the LU NAS Parallel Benchmark (NPB2.0) performed poorly compared to its companion NPB2.0 codes. The later LU release (NPB2.1 & 2.2) runs up to two and a half times faster, thanks to a revised point access scheme and related communications scheme. The new scheme sends substantially fewer messages. is cache "friendly", and has a better load balance. We detail the, observations and modifications that resulted in this efficiency improvement, and show that the poor behavior of the original code resulted from deriving a message passing scheme from an algorithm originally devised for a vector architecture.

Yarrow, Maurice↗
DOE OSTI2022

Analytical benchmark solution for 1-D neutron transport coupled with thermal conduction and material expansion

In this work we present fully analytical solutions for a class of finite, homogeneous, 1-D slab benchmark problems with nonlinear temperature feedback effects. The proposed class of benchmarks include multiplicative 1-D neutron transport (limited to quasistatic S{sub 2} with μ = ±1) coupled with thermal conduction, convection, Doppler broadening, and expansion effects along the length of the slab. This class of benchmark models, along with the corresponding analytical solutions, are valuable for validating multiphysics analysis frameworks that support coupled neutronics/thermal/structural calculations. Analytical solutions for the benchmarks are obtained by introducing an ansatz that the equilibrium flux and temperature distributions in the slab have the same shape. Specific values for the total microscopic cross section, σ{sub t,0}, and conductive heat transfer coefficient, h, that satisfy the assumed ansatz are then determined. A discussion of the procedure for generating benchmark models and analytical solutions is provided, along with numerical results for an example set of model parameters and thoughts on practical applications of the benchmarks for multiphysics code validation. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS2024

Integrating State Data Assimilation and Innovative Model Parameterization Reduces Simulated Carbon Uptake in the Arctic and Boreal Region

Model representation of carbon uptake and storage is essential for accurate projection of the response of the arctic‐boreal zone to a rapidly changing climate. Land model estimates of LAI and aboveground biomass that can have a marked influence on model projections of carbon uptake and storage vary substantially in the arctic and boreal zone, making it challenging to correctly evaluate model estimates of Gross Primary Productivity (GPP). To understand and correct bias of LAI and aboveground biomass in the Community Land Model (CLM), we assimilated the 8‐day Moderate Resolution Imaging Spectroradiometer (MODIS) LAI observation and a machine learning product of annual aboveground biomass into CLM using an Ensemble Adjustment Kalman Filter (EAKF) in an experimental region including Alaska and Western Canada. Assimilating LAI and aboveground biomass reduced these model estimates by 58% and 72%, respectively. The change of aboveground biomass was consistent with independent estimates of canopy top height at both regional and site levels. The International Land Model Benchmarking system assessment showed that data assimilation significantly improved CLM's performance in simulating the carbon and hydrological cycles, as well as in representing the functional relationships between LAI and other variables. To further reduce the remaining bias in GPP after LAI bias correction, we re‐parameterized CLM to account for low temperature suppression of photosynthesis. The LAI bias corrected model that included the new parameterization showed the best agreement with model benchmarks. Combining data assimilation with model parameterization provides a useful framework to assess photosynthetic processes in LSMs.

land data assimilation↗
DOE OSTI2024

Integrating State Data Assimilation and Innovative Model Parameterization Reduces Simulated Carbon Uptake in the Arctic and Boreal Region

Model representation of carbon uptake and storage is essential for accurate projection of the response of the arctic-boreal zone to a rapidly changing climate. Land model estimates of LAI and aboveground biomass that can have a marked influence on model projections of carbon uptake and storage vary substantially in the arctic and boreal zone, making it challenging to correctly evaluate model estimates of Gross Primary Productivity (GPP). To understand and correct bias of LAI and aboveground biomass in the Community Land Model (CLM), we assimilated the 8-day Moderate Resolution Imaging Spectroradiometer (MODIS) LAI observation and a machine learning product of annual aboveground biomass into CLM using an Ensemble Adjustment Kalman Filter (EAKF) in an experimental region including Alaska and Western Canada. Assimilating LAI and aboveground biomass reduced these model estimates by 58% and 72%, respectively. The change of aboveground biomass was consistent with independent estimates of canopy top height at both regional and site levels. The International Land Model Benchmarking system assessment showed that data assimilation significantly improved CLM's performance in simulating the carbon and hydrological cycles, as well as in representing the functional relationships between LAI and other variables. Here, to further reduce the remaining bias in GPP after LAI bias correction, we re-parameterized CLM to account for low temperature suppression of photosynthesis. The LAI bias corrected model that included the new parameterization showed the best agreement with model benchmarks. Combining data assimilation with model parameterization provides a useful framework to assess photosynthetic processes in LSMs.

58 GEOSCIENCES↗