Multi-Model Uncertainty Quantification and Model Updating for the TMD Research Challenge
Explore the source record for details and available documents.
Ortiz, Jonel Aaron↗
SEARCH · Search NASA
Results for “uncertainty modeling”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Multi-Model Uncertainty Quantification and Model Updating for the TMD Research Challenge
Explore the source record for details and available documents.
Najera-Flores, David Aaron↗
Quantification of modeling uncertainty in the Rayleigh damping model
Understanding and accurately characterizing energy dissipation mechanisms in civil structures during earthquakes is an important element of seismic assessment and design. The most commonly used model is attributed to Rayleigh. This paper proposes a systematic approach to quantify the uncertainty associated with Rayleigh's damping model. Bayesian calibration with embedded model error is employed to treat the coefficients of the Rayleigh model as random variables using modal damping ratios. Through a numerical example, we illustrate how this approach works and how the calibrated model can address modeling uncertainty associated with the Rayleigh damping model.
42 ENGINEERING↗
Leveraging Active Subspaces to Capture Epistemic Model Uncertainty in Deep Generative Models for Molecular Design
Explore the source record for details and available documents.
97 MATHEMATICS AND COMPUTING↗
Simulating organic aerosol in Delhi with WRF-Chem using the volatility-basis-set approach: exploring model uncertainty with a Gaussian process emulator
The nature and origin of organic aerosol in the atmosphere remain unclear. The gas–particle partitioning of semi-volatile organic compounds (SVOCs) that constitute primary organic aerosols (POAs) and the multigenerational chemical aging of SVOCs are particularly poorly understood. The volatility basis set (VBS) approach, implemented in air quality models such as WRF-Chem (Weather Research and Forecasting model with Chemistry), can be a useful tool to describe emissions of POA and its chemical evolution. However, the evaluation of model uncertainty and the optimal model parameterization may be expensive to probe using only WRF-Chem simulations. Gaussian process emulators, trained on simulations from relatively few WRF-Chem simulations, are capable of reproducing model results and estimating the sources of model uncertainty within a defined range of model parameters. In this study, a WRF-Chem VBS parameterization is proposed; we then generate a perturbed parameter ensemble of 111 model runs, perturbing 10 parameters of the WRF-Chem model relating to organic aerosol emissions and the VBS oxidation reactions. This allowed us to cover the model's uncertainty space and to compare outputs from each run to aerosol mass spectrometer observations of organic aerosol concentrations and O:C ratios measured in New Delhi, India. The simulations spanned the organic aerosol concentrations measured with the aerosol mass spectrometer (AMS). However, they also highlighted potential structural errors in the model that may be related to unsuitable diurnal cycles in the emissions and/or failure to adequately represent the dynamics of the planetary boundary layer. While the structural errors prevented us from clearly identifying an optimized VBS approach in WRF-Chem, we were able to apply the emulator in the following two periods: the full period (1–29 May) and a subperiod period of 14:00–16:00 h LT (local time) on 1–29 May. The combination of emulator analysis and model evaluation metrics allowed us to identify plausible parameter combinations for the analyzed periods. We demonstrate that the methodology presented in this study can be used to determine the model uncertainty and to identify the appropriate parameter combination for the VBS approach and hence to provide valuable information to improve our understanding of OA production.
54 ENVIRONMENTAL SCIENCES↗
Methods for Incorporating Model Uncertainty into Exoplanet Atmospheric Analysis
A key goal of exoplanet spectroscopy is to measure atmospheric properties, such as abundances of chemical species, in order to connect them to our understanding of atmospheric physics and planet formation. In this new era of high-quality JWST data, it is paramount that these measurement methods are robust. When comparing atmospheric models to observations, multiple candidate models may produce reasonable fits to the data. Typically, conclusions are reached by selecting the best-performing model according to some metric. This ignores model uncertainty in favor of specific model assumptions, potentially leading to measured atmospheric properties that are overconfident and/or incorrect. In this paper, we compare three ensemble methods for addressing model uncertainty by combining posterior distributions from multiple analyses: Bayesian model averaging, a variant of Bayesian model averaging using leave-one-out predictive densities, and stacking of predictive distributions. We demonstrate these methods by fitting the Hubble Space Telescope (HST) + Spitzer transmission spectrum of the hot Jupiter HD 209458b using models with different cloud and haze prescriptions. All of our ensemble methods lead to uncertainties on retrieved parameters that are larger but more realistic and consistent with physical and chemical expectations. Since they have not typically accounted for model uncertainty, uncertainties of retrieved parameters from HST spectra have likely been underreported. We recommend stacking as the most robust model combination method. Our methods can be used to combine results from independent retrieval codes and from different models within one code. They are also widely applicable to other exoplanet analysis processes, such as combining results from different data reductions.
79 ASTRONOMY AND ASTROPHYSICS↗
ECAR-7932 Rev 0 Large Eddy Simulation of MARVEL Reactor Core Subchannel to Evaluate Model Uncertainty of Reynolds-Averaged Navier-Stokes Equation Based Computational Fluid Dynamics Analysis
In the previous work (ECAR-7210), the peak cladding temperature of the MARVEL microreactor has been evaluated by steady-state Reynolds-Averaged Navier-Stokes (RANS) based computational fluid dynamics (CFD) simulations. Although numerical uncertainties of RANS-based CFD simulations has been assessed in ECAR-7210, the model uncertainty of RANS turbulence models must be investigated to resolve the issues related to inaccurate prediction of turbulent heat flux and flow pulsation in a tight lattice rod bundle using the steady-state RANS simulations. Consequently, this ECAR conducted a high-fidelity CFD analysis utilizing Large Eddy Simulation (LES) to generate reference data and investigated the model uncertainty of RANS-based CFD simulations.
21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗
Urban Flood Modeling: Uncertainty Quantification and Physics‐Informed Gaussian Processes Regression Forecasting
Abstract Estimating uncertainty in flood model predictions is important for many applications, including risk assessment and flood forecasting. We focus on uncertainty in physics‐based urban flooding models. We consider the effects of the model's complexity and uncertainty in key input parameters. The effect of rainfall intensity on the uncertainty in water depth predictions is also studied. As a test study, we choose the Interconnected Channel and Pond Routing (ICPR) model of a part of the city of Minneapolis. The uncertainty in the ICPR model's predictions of the floodwater depth is quantified in terms of the ensemble variance using the multilevel Monte Carlo (MC) simulation method. Our results show that uncertainties in the studied domain are highly localized. Model simplifications, such as disregarding the groundwater flow, lead to overly confident predictions, that is, predictions that are both less accurate and uncertain than those of the more complex model. We find that for the same number of uncertain parameters, increasing the model resolution reduces uncertainty in the model predictions (and increases the MC method's computational cost). We employ the multilevel MC method to reduce the cost of estimating uncertainty in a high‐resolution ICPR model. Finally, we use the ensemble estimates of the mean and covariance of the flood depth for real‐time flood depth forecasting using the physics‐informed Gaussian process regression method. We show that even with few measurements, the proposed framework results in a more accurate forecast than that provided by the mean prediction of the ICPR model.
Kohanpur, Amir H.↗
Towards Improved Decision Making for the Smarter Grid: Geomagnetic Disturbance Implicit Modeling Uncertainty Quantification
Geomagnetic disturbances can greatly disrupt the power grid operation. This is caused by low-frequency changes in the earth’s magnetic field, which upon interaction with the deep earth conductivity induces a changing electric field at the earth’s surface. This in turn induces quasi-direct currents which have the potential to cause protective devices to operate, widespread damage to high-voltage transformers, or voltage collapse due to induced reactive power losses. While these impacts have been known since the 1940s, only recently have standards for GMD response been implemented. Because these standards are heavily based on models, it is important to understand when it is appropriate to make certain modeling assumptions. In this work, implicit modeling uncertainty is quantified for the purpose of enhanced decision making for disturbance impact mitigation. Considering electromagnetic models, modeling assumptions of non-uniform grounding, imbalanced operation, transformer saturation, and geomagnetically induced currents, voltage estimation values are analyzed. Statistical characteristics of modeling uncertainty are obtained by analysis of synthetically generated data. Results statistically characterize geomagnetic disturbances modeling uncertainty, providing an important tool for impact mitigation.
grid analysis↗
Characterizing model uncertainties in simulated coast-to-offshore wind over the northeast U.S. using multi-platform measurements from the TCAP field campaign
Numerical weather prediction (NWP) models, such as the Weather Research and Forecasting (WRF) model, are widely used to provide estimates of the offshore wind energy resource owing to their large spatial coverage compared to available observations. Nevertheless, spatiotemporal distribution of model biases is highly dependent on factors including model configuration, location, and the interplay of multi-scale physical processes. Here, in this study, we focus on the characterization of model uncertainties in simulated coast-to-offshore winds over the northeast U.S., by varying sea surface temperature (SST) forcings, surface layer (SL) and planetary boundary layer (PBL) parameterizations, as well as identifying biases that may be directly passed from initial and boundary conditions. Multiple measurements, including aircraft data collected during the U.S. Department of Energy's Two-Column Aerosol Project (TCAP) experiment, are used to constrain the model results and facilitate quantitative comparisons. Our analysis indicates while SST forcing has notable impacts on simulated air temperature and moisture within PBL, the modeled winds are in general more sensitive to the choices of SL and PBL physics than to SST. The model’s forcing data not only controls the vertical dependence of wind speed errors, but also alters regional variability in wind speed’s spatial correlation. Bias comparisons between ERA5 reanalysis and ensemble simulations revealed significant similarity, particularly in wind speed biases during winter, underscoring their dependency on initial and boundary conditions. Coastal and offshore near-surface wind speed biases tend to exhibit much higher similarity in winter than in summer due to the presence of much stronger and more persistent synoptic wind conditions. This study highlights the importance of accurate atmospheric forcing and parameterization choices in improving wind forecasts and suggests the potential for extrapolating coastal wind biases to offshore locations, aiding wind energy forecasting and informing the Wind Forecast Improvement Project-3 (WFIP3).
17 WIND ENERGY↗
An Entropy-Based Test and Development Framework for Uncertainty Modeling in Level-Set Visualizations
We present a simple comparative framework for testing and developing uncertainty modeling in uncertain marching cubes implementations. The selection of a model to represent the probability distribution of uncertain values directly influences the memory use, run time, and accuracy of an uncertainty visualization algorithm. We use an entropy calculation directly on ensemble data to establish an expected result and then compare the entropy from various probability models, including uniform, Gaussian, histogram, and quantile models. Our results verify that models matching the distribution of the ensemble indeed match the entropy. We further show that fewer bins in nonparametric histogram models are more effective whereas large numbers of bins in quantile models approach data accuracy.
Sisneros, Robert↗
How initial conditions-, structural-, and parameter-based model uncertainty interact and influence predictions in permafrost ecosystems: Modeling Archive
This dataset contains model output and input data, as well as source code examples for the Terrestrial Ecosystem Model with the Dynamic Vegetation Model and Dynamic Organic Soil (DVM-DOS-TEM) for the field sites Imnavait creek and the Bonanza creek Long Term Ecological Research Network (LTER). The data covers simulations from the last glacial maximum (LGM) until 2100 for a selection of paleo scenarios, setting the mean temperature of the LGM up to 10°C lower than pre-industrial conditions. The model structure was modulated to represent various model versions, and this dataset contains the relevant changes in the source code. The raw output data, the processed statistical data, the setup and processing scripts as well as parameter value distribution files from a parameter sensitivity analysis are included as well. Model outputs include active layer depth, organic soil carbon, soil layer depths, gross primary productivity (GPP) with and without nitrogen limitation, net primary productivity (NPP), soil liquid water content, heterotrophic, maintenance, and growth respiration, soil temperature, and vegetation carbon (*.nc files). The Next-Generation Ecosystem Experiments in the Arctic (NGEE Arctic) project is a research effort to reduce uncertainty in the Department of Energy’s Energy Exascale Earth System Model (E3SM) by developing a predictive understanding of Arctic tundra ecosystems underlain by permafrost and to quantify feedbacks from the Arctic tundra to the Earth system. NGEE Arctic is supported by the Department of Energy's Office of Biological and Environmental Research.Over Phases 1–3, observations made by the NGEE Arctic team across a gradient of permafrost landscapes in Arctic Alaska improved the representation of tundra processes in the land surface component of E3SM (the E3SM Land Model, ELM). Model improvements emphasized unique aspects of permafrost environments and explored reductions in model complexity while retaining predictive power. The Arctic-informed ELM developed by NGEE Arctic has been used to make novel predictions on processes ranging from permafrost thaw to soil biogeochemical cycling to Earth system feedbacks associated with the unique characteristics of tundra plants. In Phase 4, the NGEE Arctic team is evaluating our new predictive understanding under novel conditions across the Arctic domain. In collaboration with partners at long-term pan-Arctic research sites we are examining whether an Arctic-informed ELM can faithfully simulate interactions among surface and subsurface processes at site, regional, and pan-Arctic scales. In turn, we are using variety of tools to dynamically extend and evaluate ELM inference, with an emphasis on data synthesis and pan-Arctic model evaluation, reintegration of code with an evolving E3SM, scaling across heterogeneous Arctic landscapes, and the appropriate representation of the impacts of increasingly frequent Arctic disturbances.
54 ENVIRONMENTAL SCIENCES↗
Evaluating the Effects of Detector Modeling Uncertainties on Sterile Neutrino Oscillation Analysis with the ICARUS Detector
he ICARUS T600 LArTPC detector was refurbished after an initial run at the underground LNGS labs and is currently taking data within its experimental hall at Fermilab after full commissioning. Regular data taking began in May 2021 with neutrinos from the Booster Neutrino Beam (BNB) and the Neutrinos at the Main Injector (NuMI) off-axis beam. As the far detector of the Short-Baseline Neutrino (SBN) Program, the ICARUS detector's capability in searching for both muon neutrino disappearance and electron neutrino appearance will allow for unprecedented sensitivity to light sterile neutrinos with eV-scale mass. The ultimate sensitivity of the detector to sterile neutrino oscillations depends on the understanding of the detector response and the uncertainties remaining after calibrating the detector model. This poster will review how detector-model-related uncertainties are quantified, evaluated and their impact on expected sterile neutrino oscillation sensitivities using the existing ICARUS data. The work pursued here is broadly applicable to other ICARUS analysis pathways such as neutrino-argon cross-sections and Beyond the Standard Model physics searches. It will finally discuss improvement pathways to reduce the systematic uncertainties to the level needed for the joint sterile neutrino oscillation analysis with the Short Baseline Near Detector (SBND) within the SBN Program.
43 PARTICLE ACCELERATORS↗
Evaluating SWAT + model uncertainties for human and natural outcomes: Application in a Great Lakes agricultural watershed
Nutrient exports from agricultural lands in the Great Lakes Region pose significant threats to water quality and ecological health through eutrophication, hypoxia, and harmful algal blooms. Climate change and agricultural adaptation practices complicate future nutrient loading due to intensified hydrologic cycles and land use decisions. Our research focuses on evaluating the Soil and Water Assessment Tool (SWAT) plus model parametric uncertainties for human and natural outcomes across different scales. These factors are integral to ensuring a balance between productive agricultural practices and maintaining the health of watershed hydrology. However, uncertainties in modeling such complex interactions pose significant challenges, limiting our ability to precisely determine critical factors that influence crop yield and soil moisture. Our analysis employs Sobol global sensitivity analysis to evaluate first-order, second order, and total-order indices for SWAT crop growth parameter, ensuring comprehensive assessment of individual and interactive effects on model outputs. The objective is to identify the parameters that significantly affect model outputs for crop yield and soil moisture and improve our understanding of their interactions at the basin and hydrological response unit (HRU) scale. Our case study, the Portage River Watershed, which drains into Lake Erie, is chosen to better capture finer scale interactions crucial for predicting nutrient loading under future climate scenarios. This foundational work is aimed at setting the stage for the future development of an agent-based model (ABM). The ABM model would incorporate SWAT outputs to dynamically simulate decision-making processes.
Bunyon, Enock↗
Combined Data and Deep Learning Model Uncertainties: An Application to the Measurement of Solid Fuel Regression Rate
In complex physical process characterization, such as the measurement of the regression rate for solid hybrid rocket fuels, where both the observation data and the model used have uncertainties originating from multiple sources, combining these in a systematic way for quantities of interest (QoI) remains a challenge. In this paper, we present a forward propagation uncertainty quantification (UQ) process to produce a probabilistic distribution for the observed regression rate r. We characterized two input data uncertainty sources from the experiment (the distortion from the camera U c and the non-zero-angle fuel placement U Y ), the prediction and model form uncertainty from the deep neural network (U m ), as well as the variability from the manually segmented images used for training it (U s ). Here, we conducted seven case studies on combinations of these uncertainty sources with the model form uncertainty. The main contribution of this paper is the investigation and inclusion of the experimental image data uncertainties involved, and how to include them in a workflow when the QoI is the result of multiple sequential processes.
42 ENGINEERING↗
Impact of Systematic Modeling Uncertainties on Kilonova Property Estimation
The precise atomic structure and therefore the wavelength-dependent opacities of lanthanides are highly uncertain. This uncertainty introduces systematic errors in modeling transients like kilonovae and estimating key properties such as mass, characteristic velocity, and heavy metal content. Here, we quantify how atomic data from across the literature as well as choices of thermalization efficiency of r-process radioactive decay heating impact the light curve and spectra of kilonovae. Specifically, we analyze the spectra of a grid of models produced by the radiative transfer code Sedona that span the expected range of kilonova properties to identify regions with the highest systematic uncertainty. Our findings indicate that differences in atomic data have a substantial impact on estimates of lanthanide mass fraction, spanning approximately 1 order of magnitude for lanthanide-rich ejecta, and demonstrate the difficulty in precisely measuring the lanthanide fraction in lanthanide-poor ejecta. Mass estimates vary typically by 25%–40% for differing atomic data. Similarly, the choice of thermalization efficiency can affect mass estimates by 20%–50%. Observational properties such as color and decay rate are highly model dependent. Velocity estimation, when fitting solely based on the light curve, can have a typical error of ∼100%. Atomic data of light r-process elements can strongly affect blue emission. Even for well-observed events like GW170817, the total lanthanide production estimated using different atomic data sets can vary by a factor of ∼6.
Gravitational wave sources↗
Temperature Uncertainty Modeling with Proxy Structural Data as Geostatistical Constraints for Well Siting: An Example Applied to Granite Springs Valley, NV, USA
Utilizing existing temperature and structural information around Granite Springs Valley, Nevada, we build 3D stochastic temperature models with the aim of evaluating the 3D uncertainty of temperature and choosing between candidate exploration well locations . The data used to support the modeling are measured temperatures and structural proxies from 3D geologic modeling, the latter considered "secondary" data. Two stochastic geostatistical techniques are explored for incorporating the structural proxies: cosimulation and local varying mean. With both the cosimulation and local varying mean methods, many equally likely temperature models (i.e., realizations) are produced, from which temperature probability profiles are calculated at candidate well locations. To aid in choosing between the candidate locations, two quantities summarize the temperature probabilities: Vprior and entropy. Vprior quantifies the likelihood for economic temperatures at each candidate location, whereas entropy identifies where new information has the most potential to reduce uncertainty. In general, the cosimulation realizations have smoother spatial structure, and extrapolate high temperatures at candidate locations that are located along the direction of the longest spatial correlation, which are down dip from existing temperature logs. The smooth realizations result in tight temperature probability profiles that are easier to interpret, but they have unrealistic temperature reversals in some locations because the cosimulation technique does not enforce a conductive geothermal gradient as a baseline (i.e., linearly increasing temperature with depth). The local varying mean results produce realizations with more realistic geothermal gradients, with temperatures increasing downward since a depth-temperature relationship is included. However, because they have much noisier spatial nature compared to cosimulation, it is harder to interpret the temperature probability profiles. The different local varying mean results allow the geologist to determine which proxy (e.g., dilation versus distance to fault termination) should be used given the specific geothermal system. In general, Vprior from local varying mean results identify locations that are close to high values for the structural proxies: areas with highe r probabilities for higher temperatures. The entropy results identify where uncertainty is greatest and therefore new drilling information could be most useful. Though these techniques provide useful information, even when applied to areas of sparse data, our comp arison of these two techniques demonstrates the need for new geothermal geostatistics techniques that combine the advantages of these two methods and that are tailored to the spatial uncertainty issues inherent in geothermal exploration.
3D temperature modeling↗
Temperature uncertainty modelling with proxy structural data as geostatistical constraints for well siting: an example applied to Granite Springs Valley, NV, USA
Utilizing existing temperature and structural geology information around Granite Springs Valley, Nevada, we build 3D stochastic temperature models with the aims of evaluating the 3D uncertainty of temperature and choosing between candidate exploration well locations. The data used to support the modelling are measured temperatures and structural proxies from 3D geologic modelling (distance to fault, distance to fault intersections and terminations, Coulomb stress change and dilation tendency), the latter considered ‘secondary’ data. Two stochastic geostatistical techniques are explored for incorporating the structural proxies: cosimulation and local varying mean. With both the cosimulation and local varying mean methods, many equally-likely temperature models (i.e. realizations) are produced, from which temperature probability profiles are calculated at candidate well locations. To aid in choosing between the candidate locations, two quantities summarize the temperature probabilities: V prior and entropy. V prior quantifies the likelihood for economic temperatures at each candidate location, whereas entropy identifies where new information has the most potential to reduce uncertainty. In general, the cosimulation realizations have smoother spatial structure, and extrapolate high temperatures at candidate locations that are located along the direction of the longest spatial correlation, which are down dip from existing temperature logs. The smooth realizations result in tight temperature probability profiles that are easier to interpret, but they have unrealistic temperature reversals in some locations because of the dipping ellipsoid shape created and that the cosimulation technique does not enforce a conductive geothermal gradient as a baseline (i.e. linearly increasing temperature with depth). The local varying mean results produce realizations with more realistic geothermal gradients, with temperatures increasing downward since a depth-temperature relationship is included. However, because they have much noisier spatial nature compared to cosimulation, it is harder to interpret the temperature probability profiles. The different local varying mean results allow the geologist to determine which proxy (e.g. dilation v. distance to fault termination) should be used given the specific geothermal system. In general, V prior from local varying mean results identify locations that are close to high values for the structural proxies: areas with higher probabilities for higher temperatures. The entropy results identify where uncertainty is greatest and therefore new drilling information could be most useful. Though these techniques provide useful information, even when applied to areas of sparse data, our comparison of these two techniques demonstrates the need for new geothermal geostatistics techniques that combine the advantages of these two methods and that are tailored to the spatial uncertainty issues inherent in geothermal exploration.
15 GEOTHERMAL ENERGY↗