Characterizing model uncertainties in simulated coast-to-offshore wind over the northeast U.S. using multi-platform measurements from the TCAP field campaign
Explore the source record for details and available documents.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Context: This webinar was hosted by the MultiSector Dynamics Community of Practice (MSD CoP; https://multisectordynamics.org). Abstract: The energy transition is a game-changer across multiple sectors, requiring researchers and practitioners to re-evaluate the evolving connections between climate, water, and power systems, as well as the associated co-management of resources and decision-making. In anticipation of new technologies, policies, business models, and other solutions, applied science for decision-making at the climate-water-energy nexus is needed to support transitions with the expected climate resilience, maintained and enhanced energy security, thriving economies, and equity considerations. Through panelists' applied research and operational experience, the webinar will provide lesson-learnt in how to increase the readiness of water-energy research and become actionable in the context of energy transitions. A number of themes will be discussed, including the connection between growing computational resources and complexity of models, and the implications on the intended users, the actual actionable products, the availability of data for validation of the models, uncertainty characterization and decision-making under uncertainty, and the communication of the novelty to decision-makers and the public. Presenters : David McCollum (Oak Ridge National Laboratory; Co-Chair), and Gokul Iyer (Pacific Northwest National Laboratory; Co-Chair), Curt Jawdy (Tennessee Valley Authority), Nathalie Voisin (Pacific Northwest National Laboratory), and Andrew D. Jones (Lawrence Berkeley National Laboratory) Moderator: Pat M. Reed (MSD CoP Facilitation Team) This webinar was held on: April 30, 2024 from 1-2 PM ET
Navigating uncertainty is a critical challenge in all fields of science, especially when translating knowledge into real-world policies or management decisions. However, the wide variance in concepts and definitions of uncertainty across scientific fields hinders effective communication. As a microcosm of diverse fields within Earth Science, NASA’s Carbon Monitoring System (CMS) provides a useful crucible in which to identify cross-cutting concepts of uncertainty. The CMS convened the Uncertainty Working Group (UWG), a group of specialists across disciplines, to evaluate and synthesize efforts to characterize uncertainty in CMS projects. This paper represents efforts by the UWG to build a heuristic framework designed to evaluate data products and communicate uncertainty to both scientific and non-scientific end users. We consider four pillars of uncertainty: origins, severity, stochasticity versus incomplete knowledge, and spatial and temporal autocorrelation. Using a common vocabulary and a generalized workflow, the framework introduces a graphical heuristic accompanied by a narrative, exemplified through contrasting case studies. Envisioned as a versatile tool, this framework provides clarity in reporting uncertainty, guiding users and tempering expectations. Beyond CMS, it stands as a simple yet powerful means to communicate uncertainty across diverse scientific communities.
The following assessment evaluates the efficacy of the control script for carrying out a linear signal to generate a linear mechanical response of the system: the TRUST nonlinear dynamics (TRUSTND) testbed. The hardware has three main components: the controller (NI PXIe 8861) embedded in the National Instruments chassis (NI PXIe 1092), the signal amplifier (The Modal Shop Linear Power Amplifier 2050E09) to amplify the output of a custom LabVIEW script, and a shaker (The Modal Shop Electrodynamic Exciter 2075E) where wave spring specimens live and are tested within the attached aluminum testbed (center mass). Different kinds of signals (single-tone, swept frequency and white noise) were sent through this hardware in the E-1 lab space at TA-53. First, the linearity of the setup was tested by specifying a sine wave in the control script and checking the quality of the oscillations in a wave spring (McMaster-Carr 9714K19 [1]). When passing a sine wave through the amplifier was succeeded, the frequency response of the wave spring was tested with white noise for comparison to finite-element model predictions. The testbed outputs a repeatable, linear response to the input excitation when used with the wave spring. These measurements reduce the uncertainty associated with the testbed and test procedure to better characterize uncertainty associated with material behavior (SX358 foam samples) in future tests
This project successfully developed a multiphysics, multiscale computational framework to enhance the design and development of CMCs, with a focus on modeling highly nonlinear, time-dependent damage mechanisms and material degradation under extreme conditions, such as those experienced in turbine service environments. The project made significant advances in improving our understanding of progressive damage, oxidative degradation, and time-dependent inelastic deformation in CMCs, with particular attention to the role of uncertainties in predictions. Key outcomes include the integration of advanced material characterization, uncertainty quantification, and multiphysics constitutive models to predict the behavior of CMCs over their service life. A novel multiscale methodology was employed, which integrated microscale constituent behaviors with structural-scale responses, enabling the manufacturing defects in the microstructure that are prone to damage nucleation. Through the development of DL algorithms, the project advanced the prediction of damage initiation and crack propagation, taking into account the defect morphology and statistical variations across multiple scales. The framework was rigorously validated using thermomechanical experiments, which tested CMCs under various mechanical loadings at elevated temperatures, further enhancing the model's predictive capability. Overall, the research outcomes have provided a more accurate, reliable method for predicting CMC component life, significantly advancing material design, and improving component reliability in extreme environments. This work has strong implications for the optimization of turbine components and other high-performance applications where CMCs are used.
An interlaboratory effort has developed a probabilistic framework to characterize uncertainty in data products that are developed by the US Department of Energy Consequence Management Program in support of the Federal Radiological Monitoring and Assessment Center. The purpose of this paper is to provide an overview of the probability distributions of input variables and the statistical methods used to propagate and quantify the overall uncertainty of the derived response levels that are used as contours on data products due to the uncertainty in input parameters. Uncertainty analysis results are also presented for several study scenarios. This paper includes an example data product to illustrate the potential real-world implications of incorporating uncertainty analysis results into data products that inform protective action decisions. Data product contours that indicate areas where public protection actions may be warranted can be customized to an acceptable level of uncertainty. Finally, the investigators seek feedback from decision makers and the radiological emergency response community to determine how uncertainty information can be used to support the protective action decision-making process and how it can be presented on data products.
Sustained advances in the mathematics of modeling and simulation have resulted in the capability today for routine simulation of a number of large scale complex DOE-relevant systems. As remarkable as this capability for solving the so-called forward problem is, it is typically only the first step-an inner loop within an outer loop that explores the simulation model's parameter space and decision space to characterize uncertainty in the model's predictions, learn unknown model parameters from data, design the most informative experiments, determine optimal control strategies, and create optimal designs. Broadly, what unifies all of these outer loop problems is that they are, in one form or another, optimization problems over parameter/control/design space that are constrained by complex uncertain models. To fully realize the power of scientific simulation as a basis for scientific discovery, technological innovation, and rational decision-making, it is imperative to move beyond simulation to tackle the outer loop of optimization for learning from data, experimental design, and control with complex uncertain models. When the models under consideration are large-scale and complex, and when the optimization variable and uncertain parameter spaces are high (or infinite) dimensional, this constitutes a grand challenge of the highest order, and is intractable with conventional methods. To overcome these challenges, the AEOLUS Center was established to develop a unified mathematical, computational, and statistical framework for (1) Learning predictive models from complex data via Bayesian inference and optimization, and (2) Optimizing experiments, processes, and designs using the resulting uncertain models. These problems are intractable with conventional methods, for several reasons: (1) The simulation problems that govern the inner loops of the optimization problems are expensive to execute (due to severe nonlinearity, heterogeneity, multiphysics/multiscale coupling); (2) The optimization variable and uncertain parameter spaces are high dimensional, often stemming from discretizations of infinite dimensional fields such as initial conditions, sources, or material properties. We argue that the key to overcoming these challenges is to develop new mathematical, computational, and statistical methods that exploit the structure of the Bayesian inference and optimization problems mediated by their underlying complex uncertain models. This structure includes the regularity, sparsity, geometry, low intrinsic dimensionality, and multifidelity nature of the maps from uncertain parameter/optimization variable spaces to the specific objectives targeted: Bayesian inference, optimal experimental design, and optimal control design. Black box methods developed as generic tools are incapable of exploiting this structure. To be successful, we must create, integrate, and cross-fertilize ideas across multiple areas of applied math--including approximation theory, Bayesian inference, data science, experimental design, information theory, machine learning, model reduction, optimal control theory, parallel algorithms, PDE-constrained optimization, randomized algorithms, stochastic optimization, and uncertainty quantification--all while exploiting the structure of the problems at hand. With this goal in mind, we have marshaled a team of leading authorities in these areas. While the methods we develop will be broadly applicable across a wide spectrum of DOE problems in which experiments inform models and the systems those models describe must be optimized under uncertainty, we have chosen a specific area, advanced manufacturing and materials, to drive our work. AMM is characterized by complex models across multiple scales, and is a rich source of challenging problems in inference, experimental design, and optimal control, requiring multifaceted and integrated advances in applied mathematics. As such, AMM serves as an excellent vehicle to motivate and demonstrate the advances in applied mathematics developed by our center.
Uncertainty is a key metric in computational modeling that must be evaluated for results to have wide ranging applicability. A well characterized uncertainty range is ideal with clear error bars on results that can be presented to stakeholders. In the field of spent fuel cask modeling, this ideal has been historically difficult to achieve in practice because of the computationally intensive nature of the models used and the difficulty assigning reasonable uncertainties to quantities in as-built systems. The work in this report has been conducted to evaluate the overall state of uncertainty and sensitivity in spent fuel cask models and develop methodologies for evaluating these uncertainties. These methodologies must be practical for engineering applications. They should not require excessive computational resources or calendar time to achieve results. In engineering, the model must be on a scale such that it can be changed and adapted throughout a project as new information is discovered and project goals evolve. This report covers three major modeling task areas that provide an overview of the types of sensitivity and uncertainty present in a spent fuel storage and transportation system. Section 3 discusses sensitivity and uncertainty analysis in the effective thermal conductivity model for the fuel region and applies these results to a single assembly model. Section 4 shows sensitivity analysis of a full cask model in the TN-32B and Section 5 demonstrates the overall uncertainty workflow using Coolant Boiling in Rod Arrays – Spent Fuel Storage and STAR-CCM+ developed from the sensitivity work in the preceding sections.
We synthesize five complementary tropical cyclone (TC) hazard frameworks—RAFT (physics-based machine learning), CHAZ and MIT (statistical–dynamical), STORM (fully statistical), and CLIMADA (observation-driven resampling)—to characterize uncertainty in wind-related TC metrics relevant to energy applications. All datasets and the IBTrACS observational record are harmonized to a common 6-hourly, 2.5° grid. We compare basin-wide and coastal properties using consistent definitions for TC frequency, mean and maximum intensity, 24-hour intensification, and 6-hour translation speed, and quantify agreement with Pearson r, RMSE, and Kling–Gupta efficiency (KGE) alongside resampling-based confidence intervals. CLIMADA is included for basin context but excluded from coastal skill scoring because it resamples historical IBTrACS; if supplied with projected future tracks from an external hazard model, CLIMADA can be used to simulate future TC scenarios. Results show robust, cross-model signals: (i) a corridor of activity from the tropical Atlantic through the Caribbean into the Bahamas and western subtropical Atlantic; (ii) a meridional dipole in 24-hour intensification (low-latitude strengthening, subtropical weakening); and (iii) a transition from slower tropical motion to faster midlatitude translation. Coastal winds (mean and maximum) consistently cluster from the eastern Gulf into the Bahamas–western Atlantic transition. The largest structural spread occurs in the amplitude and footprint of lifetime maximum intensity and, secondarily, in translation speed; intensification exhibits similar central behavior across frameworks with variability in extremes. Translation speed shows the most uniform coastal agreement. These findings provide a decision envelope for wind-focused risk screening and clarify where uncertainty should be carried forward; wind-only results represent a lower bound on total hazard, motivating integration of surge and rainfall modules and a companion, asset-level damage analysis.
Stratospheric aerosols, most of which originate from explosive volcanic sulfur emissions into the stratosphere, are a key natural driver of climate variability. They are thus a forcing provided by the Coupled Model Intercomparison Project (CMIP) Climate Forcings Task Team to climate modelling groups participating in phase 7 of CMIP. For the historical period, we provide two datasets covering 1750–2023: (i) a volcanic upper tropospheric-stratospheric sulfur emission dataset, documented in a companion paper; and (ii) a stratospheric sulfate aerosol optical property dataset, documented in this paper. For the satellite era (from 1979 onwards), stratospheric aerosol optical properties are derived from the Global Space-based Stratospheric Aerosol Climatology (GloSSAC) dataset. For the pre-satellite era (1750–1978), optical properties are derived from our volcanic SO 2 emission dataset using a new version of the reduced-complexity volcanic aerosol model Easy Volcanic Aerosol (Height) (EVA_H v2). A background, non-volcanic stratospheric aerosol climatology is derived from the 1998–2001 period with a trend over 1850–1978 accounting for increasing anthropogenic aerosols. A monthly stratospheric aerosol climatology is derived from the 1850–2021 average for both pre-industrial and scenario (future) simulations, with a 9-year ramp over 2022–2030 for scenario simulations to ensure a smooth transition from the historical period. CMIP7 simulations should use version 2.2.1 of our pre-industrial and historical datasets, and version 2.2.2 of our scenario dataset. Our methodology to produce historical aerosol optical properties significantly differs from CMIP6 for the pre-satellite era, and the resulting forcings in turn largely differ. In particular, the CMIP6 dataset was mostly based on the sparse and uncertain pyrheliometer record, which resulted in strongly underrepresented emissions from small-to-moderate magnitude eruptions. The resulting bias is addressed in CMIP7, which is entirely emission-derived in the pre-satellite era and uses more recent ice-core-based volcanic sulfur emission inventories than CMIP6. Our approach results in an overall larger volcanic aerosol forcing for CMIP7, with the 1850–2014 mean mid-visible global mean stratospheric aerosol optical depth (SAOD) in CMIP7 (0.0138) being 29 % higher than in CMIP6 (0.0107). The pre-industrial mean of the same variable is 26 % higher in CMIP7 (0.0135, derived from the historical 1850–2021) than CMIP6 (0.0107, derived from the historical 1850–2014 mean). Using a reduced-complexity climate model, we simulate a global mean surface temperature that is 0.07 °C colder for 1850–1900 when using the CMIP7 dataset instead of CMIP6, whereas 2000–2014 is 0.03 °C warmer in CMIP7. Our dataset also exhibits lower forcing for 1960–1980, resulting in temperatures 0.06 °C warmer when averaged over 1960–1990, a period for which CMIP6 climate models exhibit a cold bias. Given the large uncertainties characterizing the dataset, in particular for the pre-satellite era, we advise against treating the CMIP7 or CMIP6 dataset as uniquely superior for any specific year and highlight the need for further evaluation. We conclude the study by discussing sources of uncertainty for the dataset, future research avenues to improve it, as well as requirements to operationalize the production of the dataset, i.e. extend it and update it on an annual basis instead of every 5–7 years following CMIP cycles.
Global Sensitivity Analysis (GSA) aims to understand the relative importance of uncertain input variables to model response. Conventional GSA involves calculating sensitivity (Sobol’) indices for a model with known model parameter distributions. However, model parameters are affected by aleatory and epistemic uncertainty, with the latter often caused by lack of data. In this paper, we propose a new framework to quantify uncertainty in probability model-form and model parameters resulting from small datasets and integrate these uncertainties into Sobol’ index estimates. First, the process establishes, through Bayesian multimodel inference, a set of candidate probability models and their associated probabilities. Imprecise Sobol’ indices are calculated from these probability models using an importance sampling reweighting approach. This results in probabilistic Sobol’ indices, whose distribution characterizes uncertainty in the sensitivity resulting from small dataset size. The imprecise Sobol’ indices thus provide a measure of confidence in the sensitivity estimate and, moreover, can be used to inform data collection efforts targeted to minimize the impact of uncertainties. Through an example studying the parameters of a Timoshenko beam, we show that these probabilistic Sobol’ indices converge to the true/deterministic Sobol’ indices as the dataset size increases and hence, distribution-form uncertainty reduces. The approach is then applied to assess the sensitivity of the out-of-plane properties of an E-glass fiber composite material to its constituent properties. This second example illustrates the approach for an important class of materials with wide-ranging applications when data may be lacking for some input parameters.
Computational and mathematical models are essential to understanding complex systems and phenomena. However, when developing such models, limited knowledge and/or resources necessitates the use of simplifying assumptions. It is therefore crucial to quantify the impact of such simplifying assumptions on the reliability and accuracy of resulting model predictions. This work develops a first-of-its-kind approach to quantify the impact of physics modeling assumptions on predictions. Here, we leverage the emerging field of model-form uncertainty (MFU) representations, which are parameterized modifications to modeling assumptions, in combination with grouped Sobol’ indices to quantitatively measure an assumption’s importance. Specifically, we compute the grouped Sobol’ index for the MFU representation’s parameters as a single importance measure of the assumption for which the MFU representation characterizes uncertainty. To ensure this approach is robust to the subjective choice of how to parameterize a MFU representation, we establish bounds for the difference between sensitivity results for two different MFU representations based on differences in model prediction statistics. The capabilities associated with this approach are demonstrated on three exemplar problems: an upscaled subsurface contaminant transport problem, ablation modeling for hypersonic flight, and nuclear waste repository modeling. We found that our grouped approach is able to assess the impact of modeling assumptions on predictions and offers computational advantages over classical Sobol’ index computation while providing more interpretable results.
There is a dearth in the literature on how to capture the uncertainty generated by material surface evolution in thermal modeling. This leads to inadequate or highly variable uncertainty representations for material properties, specifically emissivity when minimal information is available. Inaccurate understandings of prediction uncertainties may lead decision makers to incorrect conclusions, so best engineering practices should be developed for this domain. In order to mitigate the aforementioned issues, this study explores different strategies to better capture the thermal uncertainty response of engineered systems exposed to fire environments via defensible emissivity uncertainty characterizations that can be easily adapted to a variety of use cases. Here, two unique formulations (one physics-informed and one mathematically based) are presented. The formulations and methodologies presented herein are not exhaustive but more so are a starting point and give the reader a basis for how to customize their uncertainty definitions for differing fire scenarios and materials. Lastly, the impact of using this approach versus other commonly used strategies and the usefulness of adding rigor to material surface evolution uncertainty is demonstrated.
Radon is a naturally occurring radioactive gas that poses a serious health risk as the primary cause of lung cancer in non-smokers. Despite the well-known adverse association with health outcomes, current radon exposure assessments are limited to county-level or average-level estimates, which fail to capture regional variability. This study uses Machine Learning models, including Random Forest (RF) and Quantile Regression Forest (QRF), to estimate the indoor radon concentrations at the ZCTA (Zip code tabulation area)-level and characterize uncertainties in model estimates. Incorporating geological, meteorological, and building-specific data, the models aim to improve radon risk assessment by capturing mean exposure, variability, and extreme concentration levels. Processed radon test data (n = 718,111) were analyzed using average, variability, and quantile prediction methods. Models that estimate the average radon exposure at the ZCTA-level can yield promising model-fit results, but they do not capture the underlying variability of indoor radon exposure within a ZCTA. We utilize volatility analyses to identify characteristics indicative of high variability of indoor radon exposure. We also show that a QRF model can be used to estimate upper quantiles of residential radon exposure, thereby uncovering localized areas of elevated exposure that were not apparent in mean estimates. The results highlighted the need for a deep characterization of exposure risk and show that regions with moderate average exposure levels could still harbor extreme outliers with implications for evaluating health risks. Utilizing multiple radon exposure models allows for a deeper characterization of radon risk within a geographic area and can better identify high-risk areas. The results from this study provide a foundation for developing mitigation strategies and examining associations between radon exposure and health outcomes at fine scales. Future research should extend the geographic scope and incorporate additional environmental risk factors to establish a comprehensive framework for risk assessment.
Land surface models such as Community Land Model Version 5 (CLM5) are essential tools for simulating the behaviors of the terrestrial system. Despite the extensive application of CLM5, limited attention has been paid to the underlying uncertainties associated with its hydrologic parameters and the implications that these uncertainties have on water resources applications. To address this long-standing issue, we conduct a comprehensive hydrologic parameter uncertainty characterization (UC) of CLM5 over the hydroclimatic gradients of the Conterminous United States using five meteorological datasets. Key datasets produced from the UC experiment include a benchmark dataset of CLM5 default hydrological performance, parameter sensitivity identified for 28 hydrological metrics, and large ensemble outputs for hydrological predictions. The presented datasets can assist CLM5 calibration and to support broad applications such as evaluating vulnerabilities to droughts and floods. The dataset can be used to identify under what hydroclimate conditions parametric uncertainties demonstrate substantial effects on hydrological predictions and clarify where further investigations are needed to understand how land runoff uncertainties interact with other Earth system processes. Please refer to the included README.pdf for a description of the included files. Note that raw CLM5 model outputs for each forcing dataset are hosted on a Globus endpoint: https://app.globus.org/file-manager?destination_id=d22ef858-27b0-11ed-a910-fd3165076336.
Land surface models such as Community Land Model Version 5 (CLM5) are essential tools for simulating the behaviors of the terrestrial system. Despite the extensive application of CLM5, limited attention has been paid to the underlying uncertainties associated with its hydrologic parameters and the implications that these uncertainties have on water resources applications. To address this long-standing issue, we conduct a comprehensive hydrologic parameter uncertainty characterization (UC) of CLM5 over the hydroclimatic gradients of the Conterminous United States using five meteorological datasets. Key datasets produced from the UC experiment include a benchmark dataset of CLM5 default hydrological performance, parameter sensitivity identified for 28 hydrological metrics, and large ensemble outputs for hydrological predictions. The presented datasets can assist CLM5 calibration and to support broad applications such as evaluating vulnerabilities to droughts and floods. The dataset can be used to identify under what hydroclimate conditions parametric uncertainties demonstrate substantial effects on hydrological predictions and clarify where further investigations are needed to understand how land runoff uncertainties interact with other Earth system processes. Please refer to the README.pdf for a description of the included files. Note that raw CLM5 model outputs for each forcing dataset are hosted on a Globus endpoint: https://app.globus.org/file-manager?destination_id=d22ef858-27b0-11ed-a910-fd3165076336.
This proposal is emerging from GISS ModelE3 ESM development in the area of cloud physics, so we begin with an example of research needs/gaps from that work. Here, some of our greatest development concerns arise where we lack fundamental process-level understanding, as in ice formation. Namely, it is currently unclear what is the main process that is forming the majority of ice crystals in commonly occurring convection, apparently via secondary ice production at warm temperatures. We are keenly awaiting laboratory data for candidate mechanisms, which is not yet in hand to crucially establish their efficiency. Our progress is also hampered by a lack of uncertainty characterization in currently available measurements of ice crystal number size distributions. Furthermore, the same multiplication process may be responsible for a majority of ice crystals in many extratropical mixed-phase clouds, whose variable representation in CMIP6 ESMs may be a leading cause of differences in cloud phase feedback and ECS. Yet we have been required to deliver an ESM with the cloud physics knowledge at hand. The proposed grand challenge project is AI-enabled via application of machine learning (ML) to climate model and observational data streams (focal area 3), and applications include AI-guided observing system design and model/component/parameterization selection (areas 1 and 2). The project is structurally agnostic as to whether model or observing system components use AI approaches or not, but uncertainties must be estimated and propagatable in both.
Abstract The increasingly urgent need to develop knowledge and practices to manage flood risks drives innovative information design. However, experts often disagree about design practices. As a result, flood‐risk estimates can diverge, leading to different conclusions for decision‐making. Using examples of household‐scale fluvial (riverine) flood‐risk information in the United States, we assess design features and risk communication approaches that may lead to more actionable information for decision‐making. We argue that increased attention to uncertainty characterization and model diagnostics is a critical intermediate step for developing simpler approaches for designing flood‐risk information. Simpler frameworks are desirable because flood risks change over time, and simpler frameworks are less costly to update. Developing frameworks for large spatial domains require collaboration grounded in principles of open science. Finally, systematically evaluating how decision‐makers access and use information can provide new insights to guide risk communication and information design.