Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model Uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,297 records · Page 72

Predicting long-term carbon sequestration in response to CO 2 enrichment: How and why do current ecosystem models differ?

Large uncertainty exists in model projections of the land carbon (C) sink response to increasing atmospheric CO 2 . Free-Air CO 2 Enrichment (FACE) experiments lasting a decade or more have investigated ecosystem responses to a step change in atmospheric CO 2 concentration. To interpret FACE results in the context of gradual increases in atmospheric CO 2 over decades to centuries, we used a suite of seven models to simulate the Duke and Oak Ridge FACE experiments extended for 300 years of CO 2 enrichment. We also determine key modeling assumptions that drive divergent projections of terrestrial C uptake and evaluate whether these assumptions can be constrained by experimental evidence. All models simulated increased terrestrial C pools resulting from CO 2 enrichment, though there was substantial variability in quasi-equilibrium C sequestration and rates of change. In two of two models that assume that plant nitrogen (N) uptake is solely a function of soil N supply, the net primary production response to elevated CO 2 became progressively N limited. In four of five models that assume that N uptake is a function of both soil N supply and plant N demand, elevated CO 2 led to reduced ecosystem N losses and thus progressively relaxed nitrogen limitation. Many allocation assumptions resulted in increased wood allocation relative to leaves and roots which reduced the vegetation turnover rate and increased C sequestration. Additionally, self-thinning assumptions had a substantial impact on C sequestration in two models. As a result, accurate representation of N process dynamics (in particular N uptake), allocation, and forest self-thinning is key to minimizing uncertainty in projections of future C sequestration in response to elevated atmospheric CO 2 .

54 ENVIRONMENTAL SCIENCES↗

Statistical analysis of the modal properties of large structural systems.

A theory is developed to predict eigenvalue and eigenvector uncertainty in large dynamic models. The uncertainty is based on physical property uncertainty and should not be confused with numerical roundoff, although the method can be extended to include the latter. The theory, when implemented on a computer, is used to analyze the uncertainties in frequencies and mode shapes based on uncertainties in mass, stiffness, modulus of elasticity, etc. The method incorporates a linear statistical model which is quite adequate for handling property uncertainties of 10% or more. The model is not limited to small systems but uses certain statistical assumptions as well as selective matrix manipulations to keep the size of all matrix operations to within the number of degrees of freedom of the system. Examples are given for two longitudinal vibration problems, and the results are supported by a Monte Carlo analysis.

Collins, J. D.↗

LLNL ARRA Report on CCSI Uncertainty Quantification Capabilities with Applications on MEA Process Models

The Carbon Capture Simulation Initiative (CCSI) is a partnership among national laboratories, industry and academic institutions that will develop and deploy state-of-the-art computational modeling and simulation tools to accelerate the commercialization of carbon capture technologies from discovery to development, demonstration, and ultimately the widespread deployment to hundreds of power plants. The CCSI Toolset will provide end users in industry with a comprehensive, integrated suite of scientifically validated models with uncertainty quantification, optimization, risk analysis and decision making capabilities. The CCSI Toolset will incorporate commercial and open-source software currently in use by industry and will also develop new software tools as necessary to fill technology gaps identified during execution of the project. The CCSI Toolset will (1) enable promising concepts to be more quickly identified through rapid computational screening of devices and processes; (2) reduce the time to design and troubleshoot new devices and processes; (3) quantify the technical risk in taking technology from laboratory-scale to commercial-scale; and (4) stabilize deployment costs more quickly by replacing some of the physical operational tests with virtual power plant simulations. The goal of CCSI is to deliver a toolset that can simulate the scale-up of a broad set of new carbon capture technologies from laboratory scale to full commercial scale. To provide a framework around which the toolset can be developed and demonstrated, we will focus on three Industrial Challenge Problems (ICPs) related to carbon capture technologies relevant to U.S. pulverized coal (PC) power plants. Post combustion capture by solid sorbents is the technology focus of the initial ICP (referred to as ICP A). The goal of the uncertainty quantification (UQ) task (Task 6) is to provide a set of capabilities to the user community for the quantification of uncertainties associated with the carbon capture processes. As such, we will develop, as needed and beyond existing capabilities, a suite of robust and efficient computational tools for UQ to be integrated into a CCSI UQ software framework.

97 MATHEMATICS AND COMPUTING↗

Uncertainty quantification for neural network potential foundation models

Abstract For neural network potentials (NNPs) to gain widespread use, researchers must be able to trust model outputs. However, the blackbox nature of neural networks and their inherent stochasticity are often deterrents, especially for foundation models trained over broad swaths of chemical space. Uncertainty information provided at the time of prediction can help reduce aversion to NNPs. In this work, we detail two uncertainty quantification (UQ) methods. Readout ensembling, by finetuning the readout layers of an ensemble of foundation models, provides information about model uncertainty, while quantile regression, by replacing point predictions with distributional predictions, provides information about uncertainty within the underlying training data. We demonstrate our approach with the MACE-MP-0 model, applying UQ to the foundation model and a series of finetuned models. The uncertainties produced by the readout ensemble and quantile methods are demonstrated to be distinct measures by which the quality of the NNP output can be judged.

36 MATERIALS SCIENCE↗

Calibration under uncertainty for finite element models of masonry monuments

Historical unreinforced masonry buildings often include features such as load bearing unreinforced masonry vaults and their supporting framework of piers, fill, buttresses, and walls. The masonry vaults of such buildings are among the most vulnerable structural components and certainly among the most challenging to analyze. The versatility of finite element (FE) analyses in incorporating various constitutive laws, as well as practically all geometric configurations, has resulted in the widespread use of the FE method for the analysis of complex unreinforced masonry structures over the last three decades. However, an FE model is only as accurate as its input parameters, and there are two fundamental challenges while defining FE model input parameters: (1) material properties and (2) support conditions. The difficulties in defining these two aspects of the FE model arise from the lack of knowledge in the common engineering understanding of masonry behavior. As a result, engineers are unable to define these FE model input parameters with certainty, and, inevitably, uncertainties are introduced to the FE model.

42 ENGINEERING↗

Uncertainty Quantification of Turbulence Model Closure Coefficients for Transonic Wall-Bounded Flows

The goal of this work was to quantify the uncertainty and sensitivity of commonly used turbulence models in Reynolds-Averaged Navier-Stokes codes due to uncertainty in the values of closure coefficients for transonic, wall-bounded flows and to rank the contribution of each coefficient to uncertainty in various output flow quantities of interest. Specifically, uncertainty quantification of turbulence model closure coefficients was performed for transonic flow over an axisymmetric bump at zero degrees angle of attack and the RAE 2822 transonic airfoil at a lift coefficient of 0.744. Three turbulence models were considered: the Spalart-Allmaras Model, Wilcox (2006) k-w Model, and the Menter Shear-Stress Trans- port Model. The FUN3D code developed by NASA Langley Research Center was used as the flow solver. The uncertainty quantification analysis employed stochastic expansions based on non-intrusive polynomial chaos as an efficient means of uncertainty propagation. Several integrated and point-quantities are considered as uncertain outputs for both CFD problems. All closure coefficients were treated as epistemic uncertain variables represented with intervals. Sobol indices were used to rank the relative contributions of each closure coefficient to the total uncertainty in the output quantities of interest. This study identified a number of closure coefficients for each turbulence model for which more information will reduce the amount of uncertainty in the output significantly for transonic, wall-bounded flows.

Schaefer, John↗

Hierarchical Multi-Scale Approach To Validation and Uncertainty Quantification of Hyper-Spectral Image Modeling

Validating predictive models and quantifying uncertainties inherent in the modeling process is a critical component of the HARD Solids Venture program [1]. Our current research focuses on validating physics-based models predicting the optical properties of solid materials for arbitrary surface morphologies and characterizing the uncertainties in these models. We employ a systematic and hierarchical approach by designing physical experiments and comparing the experimental results with the outputs of computational predictive models. We illustrate this approach through an example comparing a micro-scale forward model to an idealized solid-material system and then propagating the results through a system model to the sensor level. Our efforts should enhance detection reliability of the hyper-spectral imaging technique and the confidence in model utilization and model outputs by users and stakeholders.

Engel, David W.↗

Parametric Sensitivity and Uncertainty Quantification in the Version 1 of E3SM Atmosphere Model Based on Short Perturbed Parameter Ensemble Simulations

Abstract The atmospheric component of Energy Exascale Earth System Model version 1 has included many new features in the physics parameterizations compared to its predecessors. Potential complex nonlinear interactions among the new features create a significant challenge for understanding the model behaviors and parameter tuning. Using the one‐at‐a‐time method, the benefit of tuning one parameter may offset the benefit of tuning another parameter, or improvement in one target variable may lead to degradation in another target variable. To better understand the Energy Exascale Earth System Model version 1 model behaviors and physics, we conducted a large number of short simulations (three days) in which 18 parameters carefully selected from parameterizations of deep convection, shallow convection, and cloud macrophysics and microphysics were perturbed simultaneously using the Latin hypercube sampling method. From the perturbed parameter ensemble simulations and use of different skill score functions, we identified the most sensitive parameters, quantified how the model responds to changes of the parameters for both global mean and spatial distribution, and estimated the maximum likelihood of model parameter space for a number of important fidelity metrics. Comparison of the parametric sensitivity using simulations of two different lengths suggests that perturbed parameter ensemble using short simulations has some bearing on understanding parametric sensitivity of longer simulations. Results from this analysis provide a more comprehensive picture of the Energy Exascale Earth System Model version 1 behavior. The difficulty in reducing biases in multiple variables simultaneously highlights the need of characterizing model structural uncertainty (so‐called embedded errors) to inform future development efforts.

54 ENVIRONMENTAL SCIENCES↗

Attributing runoff changes to climate variability and human activities: Uncertainty analysis using four monthly water balance models

Hydrological simulations to delineate the impacts of climate variability and human activities are subjected to uncertainties related to both parameter and structure of the hydrological models. To analyze the impact of these uncertainties on the model performance and to yield more reliable simulation results, a global calibration and multimodel combination method that integrates the Shuffled Complex Evolution Metropolis (SCEM) and Bayesian Model Averaging (BMA) of four monthly water balance models was proposed. The method was applied to the Weihe River Basin (WRB), the largest tributary of the Yellow River, to determine the contribution of climate variability and human activities to runoff changes. The change point, which was used to determine the baseline period (1956-1990) and human-impacted period (1991-2009), was derived using both cumulative curve and Pettitt’s test. Results show that the combination method from SCEM provides more skillful deterministic predictions than the best calibrated individual model, resulting in the smallest uncertainty interval of runoff changes attributed to climate variability and human activities. This combination methodology provides a practical and flexible tool for attribution of runoff changes to climate variability and human activities by hydrological models.

runoff change, uncertainty assessment, monthly wat↗

Uncertainty Quantification and Sensitivity Analysis of Nuclear Construction Cost Reduction Pathways

High construction costs have plagued recent nuclear projects and they hamper the widespread deployment of nuclear technology. The Nuclear Cost Reduction Tool is a plant economic framework that quantifies the impact that various plant design and construction attributes have on construction costs and cost overruns and shows the positive effects of learning over a series of deployments. However, a downside of the current model is that all model outputs and capabilities are deterministic. To provide a more comprehensive view, this study evaluated the impact of model parameter uncertainty through a sensitivity analysis applied to 18 model parameters. This approach quantified the impact of model uncertainty on the output variables of Net Overnight Capital Cost (Net OCC), Construction Duration (CD), and Levelized Cost of Electricity (LCOE). Monte Carlo analysis revealed uncertainty distributions for these variables, showing that absolute uncertainty decreases over a series of deployments. A local sensitivity analysis showed that even small parameter perturbations (5%) can have a significant impact on project execution, highlighting areas that could reduce costs by billions across an order book of plants. The results of this study have improved the understanding of the model and identified the most impactful model parameters and construction attributes.

Levelized Cost of Electricity↗

PyApprox: A software package for sensitivity analysis, Bayesian inference, optimal experimental design, and multi-fidelity uncertainty quantification and surrogate modeling

PyApprox is a Python-based one-stop-shop for probabilistic analysis of numerical models such as those used in the earth, environmental and engineering sciences. Easy to use and extendable tools are provided for constructing surrogates, sensitivity analysis, Bayesian inference, experimental design, and forward uncertainty quantification. The algorithms implemented represent a wide range of methods for model analysis developed over the past two decades, including recent advances in multi-fidelity approaches that use multiple model discretizations and/or simplified physics to significantly reduce the computational cost of various types of analyses. An extensive set of Benchmarks from the literature is also provided to facilitate the easy comparison of new or existing algorithms for a wide range of model analyses. Here, this paper introduces PyApprox and its various features, and presents results demonstrating the utility of PyApprox on a benchmark problem modeling the advection of a tracer in groundwater.

54 ENVIRONMENTAL SCIENCES↗

Regional and Model-Specific Response Types in A Global Gridded Crop Model Ensemble

Crop models are often employed to project crop yields under changing conditions such as global warming and associated management change for adaptation. Multi-model ensembles are promoted to enhance the robustness of projections, but questions remain on what causes often large differences between projections of individual models. Global Gridded Crop Models (GGCMs) are especially exposed to this question when applied for assessing climate change impacts, adaptation, environmental impacts of agricultural production, because their results are used in downstream analyses, such as in integrated assessment or economic modeling for projecting future land-use change. Even though global gridded crop models are often based on detailed field-scale models or have implemented similar modeling principles in other ecosystem models, global-scale models are subject to substantial uncertainties from both model structure and parametrization as well as from calibration and input data quality. AgMIP’s Global Gridded Crop Model Intercomparison (GGCMI) has thus set out to intercompare GGCMs in order to evaluate model performance, describe model uncertainties, identify inconsistencies within the ensemble and underlying reasons, and to ultimately improve models and modeling capacities. In phase 2 of the GGCMI activities, 12 modeling groups followed a modeling protocol that asked for up to 1404 31-year global simulations at 0.5 arc-degree spatial resolution to assess models’ sensitivities to changes in carbon dioxide (C; 4 different levels) temperature (T; 7 different offset levels), water supply (W; 9 levels), and nitrogen (N; 3 levels), the so-called CTWN experiment (Franke et al. 2020; http://dx.doi.org/10.5194/gmd-13-2315-2020). We here present analyses of model response types using impact response surfaces along the C, T, W, and N dimensions, respectively and collectively. Doing so, we can understand differences in simulated responses per driver rather than aggregated changes in yields. We find that models’ sensitivities to the individual driver dimensions are substantially different and often more different across models than across regions. A cluster analysis finds regional and model-specific patterns. There is some agreement across models with respect to the spatial patterns of response types but strong differences in the distribution of response type clusters across models suggests that models need to undergo further scrutiny. We suggest establishing standards in model process evaluation not only against historical dynamics but also against dedicated experiments across the CTWN dimensions.

crop models↗

Quantifying the Multi-Objective Cost of Uncertainty

Various real-world applications involve modeling complex systems with immense uncertainty and optimizing multiple objectives based on the uncertain model. Quantifying the impact of the model uncertainty on the given operational objectives is critical for designing optimal experiments that can most effectively reduce the uncertainty that affect the objectives pertinent to the application at hand. In this paper, we propose the concept of mean multi-objective cost of uncertainty (multi-objective MOCU) that can be used for objective-based quantification of uncertainty for complex uncertain systems considering multiple operational objectives. We provide several illustrative examples that demonstrate the concept and strengths of the proposed multi-objective MOCU. Furthermore, we present a real-world example based on the mammalian cell cycle network to demonstrate how the multi-objective MOCU can be used for quantifying the operational impact of model uncertainty when there are multiple, possibly competing, objectives.

42 ENGINEERING↗

Industry Requirements for Geomagnetic Disturbance Models

This research project will address the most pressing uncertainties in modeling and measuring the electric power grid effects of geomagnetic disturbances (GMDs) and the E3 portion of nuclear electromagnetic pulse (EMP). The primary goal is to help decision-makers in the electric power sector have the knowledge and tools they need to most effectively mitigate GMD effects on our nation’s electric grid, with a secondary focus on EMP. Comprehensive modeling, model assessment through sensitivity analysis, and validation with field measurement data will be the primary tasks undertaken. The goal will be a more widespread adoption of these modeling approaches, namely characterizing the uncertainty associated with these models and how that uncertainty will affect decision-making by industry. This document summarizes industry requirements for better decision-making tools, informed by feedback from an industry advisory board. These requirements help guide planning and execution of the project's remaining tasks.

24 POWER TRANSMISSION AND DISTRIBUTION↗