Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model Uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63

Reducing Aerosol Forcing Uncertainty by Combining Models With Satellite and Within-The-Atmosphere Observations: A Three-Way Street

Aerosol forcing uncertainty represents the largest climate forcing uncertainty overall. Its magnitude has remained virtually undiminished over the past 20 years despite considerable advances in understanding most of the key contributing elements. Recent work has produced modest increases only in the confidence of the uncertainty estimate itself. This review summarizes the contributions toward reducing the uncertainty in the aerosol forcing of climate made by satellite observations, measurements taken within the atmosphere, as well as modeling and data assimilation. We adopt a more measurement-oriented perspective than most reviews of the subject in assessing the strengths and limitations of each; gaps and possible ways to fill them are considered. Currently planned programs supporting advanced, global-scale satellite and surface-based aerosol, cloud, and precursor gas observations, climate modeling, and intensive field campaigns aimed at characterizing the underlying physical and chemical processes involved, are all essential. But in addition, new efforts are needed: (a) to obtain systematic aircraft in situ measurements capturing the multi-variate probability distribution functions of particle optical, microphysical, and chemical properties (and associated uncertainty estimates), as well as co-variability with meteorology, for the major aerosol airmass types; (b) to conceive, develop, and implement a suborbital (aircraft plus surface-based) program aimed at systematically quantifying the cloud-scale microphysics, cloud optical properties, and cloud-related vertical velocities associated with aerosol-cloud interactions; and (c) to focus much more research on integrating the unique contributions of satellite observations, suborbital measurements, and modeling, to reduce the persistent uncertainty in aerosol climate forcing.

aerosol climate forcing↗

Patterns of Precipitation Change and Climatological Uncertainty among CMIP5 Models, with a Focus on the Midlatitude Pacific Storm Track

Projections of modeled precipitation ( P ) change in global warming scenarios demonstrate marked intermodel disagreement at regional scales. Empirical orthogonal functions (EOFs) and maximum covariance analysis (MCA) are used to diagnose spatial patterns of disagreement in the simulated climatology and end-of-century P changes in phase 5 of the Coupled Model Intercomparison Project (CMIP5) archive. The term principal uncertainty pattern (PUP) is used for any robust mode calculated when applying these techniques to a multimodel ensemble. For selected domains in the tropics, leading PUPs highlight features at the margins of convection zones and in the Pacific cold tongue. The midlatitude Pacific storm track is emphasized given its relevance to wintertime P projections over western North America. The first storm-track PUP identifies a sensitive region of disagreement in P increases over the eastern midlatitude Pacific where the storm track terminates, related to uncertainty in an eastward extension of the climatological jet. The second PUP portrays uncertainty in a zonally asymmetric meridional shift of storm-track P , related to uncertainty in the extent of a poleward jet shift in the western Pacific. Both of these modes appear to arise primarily from intermodel differences in the response to radiative forcing, distinct from sampling of internal variability. The leading storm-track PUPs for P and zonal wind change exhibit similarities to the leading uncertainty patterns for the historical climatology, indicating important and parallel sensitivities in the eastern Pacific storm-track terminus region. However, expansion coefficients for climatological uncertainties tend to be weakly correlated with those for end-of-century change.

54 ENVIRONMENTAL SCIENCES↗

Parameterization of Model Validating Sets for Uncertainty Bound Optimizations

Given measurement data, a nominal model and a linear fractional transformation uncertainty structure with an allowance on unknown but bounded exogenous disturbances, easily computable tests for the existence of a model validating uncertainty set are given. Under mild conditions, these tests are necessary and sufficient for the case of complex, nonrepeated, block-diagonal structure. For the more general case which includes repeated and/or real scalar uncertainties, the tests are only necessary but become sufficient if a collinearity condition is also satisfied. With the satisfaction of these tests, it is shown that a parameterization of all model validating sets of plant models is possible. The new parameterization is used as a basis for a systematic way to construct or perform uncertainty tradeoff with model validating uncertainty sets which have specific linear fractional transformation structure for use in robust control design and analysis. An illustrative example which includes a comparison of candidate model validating sets is given.

Lim, K. B.↗

EFFECT OF UNCERTAINTIES IN STELLAR MODEL PARAMETERS ON ESTIMATED MASSES AND RADII OF SINGLE STARS

Accurate and precise values of radii and masses of stars are needed to correctly estimate properties of extrasolar planets. We examine the effect of uncertainties in stellar model parameters on estimates of the masses, radii, and average densities of solar-type stars. We find that in the absence of seismic data on solar-like oscillations, stellar masses can be determined to a greater accuracy than either stellar radii or densities; but to get reasonably accurate results the effective temperature, log g, and metallicity must be measured to high precision. When seismic data are available, stellar density is the most well-determined property, followed by radius, with mass the least well-determined property. Uncertainties in stellar convection, quantified in terms of uncertainties in the value of the mixing length parameter, cause the most significant errors in the estimates of stellar properties.

79 ASTRONOMY AND ASTROPHYSICS↗

Selection of representative general circulation models under climatic uncertainty for Western North America

Abstract Climate change research uses an ensemble of general circulation model runs (GCMs-runs) to predict future climate under uncertainties. To reduce computational costs, this study selects representative GCM-runs (RGCM-runs) for Western North America (WNA) based on their performance in replicating historical climate conditions from 1981 to 2005 and projecting future changes from 1981–2010 to 2071–2100. This evaluation is conducted under two representative concentration pathways (RCPs) scenarios, RCP4.5 and RCP8.5, from the Coupled Model Intercomparison Project 5. By using an envelope-based selection technique and a multi-objective distance-based approach, we identify four RGCM-runs per RCP representing diverse climatic conditions, including wet-warm, wet-cold, dry-warm, and dry-cold. Compared to the full-set, these selected runs show a decreased mean absolute error (MAE) between the reference and RGCM-runs concerning the monthly average mean air temperature (T̄) and precipitation (P̄). For RCP4.5, T̄ MAE is 0.45 (vs. 0.58 in the full-set) and P̄ MAE is 0.31 (vs. 0.42). For RCP8.5, T̄ MAE is 0.51 (vs. 0.75) and P̄ MAE is 0.25 (vs. 0.36). The lower MAE values in the RGCM-run set indicate closer alignment between predicted and reference values, making the RGCM-run suitable for climate impact assessments in the region.

Water Resources↗

Managing Model Data Introduced Uncertainties in Simulator Predictions for Generation IV Systems via Optimum Experimental Design

An optimization technique has been developed to select optimized experimental design specifications to produce data specifically designed to be assimilated to optimize a given reactor concept. Data from the optimized experiment is assimilated to generate posteriori uncertainties on the reactor concept’s core attributes from which the design responses are computed. The reactor concept is then optimized with the new data to realize cost savings by reducing margin. The optimization problem iterates until an optimal experiment is found to maximize the savings. A new generation of innovative nuclear reactor designs, in particular fast neutron spectrum recycle reactors, are being considered for the application of closing the nuclear fuel cycle in the future. Safe and economical design of these reactors will require uncertainty reduction in basic nuclear data which are input to the reactor design. These data uncertainty propagate to design responses which in turn require the reactor designer to incorporate additional safety margin into the design, which often increases the cost of the reactor. Therefore basic nuclear data needs to be improved and this is accomplished through experimentation. Considering the high cost of nuclear experiments, it is desired to have an optimized experiment which will provide the data needed for uncertainty reduction such that a reactor design concept can meet its target accuracies or to allow savings to be realized by reducing the margin required due to uncertainty propagated from basic nuclear data. However, this optimization is coupled to the reactor design itself because with improved data the reactor concept can be re-optimized itself. It is thus desired to find the experiment that gives the best optimized reactor design. Methods are first established to model both the reactor concept and the experiment and to efficiently propagate the basic nuclear data uncertainty through these models to outputs. The representativity of the experiment to the design concept is quantitatively determined. A technique is then established to assimilate this data and produce posteriori uncertainties on key attributes and responses of the design concept. Several experiment perturbations based on engineering judgment are used to demonstrate these methods and also serve as an initial generation of the optimization problem. Finally, an optimization technique is developed which will simultaneously arrive at an optimized experiment to produce an optimized reactor design. Solution of this problem is made possible by the use of the simulated annealing algorithm for solution of optimization problems. The optimization examined in this work is based on maximizing the reactor cost savings associated with the modified design made possible by using the design margin gained through reduced basic nuclear data uncertainties. Cost values for experiment design specifications and reactor design specifications are established and used to compute a total savings by comparing the posteriori reactor cost to the a priori cost plus the cost of the experiment. The optimized solution arrives at a maximized cost savings.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Bayesian characterization and forward propagation of the uncertainty in thermodynamic models for redox active materials

Demonstrating the thermodynamic efficiency of hydrogen conversion processes using various materials is a critical step in developing new technologies for storing concentrated solar energy, and is largely accomplished by using a thermodynamic model derived from experimental data. A main goal of this project is to calculate the uncertainty of the thermodynamic efficiency by calculating the uncertainty of the components that feed into the efficiency. Many different models and data sets were used to test the workflow. First, the models were fit to the data using a Bayesian Inference and a method called Markov Chain Monte Carlo (MCMC), which found the maximum a priori parameters, and a posterior probability distribution of the parameters. Next, the different models were compared to each other using model evidence values. It was found that for cleaner data sets, overfitting had not yet been reached, and the most complicated model was ideal, but on the noisier data sets, the less complex models were favored because the more complicated models resulted in overfitting. Next, forward propagation was used to calculate the enthalpy change and its associated uncertainty. A few variations on the models were tried, such as fitting in a different variable, producing negligible or negative effects on the fits of the models. Thus, the original models were used. A sensitivity analysis was performed, and used to calculate the model error. On the cleaner data sets, there was very minimal experimental noise, and thus, all resulting error was from the model. With consideration of the model error, the models fit the data very well, and the simpler model had a high model error, as expected. All these components will then be used to calculate the thermodynamic efficiency of the different materials.

14 SOLAR ENERGY↗

Performance of Trajectory Models with Wind Uncertainty

Typical aircraft trajectory predictors use wind forecasts but do not account for the forecast uncertainty. A method for generating estimates of wind prediction uncertainty is described and its effect on aircraft trajectory prediction uncertainty is investigated. The procedure for estimating the wind prediction uncertainty relies uses a time-lagged ensemble of weather model forecasts from the hourly updated Rapid Update Cycle (RUC) weather prediction system. Forecast uncertainty is estimated using measures of the spread amongst various RUC time-lagged ensemble forecasts. This proof of concept study illustrates the estimated uncertainty and the actual wind errors, and documents the validity of the assumed ensemble-forecast accuracy relationship. Aircraft trajectory predictions are made using RUC winds with provision for the estimated uncertainty. Results for a set of simulated flights indicate this simple approach effectively translates the wind uncertainty estimate into an aircraft trajectory uncertainty. A key strength of the method is the ability to relate uncertainty to specific weather phenomena (contained in the various ensemble members) allowing identification of regional variations in uncertainty.

Lee, Alan G.↗

Calibration of Predictor Models Using Multiple Validation Experiments

This paper presents a framework for calibrating computational models using data from several and possibly dissimilar validation experiments. The offset between model predictions and observations, which might be caused by measurement noise, model-form uncertainty, and numerical error, drives the process by which uncertainty in the models parameters is characterized. The resulting description of uncertainty along with the computational model constitute a predictor model. Two types of predictor models are studied: Interval Predictor Models (IPMs) and Random Predictor Models (RPMs). IPMs use sets to characterize uncertainty, whereas RPMs use random vectors. The propagation of a set through a model makes the response an interval valued function of the state, whereas the propagation of a random vector yields a random process. Optimization-based strategies for calculating both types of predictor models are proposed. Whereas the formulations used to calculate IPMs target solutions leading to the interval value function of minimal spread containing all observations, those for RPMs seek to maximize the models' ability to reproduce the distribution of observations. Regarding RPMs, we choose a structure for the random vector (i.e., the assignment of probability to points in the parameter space) solely dependent on the prediction error. As such, the probabilistic description of uncertainty is not a subjective assignment of belief, nor is it expected to asymptotically converge to a fixed value, but instead it casts the model's ability to reproduce the experimental data. This framework enables evaluating the spread and distribution of the predicted response of target applications depending on the same parameters beyond the validation domain.

Crespo, Luis G.↗

A New Perspective for the Calibration of Computational Predictor Models

This paper presents a framework for calibrating computational models using data from several and possibly dissimilar validation experiments. The offset between model predictions and observations, which might be caused by measurement noise, model-form uncertainty, and numerical error, drives the process by which uncertainty in the models parameters is characterized. The resulting description of uncertainty along with the computational model constitute a predictor model. Two types of predictor models are studied: Interval Predictor Models (IPMs) and Random Predictor Models (RPMs). IPMs use sets to characterize uncertainty, whereas RPMs use random vectors. The propagation of a set through a model makes the response an interval valued function of the state, whereas the propagation of a random vector yields a random process. Optimization-based strategies for calculating both types of predictor models are proposed. Whereas the formulations used to calculate IPMs target solutions leading to the interval value function of minimal spread containing all observations, those for RPMs seek to maximize the models’ ability to reproduce the distribution of observations. Regarding RPMs, we choose a structure for the random vector (i.e., the assignment of probability to points in the parameter space) solely dependent on the prediction error. As such, the probabilistic description of uncertainty is not a subjective assignment of belief, nor is it expected to asymptotically converge to a fixed value, but instead it is a description of the model’s ability to reproduce the experimental data. This framework enables evaluating the spread and distribution of the predicted response of target applications depending on the same parameters beyond the validation domain (i.e., roll-up and extrapolation).

42 ENGINEERING↗

An Efficient Deterministic Approach to Model-based Prediction Uncertainty Estimation

Prognostics deals with the prediction of the end of life (EOL) of a system. EOL is a random variable, due to the presence of process noise and uncertainty in the future inputs to the system. Prognostics algorithm must account for this inherent uncertainty. In addition, these algorithms never know exactly the state of the system at the desired time of prediction, or the exact model describing the future evolution of the system, accumulating additional uncertainty into the predicted EOL. Prediction algorithms that do not account for these sources of uncertainty are misrepresenting the EOL and can lead to poor decisions based on their results. In this paper, we explore the impact of uncertainty in the prediction problem. We develop a general model-based prediction algorithm that incorporates these sources of uncertainty, and propose a novel approach to efficiently handle uncertainty in the future input trajectories of a system by using the unscented transformation. Using this approach, we are not only able to reduce the computational load but also estimate the bounds of uncertainty in a deterministic manner, which can be useful to consider during decision-making. Using a lithium-ion battery as a case study, we perform several simulation-based experiments to explore these issues, and validate the overall approach using experimental data from a battery testbed.

Daigle, Matthew J.↗

Sparsifying priors for Bayesian uncertainty quantification in model discovery

We propose a probabilistic model discovery method for identifying ordinary differential equations governing the dynamics of observed multivariate data. Our method is based on the sparse identification of nonlinear dynamics (SINDy) framework, where models are expressed as sparse linear combinations of pre-specified candidate functions. Promoting parsimony through sparsity leads to interpretable models that generalize to unknown data. Instead of targeting point estimates of the SINDy coefficients, we estimate these coefficients via sparse Bayesian inference. The resulting method, uncertainty quantification SINDy (UQ-SINDy), quantifies not only the uncertainty in the values of the SINDy coefficients due to observation errors and limited data, but also the probability of inclusion of each candidate function in the linear combination. UQ-SINDy promotes robustness against observation noise and limited data, interpretability (in terms of model selection and inclusion probabilities) and generalization capacity for out-of-sample forecast. Sparse inference for UQ-SINDy employs Markov chain Monte Carlo, and we explore two sparsifying priors: the spike and slab prior, and the regularized horseshoe prior. UQ-SINDy is shown to discover accurate models in the presence of noise and with orders-of-magnitude less data than current model discovery methods, thus providing a transformative method for real-world applications which have limited data.

97 MATHEMATICS AND COMPUTING↗

Quantifying uncertainty in LCA-modelling of waste management systems

Highlights: Black-Right-Pointing-Pointer Uncertainty in LCA-modelling of waste management is significant. Black-Right-Pointing-Pointer Model, scenario and parameter uncertainties contribute. Black-Right-Pointing-Pointer Sequential procedure for quantifying uncertainty is proposed. Black-Right-Pointing-Pointer Application of procedure is illustrated by a case-study. - Abstract: Uncertainty analysis in LCA studies has been subject to major progress over the last years. In the context of waste management, various methods have been implemented but a systematic method for uncertainty analysis of waste-LCA studies is lacking. The objective of this paper is (1) to present the sources of uncertainty specifically inherent to waste-LCA studies, (2) to select and apply several methods for uncertainty analysis and (3) to develop a general framework for quantitative uncertainty assessment of LCA of waste management systems. The suggested method is a sequence of four steps combining the selected methods: (Step 1) a sensitivity analysis evaluating the sensitivities of the results with respect to the input uncertainties, (Step 2) an uncertainty propagation providing appropriate tools for representing uncertainties and calculating the overall uncertainty of the model results, (Step 3) an uncertainty contribution analysis quantifying the contribution of each parameter uncertainty to the final uncertainty and (Step 4) as a new approach, a combined sensitivity analysis providing a visualisation of the shift in the ranking of different options due to variations of selected key parameters. This tiered approach optimises the resources available to LCA practitioners by only propagating the most influential uncertainties.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Bayesian automated weighting of aggregated DFT, MD, and experimental data for candidate thermodynamic models of aluminum with uncertainty quantification

Atomic-scale modeling methods such as density functional theory (DFT) and molecular dynamics (MD) can predict the thermodynamic properties of materials at a lower cost than experimental measurements. However, their regular usage in thermodynamic model construction is hampered by the lack of quantitative agreement with experimental measurements and the lack of uncertainty estimates on the data. To make regular usage of this atomistic simulation data, it is important to assess whether the atomistic simulation datasets, by themselves or in combination with experimental measurements, result in the same physics-informed models best supported by experimental measurements alone. Here, models of aluminum thermodynamic properties are discussed using three data sources: atomistic calculations (DFT and MD), experiments, and a combination of atomistic calculations and experiments. The study shows that, after ensuring self-consistency in predicting key invariant points, both experimental measurements and atomistic calculations can significantly contribute to an optimal model.

36 MATERIALS SCIENCE↗

Bayesian calibration of groundwater models with input data uncertainty

Effective water resources management typically relies on numerical models to analyze groundwater flow and solute transport processes. Groundwater models are often subject to input data uncertainty, as some inputs (such as recharge and well pumping rates) are estimated and subject to uncertainty. Current practices of groundwater model calibration often overlook uncertainties in input data; this can lead to biased parameter estimates and compromised predictions. Through a synthetic case study of surface–ground water interaction under changing pumping conditions and land use, we investigate the impacts of uncertain pumping and recharge rates on model calibration and uncertainty analysis. We then present a Bayesian framework of model calibration to handle uncertain input of groundwater models. The framework implements a marginalizing step to account for input data uncertainty when evaluating likelihood. It was found that not accounting for input uncertainty may lead to biased, overconfident parameter estimates because parameters could be over–adjusted to compensate for possible input data errors. Parameter compensation can have deleterious impacts when the calibrated model is used to make forecast under a scenario that is different from calibration conditions. By marginalizing input data uncertainty, the Bayesian calibration approach effectively alleviates parameter compensation and gives more accurate predictions in the synthetic case study. The marginalizing Bayesian method also decomposes prediction uncertainty into uncertainties contributed by parameters, input data, and measurements. Furthermore, the results underscore the need to account for input uncertainty to better inform postmodeling decision making.

54 ENVIRONMENTAL SCIENCES↗

Earth System Model Improvement Pipeline via Uncertainty Attribution and Active Learning

Primary focal area: 2 (Predictive Modeling via AI): We develop methods to formally quantify uncertainties in Earth System models for the land-atmosphere coupled system. Science Challenge: Earth system models still have significant biases in historical predictions of the intensity and frequency of water cycling extremes (e.g., droughts and flood events), leading to low confidence in future projections. Uncertainties arise from incomplete understanding of land and atmospheric processes, and insufficient observational constraints on model parameters. Many observations, including those from key DOE investments such as ARM and AmeriFlux, are used to evaluate model performance but have not been used to formally quantify model uncertainty because of the expense of running ESM simulations. An efficient pipeline engaging cutting-edge machine learning (ML) and uncertainty quantification (UQ) methods is needed to improve the predictive understanding of water cycle extremes in the Earth system.

54 ENVIRONMENTAL SCIENCES↗