Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model Uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,711 records · Page 95

Model-free estimation of completeness, uncertainties, and outliers in atomistic machine learning using information theory

Abstract An accurate description of information is relevant for a range of problems in atomistic machine learning (ML), such as crafting training sets, performing uncertainty quantification (UQ), or extracting physical insights from large datasets. However, atomistic ML often relies on unsupervised learning or model predictions to analyze information contents from simulation or training data. Here, we introduce a theoretical framework that provides a rigorous, model-free tool to quantify information contents in atomistic simulations. We demonstrate that the information entropy of a distribution of atom-centered environments explains known heuristics in ML potential developments, from training set sizes to dataset optimality. Using this tool, we propose a model-free UQ method that reliably predicts epistemic uncertainty and detects out-of-distribution samples, including rare events in systems such as nucleation. This method provides a general tool for data-driven atomistic modeling and combines efforts in ML, simulations, and physical explainability.

36 MATERIALS SCIENCE↗

Validation of Heat Transfer Thermal Decomposition and Container Pressurization of Polyurethane Foam.

Polymer foam encapsulants provide mechanical, electrical, and thermal isolation in engineered systems. In fire environments, gas pressure from thermal decomposition of polymers can cause mechanical failure of sealed systems. In this work, a detailed uncertainty quantification study of PMDI-based polyurethane foam is presented to assess the validity of the computational model. Both experimental measurement uncertainty and model prediction uncertainty are examined and compared. Both the mean value method and Latin hypercube sampling approach are used to propagate the uncertainty through the model. In addition to comparing computational and experimental results, the importance of each input parameter on the simulation result is also investigated. These results show that further development in the physics model of the foam and appropriate associated material testing are necessary to improve model accuracy.

Scott, Sarah Nicole↗

Robust control of hypersonic vehicles considering propulsive and aeroelastic effects

The influence of propulsion system variations and elastic fuselage behavior on the flight control system of an airbreathing hypersonic vehicle is investigated. Thrust vector magnitude and direction changes due to angle of attack variations affect the pitching moment. Low structural vibration frequencies may occur close to the rigid body modes influencing the angle of attack and lead to possible cross coupling. These effects are modeled as uncertainties in the context of a robust control study of a hypersonic vehicle model accelerating through Mach 8 using H-infinity and mu synthesis techniques. Various levels of uncertainty are introduced into the system. Both individual and simultaneous appearance of uncertainty are considered. The results indicate that the chosen design technique is suitable for this kind of problem provided that a fairly good knowledge of the effects mentioned above is available. The order of the designed controller is reduced but robust performance is lost which shows the need for fixed order design techniques.

Buschek, Harald↗

Multi-Model and Multi-Scale Global Sensitivity Analysis for Identifying Controlling Processes of Complex Systems

An environmental model consists of multiple process level sub-models, and each sub-model represents a process that is key to the operation of the simulated system. Global sensitivity analysis methods have been widely used to identify important processes for system model development and improvement. The existing methods of global sensitivity analysis only consider parametric uncertainty, and are not capable of handling model uncertainty caused by multiple process models that arise from competing hypotheses about one or more processes. To address this problem, this project develops a new method to probe model output sensitivity to competing process models by integrating model averaging methods with variance-based global sensitivity analysis to address uncertainty in process models and parameters. The new method yields three process sensitivity indices. The first one is called first-order process sensitivity index, and it is derived as a single summary measure of relative process importance. Evaluating the index is computationally expensive, because it relies in a Monte Carlo scheme that requires thousands and even millions of model executions. To reduce computational cost, this project develops a computationally efficient, quasi Monte Carlo method, and this method is presented in Chapter 2 of this report with and a numerical example for demonstration. The numerical example shows that the results of the quasi Monte Carlo method are substantially close to those of the full Monte Carlo method, but the computational cost of the quasi Monte Carlo method is only 0.7% of that of the full Monte Carlo method. The second index is called total-effect process sensitivity index, and it measures interactions between different processes. Therefore, this sensitivity index includes the first-order process sensitivity index, and can be used to identify influential processes. On the other hand, the total-effect process sensitivity index can also be used to screen non-influential processes. This is demonstrated by two numerical examples using the Sobol-G* functions and groundwater flow models that consider recharge process, geological process, and snowmelt process. The numerical examples shows that the total-effect process sensitivity index is more informative than the first-order process sensitivity. The derivation of the process sensitivity index and the numerical examples are discussed in Chapter 3. Chapter 4 presents two computationally efficient methods for screening non-influential processes to exclude them from further investigation. The two methods are the multi-model difference-based sensitivity (MMDS) analysis method, which can be implemented using the Latin Hypercube Sampling. The second one is the implementation of MMDS method using a binning method. The numerical example for the Sobol-G* function indicates the two methods are capable of identifying non-influential models, and the numerical examples for the groundwater flow and reactive transport show that the two methods are effective for groundwater problems. However, it should be noted that the two methods are numerical approximations, and they can only be used for screening non-influential processes, not for ranking importance of system processes. All the sensitivity analysis methods are implemented by developing python codes, and the codes are in a software called SAMMPY: a python package for process sensitivity analysis under multiple models. The SAMMPY design and structure are discussed in Chapter 5, and the package is released to the public for free download.

54 ENVIRONMENTAL SCIENCES↗

UQ Toolkit v 2.0

The Uncertainty Quantification (UQ) Toolkit is a software library for the characterizaton and propagation of uncertainties in computational models. For the characterization of uncertainties, Bayesian inference tools are provided to infer uncertain model parameters, as well as Bayesian compressive sensing methods for discovering sparse representations of high-dimensional input-output response surfaces, and also Karhunen-Loève expansions for representing stochastic processes. Uncertain parameters are treated as random variables and represented with Polynomial Chaos expansions (PCEs). The library implements several spectral basis function types (e.g. Hermite basis functions in terms of Gaussian random variables or Legendre basis functions in terms of uniform random variables) that can be used to represent random variables with PCEs. For propagation of uncertainty, tools are provided to propagate PCEs that describe the input uncertainty through the computational model using either intrusive methods (Galerkin projection of equations onto basis functions) or non-intrusive methods (perform deterministic operation at sampled values of the random values and project the obtained results onto basis functions).

Safta, Cosmin↗

UQ Toolkit v 2.0

The Uncertainty Quantification (UQ) Toolkit is a software library for the characterizaton and propagation of uncertainties in computational models. For the characterization of uncertainties, Bayesian inference tools are provided to infer uncertain model parameters, as well as Bayesian compressive sensing methods for discovering sparse representations of high-dimensional input-output response surfaces, and also Karhunen-Loève expansions for representing stochastic processes. Uncertain parameters are treated as random variables and represented with Polynomial Chaos expansions (PCEs). The library implements several spectral basis function types (e.g. Hermite basis functions in terms of Gaussian random variables or Legendre basis functions in terms of uniform random variables) that can be used to represent random variables with PCEs. For propagation of uncertainty, tools are provided to propagate PCEs that describe the input uncertainty through the computational model using either intrusive methods (Galerkin projection of equations onto basis functions) or non-intrusive methods (perform deterministic operation at sampled values of the random values and project the obtained results onto basis functions).

Source record↗

Bayesian calibration of thermodynamic parameters for geochemical speciation modeling of cementitious materials

Chemical equilibrium modeling of cementitious materials requires aqueous-solid equilibrium constants of the controlling mineral phases (K{sub sp}) and the available concentrations of primary components. Inherent randomness of the input and model parameters, experimental measurement error, the assumptions and approximations required for numerical simulation, and inadequate knowledge of the chemical process contribute to uncertainty in model prediction. A numerical simulation framework is developed in this paper to assess uncertainty in K{sub sp} values used in geochemical speciation models. A Bayesian statistical method is used in combination with an efficient, adaptive Metropolis sampling technique to develop probability density functions for K{sub sp} values. One set of leaching experimental observations is used for calibration and another set is used for comparison to evaluate the applicability of the approach. The estimated probability distributions of K{sub sp} values can be used in Monte Carlo simulation to assess uncertainty in the behavior of aqueous-solid partitioning of constituents in cement-based materials.

36 MATERIALS SCIENCE↗

Development of a multi-layer canopy model for E3SM Land Model with support for heterogeneous computing

The vertical structure of vegetation canopies creates micro-climates. However, the land components of most Earth System Models, including the Energy Exascale Earth System Model (E3SM), typically neglect vertical canopy structure by using a single layer big-leaf representation to simulate water, CO 2 , and energy exchanges between the land and the atmosphere. In this study, we developed a Multi-Layer Canopy Model for the E3SM Land Model to resolve the micro-climate created by vegetation canopies. The model developed in this study re-implements the CLM-ml_v1 to support heterogeneous computing architectures consisting of CPUs and GPUs and includes three additional optimization-based stomatal conductance models. The use of Portable, Extensible Toolkit for Scientific Computation provides a speedup of 25–50 times on a GPU relative to a CPU. The numerical implementation of the model was verified against CLM-ml_v1 for a month-long simulation using data from the Ameriflux US-University of Michigan Biological Station site. Model structural uncertainty was explored by performing control simulations for five stomatal conductance models that exclude and include the control of plant hydrodynamics (PHD) on photosynthesis. The bias in simulated sensible and latent heat fluxes was lower when PHD was accounted for in the model. Additionally, six idealized simulations were performed to study the impact of three environmental variables (i.e. air temperature, atmospheric CO 2 , and soil moisture) on canopy processes (i.e. net CO 2 assimilation, leaf temperature, and leaf water potential). Increasing air temperature reduced net CO 2 assimilation and increased air temperature. Net CO 2 assimilation increased at higher atmospheric CO 2 , while decreasing soil moisture resulted in lower leaf water potential.

54 ENVIRONMENTAL SCIENCES↗

Performance and transport in the ARC tokamak

The ARC TM tokamak, a high-field (𝐵 𝑇 = 11.4 T) fusion power plant, under development by Commonwealth Fusion Systems, is studied using a suite of integrated modelling tools to predict its fusion power generation (𝑃𝑓⁡𝑢⁢𝑠), transport and confinement properties. Analysis is based off an ARC operational point scoped first with zero-dimensional (0-D) plasma operational contour (POPCON) modelling to produce 1.13 GW of fusion power. A suite of integrated modelling tools (TRANSP, ASTRA and TORAX) were applied to predict the performance and kinetic profiles of the ARC design point, yielding a range of predicted performance spanning from ∼900 to 1300 MW in rough quantitative agreement with POPCON predictions. The sensitivity of these results to uncertain modelling inputs was probed using scans of pedestal boundary conditions around EPED-predicted values (total pressure and temperature ratios), tungsten concentration and seperatrix density around their nominal assumptions. Pedestal pressure and pedestal top (𝑇 𝑖 /𝑇 𝑒 ) play a large role in 1.5-dimensional performance predictions, able to modify the predicted 𝑃 𝑓⁡𝑢⁢𝑠 by a factor of 2 within reasonable assumptions. High-fidelity core nonlinear gyrokinetic profile predictions, performed using CGYRO (Candy et al. 2016 J. Comput. Phys., vol. 324, pp. 73–93) coupled with the PORTALS (Rodriguez-Fernandez et al. 2024 Nucl. Fusion, vol. 64, 076034; Phys. Plasmas, vol. 31, 2024, 062501) framework, yield substantially lower performance (𝑃 𝑓⁡𝑢⁢𝑠 =677 MW) compared with 0-D and medium-fidelity modelling for nominal assumptions, showing that there is non-negligible uncertainty between models and that future work on SPARC may help resolve discrepancies. Lower overall performance results from significantly reduced volume-averaged densities and temperatures, along with reduced levels of density and temperature peaking. Turbulence and transport are largely dominated by ion temperature gradient across the profile, confirmed by both linear stability and the response of the nonlinear fluxes to changes in gradients, with some impact of kinetic ballooning modes in the deep core. This work represents one of the most complete scoping of potential fusion power plant conditions performed to date. The extensive integrated modelling provides confidence in ARC performance approaching 1 GW, while nonlinear gyrokinetic modelling results in open questions into the physics of density and temperature peaking in fusion-power-plant-relevant operational space. A discussion of results and the role that the SPARC tokamak (Creely et al. 2020 J. Plasma Phys., vol. 86, 865860502) will play in informing ARC design, performance and operation is presented.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

THE PHYSICAL CONDITIONS IN STARBURSTS DERIVED FROM BAYESIAN FITTING OF MID-INFRARED SPECTRAL ENERGY DISTRIBUTION MODELS: 30 DORADUS AS A TEMPLATE

To understand and interpret the observed spectral energy distributions (SEDs) of starbursts, theoretical or semi-empirical SED models are necessary. Yet, while they are well founded in theory, independent verification and calibration of these models, including the exploration of possible degeneracies between their parameters, are rarely made. As a consequence, a robust fitting method that leads to unique and reproducible results has been lacking. Here we introduce a novel approach based on Bayesian analysis to fit the Spitzer-Infrared Spectrometer spectra of starbursts using the SED models proposed by Groves et al.. We demonstrate its capabilities and verify the agreement between the derived best-fit parameters and actual physical conditions by modeling the nearby, well-studied, giant H II region 30 Doradus in the LMC. The derived physical parameters, such as cluster mass, cluster age, interstellar medium pressure, and covering fraction of photodissociation regions, are representative of the 30 Doradus region. The inclusion of the emission lines in the modeling is crucial to break degeneracies. We investigate the limitations and uncertainties by modeling subregions, which are dominated by single components, within 30 Doradus. A remarkable result for 30 Doradus in particular is a considerable contribution to its mid-infrared spectrum from hot ({approx}300 K) dust. The demonstrated success of our approach will allow us to derive the physical conditions in more distant, spatially unresolved starbursts.

79 ASTRONOMY AND ASTROPHYSICS↗

Benchmarks for Model Validation based on LiDAR Wake Measurements

Technology development and design decisions in wind energy are often based on results from simulations performed for individual wind turbines or entire wind plants. It is therefore critical to ensure that the models being used for research and industry applications in wind energy be thoroughly validated against measurements. A full-system validation of wind plant simulations must consider the atmospheric inflow, the response of the wind turbines, and their wakes. This task is complicated by the lack of freely available, quality-controlled, high-quality measurements. Here, such measurements are used to offer a validation exercise that can be used to assess the accuracy of models of any fidelity level. When it comes to real-world measurements, the dataset considered herein is simple in terms of terrain but exhibits pronounced diurnal cycles. Instead of a full-scale wind plant, we consider an individual research-scale, utility wind turbine instrumented for power and loads measurements. Three benchmarks are defined, with increasing levels of complexity: near neutral, slightly unstable, and very stable atmospheric stratification. Through comparisons between observations and simulations, the benchmarks provide complementary information about the model performance and its ability to reproduce mean and dynamic wake characteristics. This article describes the measurements and methodology used to define these benchmarks and provides the information required to perform simulations and conduct the model-measurement comparison. The objective is to provide a robust wake model validation exercise open to anyone, which will serve to minimize uncertainty in model validation practices related to varying methodologies across simulation tools and users.

17 WIND ENERGY↗

Optimizing Service Restoration in Distribution Systems With Uncertain Repair Time and Demand

This paper proposes a novel method to co-optimize the distribution system operation and repair crew routing for outage restoration after extreme weather events. A two-stage stochastic mixed integer linear program is developed. The first stage is to dispatch the repair crews to the damaged components. The second stage is distribution system restoration using distributed generators, and reconfiguration. We consider demand uncertainty in terms of a truncated normal forecast error distribution, and model the uncertainty of the repair time using a lognormal distribution. A new decomposition approach, combined with the progressive hedging algorithm, is developed for solving large-scale outage management problems in an effective and timely manner. Furthermore, the proposed method is validated on modified IEEE 34- and 8500-bus distribution test systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Subpolar North Atlantic Mean State Affects the Response of the Atlantic Meridional Overturning Circulation to the North Atlantic Oscillation in CMIP6 Models

Abstract The Atlantic meridional overturning circulation (AMOC) plays an important role in climate, transporting heat and salt to the subpolar North Atlantic. The AMOC’s variability is sensitive to atmospheric forcing, especially the North Atlantic Oscillation (NAO). Because AMOC observations are short, climate models are a valuable tool to study the AMOC’s variability. Yet, there are known issues with climate models, like uncertainties and systematic biases. To investigate this, preindustrial control experiments from models participating in the phase 6 of Coupled Model Intercomparison Project (CMIP6) are evaluated. There is a large, but correlated, spread in the models’ subpolar gyre mean surface temperature and salinity. By splitting models into groups of either a warm–salty or cold–fresh subpolar gyre, it is shown that warm–salty models have a lower sea ice cover in the Labrador Sea and, hence, enable a larger heat loss during a positive NAO. Stratification in the Labrador Sea is also weaker in warm–salty models, such that the larger NAO-related heat loss can also affect greater depths. As a result, subsurface density anomalies are much stronger in the warm–salty models than in those that tend to be cold and fresh. As these anomalies propagate southward along the western boundary, they establish a zonal density gradient anomaly that promotes a stronger delayed AMOC response to the NAO in the warm–salty models. These findings demonstrate how model mean state errors are linked across variables and affect variability, emphasizing the need for improvement of the subpolar North Atlantic mean states in models.

54 ENVIRONMENTAL SCIENCES↗

What is the Uncertainty in MODIS Aerosol Optical Depth in the Vicinity of Clouds?

MODIS dark-target (DT) algorithm retrieves aerosol optical depth (AOD) using a Look Up Table (LUT) approach. Global comparison of AOD (Collection 6 ) with ground-based sun photometer gives an Estimated Error (EE) of +/-(0.04 + 10%) over ocean. However, EE does not represent per-retrieval uncertainty. For retrievals that are biased high compared to AERONET, here we aim to closely examine the contribution of biases due to presence of clouds and per-pixel retrieval uncertainty. We have characterized AOD uncertainty at 550 nm, due to standard deviation of reflectance in 10 km retrieval region, uncertainty related to gas (H2O, O3) absorption, surface albedo, and aerosol models. The uncertainty in retrieved AOD seems to lie within the estimated over ocean error envelope of +/-(0.03+10%). Regions between broken clouds tend to have higher uncertainty. Compared to C6 AOD, a retrieval omitting observations in the vicinity of clouds (< or = 1 km) is biased by about +/- 0.05. For homogeneous aerosol distribution, clear sky retrievals show near zero bias. Close look at per-pixel reflectance histograms suggests retrieval possibility using median reflectance values.

Patadia, Falguni↗

Occlusion-Perturbed Deep Learning for Probabilistic Solar Forecasting via Sky Images

Solar forecasting is shifting to the probabilistic paradigm due to the inherent uncertainty within the solar resource. Input uncertainty quantification is one of the widely used and best-performing ways to model solar uncertainty. However, compared to other sources of inputs, such as numerical weather prediction models, pure sky image-based probabilistic solar forecasting lags behind. In this research, an occlusion-perturbed convolutional neural network, named the PSolarNet, is developed. The PSolarNet provides very short-term deterministic forecasts, forecast scenarios, and probabilistic forecasts of the global horizontal irradiance from sky image sequences. Case studies based on 6 years of open-source data show that the developed PSolarNet is able to generate accurate 10-minute ahead deterministic forecasts with a 5.62% normalized root mean square error, realistic and diverse forecast scenarios with a 0.966 average correlation with the actual time series, and reliable and sharp probabilistic forecasts with a 2.77% normalized continuous ranked probability score.

Bayesian model averaging↗

Occlusion-Perturbed Deep Learning for Probabilistic Solar Forecasting via Sky Images: Preprint

Solar forecasting is shifting to the probabilistic paradigm due to the inherent uncertainty within the solar resource. Input uncertainty quantification is one of the widely used and best-performing ways to model solar uncertainty. However, compared to other sources of inputs, such as numerical weather prediction models, pure sky image-based probabilistic solar forecasting lags behind. In this research, an occlusion-perturbed convolutional neural network, named the PSolarNet, is developed. The PSolarNet provides very short-term deterministic forecasts, forecast scenarios, and probabilistic forecasts of the global horizontal irradiance from sky image sequences. Case studies based on 6 years of open-source data show that the developed PSolarNet is able to generate accurate 10-minute ahead deterministic forecasts with a 5.62% normalized root mean square error, realistic and diverse forecast scenarios with a 0.966 average correlation with the actual time series, and reliable and sharp probabilistic forecasts with a 2.77% normalized continuous ranked probability score.

Bayesian model averaging↗

Cold Nuclear Matter Effects on J/{psi} Yields as a Function of Rapidity and Nuclear Geometry in d+A Collisions at {radical}(s{sub NN})=200 GeV

We present measurements of J/{psi} yields in d+Au collisions at {radical}(s{sub NN})=200 GeV recorded by the PHENIX experiment and compare them with yields in p+p collisions at the same energy per nucleon-nucleon collision. The measurements cover a large kinematic range in J/{psi} rapidity (-2.2<y<2.4) with high statistical precision and are compared with two theoretical models: one with nuclear shadowing combined with final state breakup and one with coherent gluon saturation effects. In order to remove model dependent systematic uncertainties we also compare the data to a simple geometric model. The forward rapidity data are inconsistent with nuclear modifications that are linear or exponential in the density weighted longitudinal thickness, such as those from the final state breakup of the bound state.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Structural Uncertainty in the Sensitivity of Urban Temperatures to Anthropogenic Heat Flux

One key source of uncertainty for weather and climate models is structural uncertainty arising from the fact that these models must simplify or approximate complex physical, chemical, and biological processes that occur in the real world. However, structural uncertainty is rarely examined in the context of simulated effects of anthropogenic heat flux in cities. Using the Weather Research and Forecasting (WRF) model coupled with a single-layer urban canopy model, it is found that the sensitivity of urban canopy air temperature to anthropogenic heat flux can differ by an order of magnitude depending on how anthropogenic heat flux is released to the urban environment. Moreover, varying model structures through changing the treatment of roof-air interaction and the parameterization of convective heat transfer between the canopy air and the atmosphere can affect the sensitivity of urban canopy air temperature by a factor of 4. Urban surface temperature and 2-m air temperature are less sensitive to the methods of anthropogenic heat flux release and the examined model structural variants than urban canopy air temperature, but their sensitivities to anthropogenic heat flux can still vary by as much as a factor of 4 for surface temperature and 2 for 2-m air temperature. Our study recommends using temperature sensitivity instead of temperature response to understand how various physical processes (and their representations in numerical models) modulate the simulated effects of anthropogenic heat flux.

54 ENVIRONMENTAL SCIENCES↗