SEARCH · Search NASA
Results for “Sobol’ indices”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Computation of Sobol' indices using Embedded Variance Deconvolution.
Abstract not provided.
A sampling-based approach to solve Sobol' Indices using variance deconvolution for arbitrary uncertainty distributions.
Abstract not provided.
A sampling-based approach to solve Sobol' Indices using variance deconvolution for arbitrary uncertainty distributions.
Abstract not provided.
Imprecise global sensitivity analysis using bayesian multimodel inference and importance sampling
Global Sensitivity Analysis (GSA) aims to understand the relative importance of uncertain input variables to model response. Conventional GSA involves calculating sensitivity (Sobol’) indices for a model with known model parameter distributions. However, model parameters are affected by aleatory and epistemic uncertainty, with the latter often caused by lack of data. In this paper, we propose a new framework to quantify uncertainty in probability model-form and model parameters resulting from small datasets and integrate these uncertainties into Sobol’ index estimates. First, the process establishes, through Bayesian multimodel inference, a set of candidate probability models and their associated probabilities. Imprecise Sobol’ indices are calculated from these probability models using an importance sampling reweighting approach. This results in probabilistic Sobol’ indices, whose distribution characterizes uncertainty in the sensitivity resulting from small dataset size. The imprecise Sobol’ indices thus provide a measure of confidence in the sensitivity estimate and, moreover, can be used to inform data collection efforts targeted to minimize the impact of uncertainties. Through an example studying the parameters of a Timoshenko beam, we show that these probabilistic Sobol’ indices converge to the true/deterministic Sobol’ indices as the dataset size increases and hence, distribution-form uncertainty reduces. The approach is then applied to assess the sensitivity of the out-of-plane properties of an E-glass fiber composite material to its constituent properties. This second example illustrates the approach for an important class of materials with wide-ranging applications when data may be lacking for some input parameters.
Variance Decomposition of MEDLI2 Reconstructed Heating Using Neural Networks
The Mars Entry, Descent, and Landing Instrumentation (MEDLI2) sensor suite collected data during entry of the Mars 2020 Perseverance rover into Mars’ atmosphere. This suite included a network of MEDLI2 Instrumented Sensor Plugs (MISPs). Each MISP was comprised of a cylinder made of Thermal Protection System (TPS) material with 1-3 embedded thermocouples (TCs), and it was flush mounted into the heatshield or backshell. Data from these in-depth TCs were used to reconstruct the aeroheating environment of the vehicle throughout entry. Surface heating was posed as an inverse problem, with the goal of estimating the surface heating by minimizing an objective function of the difference between MISP temperature measurements during flight and the temperature predictions derived from the Fully Implicit Ablation and Thermal response (FIAT) program. Given an aerothermal environment, FIAT calculates the material response and provides in-depth temperatures throughout the TPS material. To achieve the reverse, an internal tool called FIAT_Opt runs through multiple different environments until the output temperature at the TC depth closely matches the flight data. 95% confidence intervals on the reconstructed surface heating were obtained using Monte Carlo analysis, in which uncertainties in the thermocouple depth and the TPS material properties (e.g., density, thermal conductivity, heat capacity, emissivity) based on flight-lot material testing were included. A variance decomposition method using Sobol indices was employed to assess the sensitivity of the reconstructed peak heating to the TC placement and material property uncertainties. Variance decomposition was found to require tens of thousands of FIAT_Opt runs in order for the Sobol indices to converge. With a single FIAT_Opt run taking on the order of 40 minutes, the required number of computations would take months to complete, even if using multiple CPUs. To mitigate this problem, three machine learning models (ridge regression with cross-validation, random forest regression, and a deep neural network) were trained and tested using the 2000 Monte Carlo runs that were already completed. A subset of 1600 runs were used to train the model (i.e., training set), while the remaining 400 runs were used as the test set. The predictions from the deep neural network (DNN) on the test set showed nearly perfect agreement to the actual values computed with FIAT_Opt (R2 > 0.99). Using the DNN as a surrogate model, the variance decomposition using 50,000 runs was completed within minutes. The resulting Sobol indices showed that the reconstructed peak surface heating was most sensitive to the uncertainties in the thermal conductivity (ST = 0.37) and heat capacity (ST = 0.26). This method can be leveraged to provide requirements for material property measurements needed to improve the accuracy of surface heating prediction and ultimately lead to the reduction of design margins in the future. This presentation will include background on the MEDLI2 suite; the method used for inverse heating estimation; the way that material property uncertainties were accounted for using Monte Carlo analysis; a brief background on variance decomposition; the motivation for using machine learning in this context; how a neural network was trained on the data to enable variance decomposition in a fraction of the time; and the variance decomposition results for one of the MISPs.
A Data-Driven Global Sensitivity Analysis Framework for Three-Phase Distribution System with PVs
Global sensitivity analysis (GSA) of distribution systems with respect to stochastic PV and load variations plays an important role in designing optimal voltage control schemes. This paper proposes a data-driven framework for GSA of distribution systems. In particular, two representative surrogate modeling-based approaches are developed, including the traditional Gaussian process-based and the analysis of variance (ANOVA) kernel ones. The key idea is to develop a surrogate model that captures the hidden global relationship between voltage and real and reactive power injections from the historical data. With the surrogate model, the Sobol indices can be conveniently calculated through either the sampling-based method or the analytical method to assess the global sensitivity of voltage to variations of PV and load power injections. The sampling-based method approximates the Sobol indices using Monte Carlo simulations while the analytical method calculates them by resorting to the ANOVA expansion framework. Comparison results with other model-based GSA methods on the unbalanced three-phase IEEE 37-bus and 123-bus distribution systems show that the proposed framework can achieve much higher computational efficiency with negligible loss of accuracy. The results on a real 240-node distribution system using actual smart meter data further validate the feasibility and scalability of the proposed framework.
Uncertainty Quantification Framework for Predicting Material Response with Large Number of Parameters: Application to Creep Prediction in Ferritic-Martensitic Steels Using Combined Crystal Plasticity and Grain Boundary Models
This paper presents an uncertainty quantification (UQ) framework for the physics-based model prediction of material response with a large number of parameters. The application problem presented in this work is that of predicting creep in Grade 91 steel at 600°C. The material response is defined with a physically based microstructural model with constitutive equations emulating several observed phenomena in Grade 91 and embodied into an explicit geometry mesoscale finite element model for prior austenite grains and grain boundaries. Creep within the grains and in grain boundaries are represented by crystal plasticity for dislocation motion and a physics-based model for cavity growth and nucleation, respectively. The creep behavior of this material is influenced by several parameters, some of which have a wide range of variation based on experimental data. UQ combined with microstructural modeling can discover the core microstructural causes of experimental variability, leading to improved materials with lower variability in critical long-term material properties. In this study, we investigate the model's uncertainty to identify material properties that may be modified during production to increase creep life and analyze different components of the crystal plasticity model for improvements. For this purpose, a quantity of interest is defined as time to minimum creep rate, which correlates well to the creep failure of the material. A deep neural network model was trained and validated to be used as a surrogate for the finite element model. Then, a variance-based sensitivity analysis is performed on the surrogate model to find the Sobol indices of the input parameters in respect to the output quantity of interest. The Sobol indices are used to reduce the dimensionality of the model. Generalized polynomial chaos expansion is used on the reduced basis models to propagate the uncertainty from the input parameters to the quantity of interest using the deep neural network surrogate model. These results are benchmarked against uncertainty propagation using Monte Carlo simulations. In conclusion, the UQ performed through the reduced basis model captures almost all the uncertainty in the model with significantly fewer simulations, making it possible to perform the UQ directly via simulations with the finite element model rather than surrogate machine-learned models.
Global Sensitivity Analysis of Simulated Remote Sensing Polarimetric Observations Over Snow
This study presents a detailed theoretical assessment of the information content of passive polarimetric observations over snow scenes, using a global sensitivity analysis (GSA) method. Conventional sensitivity studies focus on varying a single parameter while keeping all other parameters fixed. In contrast, the GSA correctly addresses the covariance of state parameters across their entire parameter space, hence favoring a more correct interpretation of inversion algorithms and the optimal design of their state vectors. The forward simulations exploit a vector radiative transfer model to obtain the Stokes vector emerging at the top of the atmosphere for different solar zenith angles, when the bottom boundary consists of a vertically resolved snowpack of non-spherical grains. The presence of light-absorbing particulates (LAPs), either embedded in the snow or aloft in the atmosphere above in the form of aerosols, is also considered. The results are presented for a set of wavelengths spanning the visible (VIS), near-infrared (NIR), and shortwave infrared (SWIR) region of the spectrum. The GSA correctly captures the expected, high sensitivity of the reflectance to LAPs in the VIS–NIR and to grain size at different depths in the snowpack in the NIR–SWIR. With adequate viewing geometries, mono-angle measurements of total reflectance in the VIS–SWIR (akin to those of the Moderate Resolution Imaging Spectroradiometer, MODIS) resolve grain size in the top layer of the snowpack sufficiently well. The addition of multi-angle polarimetric observations in the VIS–NIR provides information on grain shape and microscale roughness. The simultaneous sensitivity in the VIS–NIR to both aerosols and snow-embedded impurities can be disentangled by extending the spectral range to the SWIR, which contains information on aerosol optical depth while remaining essentially unaffected when the same particulates are mixed with the snow. Multi-angle polarimetric observations can therefore (i) effectively partition LAPs between the atmosphere and the surface, which represents a notorious challenge for snow remote sensing based on measurements of total reflectance only and (ii) lead to better estimates of grain shape and roughness and, in turn, the asymmetry parameter, which is critical for the determination of albedo. The retrieval uncertainties are minimized when the degree of linear polarization is used in place of the polarized reflectance. The Sobol indices, which are the main metric for the GSA, were used to select the state parameters in retrievals performed on data simulated for multiple instrument configurations. Improvements in retrieval quality with the addition of measurements of polarization, multi-angle views, and different spectral channels reflect the information content, identified by the Sobol indices, relative to each configuration. The results encourage the development of new remote sensing algorithms that fully leverage multi-angle and polarimetric capabilities of modern remote sensors. They can also aid flight planning activities, since the optimal exploitation of the information content of multi-angle measurements depends on the viewing geometry. The better characterization of surface and atmospheric parameters in snow-covered regions advances research opportunities for scientists of the cryosphere and ultimately benefits albedo estimates in climate models.
Global Sensitivity Analysis of Large Distribution System with PVs using Deep Gaussian Process
Global sensitivity analysis (GSA) of the voltage to uncertain power injection variations plays an important role for appropriate Volt-VAR optimization. This paper proposes a data-driven GSA method for large-scale distribution systems with a large number of uncertain sources. Specifically, the deep Gaussian process is used to identify the mapping relationship between uncertain power injections and voltages. This allows us to resort to the analysis of variance framework to calculate the Sobol indices for GSA. Unlike the existing polynomial chaos expansion and Gaussian process-based approaches, our proposed method has much better scalability. Test results on the EPRI 1747-node K1 circuit with a different number and different probability distributions of uncertain sources demonstrate that the proposed method can achieve accurate GSA under various conditions.
Uncertainty quantification and sensitivity analysis for SPERT III E-core reactivity measurement benchmarking
The Special Power Excursion Reactor Test (SPERT) III E-core experiment is important because it provides critical data on reactor behavior under significant reactivity insertions, which is essential for validating computer simulations and ensuring the safety of modern light water reactors. Its design similarities to contemporary reactors make it a valuable resource for understanding and mitigating extreme hazards in nuclear operations. The current study details the application of formal parametric uncertainty quantification and sensitivity analysis to a model of the SPERT-III E-core model for zero power reactivity benchmarking. Additionally, the reactivity impact from various modeling assumptions is quantified. Overall, an conservative estimate for an uncertainty in k$_{\text{eff}}$ of $\pm$1257 was observed. A less conservative, more realistic, uncertainty estimate of $\pm$1096 pcm can be justified by the potential for various parametric uncertainties to become negligible when sampled independently across the ~1400 pins in the core. The experimental results fall within both of these uncertainty bounds. Standardized regression coefficient as well as Sobol indices are used to identify the guide tube thicknesses as the primary contributors to the uncertainty in k$_{\text{eff}}$. Overall, this study provides information on how the uncertainties in input parameters and modeling methods impact simulated k$_{\text{eff}}$ values and can be used to aid model building efforts for future code validation with the SPERT-III E-core experiment.
Polynomial Chaos Surrogate Construction for Random Fields with Parametric Uncertainty
Engineering and applied science rely on computational experiments to rigorously study physical systems. The mathematical models used to probe these systems are highly complex, and sampling-intensive studies often require prohibitively many simulations for acceptable accuracy. Surrogate models provide a means of circumventing the high computational expense of sampling such complex models. In particular, polynomial chaos expansions (PCEs) have been successfully used for uncertainty quantification studies of deterministic models where the dominant source of uncertainty is parametric. We discuss an extension to conventional PCE surrogate modeling to enable surrogate construction for stochastic computational models that have intrinsic noise in addition to parametric uncertainty. We develop a PCE surrogate on a joint space of intrinsic and parametric uncertainty, enabled by Rosenblatt transformations, which are evaluated via kernel density estimation of the associated conditional cumulative distributions. Furthermore, we extend the construction to random field data via the Karhunen–Loève expansion. We then take advantage of closed-form solutions for computing PCE Sobol indices to perform a global sensitivity analysis of the model which quantifies the intrinsic noise contribution to the overall model output variance. Additionally, the resulting joint PCE is generative in the sense that it allows generating random realizations at any input parameter setting that are statistically approximately equivalent to realizations from the underlying stochastic model. The method is demonstrated on a chemical catalysis example model and a synthetic example controlled by a parameter that enables a switch from unimodal to bimodal response distributions.
Quantifying model prediction sensitivity to model-form uncertainty
Computational and mathematical models are essential to understanding complex systems and phenomena. However, when developing such models, limited knowledge and/or resources necessitates the use of simplifying assumptions. It is therefore crucial to quantify the impact of such simplifying assumptions on the reliability and accuracy of resulting model predictions. This work develops a first-of-its-kind approach to quantify the impact of physics modeling assumptions on predictions. Here, we leverage the emerging field of model-form uncertainty (MFU) representations, which are parameterized modifications to modeling assumptions, in combination with grouped Sobol’ indices to quantitatively measure an assumption’s importance. Specifically, we compute the grouped Sobol’ index for the MFU representation’s parameters as a single importance measure of the assumption for which the MFU representation characterizes uncertainty. To ensure this approach is robust to the subjective choice of how to parameterize a MFU representation, we establish bounds for the difference between sensitivity results for two different MFU representations based on differences in model prediction statistics. The capabilities associated with this approach are demonstrated on three exemplar problems: an upscaled subsurface contaminant transport problem, ablation modeling for hypersonic flight, and nuclear waste repository modeling. We found that our grouped approach is able to assess the impact of modeling assumptions on predictions and offers computational advantages over classical Sobol’ index computation while providing more interpretable results.
Sensitivity analysis of in-pile critical heat flux experiments in TREAT for characterization of RIA power-transient effects
A reactivity-initiated accident (RIA) is one type of postulated design basis accident (DBA) that can cause a departure from nucleate boiling (DNB) event in pressurized-water reactors (PWRs). A DNB occurrence and its consequences depend on the thermophysical properties of the fuel components and coolant, characteristics of the transient energy insertion into the fuel rod, and the onset of the critical heat flux (CHF) phenomenon. To leverage the restart of the Transient Reactor Test (TREAT) Facility, an effort is currently underway to better understand the cladding-to-coolant heat transfer mechanisms and the CHF phenomenon under fast-transient irradiation conditions. This paper characterizes the impact of power transients on the thermal-hydraulic behavior of a TREAT Facility reactor heater rodlet CHF experiment to provide the priority of parameters that need to be investigated for an improved CHF model. Sobol sensitivity analysis methods and the Reactor Excursion and Leak Analysis Program (RELAP5-3D) code were used to identify key input parameters on the uncertainty in the prediction of peak outer- and inner-surface temperatures of the heater tube, as well as the time of the DNB event. A series of sensitivity analyses revealed the total energy deposition on the tube and the transient effects of power pulse had large impacts on the maximum temperatures. The CHF multiplier had the largest impact on the time occurrence of CHF. The overall results show the energy deposition rate in the tube is the most influencing factor to the manifestation of CHF and the resulting thermal-hydraulic behaviors of the tube. The multiplier for the CHF, which is interpreted as the predicted CHF value, has the largest Sobol indices for the time of the CHF in all cases, since it directly determines the occurrence of CHF. It is inferred that the uncertainties in the thermal-hydraulic behaviors of fuels increase with respect to the key parameters as the power pulse becomes broader, and an accurate estimation of the energy deposition rate is required to reduce the uncertainty in the evaluation of the integrity of fuel if the CHF is expected to occur near the peak power. Therefore, the outputs are expected to provide rigorous interpretation of ongoing in-pile CHF experiments in the TREAT Facility reactor regarding thermal-hydraulic behavior of the fuel system aiming for a new transient in-pile CHF model.
Uncertainty quantification of bank vegetation impacts on the flood flow field in the American River, California, using large‐eddy simulations
Bank vegetation plays a key role in both hydrodynamics and morphodynamics of natural rivers; however, these effects are often unaccounted for in the computational flow dynamics of natural waterways. Recent studies using the large‐eddy simulation (LES), however, have attempted to gain insights into the impacts of bank vegetation on the mean flow field of the natural rivers using a vegetation model, which applies a sink term to the momentum equations of motion. This approach accounts for the effects of the vegetation and provides a practical approach to account for the complex patches of bank vegetation in large‐scale rivers. To implement the vegetation model, a drag coefficient reflecting the overall resistance of vegetal structures to the flow is needed, but due to the lack of calibrated data and range of size, density and type of vegetation, this parameter can be a significant source of uncertainty in the model results. Here, in this study, we use uncertainty quantification (UQ) to investigate the hydrodynamics and bed shear results when a bank vegetation is incorporated in an LES model. To this end, we used the polynomial chaos expansion and Monte Carlo sampling techniques to determine the uncertainties associated with the drag coefficient in the vegetation model and from uncertainties in the bed roughness and inflow discharge. The UQ analysis provided spatially varying confidence levels for the spanwise and vertical distribution of velocity magnitude and for the bed shear stress distributions. In addition, Sobol indices were computed to indicate the relative influence that each parameter had on the overall uncertainty. In general, it was found that uncertainty in flow discharge was the dominant source of uncertainty; however, the drag coefficient in the vegetation model and the bed roughness parameter also made significant contribution to the uncertainty near the banks and bed, respectively.
Benchmarking FFTF LOFWOS Test# 13 using SAM code: Baseline model development and uncertainty quantification
The development and deployment of advanced reactors, such as the sodium-cooled fast reactor (SFR), relies on sophisticated modeling tools to ensure the safety of the design under various transients. The predictive capability of these advanced modeling tools requires validation to garner trust in supporting the licensing of the advanced reactors. For this reason, the International Atomic Energy Agency (IAEA) initiated a coordinated research project (CRP) in 2018 for the analysis of the Fast Flux Test Facility (FFTF) Loss of Flow Without Scram (LOFWOS) Test #13.In this study, we present and discuss the benchmarking efforts of the modern system code SAM on the FFTF LOFWOS Test #13. Further, the SAM baseline model was developed according to the benchmark specification, which included a detailed core model with reactivity feedback. Generally, good agreement was observed between the baseline results and benchmark measurements; however, discrepancies persisted, particularly in predicted fuel assembly coolant outlet temperatures. Utilizing the baseline model, uncertainty quantification (UQ) and sensitivity analysis (SA) were conducted with the assistance of various statistical learning and machine learning methods, including kernel density estimation, Gaussian processes, and Sobol indices. Following the baseline model prediction and UQ and SA results, we discuss the reasons for the simulation discrepancies and propose further improvements to the model. This benchmarking effort adheres to the best-estimate plus uncertainty approach and can serve as a valuable example for supporting risk-informed licensing of advanced reactors.
Uncertainty quantification for competing failure mechanisms in unidirectionally reinforced carbon–carbon composites
Microstructure-informed finite element models play a key role in the carbon–carbon composite design process. Variability in manufacturing process parameters and experimental limitations introduce model parameter uncertainty. This study quantifies the effect of model parameter uncertainty on transverse tensile fracture behavior and proposes a methodology to predict the failure mode based on competing microscale damage mechanisms. Finite element simulations incorporate fiber–matrix interface debonding with cohesive zones and matrix damage with a smeared crack band approach in a unidirectional carbon–carbon composite. Results from a variance-based global sensitivity analysis identifies interfacial and matrix damage parameters as the primary source of variability in fracture behavior. Sobol’ indices indicate that matrix and cohesive zone strengths contribute 94% of the variance in the effective ultimate stress. A local analysis elucidates the relationship between these constituent strength parameters and failure mode by estimating the probability of cohesive, matrix, and mixed-mode dominated failure. Based on the results for 4000 simulations, 93% exhibit mixed-mode or interfacial dominated failure, which underscores the crucial role of fiber–matrix interface debonding in the transverse tensile failure of carbon–carbon composites. These uncertainty quantification results facilitate more efficient model calibration and provide a framework for microstructure-informed failure predictions in the face of manufacturing-induced uncertainty.