SEARCH · Search NASA

Results for “Uncertainty Quantification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

SLS Integrated Modal Test Uncertainty Quantification using the Hybrid Parametric Variation Method

Uncertainty in structural loading during launch is a significant concern in the development of spacecraft and launch vehicles. Small variations in launch vehicle and payload mode shapes and their interaction can result in significant variation in system loads. In many cases involving large aerospace systems it is difficult, not economical, or impossible to perform a system modal test. However, it is still vital to obtain test results that can be compared with analytical predictions to validate models. Instead, the “Building Block Approach” is used in which system components are tested individually. Component models are correlated and updated to agree as best they can with test results. The Space Launch System consists of a number of components that are assembled into a launch vehicle. Finite element models of the components are developed, reduced to Hurty/Craig-Bampton models and assembled to represent different phases of flight. The only opportunity to obtain modal test data from an assembled Space Launch System will be during the Integrated Modal Test. There is always uncertainty in every model, which flows into uncertainty in predicted system results. Uncertainty Quantification is used to determine statistical bounds on prediction accuracy based on model uncertainty. For the Space Launch System, model uncertainty is at the Hurty/Craig-Bampton component level. Uncertainty in the Hurty/Craig-Bampton components is quantified using the hybrid parametric variation approach that combines parametric and nonparametric uncertainty. Uncertainty in model form is one of the biggest contributors to uncertainty in complex built-up structures. This type of uncertainty cannot be represented by variations infinite element model input parameters and thus cannot be included in a parametric approach. However, model-form uncertainty can be modeled using a nonparametric approach based on random matrix theory. The hybrid parametric variation method requires the selection of dispersion values for the Hurty/Craig-Bampton fixed-interface eigenvalues, and the Hurty/Craig-Bampton stiffness matrices. Component test/analysis frequency error is used to identify the fixed-interface eigenvalue dispersions, while test/analysis cross-orthogonality is used to identify stiffness dispersion values. The hybrid parametric variation uncertainty quantification approach is applied to the Space Launch System Integrated Modal Test configuration. Monte Carlo analysis is performed, and statistics are determined for modal correlation metrics, frequency response from Integrated Modal Test shakers to selected accelerometers, as well as other metrics for determining how well target modes are excited and identified. If the predicted uncertainty envelopes future Integrated Modal Test results, then there will be increased confidence in the utility of the component-based hybrid parametric variation uncertainty quantification approach.

Uncertainty Quantification↗

NASA NTRS2019

Uncertainty Quantification of a Rotorcraft Conceptual Sizing Toolsuite

A computational framework to support the quantification of system uncertainties and sensitivities for rotorcraft applications is presented using the NASA Design and Analysis of Rotorcraft (NDARC) conceptual sizing tool. A 90 passenger conceptual tiltrotor configuration was used for case demonstration in the modeling of uncertainties in NDARCs emission module. A non-intrusive forward propagation uncertainty quantification approach was applied to ensemble simulations using a Monte Carlo methodology with stratified Latin hypercube sampling. An off-the-shelf software, DAKOTA, which supports trade studies and design space exploration, including optimization, surrogate modeling and uncertainty analysis was used to address the research goals. A toolsuite was further developed incorporating DAKOTA with automated design processes and methods using function wrappers to execute program routines including support for data post-processing. Uncertainties in rotorcraft emissions modeling using the Average Temperature Response metric for a set mission profile were studied. It was shown that for the current study, using the base-line best estimate modeling parameters for the Average Temperature Response metric, NDARC under-estimates the effects of emissions when compared with results from Monte Carlo simulations. A global sensitivity analysis was further undertaken to quantify the contribution of the various emission species on output sensitivity, hence uncertainty. The work demonstrates that the developed toolsuite is robust and will support the quantification of system uncertainties and sensitivities in future rotorcraft design efforts.

Rotorcraft↗

DOE OSTI2020

Multilevel and multifidelity uncertainty quantification for cardiovascular hemodynamics

Standard approaches for uncertainty quantification in cardiovascular modeling pose challenges due to the large number of uncertain inputs and the significant computational cost of realistic three-dimensional simulations. Here, we propose an efficient uncertainty quantification framework utilizing a multilevel multifidelity Monte Carlo (MLMF) estimator to improve the accuracy of hemodynamic quantities of interest while maintaining reasonable computational cost. This is achieved by leveraging three cardiovascular model fidelities, each with varying spatial resolution to rigorously quantify the variability in hemodynamic outputs. We employ two low-fidelity models (zero- and one-dimensional) to construct several different estimators. Our goal is to investigate and compare the efficiency of estimators built from combinations of these two low-fidelity model alternatives and our high-fidelity three-dimensional models. We demonstrate this framework on healthy and diseased models of aortic and coronary anatomy, including uncertainties in material property and boundary condition parameters. Our goal is to demonstrate that for this application it is possible to accelerate the convergence of the estimators by utilizing a MLMF paradigm. Therefore, we compare our approach to single fidelity Monte Carlo estimators and to a multilevel Monte Carlo approach based only on three-dimensional simulations, but leveraging multiple spatial resolutions. We demonstrate significant, on the order of 10 to 100 times, reduction in total computational cost with the MLMF estimators. We also examine the differing properties of the MLMF estimators in healthy versus diseased models, as well as global versus local quantities of interest. As expected, global quantities such as outlet pressure and flow show larger reductions than local quantities, such as those relating to wall shear stress, as the latter rely more heavily on the highest fidelity model evaluations. Similarly, healthy models show larger reductions than diseased models. In all cases, our workflow coupling Dakota’s MLMF estimators with the SimVascular cardiovascular modeling framework makes uncertainty quantification feasible for constrained computational budgets.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

Bayesian uncertainty quantification of tristructural isotropic particle fuel silver release: Decomposing model inadequacy plus experimental noise and parametric uncertainties

Tristructural isotropic (TRISO) particle fuel is one of the most promising fuel concepts enabling high temperature and high burnup reactor operation. One dominant source of radioactivity released from the TRISO particles is silver (Ag), which is subject to a high release fraction and long decay life compared to other fission products. Previous modeling efforts using the fuel performance code BISON indicated nonnegligible uncertainties in modeling the diffusion process of fission products in TRISO compared to the Advanced Gas Reactor experiments. The overall uncertainties observed when modeling the fission product diffusion can result from uncertainties in model parameters, noisy experimental measurements, and deficiencies in the developed models. The three types of underlying uncertainties have not yet been properly quantified in open literature. Here, this paper presents the Bayesian uncertainty quantification (UQ) using massively parallelizable Markov chain Monte Carlo samplers. The uncertainties due to model parameters, model inadequacy, and experimental measurement noise are quantified, with the σ term used to represent the sum of the model inadequacy and measurement noise uncertainties. It is worth noting that this is the first time the σ term is inferred for nuclear fuel experiments, as compared to using prescribed values for uncertainty quantification in previous work. The parallelizable Markov chain Monte Carlo samplers efficiently infer the model parameters and the σ term, giving insight into physical parameters like diffusion coefficients and the combined model discrepancy and measurement noise. A subsequent forward uncertainty quantification (UQ) is also performed based on the calibration results to generate more accurate predictions of the Ag release. The model inadequacy plus experimental noise is the most dominant source of uncertainty compared to the parametric uncertainty. All the UQ analyses presented in this work are based on the second series of the irradiation experiments in the Advanced Gas Reactor program.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2020

Modern Monte Carlo methods for efficient uncertainty quantification and propagation: A survey

Uncertainty quantification (UQ) includes the characterization, integration, and propagation of uncertainties that result from stochastic variations and a lack of knowledge or data in the natural world. Monte Carlo (MC) method is a sampling-based approach that has widely used for quantification and propagation of uncertainties. However, the standard MC method is often time-consuming if the simulation-based model is computationally intensive. This article gives an overview of modern MC methods to address the existing challenges of the standard MC in the context of UQ. Specifically, multilevel Monte Carlo (MLMC) extending the concept of control variates achieves a significant reduction of the computational cost by performing most evaluations with low accuracy and corresponding low cost, and relatively few evaluations at high accuracy and corresponding high cost. Multifidelity Monte Carlo (MFMC) accelerates the convergence of standard Monte Carlo by generalizing the control variates with different models having varying fidelities and varying computational costs. Multimodel Monte Carlo method (MMMC), having a different setting of MLMC and MFMC, aims to address the issue of UQ and propagation when data for characterizing probability distributions are limited. Multimodel inference combined with importance sampling is proposed for quantifying and efficiently propagating the uncertainties resulting from small data sets. All of these three modern MC methods achieve a significant improvement of computational efficiency for probabilistic UQ, particularly uncertainty propagation. Finally, an algorithm summary and the corresponding code implementation are provided for each of the modern MC methods. The extension and application of these methods are discussed in detail.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Interpretable Uncertainty Quantification in AI for HEP

Estimating uncertainty is at the core of performing scientific measurements in HEP: a measurement is not useful without an estimate of its uncertainty. The goal of uncertainty quantification (UQ) is inextricably linked to the question, "how do we physically and statistically interpret these uncertainties?" The answer to this question depends not only on the computational task we aim to undertake, but also on the methods we use for that task. For artificial intelligence (AI) applications in HEP, there are several areas where interpretable methods for UQ are essential, including inference, simulation, and control/decision-making. There exist some methods for each of these areas, but they have not yet been demonstrated to be as trustworthy as more traditional approaches currently employed in physics (e.g., non-AI frequentist and Bayesian methods). Shedding light on the questions above requires additional understanding of the interplay of AI systems and uncertainty quantification. We briefly discuss the existing methods in each area and relate them to tasks across HEP. We then discuss recommendations for avenues to pursue to develop the necessary techniques for reliable widespread usage of AI with UQ over the next decade.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2020

CALPHAD Uncertainty Quantification and TDBX

CALPHAD uncertainty quantification (UQ) is the foundation of materials design with quantified confidence. We report a framework and software packages to enable CALPHAD UQ assessment and calculation using commercial CALPHAD software (Thermo-Calc). This Bayesian inference framework is coupled with a Markov chain Monte Carlo algorithm to establish uncertainty traces with a given thermodynamic database file (TDB) and corresponding experimental data points. This general framework is demonstrated with the Ni–Cr binary system. The algorithm is firstly validated on synthetic data with known ground truth. Then it is applied to real experimental data to generate posterior traces. We develop a file format named TDBX, which provides a single source of truth by combining the original TDB content and the traces for each assessed Gibbs energy parameter. CALPHAD UQ calculations are performed based on the TDBX file, from which uncertainties for phase boundaries, enthalpy curves, and solidification range are collected as examples of basic design parameters. This TDBX file with corresponding scripts are made open-source. Finally, the combination of CALPHAD UQ assessments and calculations connected by TDBX supports uncertainty-assisted modeling, enabling the integrated application of modern design with uncertainty methodologies to computational materials design.

36 MATERIALS SCIENCE↗

NASA NTRSDate not supplied

End-To-End Uncertainty Quantification with Analytical Derivatives for Design Under Uncertainty

Uncertainty quantification (UQ) is a rapidly growing and evolving discipline, especially within the aerospace community. Performing analysis with UQ can provide decision makers with a wealth of information about a candidate design. However, the value of UQ is fully realized when the information gained during UQ analysis is leveraged in a feedback loop of a design optimization process, often referred to as design under uncertainty. Although design under uncertainty can be a powerful risk mitigation technique, there are a number of roadblocks that prevent its implementation. Two primary factors are computational costs and added complexity of the analysis. High fidelity simulations on the order tens of uncertain variables quickly become computationally infeasible. Also, implementing UQ into an existing multidisciplinary design and optimization (MDO) process often requires extensive knowledge of the UQ methods and careful treatment of the problem formulation. The objective of this work is to address these two primary roadblocks and enable practitioners to efficiently perform design under uncertainty with limited knowledge of the UQ discipline. Methods outlined in this paper demonstrate MDO incorporating UQ into the design process, leveraging an analytic derivative tool chain through the entire optimization. The proposed approach leverages machine learning techniques to generate a differentiable confidence interval output from polynomial chaos models. This technique, coupled with the incorporation of analytical derivatives through the Polynomial Chaos Expansion (PCE) process, eliminates the need to estimate derivatives which are usually obtained from finite difference, complex step, or similar methods. Developing a differentiable confidence interval allows mixed uncertainty problems (both epistemic and aleatory) to be modeled. Without such modeling, these problems cannot accurately predict objective functions containing statistical quantities such as mean and variance. The addition of analytic derivatives to a polynomial chaos-based UQ method decreases the computational costs of performing design under uncertainty by orders of magnitude in comparison with methods such as complex step. The method and codes developed are modular in nature and are a drop-in solution for design under uncertainty within existing MDO problems. A low-fidelity analytical multidisciplinary optimization under uncertainty for a wing design in OpenMDAO is detailed in this paper. This demonstration case will include both objective functions and constraints which are influenced by uncertain parameters.

Ben D Phillips↗

NASA NTRSDate not supplied

End-To-End Uncertainty Quantification with Analytical Derivatives for Design Under Uncertainty

Ben Phillips↗

DOE OSTI2023

Deep neural network uncertainty quantification for LArTPC reconstruction

We evaluate uncertainty quantification (UQ) methods for deep learning applied to liquid argon time projection chamber (LArTPC) physics analysis tasks. As deep learning applications enter widespread usage among physics data analysis, neural networks with reliable estimates of prediction uncertainty and robust performance against overconfidence and out-of-distribution (OOD) samples are critical for their full deployment in analyzing experimental data. While numerous UQ methods have been tested on simple datasets, performance evaluations for more complex tasks and datasets are scarce. Here we assess the application of selected deep learning UQ methods on the task of particle classification using the PiLArNet monte carlo 3D LArTPC point cloud dataset. We observe that UQ methods not only allow for better rejection of prediction mistakes and OOD detection, but also generally achieve higher overall accuracy across different task settings. We assess the precision of uncertainty quantification using different evaluation metrics, such as distributional separation of prediction entropy across correctly and incorrectly identified samples, receiver operating characteristic curves (ROCs), and expected calibration error from observed empirical accuracy. We conclude that ensembling methods can obtain well calibrated classification probabilities and generally perform better than other existing methods in deep learning UQ literature.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Bi-fidelity variational auto-encoder for uncertainty quantification

Quantifying the uncertainty of quantities of interest (QoIs) from physical systems is a primary objective in model validation. However, achieving this goal entails balancing the need for computational efficiency with the requirement for numerical accuracy. To address this trade-off, we propose a novel bi-fidelity formulation of variational auto-encoders (BF-VAE) designed to estimate the uncertainty associated with a QoI from low-fidelity (LF) and high-fidelity (HF) samples of the QoI. Here, this model allows for the approximation of the statistics of the HF QoI by leveraging information derived from its LF counterpart. Specifically, we design a bi-fidelity auto-regressive model in the latent space which is integrated within the VAE’s probabilistic encoder–decoder structure. An effective algorithm is proposed to maximize the variational lower bound of the HF log-likelihood in the presence of limited HF data, resulting in the synthesis of HF realizations with a reduced computational cost. Additionally, we introduce the concept of the bi-fidelity information bottleneck (BF-IB) to provide an information-theoretic interpretation of the proposed BF-VAE model. Our numerical results demonstrate that the BF-VAE leads to considerably improved accuracy, as compared to a VAE trained using only HF data, when limited HF data is available.

42 ENGINEERING↗

DOE OSTI2021

Pre-conditioned BFGS-based uncertainty quantification in elastic full-waveform inversion

SUMMARY Full-waveform inversion has become an essential technique for mapping geophysical subsurface structures. However, proper uncertainty quantification is often lacking in current applications. In theory, uncertainty quantification is related to the inverse Hessian (or the posterior covariance matrix). Even for common geophysical inverse problems its calculation is beyond the computational and storage capacities of the largest high-performance computing systems. In this study, we amend the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm to perform uncertainty quantification for large-scale applications. For seismic inverse problems, the limited-memory BFGS (L-BFGS) method prevails as the most efficient quasi-Newton method. We aim to augment it further to obtain an approximate inverse Hessian for uncertainty quantification in FWI. To facilitate retrieval of the inverse Hessian, we combine BFGS (essentially a full-history L-BFGS) with randomized singular value decomposition to determine a low-rank approximation of the inverse Hessian. Setting the rank number equal to the number of iterations makes this solution efficient and memory-affordable even for large-scale problems. Furthermore, based on the Gauss–Newton method, we formulate different initial, diagonal Hessian matrices as pre-conditioners for the inverse scheme and compare their performances in elastic FWI applications. We highlight our approach with the elastic Marmousi benchmark model, demonstrating the applicability of pre-conditioned BFGS for large-scale FWI and uncertainty quantification.

58 GEOSCIENCES↗

DOE OSTI2025

Uncertainty Quantification for Neutron Shield Using Convolutional Neural Networks

Uncertainty quantification from radiation transport calculations was conducted using a Bayesian inference approach. A surrogate model, using a convolutional neural network, was employed to emulate the neutron fluence, which was simulated with a Monte Carlo radiation transport model. This allowed for a computationally cheap approach to evaluate input parameters and to sample their corresponding posterior probability distributions. Experimental data from the literature were employed to perform uncertainty quantification studies for concrete shields. As a result, the method is a nonintrusive approach that enables studies with multiple input parameters and can be applied to any radiation transport model.

Bayesian inference↗

NASA NTRS2016

Uncertainty Reduction using Bayesian Inference and Sensitivity Analysis: A Sequential Approach to the NASA Langley Uncertainty Quantification Challenge

This paper presents a computational framework for uncertainty characterization and propagation, and sensitivity analysis under the presence of aleatory and epistemic un- certainty, and develops a rigorous methodology for efficient refinement of epistemic un- certainty by identifying important epistemic variables that significantly affect the overall performance of an engineering system. The proposed methodology is illustrated using the NASA Langley Uncertainty Quantification Challenge (NASA-LUQC) problem that deals with uncertainty analysis of a generic transport model (GTM). First, Bayesian inference is used to infer subsystem-level epistemic quantities using the subsystem-level model and corresponding data. Second, tools of variance-based global sensitivity analysis are used to identify four important epistemic variables (this limitation specified in the NASA-LUQC is reflective of practical engineering situations where not all epistemic variables can be refined due to time/budget constraints) that significantly affect system-level performance. The most significant contribution of this paper is the development of the sequential refine- ment methodology, where epistemic variables for refinement are not identified all-at-once. Instead, only one variable is first identified, and then, Bayesian inference and global sensi- tivity calculations are repeated to identify the next important variable. This procedure is continued until all 4 variables are identified and the refinement in the system-level perfor- mance is computed. The advantages of the proposed sequential refinement methodology over the all-at-once uncertainty refinement approach are explained, and then applied to the NASA Langley Uncertainty Quantification Challenge problem.

Uncertainty↗

DOE OSTI2025

The Fluid Dynamics Uncertainty Quantification Challenge Problem: XFOIL vs. MFOIL

Uncertainty quantification (UQ) has become more critical in aerospace engineering due to the growing dependence on computational tools for design optimization and performance analyses of aerospace vehicles. Even though the significance of UQ in assessing the credibility of computational analyses is well recognized, its costs and complexity impede its integration into standard practices, particularly in computational fluid dynamics (CFD) and other fluid analyses. This paper presents a UQ study for low-fidelity computational aerodynamics analyses with XFOIL and mfoil (i.e., the MATLAB version of XFOIL with several implementation modifications); these tools are utilized widely in both research and education. The main contributions of this paper are as follows: 1) improved precision in quantifying the uncertainty of the baseline Monte Carlo results used to benchmark surrogate modeling techniques for UQ, 2) quantification of the effect of the implementation differences between XFOIL and mfoil on solution quantities of interest (QoIs), such as lift and pitching moment coefficients, and 3) development of an open-source UQ library for use with XFOIL and mfoil, which has educational values and helps promote UQ for fluid analyses with aerospace applications. Results and discussions revolve around cases 1-4 of the challenge problem posed by the AIAA Fluid Dynamics Technical Committee’s Uncertainty Quantification Discussion Group (UQDG). In case 3, this work employs CFDverify, an open-source solution verification software, to quantify the discretization error and evaluate the extrapolated QoIs based on the grid convergence index (GCI). This UQ study differentiates itself from previous studies in the rigor of handling baseline Monte Carlo uncertainty and in including mfoil, which is a more accessible alternative to XFOIL. Finally, despite the growing computing power, low-fidelity computational tools remain valuable, such as for aerodynamic shape optimization at Mach numbers below 0.65 and low-to-mid Reynolds numbers.

Lay, Aidan S [University of Tennessee, Knoxville (↗

DOE OSTI2023

Multifidelity Monte Carlo estimation for efficient uncertainty quantification in climate-related modeling

Abstract. Uncertainties in an output of interest that depends on the solution of a complex system (e.g., of partial differential equations with random inputs) are often, if not nearly ubiquitously, determined in practice using Monte Carlo (MC) estimation. While simple to implement, MC estimation fails to provide reliable information about statistical quantities (such as the expected value of the output of interest) in application settings such as climate modeling, for which obtaining a single realization of the output of interest is a costly endeavor. Specifically, the dilemma encountered is that many samples of the output of interest have to be collected in order to obtain an MC estimator that has sufficient accuracy – so many, in fact, that the available computational budget is not large enough to effect the number of samples needed. To circumvent this dilemma, we consider using multifidelity Monte Carlo (MFMC) estimation which leverages the use of less costly and less accurate surrogate models (such as coarser grids, reduced-order models, simplified physics, and/or interpolants) to achieve, for the same computational budget, higher accuracy compared to that obtained by an MC estimator – or, looking at it another way, an MFMC estimator obtains the same accuracy as the MC estimator at lower computational cost. The key to the efficacy of MFMC estimation is the fact that most of the required computational budget is loaded onto the less costly surrogate models so that very few samples are taken of the more expensive model of interest. We first provide a more detailed discussion about the need to consider an alternative to MC estimation for uncertainty quantification. Subsequently, we present a review, in an abstract setting, of the MFMC approach along with its application to three climate-related benchmark problems as a proof-of-concept exercise.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Economic Parameter Uncertainty Quantification Demonstration

HERON has recently added uncertainty quantification capabilities for economic parameters. Users may now associate a distribution with certain cash flow parameters to simulate uncertainty in cost or other economic inputs. For example, a user may want to capture the uncertainty of capital expenditures of an advanced nuclear power plant because public data is hard to find or is not available yet. This new addition allows users to account for that uncertainty.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

NASA NTRSDate not supplied

Probing Model Sensitivity in Discharge Characteristics for a Li-O2 Battery using Uncertainty Quantification

For this presentation, we looked into using uncertainty quantification to understand the uncertainty in model parameters in predicting voltage and time response of a lithium-air battery.

Lithium-air↗