Search NASA⌕ Search

SEARCH · Search NASA

Results for “Approximate Bayesian computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

CheKiPEUQ Intro 1: Bayesian Parameter Estimation Considering Uncertainty or Error from both Experiments and Theory**

A common goal is extraction of physico-chemical parameter values such as pre-exponentials and activation energies from experiment. Ever increasing knowledge from experiments and computations is enabling semi-quantitative prior predictions of such values. When prior knowledge of physically realistic ranges is available, a method named Bayesian parameter estimation (BPE) enables more physically realistic parameter estimation relative to unsophisticated fitting by seeking the most probable value when considering together the uncertainties from prior knowledge, experimental data, and approximations in the model. An impediment to widespread use of BPE is a lack of understanding, training, and user-friendly software. Along with this invited publication, a general software package for BPE is being released that is user-friendly and that does not require understanding of the math behind the methodology. Overall, two previously unpublished catalysis science examples are provided along with considerations and guidelines for successful application of BPE. Following this work, BPE can become more widespread to enable extraction of physically meaningful parameter values.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

A Gaussian process based surrogate approach for the optimization of cylindrical targets

Simulating direct-drive inertial confinement experiments presents significant computational challenges, both due to the complexity of the codes required for such simulations and the substantial computational expense associated with target design studies. Machine learning models, and in particular, surrogate models, offer a solution by replacing simulation results with a simplified approximation. In this study, we apply surrogate modeling and optimization techniques that are well established in the existing literature to one-dimensional simulation data of a new cylindrical target design containing deuterium–tritium fuel. These models predict yields without the need for expensive simulations. We find that Bayesian optimization with Gaussian process surrogates enhances sampling efficiency in low-dimensional design spaces but becomes less efficient as dimensionality increases. Nonetheless, optimization routines within two-dimensional and five-dimensional design spaces can identify designs that maximize yield, while also aligning with established physical intuition. Optimization routines, which ignore constraints on hydrodynamic instability growth, are shown to lead to unstable designs in 2D, resulting in yield loss. However, routines that utilize 1D simulations and impose constraints on the in-flight aspect ratio converge on novel cylindrical target designs that are stable against hydrodynamic instability growth in 2D and achieve high yield.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2004

Automatic Generation of Algorithms for the Statistical Analysis of Planetary Nebulae Images

Analyzing data sets collected in experiments or by observations is a Core scientific activity. Typically, experimentd and observational data are &aught with uncertainty, and the analysis is based on a statistical model of the conjectured underlying processes, The large data volumes collected by modern instruments make computer support indispensible for this. Consequently, scientists spend significant amounts of their time with the development and refinement of the data analysis programs. AutoBayes [GF+02, FS03] is a fully automatic synthesis system for generating statistical data analysis programs. Externally, it looks like a compiler: it takes an abstract problem specification and translates it into executable code. Its input is a concise description of a data analysis problem in the form of a statistical model as shown in Figure 1; its output is optimized and fully documented C/C++ code which can be linked dynamically into the Matlab and Octave environments. Internally, however, it is quite different: AutoBayes derives a customized algorithm implementing the given model using a schema-based process, and then further refines and optimizes the algorithm into code. A schema is a parameterized code template with associated semantic constraints which define and restrict the template s applicability. The schema parameters are instantiated in a problem-specific way during synthesis as AutoBayes checks the constraints against the original model or, recursively, against emerging sub-problems. AutoBayes schema library contains problem decomposition operators (which are justified by theorems in a formal logic in the domain of Bayesian networks) as well as machine learning algorithms (e.g., EM, k-Means) and nu- meric optimization methods (e.g., Nelder-Mead simplex, conjugate gradient). AutoBayes augments this schema-based approach by symbolic computation to derive closed-form solutions whenever possible. This is a major advantage over other statistical data analysis systems which use numerical approximations even in cases where closed-form solutions exist. AutoBayes is implemented in Prolog and comprises approximately 75.000 lines of code. In this paper, we take one typical scientific data analysis problem-analyzing planetary nebulae images taken by the Hubble Space Telescope-and show how AutoBayes can be used to automate the implementation of the necessary anal- ysis programs. We initially follow the analysis described by Knuth and Hajian [KHO2] and use AutoBayes to derive code for the published models. We show the details of the code derivation process, including the symbolic computations and automatic integration of library procedures, and compare the results of the automatically generated and manually implemented code. We then go beyond the original analysis and use AutoBayes to derive code for a simple image segmentation procedure based on a mixture model which can be used to automate a manual preproceesing step. Finally, we combine the original approach with the simple segmentation which yields a more detailed analysis. This also demonstrates that AutoBayes makes it easy to combine different aspects of data analysis.

Fischer, Bernd↗
DOE OSTI2023

Generalized Bayesian Framework for Evaluation of Integral Benchmark Experiments [Slides]

This presentation is on generalized Bayesian framework for evaluation of integral benchmark experiments. This presentation starts off with assumptions and approximations used with Bayes Theorem. then an overview of approximations used by ORNL codes, and Generalized Bayesian Monto Carlo (GBMC). The presentation then details out a precise framework. This presentation then concludes with considerations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Bayesian, Multifidelity Operator Learning for Complex Engineering Systems–A Position Paper

Abstract Deep learning has significantly improved the state-of-the-art in computer vision and natural language processing, and holds great potential to design effective tools for predicting and simulating complex engineering systems. In particular, scientific machine learning seeks to apply the power of deep learning to scientific and engineering tasks, with operator learning (OL) emerging as a particularly effective tool. OL can approximate nonlinear operators arising in complex engineering systems, making it useful for simulating, designing, and controlling those systems. In this position paper, we provide a comprehensive overview of OL, including its potential applications to complex engineering domains. We cover three variations of OL approaches: deterministic OL for modeling nonautonomous systems, OL with uncertainty quantification (UQ) capabilities, and multifidelity OL. For each variation, we discuss drawbacks and potential applications to engineering, in addition to providing a detailed explanation. We also highlight how multifidelity OL approaches with UQ capabilities can be used to design, optimize, and control engineering systems. Finally, we outline some potential challenges for OL within the engineering domain.

Computer Science↗
DOE OSTI2022

Model-Form Epistemic Uncertainty Quantification for Modeling with Differential Equations: Application to Epidemiology

Modeling real-world phenomena to any degree of accuracy is a challenge that the scientific research community has navigated since its foundation. Lack of information and limited computational and observational resources necessitate modeling assumptions which, when invalid, lead to model-form error (MFE). The work reported herein explored a novel method to represent model-form uncertainty (MFU) that combines Bayesian statistics with the emerging field of universal differential equations (UDEs). The fundamental principle behind UDEs is simple: use known equational forms that govern a dynamical system when you have them; then incorporate data-driven approaches – in this case neural networks (NNs) – embedded within the governing equations to learn the interacting terms that were underrepresented. Utilizing epidemiology as our motivating exemplar, this report will highlight the challenges of modeling novel infectious diseases while introducing ways to incorporate NN approximations to MFE. Prior to embarking on a Bayesian calibration, we first explored methods to augment the standard (non-Bayesian) UDE training procedure to account for uncertainty and increase robustness of training. In addition, it is often the case that uncertainty in observations is significant; this may be due to randomness or lack of precision in the measurement process. This uncertainty typically manifests as “noisy” observations which deviate from a true underlying signal. To account for such variability, the NN approximation to MFE is endowed with a probabilistic representation and is updated using available observational data in a Bayesian framework. By representing the MFU explicitly and deploying an embedded, data-driven model, this approach enables an agile, expressive, and interpretable method for representing MFU. In this report we will provide evidence that Bayesian UDEs show promise as a novel framework for any science-based, data-driven MFU representation; while emphasizing that significant advances must be made in the calibration of Bayesian NNs to ensure a robust calibration procedure.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Scalable Bayesian Physics-Informed Kolmogorov-Arnold Networks

Uncertainty quantification (UQ) plays a pivotal role in scientific machine learning, especially when surrogate models are used to approximate complex systems. Although multilayer perceptions (MLPs) are commonly employed as surrogates, they often suffer from overfitting due to their large number of parameters. Kolmogorov-Arnold networks (KANs) offer an alternative solution with fewer parameters. However, gradient-based inference methods, such as Hamiltonian Monte Carlo (HMC), may result in computational inefficiency when applied to KANs, especially for large-scale datasets, due to the high cost of back-propagation. To address these challenges, we propose a novel approach, combining the dropout Tikhonov ensemble Kalman inversion (DTEKI) with Chebyshev KANs. This gradient-free method effectively mitigates overfitting and enhances numerical stability. In addition, we incorporate the active subspace method to reduce the parameter-space dimensionality, allowing us to improve the accuracy of predictions and obtain more reliable uncertainty estimates. Extensive experiments demonstrate the efficacy of our approach in various test cases, including scenarios with large datasets and high noise levels. Our results show that the new method achieves comparable or better accuracy, much higher efficiency as well as stability compared to HMC, in addition to scalability. Moreover, by leveraging the low-dimensional parameter subspace, our method preserves prediction accuracy while substantially reducing further the computational cost.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

AEOLUS: Advances in Experimental Design, Optimal Control, and Learning for Uncertain Complex Systems

Sustained advances in the mathematics of modeling and simulation have resulted in the capability today for routine simulation of a number of large scale complex DOE-relevant systems. As remarkable as this capability for solving the so-called forward problem is, it is typically only the first step-an inner loop within an outer loop that explores the simulation model's parameter space and decision space to characterize uncertainty in the model's predictions, learn unknown model parameters from data, design the most informative experiments, determine optimal control strategies, and create optimal designs. Broadly, what unifies all of these outer loop problems is that they are, in one form or another, optimization problems over parameter/control/design space that are constrained by complex uncertain models. To fully realize the power of scientific simulation as a basis for scientific discovery, technological innovation, and rational decision-making, it is imperative to move beyond simulation to tackle the outer loop of optimization for learning from data, experimental design, and control with complex uncertain models. When the models under consideration are large-scale and complex, and when the optimization variable and uncertain parameter spaces are high (or infinite) dimensional, this constitutes a grand challenge of the highest order, and is intractable with conventional methods. To overcome these challenges, the AEOLUS Center was established to develop a unified mathematical, computational, and statistical framework for (1) Learning predictive models from complex data via Bayesian inference and optimization, and (2) Optimizing experiments, processes, and designs using the resulting uncertain models. These problems are intractable with conventional methods, for several reasons: (1) The simulation problems that govern the inner loops of the optimization problems are expensive to execute (due to severe nonlinearity, heterogeneity, multiphysics/multiscale coupling); (2) The optimization variable and uncertain parameter spaces are high dimensional, often stemming from discretizations of infinite dimensional fields such as initial conditions, sources, or material properties. We argue that the key to overcoming these challenges is to develop new mathematical, computational, and statistical methods that exploit the structure of the Bayesian inference and optimization problems mediated by their underlying complex uncertain models. This structure includes the regularity, sparsity, geometry, low intrinsic dimensionality, and multifidelity nature of the maps from uncertain parameter/optimization variable spaces to the specific objectives targeted: Bayesian inference, optimal experimental design, and optimal control design. Black box methods developed as generic tools are incapable of exploiting this structure. To be successful, we must create, integrate, and cross-fertilize ideas across multiple areas of applied math--including approximation theory, Bayesian inference, data science, experimental design, information theory, machine learning, model reduction, optimal control theory, parallel algorithms, PDE-constrained optimization, randomized algorithms, stochastic optimization, and uncertainty quantification--all while exploiting the structure of the problems at hand. With this goal in mind, we have marshaled a team of leading authorities in these areas. While the methods we develop will be broadly applicable across a wide spectrum of DOE problems in which experiments inform models and the systems those models describe must be optimized under uncertainty, we have chosen a specific area, advanced manufacturing and materials, to drive our work. AMM is characterized by complex models across multiple scales, and is a rich source of challenging problems in inference, experimental design, and optimal control, requiring multifaceted and integrated advances in applied mathematics. As such, AMM serves as an excellent vehicle to motivate and demonstrate the advances in applied mathematics developed by our center.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Bayesian Model Calibration for Extrapolative Prediction via Gibbs Posteriors

The current standard Bayesian approach to model calibration, which assigns a Gaussian process prior to the discrepancy term, often suffers from issues of unidentifiability and computational complexity and instability. When the goal is to quantify uncertainty in physical parameters for extrapolative prediction, then there is no need to perform inference on the discrepancy term. With this in mind, we introduce Gibbs posteriors as an alternative Bayesian method for model calibration, which updates the prior with a loss function connecting the data to the parameter. The target of inference is the physical parameter value which minimizes the expected loss. We propose to tune the loss scale of the Gibbs posterior to maintain nominal frequentist coverage under assumptions of the form of model discrepancy, and present a bootstrap implementation for approximating coverage rates. Our approach is highly modular, allowing an analyst to easily encode a wide variety of such assumptions. Furthermore, we provide a principled method of combining posteriors calculated from data subsets. We apply our methods to data from an experiment measuring the material properties of tantalum.

36 MATERIALS SCIENCE↗
DOE OSTI2026

A score-based diffusion model approach for adaptive learning of stochastic partial differential equation solutions

In this paper, we propose a novel framework for adaptively learning the time-evolving solutions of stochastic partial differential equations (SPDEs) using score-based diffusion models within a recursive Bayesian inference setting. SPDEs play a central role in modeling complex physical systems under uncertainty, but their numerical solutions often suffer from model errors and reduced accuracy due to incomplete physical knowledge and environmental variability. To address these challenges, we encode the governing physics into the score function of a diffusion model using simulation data and incorporate observational information via a likelihood-based correction in a reverse-time stochastic differential equation. This enables adaptive learning through iterative refinement of the solution as new data becomes available. To improve computational efficiency in high-dimensional settings, we introduce the ensemble score filter, a training-free approximation of the score function designed for real-time inference. Numerical experiments on benchmark SPDEs demonstrate the accuracy and robustness of the proposed method under sparse and noisy observations.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

Probabilistic Calibration of Expensive Models using Efficiently Trained Surrogates

Calibration of computational models in the presence of uncertainty is often cast as a Bayesian inference problem and solved via sampling methods, e.g., Markov chain Monte Carlo. When the computational model is expensive, this task becomes intractable due to the large number of samples required to accurately estimate the posterior distribution of the calibration parameters. A popular solution to this problem is to use machine learning to develop a faster-to-evaluate, lower-fidelity substitute for the original model to serve as a surrogate while solving the inference problem. Although considered an offline cost, generating training data to construct this surrogate model can still be an expensive task in practice. An active learning algorithm is presented that focuses training on improving surrogate accuracy specifically in and around the bulk of the posterior distribution, as this is where the model is exercised during calibration. Candidate samples are drawn from families of distributions related to an approximation of the posterior. The sample maximizing predictive variance is then selected for evaluation by the original computational model, yielding a label for the training point. Iterating this approach increases efficiency relative to space filling designs (e.g., Latin hypercube sampling) by avoiding low probability points. Practical considerations are discussed, including the benefits of using a sequential Monte Carlo sampling approach, convergence heuristics, and the importance of both exploration and exploitation given that the true posterior is unknown a priori.

uncertainty quantification↗
DOE OSTI2026

Quantum-Inspired Bayesian Sampling for Uncertainty Quantification and Machine Learning (Final Technical Report)

With increasing simulation and measurement data, machine learning and artificial intelligence have been widely used in computational decision-making of complex engineering systems. The resulting tools, such as uncertainty quantification solvers, reinforcement learning, and physics-informed machine learning, have achieved great success in critical DOE tasks such as material discovery and design, energy system modeling and control, and numerical weather and climate prediction. A core topic in scientific machine learning and artificial intelligence is Bayesian inference: given an observed data set, people want to estimate the posterior distribution of a (possibly large) number of hidden parameters. Due to the flexibility and weak assumptions, Bayesian sampling has been the mainstream Bayesian inference solvers despite the rapid progress of approximate Bayesian inference. Classical Bayesian sampling methods such as Markov-chain Monte Carlo suffer from a low-acceptance rate due to the random walk nature, therefore state-of-the-art techniques use Hamiltonian Monte Carlo and its variants to efficiently draw posterior samples in a high dimension. The key idea of Hamiltonian Monte Carlo and its variants is to simulate the Hamiltonian dynamics of a classical particle with a fixed mass, and their performance significantly degrades when the posterior distribution is highly spiky or has multiple modes. Leveraging the idea of quantum physics, this project has investigated new theory, algorithms and applications of Bayesian inference (especially Bayesian sampling). The main results include: (1) novel quantum-inspired Bayesian sampling methods that can lead to better accuracy for challenging multi-modal or spiky distributions, (2) more scalable machine learning framework leveraging tensor-compressed Bayesian inference, and (3) Bayesian and sampling approaches for verifying the robustness of continuous and binary neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

gpCAM v8

gpCAM is a Python software for large-scale Gaussian-Process driven uncertainty quantification, Bayesian Optimization, and Autonomous Experimentation. It is designed with maximum flexibility and customizability while offering record-breaking computing capabilities. In 2023 gpCAM broke the world record for Gaussian processes on large datasets set in 2019. Through clever programming and math, Gaussian Process building blocks, such as prior mean, kernel, and noise functions maintain their mathematical flexibility while supporting acceleration and scaling. This flexibility means that the approximation and uncertainty quantification can be domain-aware and extended over exotic input and output spaces.

Noack, Marcus↗
DOE OSTI2025

Surrogate models for linear response

Linear response theory is a well-established method in physics and chemistry for exploring excitations of many-body systems. In particular, the quasiparticle random-phase approximation (QRPA) provides a powerful microscopic framework by building excitations on top of the mean-field vacuum; however, its high computational cost limits model calibration and uncertainty quantification studies. Here, we present two complementary QRPA surrogate models and apply them to study response functions of finite nuclei. One is a reduced-order model that exploits the underlying QRPA structure, while the other utilizes the recently developed parametric matrix model algorithm to construct a map between the system’s Hamiltonian and observables. Our benchmark applications, the calculation of the electric dipole polarizability of 180 Yb and the 𝛽-decay half-life of 80 Ni, show that both emulators can achieve 0.1%–1% accuracy while offering a 6–7 orders of magnitude speedup compared to state-of-the-art QRPA solvers. These results demonstrate that the developed QRPA emulators are well positioned to enable Bayesian calibration and large-scale studies of computationally expensive physics models describing the properties of many-body systems.

Beta decay↗
NASA NTRS1993

Asteroid orbit determination using Bayesian probabilities

Bayesian a priori and a posteriori probability densities are employed to determine asteroid orbital elements from optical astrometric observations. For near-earth asteroids and Jupiter Trojans, the two-body approximation can yield grossly erroneous uncertainty estimates because of the scattering effect of close planetary encounters or prolonged weak planetary perturbations. For main-belt asteroids, the two-body approximation works efficiently, from which it is inferred that, although the ephemeris must be computed from an integrated orbit, only approximate partial derivatives are needed to attain sufficient accuracy in predicting ephemeris uncertainties. The semimajor and semiminor axes of the 2D 1-sigma sky-plane uncertainty ellipse during the years 1993 to 2005 in the many-body and two-body linear approximations are shown.

Muinonen, Karri↗
DOE OSTI2022

A mixed, unified forward/inverse framework for earthquake problems: fault implementation and coseismic slip estimate

SUMMARY We introduce a new finite-element (FE) based computational framework to solve forward and inverse elastic deformation problems for earthquake faulting via the adjoint method. Based on two advanced computational libraries, FEniCS and hIPPYlib for the forward and inverse problems, respectively, this framework is flexible, transparent and easily extensible. We represent a fault discontinuity through a mixed FE elasticity formulation, which approximates the stress with higher order accuracy and exposes the prescribed slip explicitly in the variational form without using conventional split node and decomposition discrete approaches. This also allows the first order optimality condition, that is the vanishing of the gradient, to be expressed in continuous form, which leads to consistent discretizations of all field variables, including the slip. We show comparisons with the standard, pure displacement formulation and a model containing an in-plane mode II crack, whose slip is prescribed via the split node technique. We demonstrate the potential of this new computational framework by performing a linear coseismic slip inversion through adjoint-based optimization methods, without requiring computation of elastic Green’s functions. Specifically, we consider a penalized least squares formulation, which in a Bayesian setting—under the assumption of Gaussian noise and prior—reflects the negative log of the posterior distribution. The comparison of the inversion results with a standard, linear inverse theory approach based on Okada’s solutions shows analogous results. Preliminary uncertainties are estimated via eigenvalue analysis of the Hessian of the penalized least squares objective function. Our implementation is fully open-source and Jupyter notebooks to reproduce our results are provided. The extension to a fully Bayesian framework for detailed uncertainty quantification and non-linear inversions, including for heterogeneous media earthquake problems, will be analysed in a forthcoming paper.

58 GEOSCIENCES↗
DOE OSTI2021

Cholesky-based experimental design for Gaussian process and kernel-based emulation and calibration.

Gaussian processes and other kernel-based methods are used extensively to construct approximations of multivariate data sets. The accuracy of these approximations is dependent on the data used. This paper presents a computationally efficient algorithm to greedily select training samples that minimize the weighted L p error of kernel-based approximations for a given number of data. The method successively generates nested samples, with the goal of minimizing the error in high probability regions of densities specified by users. The algorithm presented is extremely simple and can be implemented using existing pivoted Cholesky factorization methods. Training samples are generated in batches which allows training data to be evaluated (labeled) in parallel. For smooth kernels, the algorithm performs comparably with the greedy integrated variance design but has significantly lower complexity. Numerical experiments demonstrate the efficacy of the approach for bounded, unbounded, multi-modal and non-tensor product densities. We also show how to use the proposed algorithm to efficiently generate surrogates for inferring unknown model parameters from data using Bayesian inference.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Understanding Model Inadequacy in TRISO Nuclear Fuel Fission Products Release Models: Empirical and Mechanistic Approaches

The increasing use of tristructural isotropic (TRISO) particle fuel in both advanced and existing reactors necessitates a thorough evaluation of uncertainties and shortcomings in TRISO fission product release models. These inadequacies arise from the simplifications made in computational models compared to experimental data. Utilizing the BISON fuel performance code and experimental data from the Advanced Gas Reactor (AGR) program provides a unique chance to rigorously assess these inadequacies within a Bayesian uncertainty quantification (UQ) framework. This study contrasts the standard Bayesian framework with the Kennedy-O'Hagan (KOH) framework, which explicitly accounts for modeling inadequacies, in the context of UQ for TRISO silver release models. It examines both the traditional Arrhenius equation and a more advanced lower-length-scale (LLS)-informed model that incorporates microstructure information. The inverse UQ process applied to AGR-2 and AGR-3/4 datasets identified modeling inadequacy as the primary source of uncertainty, with experimental noise also being significant, while model parameter uncertainty was minimal. Both the Arrhenius and LLS-informed models showed similar levels of modeling inadequacy. For forward predictive UQ using the AGR-1 dataset, the KOH framework enhanced the accuracy and quality of quantified uncertainties by approximately 30% and 40%, respectively, compared to the standard Bayesian framework. This improvement was observed for both the Arrhenius and LLS-informed models. At the engineering scale, both models performed similarly, but the LLS-informed model outperformed the Arrhenius equation at the mesoscale. These findings underscore the importance of explicitly considering modeling inadequacy in the UQ process and highlight the need for ongoing refinement of physics-based models to address these shortcomings.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗