Search NASA⌕ Search

SEARCH · Search NASA

Results for “Variational Bayesian”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Bayesian Poroelastic Aquifer Characterization From InSAR Surface Deformation Data. Part I: Maximum A Posteriori Estimate

Characterizing the properties of groundwater aquifers is essential for predicting aquifer response and managing groundwater resources. In this work, we develop a high-dimensional scalable Bayesian inversion framework governed by a three-dimensional quasi-static linear poroelastic model to characterize lateral permeability variations in groundwater aquifers. In this work, we determine the maximum a posteriori (MAP) point of the posterior permeability distribution from centimeter-level surface deformation measurements obtained from Interferometric Synthetic Aperture Radar (InSAR). The scalability of our method to high parameter dimension is achieved through the use of adjoint-based derivatives, inexact Newton methods to determine the MAP point, and a Mat´ern class sparse prior precision operator. Together, these guarantee that the MAP point is found at a cost, measured in number of forward/adjoint poroelasticity solves, that is independent of the parameter dimension. We apply our methodology to a test case for a municipal well in Mesquite, Nevada, in which InSAR and GPS surface deformation data are available. We solve problems with up to 320,824 state variable degrees of freedom (DOFs) and 16,896 parameter DOFs. A consistent treatment of noise level is employed so that the aquifer characterization result does not depend on the pixel spacing of surface deformation data. Our results show that the use of InSAR data significantly improves characterization of lateral aquifer heterogeneity, and the InSAR-based aquifer characterization recovers complex lateral displacement trends observed by independent daily GPS measurements.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Preparation of the SU(3) lattice Yang-Mills vacuum with variational quantum methods

Studying QCD and other gauge theories on quantum hardware requires the preparation of physically interesting states. The variational quantum eigensolver provides a way of performing vacuum state preparation on quantum hardware. Here in this work, variational quantum eigensolver is applied to pure SU(3) lattice Yang-Mills on a single plaquette and one dimensional plaquette chains. Bayesian optimization and gradient descent were investigated for performing the classical optimization. Ansatz states for plaquette chains are constructed in a scalable manner from smaller systems using domain decomposition and a stitching procedure analogous to the density matrix renormalization group. Small examples are performed on IBM’s superconducting Manila processor.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

Scalable and efficient learning from crowds with Gaussian processes

Over the last few years, multiply-annotated data has become a very popular source of information. Online platforms such as Amazon Mechanical Turk have revolutionized the labelling process needed for any classification task, sharing the effort between a number of annotators (instead of the classical single expert). This crowdsourcing approach has introduced new challenging problems, such as handling disagreements on the annotated samples or combining the unknown expertise of the annotators. Probabilistic methods, such as Gaussian Processes (GP), have proven successful to model this new crowdsourcing scenario. However, GPs do not scale up well with the training set size, which makes them prohibitive for medium-to-large datasets (beyond 10K training instances). This constitutes a serious limitation for current real-world applications. In this work, we introduce two scalable and efficient GP-based crowdsourcing methods that allow for processing previously-prohibitive datasets. The first one is an efficient and fast approximation to GP with squared exponential (SE) kernel. The second allows for learning a more flexible kernel at the expense of a heavier training (but still scalable to large datasets). Since the latter is not a GP-SE approximation, it can be also considered as a whole new scalable and efficient crowdsourcing method, useful for any dataset size. Both methods use Fourier features and variational inference, can predict the class of new samples, and estimate the expertise of the involved annotators. A complete experimentation compares them with state-of-the-art probabilistic approaches in synthetic and real crowdsourcing datasets of different sizes. Finally, they stand out as the best performing approach for large scale problems. Moreover, the second method is competitive with the current state-of-the-art for small datasets.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2026

mmbo (Multi-modal Bayesian Optimization) [SWR-25-119]

This package implements Bayesian Neural Network (BNN) based surrogate models for multi-modal data. These models are fit using a custom Variational Bayes procedure leveraging conjugate posteriors in the last layer of the network, offering more accurate predictions with better-calibrated uncertainty quantification than mean-field Variational Bayes.

Taylor, Ian [National Laboratory of the Rockies (N↗
DOE OSTI2025

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Assessing the Influence of Microphysical and Environmental Parameter Perturbations on Orographic Precipitation

Microphysical (MP) schemes contain parameters whose values can impact the amount and location of forecasted precipitation, and sensitivity is typically explored by perturbing one parameter at a time while holding the rest constant. Although much can be learned from these “one-at-a-time” studies, the results are limited as these methods do not allow for nonlinear interactions of multiple perturbed parameters. This study applies the Morris one-at-a-time (MOAT) method, a robust statistical tool allowing for simultaneous perturbation of numerous parameters, to explore orographic precipitation sensitivity to changes in microphysical and environmental parameters within an environment characteristic of an atmospheric river. Results show parameters associated with snow fall speed coefficient A s and density ρ s , ice-cloud water collection efficiency (ECI), rain accretion (WRA), relative humidity, zonal wind speed, and surface potential temperature cause the largest influence on simulated precipitation. MP and environmental parameter perturbations can cause precipitation changes of similar magnitude, but results vary by location on the mountain. Different environments are also tested, with A s being the most influential MP parameter regardless of environment. Fewer MP parameters influence precipitation in a faster-wind-speed environment, possibly due to the stronger dynamical forcing upwind and different wave dynamics downwind, compared to a slower-wind-speed environment. Finally, perturbing MP parameters within a single scheme can result in precipitation variations of similar magnitude compared to using entirely different microphysics schemes. MOAT results presented in this study have implications for Bayesian parameter estimation methods and stochastic parameterization within ensemble forecasting.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2017

Analyzing degradation data with a random effects spline regression model

This study proposes using a random effects spline regression model to analyze degradation data. Spline regression avoids having to specify a parametric function for the true degradation of an item. A distribution for the spline regression coefficients captures the variation of the true degradation curves from item to item. We illustrate the proposed methodology with a real example using a Bayesian approach. The Bayesian approach allows prediction of degradation of a population over time and estimation of reliability is easy to perform.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Data-driven prediction of scaling and ignition of inertial confinement fusion experiments

Recent advances in inertial confinement fusion (ICF) at the National Ignition Facility (NIF), including ignition and energy gain, are enabled by a close coupling between experiments and high-fidelity simulations. Neither simulations nor experiments can fully constrain the behavior of ICF implosions on their own, meaning pre- and postshot simulation studies must incorporate experimental data to be reliable. Linking past data with simulations to make predictions for upcoming designs and quantifying the uncertainty in those predictions has been an ongoing challenge in ICF research. We have developed a data-driven approach to prediction and uncertainty quantification that combines large ensembles of simulations with Bayesian inference and deep learning. The approach builds a predictive model for the statistical distribution of key performance parameters, which is jointly informed by past experiments and physics simulations. The prediction distribution captures the impact of experimental uncertainty, expert priors, design changes, and shot-to-shot variations. We have used this new capability to predict a 10× increase in ignition probability between Hybrid-E shots driven with 2.05 MJ compared to 1.9 MJ, and validated our predictions against subsequent experiments. We describe our new Bayesian postshot and prediction capabilities, discuss their application to NIF ignition and validate the results, and finally investigate the impact of data sparsity on our prediction results.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2010

Understanding the Scalability of Bayesian Network Inference Using Clique Tree Growth Curves

One of the main approaches to performing computation in Bayesian networks (BNs) is clique tree clustering and propagation. The clique tree approach consists of propagation in a clique tree compiled from a Bayesian network, and while it was introduced in the 1980s, there is still a lack of understanding of how clique tree computation time depends on variations in BN size and structure. In this article, we improve this understanding by developing an approach to characterizing clique tree growth as a function of parameters that can be computed in polynomial time from BNs, specifically: (i) the ratio of the number of a BN s non-root nodes to the number of root nodes, and (ii) the expected number of moral edges in their moral graphs. Analytically, we partition the set of cliques in a clique tree into different sets, and introduce a growth curve for the total size of each set. For the special case of bipartite BNs, there are two sets and two growth curves, a mixed clique growth curve and a root clique growth curve. In experiments, where random bipartite BNs generated using the BPART algorithm are studied, we systematically increase the out-degree of the root nodes in bipartite Bayesian networks, by increasing the number of leaf nodes. Surprisingly, root clique growth is well-approximated by Gompertz growth curves, an S-shaped family of curves that has previously been used to describe growth processes in biology, medicine, and neuroscience. We believe that this research improves the understanding of the scaling behavior of clique tree clustering for a certain class of Bayesian networks; presents an aid for trade-off studies of clique tree clustering using growth curves; and ultimately provides a foundation for benchmarking and developing improved BN inference and machine learning algorithms.

Mengshoel, Ole J.↗
DOE OSTI2016

Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction

Two prime bottlenecks to the solution of large-scale Bayesian inverse problems are the scaling of posterior sampling algorithms to high-dimensional parameter spaces and the computational cost of forward model evaluations. However, incomplete or noisy data, the state variation and parameter dependence of the forward model, and correlations in the prior collectively provide useful structure that can be exploited for dimension reduction in this setting—both in the parameter space of the inverse problem and in the state space of the forward model. To this end, we show how to jointly construct low-dimensional subspaces of the parameter space and the state space in order to accelerate the Bayesian solution of the inverse problem. As a byproduct of state dimension reduction, we also show how to identify low-dimensional subspaces of the data in problems with high-dimensional observations. These subspaces enable approximation of the posterior as a product of two factors: (i) a projection of the posterior onto a low-dimensional parameter subspace, wherein the original likelihood is replaced by an approximation involving a reduced model; and (ii) the marginal prior distribution on the high-dimensional complement of the parameter subspace. We introduce and compare several strategies for constructing these subspaces using only a limited number of forward and adjoint model simulations. The resulting posterior approximations can rapidly be characterized using standard sampling techniques, e.g., Markov chain Monte Carlo. Two numerical examples demonstrate the accuracy and efficiency of our approach: inversion of an integral equation in atmospheric remote sensing, where the data dimension is very high; and the inference of a heterogeneous transmissivity field in a groundwater system, which involves a partial differential equation forward model with high dimensional state and parameters.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Spike-and-Slab Shrinkage Priors for Structurally Sparse Bayesian Neural Networks

Network complexity and computational efficiency have become increasingly significant aspects of deep learning. Sparse deep learning addresses these challenges by recovering a sparse representation of the underlying target function by reducing heavily overparameterized deep neural networks. Specifically, deep neural architectures compressed via structured sparsity (e.g., node sparsity) provide low-latency inference, higher data throughput, and reduced energy consumption. In this article, we explore two well-established shrinkage techniques, Lasso and Horseshoe, for model compression in Bayesian neural networks (BNNs). To this end, we propose structurally sparse BNNs, which systematically prune excessive nodes with the following: 1) spike-and-slab group Lasso (SS-GL) and 2) SS group Horseshoe (SS-GHS) priors, and develop computationally tractable variational inference, including continuous relaxation of Bernoulli variables. We establish the contraction rates of the variational posterior of our proposed models as a function of the network topology, layerwise node cardinalities, and bounds on the network weights. Furthermore, we empirically demonstrate the competitive performance of our models compared with the baseline models in prediction accuracy, model compression, and inference latency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

GIGA-Lens: Fast Bayesian Inference for Strong Gravitational Lens Modeling

We present GIGA-Lens: a gradient-informed, GPU-accelerated Bayesian framework for modeling strong gravitational lensing systems, implemented in TensorFlow and JAX. The three components, optimization using multistart gradient descent, posterior covariance estimation with variational inference, and sampling via Hamiltonian Monte Carlo, all take advantage of gradient information through automatic differentiation and massive parallelization on graphics processing units (GPUs). We test our pipeline on a large set of simulated systems and demonstrate in detail its high level of performance. The average time to model a single system on four Nvidia A100 GPUs is 105 s. The robustness, speed, and scalability offered by this framework make it possible to model the large number of strong lenses found in current surveys and present a very promising prospect for the modeling of ${ \mathcal O }({10}^{5})$ lensing systems expected to be discovered in the era of the Vera C. Rubin Observatory, Euclid, and the Nancy Grace Roman Space Telescope.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

Approximate Bayesian Model Inversion for PDEs with Heterogeneous and State-Dependent Coefficients

We present two approximate Bayesian inference methods for parameter estimation in partial differential equation (PDE) models with space-dependent and state-dependent parameters. We demonstrate that these methods provide accurate and cost-effective alternatives to Markov Chain Monte Carlo simulation. We assume a parameterized Gaussian prior on the unknown functions, and approximate the posterior density by a parameterized multivariate Gaussian density. The parameters of the prior and posterior are estimated from sparse observations of the PDE model's states and the unknown functions themselves by maximizing the evidence lower bound (ELBO), a lower bound on the log marginal likelihood of the observations. The first method, Laplace-EM, employs the expectation maximization algorithm to maximize the ELBO, with a Laplace approximation of the posterior on the E-step, and minimization of a Kullback-Leibler divergence on the M-step. The second method, DSVI-EB, employs the doubly stochastic variational inference (DSVI) algorithm, in which the ELBO is maximized via gradient-based stochastic optimization, with nosiy gradients computed via simple Monte Carlo sampling and Gaussian backpropagation. We apply these methods to identifying diffusion coefficients in linear and nonlinear diffusion equations, and we find that both methods provide accurate estimates of posterior densities and the hyperparameters of Gaussian priors. While the Laplace-EM method is more accurate, it requires computing Hessians of the physics model. The DSVI-EB method is found to be less accurate but only requires gradients of the physics model.

approximate Bayesian inference, model inversion, v↗
NASA NTRS1993

The inference of mantle viscosity from an inversion of the Fennoscandian relaxation spectrum

The theory of Peltier (1976) for mantle viscosity is adopted, within the framework of nonlinear Bayesian Inference, to invert the Fennoscandian relaxation spectra derived by McConnell (1968). A set of rigorous constraints which all models for the viscosity variation beneath Fennoscandian must satisfy is derived. These constraints are used to test the plausibility of a wide class of viscosity models. It is shown that a model with a weak asthenosphere overlying an isoviscous 10 exp 21 Pa s deep mantle provides to good fit to the relaxation spectrum. This is also true of models with a thin sublithospheric low-viscosity zone overlying a two-layer deep mantle with a moderate jump in viscosity across 670 km depth, and models with a viscosity jump of between four and six across isoviscous upper and lower mantle regions.

Mitrovica, J. X.↗
DOE OSTI2020

Dependence of the impurity transport on the dominant turbulent regime in ELM-y H-mode discharges on the DIII-D tokamak

Laser blow-off injections of aluminum and tungsten have been performed on the DIII-D tokamak to investigate the variation of impurity transport in a set of dedicated ion and electron heating scans with a fixed value of the external torque. The particle transport is quantified via the Bayesian inference method, which, constrained by a combination of a charge exchange recombination spectroscopy, soft X-ray measurements, and VUV spectroscopy provides a detailed uncertainty quantification of transport coefficients. Contrasting discharge phases with a dominant electron and ion heating reveal 3-fold drop in the impurity confinement time and order of magnitude increase in midradius impurity diffusion, when additional electron heating is applied. Further, the calculated stationary aluminum density profiles reverse from peaked in electron heated to hollow in the ion heated case, following a similar trend as electron and carbon density. Comparable values of a core diffusion have been observed for W and Al ions, while differences in the propagation dynamics of these impurities are attributed to pedestal and edge transport. Modeling of the core transport with non-linear gyrokinetics code CGYRO, significantly underpredicts the magnitude of the variation in Al transport. Diffusion increases 3-times steeper with additional electron heat flux and 10-times lower diffusion is observed in ion heated case than predicted by the modeling. Here, the CGYRO model quantitatively matches the increase of the Al diffusion when approaching the linear threshold for the transition from the ion temperature gradient (ITG) to trapped electron mode (TEM).

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2015

MODEL COMPARISON FOR THE DENSITY STRUCTURE ACROSS SOLAR CORONAL WAVEGUIDES

The spatial variation of physical quantities, such as the mass density, across solar atmospheric waveguides governs the timescales and spatial scales for wave damping and energy dissipation. The direct measurement of the spatial distribution of density, however, is difficult, and indirect seismology inversion methods have been suggested as an alternative. We applied Bayesian inference, model comparison, and model-averaging techniques to the inference of the cross-field density structuring in solar magnetic waveguides using information on periods and damping times for resonantly damped magnetohydrodynamic transverse kink oscillations. Three commonly employed alternative profiles were used to model the variation of the mass density across the waveguide boundary. Parameter inference enabled us to obtain information on physical quantities such as the Alfvén travel time, the density contrast, and the transverse inhomogeneity length scale. The inference results from alternative density models were compared and their differences quantified. Then, the relative plausibility of the considered models was assessed by performing model comparison. Our results indicate that the evidence in favor of any of the three models is minimal, unless the oscillations are strongly damped. In such a circumstance, the application of model-averaging techniques enables the computation of an evidence-weighted inference that takes into account the plausibility of each model in the calculation of a combined inversion for the unknown physical parameters.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

A Graphical Model for Fusing Diverse Microbiome Data

This paper develops a Bayesian graphical model for fusing disparate types of count data. The motivating application is the study of bacterial communities from diverse high-dimensional features, in this case, transcripts, collected from different treatments. In such datasets, there are no explicit correspondences between the communities and each corresponds to different factors, making data fusion challenging. We introduce a flexible multinomial-Gaussian generative model for jointly modeling such count data. This latent variable model jointly characterizes the observed data through a common multivariate Gaussian latent space that parameterizes the set of multinomial probabilities of the transcriptome counts. The covariance matrix of the latent variables induces a covariance matrix of co-dependencies between all the transcripts, effectively fusing multiple data sources. We present a computationally scalable variational Expectation-Maximization (EM) algorithm for inferring the latent variables and the parameters of the model. Here, the inferred latent variables provide a common dimensionality reduction for visualizing the data and the inferred parameters provide a predictive posterior distribution. In addition to simulation studies that demonstrate the variational EM procedure, we apply our model to a bacterial microbiome dataset.

59 BASIC BIOLOGICAL SCIENCES↗