SEARCH · Search NASA

Results for “approximate Bayesian inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2022

Optimal Estimation Framework for Ocean Color Atmospheric Correction and Pixel-level Uncertainty Quantification

Ocean color remote sensing requires compensation for atmospheric scattering and absorption (aerosol, Rayleigh, and trace gases), referred to as atmospheric correction (AC). AC allows inference of parameters such as spectrally resolved remote sensing reflectance ( R rs )(λ) ; sr 1 ) at the ocean surface from the top-of-atmosphere reflectance. Often, the uncertainty of this process is not fully explored. Bayesian inference techniques provide a simultaneous AC and uncertainty assessment via a full posterior distribution of the relevant variables, given the prior distribution of those variables and the radiative transfer (RT) likelihood function. Given uncertainties in the algorithm inputs, the Bayesian framework enables better constraints on the AC process by using the complete spectral information compared to traditional approaches that use only a subset of bands for AC. This paper investigates a Bayesian inference research method (Optimal Estimation, OE) for ocean color AC by simultaneously retrieving atmospheric and ocean properties using all visible and near-infrared spectral bands. The OE algorithm analytically approximates the posterior distribution of parameters based on normality assumptions and provides a potentially viable operational algorithm with a reduced computational expense. We developed a Neural Network (NN) RT forward model look-up-table-based emulator to increase algorithm efficiency further and thus speed up the likelihood computations. We then applied the OE algorithm to synthetic data and observations from the MODerate resolution Imaging Spectroradiometer (MODIS) on NASA’s Aqua spacecraft. We compared the R rs )(λ) retrieval and its uncertainty estimates from the OE method with in-situ validation data from the SeaWiFS Bio-optical Archive and Storage System (SeaBASS) and Aerosol Robotic Network Ocean Color (AERONET-OC) datasets. The OE algorithm improved R rs )(λ) estimates relative to the NASA standard operational algorithm by improving all statistical metrics at 443, 555, and 667 nm. Unphysical negative R rs )(λ) , which often appear in complex water conditions, was reduced by a factor of 3. The OE-derived pixel-level R rs )(λ) uncertainty estimates were also assessed relative to in-situ data and were shown to have skill.

Atmospheric correction↗

NASA NTRS2001

Portfolios in Stochastic Local Search: Efficiently Computing Most Probable Explanations in Bayesian Networks

Portfolio methods support the combination of different algorithms and heuristics, including stochastic local search (SLS) heuristics, and have been identified as a promising approach to solve computationally hard problems. While successful in experiments, theoretical foundations and analytical results for portfolio-based SLS heuristics are less developed. This article aims to improve the understanding of the role of portfolios of heuristics in SLS. We emphasize the problem of computing most probable explanations (MPEs) in Bayesian networks (BNs). Algorithmically, we discuss a portfolio-based SLS algorithm for MPE computation, Stochastic Greedy Search (SGS). SGS supports the integration of different initialization operators (or initialization heuristics) and different search operators (greedy and noisy heuristics), thereby enabling new analytical and experimental results. Analytically, we introduce a novel Markov chain model tailored to portfolio-based SLS algorithms including SGS, thereby enabling us to analytically form expected hitting time results that explain empirical run time results. For a specific BN, we show the benefit of using a homogenous initialization portfolio. To further illustrate the portfolio approach, we consider novel additive search heuristics for handling determinism in the form of zero entries in conditional probability tables in BNs. Our additive approach adds rather than multiplies probabilities when computing the utility of an explanation. We motivate the additive measure by studying the dramatic impact of zero entries in conditional probability tables on the number of zero-probability explanations, which again complicates the search process. We consider the relationship between MAXSAT and MPE, and show that additive utility (or gain) is a generalization, to the probabilistic setting, of MAXSAT utility (or gain) used in the celebrated GSAT and WalkSAT algorithms and their descendants. Utilizing our Markov chain framework, we show that expected hitting time is a rational function - i.e. a ratio of two polynomials - of the probability of applying an additive search operator. Experimentally, we report on synthetically generated BNs as well as BNs from applications, and compare SGSs performance to that of Hugin, which performs BN inference by compilation to and propagation in clique trees. On synthetic networks, SGS speeds up computation by approximately two orders of magnitude compared to Hugin. In application networks, our approach is highly competitive in Bayesian networks with a high degree of determinism. In addition to showing that stochastic local search can be competitive with clique tree clustering, our empirical results provide an improved understanding of the circumstances under which portfolio-based SLS outperforms clique tree clustering and vice versa.

Mengshoel, Ole J.↗

DOE OSTI2024

Active learning for SNAP interatomic potentials via Bayesian predictive uncertainty

Bayesian inference with a simple Gaussian error model is used to efficiently compute prediction variances for energies, forces, and stresses in the linear SNAP interatomic potential. Here, the prediction variance is shown to have a strong correlation with the absolute error over approximately 24 orders of magnitude. Using this prediction variance, an active learning algorithm is constructed to iteratively train a potential by selecting the structures with the most uncertain properties from a pool of candidate structures. The relative importance of the energy, force, and stress errors in the objective function is shown to have a strong impact upon the trajectory of their respective net error metrics when running the active learning algorithm. Batched training of different batch sizes is also tested against singular structure updates, and it is found that batches can be used to significantly reduce the number of retraining steps required with only minor impact on the active learning trajectory.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Kernel learning backward SDE filter for data assimilation

In this paper, we develop a kernel learning backward SDE filter method to estimate the state of a stochastic dynamical system based on its partial noisy observations. A system of forward backward stochastic differential equations is used to propagate the state of the target dynamical model, and Bayesian inference is applied to incorporate the observational information. Further, to characterize the dynamical model in the entire state space, we introduce a kernel learning method to learn a continuous global approximation for the conditional probability density function of the target state by using discrete approximated density values as training data. Numerical experiments demonstrate that the kernel learning backward SDE is highly effective.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Recombination smooths the time-signal disrupted by latency in within-host HIV phylogenies

Within-host HIV evolution involves several features that may disrupt standard phylogenetic reconstruction. One important feature is re-activation of latently integrated provirus, which has the potential to disrupt the temporal signal, leading to variation in the branch lengths and apparent evolutionary rates in a tree. Yet, real within-host HIV phylogenies tend to show clear, ladder-like trees structured by the time of sampling. Another important feature is recombination, which violates the fundamental assumption that evolutionary history can be represented by a single bifurcating tree. Thus, recombination complicates the within-host HIV dynamic by mixing genomes and creating evolutionary loop structures that cannot be represented in bifurcating trees. In this paper, we develop a coalescent-based simulator of within-host HIV evolution that includes latency, recombination, and effective population size dynamics that allows us to study the relationship between the true, complex genealogy of within-host HIV evolution, encoded as an Ancestral Recombination Graph (ARG), and the observed phylogenetic tree. To compare our ARG results to the familiar phylogeny format, we calculate the expected bifurcating tree after decomposing the ARG into all unique site trees, their combined distance matrix, and the overall corresponding bifurcating tree. While latency and recombination separately disrupt the phylogenetic signal, remarkably, we find that recombination recovers the temporal signal of within-host HIV evolution caused by latency by mixing fragments of old, latent genomes into the contemporary population. In effect, recombination averages over extant heterogeneity, whether it stems from mixed time-signals or population bottlenecks. Further, we establish that the signals of latency and recombination can be observed in phylogenetic trees despite being an incorrect representation of the true evolutionary history. Using an Approximate Bayesian Computation method, we develop a set of statistical probes to tune our simulation model to nine longitudinally-sampled within-host HIV phylogenies. Because ARGs are exceedingly difficult to infer from real HIV data, our simulation system allows investigating effects of latency, recombination, and population size bottlenecks by matching decomposed ARGs to real data as observed in standard phylogenies.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2021

PNNL-CompBio/bayesian-inference

Bayesian Metabolic Inference is a platform to integrate genome-scale multi-omics data into an iterative method to obtain optimized recommendations for optimal target compound production. This method uses a variational inference approach to approximate new posterior data to analyze in order to obtain useful correlations and control coefficients.

Kumar, Neeraj↗

DOE OSTI2021

B-PINNs: Bayesian Physics-informal Neural Networks for Forward and Inverse PDE Problems with Noisy Data

We propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framework. Compared with PINNs, in addition to uncertainty quantification, B-PINNs obtain more accurate predictions in scenarios with large noise due to their capability of avoiding overfitting. We conduct a systematic comparison between the two different approaches for the B-PINNs posterior estimation (i.e., HMC or VI), along with dropout used for quantifying uncertainty in deep neural networks. Our experiments show that HMC is more suitable than VI with mean field Gaussian approximation for the B-PINNs posterior estimation, while dropout employed in PINNs can hardly provide accurate predictions with reasonable uncertainty. Finally, we replace the BNN in the prior with a truncated Karhunen-Loève (KL) expansion combined with HMC or a deep normalizing flow (DNF) model as posterior estimators. The KL is as accurate as BNN and much faster but this framework cannot be easily extended to high-dimensional problems unlike the BNN based framework.

Non linear PDEs, Noisy data, Bysian physics inform↗

NASA NTRS1991

Autoclass: An automatic classification system

The task of inferring a set of classes and class descriptions most likely to explain a given data set can be placed on a firm theoretical foundation using Bayesian statistics. Within this framework, and using various mathematical and algorithmic approximations, the AutoClass System searches for the most probable classifications, automatically choosing the number of classes and complexity of class descriptions. A simpler version of AutoClass has been applied to many large real data sets, has discovered new independently-verified phenomena, and has been released as a robust software package. Recent extensions allow attributes to be selectively correlated within particular classes, and allow classes to inherit, or share, model parameters through a class hierarchy. The mathematical foundations of AutoClass are summarized.

Stutz, John↗

NASA NTRS1991

Bayesian classification theory

The task of inferring a set of classes and class descriptions most likely to explain a given data set can be placed on a firm theoretical foundation using Bayesian statistics. Within this framework and using various mathematical and algorithmic approximations, the AutoClass system searches for the most probable classifications, automatically choosing the number of classes and complexity of class descriptions. A simpler version of AutoClass has been applied to many large real data sets, has discovered new independently-verified phenomena, and has been released as a robust software package. Recent extensions allow attributes to be selectively correlated within particular classes, and allow classes to inherit or share model parameters though a class hierarchy. We summarize the mathematical foundations of AutoClass.

Hanson, Robin↗

NASA NTRS2009

Understanding the Scalability of Bayesian Network Inference using Clique Tree Growth Curves

Bayesian networks (BNs) are used to represent and efficiently compute with multi-variate probability distributions in a wide range of disciplines. One of the main approaches to perform computation in BNs is clique tree clustering and propagation. In this approach, BN computation consists of propagation in a clique tree compiled from a Bayesian network. There is a lack of understanding of how clique tree computation time, and BN computation time in more general, depends on variations in BN size and structure. On the one hand, complexity results tell us that many interesting BN queries are NP-hard or worse to answer, and it is not hard to find application BNs where the clique tree approach in practice cannot be used. On the other hand, it is well-known that tree-structured BNs can be used to answer probabilistic queries in polynomial time. In this article, we develop an approach to characterizing clique tree growth as a function of parameters that can be computed in polynomial time from BNs, specifically: (i) the ratio of the number of a BN's non-root nodes to the number of root nodes, or (ii) the expected number of moral edges in their moral graphs. Our approach is based on combining analytical and experimental results. Analytically, we partition the set of cliques in a clique tree into different sets, and introduce a growth curve for each set. For the special case of bipartite BNs, we consequently have two growth curves, a mixed clique growth curve and a root clique growth curve. In experiments, we systematically increase the degree of the root nodes in bipartite Bayesian networks, and find that root clique growth is well-approximated by Gompertz growth curves. It is believed that this research improves the understanding of the scaling behavior of clique tree clustering, provides a foundation for benchmarking and developing improved BN inference and machine learning algorithms, and presents an aid for analytical trade-off studies of clique tree clustering using growth curves.

Mengshoel, Ole Jakob↗

DOE OSTI2023

An Assessment of the Laminar Hypersonic Double-Cone Experiments in the LENS-XX Tunnel

This is an investigation on two experimental datasets of laminar hypersonic flows, over a double-cone geometry, acquired in Calspan—University at Buffalo Research Center’s Large Energy National Shock (LENS)-XX expansion tunnel. These datasets have yet to be modeled accurately. A previous paper suggested that this could partly be due to mis-specified inlet conditions. The authors of this paper solved a Bayesian inverse problem to infer the inlet conditions of the LENS-XX test section and found that in one case they lay outside the uncertainty bounds specified in the experimental dataset. However, the inference was performed using approximate surrogate models. Here in this paper, the experimental datasets are revisited and inversions for the tunnel test-section inlet conditions are performed with a Navier–Stokes simulator. The inversion is deterministic and can provide uncertainty bounds on the inlet conditions under a Gaussian assumption. It was found that deterministic inversion yields inlet conditions that do not agree with what was stated in the experiments. An a posteriori method is also presented to check the validity of the Gaussian assumption for the posterior distribution. This paper contributes to ongoing work on the assessment of datasets from challenging experiments conducted in extreme environments, where the experimental apparatus is pushed to the margins of its design and performance envelopes.

42 ENGINEERING↗

DOE OSTI2021

Bayesian estimations of orientation distribution functions from small-angle scattering enable direct prediction of mechanical stress in anisotropic materials

Properties of soft materials are influenced by their anisotropic structuring under nonequilibrium fields. Although anisotropic structure-property relationships have been extensively explored theoretically, comparison to experiments requires determination of the microstructural orientation probability distribution function (OPDF) of microstructural elements. Small angle scattering (SAS) measurements encode information about the OPDF, but tools to navigate this connection are incomplete. Here, we develop and validate an explicit framework to link arbitrary OPDFs to SAS measurements. Specifically, we propose, validate, and apply a method, maximum a posteriori scattering inference (MAPSI), whereby the OPDF may be obtained from SAS measurements using a Bayesian estimation method. Using this method, we obtain estimates of the full 3D OPDF for two model semidilute fd-virus (rodlike) dispersions at concentrations that are approximately equal to and twice the overlap concentration. From the OPDF, we calculate its second and fourth moments and compare these to predictions for a dilute suspension of rigid rods and to a recent theory for semidilute suspensions. Finally, we use both the theoretical and measured moments to calculate the stress, both for dilute and semidilute suspensions. These predictions are not only compared to each other, but also to measured values of the shear stress, and point to new insights into the behavior of suspensions of highly elongated particles in the transition between dilute and semidilute behavior. We also use this new framework to provide perspective on the connection between scalar parameterizations of scattering and the OPDF that have frequently been used in the past. Here, the new tools developed in this work provide an unprecedented path toward experimental validation of dynamical theories of rodlike colloids and polymers, and for measurement of nonequilibrium structures and stresses of other complex fluids and soft materials with SAS.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Persistent Sampling: Enhancing the Efficiency of Sequential Monte Carlo

Sequential Monte Carlo (SMC) samplers are powerful tools for Bayesian inference but suffer from high computational costs due to their reliance on large particle ensembles for accurate estimates. We introduce persistent sampling (PS), an extension of SMC that systematically retains and reuses particles from all prior iterations to construct a growing, weighted ensemble. By leveraging multiple importance sampling and resampling from a mixture of historical distributions, PS mitigates the need for excessively large particle counts, directly addressing key limitations of SMC such as particle impoverishment and mode collapse. Crucially, PS achieves this without additional likelihood evaluations-weights for persistent particles are computed using cached likelihood values. This framework not only yields more accurate posterior approximations but also produces marginal likelihood estimates with significantly lower variance, enhancing reliability in model comparison. Furthermore, the persistent ensemble enables efficient adaptation of transition kernels by leveraging a larger, decorrelated particle pool. Experiments on high-dimensional Gaussian mixtures, hierarchical models, and non-convex targets demonstrate that PS consistently outperforms standard SMC and related variants, including recycled and waste-free SMC, achieving substantial reductions in mean squared error for posterior expectations and evidence estimates, all at reduced computational cost. PS thus establishes itself as a robust, scalable, and efficient alternative for complex Bayesian inference tasks.

Karamanis, Minas↗

DOE OSTI2014

Generic inference of inflation models by non-Gaussianity and primordial power spectrum reconstruction

We present a generic inference method for inflation models from observational data by the usage of higher-order statistics of the curvature perturbation on uniform density hypersurfaces. This method is based on the calculation of the posterior for the primordial non-Gaussianity parameters f{sub NL} and g{sub NL}, which in general depend on specific parameters of inflation and reheating models, and enables to discriminate among the still viable inflation models. To keep analyticity as far as possible to dispense with numerically expensive sampling techniques a saddle-point approximation is introduced, whose precision is validated for a numerical toy example. The mathematical formulation is done in a generic way so that the approach remains applicable to cosmic microwave background data as well as to large scale structure data. Additionally, we review a few currently interesting inflation models and present numerical toy examples thereof in two and three dimensions to demonstrate the efficiency of the higher-order statistics method. A second quantity of interest is the primordial power spectrum. Here, we present two Bayesian methods to infer it from observational data, the so called critical filter and an extension thereof with smoothness prior, both allowing for a non-parametric spectrum reconstruction. These methods are able to reconstruct the spectra of the observed perturbations and the primordial ones of curvature perturbation even in case of non-Gaussianity and partial sky coverage. We argue that observables like T- and B-modes permit to measure both spectra. This also allows to infer the level of non-Gaussianity generated since inflation.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2019

Rigorous Data Fusion for Computationally Expensive Simulations

This manuscript comprises the final report for the 1-year, FY19 LDRD project "Rigorous Data Fusion for Computationally Expensive Simulations," wherein an alternative approach to Bayesian calibration was developed based a new sampling technique called VoroSpokes. Vorospokes is a novel quadrature and sampling framework defined with respect to Voronoi tessellations of bounded domains in $R^d$ developed within this project. In this work, we first establish local quadrature and sampling results on convex polytopes using randomly directed rays, or spokes, to approximate the quantities of interest for a specified target function. A theoretical justification for both procedures is provided along with empirical results demonstrating the unbiased convergence in the resulting estimates/samples. The local quadrature and sampling procedures are then extended to global procedures defined on more general domains by applying the local results to the cells of a Voronoi tessellation covering the domain in consideration. We then demonstrate how the proposed global sampling procedure can be used to define a natural framework for adaptively constructing Voronoi Piecewise Surrogate (VPS) approximations based on local error estimates. Finally, we show that the adaptive VPS procedure can be used to form a surrogate model approximation to a specified, potentially unnormalized, density function, and that the global sampling procedure can be used to efficiently draw independent samples from the surrogate density in parallel. The performance of the resulting VoroSpokes sampling framework is assessed on a collection of Bayesian inference problems and is shown to provide highly accurate posterior predictions which align with the results obtained using traditional methods such as Gibbs sampling and random-walk Markov Chain Monte Carlo (MCMC). Importantly, the proposed framework provides a foundation for performing Bayesian inference tasks which is entirely independent from the theory of Markov chains.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Characterizing volumetric strain at Brady Hot Springs, Nevada, USA using geodetic data, numerical models and prior information

The geothermal field at Brady Hot Springs, Nevada has subsided over the past decade. Between 2004 and 2014, the rate of downward-vertical displacement was on the order of 10 mm yr -1 , as measured by two independent geodetic techniques: interferometric synthetic aperture radar (InSAR) and Global Positioning System. The observed deformation field forms an approximately elliptical bowl that is 4 km long and aligned with the trace of the NNE striking normal fault system. We use modelling to estimate the plausibility of pressure changes or thermal contraction as the cause of the observed subsidence. As a result, Bayesian inference favours with ‘very strong evidence’ thermal contraction over other hypotheses as the dominant driving mechanism for the observed subsidence. Using InSAR data spanning from 2016 July 22 to 2017 August 22, we estimate the volume change rate in the significantly deforming volume to be (-29 ± 3) thousand m 3 yr -1 and the total rate of change in thermal energy between -53 and -79 MW. We infer the total volume of cubes where the estimated volumetric strain rate is significantly different from zero with 95 per cent confidence to be 119 million m 3 . We find that the main region of significant cooling occurs between the injection and production well locations. This result supports the idea that highly permeable conduits along faults channel fluids from shallow aquifers to the deep reservoir tapped by the production wells.

15 GEOTHERMAL ENERGY↗

DOE OSTI2016

A critical reassessment of particle Dark Matter limits from dwarf satellites

Dwarf satellite galaxies are ideal laboratories for identifying particle Dark Matter signals. When setting limits on particle Dark Matter properties from null searches, it becomes however crucial the level at which the Dark Matter density profile within these systems is constrained by observations. In the limit in which the spherical Jeans equation is assumed to be valid for a given tracer stellar population, we study the solution of this equation having the Dark Matter mass profile as an output rather than as a trial parametric input. Within our new formulation, we address to what level dwarf spheroidal galaxies feature a reliable mass estimator. We assess then possible extrapolation of the density profiles in the inner regions and—keeping explicit the dependence on the orbital anisotropy profile of the tracer population—we derive general trends on the line-of-sight integral of the density profile squared, a quantity commonly dubbed J -factor and crucial to estimate fluxes from prompt Dark Matter pair annihilations. Taking Ursa Minor as a study case among Milky Way satellites, we perform Bayesian inference using the available kinematical data for this galaxy. Contrary to all previous studies, we avoid marginalization over quantities poorly constrained by observations or by theoretical arguments. We find minimal J -factors to be about 2 to 4 times smaller than commonly quoted estimates, approximately relaxing by the same amount the limit on Dark Matter pair annihilation cross section from gamma-ray surveys of Ursa Minor. At the same time, if one goes back to a fixed trial parametric form for the density, e.g. using a NFW or Burkert profile, we show that the minimal J can hardly be reduced by more than a factor of 1.5.

79 ASTRONOMY AND ASTROPHYSICS↗