Search NASA⌕ Search

SEARCH · Search NASA

Results for “approximate Bayesian inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Accelerating multilevel Markov Chain Monte Carlo using machine learning models

Here, this work presents an efficient approach for accelerating multilevel Markov Chain Monte Carlo (MCMC) sampling for large-scale problems using low-fidelity machine learning models. While conventional techniques for large-scale Bayesian inference often substitute computationally expensive high-fidelity models with machine learning models, thereby introducing approximation errors, our approach offers a computationally efficient alternative by augmenting high-fidelity models with low-fidelity ones within a hierarchical framework. The multilevel approach utilizes the low-fidelity machine learning model (MLM) for inexpensive evaluation of proposed samples thereby improving the acceptance of samples by the high-fidelity model. The hierarchy in our multilevel algorithm is derived from geometric multigrid hierarchy. We utilize an MLM to accelerate the coarse level sampling. Training machine learning model for the coarsest level significantly reduces the computational cost associated with generating training data and training the model. We present an MCMC algorithm to accelerate the coarsest level sampling using MLM and account for the approximation error introduced. We provide theoretical proofs of detailed balance and demonstrate that our multilevel approach constitutes a consistent MCMC algorithm. Additionally, we derive the expression for cost reduction due to machine learning model to facilitate cost analysis of the hierarchical sampling algorithm. Our technique is demonstrated on a standard benchmark inference problem in groundwater flow, where we estimate the probability density of a quantity of interest using a four-level MCMC algorithm. Our proposed algorithm accelerates multilevel sampling by a factor of two while achieving similar accuracy compared to sampling using the standard multilevel algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗
DOE OSTI2021

Overview of Algorithms for Using Particle Morphology in Pre-Detonation Nuclear Forensics

A major goal in pre-detonation nuclear forensics is to infer the processing conditions and/or facility type that produced radiological material. This review paper focuses on analyses of particle size, shape, texture (“morphology”) signatures that could provide information on the provenance of interdicted materials. For example, uranium ore concentrates (UOC or yellowcake) include ammonium diuranate (ADU), ammonium uranyl carbonate (AUC), sodium diuranate (SDU), magnesium diuranate (MDU), and others, each prepared using different salts to precipitate U from solution. Once precipitated, UOCs are often dried and calcined to remove adsorbed water. The products can be allowed to react further, forming uranium oxides UO3, U3O8, or UO2 powders, whose surface morphology can be indicative of precipitation and/or calcination conditions used in their production. This review paper describes statistical issues and approaches in using quantitative analyses of measurements such as particle size and shape to infer production conditions. Statistical topics include multivariate t tests (Hotelling’s T 2 ), design of experiments, and several machine learning (ML) options including decision trees, learning vector quantization neural networks, mixture discriminant analysis, and approximate Bayesian computation (ABC). ABC is emphasized as an attractive option to include the effects of model uncertainty in the selected and fitted forward model used for inferring processing conditions.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2024

Prime VI

SAND2025-03757O Prime VI is a distribution-of-disease outbreak model calibration code based on variational inference. It accompanies a publication for submission to Statistics in Medicine journal, and the code will be maintained for open-source use on Sandia's GitLab. The software provides methods for calibrating an epidemiological model to measured case-count data for a multitude of correlated spatial regions. The code solves a Bayesian inverse problem for model calibration where the posterior over-model parameters are approximated through a custom implementation of variational inference. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Safta, Cosmin↗
DOE OSTI2015

OBJECTIVE BAYESIAN ANALYSIS OF ''ON/OFF'' MEASUREMENTS

In high-energy astrophysics, it is common practice to account for the background overlaid with counts from the source of interest with the help of auxiliary measurements carried out by pointing off-source. In this ''on/off'' measurement, one knows the number of photons detected while pointing toward the source, the number of photons collected while pointing away from the source, and how to estimate the background counts in the source region from the flux observed in the auxiliary measurements. For very faint sources, the number of photons detected is so low that the approximations that hold asymptotically are not valid. On the other hand, an analytical solution exists for the Bayesian statistical inference, which is valid at low and high counts. Here we illustrate the objective Bayesian solution based on the reference posterior and compare the result with the approach very recently proposed by Knoetig, and discuss its most delicate points. In addition, we propose to compute the significance of the excess with respect to the background-only expectation with a method that is able to account for any uncertainty on the background and is valid for any photon count. This method is compared to the widely used significance formula by Li and Ma, which is based on asymptotic properties.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Parsimonious Inference Information-Theoretic Foundations for a Complete Theory of Machine Learning (CIS-LDRD Project 218313 Final Technical Report)

This work examines how we may cast machine learning within a complete Bayesian framework to quantify and suppress explanatory complexity from first principles. Our investigation into both the philosophy and mathematics of rational belief leads us to emphasize the critical role of Bayesian inference in learning well-justified predictions within a rigorous and complete extended logic. The Bayesian framework allows us to coherently account for evidence in the learned plausibility of potential explanations. As an extended logic, the Bayesian paradigm regards probability as a notion of degrees of truth. In order to satisfy critical properties of probability as a coherent measure, as well as maintain consistency with binary propositional logic, we arrive at Bayes' Theorem as the only justifiable mechanism to update our beliefs to account for empiracle evidence. Yet, in the machine learning paradigm, where explanations are unconstrained algorithmic abstractions, we arrive at a critical challenge: Bayesian inference requires prior belief. Conventional approaches fail to yield a consistent framework in which we could compare prior plausibility among the infinities of potential choices in learning architectures. The difficulty of articulating well-justified prior belief over abstract models is the provinence of memorization in traditional machine learning training practices. This becomes exceptionally problematic in the context of limited datasets, when we wish to learn justifiable predictions from only a small amount of data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Inference of reaction rate parameters based on summary statistics from experiments

Here, we present the results of an application of Bayesian inference and maximum entropy methods for the estimation of the joint probability density for the Arrhenius rate para meters of the rate coefficient of the H 2 /O 2 -mechanism chain branching reaction H + O 2 → OH + O. Available published data is in the form of summary statistics in terms of nominal values and error bars of the rate coefficient of this reaction at a number of temperature values obtained from shock-tube experiments. Our approach relies on generating data, in this case OH concentration profiles, consistent with the given summary statistics, using Approximate Bayesian Computation methods and a Markov Chain Monte Carlo procedure. The approach permits the forward propagation of parametric uncertainty through the computational model in a manner that is consistent with the published statistics. A consensus joint posterior on the parameters is obtained by pooling the posterior parameter densities given each consistent data set. To expedite this process, we construct efficient surrogates for the OH concentration using a combination of Pad'e and polynomial approximants. These surrogate models adequately represent forward model observables and their dependence on input parameters and are computationally efficient to allow their use in the Bayesian inference procedure. We also utilize Gauss-Hermite quadrature with Gaussian proposal probability density functions for moment computation resulting in orders of magnitude speedup in data likelihood evaluation. Despite the strong non-linearity in the model, the consistent data sets all res ult in nearly Gaussian conditional parameter probability density functions. The technique also accounts for nuisance parameters in the form of Arrhenius parameters of other rate coefficients with prescribed uncertainty. The resulting pooled parameter probability density function is propagated through stoichiometric hydrogen-air auto-ignition computations to illustrate the need to account for correlation among the Arrhenius rate parameters of one reaction and across rate parameters of different reactions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Scalable Algorithms for Inverse Problems With High-Dimensional Parameter Spaces

Inverse problems, which involve inferring unknown parameters from observed data, present significant computational challenges, especially in large-scale settings with high-dimensional unknown parameters and nonlinear relationships between the unknowns and observations. Bayesian inference provides an approach for addressing these problems, often relying on sequential sampling methods like Markov chain Monte Carlo (MCMC) to approximate the posterior distribution of the parameters. However, MCMC methods become computationally demanding as the dimensionality of the problem increases, particularly in large-scale systems where likelihood evaluations rely on solving partial differential equations (PDEs) on large spatial domains with finely resolved meshes. To overcome these limitations, recent advancements have focused on designing scalable computa tional techniques – for both PDE simulations and sampling strategies – to make Bayesian methods feasible for high-dimensional problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Capabilities of multivariate Bayesian inference toward seismic hazard assessment

Multivariate Bayesian analysis can bring significant benefits to seismic hazard analysis: Its multivariate feature enables computing scalar and vector hazard without making any approximations; Correlations between intensity measures are implicitly modeled, permitting direct simulation of ground motion selection tools such as the conditional mean spectrum and the generalized conditioning intensity measure; and Its updating feature enables a seamless integration of new ground motion data into the hazard results. Here, we first develop a multivariate Bayesian ground motion model through the NGA-West2 database. The model functional form considers fault-type, magnitude, and distance dependencies, and also the linear and the rock intensity dependent site response. We use a hybrid Markov Chain Monte Carlo sampling to perform Bayesian inference consisting of Gibbs step and a multilevel Metropolis-Hastings step. We then perform several checks on the model and note that its performance is satisfactory. Finally, we illustrate the merits of this multivariate Bayesian analysis, which include: ground motion model updating with ground motion data recorded in the last four years not part of the NGA-West2 database; computation of scalar and vector seismic hazard using the un-updated and updated ground motion models for Los Angeles, CA; and simulation of the conditional mean spectrum under scalar and vector IM conditioning while accounting for different sources of aleatoric and epistemic uncertainties.

58 GEOSCIENCES↗
DOE OSTI2025

Sequential spectral line analysis for accurate density and temperature diagnosis of laboratory opacity measurements

The accuracy of iron opacity calculated in stellar interiors has been questioned since the discovery of the “solar problem” and the discrepancies between the measured and modeled iron opacity reported in 2015. Experimental opacity benchmarks require accurate temperature and density measurements, which were inferred by analyzing tracer magnesium spectra in those experiments. Could the observed discrepancy be explained by insufficient accuracy in the inferred temperature, density, and their uncertainties? Previous analyses may have yielded biased results due to three limitations: (1) simultaneous multi-line fitting, (2) approximations in line-shape models, and (3) exclusion of certain spectral lines due to insufficient background characterization. Notably, the first issue is a common concern for many inversion methods, including Bayesian inferences. We present a refined analysis method that overcomes these limitations, applied to three categories of iron opacity experiments (Anchor 1, 2, and 3). In particular, the sequential fitting method yields unbiased results with more realistic uncertainties by accounting for line inconsistencies in the parameter uncertainties. The average electron temperature and density values are 162 ± 6 eV and (7.0 ± 1.9) × 10 21 cm −3 for six Anchor 1 experiments, 189 ± 7 eV and (3.4 ± 0.3) × 10 22 cm −3 for 21 Anchor 2 experiments, and 201 ± 6 eV and (4.8 ± 1.1) × 10 22 cm −3 for nine Anchor 3 experiments. These results show ∼4% temperature and ∼20% density reproducibility over a decade, which also aligns with the inferred parameter uncertainties. In conclusion, the resulting temperature and density uncertainties lead to a quasi-continuum iron opacity variation of ±4%–7% for wavelengths below 9.5 Å, which is insufficient to explain the significant model-data discrepancies reported in 2015.

Absorption spectroscopy↗
DOE OSTI2015

Scalable and efficient algorithms for the propagation of uncertainty from data through inference to prediction for large-scale problems, with application to flow of the Antarctic ice sheet

The majority of research on efficient and scalable algorithms in computational science and engineering has focused on the forward problem : given parameter inputs, solve the governing equations to determine output quantities of interest. In contrast, in this paper, we consider the broader question: given a (large-scale) model containing uncertain parameters, (possibly) noisy observational data, and a prediction quantity of interest, how do we construct efficient and scalable algorithms to (1) infer the model parameters from the data (the deterministic inverse problem ), (2) quantify the uncertainty in the inferred parameters (the Bayesian inference problem), and (3) propagate the resulting uncertain parameters through the model to issue predictions with quantified uncertainties (the forward uncertainty propagation problem )? We present efficient and scalable algorithms for this end-to-end, data-to-prediction process under the Gaussian approximation and in the context of modeling the flow of the Antarctic ice sheet and its effect on loss of grounded ice to the ocean. The ice is modeled as a viscous, incompressible, creeping, shear-thinning fluid. The observational data come from satellite measurements of surface ice flow velocity, and the uncertain parameter field to be inferred is the basal sliding parameter, represented by a heterogeneous coefficient in a Robin boundary condition at the base of the ice sheet. The prediction quantity of interest is the present-day ice mass flux from the Antarctic continent to the ocean. We show that the work required for executing this data-to-prediction process—measured in number of forward (and adjoint) ice sheet model solves—is independent of the state dimension, parameter dimension, data dimension, and the number of processor cores. The key to achieving this dimension independence is to exploit the fact that, despite their large size, the observational data typically provide only sparse information on model parameters. This property can be exploited to construct a low rank approximation of the linearized parameter-to-observable map via randomized SVD methods and adjoint-based actions of Hessians of the data misfit functional.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Probabilistic Mixture Model-Based Spectral Unmixing

Spectral unmixing attempts to decompose a spectral ensemble into the constituent pure spectral signatures (called endmembers) along with the proportion of each endmember. This is essential for techniques like hyperspectral imaging (HSI) used in environment monitoring, geological exploration, etc. Several spectral unmixing approaches have been proposed, many of which are connected to hyperspectral imaging. However, most extant approaches assume highly diverse collections of mixtures and extremely low-loss spectroscopic measurements. Additionally, current non-Bayesian frameworks do not incorporate the uncertainty inherent in unmixing. We propose a probabilistic inference algorithm that explicitly incorporates noise and uncertainty, enabling us to unmix endmembers in collections of mixtures with limited diversity. We use a Bayesian mixture model to jointly extract endmember spectra and mixing parameters while explicitly modeling observation noise and the resulting inference uncertainties. We obtain approximate distributions over endmember coordinates for each set of observed spectra while remaining robust to inference biases from the lack of pure observations and the presence of non-isotropic Gaussian noise. As a direct impact of our methodology, access to reliable uncertainties on the unmixing solutions would enable robust solutions to noise, as well as informed decision-making for HSI applications and other unmixing problems.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Hamiltonian parameter inference from resonant inelastic x-ray scattering with active learning

Identifying model Hamiltonians is a vital step toward creating predictive models of materials. Here, in this study, we combine Bayesian optimization with the EDRIXS numerical package to infer Hamiltonian parameters from resonant inelastic x-ray scattering (RIXS) spectra within the single atom approximation. To evaluate the efficacy of our method, we test it on experimental RIXS spectra of NiPS 3 , NiCl 2 , Ca 3 ⁢LiOsO 6 , and Fe 2⁢ O 3 , and demonstrate that it can reproduce results obtained from hand-fitted parameters to a precision similar to expert human analysis while providing a more systematic mapping of parameter space. Our work provides a key first step toward solving the inverse scattering problem to extract effective multi-orbital models from information-dense RIXS measurements, which can be applied to a host of quantum materials. We also propose atomic model parameter sets for two materials, Ca 3⁢ LiOsO 6 and Fe 2⁢ O 3 , that were previously missing from the literature.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2024

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Neural network ensembles and uncertainty estimation for predictions of inelastic mechanical deformation using a finite element method-neural network approach

The finite element method (FEM) is widely used to simulate a variety of physics phenomena. Approaches that integrate FEM with neural networks (NNs) are typically leveraged as an alternative to conducting expensive FEM simulations in order to reduce the computational cost without significantly sacrificing accuracy. However, these methods can produce biased predictions that deviate from those obtained with FEM, since these hybrid FEM-NN approaches rely on approximations trained using physically relevant quantities. In this work, an uncertainty estimation framework is introduced that leverages ensembles of Bayesian neural networks to produce diverse sets of predictions using a hybrid FEM-NN approach that approximates internal forces on a deforming solid body. The uncertainty estimator developed herein reliably infers upper bounds of bias/variance in the predictions for a wide range of interpolation and extrapolation cases using a three-element FEM-NN model of a bar undergoing plastic deformation. This proposed framework offers a powerful tool for assessing the reliability of physics-based surrogate models by establishing uncertainty estimates for predictions spanning a wide range of possible load cases.

42 ENGINEERING↗
DOE OSTI2016

Development of a Severe Accident Analysis Engine Using Approximate Reasoning

In hypothetical severe accident scenarios, nuclear power plant safety is challenged by core degradation and release of fission products. To aid the plant operator and technical support center staff, research has been performed to develop understanding of severe accident phenomenology, assessing their risk and devise measures to mitigate accident consequences (Sehgal, 2011). Nuclear power plant accidents involve highly complex physical and chemical interactions, a large number of safety related systems, and emergency operating procedures (EOP) and severe accident management guidelines (SAMG). Large number of factors with uncertainty and stress can overwhelm the operators. The overall objective of the present research is to develop a methodology and tools that an operator can use to predict severe accident progression and make effective decisions to arrest and manage the accident under large uncertainties. Towards this goal, this study examines a novel approach that brings to bear a combined use of techniques in artificial intelligence (Russel and Norvig, 2010), including Bayesian Network (BN), Answer Set Programming (ASP) and Fuzzy Inference (FI) to achieve faster-than-real-time, yet sufficiently accurate analysis of severe accident progression. Bayesian network (BN) is a type of probabilistic graphical models that can be used to represent knowledge and conditional dependencies about an uncertain realm. It is useful for capturing the causal relationships in a complex system. It also allows prediction via Bayesian inference (Kelly and Smith, 2012). It is composed of nodes and edges, which express random variables and probabilistic relationships between the nodes, respectively. This relationship, given by conditional probability is calculated using stochastic methods. The BN is used in several systems for analysis and management of accident risk; e.g. (Knochenhauer and Frid, 2010). In those systems, the nodes or so-called 'effective factors' are presumably known within the network by system descriptions in SAMGs. There are two steps to configure the BN, namely (i) identifying compositions within the network and (ii) defining conditional dependency between nodes. The first step is performed following the spirit of ASP (Baral, 2003) using Prolog (Wielemaker, 2005). Knowledge representation is used to build databases required for processing by ASP. Knowledge representation is defined as the expression of knowledge in a way that is understandable to both human and computers. Originally knowledge indicates data, information and facts. To focus, knowledge should be formed to achieve a goal. In a severe accident, key physics and safety related components status are defined according to a failure mode. The conditional dependencies between nodes are generated by approximate reasoning using Fuzzy inference (Zadeh, 1995). Fuzzy logic is a multiple-valued logic to treat ambiguous features such as linguistic description. It indicates possibility, which is distinguished from probability. Fuzzy set consists of linguistic variables according to the fixed rule. In other words, even uncertain states could be expressed as not binary logic, but as many-value logic by rule-based technology. Fuzzy control system is comprised of Fuzzy inference engine that is performed based on the 'If-Then Rule'. The 'If-Then Rule' is of the following form. If Y1 is B11 and Y2 is B12 then Z is C1. If Y1 is B21 and Y2 is B22 then Z is C2. Y1 and Y2 are inputs, which indicate conditional variables. Z indicates decision-making and both B and C are linguistic variables. With this rule-based technique, the approximate reasoning can be realized by Fuzzy inference. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2015

Adaptive Bayesian Inference for Prediction

Performing rigorous predictive simulation necessitates that all available data be incorporated into the modeling process in order to accurately quantify uncertainties in predictions of the model. Statistical Bayesian inference is the most commonly used approach for incorporating data into uncertainty estimates. Statistical Bayesian inference uses data and an assumed error model to inform posterior distributions of model input variables and model discrepancies. An explicit characterization of the posterior distribution is not formed, but rather methods such as Markov chain Monte-Carlo (MCMC) are used to draw samples from the posterior. These samples are then used to evaluate statistics, such as integrals, of predicted quantities of interest (QoI). Recently the authors have developed an alternative reformulation of Bayesian inference based upon a measure-theoretic inversion technique. Unlike traditional statistical Bayesian inference, this reformulation allows us to directly interrogate the posterior distribution and, consequently, to generate samples from the posterior without MCMC. In this work we use forward propagation uncertainty quantification techniques to reduce the cost of approximating the posterior distribution. Specifically we investigate goal-oriented adaptive sparse grid approximation methods. These adaptive methods concentrate high-fidelity model evaluations in regions of the parameter space that both significantly contribute to the uncertainty in specified QoI and are informed by the available data. Numerical results are presented to compare our approach with the standard statistical Bayesian approach and to demonstrate the effectiveness of the goal-oriented adaptivity.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗