Search NASA⌕ Search

SEARCH · Search NASA

Results for “variational distribution approximations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

Variational Dirichlet Blur Kernel Estimation

Blind image deconvolution involves two key objectives: 1) latent image and 2) blur estimation. For latent image estimation, we propose a fast deconvolution algorithm, which uses an image prior of nondimensional Gaussianity measure to enforce sparsity and an undetermined boundary condition methodology to reduce boundary artifacts. For blur estimation, a linear inverse problem with normalization and nonnegative constraints must be solved. However, the normalization constraint is ignored in many blind image deblurring methods, mainly because it makes the problem less tractable. In this paper, we show that the normalization constraint can be very naturally incorporated into the estimation process by using a Dirichlet distribution to approximate the posterior distribution of the blur. Making use of variational Dirichlet approximation, we provide a blur posterior approximation that considers the uncertainty of the estimate and removes noise in the estimated kernel. Furthermore, experiments with synthetic and real data demonstrate that the proposed method is very competitive to the state-of-the-art blind image restoration methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Bayesian and variational Bayesian approaches for flows in heterogeneous random media

In this paper, we study porous media flows in heterogeneous stochastic media. We propose an efficient forward simulation technique that is tailored for variational Bayesian inversion. As a starting point, the proposed forward simulation technique decomposes the solution into the sum of separable functions (with respect to randomness and the space), where each term is calculated based on a variational approach. This is similar to Proper Generalized Decomposition (PGD). Next, we apply a multiscale technique to solve for each term (as in ) and, further, decompose the random function into 1D fields. As a result, our proposed method provides an approximation hierarchy for the solution as we increase the number of terms in the expansion and, also, increase the spatial resolution of each term. We use the hierarchical solution distributions in a variational Bayesian approximation to perform uncertainty quantification in the inverse problem. We conduct a detailed numerical study to explore the performance of the proposed uncertainty quantification technique and show the theoretical posterior concentration.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations

Predictive modeling typically relies on Bayesian model calibration to provide uncertainty quantification. Variational inference utilizing fully independent (“mean-field”) Gaussian distributions are often used as approximate probability density functions. This simplification is attractive since the number of variational parameters grows only linearly with the number of unknown model parameters. However, the resulting diagonal covariance structure and unimodal behavior can be too restrictive to provide useful approximations of intractable Bayesian posteriors that exhibit highly non-Gaussian behavior, including multimodality. High-fidelity surrogate posteriors for these problems can be obtained by considering the family of Gaussian mixtures. Gaussian mixtures are capable of capturing multiple modes and approximating any distribution to an arbitrary degree of accuracy, while maintaining some analytical tractability. Unfortunately, variational inference using Gaussian mixtures with full-covariance structures suffers from a quadratic growth in variational parameters with the number of model parameters. The existence of multiple local minima due to strong nonconvex trends in the loss functions often associated with variational inference present additional complications, These challenges motivate the need for robust initialization procedures to improve the performance and computational scalability of variational inference with mixture models. In this work, we propose a method for constructing an initial Gaussian mixture model approximation that can be used to warm-start the iterative solvers for variational inference. The procedure begins with a global optimization stage in model parameter space. In this step, local gradient-based optimization, globalized through multistart, is used to determine a set of local maxima, which we take to approximate the mixture component centers. Around each mode, a local Gaussian approximation is constructed via the Laplace approximation. Finally, the mixture weights are determined through constrained least squares regression. The robustness and scalability of the proposed methodology is demonstrated through application to an ensemble of synthetic tests using high-dimensional, multimodal probability density functions. Here, the practical aspects of the approach are demonstrated with inversion problems in structural dynamics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗
DOE OSTI2016

Multivariate Predictive Analytics of Wind Power Data for Robust Control of Energy Storage

Short-term forecasting is frequently identified as an important tool for the effective management of wind generation. However, forecasting errors, inherent to the point forecasts, increase requirements for energy storage and can affect optimal system operation. Probabilistic forecasts can help tackle this issue by providing a proper characterization of forecasting errors in the optimization process. This paper proposes a multivariate model of forecasting data for wind generation. Predictive uncertainty intervals of wind power can be obtained by sampling from the proposed model. The main goal is to use empirical data models without linear or Gaussian approximations of the distributional or temporal variations. The predictive modeling is utilized within a case study of an energy storage system. Finally, a modified robust convex programming is used to maintain the practical robustness and feasibility of the solution based on the sampled scenarios from the model.

17 WIND ENERGY↗
DOE OSTI2024

Posterior Covariance Matrix Approximations

Here, the Davis equation of state (EOS) is commonly used to model thermodynamic relationships for high explosive (HE) reactants. Typically, the parameters in the EOS are calibrated, with uncertainty, using a Bayesian framework and Markov Chain Monte Carlo (MCMC) methods. However, MCMC methods are computationally expensive, especially for complex models with many parameters. This paper provides a comparison between MCMC and less computationally expensive Variational methods (Variational Bayesian and Hessian Variational Bayesian) for computing the posterior distribution and approximating the posterior covariance matrix based on heterogeneous experimental data. All three methods recover similar posterior distributions and posterior covariance matrices. This study demonstrates that for this EOS parameter calibration application, the assumptions made in the two Variational methods significantly reduce the computational cost but do not substantially change the results compared to MCMC.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

How Snow Aggregate Ellipsoid Shape and Orientation Variability Affects Fall Speed and Self-Aggregation Rates

Snow aggregate shapes and orientations have long been known to exhibit substantial variability. Despite this observed variability, most weather and climate prediction models use fixed power-law functions that deterministically map particle size to mass and fall speed. As such, integrated quantities like precipitation and self-aggregation rates currently ignore nonlinear effects resulting from variation in shape and orientation for aggregates of the same size. This study therefore develops an analytic framework that couples an empirically based bivariate distribution of ellipsoid shapes to classical hydrodynamic theory so as to capture an appropriate dispersion of masses, projected areas, and fall speeds for an assumed size distribution. For a fixed aggregate size, shape variations produce approximately ±0.13 m s -1 standard deviation of fall speed which increases the mass flux fall speed dispersion by more than 100% over traditional microphysics models. This increased fall speed dispersion results predominantly from shape-induced mass dispersion whereas orientation and drag dispersion play a lesser role. Shape variations can increase mass- and reflectivity-weighted fall speeds by up to 60% of traditional models whereas self-aggregation rates can increase by a factor of 100 for very small slope parameters. This implies that aggregate shape variations effectively forestall the theorized onset of fall speed distribution narrowing and subsequent quenching of the aggregation process. As a result, it is likely that secondary ice formation is necessary to prevent an ever decreasing slope parameter. The mathematical theory presented in this study is used to develop simple correction factors for snow forecast and climate models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Measurement and Analysis of Porosity in Al-10Si-1Mg Components Additively Manufactured by Selective Laser Melting

Aluminum alloys are candidate materials for weight critical applications because of their excellent strength and stiffness to weight ratio. However, defects such as voids decrease the strength and fatigue life of these alloys, which can limit the application of Selective Laser Melting. In this study, the average volume fraction, average size, and size distribution of pores in Al10-Si-1Mg samples built using Selective Laser Melting have been characterized. Synchrotron high energy X-rays were used to perform computed tomography on volumes of order one cubic millimeter with a resolution of approximately 1.5 μm. Substantial variations in the pore size distributions were found as a function of process conditions. Even under conditions that ensured that all locations were melted at least once, a significant number density was found of pores above 5 μm in diameter.

36 MATERIALS SCIENCE↗
DOE OSTI2015

Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations

We construct a new framework for accelerating Markov chain Monte Carlo in posterior sampling problems where standard methods are limited by the computational cost of the likelihood, or of numerical models embedded therein. Our approach introduces local approximations of these models into the Metropolis–Hastings kernel, borrowing ideas from deterministic approximation theory, optimization, and experimental design. Previous efforts at integrating approximate models into inference typically sacrifice either the sampler’s exactness or efficiency; our work seeks to address these limitations by exploiting useful convergence characteristics of local approximations. We prove the ergodicity of our approximate Markov chain, showing that it samples asymptotically from the exact posterior distribution of interest. We describe variations of the algorithm that employ either local polynomial approximations or local Gaussian process regressors. Our theoretical results reinforce the key observation underlying this article: when the likelihood has some local regularity, the number of model evaluations per Markov chain Monte Carlo (MCMC) step can be greatly reduced without biasing the Monte Carlo average. Numerical experiments demonstrate multiple order-of-magnitude reductions in the number of forward model evaluations used in representative ordinary differential equation (ODE) and partial differential equation (PDE) inference problems, with both synthetic and real data. Supplementary materials for this article are available online.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

36 MATERIALS SCIENCE↗
DOE OSTI2020

A Deep Generative Model for Non-Intrusive Identification of EV Charging Profiles

The proliferation of electric vehicles (EVs) brings environmental benefits and technical challenges to power grids. An identification algorithm which can accurately extract individual EV charging profiles out of widely available smart meter measurements has attracted great interests. This paper proposes a non-intrusive identification framework for EV charging profile extraction, which is driven by deep generative models (DGM). First, the proposed DGM is designed as a representation layer embedded into the Markov process and used to model the joint probability distribution of available time-series data. A novel contribution is to approximate posterior distributions by neural networks whose parameters are obtained by variational inference and supervised learning. Second, the EV charging status is inferred from the DGM via dynamic programming. Lastly, the desired EV charging profile can be reconstructed by the rated power of EV models and inferred status. Compared with the benchmark Hidden Markov Models, the proposed framework can better handle noise in data with less computational complexity and better overall accuracy performances with smaller recall. The proposed framework is validated by numerical experiments on the Pecan Street dataset.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Projective Integral Updates for High-Dimensional Variational Inference

Variational inference is an approximation framework for Bayesian inference that seeks to improve quantified uncertainty in predictions by optimizing a simplified distribution over parameters to stand in for the full posterior. Capturing model variations that remain consistent with training data enables more robust predictions by reducing parameter sensitivity. This work introduces a fixed-point optimization for variational inference that is applicable when every feasible log density can be expressed as a linear combination of functions from a given basis. In such cases, the optimizer becomes a fixed-point of projective integral updates. When the basis spans univariate quadratics in each parameter, the feasible distributions are Gaussian mean-fields and the projective integral updates yield quasi-Newton variational Bayes (QNVB). Other bases and updates are also possible. Since these updates require high-dimensional integration, this work begins by proposing an efficient quasirandom sequence of quadratures for mean-field distributions. Each iterate of the sequence contains two evaluation points that combine to correctly integrate all univariate quadratic functions and, if the mean-field factors are symmetric, all univariate cubics. More importantly, averaging results over short subsequences achieves periodic exactness on a much larger space of multivariate polynomials of quadratic total degree. The corresponding variational updates require four loss evaluations with standard (not second-order) backpropagation to eliminate error terms from over half of all multivariate quadratic basis functions. Furthermore, this integration technique is motivated by first proposing stochastic blocked mean-field quadratures, which may be useful in other contexts. A PyTorch implementation of QNVB allows for better control over model uncertainty during training than competing methods. Experiments demonstrate superior generalizability for multiple learning problems and architectures.

Gaussian mean-field↗
DOE OSTI2016

SU-G-BRC-16: Theory and Clinical Implications of the Constant Dosimetric Leaf Gap (DLG) Approximation

Purpose: Commercial dose calculation algorithms incorporate a single DLG value for a given beam energy that is applied across an entire treatment field. However, the physical processes associated with beam generation and dose delivery suggest that the DLG is not constant. The aim of this study is to evaluate the variation of DLG among all leaf pairs, to quantify how this variation impacts delivered dose, and to establish a novel method to correct dose distributions calculated using the approximation of constant DLG. Methods: A 2D diode array was used to measure the DLG for all 60 leaf pairs at several points along each leaf pair travel direction. This approach was validated by comparison to DLG values measured at select points using a 0.6 cc ion chamber with the standard formalism. In-house software was developed to enable incorporation of position dependent DLG values into dose distribution optimization and calculation. The accuracy of beam delivery of both the corrected and uncorrected treatment plans was studied through gamma pass rate evaluation. A comparison of DVH statistics in corrected and uncorrected treatment plans was made. Results: The outer 20 MLC leaf pairs (1.0 cm width) have DLG values that are 0.32 mm (mean) to 0.65 mm (maximum) lower than the central leaf-pair. VMAT plans using a large number of 1 cm wide leaves were more accurately delivered (gamma pass rate increased by 5%) and dose coverage was higher (D100 increased by 3%) when the 2D DLG was modeled. Conclusion: Using a constant DLG value for a given beam energy will result in dose optimization, dose calculation and treatment delivery inaccuracies that become significant for treatment plans with high modulation complexity scores delivered with 1 cm wide leaves.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

Understanding the optical properties of ambient sub- and supermicron particulate matter: results from the CARES 2010 field study in northern California

Abstract. Measurements of the optical properties (absorption, scattering and extinction) of PM1, PM2.5 and PM10 made at two sites around Sacramento, CA, during the June 2010 Carbonaceous Aerosols and Radiative Effects Study (CARES) are reported. These observations are used to establish relationships between various intensive optical properties and to derive information about the dependence of the optical properties on photochemical aging and sources. Supermicron particles contributed substantially to the total light scattering at both sites, about 50 % on average. A strong, linear relationship is observed between the scattering Ångström exponent for PM10 and the fraction of the scattering that is contributed by submicron particles (fsca, PM1) at both sites and with similar slopes and intercepts (for a given pair of wavelengths), suggesting that the derived relationship may be generally applicable for understanding variations in particle size distributions from remote sensing measurements. At the more urban T0 site, the fsca, PM1 increased with photochemical age, whereas at the downwind, more rural T1 site the fsca, PM1 decreased slightly with photochemical age. This difference in behavior reflects differences in transport, local production and local emission of supermicron particles between the sites. Light absorption is dominated by submicron particles, but there is some absorption by supermicron particles ( ∼ 15 % of the total). The supermicron absorption derives from a combination of black carbon that has penetrated into the supermicron mode and from dust, and there is a clear increase in the mass absorption coefficient of just the supermicron particles with increasing average particle size. The mass scattering coefficient (MSC) for the supermicron particles was directly observed to vary inversely with the average particle size, demonstrating that MSC cannot always be treated as a constant in estimating mass concentrations from scattering measurements, or vice versa. The total particle backscatter fraction exhibited some dependence upon the relative abundance of sub- versus supermicron particles; however this was modulated by variations in the median size of particles within a given size range; variations in the submicron size distribution had a particularly large influence on the observed backscatter efficiency and an approximate method to account for this variability is introduced. The relationship between the absorption and scattering Ångström exponents is examined and used to update a previously suggested particle classification scheme. Differences in composition led to differences in the sensitivity of PM2.5 to heating in a thermodenuder to the average particle size, with more extensive evaporation (observed as a larger decrease in the PM2.5 extinction coefficient) corresponding to smaller particles; i.e., submicron particles were generally more susceptible to heating than the supermicron particles. The influence of heating on the particle hygroscopicity varied with the effective particle size, with larger changes observed when the PM2.5 distribution was dominated by smaller particles.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

An Improved Energy Deposition Model in MPACT and Explicit Heat Generation Coupling with CTF

The default energy deposition model in the CASL neutronics code MPACT assumes all fission energy is deposited locally in fuel rods. Furthermore, equilibrium delayed energy release is assumed for both steady-state and transient calculations. These approximations limit the accurate representation of the heat generation distribution in space and its variations over time, which are essential for power distribution and thermal-hydraulic coupling calculations. In this paper, an improved energy deposition model is presented in both the spatial and time domains. Spatially, the energy deposition through fission, neutron capture, and slowing-down reactions are explicitly modeled to account for the heat generation from all regions of a reactor core, and a gamma smearing scheme is developed that utilizes the gamma sources from neutron fission and capture. In the time domain, the delayed energy release is modeled by solving an additional equation of delayed heat emitters, similar to the equation of delayed neutron precursors.To allow the explicit heat generation coupling, the interfaces between MPACT and CTF were updated to transfer separate heat sources for different material regions (fuel, clad, moderator and guide tube). The results show that the distributions of the energy deposition between MPACT and MCNP agree very well for various 2-D assembly and quarter-core problems without TH feedback. The MPACT/CTF coupled calculation for the hot full power quarter-core case exhibited a reduced peak pin power by 2.3% and a reduced peak fuel centerline temperature by 17 K when using the explicit energy deposition and heat transfer. The new model also shows a maximum 100 pcm keff effect on assembly depletion problems and an increased overall energy release by 7% in a PWR reactivity-initiated accident (RIA) problem.

Liu, Yuxuan↗
DOE OSTI2016

Deterministic Mean-Field Ensemble Kalman Filtering

The proof of convergence of the standard ensemble Kalman filter (EnKF) from Le Gland, Monbet, and Tran [Large sample asymptotics for the ensemble Kalman filter, in The Oxford Handbook of Nonlinear Filtering, Oxford University Press, Oxford, UK, 2011, pp. 598--631] is extended to non-Gaussian state-space models. In this paper, a density-based deterministic approximation of the mean-field limit EnKF (DMFEnKF) is proposed, consisting of a PDE solver and a quadrature rule. Given a certain minimal order of convergence κ between the two, this extends to the deterministic filter approximation, which is therefore asymptotically superior to standard EnKF for dimension d < 2κ. The fidelity of approximation of the true distribution is also established using an extension of the total variation metric to random measures. Lastly, this is limited by a Gaussian bias term arising from nonlinearity/non-Gaussianity of the model, which arises in both deterministic and standard EnKF. Numerical results support and extend the theory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Uncertainty Quantification via Stable Distribution Propagation

We propose a new approach for propagating stable probability distributions through neural networks. Our method is based on local linearization, which we show to be an optimal approximation in terms of total variation distance for the ReLU non-linearity. This allows propagating Gaussian and Cauchy input uncertainties through neural networks to quantify their output uncertainties. To demonstrate the utility of propagating distributions, we apply the proposed method to predicting calibrated confidence intervals and selective prediction on out-of-distribution data. The results demonstrate a broad applicability of propagating distributions and show the advantages of our method over other approaches such as moment matching.

Artificial Intelligence (cs.AI)↗
DOE OSTI2024

Distributionally Robust Variational Quantum Algorithms With Shifted Noise

Given their potential to demonstrate near-term quantum advantage, variational quantum algorithms (VQAs) have been extensively studied. Although numerous techniques have been developed for VQA parameter optimization, it remains a significant challenge. A practical issue is the high sensitivity of quantum noise to environmental changes, and its propensity to shift in real time. This presents a critical problem as an optimized VQA ansatz may not perform effectively under a different noise environment. For the first time, we explore how to optimize VQA parameters to be robust against unknown shifted noise. We model the noise level as a random variable with an unknown probability density function (PDF), and we assume that the PDF may shift within an uncertainty set. This assumption guides us to formulate a distributionally robust optimization problem, with the goal of finding parameters that maintain effectiveness under shifted noise. We utilize a distributionally robust Bayesian optimization solver for our proposed formulation. This provides numerical evidence in both the Quantum Approximate Optimization Algorithm (QAOA) and the Variational Quantum Eigensolver (VQE) with hardware-efficient ansatz, indicating that we can identify parameters that perform more robustly under shifted noise. We regard this work as the first step towards improving the reliability of VQAs influenced by real-time noise.

97 MATHEMATICS AND COMPUTING↗