Calibration of stochastic agent-based model with Gaussian process surrogate and Stein variational inference
Explore the source record for details and available documents.
Robertson, Connor James↗
SEARCH · Search NASA
Results for “Stein variational inference”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Stein variational inference and sparsification for uncertainty quantification of physical neural networks
Explore the source record for details and available documents.
Jones, Reese Edward [Sandia National Laboratories ↗
Uncertainty quantification of graph convolution neural network models of evolving processes
The application of neural network models to scientific machine learning tasks has proliferated in recent years. In particular, neural networks have proved to be adept at modeling processes with spatial–temporal complexity. Nevertheless, these highly parameterized models have garnered skepticism in their ability to produce outputs with quantified error bounds over the regimes of interest. Hence there is a need to find uncertainty quantification methods that are suitable for neural networks. In this work we present comparisons of the parametric uncertainty quantification of neural networks modeling complex spatial–temporal processes with Hamiltonian Monte Carlo and Stein variational gradient descent and its projected variant. Specifically we apply these methods to graph convolutional neural network models of evolving systems modeled with recurrent neural network and neural ordinary differential equations architectures. We show that Stein variational inference is a viable alternative to Monte Carlo methods with some clear advantages for complex neural network models. For our exemplars, Stein variational interference gave similar pushed forward uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit with generally more generous variance. As a result, projected Stein variational gradient descent also produced similar uncertainty profiles to the non-projected counterpart, but large reductions in the active weight space were confounded by the stability of the neural network predictions and the convoluted likelihood landscape.
36 MATERIALS SCIENCE↗
CONCURRENT, CONDENSED STEIN VARIATIONAL GRADIENT DESCENT FOR UNCERTAINTY QUANTIFICATION OF NEURAL NETWORKS
In this work, we propose a Stein variational gradient descent (SVGD) method to concurrently sparsify, train, and provide uncertainty quantification (UQ) of a complexly parameterized model, such as a neural network (NN). It employs a graph reconciliation and condensation process to reduce complexity and increase similarity in the Stein ensemble of parameterizations. Therefore, the proposed concurrent, condensed SVGD (ccSVGD) method can provide UQ on parameters, not just outputs. Furthermore, the parameter reduction speeds up the convergence of the Stein gradient descent as it reduces the combinatorial complexity by aligning and differentiating the sensitivity to parameters. These properties are demonstrated with an illustrative example and an application to a mechanical response representation problem in solid mechanics.
42 ENGINEERING↗
Bayesian inference of heterogeneous epidemic models: Application to COVID-19 spread accounting for long-term care facilities
Here we propose a high dimensional Bayesian inference framework for learning heterogeneous dynamics of a COVID-19 model, with a specific application to the dynamics and severity of COVID-19 inside and outside long-term care (LTC) facilities. We develop a heterogeneous compartmental model that accounts for the heterogeneity of the time-varying spread and severity of COVID-19 inside and outside LTC facilities, which is characterized by time-dependent stochastic processes and time-independent parameters in ~ 1500 dimensions after discretization. To infer these parameters, we use reported data on the number of confirmed, hospitalized, and deceased cases with suitable post-processing in both a deterministic inversion approach with appropriate regularization as a first step, followed by Bayesian inversion with proper prior distributions. To address the curse of dimensionality and the ill-posedness of the high-dimensional inference problem, we propose use of a dimension-independent projected Stein variational gradient descent method, and demonstrate the intrinsic low-dimensionality of the inverse problem. We present inference results with quantified uncertainties for both New Jersey and Texas, which experienced different epidemic phases and patterns. Moreover, we also present forecasting and validation results based on the empirical posterior samples of our inference for the future trajectory of COVID-19.
42 ENGINEERING↗
General-Purpose Bayesian Tensor Learning With Automatic Rank Determination and Uncertainty Quantification
A major challenge in many machine learning tasks is that the model expressive power depends on model size. Low-rank tensor methods are an efficient tool for handling the curse of dimensionality in many large-scale machine learning models. The major challenges in training a tensor learning model include how to process the high-volume data, how to determine the tensor rank automatically, and how to estimate the uncertainty of the results. While existing tensor learning focuses on a specific task, this paper proposes a generic Bayesian framework that can be employed to solve a broad class of tensor learning problems such as tensor completion, tensor regression, and tensorized neural networks. We develop a low-rank tensor prior for automatic rank determination in nonlinear problems. Our method is implemented with both stochastic gradient Hamiltonian Monte Carlo (SGHMC) and Stein Variational Gradient Descent (SVGD). We compare the automatic rank determination and uncertainty quantification of these two solvers. We demonstrate that our proposed method can determine the tensor rank automatically and can quantify the uncertainty of the obtained results. We validate our framework on tensor completion tasks and tensorized neural network training tasks.
Bayesian inference↗
Ca-isotopes as a robust tracer of magmatic differentiation
The large mass difference (~10%) between the two most abundant isotopes of calcium, 40 Ca and 44 Ca, gives Ca great potential in tracking mass-dependent fractionation during magmatic processes. Resolvable Ca-isotope fractionation during fractional crystallization of magma, particularly by feldspar in evolved melts, has been theoretically inferred but not robustly tested in nature. To further explore the effects of magmatic differentiation on Ca-isotope systematics, we studied the late-Permian alkaline igneous suite of the Øyangen Caldera, Oslo Rift, Norway, consisting of volcanic and intrusive units ranging from basanitic to rhyolitic compositions. Major and trace element variations and modeling demonstrate that the main series of samples (N = 21), including basanites, ring-dyke syenites, and central-dome syenites, likely documents a co-genetic and closed-system fractional crystallization sequence. Our data show minimal δ 44/40 Ca variation (< 0.05 ‰) in the intermediate magma and a marked increase in δ 44/40 Ca in the felsic magma of the Øyangen Caldera (from 0.62 ± 0.02 ‰ to 1.15 ± 0.03 ‰ relative to Ca standard, SRM915a). The systematic increase is best explained by equilibrium isotopic fractionation dominated by alkali feldspar in the fractionating mineral assemblage. This is further supported by strong correlations between δ 44/40 Ca, CaO, and Eu/Eu* in the main-series samples. Implementing a Monte Carlo approach, isotopic modeling of the liquid line of descent using Rayleigh fractionation is highly consistent with the observed Ca-isotope evolution. For the first time, we confirm prominent Ca stable isotope fractionation in felsic-stage differentiation of alkaline magma and constrain the isotope fractionation factors of plagioclase and K-feldspar. Integrated with extant estimations on mineral fractionation factors from the literature, our results suggest increasing fractionation effects of rock-forming minerals with decreasing Ca content. Finally, the affirmation of significant Ca-isotope fractionation in alkaline magma by feldspar empowers the application of Ca as a versatile tracer of crustal evolution, allowing further tests in other magmatic conditions across various planetary objects.
58 GEOSCIENCES↗