Calibration of stochastic agent-based model with Gaussian process surrogate and Stein variational inference
Explore the source record for details and available documents.
Robertson, Connor James↗
SEARCH · Search NASA
Results for “Stein variational inference”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Stein variational inference and sparsification for uncertainty quantification of physical neural networks
Explore the source record for details and available documents.
Jones, Reese Edward [Sandia National Laboratories ↗
Uncertainty quantification of graph convolution neural network models of evolving processes
The application of neural network models to scientific machine learning tasks has proliferated in recent years. In particular, neural networks have proved to be adept at modeling processes with spatial–temporal complexity. Nevertheless, these highly parameterized models have garnered skepticism in their ability to produce outputs with quantified error bounds over the regimes of interest. Hence there is a need to find uncertainty quantification methods that are suitable for neural networks. In this work we present comparisons of the parametric uncertainty quantification of neural networks modeling complex spatial–temporal processes with Hamiltonian Monte Carlo and Stein variational gradient descent and its projected variant. Specifically we apply these methods to graph convolutional neural network models of evolving systems modeled with recurrent neural network and neural ordinary differential equations architectures. We show that Stein variational inference is a viable alternative to Monte Carlo methods with some clear advantages for complex neural network models. For our exemplars, Stein variational interference gave similar pushed forward uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit with generally more generous variance. As a result, projected Stein variational gradient descent also produced similar uncertainty profiles to the non-projected counterpart, but large reductions in the active weight space were confounded by the stability of the neural network predictions and the convoluted likelihood landscape.
36 MATERIALS SCIENCE↗
CONCURRENT, CONDENSED STEIN VARIATIONAL GRADIENT DESCENT FOR UNCERTAINTY QUANTIFICATION OF NEURAL NETWORKS
In this work, we propose a Stein variational gradient descent (SVGD) method to concurrently sparsify, train, and provide uncertainty quantification (UQ) of a complexly parameterized model, such as a neural network (NN). It employs a graph reconciliation and condensation process to reduce complexity and increase similarity in the Stein ensemble of parameterizations. Therefore, the proposed concurrent, condensed SVGD (ccSVGD) method can provide UQ on parameters, not just outputs. Furthermore, the parameter reduction speeds up the convergence of the Stein gradient descent as it reduces the combinatorial complexity by aligning and differentiating the sensitivity to parameters. These properties are demonstrated with an illustrative example and an application to a mechanical response representation problem in solid mechanics.
42 ENGINEERING↗
Bayesian inference of heterogeneous epidemic models: Application to COVID-19 spread accounting for long-term care facilities
Here we propose a high dimensional Bayesian inference framework for learning heterogeneous dynamics of a COVID-19 model, with a specific application to the dynamics and severity of COVID-19 inside and outside long-term care (LTC) facilities. We develop a heterogeneous compartmental model that accounts for the heterogeneity of the time-varying spread and severity of COVID-19 inside and outside LTC facilities, which is characterized by time-dependent stochastic processes and time-independent parameters in ~ 1500 dimensions after discretization. To infer these parameters, we use reported data on the number of confirmed, hospitalized, and deceased cases with suitable post-processing in both a deterministic inversion approach with appropriate regularization as a first step, followed by Bayesian inversion with proper prior distributions. To address the curse of dimensionality and the ill-posedness of the high-dimensional inference problem, we propose use of a dimension-independent projected Stein variational gradient descent method, and demonstrate the intrinsic low-dimensionality of the inverse problem. We present inference results with quantified uncertainties for both New Jersey and Texas, which experienced different epidemic phases and patterns. Moreover, we also present forecasting and validation results based on the empirical posterior samples of our inference for the future trajectory of COVID-19.
42 ENGINEERING↗
General-Purpose Bayesian Tensor Learning With Automatic Rank Determination and Uncertainty Quantification
A major challenge in many machine learning tasks is that the model expressive power depends on model size. Low-rank tensor methods are an efficient tool for handling the curse of dimensionality in many large-scale machine learning models. The major challenges in training a tensor learning model include how to process the high-volume data, how to determine the tensor rank automatically, and how to estimate the uncertainty of the results. While existing tensor learning focuses on a specific task, this paper proposes a generic Bayesian framework that can be employed to solve a broad class of tensor learning problems such as tensor completion, tensor regression, and tensorized neural networks. We develop a low-rank tensor prior for automatic rank determination in nonlinear problems. Our method is implemented with both stochastic gradient Hamiltonian Monte Carlo (SGHMC) and Stein Variational Gradient Descent (SVGD). We compare the automatic rank determination and uncertainty quantification of these two solvers. We demonstrate that our proposed method can determine the tensor rank automatically and can quantify the uncertainty of the obtained results. We validate our framework on tensor completion tasks and tensorized neural network training tasks.
Bayesian inference↗
Sr isotopic variations in Upper Proterozoic carbonates from Svalbard and East Greenland
Precambrian Sr isotope stratigraphy was investigated by determining variations in Sr-87/Sr-86 ratios in the Upper Proterozoic carbonate succession from Svalbard and East Greenland. Data from this study were combined with those from literature to construct a curve of Sr-87/Sr-86 versus time for Upper Proterozoic seawater. The curve for the Upper Riphean-Vandian showed that the isotopic composition of Sr in seawater was low (Delta Sr-87 of about -500) between 900 and 650 Ma but rose rapidly to about +30 by 600 Ma (this range of long-term variation exceeds the total Phanerozoic variation). The very low values of Delta Sr-87 inferred for the Riphean require that, for this time, the submarine hydrothermal water flux was a large fraction of the Sr input to the oceans, while the rise in Delta Sr-87 in the Upper Proterozoic seawater reflects both a change in the ratio of hydrothermal and continental fluxes of Sr to the oceans, and a change in the isotopic composition of Sr from continental sources.
Derry, Louis A.↗
Ca-isotopes as a robust tracer of magmatic differentiation
The large mass difference (~10%) between the two most abundant isotopes of calcium, 40 Ca and 44 Ca, gives Ca great potential in tracking mass-dependent fractionation during magmatic processes. Resolvable Ca-isotope fractionation during fractional crystallization of magma, particularly by feldspar in evolved melts, has been theoretically inferred but not robustly tested in nature. To further explore the effects of magmatic differentiation on Ca-isotope systematics, we studied the late-Permian alkaline igneous suite of the Øyangen Caldera, Oslo Rift, Norway, consisting of volcanic and intrusive units ranging from basanitic to rhyolitic compositions. Major and trace element variations and modeling demonstrate that the main series of samples (N = 21), including basanites, ring-dyke syenites, and central-dome syenites, likely documents a co-genetic and closed-system fractional crystallization sequence. Our data show minimal δ 44/40 Ca variation (< 0.05 ‰) in the intermediate magma and a marked increase in δ 44/40 Ca in the felsic magma of the Øyangen Caldera (from 0.62 ± 0.02 ‰ to 1.15 ± 0.03 ‰ relative to Ca standard, SRM915a). The systematic increase is best explained by equilibrium isotopic fractionation dominated by alkali feldspar in the fractionating mineral assemblage. This is further supported by strong correlations between δ 44/40 Ca, CaO, and Eu/Eu* in the main-series samples. Implementing a Monte Carlo approach, isotopic modeling of the liquid line of descent using Rayleigh fractionation is highly consistent with the observed Ca-isotope evolution. For the first time, we confirm prominent Ca stable isotope fractionation in felsic-stage differentiation of alkaline magma and constrain the isotope fractionation factors of plagioclase and K-feldspar. Integrated with extant estimations on mineral fractionation factors from the literature, our results suggest increasing fractionation effects of rock-forming minerals with decreasing Ca content. Finally, the affirmation of significant Ca-isotope fractionation in alkaline magma by feldspar empowers the application of Ca as a versatile tracer of crustal evolution, allowing further tests in other magmatic conditions across various planetary objects.
58 GEOSCIENCES↗
Use of Minute-by-Minute Cardiovascular Measurements During Tilt Tests to Strengthen Inference on the Effect of Long-Duration Space Flight on Orthostatic Hypotension
Typical methodology for evaluating the effects of spaceflight on orthostatic hypotension (OH) has been survival analysis of tolerance times from 80 head-up tilt tests. However when scheduled test durations are short, there may not be enough failures to allow survival analysis to adequately estimate and compare the effects of flight phase (e.g. pre-flight, number of days post-flight), flight duration, and their interaction, as well as interactions with effects of interventions or countermeasures. The problem is exacerbated in the presence of a repeated measures design, in which subjects participate in tilt tests during various flight phases. Here we show how it is possible to dramatically improve the efficiency of statistical inference in this setting by making use of the additional information contained in minute-by-minute observations of cardiovascular parameters thought to be reflective of progression towards presyncope during tilt testing. Methods: We retrospectively examined operational tilt test (OTT; 10 -min 80 head-up tilt) data from 20 International Space Station (ISS) and 66 Shuttle astronauts 10 d before launch (L-10), on landing day (R+0) and during recovery (R+1, R+3, R+6-10) depending on the level of participation. Data from 5 ISS astronauts tested on R+0 or R+1 who used non-standard countermeasures were excluded. In addition to OTT survival time, 8 cardiovascular parameters (CP: heart rate, systolic, diastolic, and mean arterial blood pressure, pulse pressure, stroke volume, cardiac output, and total peripheral resistance) that might be predictive of progression towards presyncope were measured every minute of each OTT. Statistical analysis was predicated on a two ]stage model of causation. In the first stage, flight duration and time from landing affect the astronauts' degree of OH, which is manifested in the time trends and variation of the above CPs during OTTs. In the second stage, the behavior of these parameters directly affects OTT survival time. Actual analysis proceeded in the opposite direction. First we identified those CPs or linear combinations that best predicted OTT survival regardless of what spaceflight conditions led to OTT completion or presyncope. From these, we calculated a summary statistic (one per OTT) that best predicted survival. We then used mixed ]model regression analysis to relate changes in the summary statistic to flight phase and duration. Inference on the effects of phase, duration, and their interaction on OH follows directly from this second analysis. Results: A linear combination (W) of diastolic blood pressure (DBP) and stroke volume (SV) was found to be the best predictor of OTT survival using the complete data set of minute-by-minute observations of CPs for each OTT. Furthermore, the log-transformed standard deviation of W (Z = log SW) was found to be a strong predictor of survival in the reduced data set consisting of one observation per OTT. In other words, this measure of variability of W during an OTT was the best indicator of whether or not the subject could complete the 10-min test, with higher variability (i.e. higher values of Z) being associated with greater probability of failure. In the mixed-model regression analysis where Z was now treated as a outcome with flight phase and duration groups (ISS and STS) as predictors, we found that there was a significantly more variability in W (higher values of Z) for both groups at R+0, but with no evidence of an interaction until R+3, when the ISS group still had inflated variability, but not the STS group. Conclusions: Variability of the cardiovascular index W recovers more slowly after long-compared to short-duration spaceflight. Since high variability of W has also been shown to be predictive of OTT failure, a primary manifestation of OH, a logical conclusion is that recovery from OH also is slower after long-duration compared to short-duration spaceflights.
Feiveson, Alan H.↗
Constraints on hydrothermal heat flux through the oceanic lithosphere from global heat flow
A significant discrepancy exists between the heat flow measured at the seafloor and the higher values predicted by thermal models of the cooling lithosphere. This discrepancy is generally interpreted as indicating that the upper oceanic crust is cooled significantly by hydrothermal circulation. The magnitude of this heat flow discrepancy is the primary datum used to estimate the volume of hydrothermal flow, and the variation in the discrepancy with lithospheric age is the primary constraint on how the hydrothermal flux is divided between near-ridge and off-ridge environments. The resulting estimates are important for investigation of both the thermal structure of the lithosphere and the chemistry of the oceans. We reevaluate the magnitude and age variation of the discrepancy using a global heat flow data set substantially larger than in earlier studies, and the GDHI (Global Depth and Heat Flow) model that better predicts the heat flow. We estimate that of the predicted global oceanic heat flux of 32 x 10(exp 12) W, 34% (11 x 10(exp 12) W) occurs by hydrothermal flow. Approximately 30% of the hydrothermal heat flux occurs in crust younger than 1 Ma, so the majority of this flux is off-ridge. These hydrothermal heat flux estimates are upper bounds, because heat flow measurements require sediment at the site and so are made preferentially at topographic lows, where heat flow may be depressed. Because the water temperature for the near-ridge flow exceeds that for the off-ridge flow, the near-ridge water flow will be even a smaller fraction of the total water flow. As a result, in estimating fluxes from geochemical data, use of the high water temperatures appropriate for the ridge axis may significantly overestimate the heat flux for an assumed water flux or underestimate the water flux for an assumed heat flux. Our data also permit improved estimates of the 'sealing' age, defined as the age where the observed heat flow approximately equals that predicted, suggesting that hydrothermal heat transfer has largely ceased. Although earlier studies suggested major differences in sealing ages for different ocean basins, we find that the sealing ages for the Atlantic, Pacific, and Indian oceans are similar and consistent with the sealing age for the entire data set, 65 +/- 10 Ma. The previous inference of a young (approximately 20 Ma) sealing age for the Pacific appears to have biased downward several previous estimates of the global hydrothermal flux. The heat flow data also provide indirect evidence for the mechanism by which the hydrothermal heat flux becomes small, which has often been ascribed to isolation of the igneous crust from seawater due to the hydraulic conductivity of the intervening sediment. We find, however, that even the least sedimented sites show the systematic increase of the ratio of observed to predicted heat flow with age, although the more sedimented sites have a younger sealing age. Moreover, the heat flow discrepancy persists at heavily sedimented sites until approximately 50 Ma. It thus appears that approximately 100-200 m of sediment is neither necessary nor sufficient to stop hydrothermal heat transfer. We therefore conclude that the age of the crust is the primary control on the fraction of heat transported by hydrothermal flow and that sediment thickness has a lesser effect. This inference is consistent with models in which hydrothermal flow decreases with age due to reduced crustal porosity and hence permeability.
Stein, Carol A.↗