Search NASA⌕ Search

SEARCH · Search NASA

Results for “Bayesian methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Active learning of reactive Bayesian force fields applied to heterogeneous catalysis dynamics of H/Pt

Abstract Atomistic modeling of chemically reactive systems has so far relied on either expensive ab initio methods or bond-order force fields requiring arduous parametrization. Here, we describe a Bayesian active learning framework for autonomous “on-the-fly” training of fast and accurate reactive many-body force fields during molecular dynamics simulations. At each time-step, predictive uncertainties of a sparse Gaussian process are evaluated to automatically determine whether additional ab initio training data are needed. We introduce a general method for mapping trained kernel models onto equivalent polynomial models whose prediction cost is much lower and independent of the training set size. As a demonstration, we perform direct two-phase simulations of heterogeneous H 2 turnover on the Pt(111) catalyst surface at chemical accuracy. The model trains itself in three days and performs at twice the speed of a ReaxFF model, while maintaining much higher fidelity to DFT and excellent agreement with experiment.

42 ENGINEERING↗
DOE OSTI2025

Tracing the impacts of Mount Pinatubo eruption on regional climate using spatially-varying changepoint detection

Significant events, such as volcanic eruptions, can have global and long-lasting impacts on climate. These global impacts, however, are not uniform across space and time. Understanding how the Mt. Pinatubo eruption affects global and regional climate is of great interest for predicting the impact on climate due to similar events as well as understanding the possible effect of the stratospheric aerosol injections proposed to combat climate change. While many studies illustrated the impact of the Pinatubo eruption on a global scale, studies at a fine regional scale are scarce. Here, we propose a Bayesian spatially-varying changepoint detection and estimation method to trace the impact of Mt. Pinatubo eruption on regional climate. Our approach takes into account the diffusing nature and spatial correlation of the climate changes attributed to the volcanic eruption. We illustrate our method and demonstrate its advantages over an existing changepoint detection method through simulations. Finally, we apply our method to monthly stratospheric aerosol optical depth and surface temperature data from 1985 to 1995 to detect and estimate changepoints following the 1991 Mt. Pinatubo eruption. Our results quantitatively characterize the spatial pattern of the eruption’s impact on regional climate, complementing the previous studies on the global impact of the Pinatubo eruption.

Aerosol optical depth↗
DOE OSTI2022

Physics makes the difference: Bayesian optimization and active learning via augmented Gaussian process

Abstract Both experimental and computational methods for the exploration of structure, functionality, and properties of materials often necessitate the search across broad parameter spaces to discover optimal experimental conditions and regions of interest in the image space or parameter space of computational models. The direct grid search of the parameter space tends to be extremely time-consuming, leading to the development of strategies balancing exploration of unknown parameter spaces and exploitation towards required performance metrics. However, classical Bayesian optimization (BO) strategies based on the Gaussian process (GP) do not readily allow for the incorporation of the known physical behaviors or past knowledge. Here we explore a hybrid optimization/exploration algorithm created by augmenting the standard GP with a structured probabilistic model of the expected system’s behavior. This approach balances the flexibility of the non-parametric GP approach with a rigid structure of physical knowledge encoded into the parametric model. The fully Bayesian treatment of the latter allows additional control over the optimization via the selection of priors for the model parameters. The method is demonstrated for a noisy version of a standard univariate test function used to evaluate optimization algorithms and further extended to physical lattice models. This methodology is expected to be universally suitable for injecting prior knowledge in the form of physical models and past data in the BO framework.

42 ENGINEERING↗
DOE OSTI2020

Extended experimental inferential structure determination method in determining the structural ensembles of disordered protein states

Proteins with intrinsic or unfolded state disorder comprise a new frontier in structural biology, requiring the characterization of diverse and dynamic structural ensembles. Here we introduce a comprehensive Bayesian framework, the Extended Experimental Inferential Structure Determination (X-EISD) method, which calculates the maximum log-likelihood of a disordered protein ensemble. X-EISD accounts for the uncertainties of a range of experimental data and back-calculation models from structures, including NMR chemical shifts, J-couplings, Nuclear Overhauser Effects (NOEs), paramagnetic relaxation enhancements (PREs), residual dipolar couplings (RDCs), hydrodynamic radii (R h ), single molecule fluorescence Förster resonance energy transfer (smFRET) and small angle X-ray scattering (SAXS). We apply X-EISD to the joint optimization against experimental data for the unfolded drkN SH3 domain and find that combining a local data type, such as chemical shifts or J-couplings, paired with long-ranged restraints such as NOEs, PREs or smFRET, yields structural ensembles in good agreement with all other data types if combined with representative IDP conformers.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data

We present a machine learning framework (GP-NODE) for Bayesian model discovery from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. The use of the Finnish Horseshoe as a sparsity-promoting prior for free model parameters also enables the discovery of parsimonious representations for the latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator–prey systems, systems biology and a 50-dimensional human motion dynamical system. This article is part of the theme issue ‘Data-driven prediction in dynamical systems’.

Science & Technology - Other Topics↗
DOE OSTI2024

Forward variable selection enables fast and accurate dynamic system identification with Karhunen-Loève decomposed Gaussian processes

A promising approach for scalable Gaussian processes (GPs) is the Karhunen-Loève (KL) decomposition, in which the GP kernel is represented by a set of basis functions which are the eigenfunctions of the kernel operator. Such decomposed kernels have the potential to be very fast, and do not depend on the selection of a reduced set of inducing points. However KL decompositions lead to high dimensionality, and variable selection thus becomes paramount. This paper reports a new method of forward variable selection, enabled by the ordered nature of the basis functions in the KL expansion of the Bayesian Smoothing Spline ANOVA kernel (BSS-ANOVA), coupled with fast Gibbs sampling in a fully Bayesian approach. It quickly and effectively limits the number of terms, yielding a method with competitive accuracies, training and inference times for tabular datasets of low feature set dimensionality. Theoretical computational complexities are O ( N P 2 ) in training and O ( P ) per point in inference, where N is the number of instances and P the number of expansion terms. The inference speed and accuracy makes the method especially useful for dynamic systems identification, by modeling the dynamics in the tangent space as a static problem, then integrating the learned dynamics using a high-order scheme. The methods are demonstrated on two dynamic datasets: a ‘Susceptible, Infected, Recovered’ (SIR) toy problem, along with the experimental ‘Cascaded Tanks’ benchmark dataset. Comparisons on the static prediction of time derivatives are made with a random forest (RF), a residual neural network (ResNet), and the Orthogonal Additive Kernel (OAK) inducing points scalable GP, while for the timeseries prediction comparisons are made with LSTM and GRU recurrent neural networks (RNNs) along with the SINDy package.

Hayes, Kyle↗
DOE OSTI2026

Designing the Protocols for Programmable Ammonia Catalysis

Programmable catalysis can provide a more energy-efficient and cost-effective route to enhancing commercial ammonia production, a key process in the advancement of renewable energy technologies and the manufacture of fertilizers and basic chemicals. This work explores the computational discovery of optimal forcing protocols to drive such dynamic catalysis models. By employing matrix-free time-stepper methods, coupled with an optimization approach, that integrates Bayesian optimization with a Bayesian continuation strategy to efficiently discover the periodic steady states of such periodically forced systems, we enable the discovery of complex optimal catalyst strain waveforms, while ensuring robust solver convergence. We demonstrate the flexibility of our approach to discover optimized forcing protocols under varying physical constraints on strain modulation or other catalyst operating parameters. We show that these can have a temporal structure more complex than simple step functions. In order to detect undesirable catalytic loops that may correlate with overall reduced performance, we perform a study using graph-theoretical analysis to investigate the dynamics of catalytic kinetic networks formed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Bayesian event categorization matrix approach for explosion monitoring

Current efforts to correctly categorize natural events from suspected explosion sources with data that is collected by ground- or space-based sensors presents historical challenges that remain unaddressed by the Event Categorization Matrix (ECM) model. Smaller historical events (lower yield explosions) may have data available from fewer measurement techniques than are available today, and therefore, a historical event record can lack a complete set of discriminants. The covariance structures can also differ between such observations of event (source-type) categories. Both obstacles are problematic for the classic ECM model. Our work addresses this gap and presents a Bayesian update to the previous ECM model, termed the Bayesian Event Categorization Matrix model, which can be trained on partial observations and does not rely on a pooled covariance structure. We further augment the ECM model with Bayesian Decision Theory so that false negative or false positive rates of an event categorization can be reduced in an intuitive manner. To demonstrate improved categorization rates for the Bayesian Event Categorization Matrix model, we compare an array of Bayesian and classic models with multiple performance metrics using Monte Carlo experiments. We use both synthetic and real data. Our Bayesian models show consistent gains in overall accuracy and lower false negative rates relative to the classic ECM model. Here, we propose future avenues to improve Bayesian Event Categorization Matrix models’ decision making and predictive capability.

58 GEOSCIENCES↗
DOE OSTI2025

pnnl/MCRASTA

McRasta (Markov Chain Rate and State Analysis) was developed to estimate parameter uncertainty in constitutive friction models via Bayesian inverse and Markov Chain Monte Carlo (MCMC) methods.

Fichera, Marissa [Pacific Northwest National Labor↗
DOE OSTI2021

Machine Learning Assisted Safety Modeling and Analysis of Advanced Reactors

With the advances in computational power and numerical methods, analysts can now rely on first-principle simulations to predict ultra-fine details in a variety of applications. Advances in machine learning (ML) have produced algorithms that can now learn high-level abstractions via hierarchical models. This project aims to leverage advances in ML techniques and the available high-resolution simulation data to develop a novel modeling and simulation (M\&S) methodology for reactor safety analysis. While application-agnostic ML techniques are available, complex physics constraints need to be incorporated into ML techniques to build ML-based closures for computationally efficient predictive simulations. This project intends to develop a physics-guided data-driven multi-scale methodology for M\&S of advanced reactors. The project focuses on thermal fluid (T/F) phenomena, which play major roles in advanced reactor safety. Specifically, we propose a data-driven coarse-mesh turbulence model based on local flow features for the transient analysis of thermal mixing and stratification in a sodium-cooled fast reactor (SFR). The model has a coarse-mesh setup to ensure computational efficiency, while it is trained by fine-mesh computational fluid dynamics (CFD) data with Reynolds-averaged Navier-Stokes (RANS) turbulence model to ensure accuracy. Three different neural networks are developed and tested for loss-of-flow transients in the hot pool of SFR, i.e. the densely connected convolutional neural network (DCNN), long-short-term-memory network based on proper orthogonal decomposition (POD-LSTM), and the DCNN informed by LSTM (DCNN-LSTM). The performances of these three neural networks are evaluated based on baseline models. The DCNN-LSTM model has been chosen for further hyperparameter optimization. Furthermore, based on a simplified two-dimensional case, uncertainty quantification (UQ) of the developed ML-based closure are investigated with three methods, i.e. Monte Carlo dropout, deep ensemble, and Bayesian neural network. The developed ML-based turbulent viscosity closure relation based on deep ensemble is then integrated into the system analysis module SAM and serves as a term in the conservation equations. Such a SAM-ML based procedure guarantees that the obtained results are consistent with the physical constraints of the thermal-fluid system. The SAM-ML simulation on the same loss-of-flow transient showed comparable accuracy with the CFD simulation but with a much coarser mesh setup. Last but not least, the ML-based closure improvement with the support of higher-fidelity data from large eddy simulation (LES) is discussed. As a first step towards this direction, a baseline LES simulation is performed to obtain comparable data with RANS results. Based on the early results, future investigation on further improving the ML-based closure is discussed. We believe the developed approach that combines scientific machine learning with nuclear system analysis code can benefit the advanced reactor community as more accurate safety analyses will better characterize reactor safety margins and reduce licensing efforts.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

How to Obtain the Redshift Distribution from Probabilistic Redshift Estimates

Abstract A reliable estimate of the redshift distribution n ( z ) is crucial for using weak gravitational lensing and large-scale structures of galaxy catalogs to study cosmology. Spectroscopic redshifts for the dim and numerous galaxies of next-generation weak-lensing surveys are expected to be unavailable, making photometric redshift (photo- z ) probability density functions (PDFs) the next best alternative for comprehensively encapsulating the nontrivial systematics affecting photo- z point estimation. The established stacked estimator of n ( z ) avoids reducing photo- z PDFs to point estimates but yields a systematically biased estimate of n ( z ) that worsens with a decreasing signal-to-noise ratio, the very regime where photo- z PDFs are most necessary. We introduce Cosmological Hierarchical Inference with Probabilistic Photometric Redshifts ( CHIPPR ), a statistically rigorous probabilistic graphical model of redshift-dependent photometry that correctly propagates the redshift uncertainty information beyond the best-fit estimator of n ( z ) produced by traditional procedures and is provably the only self-consistent way to recover n ( z ) from photo- z PDFs. We present the chippr prototype code, noting that the mathematically justifiable approach incurs computational cost. The CHIPPR approach is applicable to any one-point statistic of any random variable, provided the prior probability density used to produce the posteriors is explicitly known; if the prior is implicit, as may be the case for popular photo- z techniques, then the resulting posterior PDFs cannot be used for scientific inference. We therefore recommend that the photo- z community focus on developing methodologies that enable the recovery of photo- z likelihoods with support over all redshifts, either directly or via a known prior probability density.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

IceCube Search for Neutrinos Coincident with Gravitational Wave Events from LIGO/Virgo Run O3

Using data from the IceCube Neutrino Observatory, we searched for high-energy neutrino emission from the gravitational-wave events detected by the advanced LIGO and Virgo detectors during their third observing run. We did a low-latency follow-up on the public candidate events released during the detectors' third observing run and an archival search on the 80 confident events reported in the GWTC-2.1 and GWTC-3 catalogs. An extended search was also conducted for neutrino emission on longer timescales from neutron star containing mergers. Follow-up searches on the candidate optical counterpart of GW190521 were also conducted. We used two methods; an unbinned maximum likelihood analysis and a Bayesian analysis using astrophysical priors, both of which were previously used to search for high-energy neutrino emission from gravitational-wave events. No significant neutrino emission was observed by any analysis, and upper limits were placed on the time-integrated neutrino flux as well as the total isotropic equivalent energy emitted in high-energy neutrinos.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Bayesian filtering for model predictive control of stochastic gene expression in single cells

This study describes a method for controlling the production of protein in individual cells using stochastic models of gene expression. By combining modern microscopy platforms with optogenetic gene expression, experimentalists are able to accurately apply light to individual cells, which can induce protein production. Here we use a finite state projection based stochastic model of gene expression, along with Bayesian state estimation to control protein copy numbers within individual cells. We compare this method to previous methods that use population based approaches. We also demonstrate the ability of this control strategy to ameliorate discrepancies between the predictions of a deterministic model and stochastic switching system.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Probabilistic Nanomagnetic Memories for Uncertain and Robust Machine Learning

This project evaluated the use of emerging spintronic memory devices for robust and efficient variational inference schemes. Variational inference (VI) schemes, which constrain the distribution for each weight to be a Gaussian distribution with a mean and standard deviation, are a tractable method for calculating posterior distributions of weights in a Bayesian neural network such that this neural network can also be trained using the powerful backpropagation algorithm. Our project focuses on domain-wall magnetic tunnel junctions (DW-MTJs), a powerful multi-functional spintronic synapse design that can achieve low power switching while also opening the pathway towards repeatable, analog operation using fabricated notches. Our initial efforts to employ DW-MTJs as an all-in-one stochastic synapse with both a mean and standard deviation didn’t end up meeting the quality metrics for hardware-friendly VI. In the future, new device stacks and methods for expressive anisotropy modification may make this idea still possible. However, as a fall back that immediately satisfies our requirements, we invented and detailed how the combination of a DW-MTJ synapse encoding the mean and a probabilistic Bayes-MTJ device, programmed via a ferroelectric or ionically modifiable layer, can robustly and expressively implement VI. This design includes a physics-informed small circuit model, that was scaled up to perform and demonstrate rigorous uncertainty quantification applications, up to and including small convolutional networks on a grayscale image classification task, and larger (Residual) networks implementing multi-channel image classification. Lastly, as these results and ideas all depend upon the idea of an inference application where weights (spintronic memory states) remain non-volatile, the retention of these synapses for the notched case was further interrogated. These investigations revealed and emphasized the importance of both notch geometry and anisotropy modification in order to further enhance the endurance of written spintronic states. In the near future, these results will be mapped to effective predictions for room temperature and elevated operation DW-MTJ memory retention, and experimentally verified when devices become available.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2020

Bayesian differential programming for robust systems identification under uncertainty

This paper presents a machine learning framework for Bayesian systems identification from noisy, sparse and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling. This allows an efficient inference of the posterior distributions over plausible models with quantified uncertainty, while the use of sparsity-promoting priors enables the discovery of interpretable and parsimonious representations for the underlying latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed methods, including nonlinear oscillators, predator–prey systems and examples from systems biology. Taken together, our findings put forth a flexible and robust workflow for data-driven model discovery under uncertainty. All codes and data accompanying this article are available at https://bit.ly/34FOJMj .

Science & Technology - Other Topics↗
DOE OSTI2021

Sparse Bayesian mass mapping with uncertainties: hypothesis testing of structure

ABSTRACT A crucial aspect of mass mapping, via weak lensing, is quantification of the uncertainty introduced during the reconstruction process. Properly accounting for these errors has been largely ignored to date. We present a new method to reconstruct maximum a posteriori (MAP) convergence maps by formulating an unconstrained Bayesian inference problem with Laplace-type l1-norm sparsity-promoting priors, which we solve via convex optimization. Approaching mass mapping in this manner allows us to exploit recent developments in probability concentration theory to infer theoretically conservative uncertainties for our MAP reconstructions, without relying on assumptions of Gaussianity. For the first time, these methods allow us to perform hypothesis testing of structure, from which it is possible to distinguish between physical objects and artefacts of the reconstruction. Here, we present this new formalism, and demonstrate the method on simulations, before applying the developed formalism to two observational data sets of the Abell 520 cluster. Initial reconstructions of the Abell 520 catalogues reported the detection of an anomalous ‘dark core’ – an overdense region with no optical counterpart – which was taken to be evidence for self-interacting dark matter. In our Bayesian framework, it is found that neither Abell 520 data set can conclusively determine the physicality of such dark cores at $99{{\ \rm per\ cent}}$ confidence. However, in both cases the recovered MAP estimators are consistent with both sets of data.

Price, M. A.↗
DOE OSTI2024

Seismic moment tensor classification using elliptical distribution functions on the hypersphere

Discrimination of underground explosions from naturally occurring earthquakes and other anthropogenic sources is one of the fundamental challenges of nuclear explosion monitoring. In an operational setting, the number of events that can be thoroughly investigated by analysts is limited by available resources. The capability to rapidly screen out events that can be robustly identified as not being explosions is, therefore, of great potential benefit. Nevertheless, possible mis-classification of explosions as earthquakes currently limits the use of screening methods for verification of test-ban treaties. Moment tensors provide a physics-based classification tool for the characterization of different seismic sources and have enabled the advent of new techniques for discriminating between earthquakes and explosions. Following normalization and projection of their six-degree vectors onto the hypersphere, existing screening approaches use spherically symmetric metrics to determine whether any new moment tensor may have been an explosion. Here, we show that populations of moment tensors for both earthquakes and explosions are anisotropically distributed on the hypersphere. Distributions possessing elliptical symmetry, such as the scaled von Mises–Fisher distribution, therefore provide a better description of these populations than the existing spherically symmetric models. We describe a method that uses these elliptical distributions in combination with a Bayesian classifier to achieve successful classification rates of 99 per cent for explosions and 98 per cent for earthquakes using existing catalogues of events from the western United States. The 1983 May 5 Crowdie underground nuclear test and 2018 July 20 DAG-1 deep-borehole chemical explosion are the only two explosions out of 140 that are incorrectly classified. Application of the method to the 2006–2017 nuclear tests in the Democratic People’s Republic of Korea yields 100 per cent identification rates and we provide a simple routine MTid for general usage. The approach provides a means to rapidly assess the likelihood of an event being an explosion and can be built into monitoring workflows that rely on simultaneously assessing multiple different discrimination metrics.

58 GEOSCIENCES↗