Search NASA⌕ Search

SEARCH · Search NASA

Results for “Gaussian Process (GP)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

117 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Towards Efficient Uncertainty estimation in deep learning for robust energy prediction in crystal materials

In material science, recent studies have started to explore the potential of using deep learning to improve property prediction from high-fidelity simulations, e.g, density functional theory (DFT). However, the design spaces are sometimes too large and intractable to sample completely. This results in a critical question that is how to evaluate the confidence and robustness of the prediction. In this paper, we propose an efficient approach to estimate uncertainty in deep learning using a single forward pass and then apply it for robust prediction of the total energy in crystal lattice structures. Our approach is built upon the deep kernel learning (DKL) that originally introduces to leverage the expressiveness of deep neural networks as input with a probabilistic prediction of Gaussian processes (GPs) as output. Existing DKL methods have difficulties in the accuracy of predictive uncertainty, training stability, and scaling to large datasets, which lead to significant barriers in real-world applications. We propose to address these challenges by using an inducing point approximate GP in feature space combined with spectral normalization as a regularization. We finally demonstrate our robust performance on an artificial example and a real-world application from materials chemistry.

Bi, Sirui↗
DOE OSTI2024

Exploration with Scalable Gaussian Process Reinforcement Learning

Exploration is a challenging problem in reinforcement learning (RL), especially in environments with sparse rewards. Quantifying and utilizing the parametric uncertainty has been shown to be paramount for successful exploration [Osband et al., 2018]. Bayesian, or approximately Bayesian, methods present a principled means of estimating the parametric uncertainty in RL problems. Gaussian processes, nonparametric Bayesian models, are often impractical due to poor scalability and computational bottlenecks. We introduce a scalable Gaussian process RL (GPRL) method which directly induces sparsity in the covariance matrix to facilitate faster computation. This is a departure from previous GPRL methods which instead rely on data reduction and subsampling. We compare various covariance-based exploration techniques (Thompson sampling, upper confidence bound, and probabilistic maximum variance) which leverage our scalable GP framework in sparse reward environments. Finally, we show favorable comparison against the bootstrapped deep Q-Network.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Sparse Cholesky factorization for solving nonlinear PDEs via Gaussian processes

In recent years, there has been widespread adoption of machine learning-based approaches to automate the solving of partial differential equations (PDEs). Among these approaches, Gaussian processes (GPs) and kernel methods have garnered considerable interest due to their flexibility, robust theoretical guarantees, and close ties to traditional methods. They can transform the solving of general nonlinear PDEs into solving quadratic optimization problems with nonlinear, PDE-induced constraints. However, the complexity bottleneck lies in computing with dense kernel matrices obtained from pointwise evaluations of the covariance kernel, and its partial derivatives, a result of the PDE constraint and for which fast algorithms are scarce. The primary goal of this paper is to provide a near-linear complexity algorithm for working with such kernel matrices. We present a sparse Cholesky factorization algorithm for these matrices based on the near-sparsity of the Cholesky factor under a novel ordering of pointwise and derivative measurements. The near-sparsity is rigorously justified by directly connecting the factor to GP regression and exponential decay of basis functions in numerical homogenization. We then employ the Vecchia approximation of GPs, which is optimal in the Kullback-Leibler divergence, to compute the approximate factor. This enables us to compute ϵ-approximate inverse Cholesky factors of the kernel matrices with complexity O(N log d (N/ϵ)) in space and O(N log 2d (N/ϵ)) in time. We integrate sparse Cholesky factorizations into optimization algorithms to obtain fast solvers of the nonlinear PDE. We numerically illustrate our algorithm’s near-linear space/time complexity for a broad class of nonlinear PDEs such as the nonlinear elliptic, Burgers, and Monge-Ampère equations. In summary, we provide a fast, scalable, and accurate method for solving general PDEs with GPs and kernel methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Stochastic machine learning via sigma profiles to build a digital chemical space

This work establishes a different paradigm on digital molecular spaces and their efficient navigation by exploiting sigma profiles. To do so, the remarkable capability of Gaussian processes (GPs), a type of stochastic machine learning model, to correlate and predict physicochemical properties from sigma profiles is demonstrated, outperforming state-of-the-art neural networks previously published. The amount of chemical information encoded in sigma profiles eases the learning burden of machine learning models, permitting the training of GPs on small datasets which, due to their negligible computational cost and ease of implementation, are ideal models to be combined with optimization tools such as gradient search or Bayesian optimization (BO). Gradient search is used to efficiently navigate the sigma profile digital space, quickly converging to local extrema of target physicochemical properties. While this requires the availability of pretrained GP models on existing datasets, such limitations are eliminated with the implementation of BO, which can find global extrema with a limited number of iterations. A remarkable example of this is that of BO toward boiling temperature optimization. Holding no knowledge of chemistry except for the sigma profile and boiling temperature of carbon monoxide (the worst possible initial guess), BO finds the global maximum of the available boiling temperature dataset (over 1,000 molecules encompassing more than 40 families of organic and inorganic compounds) in just 15 iterations (i.e., 15 property measurements), cementing sigma profiles as a powerful digital chemical space for molecular optimization and discovery, particularly when little to no experimental data is initially available.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Fast Emulation of Expensive Simulations using Approximate Gaussian Processes [Slides]

Nuclear Computational Low-Energy Initiative (NUCLEI) collaboration uses Density Functional Theory (DFT) simulations to predict the structure and binding energies of nuclei over a wide range of proton (Z) and neutron (N) numbers. The DFT simulations utilize a particular parameterization of a Skyrme energy density functional called UNEDF1 which depends on 12 free parameters that must be fit to data (M Kortelainen et al 2014). Fitting involves comparing (e.g.) predicted binding energies of nuclei to experimentally measured values. We use only binding energies as observables, but DFT with UNEDF1 will predict structure (shape) observables as well. In this work, assessing the capability of approximate GP emulators to balance emulator accuracy with computational speed to facilitate improved UNEDF1 calibration. Sparse GPs are straightforward to train and accurate. Calibration is not straightforward with MCMC (using MH or HMC/NUTS). We produced reusable software for continuing and building on this work as well as accessing and using Darwin cluster compute resources

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes

Uncertainty propagation in complex engineering systems often poses significant computational challenges related to modeling and quantifying probability distributions of model outputs, as those emerge as the result of various sources of uncertainty that are inherent in the system under investigation. Gaussian Processes regression (GPs) is a robust meta-modeling technique that allows for fast model prediction and exploration of response surfaces. Multi-fidelity variations of GPs further leverage information from cheap and low fidelity model simulations in order to improve their predictive performance on the high fidelity model. In order to cope with the high volume of data required to train GPs in high dimensional design spaces, a common practice is to introduce latent design variables that are typically projections of the original input space to a lower dimensional subspace, and therefore substitute the problem of learning the initial high dimensional mapping, with that of training a GP on a low dimensional space. Here in this paper, we present a Bayesian approach to identify optimal transformations that map the input points to low dimensional latent variables. The \projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters, conditioned on the available training data. The proposed Bayesian inference scheme relies on a two-step iterative algorithm that samples from the marginal posteriors of the GP parameters and the projection matrix respectively, both using Markov Chain Monte Carlo (MCMC) sampling. In order to take into account the orthogonality constraints imposed on the orthonormal projection matrix, a Geodesic Monte Carlo sampling algorithm is employed, that is suitable for exploiting probability measures on manifolds. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. We validate our framework on three synthetic problems with a known lower-dimensional subspace. The benefits of our proposed framework, are illustrated on the computationally challenging aerodynamic optimization of a last-stage blade for an industrial gas turbine, where we study the effect of an 85-dimensional shape parameterization of a three-dimensional airfoil on two output quantities of interest, specifically on the aerodynamic efficiency and the degree of reaction

42 ENGINEERING↗
DOE OSTI2023

Validating the Use of Gaussian Process Regression for Adaptive Mapping of Residual Stress Fields

Probing the stress state using a high density of measurement points is time intensive and presents a limitation for what is experimentally feasible. Alternatively, individual strain fields used for determining stresses can be reconstructed from a subset of points using a Gaussian process regression (GPR). Results presented in this paper evidence that determining stresses from reconstructed strain fields is a viable approach for reducing the number of measurements needed to fully sample a component’s stress state. The approach was demonstrated by reconstructing the stress fields in wire-arc additively manufactured walls fabricated using either a mild steel or low-temperature transition feedstock. Effects of errors in individual GP reconstructed strain maps and how these errors propagate to the final stress maps were assessed. Implications of the initial sampling approach and how localized strains affect convergence are explored to give guidance on how best to implement a dynamic sampling experiment.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Microscopic constraints for the equation of state and structure of neutron stars: A Bayesian model mixing framework

Bayesian model mixing (BMM) is a statistical technique that can combine constraints from different regions of an input space in a principled way. Here we extend our BMM framework for the equation of state (EOS) of strongly interacting matter from symmetric nuclear matter to asymmetric matter, specifically focusing on zero-temperature, charge-neutral, 𝛽-equilibrated matter. We use Gaussian processes (GPs) to infer constraints on the neutron-star matter EOS at intermediate densities from two different microscopic theories: chiral effective-field theory (𝜒⁢EFT) at baryon densities around nuclear saturation, 𝑛 𝐵 ∼ 𝑛 0 , and perturbative QCD at asymptotically high baryon densities, 𝑛 𝐵 ⩾ 20⁢𝑛 0 . The uncertainties of the 𝜒⁢EFT and pQCD EOSs are obtained using the BUQEYE truncation error model. We demonstrate the flexibility of our framework through the use of two categories of GP kernels: conventional stationary kernels and a nonstationary changepoint kernel. We use the latter to explore potential constraints on the dense matter EOS by including exogenous data representing theory predictions and heavy-ion collision measurements at densities ⩾ 2⁢𝑛 0 . We also use our EOSs to obtain neutron-star mass-radius relations and their uncertainties. Finally, our framework, whose implementation will be available through a GitHub repository, provides a prior distribution for the EOS that can be used in large-scale neutron-star inference frameworks.

Bayesian methods↗
DOE OSTI2025

Effect of likelihood misspecification in Gaussian process-driven autonomous experimentation

In recent years, several groups have designed Autonomous Experiment (AE) models with the aim of using them as an alternative method for neutron scattering scanning. In an AE, Gaussian processes (GPs) are most frequently used due to their interpretability, their non-parametric nature, their universal approximation, and their closed-form predictive distribution. GPs have two key components, namely, the model for the likelihood of a neutron count knowing the underlying dynamic structure factor and the acquisition function. In this paper, we investigate the impact, on the quality of an AE, of the likelihood and acquisition function choices, in energy scans and (Q, ω) ones, with respect to the signal-over-noise ratio. While we hypothesized that the quality of GP predictions would decrease when the normal to Poisson likelihood approximation breaks down at low count rates, we found that the use of the correct Poisson likelihood does not improve the quality of the data collected, as well as yields very poor results in (Q, ω) scans at low count rates. In fact, the best results are obtained with a combination of normal likelihood, including the observation noise, and the change in variance acquisition function. In addition, we find that the performance, or quality of the predictive distribution, is a misleading measure of efficiency, that is, of the quality of the data collected.

Perryman, David Elliott [Inst. Laue-Langevin (ILL)↗