Search NASA⌕ Search

SEARCH · Search NASA

Results for “Gaussian Process (GP)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

123 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Fast Gaussian Process Estimation for Large-Scale In Situ Inference using Convolutional Neural Networks

Exascale computing will bring with it significant I/O limitations. One foreseeable consequence of such restrictions is that the user can save only a small fraction of complex simulation data to disk for subsequent analysis. An alternative is to fit statistical models to data in situ, that is, inside the simulation as it runs. This option requires extremely fast statistical estimation to avoid slowing down the simulation. Gaussian processes (GPs) have state-of-the-art predictive performance for modeling spatial data. However, standard estimation methods for GPs scale quite poorly to large data sets as parameter estimation requires inverting a covariance matrix to the size of the data set. In the presented work, we use a convolutional neural network (CNN) to predict the GP parameters for a spatial data set, from a simulation or otherwise, rather than optimize the parameters directly. Here, our presented case study models spatial data from E3SM, the Department of Energy’s Exascale climate model. The CNN is trained on synthetic data simulated from GP models with known parameters and then applied to data from the climate simulation. In the presented examples, the neural network scheme produces parameter estimates that compare well with standard methods such as maximum likelihood estimation in predictive performance but is obtained four orders of magnitude faster.

big data↗
DOE OSTI2020

Applying Constrained Bayesian Optimization to the Design of Critical Experiments

Often when planning a criticality experiment, many design configurations are iteratively investigated with a Monte Carlo transport code. The goal is that the experiment will be optimal with respect to some variable, like the fraction of fissions occurring at a certain energy range, while simultaneously being critical. Unfortunately, the Monte Carlo transport simulations are expensive, which can ultimately limit the number of configurations that can be explored. In this work, we present how Gaussian processes (GPs) can be used as a reduced-order model in a constrained Bayesian optimization (CBO) algorithm to design a criticality experiment. The GPs replace the Monte Carlo transport simulations that explore the design space. The CBO algorithm efficiently identifies new points in the design space to run the Monte Carlo transport code while respecting the criticality constraint. It does so in a manner that both improves the accuracy of the GP and finds the approximate global optimum. We demonstrate the performance of CBO with the design of a Thermal Epithermal eXperiment (TEX) for the criticality safety validation of nuclear waste models of the Hanford Tank Farm.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Star–Galaxy Image Separation with Computationally Efficient Gaussian Process Classification

Abstract We introduce a novel method for discerning optical telescope images of stars from those of galaxies using Gaussian processes (GPs). Although applications of GPs often struggle in high-dimensional data modalities such as optical image classification, we show that a low-dimensional embedding of images into a metric space defined by the principal components of the data suffices to produce high-quality predictions from real large-scale survey data. We develop a novel method of GP classification hyperparameter training that scales approximately linearly in the number of image observations, which allows for application of GP models to large-size Hyper Suprime-Cam Subaru Strategic Program data. In our experiments, we evaluate the performance of a principal component analysis embedded GP predictive model against other machine-learning algorithms, including a convolutional neural network and an image photometric morphology discriminator. Our analysis shows that our methods compare favorably with current methods in optical image classification while producing posterior distributions from the GP regression that can be used to quantify object classification uncertainty. We further describe how classification uncertainty can be used to efficiently parse large-scale survey imaging data to produce high-confidence object catalogs.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

A Fast, Two-dimensional Gaussian Process Method Based on Celerite: Applications to Transiting Exoplanet Discovery and Characterization

Gaussian processes (GPs) are commonly used as a model of stochastic variability in astrophysical time series. In particular, GPs are frequently employed to account for correlated stellar variability in planetary transit light curves. The efficient application of GPs to light curves containing thousands to tens of thousands of data points has been made possible by recent advances in GP methods, including the celerite method. Here we present an extension of the celerite method to two input dimensions where, typically, the second dimension is small. This method scales linearly with the total number of data points when the noise in each large dimension is proportional to the same celerite kernel and only the amplitude of the correlated noise varies in the second dimension. We demonstrate the application of this method to the problem of measuring precise transit parameters from multiwavelength light curves and show that it has the potential to improve transit parameters measurements by orders of magnitude. Applications of this method include transit spectroscopy and exomoon detection, as well a broader set of astronomical problems.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Non-smooth Bayesian optimization in tuning scientific applications

Tuning algorithmic parameters to optimize the performance of large, complicated computational codes is an important problem involving finding the optima and identifying regimes defined by non-smooth boundaries in black-box functions. Within the Bayesian optimization framework, the Gaussian process surrogate model produces smooth mean functions, but functions in the tuning problem are often non-smooth, which is exacerbated by the fact that we usually have limited sequential samples from the black-box function. Here, motivated by these issues encountered in tuning, we propose a novel Gaussian process model called a clustered Gaussian process (cGP), where the components are dynamically updated by clustering. In our studies, the performance of cGP can be better than stationary GPs in nearly 90% of the experiments and better than non-stationary GPs in nearly 70% of the repeated experiments while requiring less computational cost. cGP provides a novel approach for dynamic GP, computes more efficiently than recursive partitioning, and discovers non-smoothness regimes. We provide extensive experiments including high-performance computing (HPC) and industrial simulation functions to show the effectiveness of our methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Mapping Stellar Surfaces. II. An Interpretable Gaussian Process Model for Light Curves

The use of Gaussian processes (GPs) as models for astronomical time series data sets has recently become almost ubiquitous, given their ease of use and flexibility. In particular, GPs excel at marginalization over the stellar signal when the variability due to starspots is treated as a nuisance, as in exoplanet transit modeling. However, these effective models are less useful in cases where the starspot signal is of primary interest, since it is not obvious how the parameters of the GP relate to physical parameters like the spot size, contrast, and latitudinal distribution. Instead, it is common practice to explicitly model the effect of individual starspots on the light curve and attempt to infer their properties via optimization or posterior inference. Unfortunately, this process is ill-posed and often computationally intractable when applied to stars with more than a few spots and/or to ensembles of many stars. Here we derive a closed-form expression for a GP that describes the light curve of a rotating, evolving stellar surface conditioned on a given distribution of starspot sizes, contrasts, and latitudes. We demonstrate that this model is correctly calibrated, allowing one to robustly infer physical parameters of interest from one or more light curves, including the typical spot radii and latitudes. Our GP has far-ranging implications for understanding the variability and magnetic activity of stars from light curves and radial velocity measurements, as well as for modeling correlated noise in exoplanet searches. Our implementation is efficient, user-friendly, and open-source, available in the package starry-process.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Towards Efficient Uncertainty estimation in deep learning for robust energy prediction in crystal materials

In material science, recent studies have started to explore the potential of using deep learning to improve property prediction from high-fidelity simulations, e.g, density functional theory (DFT). However, the design spaces are sometimes too large and intractable to sample completely. This results in a critical question that is how to evaluate the confidence and robustness of the prediction. In this paper, we propose an efficient approach to estimate uncertainty in deep learning using a single forward pass and then apply it for robust prediction of the total energy in crystal lattice structures. Our approach is built upon the deep kernel learning (DKL) that originally introduces to leverage the expressiveness of deep neural networks as input with a probabilistic prediction of Gaussian processes (GPs) as output. Existing DKL methods have difficulties in the accuracy of predictive uncertainty, training stability, and scaling to large datasets, which lead to significant barriers in real-world applications. We propose to address these challenges by using an inducing point approximate GP in feature space combined with spectral normalization as a regularization. We finally demonstrate our robust performance on an artificial example and a real-world application from materials chemistry.

Bi, Sirui↗
DOE OSTI2024

Exploration with Scalable Gaussian Process Reinforcement Learning

Exploration is a challenging problem in reinforcement learning (RL), especially in environments with sparse rewards. Quantifying and utilizing the parametric uncertainty has been shown to be paramount for successful exploration [Osband et al., 2018]. Bayesian, or approximately Bayesian, methods present a principled means of estimating the parametric uncertainty in RL problems. Gaussian processes, nonparametric Bayesian models, are often impractical due to poor scalability and computational bottlenecks. We introduce a scalable Gaussian process RL (GPRL) method which directly induces sparsity in the covariance matrix to facilitate faster computation. This is a departure from previous GPRL methods which instead rely on data reduction and subsampling. We compare various covariance-based exploration techniques (Thompson sampling, upper confidence bound, and probabilistic maximum variance) which leverage our scalable GP framework in sparse reward environments. Finally, we show favorable comparison against the bootstrapped deep Q-Network.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Sparse Cholesky factorization for solving nonlinear PDEs via Gaussian processes

In recent years, there has been widespread adoption of machine learning-based approaches to automate the solving of partial differential equations (PDEs). Among these approaches, Gaussian processes (GPs) and kernel methods have garnered considerable interest due to their flexibility, robust theoretical guarantees, and close ties to traditional methods. They can transform the solving of general nonlinear PDEs into solving quadratic optimization problems with nonlinear, PDE-induced constraints. However, the complexity bottleneck lies in computing with dense kernel matrices obtained from pointwise evaluations of the covariance kernel, and its partial derivatives, a result of the PDE constraint and for which fast algorithms are scarce. The primary goal of this paper is to provide a near-linear complexity algorithm for working with such kernel matrices. We present a sparse Cholesky factorization algorithm for these matrices based on the near-sparsity of the Cholesky factor under a novel ordering of pointwise and derivative measurements. The near-sparsity is rigorously justified by directly connecting the factor to GP regression and exponential decay of basis functions in numerical homogenization. We then employ the Vecchia approximation of GPs, which is optimal in the Kullback-Leibler divergence, to compute the approximate factor. This enables us to compute ϵ-approximate inverse Cholesky factors of the kernel matrices with complexity O(N log d (N/ϵ)) in space and O(N log 2d (N/ϵ)) in time. We integrate sparse Cholesky factorizations into optimization algorithms to obtain fast solvers of the nonlinear PDE. We numerically illustrate our algorithm’s near-linear space/time complexity for a broad class of nonlinear PDEs such as the nonlinear elliptic, Burgers, and Monge-Ampère equations. In summary, we provide a fast, scalable, and accurate method for solving general PDEs with GPs and kernel methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Stochastic machine learning via sigma profiles to build a digital chemical space

This work establishes a different paradigm on digital molecular spaces and their efficient navigation by exploiting sigma profiles. To do so, the remarkable capability of Gaussian processes (GPs), a type of stochastic machine learning model, to correlate and predict physicochemical properties from sigma profiles is demonstrated, outperforming state-of-the-art neural networks previously published. The amount of chemical information encoded in sigma profiles eases the learning burden of machine learning models, permitting the training of GPs on small datasets which, due to their negligible computational cost and ease of implementation, are ideal models to be combined with optimization tools such as gradient search or Bayesian optimization (BO). Gradient search is used to efficiently navigate the sigma profile digital space, quickly converging to local extrema of target physicochemical properties. While this requires the availability of pretrained GP models on existing datasets, such limitations are eliminated with the implementation of BO, which can find global extrema with a limited number of iterations. A remarkable example of this is that of BO toward boiling temperature optimization. Holding no knowledge of chemistry except for the sigma profile and boiling temperature of carbon monoxide (the worst possible initial guess), BO finds the global maximum of the available boiling temperature dataset (over 1,000 molecules encompassing more than 40 families of organic and inorganic compounds) in just 15 iterations (i.e., 15 property measurements), cementing sigma profiles as a powerful digital chemical space for molecular optimization and discovery, particularly when little to no experimental data is initially available.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Fast Emulation of Expensive Simulations using Approximate Gaussian Processes [Slides]

Nuclear Computational Low-Energy Initiative (NUCLEI) collaboration uses Density Functional Theory (DFT) simulations to predict the structure and binding energies of nuclei over a wide range of proton (Z) and neutron (N) numbers. The DFT simulations utilize a particular parameterization of a Skyrme energy density functional called UNEDF1 which depends on 12 free parameters that must be fit to data (M Kortelainen et al 2014). Fitting involves comparing (e.g.) predicted binding energies of nuclei to experimentally measured values. We use only binding energies as observables, but DFT with UNEDF1 will predict structure (shape) observables as well. In this work, assessing the capability of approximate GP emulators to balance emulator accuracy with computational speed to facilitate improved UNEDF1 calibration. Sparse GPs are straightforward to train and accurate. Calibration is not straightforward with MCMC (using MH or HMC/NUTS). We produced reusable software for continuing and building on this work as well as accessing and using Darwin cluster compute resources

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes

Uncertainty propagation in complex engineering systems often poses significant computational challenges related to modeling and quantifying probability distributions of model outputs, as those emerge as the result of various sources of uncertainty that are inherent in the system under investigation. Gaussian Processes regression (GPs) is a robust meta-modeling technique that allows for fast model prediction and exploration of response surfaces. Multi-fidelity variations of GPs further leverage information from cheap and low fidelity model simulations in order to improve their predictive performance on the high fidelity model. In order to cope with the high volume of data required to train GPs in high dimensional design spaces, a common practice is to introduce latent design variables that are typically projections of the original input space to a lower dimensional subspace, and therefore substitute the problem of learning the initial high dimensional mapping, with that of training a GP on a low dimensional space. Here in this paper, we present a Bayesian approach to identify optimal transformations that map the input points to low dimensional latent variables. The \projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters, conditioned on the available training data. The proposed Bayesian inference scheme relies on a two-step iterative algorithm that samples from the marginal posteriors of the GP parameters and the projection matrix respectively, both using Markov Chain Monte Carlo (MCMC) sampling. In order to take into account the orthogonality constraints imposed on the orthonormal projection matrix, a Geodesic Monte Carlo sampling algorithm is employed, that is suitable for exploiting probability measures on manifolds. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. We validate our framework on three synthetic problems with a known lower-dimensional subspace. The benefits of our proposed framework, are illustrated on the computationally challenging aerodynamic optimization of a last-stage blade for an industrial gas turbine, where we study the effect of an 85-dimensional shape parameterization of a three-dimensional airfoil on two output quantities of interest, specifically on the aerodynamic efficiency and the degree of reaction

42 ENGINEERING↗
DOE OSTI2023

Validating the Use of Gaussian Process Regression for Adaptive Mapping of Residual Stress Fields

Probing the stress state using a high density of measurement points is time intensive and presents a limitation for what is experimentally feasible. Alternatively, individual strain fields used for determining stresses can be reconstructed from a subset of points using a Gaussian process regression (GPR). Results presented in this paper evidence that determining stresses from reconstructed strain fields is a viable approach for reducing the number of measurements needed to fully sample a component’s stress state. The approach was demonstrated by reconstructing the stress fields in wire-arc additively manufactured walls fabricated using either a mild steel or low-temperature transition feedstock. Effects of errors in individual GP reconstructed strain maps and how these errors propagate to the final stress maps were assessed. Implications of the initial sampling approach and how localized strains affect convergence are explored to give guidance on how best to implement a dynamic sampling experiment.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Microscopic constraints for the equation of state and structure of neutron stars: A Bayesian model mixing framework

Bayesian model mixing (BMM) is a statistical technique that can combine constraints from different regions of an input space in a principled way. Here we extend our BMM framework for the equation of state (EOS) of strongly interacting matter from symmetric nuclear matter to asymmetric matter, specifically focusing on zero-temperature, charge-neutral, 𝛽-equilibrated matter. We use Gaussian processes (GPs) to infer constraints on the neutron-star matter EOS at intermediate densities from two different microscopic theories: chiral effective-field theory (𝜒⁢EFT) at baryon densities around nuclear saturation, 𝑛 𝐵 ∼ 𝑛 0 , and perturbative QCD at asymptotically high baryon densities, 𝑛 𝐵 ⩾ 20⁢𝑛 0 . The uncertainties of the 𝜒⁢EFT and pQCD EOSs are obtained using the BUQEYE truncation error model. We demonstrate the flexibility of our framework through the use of two categories of GP kernels: conventional stationary kernels and a nonstationary changepoint kernel. We use the latter to explore potential constraints on the dense matter EOS by including exogenous data representing theory predictions and heavy-ion collision measurements at densities ⩾ 2⁢𝑛 0 . We also use our EOSs to obtain neutron-star mass-radius relations and their uncertainties. Finally, our framework, whose implementation will be available through a GitHub repository, provides a prior distribution for the EOS that can be used in large-scale neutron-star inference frameworks.

Bayesian methods↗
DOE OSTI2025

Effect of likelihood misspecification in Gaussian process-driven autonomous experimentation

In recent years, several groups have designed Autonomous Experiment (AE) models with the aim of using them as an alternative method for neutron scattering scanning. In an AE, Gaussian processes (GPs) are most frequently used due to their interpretability, their non-parametric nature, their universal approximation, and their closed-form predictive distribution. GPs have two key components, namely, the model for the likelihood of a neutron count knowing the underlying dynamic structure factor and the acquisition function. In this paper, we investigate the impact, on the quality of an AE, of the likelihood and acquisition function choices, in energy scans and (Q, ω) ones, with respect to the signal-over-noise ratio. While we hypothesized that the quality of GP predictions would decrease when the normal to Poisson likelihood approximation breaks down at low count rates, we found that the use of the correct Poisson likelihood does not improve the quality of the data collected, as well as yields very poor results in (Q, ω) scans at low count rates. In fact, the best results are obtained with a combination of normal likelihood, including the observation noise, and the change in variance acquisition function. In addition, we find that the performance, or quality of the predictive distribution, is a misleading measure of efficiency, that is, of the quality of the data collected.

Perryman, David Elliott [Inst. Laue-Langevin (ILL)↗