Search NASA⌕ Search

SEARCH · Search NASA

Results for “Gaussian process regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Uncertainty quantification in machine learning for engineering design and health prognostics: A tutorial

On top of machine learning (ML) models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Towards this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. In conclusion, two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage (case study 1) and the remaining useful life prediction of turbofan engines (case study 2).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes

Uncertainty propagation in complex engineering systems often poses significant computational challenges related to modeling and quantifying probability distributions of model outputs, as those emerge as the result of various sources of uncertainty that are inherent in the system under investigation. Gaussian Processes regression (GPs) is a robust meta-modeling technique that allows for fast model prediction and exploration of response surfaces. Multi-fidelity variations of GPs further leverage information from cheap and low fidelity model simulations in order to improve their predictive performance on the high fidelity model. In order to cope with the high volume of data required to train GPs in high dimensional design spaces, a common practice is to introduce latent design variables that are typically projections of the original input space to a lower dimensional subspace, and therefore substitute the problem of learning the initial high dimensional mapping, with that of training a GP on a low dimensional space. Here in this paper, we present a Bayesian approach to identify optimal transformations that map the input points to low dimensional latent variables. The \projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters, conditioned on the available training data. The proposed Bayesian inference scheme relies on a two-step iterative algorithm that samples from the marginal posteriors of the GP parameters and the projection matrix respectively, both using Markov Chain Monte Carlo (MCMC) sampling. In order to take into account the orthogonality constraints imposed on the orthonormal projection matrix, a Geodesic Monte Carlo sampling algorithm is employed, that is suitable for exploiting probability measures on manifolds. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. We validate our framework on three synthetic problems with a known lower-dimensional subspace. The benefits of our proposed framework, are illustrated on the computationally challenging aerodynamic optimization of a last-stage blade for an industrial gas turbine, where we study the effect of an 85-dimensional shape parameterization of a three-dimensional airfoil on two output quantities of interest, specifically on the aerodynamic efficiency and the degree of reaction

42 ENGINEERING↗
DOE OSTI2020

Active- and transfer-learning applied to microscale-macroscale coupling to simulate viscoelastic flows

Active- and transfer-learning are applied to microscale dynamics of polymer flows for the multiscale discovery of effective constitutive approximations required in viscoelastic flow simulation. The result is macroscopic rheology directly connected to a microstructural model. Micro and macroscale simulations are adaptively coupled by means of Gaussian process regression (GPR) to run the expensive microscale computations only as necessary. This multiscale method is demonstrated with flows of a polymer solution as a model system. At the microscale level dissipative particle dynamics (DPD) is employed to model the fluid as a suspension of bead-spring micro-structures subjected to steady shear flow. The results yield the non-Newtonian viscosity and the first normal stress difference at strain rates as training data used in a GPR model. DPD parameters are calibrated with respect to experimental data for a real polymer solution. Compliance with these data requires adjustment of the DPD model's cutoff radius, which then becomes a function of the second invariant of the strain rate tensor. The FENE-P model is chosen for the macroscale description using the spectral element method (SEM) to simulate channel flow and flow past a circular cylinder. The DPD results at the lowest possible shear strain rate yield an estimate of the zero-shear rate viscosity, which allows the initiation of the macroscale flow by SEM as a Newtonian fluid. The resulting strain-rate field is surveyed to determine additional shear strain rate sampling points for the DPD system. This new information allows an initial fitting of parameters of the constitutive equation followed by new SEM simulations at the macroscale. Additionally, guided by active-learning GPR to select new sampling points, this process continues until convergence is achieved. The effectiveness of this new simulation paradigm for viscoelastic flows is tested with different macroscale operating conditions. The effective closure learned in the channel simulation is then transferred directly to the flow past a circular cylinder at low Reynolds number, where the results show that only two additional DPD simulations are required to achieve a satisfactory constitutive model. With an increase of the Reynolds number, the active-learning scheme automatically detects the inaccuracy of the learned constitutive model, and initiates additional DPD simulations for the extra data needed to once again close the microscale-macroscale coupled system. This new paradigm of active- and transfer-learning for multiscale modeling is readily applicable to other microscale-macroscale coupled simulations of complex fluids and other materials. Furthermore, the coupling between microscale and macroscale solvers can be seamlessly implemented with our open source multiscale universal interface (MUI) library.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Uncertainty-aware molecular dynamics from Bayesian active learning for phase transformations and thermal transport in SiC

Abstract Machine learning interatomic force fields are promising for combining high computational efficiency and accuracy in modeling quantum interactions and simulating atomistic dynamics. Active learning methods have been recently developed to train force fields efficiently and automatically. Among them, Bayesian active learning utilizes principled uncertainty quantification to make data acquisition decisions. In this work, we present a general Bayesian active learning workflow, where the force field is constructed from a sparse Gaussian process regression model based on atomic cluster expansion descriptors. To circumvent the high computational cost of the sparse Gaussian process uncertainty calculation, we formulate a high-performance approximate mapping of the uncertainty and demonstrate a speedup of several orders of magnitude. We demonstrate the autonomous active learning workflow by training a Bayesian force field model for silicon carbide (SiC) polymorphs in only a few days of computer time and show that pressure-induced phase transformations are accurately captured. The resulting model exhibits close agreement with both ab initio calculations and experimental measurements, and outperforms existing empirical models on vibrational and thermal properties. The active learning workflow readily generalizes to a wide range of material systems and accelerates their computational understanding.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Predicting concrete compressive strength using hybrid ensembling of surrogate machine learning models

This study aims to implement a hybrid ensemble surrogate machine learning technique in predicting the compressive strength (CS) of concrete, an important parameter used for durability design and service life prediction of concrete structures in civil engineering projects. For this purpose, an experimental database consisting of 1030 records has been compiled from the machine learning repository of the University of California, Irvine. The database was used to train and validate four conventional machine learning (CML) models, namely Artificial Neural Network (ANN), Linear and Non-Linear Multivariate Adaptive Regression Splines (MARS-L and MARS-C), Gaussian Process Regression (GPR), and Minimax Probability Machine Regression (MPMR). Subsequently, the predicted outputs of CML models were combined and trained using ANN to construct the Hybrid Ensemble Model (HENSM). It is observed that the proposed HENSM produces higher predictive accuracy compared to the CML models used in the present study. The predictive performance of all models for CS prediction was compared using the testing dataset and it is found that the HENSM model attained the highest predictive accuracy in both phases. Based on the experimental results, the newly constructed HENSM model is very potential to be a new alternative in handling the overfitting issues of CML models and hence, can be used to predict the concrete CS, including the design of less polluting and more sustainable concrete constructions.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Data-Driven Machine Learning for Wind Plant Flow Modeling

In this study, we introduce a data-driven machine learning framework for improving the accuracy of wind plant flow models by learning turbulence model corrections based on data from higher-fidelity simulations. First, a high-dimensional PDE-constrained optimization problem is solved using gradient-based optimization with adjoints to determine optimal eddy viscosity fields that improve the agreement of a medium-fidelity Reynolds-Averaged Navier Stokes (RANS) model with large eddy simulations (LES). A supervised learning problem is then constructed to find general, predictive representations of the optimal turbulence closure. A machine learning technique using Gaussian process regression is trained to predict the eddy viscosity field based on local RANS flow field information like velocities, pressures, and their gradients. The Gaussian process is trained on LES simulations of a single turbine and implemented in a wind plant simulation with 36 turbines. We show improvement over the baseline RANS model with the machine learning correction, and demonstrate the ability to provide accurate confidence levels for the corrections that enable future uncertainty quantification studies.

17 WIND ENERGY↗
NASA NTRSDate not supplied

Characterizing Spatiotemporal Uncertainty in Interpolated Meteorological Data

Interpolated meteorological data invariably contain errors. These errors have structure in time and space, particularly autocorrelation, which can cause the effects of errors to compound when model outputs are aggregated temporally or spatially. One way to account for this uncertainty is with a probabilistic model from which samples can be drawn that are coherent with respect to underlying spatial and temporal covariance structure. This work describes a probabilistic method for spatial interpolation of point-wise meteorological time series. Observational data from weather stations are generally sparse in space and dense in time (but sometimes missing). The method works by projecting time series onto orthogonal basis vectors and spatially interpolating each resulting component independently. Under suitable assumptions, and data transformations to better satisfy those assumptions, Gaussian process regression provides a complete description of the joint predictive distribution over a Gaussian random field. Spatiotemporally coherent realizations are generated as the sum of conditional (spatial) simulations of each orthogonal (temporal) component. Data-derived and generic orthogonal bases are considered. In addition to spatial interpolation, imputation of missing observational data is examined. The method is applied using near-surface air temperature over the Western United States and validated by comparing theoretical versus actual coverage of predictive distributions and analyzing the degree to which spatial and temporal covariance structure is reproduced. Computational considerations, relating to conditional simulation of random fields, are also addressed.

Conor T Doherty↗
DOE OSTI2021

A comparison of Gaussian processes and neural networks for computer model emulation and calibration

The Department of Energy relies on complex physics simulations for prediction in domains like cosmology, nuclear theory, and materials science. These simulations are often extremely computationally intensive, with some requiring days or weeks for a single simulation. In order to assure their accuracy, these models are calibrated against observational data in order to estimate inputs and systematic biases. Because of their great computational complexity, this process typically requires the construction of an emulator, a fast approximation to the simulation. In this paper, two emulator approaches are compared: Gaussian process regression and neural networks. Their emulation accuracy and calibration performance on three real problems of Department of Energy interest is considered. On these problems, the Gaussian process emulator tends to be more accurate with narrower, but still well-calibrated uncertainty estimates. The neural network emulator is accurate, but tends to have large uncertainty on its predictions. Finally, as a result, calibration with the Gaussian process emulator produces more constrained posteriors that still perform well in prediction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Nested polynomial trends for the improvement of Gaussian process-based predictors

The role of simulation keeps increasing for the sensitivity analysis and the uncertainty quantification of complex systems. Such numerical procedures are generally based on the processing of a huge amount of code evaluations. When the computational cost associated with one particular evaluation of the code is high, such direct approaches based on the computer code only, are not affordable. Surrogate models have therefore to be introduced to interpolate the information given by a fixed set of code evaluations to the whole input space. When confronted to deterministic mappings, the Gaussian process regression (GPR), or kriging, presents a good compromise between complexity, efficiency and error control. Such a method considers the quantity of interest of the system as a particular realization of a Gaussian stochastic process, whose mean and covariance functions have to be identified from the available code evaluations. In this context, this work proposes an innovative parametrization of this mean function, which is based on the composition of two polynomials. This approach is particularly relevant for the approximation of strongly non linear quantities of interest from very little information. After presenting the theoretical basis of this method, this work compares its efficiency to alternative approaches on a series of examples.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS2021

Hot planets around cool stars – two short-period mini-Neptunes transiting the late K-dwarf TOI-1260

We present the discovery and characterization of two sub-Neptunes in close orbits, as well as a tentative outer planet of a similar size, orbiting TOI-1260 – a low metallicity K6 V dwarf star. Photometry from Transiting Exoplanet Survey Satellite(TESS) yields radii of R(b) = 2.33 ± 0.10 and R(c) = 2.82 ± 0.15 Rꚛ, and periods of 3.13 and 7.49 d for TOI-1260 b and TOI-1260 c, respectively. We combined the TESS data with a series of ground-based follow-up observations to characterize the planetary system. From HARPS-N high-precision radial velocities we obtain M(b) = 8.6(+1.4,−1.5) and M(c) = 11.8(+3.4,−3.2) Mꚛ. The star is moderately active with a complex activity pattern, which necessitated the use of Gaussian process regression for both the light-curve detrending and the radial velocity modelling, in the latter case guided by suitable activity indicators. We successfully disentangle the stellar-induced signal from the planetary signals, underlining the importance and usefulness of the Gaussian process approach. We test the system’s stability against atmospheric photoevaporation and find that the TOI-1260 planets are classic examples of the structure and composition ambiguity typical for the 2–3 Rꚛ range.

I Y Georgieva↗
DOE OSTI2022

Molecular dipole moment learning via rotationally equivariant derivative kernels in molecular-orbital-based machine learning

This study extends the accurate and transferable molecular-orbital-based machine learning (MOB-ML) approach to modeling the contribution of electron correlation to dipole moments at the cost of Hartree–Fock computations. A MOB pairwise decomposition of the correlation part of the dipole moment is applied, and these pair dipole moments could be further regressed as a universal function of MOs. The dipole MOB features consist of the energy MOB features and their responses to electric fields. An interpretable and rotationally equivariant derivative kernel for Gaussian process regression (GPR) is introduced to learn the dipole moment more efficiently. The proposed problem setup, feature design, and ML algorithm are shown to provide highly accurate models for both dipole moments and energies on water and 14 small molecules. To demonstrate the ability of MOB-ML to function as generalized density-matrix functionals for molecular dipole moments and energies of organic molecules, we further apply the proposed MOB-ML approach to train and test the molecules from the QM9 dataset. The application of local scalable GPR with Gaussian mixture model unsupervised clustering GPR scales up MOB-ML to a large-data regime while retaining the prediction accuracy. In addition, compared with the literature results, MOB-ML provides the best test mean absolute errors of 4.21 mD and 0.045 kcal/mol for dipole moment and energy models, respectively, when training on 110 000 QM9 molecules. The excellent transferability of the resulting QM9 models is also illustrated by the accurate predictions for four different series of peptides.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2018

Magellan/PFS Radial Velocities of GJ 9827, a Late K dwarf at 30 pc with Three Transiting Super-Earths

The Kepler mission showed us that planets with sizes between that of Earth and Neptune appear to be the most common type in our Galaxy. These “super-Earths” continue to be of great interest for exoplanet formation, evolution, and composition studies. However, the number of super-Earths with well-constrained mass and radius measurements remains small (40 planets with σ {sub mass} < 25%), due in part to the faintness of their host stars causing ground-based mass measurements to be challenging. Recently, three transiting super-Earth planets were detected by the K2 mission around the nearby star GJ 9827/HIP 115752, at only 30 pc away. The radii of the planets span the “radius gap” detected by Fulton et al. (2017), and all orbit within ∼6.5 days, easing follow-up observations. Here, we report radial velocity (RV) observations of GJ 9827, taken between 2010 and 2016 with the Planet Finder Spectrograph on the Magellan II Telescope. We employ two different RV analysis packages, SYSTEMIC and RadVel, to derive masses and thus densities of the GJ 9827 planets. We also test a Gaussian Process regression analysis but find the correlated stellar noise is not well constrained by the PFS data and that the GP tends to over-fit the RV semi-amplitudes resulting in a lower K value. Our RV observations are not able to place strong mass constraints on the two outer planets (c and d) but do indicate that planet b, at 1.64 R {sub ⊕} and ∼8 M {sub ⊕}, is one of the most massive (and dense) super-Earth planets detected to date.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Gaussian processes for autonomous data acquisition at large-scale synchrotron and neutron facilities

The execution and analysis of complex experiments are challenged by the vast dimensionality of the underlying parameter spaces. Although an increase in data-acquisition rates should allow broader querying of the parameter space, the complexity of experiments and the subtle dependence of the model function on input parameters remains daunting owing to the sheer number of variables. New strategies for autonomous data acquisition are being developed, with one promising direction being the use of Gaussian process regression (GPR). GPR is a quick, non-parametric and robust approximation and uncertainty quantification method that can be applied directly to autonomous data acquisition. We review GPR-driven autonomous experimentation and illustrate its functionality using real-world examples from large experimental facilities in the USA and France. We introduce the basics of a GPR-driven autonomous loop with a focus on Gaussian processes, and then shift the focus to the infrastructure that needs to be built around GPR to create a closed loop. Finally, the case studies we discuss show that Gaussian-process-based autonomous data acquisition is a widely applicable method that can facilitate the optimal use of instruments and facilities by enabling the efficient acquisition of high-value datasets.

36 MATERIALS SCIENCE↗
DOE OSTI2016

An Iterative Optimization Strategy for Nuclear Reactor Design

Optimization of a complex system involves the determination of optimum values for a set of design parameters to meet a specific set of objectives on the quantities of interest (QOI) wherein, the design parameters are a subset of the input parameters, and the QOIs are determined from the output parameters. Particularly, when the parameter space is large, optimization necessitates a significant number of executions of the simulator to obtain a desired solution in tolerance limits. When the simulations are expensive in terms of computation time, an emulator, based on regression methods is useful for predictions. This work presents a novel methodology that uses an iterative hybrid global optimization method (GOM) using genetic algorithms (GA), and simulated annealing (SA) model coupled (HYBGASA) with a Gaussian process regression method based emulator (GPMEM) to optimize a set of input parameters based on a set of defined objectives in a nuclear reactor power system. Hereafter this iterative hybrid method comprising of HYBGASA, and GPMEM would be called as 'IHGOM'. In addition to optimization, IHGOM iteratively updates the trial data, obtained from the neighborhood of the near optimal solution, used to train the GPMEM in order to reduce regression errors. The objective is to develop, model and analyze IHGOM and apply it to an optimization problem in the design of a nuclear reactor. Development and analysis of IHGOM and its implementation in a nuclear reactor power system problem is a significant contribution to the optimization and the nuclear engineering communities. The objective of this work is to develop IHGOM and apply it to an optimization problem in the design of a nuclear reactor. The research involves the development and analysis of a novel optimization strategy, IHGOM. The feasibility analysis involves the optimization of a set of defined parameters in the design and development of a complex system. A GCFBR reactor power system with a set of defined objectives and parameters based on criticality and depletion studies have been characterized as the complex system. Development and analysis of IHGOM and its implementation in an optimization problem in the design of a GCFBR is a significant contribution to the optimization and the nuclear engineering communities. In particular based on the literature review, a coupled GOM and GPMEM method for optimization and regression error reduction in nuclear engineering domain has never been explored. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2019

Data-Consistent Solutions to Stochastic Inverse Problems Using a Probabilistic Multi-Fidelity Method Based on Conditional Densities

In this work, we build upon a recently developed approach for solving stochastic inverse problems based on a combination of measure-theoretic principles and Bayes' rule. We propose a multi-fidelity method to reduce the computational burden of performing uncertainty quantification using high-fidelity models. This approach is based on a Monte Carlo framework for uncertainty quantification that combines information from solvers of various fidelities to obtain statistics on the quantities of interest of the problem. In particular, our goal is to generate samples from a high-fidelity push-forward density at a fraction of the costs of standard Monte Carlo methods, while maintaining flexibility in the number of random model input parameters. Key to this methodology is the construction of a regression model to represent the stochastic mapping between the low- and high-fidelity models, such that most of the computations can be leveraged to the low-fidelity model. To that end, we employ Gaussian process regression and present extensions to multi-level-type hierarchies as well as to the case of multiple quantities of interest. Finally, we demonstrate the feasibility of the framework in several numerical examples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Combining Observational and Computational Uncertainty in Calibration Experiments

Traditional computer experiments using emulators have treat the emulator output as containing no error; however, this is not always feasible, as there exist computational codes which produce stochastic output, and future computations may require a balance between the time or resources to run the code and the accuracy of the output. This paper proposes a framework for incorporating both computational uncertainty and observational uncertainty into an analysis using a Gaussian process regression and calibration, and provides an example using simulated output from the one-dimensional diffusion equation. The particular values of observational and computation uncertainty which minimize a cost function are obtained.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Sparse regression for plasma physics

Many scientific problems can be formulated as sparse regression, i.e., regression onto a set of parameters when there is a desire or expectation that some of the parameters are exactly zero or do not substantially contribute. This includes many problems in signal and image processing, system identification, optimization, and parameter estimation methods such as Gaussian process regression. Sparsity facilitates exploring high-dimensional spaces while finding parsimonious and interpretable solutions. In the present work, we illustrate some of the important ways in which sparse regression appears in plasma physics and point out recent contributions and remaining challenges to solving these problems in this field. Further, a brief review is provided for the optimization problem and the state-of-the-art solvers, especially for constrained and high-dimensional sparse regression.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Physics-Based Machine Learning Methods for U-235 Forensics Signatures

Signatures of low-intensity U-235 sources have been recently studied by utilizing a variety of machine learning (ML) classifiers using features derived from gamma spectral measurements collectedunder structured campaigns. Several ML classifiers, such as ensemble of tress and classification trees, revealed misleadingly-optimistic training error due to over-fitting, and furthermore,their performance is not directly relatable to the physical properties due to their data-driven, opaque designs. We present a regression-based ML method that first estimates the inverse distanceto the source and then utilizes a threshold to infer its presence, by representing the background as a source located at an infinite distance. For the inverse distance estimation, we study the ensembleof trees and Gaussian process regression methods, and a hyper parameter auto-tuning and selection method that employs five regression estimators. These methods avoid the over-fittingobserved in several ML classifiers, while providing the classification error nearly comparable to them based on independent test data. Their error is directly related to estimates of the inversephysical distance to source, and the precision of error determines the seperability property that determines the false alarm and missed detection rates. The property of monotonic decrease of thesource strength with increasing detector distance combined with Poisson distribution of measurements is utilized to analytically validate these methods by deriving the generalization equations ofunderlying regression methods.

Rao, Nageswara↗