SEARCH · Search NASA
Results for “Gaussian process learning”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Sampling Functions from Gaussian Processes and Structured Covariance Gaussian Networks
When learning aerodynamic models from data, it is critical to incorporate estimates of model uncertainty. This motivates the design of probabilistic aerodynamic databases which can be sampled to generate physically and statistically plausible aerodynamic models. In this talk we discuss how to sample deterministic functions from two different kinds of probabilistic models and demonstrate their use. First, Gaussian Process Regressors (GPRs) are a widely used probabilistic kernel-based model which can be thought of as Gaussian distributions over functions. GPRs are generally trained by maximizing the marginal likelihood of seeing the training data over the kernel parameter space. Sample functions are easily generated by drawing points from the Gaussian distribution at desired input points. However, when the points are not known ahead of time, the classical sampling approach is not possible since successive function samples will generate different function realizations. We present an approach for sampling consistent function evaluations from a GPR over multiple samples. Second, we describe a neural network architecture which learns a conditional Gaussian distribution by maximizing the marginal likelihood at each point in the input space. We then discuss and compare several options for generating sample functions which match this distribution. Finally, we demonstrate the use of these probabilistic aerodynamic models in an atmospheric reentry simulation.
Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data
We present a machine learning framework (GP-NODE) for Bayesian model discovery from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. The use of the Finnish Horseshoe as a sparsity-promoting prior for free model parameters also enables the discovery of parsimonious representations for the latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator–prey systems, systems biology and a 50-dimensional human motion dynamical system. This article is part of the theme issue ‘Data-driven prediction in dynamical systems’.
Data-Driven Machine Learning for Wind Plant Flow Modeling
In this study, we introduce a data-driven machine learning framework for improving the accuracy of wind plant flow models by learning turbulence model corrections based on data from higher-fidelity simulations. First, a high-dimensional PDE-constrained optimization problem is solved using gradient-based optimization with adjoints to determine optimal eddy viscosity fields that improve the agreement of a medium-fidelity Reynolds-Averaged Navier Stokes (RANS) model with large eddy simulations (LES). A supervised learning problem is then constructed to find general, predictive representations of the optimal turbulence closure. A machine learning technique using Gaussian process regression is trained to predict the eddy viscosity field based on local RANS flow field information like velocities, pressures, and their gradients. The Gaussian process is trained on LES simulations of a single turbine and implemented in a wind plant simulation with 36 turbines. We show improvement over the baseline RANS model with the machine learning correction, and demonstrate the ability to provide accurate confidence levels for the corrections that enable future uncertainty quantification studies.
Activity coefficient acquisition with thermodynamics–informed active learning for phase diagram construction
This work explores the use of thermodynamics-informed Gaussian processes (GPs) and active learning (AL) to model activity coefficients and construct phase diagrams. Relying on synthetic data generated from an excess Gibbs energy model, GPs were found to accurately describe the activity coefficients of several binary mixtures across large composition and temperature ranges. Moreover, GPs could estimate their own uncertainty and identify composition/temperature regions where activity coefficient data provide the most information to the models. This was leveraged to build AL algorithms targeted at modeling phase equilibria. In many cases, a single active-learning-acquired data point was sufficient to describe the phase diagrams studied. Lastly, the ability of AL to greatly reduce the amount of data needed to obtain accurate models was further verified on experimental case studies, namely individual ion activity coefficients, the solid–liquid and vapor–liquid equilibrium of deep eutectic solvents, and phase equilibria in ternary mixtures.
Adaptive, Active Learning, and Multifidelity Monte Carlo Methods in the MOOSE Stochastic Tools Module
MOOSE is an open-source computational platform for constructing multi-physics models and executing them in a massively parallel fashion. It has a stochastic tools module (STM) for forward/inverse uncertainty quantification (UQ) and surrogate modeling. This presentation details some recent developments to the STM with respect to the implementation of adaptive, active learning, and multifidelity Monte Carlo methods for forward UQ of computational models. Specifically, the adaptive Monte Carlo methods include Markov Chain Monte Carlo (MCMC)-driven algorithms like adaptive importance sampling and parallelized subset simulation for statistical QoI estimation, rare events analysis, and stochastic gradient-free optimization. The active learning methods include Gaussian Process (GP) surrogates and their training via Adam optimization, design of acquisition functions, and integration with samplers like Monte Carlo, adaptive importance, and parallelized subset simulation. These active learning methods are also designed to work in a batch mode, wherein, the required calls to the full computational model are executed in parallel whenever a user-specified batch size is met. The multifidelity methods in STM are broadly divided into two categories: hierarchical, where a defined hierarchy exists among the low-fidelity models, and peer, where all the low-fidelity models are treated equally. A GP surrogate is used to learn the differences between the low- and high-fidelity models in both multifidelity categories, and acquisition functions from the active learning classes are used to decide whether to rely on a low-fidelity model or call the expensive high-fidelity model. Alongside the software description and usage, applications are also presented to nuclear engineering computational models including a TRISO nuclear fuel particle, a reactor pressure vessel, and a heat-pipe microreactor.
Resonant di-Higgs production at gravitational wave benchmarks: a collider study using machine learning
We perform a complementarity study of gravitational waves and colliders in the context of electroweak phase transitions choosing as our template the xSM model, which consists of the Standard Model augmented by a real scalar. We carefully analyze the gravitational wave signal at benchmark points compatible with a first order phase transition, taking into account subtle issues pertaining to the bubble wall velocity and the hydrodynamics of the plasma. In particular, we comment on the tension between requiring bubble wall velocities small enough to produce a net baryon number through the sphaleron process, and large enough to obtain appreciable gravitational wave production. For the most promising benchmark models, we study resonant di-Higgs production at the high-luminosity LHC using machine learning tools: a Gaussian process algorithm to jointly search for optimum cut thresholds and tuning hyperparameters, and a boosted decision trees algorithm to discriminate signal and background. The multivariate analysis on the collider side is able either to discover or provide strong statistical evidence of the benchmark points, opening the possibility for complementary searches for electroweak phase transitions in collider and gravitational wave experiments.
Towards a Robust Adaptive Digital Twin for Fusion Applications
The development of a digital twin system for fusion applications is essential for enhancing the prediction, analysis, and optimization of complex plasma processes. Machine learning (ML), particularly deep learning has demonstrated strong capabilities in modeling such highly nonlinear and intricate systems. However, two critical challenges limit the deployment of deep learning-based digital twins: Uncertainty Quantification (UQ) and data drift. UQ is vital for ensuring trustworthy predictions, especially in decision-support scenarios. Additionally, data-driven models are often sensitive to changes in the underlying data distribution, such as shot-to-shot variations in fusion experiments, which can lead to performance degradation over time. To address these challenges, we are developing an uncertainty-aware, adaptive digital twin framework. Our approach incorporates deep learning models enhanced with Gaussian Process approximations for predictive uncertainty estimation, coupled with an online learning mechanism that enables continuous model adaptation to new experimental data. This adaptive capability allows the data driven models to respond effectively to evolving plasma behaviors and equipment conditions. Specifically, to mitigate the effects of shot-to-shot drift, our system updates itself incrementally as new data becomes available, improving both robustness and fidelity. Our vision is to evolve this data driven model into a self-sustaining digital twin system that leverages UQ based feedback to continuously refine itself and potentially support real-time decision making. This presentation will cover a brief background on uncertainty quantification for ML, our ongoing effort on development of UQ capabilities for ML, our data science pipeline from data collection to model development and analysis and online learning framework for modeling coil deflection at DIII-D. I will also briefly touch upon opportunities and challenges in development of digital twin framework.
Application of Gaussian Process Modeling to Analysis of Functional Unreliability
This paper applies Gaussian Process (GP) modeling to analysis of the functional unreliability of a “passive system.” GPs have been used widely in many ways [1]. The present application uses a GP for emulation of a system simulation code. Such an emulator can be applied in several distinct ways, discussed below. All applications illustrated in this paper have precedents in the literature; the present paper is an application of GP technology to a problem that was originally analyzed [2] using neural networks (NN), and later [3, 4] by a method called “Alternating Conditional Expectations” (ACE). This exercise enables a multifaceted comparison of both the processes and the results. Given knowledge of the range of possible values of key system variables, one could, in principle, quantify functional unreliability by sampling from their joint probability distribution, and performing a system simulation for each sample to determine whether the function succeeded for that particular setting of the variables. Using previously available system simulation codes, such an approach is generally impractical for a plant-scale problem. It has long been recognized, however, that a well-trained code emulator or surrogate could be used in a sampling process to quantify certain performance metrics, even for plant-scale problems. “Response surfaces” were used for this many years ago. But response surfaces are at their best for smoothly varying functions; in regions of parameter space where key system performance metrics may behave in complex ways, or even exhibit discontinuities, response surfaces are not the best available tool. This consideration was one of several that drove the work in [2]. In the present paper, (1) the original quantification of functional unreliability using NN [2], and later ACE [3], is reprised using GP; (2) additional information provided by the GP about uncertainty in the limit surface, generally unavailable in other representations, is discussed; (3) a simple forensic exercise is performed, analogous to the inverse problem of code calibration, but with an accident management spin: given an observation about containment pressure, what can we say about the system variables? References 1. For an introduction to GPs, see (for example) Gaussian Processes for Machine Learning, C. E. Rasmussen and C. K. I. Williams (MIT, 2006). 2. Reliability Quantification of Advanced Reactor Passive Safety Systems, J. J. Vandenkieboom, PhD Thesis (University of Michigan, 1996). 3. Z. Cui, J. C. Lee, J. J. Vandenkieboom, and R. W. Youngblood, “Unreliability Quantification of a Containment Cooling System through ACE and ANN Algorithms,” Trans. Am. Nucl. Soc. 85, 178 (2001). 4. Risk and Safety Analysis of Nuclear Systems, J. C. Lee and N. J. McCormick (Wiley, 2011). See especially §11.2.4.
Task-oriented machine learning surrogates for tipping points of agent-based models
We present a machine learning framework bridging manifold learning, neural networks, Gaussian processes, and Equation-Free multiscale approach, for the construction of different types of effective reduced order models from detailed agent-based simulators and the systematic multiscale numerical analysis of their emergent dynamics. The specific tasks of interest here include the detection of tipping points, and the uncertainty quantification of rare events near them. Our illustrative examples are an event-driven, stochastic financial market model describing the mimetic behavior of traders, and a compartmental stochastic epidemic model on an Erdös-Rényi network. We contrast the pros and cons of the different types of surrogate models and the effort involved in learning them. Importantly, the proposed framework reveals that, around the tipping points, the emergent dynamics of both benchmark examples can be effectively described by a one-dimensional stochastic differential equation, thus revealing the intrinsic dimensionality of the normal form of the specific type of the tipping point. This allows a significant reduction in the computational cost of the tasks of interest.
Toward Forecasting Geomagnetic Storms
We present a selection of attempts to infer geomagnetic storm properties from time series data. Initially, we unsuccessfully attempt to predict the duration of a geomagnetic storm based on data preceding the onset of a geomagnetic storm. Similar techniques are also used to attempt to predict the maximum depth of a geomagnetic storm. Additionally, we present a case study in Gaussian Process regression, where we are able to accurately interpolate between geomagnetic storm parameters, but are unable to predict future storm behavior. Lastly, we describe attempts at using random forest regression with sliding windows to infer geomagnetic storm behavior at future times. These results did not provide robust forecasts of geomagnetic storm behavior, but did occasionally correctly predict a storm within the margin of error. Techniques presented in this paper include machine learning classification and regression, Gaussian Process regression, and machine learning regression with sliding windows. Many of the attempts presented in this document did not result in meaningful predictions of geomagnetic storm behavior. We hope this document can be used as a guide for future readers attempting to predict geomagnetic storm properties.
Intrusive analysis for NEK5000: development of intrusive uncertainty quantification for high-dimensional, high-fidelity codes.
We study the use of lower-fidelity training data for uncertainty quantification of complex simulation models. In our approach, computationally expensive full-model outputs are approximated applying proper orthogonal-decomposition-based dimensionality reduction to the full model. A Gaussian-processes-based machine learning approach is then used to model the difference (or, rather, the correspondence) between the higher-fidelity and the lower-fidelity data. This stochastic model can be constructed by using few additional full code evaluations; it is used to calibrate arbitrarily many lower-quality outputs in order to create additional training data for sampling-based analysis. In effect, an adequate approximation of the simulation model’s response to uncertainty can be constructed at a modest computational cost. In fact, we aim at the number of full-model evaluations comparable to that required for a linear approximation (as opposed to numbers traditionally associated with sampling in high-dimensional spaces). In this report, we explain the basic algorithm, suggest some performance tuning options, and give a first characterization for the class of simulation models of nuclear engineering for which the suggested report is effective.
Towards 3rd generation organic tandem solar cells with 20% efficiency: Accelerated discovery and rational design of carbon-based photovoltaic materials through massive distributed volunteer computing
Clean, affordable, and renewable energy sources are urgently needed to satisfy the 10s of terawatts (TW) energy need of human beings. Solar cells are one promising choice to replace traditional energy sources. Our broad efforts have expanded the knowledge of possible donor materials for organic photovoltaics, while increasing access of our results to the world through the Clean Energy Project database (www.molecularspace.org). Machine learning techniques, including Gaussian Processes have been used to calibrate frontier molecular orbital energies, and OPV bulk properties (open-circuit voltage, percent conversion efficiencies, and short-circuit current). This grant allowed us to delve into the solid-state properties of OPVs (charge-carrier dynamics). One particular example allowed us to predict charge-carrier dynamics and make predictions about future hydrogen-bonded materials.
The PAU Survey: narrow-band photometric redshifts using Gaussian processes
Here, we study the performance of the hybrid template machine learning photometric redshift (photo- z ) algorithm delight , which uses Gaussian processes, on a subset of the early data release of the Physics of the Accelerating Universe Survey (PAUS). We calibrate the fluxes of the 40 PAUS narrow bands with six broad-band fluxes ( uBVriz ) in the Cosmic Evolution Survey (COSMOS) field using three different methods, including a new method that utilizes the correlation between the apparent size and overall flux of the galaxy. We use a rich set of empirically derived galaxy spectral templates as guides to train the Gaussian process, and we show that our results are competitive with other standard photometric redshift algorithms. delight achieves a photo- z 68th percentile error of σ 68 = 0.0081(1 + z ) without any quality cut for galaxies with i auto < 22.5 as compared to 0.0089(1 + z ) and 0.0202(1 + z ) for the bpz and annz 2 codes, respectively. delight is also shown to produce more accurate probability distribution functions for individual redshift estimates than bpz and annz 2. Common photo- z outliers of delight and bcnz 2 (previously applied to PAUS) are found to be primarily caused by outliers in the narrow-band fluxes, with a small number of cases potentially indicating spectroscopic redshift failures in the reference sample. In the process, we introduce performance metrics derived from the results of bcnz 2 and delight , allowing us to achieve a photo- z quality of σ 68 < 0.0035(1 + z ) at a magnitude of i auto < 22.5 while keeping 50 per cent objects of the galaxy sample.
Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes
Uncertainty propagation in complex engineering systems often poses significant computational challenges related to modeling and quantifying probability distributions of model outputs, as those emerge as the result of various sources of uncertainty that are inherent in the system under investigation. Gaussian Processes regression (GPs) is a robust meta-modeling technique that allows for fast model prediction and exploration of response surfaces. Multi-fidelity variations of GPs further leverage information from cheap and low fidelity model simulations in order to improve their predictive performance on the high fidelity model. In order to cope with the high volume of data required to train GPs in high dimensional design spaces, a common practice is to introduce latent design variables that are typically projections of the original input space to a lower dimensional subspace, and therefore substitute the problem of learning the initial high dimensional mapping, with that of training a GP on a low dimensional space. Here in this paper, we present a Bayesian approach to identify optimal transformations that map the input points to low dimensional latent variables. The \projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters, conditioned on the available training data. The proposed Bayesian inference scheme relies on a two-step iterative algorithm that samples from the marginal posteriors of the GP parameters and the projection matrix respectively, both using Markov Chain Monte Carlo (MCMC) sampling. In order to take into account the orthogonality constraints imposed on the orthonormal projection matrix, a Geodesic Monte Carlo sampling algorithm is employed, that is suitable for exploiting probability measures on manifolds. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. We validate our framework on three synthetic problems with a known lower-dimensional subspace. The benefits of our proposed framework, are illustrated on the computationally challenging aerodynamic optimization of a last-stage blade for an industrial gas turbine, where we study the effect of an 85-dimensional shape parameterization of a three-dimensional airfoil on two output quantities of interest, specifically on the aerodynamic efficiency and the degree of reaction
WearGP: A computationally efficient machine learning framework for local erosive wear predictions via nodal Gaussian processes
Computational fluid dynamics (CFD)-based wear predictions are computationally expensive to evaluate, even with a high-performance computing infrastructure. Thus, it is difficult to provide accurate local wear predictions in a timely manner. Data-driven approaches provide a more computationally efficient way to approximate the CFD wear predictions without running the actual CFD wear models. In this paper, a machine learning (ML) approach, termed WearGP, is presented to approximate the 3D local wear predictions, using numerical wear predictions from steady-state CFD simulations as training and testing datasets. The proposed framework is built on Gaussian process (GP) and utilized to predict wear in a much shorter time. The WearGP framework can be segmented into three stages. At the first stage, the training dataset is built by using a number of CFD simulations in the order of O (10 2 ). At the second stage, the data cleansing and data mining process are performed, where the nodal wear solutions are extracted from the solution database to build a training dataset. At the third stage, the wear predictions are made, using trained GP processes. Two CFD case studies including 3D slurry pump impeller and casing are used to demonstrate the WearGP framework, in which 144 training and 40 testing data points are used to train and test the proposed method, respectively. The numerical accuracy, computational effiency and effectiveness between the WearGP framework and CFD wear model for both slurry pump impellers and casings are compared. It is shown that the WearGP framework can achieve highly accurate results that are comparable with the CFD results, with a relatively small size training dataset, with a computational time reduction in an order of 10 5 to 10 6 .
High-Dimensional Intrinsic Interpolation Using Gaussian Process Regression and Diffusion Maps
This article considers the challenging task of estimating geologic properties of interest using a suite of proxy measurements. The current work recast this task as a manifold learning problem. In this process, this article introduces a novel regression procedure for intrinsic variables constrained onto a manifold embedded in an ambient space. The procedure is meant to sharpen high-dimensional interpolation by inferring non-linear correlations from the data being interpolated. The proposed approach augments manifold learning procedures with a Gaussian process regression. It first identifies, using diffusion maps, a low-dimensional manifold embedded in an ambient high-dimensional space associated with the data. It relies on the diffusion distance associated with this construction to define a distance function with which the data model is equipped. This distance metric function is then used to compute the correlation structure of a Gaussian process that describes the statistical dependence of quantities of interest in the high-dimensional ambient space. The proposed method is applicable to arbitrarily high-dimensional data sets. Here, it is applied to subsurface characterization using a suite of well log measurements. The predictions obtained in original, principal component, and diffusion space are compared using both qualitative and quantitative metrics. Considerable improvement in the prediction of the geological structural properties is observed with the proposed method.
Machine Learning Based Parallel I/O Predictive Modeling: A Case Study on Lustre File Systems
Parallel I/O hardware and software infrastructure is a key contributor to performance variability for applications running on large-scale HPC systems. This variability confounds efforts to predict application performance for characterization, modeling, optimization, and job scheduling. We propose a modeling approach that improves predictive ability by explicitly treating the variability and by leveraging the sensitivity of application parameters on performance to group applications with similar characteristics. We develop a Gaussian process-based machine learning algorithm to model I/O performance and its variability as a function of application and file system characteristics. We demonstrate the effectiveness of the proposed approach using data collected from the Edison system at the National Energy Research Scientific Computing Center. The results show that the proposed sensitivity-based models are better at prediction when compared with application-partitioned or unpartitioned models. We highlight modeling techniques that are robust to the outliers that can occur in production parallel file systems. Using the developed metrics and modeling approach, we provide insights into the file system metrics that have a significant impact on I/O performance.