Search NASA⌕ Search

SEARCH · Search NASA

Results for “robust Gaussian process regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Machine Learning Based Parallel I/O Predictive Modeling: A Case Study on Lustre File Systems

Parallel I/O hardware and software infrastructure is a key contributor to performance variability for applications running on large-scale HPC systems. This variability confounds efforts to predict application performance for characterization, modeling, optimization, and job scheduling. We propose a modeling approach that improves predictive ability by explicitly treating the variability and by leveraging the sensitivity of application parameters on performance to group applications with similar characteristics. We develop a Gaussian process-based machine learning algorithm to model I/O performance and its variability as a function of application and file system characteristics. We demonstrate the effectiveness of the proposed approach using data collected from the Edison system at the National Energy Research Scientific Computing Center. The results show that the proposed sensitivity-based models are better at prediction when compared with application-partitioned or unpartitioned models. We highlight modeling techniques that are robust to the outliers that can occur in production parallel file systems. Using the developed metrics and modeling approach, we provide insights into the file system metrics that have a significant impact on I/O performance.

I/O performance variability↗
DOE OSTI2021

Cleaning Images with Gaussian Process Regression

Many approaches to astronomical data reduction and analysis cannot tolerate missing data: corrupted pixels must first have their values imputed. This paper presents astrofix, a robust and flexible image imputation algorithm based on Gaussian process regression. Through an optimization process, astrofix chooses and applies a different interpolation kernel to each image, using a training set extracted automatically from that image. It naturally handles clusters of bad pixels and image edges and adapts to various instruments and image types. For bright pixels, the mean absolute error of astrofix is several times smaller than that of median replacement and interpolation by a Gaussian kernel. We demonstrate good performance on both imaging and spectroscopic data, including the SBIG 6303 0.4 m telescope and the FLOYDS spectrograph of Las Cumbres Observatory and the CHARIS integral-field spectrograph on the Subaru Telescope.

42 ENGINEERING↗
DOE OSTI2021

Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes

Uncertainty propagation in complex engineering systems often poses significant computational challenges related to modeling and quantifying probability distributions of model outputs, as those emerge as the result of various sources of uncertainty that are inherent in the system under investigation. Gaussian Processes regression (GPs) is a robust meta-modeling technique that allows for fast model prediction and exploration of response surfaces. Multi-fidelity variations of GPs further leverage information from cheap and low fidelity model simulations in order to improve their predictive performance on the high fidelity model. In order to cope with the high volume of data required to train GPs in high dimensional design spaces, a common practice is to introduce latent design variables that are typically projections of the original input space to a lower dimensional subspace, and therefore substitute the problem of learning the initial high dimensional mapping, with that of training a GP on a low dimensional space. Here in this paper, we present a Bayesian approach to identify optimal transformations that map the input points to low dimensional latent variables. The \projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters, conditioned on the available training data. The proposed Bayesian inference scheme relies on a two-step iterative algorithm that samples from the marginal posteriors of the GP parameters and the projection matrix respectively, both using Markov Chain Monte Carlo (MCMC) sampling. In order to take into account the orthogonality constraints imposed on the orthonormal projection matrix, a Geodesic Monte Carlo sampling algorithm is employed, that is suitable for exploiting probability measures on manifolds. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. We validate our framework on three synthetic problems with a known lower-dimensional subspace. The benefits of our proposed framework, are illustrated on the computationally challenging aerodynamic optimization of a last-stage blade for an industrial gas turbine, where we study the effect of an 85-dimensional shape parameterization of a three-dimensional airfoil on two output quantities of interest, specifically on the aerodynamic efficiency and the degree of reaction

42 ENGINEERING↗
DOE OSTI2026

Poisson Log-Normal Process for Count Data Prediction

Modeling count data is important in physics and other scientific disciplines, where measurements often involve discrete, non-negative quantities such as photon or neutrino detection events. Traditional parametric approaches can be trained to generate integer-count predictions but may struggle with capturing complex, non-linear dependencies often observed in the data. Gaussian process (GP) regression provides a robust non-parametric alternative to modeling continuous data; however, it cannot generate integer outputs. We propose the Poisson Log-Normal (PoLoN) process, a framework that employs GP to model Poisson log-rates. As in GP regression, our approach relies on the correlations between data points captured via GP kernel structure rather than explicit functional parameterizations. We demonstrate that the PoLoN predictive distribution is Poisson-LogNormal and provide an algorithm for optimizing kernel hyperparameters. Furthermore, we adapt the PoLoN approach to the problem of detecting weak localized signals superimposed on a smoothly varying background - a task of considerable interest in many areas of science and engineering. Our framework allows us to predict the strength, location and width of the detected signals. We evaluate PoLoN's performance using both synthetic and real-world datasets, including the open dataset from CERN which was used to detect the Higgs boson at the Large Hadron Collider. Our results indicate that the PoLoN process can be used as a non-parametric alternative for analyzing, predicting, and extracting signals from integer-valued data.

Saha, Anushka [Rutgers U., Piscataway]↗
DOE OSTI2024

A robust approach to Gaussian process implementation

Abstract. Gaussian process (GP) regression is a flexible modeling technique used to predict outputs and to capture uncertainty in the predictions. However, the GP regression process becomes computationally intensive when the training spatial dataset has a large number of observations. To address this challenge, we introduce a scalable GP algorithm, termed MuyGPs, which incorporates nearest-neighbor and leave-one-out cross-validation during training. This approach enables the evaluation of large spatial datasets with state-of-the-art accuracy and speed in certain spatial problems. Despite these advantages, conventional quadratic loss functions used in the MuyGPs optimization, such as root mean squared error (RMSE), are highly influenced by outliers. We explore the behavior of MuyGPs in cases involving outlying observations and, subsequently, develop a robust approach to handle and mitigate their impact. Specifically, we introduce a novel leave-one-out loss function based on the pseudo-Huber function (LOOPH) that effectively accounts for outliers in large spatial datasets within the MuyGPs framework. Our simulation study shows that the LOOPH loss method maintains accuracy despite outlying observations, establishing MuyGPs as a powerful tool for mitigating unusual observation impacts in the large data regime. In the analysis of US ozone data, MuyGPs provides accurate predictions and uncertainty quantification, demonstrating its utility in managing data anomalies. Through these efforts, we advance the understanding of GP regression in spatial contexts.

Mukangango, Juliette↗
DOE OSTI2023

Robustness of the Stochastic Parameterization of Subgrid-Scale Wind Variability in Sea Surface Fluxes

Abstract High-resolution numerical models have been used to develop statistical models of the enhancement of sea surface fluxes resulting from spatial variability of sea surface wind. In particular, studies have shown that flux enhancement is not a deterministic function of the resolved state. Previous studies focused on single geographical areas or used a single high-resolution numerical model. This study extends the development of such statistical models by considering six different high-resolution models, four different geographical regions, and three different 10-day periods, allowing for a systematic investigation of the robustness of both the deterministic and stochastic parts of the data-driven parameterization. Results indicate that the deterministic part, based on regressing the unresolved normalized flux onto resolved-scale normalized flux and precipitation, is broadly robust across different models, regions, and time periods. The statistical features of the stochastic part of the model (spatial and temporal autocorrelation and parameters of a Gaussian process fit to the regression residual) are also found to be robust and not strongly sensitive to the underlying model, modeled geographical region, or time period studied. Best-fit Gaussian process parameters display robust spatial heterogeneity across models, indicating potential for improvements to the statistical model. These results illustrate the potential for the development of a generic, explicitly stochastic parameterization of sea surface flux enhancements dependent on wind variability.

Endo, Kota↗
DOE OSTI2021

Gaussian processes for autonomous data acquisition at large-scale synchrotron and neutron facilities

The execution and analysis of complex experiments are challenged by the vast dimensionality of the underlying parameter spaces. Although an increase in data-acquisition rates should allow broader querying of the parameter space, the complexity of experiments and the subtle dependence of the model function on input parameters remains daunting owing to the sheer number of variables. New strategies for autonomous data acquisition are being developed, with one promising direction being the use of Gaussian process regression (GPR). GPR is a quick, non-parametric and robust approximation and uncertainty quantification method that can be applied directly to autonomous data acquisition. We review GPR-driven autonomous experimentation and illustrate its functionality using real-world examples from large experimental facilities in the USA and France. We introduce the basics of a GPR-driven autonomous loop with a focus on Gaussian processes, and then shift the focus to the infrastructure that needs to be built around GPR to create a closed loop. Finally, the case studies we discuss show that Gaussian-process-based autonomous data acquisition is a widely applicable method that can facilitate the optimal use of instruments and facilities by enabling the efficient acquisition of high-value datasets.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Polymers for Extreme Conditions Designed Using Syntax-Directed Variational Autoencoders

We report the design/discovery of new materials is highly nontrivial owing to the near-infinite possibilities of material candidates and multiple required property/performance objectives. Thus, machine learning tools are now commonly employed to virtually screen material candidates with desired properties by learning a theoretical mapping from material-to-property space, referred to as the forward problem. However, this approach is inefficient and severely constrained by the candidates that the human imagination can conceive. Thus, in this work on polymers, we tackle the materials discovery challenge by solving the inverse problem: directly generating candidates that satisfy desired property/performance objectives. We utilize syntax-directed variational autoencoders (VAE) in tandem with Gaussian process regression (GPR) models to discover polymers expected to be robust under three extreme conditions: (1) high temperatures, (2) high electric field, and (3) high temperature and high electric field, useful for critical structural, electrical, and energy storage applications. This approach to learn from and augment) human ingenuity is general and can be extended to discover polymers with other targeted properties and performance measures.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Advanced stationary and nonstationary kernel designs for domain-aware Gaussian processes

Gaussian process regression is a widely-applied method for function approximation and uncertainty quantification. The technique has gained popularity recently in the machine learning community due to its robustness and interpretability. The mathematical methods we discuss in this paper are an extension of the Gaussian-process framework. We are proposing advanced kernel designs that only allow for functions with certain desirable characteristics to be elements of the reproducing kernel Hilbert space (RKHS) that underlies all kernel methods and serves as the sample space for Gaussian process regression. These desirable characteristics reflect the underlying physics; two obvious examples are symmetry and periodicity constraints. In addition, non-stationary kernel designs can be defined in the same framework to yield flexible multi-task Gaussian processes. We will show the impact of advanced kernel designs on Gaussian processes using several synthetic and two scientific data sets. The results of our research show that including domain knowledge, communicated through advanced kernel designs, has a significant impact on the accuracy and relevance of the function approximation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Joint Modeling of Quasar Variability and Accretion Disk Reprocessing Using Latent Stochastic Differential Equations

Quasars are bright active galactic nuclei powered by the accretion of matter around supermassive black holes at the center of galaxies. Their stochastic brightness variability depends on the physical properties of the accretion disk and black hole. The upcoming Rubin Observatory Legacy Survey of Space and Time (LSST) is expected to observe tens of millions of quasars, so there is a need for efficient techniques like machine learning that can handle the large volume of data. Quasar variability is believed to be driven by an X-ray corona, which is reprocessed by the accretion disk and emitted as UV/optical variability. We are the first to introduce an auto-differentiable simulation of the accretion disk and reprocessing. We use the simulation as a direct component of our neural network to jointly model the driving variability and reprocessing, trained with supervised learning on simulated LSST-like 10 yr quasar light curves. We encode the light curves using a transformer encoder, and the driving variability is reconstructed using latent stochastic differential equations, a physically motivated generative deep learning method that can model continuous-time stochastic dynamics. By embedding the physical processes of the driving signal and reprocessing into our network, we achieve a model that is more robust and interpretable. We demonstrate that our model outperforms a Gaussian process regression baseline and can infer accretion disk parameters and time delays between wave bands, even for out-of-distribution driving signals. Our approach provides a powerful framework that can be adapted to solve other inverse problems in multivariate time series.

Fagin, Joshua [City Univ. of New York (CUNY), NY (↗
DOE OSTI2022

Toward Forecasting Geomagnetic Storms

We present a selection of attempts to infer geomagnetic storm properties from time series data. Initially, we unsuccessfully attempt to predict the duration of a geomagnetic storm based on data preceding the onset of a geomagnetic storm. Similar techniques are also used to attempt to predict the maximum depth of a geomagnetic storm. Additionally, we present a case study in Gaussian Process regression, where we are able to accurately interpolate between geomagnetic storm parameters, but are unable to predict future storm behavior. Lastly, we describe attempts at using random forest regression with sliding windows to infer geomagnetic storm behavior at future times. These results did not provide robust forecasts of geomagnetic storm behavior, but did occasionally correctly predict a storm within the margin of error. Techniques presented in this paper include machine learning classification and regression, Gaussian Process regression, and machine learning regression with sliding windows. Many of the attempts presented in this document did not result in meaningful predictions of geomagnetic storm behavior. We hope this document can be used as a guide for future readers attempting to predict geomagnetic storm properties.

58 GEOSCIENCES↗
DOE OSTI2024

Acceleration of Solvation Free Energy Calculation via Thermodynamic Integration Coupled with Gaussian Process Regression and Improved Gelman–Rubin Convergence Diagnostics

The determination of the solvation free energy of ions and molecules holds profound importance across a spectrum of applications spanning chemistry, biology, energy storage, and the environment. Molecular dynamics simulations are powerful tools for computing this critical parameter. Nevertheless, the accurate and efficient calculation of the solvation free energy becomes a formidable endeavor when dealing with complex systems characterized by potent Coulombic interactions and sluggish ion dynamics and, consequently, slow transition across various metastable states. Here, in the present study, we expose limitations stemming from the conventional calculation of the statistical inefficiency g in the thermodynamic integration method, a factor that can hinder the determination of convergence of the solvation free energy and its associated uncertainty. Instead, we propose a robust scheme based on Gelman–Rubin convergence diagnostics. We leverage this improved estimation of uncertainties to introduce an innovative accelerated thermodynamic integration method based on the Gaussian Process regression. This methodology is applied to the calculation of the solvation free energy of trivalent rare-earth elements immersed in ionic liquids, a scenario in which the aforementioned challenges render standard approaches ineffective. The proposed method proves to be effective in computing solvation free energy in situations where traditional thermodynamic integration methods fall short.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Quantifying experimental edge plasma evolution via multidimensional adaptive Gaussian process regression

The edge density and temperature of tokamak plasmas are strongly correlated with energy and particle confinement and their quantification is fundamental to understanding edge dynamics. These quantities exhibit behaviours ranging from sharp plasma gradients and fast transient phenomena (e.g. transitions between low and high confinement regimes) to nominal stationary phases. Analysis of experimental edge measurements therefore require robust fitting techniques to capture potentially stiff spatiotemporal evolution. Additionally, fusion plasma diagnostics inevitably involve measurement errors and data analysis requires a statistical framework to accurately quantify uncertainties. This paper outlines a generalized multidimensional adaptive Gaussian process routine capable of automatically handling noisy data and spatiotemporal correlations. We focus on the edge-pedestal region in order to underline advancements in quantifying time-dependent plasma profiles including transport barrier formation on the Alcator C-Mod tokamak.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Computationally efficient and error aware surrogate construction for numerical solutions of subsurface flow through porous media

Limiting the injection rate to restrict the pressure below a threshold at a critical location can be an important goal of simulations that model the subsurface pressure between injection and extraction wells. The pressure is approximated by the solution of Darcy’s partial differential equation for a given permeability field. The subsurface permeability is modeled as a random field since it is known only up to statistical properties. This induces uncertainty in the computed pressure. Solving the partial differential equation for an ensemble of random permeability simulations enables estimating a probability distribution for the pressure at the critical location. These simulations are computationally expensive, and practitioners often need rapid online guidance for real-time pressure management. An ensemble of numerical partial differential equation solutions is used to construct a Gaussian process regression model that can quickly predict the pressure at the critical location as a function of the extraction rate and permeability realization. The Gaussian process surrogate analyzes the ensemble of numerical pressure solutions at the critical location as noisy observations of the true pressure solution, enabling robust inference using the conditional Gaussian process distribution. Our first novel contribution is to identify a sampling methodology for the random environment and matching kernel technology for which fitting the Gaussian process regression model scales as O ( n log n ) instead of the typical O ( n 3 ) rate in the number of samples n used to fit the surrogate. The surrogate model allows almost instantaneous predictions for the pressure at the critical location as a function of the extraction rate and permeability realization. Our second contribution is a novel algorithm to calibrate the uncertainty in the surrogate model to the discrepancy between the true pressure solution of Darcy’s equation and the numerical solution. Finally, although our method is derived for building a surrogate for the solution of Darcy’s equation with a random permeability field, the framework broadly applies to solutions of other partial differential equations with random coefficients.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Optimization of Water-Alternating-CO2 Injection Field Operations Using a Machine-Learning-Assisted Workflow

Summary This paper will present a robust workflow to address multiobjective optimization (MOO) of carbon dioxide (CO2)-enhanced oil recovery (EOR)-sequestration projects with a large number of operational control parameters. Farnsworth unit (FWU) field, a mature oil reservoir undergoing CO2 alternating water injection (CO2-WAG) EOR, will be used as a field case to validate the proposed optimization protocol. The expected outcome of this work would be a repository of Pareto-optimal solutions of multiple objective functions, including oil recovery, carbon storage volume, and project economics. FWU’s numerical model is used to demonstrate the proposed optimization workflow. Because using MOO requires computationally intensive procedures, machine-learning-based proxies are introduced to substitute for the high-fidelity model, thus reducing the total computation overhead. The vector machine regression combined with the Gaussian kernel (Gaussian-SVR) is used to construct proxies. An iterative self-adjusting process prepares the training knowledge base to develop robust proxies and minimizes computational time. The proxies’ hyperparameters will be optimally designed using Bayesian optimization to achieve better generalization performance. Trained proxies will be coupled with multiobjective particle swarm Optimization (MOPSO) protocol to construct the Pareto-front solution repository. The outcomes of this workflow will be a repository containing Pareto-optimal solutions of multiple objectives considered in the CO2-WAG project. The proposed optimization workflow will be compared with another established methodology using a multilayer neural network (MLNN) to validate its feasibility in handling MOO with a large number of parameters to control. Optimization parameters used include operational variables that might be used to control the CO2-WAG process, such as the duration of the water/gas injection period, producer bottomhole pressure (BHP) control, and water injection rate of each well included in the numerical model. It is proved that the workflow coupling Gaussian-SVR proxies and the iterative self-adjusting protocol is more computationally efficient. The MOO process is made more rapid by squeezing the size of the required training knowledge base while maintaining the high accuracy of the optimized results. The outcomes of the optimization study show promising results in successfully establishing the solution repository considering multiple objective functions. Results are also verified by validating the Pareto fronts with simulation results using obtained optimized control parameters. The outcome from this work could provide field operators an opportunity to design a CO2-WAG project using as many inputs as possible from the reservoir models. The proposed work introduces a novel concept that couples Gaussian-SVR proxies with a self-adjusting protocol to increase the computational efficiency of the proposed workflow and to guarantee the high accuracy of the obtained optimized results. More importantly, the workflow can optimize a large number of control parameters used in a complex CO2-WAG process, which greatly extends its utility in solving large-scale MOO problems in various projects with similar desired outcomes.

Energy & Fuels↗
DOE OSTI2022

Single Gaussian process method for arbitrary tokamak regimes with a statistical analysis

Abstract Gaussian process regression is a Bayesian method for inferring profiles based on input data. The technique is increasing in popularity in the fusion community due to its many advantages over traditional fitting techniques including intrinsic uncertainty quantification and robustness to over-fitting. This work investigates the use of a new method, the change-point method, for handling the varying length scales found in different tokamak regimes. The use of the Student’s t-distribution for the Bayesian likelihood probability is also investigated and shown to be advantageous in providing good fits in profiles with many outliers. To compare different methods, synthetic data generated from analytic profiles is used to create a database enabling a quantitative statistical comparison of which methods perform the best. Using a full Bayesian approach with the change-point method, Matérn kernel for the prior probability, and Student’s t-distribution for the likelihood is shown to give the best results.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Latent-space time evolution of non-intrusive reduced-order models using Gaussian process emulation

Non-intrusive reduced-order models (ROMs) have recently generated considerable interest for constructing computationally efficient counterparts of nonlinear dynamical systems emerging from various domain sciences. They provide a low-dimensional emulation framework for systems that may be intrinsically high-dimensional. This is accomplished by utilizing a construction algorithm that is purely data-driven. It is no surprise, therefore, that the algorithmic advances of machine learning have led to non-intrusive ROMs with greater accuracy and computational gains. However, in bypassing the utilization of an equation-based evolution, it is often seen that the interpretability of the ROM framework suffers. This becomes more problematic when black-box deep learning methods are used which are notorious for lacking robustness outside the physical regime of the observed data. In this article, we propose the use of a novel latent-space interpolation algorithm based on Gaussian process regression. Notably, this reduced-order evolution of the system is parameterized by control parameters to allow for interpolation in space. The use of this procedure also allows for a continuous interpretation of time which allows for temporal interpolation. The latter aspect provides information, with quantified uncertainty, about full-state evolution at a finer resolution than that utilized for training the ROMs. This research assesses the viability of this algorithm for an advection-dominated system given by the inviscid shallow water equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

A sparse Gaussian sigmoid basis function approximation of hyperspectral data for detection of solids

We define a new characterization of emissivity and reflectance curves for compositional exploitation of hyperspectral data. Here, our method decomposes each spectrum into a sparse set of Gaussian sigmoid components using penalized regression. Detection is based on the combination of Gaussian sigmoid components unique to a target material. Focusing on the presence of spectral upslopes and downslopes rather than spectral correlations makes detection more robust to both target variation and spectral variability from atmosphere and background encountered during the collection process. We present simulation studies that demonstrate the potential to reduce false positive rates without compromising sensitivity. Characterization of long–wave infrared (LWIR) experimental data validates our method using minerals of different particle sizes, measurement angles, and collection conditions.

97 MATHEMATICS AND COMPUTING↗