Search NASA⌕ Search

SEARCH · Search NASA

Results for “Deep gaussian processes”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Divide and conquer: separating the two probabilities in seismic phase picking

There are two fundamental probabilities in the seismic phase picking process—the probability of the existence of a seismic phase (detection probability) and the probability associated with the phase arrival time estimation (timing probability). The nearly ubiquitous approach in developing deep learning phase picking models is to use a kernel, such as a truncated Gaussian, to mask the labelled phase arrival time and train a segmentation model. Once a model is trained, the times of the peaks in the output are taken as phase arrival times (picks), and the height of the peaks are taken as ‘probability’ of the picks. Here, we show that this ‘probability’ represents neither the detection nor the timing probability because this approach forces the output to follow the shape of the kernel. We introduce an approach using two models to estimate these two distinct probabilities. We use a binary classifier with a calibrated confidence to address the detection probability and a multiclass classifier to obtain a probability mass function to address the timing probability. This new approach can make the deep learning-based phase picking process more interpretable and provide options to logically control seismic monitoring workflows.

58 GEOSCIENCES↗
DOE OSTI2021

Correspondence of NNGP Kernel and the Matérn Kernel

Kernels representing limiting cases of neural network architectures have recently gained popularity. However, the application and performance of these new kernels compared to existing options, such as the Matérn kernel, is not well studied. We take a practical approach to explore the neural network Gaussian process (NNGP) kernel and its application to data in Gaussian process regression. We first demonstrate the necessity of normalization to produce valid NNGP kernels and explore related numerical challenges. We further demonstrate that the predictions from this model are quite inflexible, and therefore do not vary much over the valid hyperparameter sets. We then demonstrate a surprising result that the predictions given from the NNGP kernel correspond closely to those given by the Matérn kernel under specific circumstances, which suggests a deep similarity between overparameterized deep neural networks and the Matérn kernel. Finally, we demonstrate the performance of the NNGP kernel as compared to the Matérn kernel on three benchmark data cases, and we conclude that for its flexibility and practical performance, the Matérn kernel is preferred to the novel NNGP in practical applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

ProbabilisticRecalibration of Forecasts

In this work, we present a scheme by which a probabilistic forecasting system whose predictions have a poor probabilistic calibration may be recalibrated through the incorporation of past performance information in order to produce a new forecasting system that is demonstrably superior to the original, inasmuch as one may use it to win wagers consistently against someone who is using the original system. The scheme utilizes Gaussian process (GP) modeling to estimate a probability distribution over the probability integral transform (PIT) of a scalar predictand. The GP density estimate gives closed-form access to information entropy measures that are associated with the estimated distribution, which allows the prediction of winnings in wagers against the base forecasting system. A separate consequence of the procedure is that the recalibrated forecast has a uniform expected PIT distribution. One distinguishing feature of the procedure is that it is appropriate even if the PIT values are not i.i.d. The recalibration scheme is formulated in a framework that exploits the deep connections among information theory, forecasting, and betting. We demonstrate the effectiveness of the scheme in two case studies: a laboratory experiment with a nonlinear circuit and seasonal forecasts of the intensity of the El Nino-Southern Oscillation phenomenon.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Coarse-Grained Density Functional Theory Predictions via Deep Kernel Learning

Scalable electronic predictions are critical for soft materials design. Recently, the Electronic Coarse-Graining (ECG) method was introduced to renormalize all-atom quantum chemical (QC) predictions to coarse-grained (CG) resolutions using deep neural networks (DNNs). While DNNs can learn complex representations that prove challenging for kernel-based methods, they are susceptible to overfitting and the overconfidence of uncertainty estimations. Here, we develop ECG within a GPU-accelerated Deep Kernel Learning (DKL) framework to enable CG QC predictions using range-separated hybrid density functional theory (DFT), obtaining a 107 speedup relative to naive all-atom QC. By treating the predicted electronic properties as random Gaussian Processes, DKL incorporates CG mapping degeneracy by learning the distribution of electronic energies as a function of CG configuration. DKL-ECG accurately reproduces molecular orbital energies from range-separated DFT while facilitating efficient training via active learning using the uncertainties provided by DKL. Further, we show that while active learning algorithms enable efficient sampling of a more diverse configurational space relative to random sampling, all explored query methods exhibit comparable performance for the examined system. We attribute this result to the significant overlap of the feature space and output property distributions across multiple temperatures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Correspondence between neuroevolution and gradient descent

Abstract We show analytically that training a neural network by conditioned stochastic mutation or neuroevolution of its weights is equivalent, in the limit of small mutations, to gradient descent on the loss function in the presence of Gaussian white noise. Averaged over independent realizations of the learning process, neuroevolution is equivalent to gradient descent on the loss function. We use numerical simulation to show that this correspondence can be observed for finite mutations, for shallow and deep neural networks. Our results provide a connection between two families of neural-network training methods that are usually considered to be fundamentally different.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning

We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor-critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection-diffusion field. As a result, we provide explanation for the policy behaviors using knowledge of the underlying physical process.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Joint Modeling of Quasar Variability and Accretion Disk Reprocessing Using Latent Stochastic Differential Equations

Quasars are bright active galactic nuclei powered by the accretion of matter around supermassive black holes at the center of galaxies. Their stochastic brightness variability depends on the physical properties of the accretion disk and black hole. The upcoming Rubin Observatory Legacy Survey of Space and Time (LSST) is expected to observe tens of millions of quasars, so there is a need for efficient techniques like machine learning that can handle the large volume of data. Quasar variability is believed to be driven by an X-ray corona, which is reprocessed by the accretion disk and emitted as UV/optical variability. We are the first to introduce an auto-differentiable simulation of the accretion disk and reprocessing. We use the simulation as a direct component of our neural network to jointly model the driving variability and reprocessing, trained with supervised learning on simulated LSST-like 10 yr quasar light curves. We encode the light curves using a transformer encoder, and the driving variability is reconstructed using latent stochastic differential equations, a physically motivated generative deep learning method that can model continuous-time stochastic dynamics. By embedding the physical processes of the driving signal and reprocessing into our network, we achieve a model that is more robust and interpretable. We demonstrate that our model outperforms a Gaussian process regression baseline and can infer accretion disk parameters and time delays between wave bands, even for out-of-distribution driving signals. Our approach provides a powerful framework that can be adapted to solve other inverse problems in multivariate time series.

Fagin, Joshua [City Univ. of New York (CUNY), NY (↗
DOE OSTI2023

MACHINE LEARNING-ENABLED PREDICTION OF TRANSIENT INJECTION MAP IN AUTOMOTIVE INJECTORS WITH UNCERTAINTY QUANTIFICATION

Accurate prediction of injection profiles is a critical aspect of linking injector operation with engine performance and emissions. However, highly resolved injector simulations can take one to two weeks of wall-clock time, which is incompatible with engine design cycles with desired turnaround times of less than a day. Hence, it is important to reduce the time-to-solution of the internal flow simulations by several orders of magnitude to make it compatible with engine simulations. This work demonstrates a data-driven approach for tackling the computational overhead of injector simulations, whereby the transient injection profiles are emulated for a side-oriented, single-hole diesel injector using a Bayesian machine-learning framework. First, an interpretable Bayesian learning strategy was employed to understand the effect of design parameters on the total void fraction field. Then, autoencoders are utilized for efficient dimensionality reduction of the flowfields. Gaussian process models are finally used to predict the spatiotemporal void fraction field at the injector exit for unknown operating conditions. The Gaussian process models produce principled uncertainty estimates associated with the emulated flowfields, which provide the engine designer with valuable information of where the data-driven predictions can be trusted in the design space. The Bayesian flowfield predictions are compared with the corresponding predictions from a deep neural network, which has been transfer-learned from static needle simulations from a previous work by the authors. The emulation framework can predict the void fraction field at the exit of the orifice within a few seconds, thus achieving a speed-up factor of up to 38 x 10(6) over the traditional simulation-based approach of generating transient injection maps.

machine learning↗
DOE OSTI2025

Exploring the energy landscape of RBMs: reciprocal space insights into bosons, hierarchical learning and symmetry breaking

Deep generative models have become ubiquitous due to their ability to learn and sample from complex distributions. Despite the proliferation of various frameworks, the relationships among these models remain largely unexplored, a gap that hinders the development of a unified theory of AI learning. In this work, we address two central challenges: clarifying the connections between different deep generative models and deepening our understanding of their learning mechanisms. We focus on Restricted Boltzmann Machines (RBMs), a class of generative models known for their universal approximation capabilities for discrete distributions. By introducing a reciprocal space formulation for RBMs, we reveal a connection between these models, diffusion processes, and systems of coupled bosons. Our analysis shows that at initialization, the RBM operates at a saddle point, where the local curvature is determined by the singular values of the weight matrix, whose distribution follows the Marc̆enko-Pastur law and exhibits rotational symmetry. During training, this rotational symmetry is broken due to hierarchical learning, where different degrees of freedom progressively capture features at multiple levels of abstraction. This leads to a symmetry breaking in the energy landscape, reminiscent of Landau’s theory. This symmetry breaking in the energy landscape is characterized by the singular values and the weight matrix eigenvector matrix. We derive the corresponding free energy in a mean-field approximation. We show that in the limit of infinite size RBM, the reciprocal variables are Gaussian distributed. Our findings indicate that in this regime, there will be some modes for which the diffusion process will not converge to the Boltzmann distribution. To illustrate our results, we trained replicas of RBMs with different hidden layer sizes using the MNIST dataset. Our findings not only bridge the gap between disparate generative frameworks but also shed light on the fundamental processes underpinning learning in deep generative models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Machine learning for seismic low-frequency extrapolation

The cycle-skipping problem that plagues full waveform inversion (FWI) can be at least partially mitigated if low frequencies (which encode the kinematics of wave propagation in seismic data) are recorded. However, seismic sources and receivers are band-limited, so seismic data does not generally include signals down to 0 Hz. To improve our ability to solve the seismic inverse problem, one can synthesize this missing low-frequency (LF) content from the recorded high-frequency (HF) data using machine learning (ML) models. Deep learning models such as convolutional neural networks (CNNs) demonstrate impressive ability to perform low frequency extrapolation. However, such models require powerful hardware (GPU machines) and careful training. We assess the extrapolation capabilities of three different ML models that do not require GPU machines, namely, random forest, Gaussian process regression and gradient boosting, on both synthetic and real data. Experimental results on two synthetic data sets (generated from a low velocity lens embedded in a homogeneous medium, and the Marmousi model) demonstrate that FWI applied to the extrapolated data consistently improves inversion accuracy relative to FWI applied to the original data sets that do not contain low frequencies. Application of low-frequency extrapolation to real data from the Northwest Shelf of Australia demonstrates that tree-based ML models such as gradient boosting can outperform CNNs in terms of both accuracy and computational cost on non-GPU architectures.

58 GEOSCIENCES↗
DOE OSTI2021

A modular methodology for time-domain stochastic seismic wave propagation

Presented here is a modular methodology for time-domain stochastic seismic wave propagation analysis. Presented methodology is designed to analyse uncertain seismic motions as an input, propagating through uncertain material. Traditional approach for uncertain wave propagation relies on models that include deep bedrock, local soil site, and their random process and random field information. Such models can become quite large and computationally intractable. The modular approach proposed herein features two step approach that allows separate consideration of the deep bedrock and local site along with corresponding random field information. In this work, the first step considers an auxiliary stochastic motions problem in the bedrock. Stochastic local site response can then be simulated in a reduced domain within certain depth from the surface. Application of uncertain seismic motions at depth, for local uncertain site response is done using stochastic effective forces developed through the Domain Reduction Method. By using Hermite polynomial chaos expansion to represent the non-Gaussian random field of material parameters and non-stationary random process of seismic motion, the proposed modular methodology is formulated using intrusive stochastic Galerkin approach, as seen in the Stochastic Elastic–Plastic Finite Element Method (SEPFEM). Developed modular methodology is illustrated using a 1-D stochastic seismic wave propagation analysis with three cases, and simulation results are also verified with results from conventional approach.

58 GEOSCIENCES↗
DOE OSTI2021

TEMImageNet training library and AtomSegNet deep-learning models for high-precision atom segmentation, localization, denoising, and deblurring of atomic-resolution images

Abstract Atom segmentation and localization, noise reduction and deblurring of atomic-resolution scanning transmission electron microscopy (STEM) images with high precision and robustness is a challenging task. Although several conventional algorithms, such has thresholding, edge detection and clustering, can achieve reasonable performance in some predefined sceneries, they tend to fail when interferences from the background are strong and unpredictable. Particularly, for atomic-resolution STEM images, so far there is no well-established algorithm that is robust enough to segment or detect all atomic columns when there is large thickness variation in a recorded image. Herein, we report the development of a training library and a deep learning method that can perform robust and precise atom segmentation, localization, denoising, and super-resolution processing of experimental images. Despite using simulated images as training datasets, the deep-learning model can self-adapt to experimental STEM images and shows outstanding performance in atom detection and localization in challenging contrast conditions and the precision consistently outperforms the state-of-the-art two-dimensional Gaussian fit method. Taking a step further, we have deployed our deep-learning models to a desktop app with a graphical user interface and the app is free and open-source. We have also built a TEM ImageNet project website for easy browsing and downloading of the training data.

25 ENERGY STORAGE↗
DOE OSTI2023

Adaptive Deep Reinforcement Learning Algorithm for Distribution System Cyber Attack Defense With High Penetration of DERs

With grid modernization, smart inverters are increasingly used to execute advanced controls for distribution network reliability. However, this also increases the cyber-attack space. Here this paper focuses on the defense approaches to restore the system to normal operation circumstances in the presence of cyber-attacks. A unique deep reinforcement learning (DRL) method is developed to minimize voltage violations and reduce power losses for impacted feeders. The defense problem is reformulated as a Markov decision-making process to dynamically control DERs while minimizing load shedding. This is achieved via an improved soft actor-critic (SAC)-based DRL algorithm, which can govern DER set points and load-shedding scenarios in discrete and continuous modes via the auto-tune entropy and Gaussian policy features. Numerical comparison results on the modified IEEE 123-node system with other control approaches, such as Volt-VAR (VV), Volt-Watt (VW), and model predictive control (MPC) show that the proposed method can eliminate voltage violations and provide feasible control actions that perform complete mitigation of cyber-threats.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Deep learning categorization of infrasound array data

Here we develop a deep learning-based infrasonic detection and categorization methodology that uses convolutional neural networks with self-attention layers to identify stationary and non-stationary signals in infrasound array processing results. Using features extracted from the coherence and direction-of-arrival information from beamforming at different infrasound arrays, our model more reliably detects signals compared with raw waveform data. Using three infrasound stations maintained as part of the International Monitoring System, we construct an analyst-reviewed data set for model training and evaluation. We construct models using a 4-category framework, a generalized noise vs non-noise detection scheme, and a signal-of-interest (SOI) categorization framework that merges short duration stationary and non-stationary categories into a single SOI category. We evaluate these models using a combination of k-fold cross-validation, comparison with an existing “state-of-the-art” detector, and a transportability analysis. Although results are mixed in distinguishing stationary and non-stationary short duration signals, f-scores for the noise vs non-noise and SOI analyses are consistently above 0.96, implying that deep learning-based infrasonic categorization is a highly accurate means of identifying signals-of-interest in infrasonic data records.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

The Nonradiative Properties of Self–Trapped Holes in Ultra–Wide Bandgap Gallium Oxide Film

The photoluminescence (PL) of self-trapped holes (STH) in ultra-wide bandgap β-Ga 2 O 3 is commonly its most dominant light emission and is an inherent property. Thus, gaining knowledge of the crystal dynamics that impact the PL properties is vital to sensor and other technologies. The PL, Raman-phonons, and their interactions are studied at an extreme temperature range of 77–622 K. The PL is studied up to the bandgap value of ≈5 eV. It is found that the high-energy Raman modes provide a major route to the nonradiative process of the PL via STH–phonon interaction with an activation energy of 72 meV. This dynamic is modeled with the configurational coordinate scheme at the strong phonon coupling limit. The exceptionally broad Gaussian PL linewidth manifests this coupling. The weak temperature response of the PL energy peak position indicates that the STH has characteristics of a deep-level defect. This contrasts with the large redshift of ≈220 meV of the optical gap of the film, ascertained from transmission. Unlike the temperature response of the high-energy phonons, the behavior of the low-energy phonons is found to follow the Bose–Einstein population increase, indicating no strong interaction with the STH.

Raman↗
DOE OSTI2025

MOOSE ProbML: Parallelized probabilistic machine learning and uncertainty quantification for computational energy applications

Here, this paper presents the development and demonstration of massively parallel probabilistic machine learning (ML) and uncertainty quantification (UQ) capabilities within the Multiphysics Object-Oriented Simulation Environment (MOOSE), an open-source computational platform for parallel finite element and finite volume analyses. In addressing the computational expense and uncertainties inherent in complex multiphysics simulations, this paper integrates Gaussian process (GP) variants, active learning, Bayesian inverse UQ, adaptive forward UQ, Bayesian optimization, evolutionary optimization, and Markov chain Monte Carlo (MCMC) within MOOSE. It also elaborates on the interaction among key MOOSE systems — Sampler, MultiApp, Reporter, and Surrogate — in enabling these capabilities. The modularity offered by these systems enables development of a multitude of probabilistic ML and UQ algorithms in MOOSE. Example code demonstrations include parallel active learning and parallel Bayesian inference via active learning. The impact of these developments is illustrated through five applications relevant to computational energy applications: UQ of nuclear fuel fission product release, using parallel active learning Bayesian inference; very rare events analysis in nuclear microreactors using active learning; advanced manufacturing process modeling using multi-output GPs (MOGPs) and dimensionality reduction; fluid flow using deep GPs (DGPs); and tritium transport model parameter optimization for fusion energy, using batch Bayesian optimization. These capabilities are part of the MOOSE framework.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Modeling I/O Performance Variability Using Conditional Variational Auto Encoders

Storage system performance modeling is crucial for efficient use of heterogeneous shared resources on leadership class computers. Variability in application performance, particularly variability arising from concurrent applications sharing I/O resources, is a major hurdle in the development of accurate performance models. We adopt a deep learning approach based on conditional variational auto encoders (CVAE) for I/O performance modeling, and use it to quantify performance variability. We illustrate our approach using the data collected on Edison, a production supercomputing system at the National Energy Research Scientific Computing Center (NERSC). The CVAE approach is investigated by comparing it to a previously proposed sensitivity-based Gaussian process (GP) model. We find that the CVAE model performs slightly better than the GP model in cases where training and testing data come from different applications, since CVAE can inherently leverage the whole data from multiple applications whereas GP partitions the data and builds separate models for each partition. Hence, the CVAE offers an alternative modeling approach that does not need preprocessing; it has enough flexibility to handle data from a wide variety of applications without changing the inference approach.

I/O perfornance variability↗
DOE OSTI2016

The K2-ESPRINT project III: A close-in super-earth around a metal-rich mid-M dwarf

We validate a R{sub p}=2.32±0.24R{sub ⊕} planet on a close-in orbit (P = 2.260455 ± 0.000041 days) around K2-28 (EPIC 206318379), a metal-rich M4-type dwarf in the Campaign 3 field of the K2 mission. Our follow-up observations included multi-band transit observations from the optical to the near-infrared, low-resolution spectroscopy, and high-resolution adaptive optics (AO) imaging. We perform a global fit to all of the observed transits using a Gaussian process-based method and show that the transit depths in all of the passbands adopted for the ground-based transit follow-ups (r{sub 2}{sup ′},z{sub s,2},J,H,K{sub s}) are within ∼2σ of the K2 value. Based on a model of the background stellar population and the absence of nearby sources in our AO imaging, we estimate the probability that a background eclipsing binary could cause a false positive to be <2 × 10{sup −5}. We also show that K2-28 cannot have a physically associated companion of stellar type later than M4, based on the measurement of almost identical transit depths in multiple passbands. There is a low probability for an M4 dwarf companion (≈0.072{sub −0.04}{sup +0.02}), but even if this were the case, the size of K2-28b falls within the planetary regime. K2-28b has the same radius (within 1σ) and experiences irradiation from its host star similar to the well-studied GJ 1214b. Given the relative brightness of K2-28 in the near-infrared (m{sub Kep}=14.85 mag and m{sub H} = 11.03 mag) and relatively deep transit (0.6%–0.7%), a comparison between the atmospheric properties of these two planets with future observations would be especially interesting.

79 ASTRONOMY AND ASTROPHYSICS↗