Search NASA⌕ Search

SEARCH · Search NASA

Results for “Bayesian Neural Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Wide neural networks with bottlenecks are deep Gaussian processes

There has recently been much work on the "wide limit" of neural networks, where Bayesian neural networks (BNNs) are shown to converge to a Gaussian process (GP) as all hidden layers are sent to infinite width. However, these results do not apply to architectures that require one or more of the hidden layers to remain narrow. In this paper, we consider the wide limit of BNNs where some hidden layers, called "bottlenecks", are held at finite width. The result is a composition of GPs that we term a "bottleneck neural network Gaussian process" (bottleneck NNGP). Although intuitive, the subtlety of the proof is in showing that the wide limit of a composition of networks is in fact the composition of the limiting GPs. We also analyze theoretically a single-bottleneck NNGP, finding that the bottleneck induces dependence between the outputs of a multi-output network that persists through extreme post-bottleneck depths, and prevents the kernel of the network from losing discriminative power at extreme post-bottleneck depths.

97 MATHEMATICS AND COMPUTING↗

Crustal Scale Travel Time Prediction with the SALSA3D Framework and Machine Learning

The SALSA3D project aims to improve our models used in travel time prediction. The current version uses tomographic modeling for propagation through the Earth’s mantle because of the large number (order of millions) of observations of seismic phases which primarily traverse the Earth’s mantle and the ability to pose the travel time problem as a set of linear equations. However, all seismic rays traverse the crust to reach receivers at the surface and therefore models of propagation through the crust are required. Therefore, the primary motivation for this study is to explore how to increase the scope of the SALSA3D project to phases which travel primarily through the crust. In this report, we evaluate new, machine learning based and physics-based methods to model these travel times for integration into the SALSA3D framework. Our results suggest that using our existing physics-based travel time tomography method is a viable approach for the regional to global scale, but better predictive capabilities can be achieved through a neural network trained on the region of interest for near-regional offsets. We suggest future iterations of SALSA3D should incorporate machine learning tools such as Physics-Informed Neural Networks or Bayesian Neural Networks.

58 GEOSCIENCES↗

Hierarchical Inference of the Lensing Convergence from Photometric Catalogs with Bayesian Graph Neural Networks

Abstract We present a Bayesian graph neural network (BGNN) that can estimate the weak lensing convergence ( κ ) from photometric measurements of galaxies along a given line of sight (LOS). The method is of particular interest in strong gravitational time-delay cosmography (TDC), where characterizing the “external convergence” ( κ ext ) from the lens environment and LOS is necessary for precise Hubble constant ( H 0 ) inference. Starting from a large-scale simulation with a κ resolution of ∼1′, we introduce fluctuations on galaxy–galaxy lensing scales of ∼1″ and extract random sight lines to train our BGNN. We then evaluate the model on test sets with varying degrees of overlap with the training distribution. For each test set of 1000 sight lines, the BGNN infers the individual κ posteriors, which we combine in a hierarchical Bayesian model to yield constraints on the hyperparameters governing the population. For a test field well sampled by the training set, the BGNN recovers the population mean of κ precisely and without bias (within the 2 σ credible interval), resulting in a contribution to the H 0 error budget well under 1%. In the tails of the training set with sparse samples, the BGNN, which can ingest all available information about each sight line, extracts a stronger κ signal compared to a simplified version of the traditional method based on matching galaxy number counts, which is limited by sample variance. Our hierarchical inference pipeline using BGNNs promises to improve the κ ext characterization for precision TDC. The code is available as a public Python package, Node to Joy ⏬ .

79 ASTRONOMY AND ASTROPHYSICS↗

Modeling inclusive electron-nucleus scattering with Bayesian artificial neural networks

We introduce a Bayesian protocol based on artificial neural networks that is suitable for modeling inclusive electron-nucleus scattering on a variety of nuclear targets with quantified uncertainties. Unlike previous applications in the field, which directly parameterize the cross sections, our approach employs artificial neural networks to represent the longitudinal and transverse response functions. In contrast to cross sections, which depend on the incoming energy, scattering angle, and energy transfer, the response functions are determined solely by the energy and momentum transfer to the system, allowing the angular component to be treated analytically. We assess the accuracy and predictive power of our framework against the extensive data in the quasielastic inclusive electron-scattering database. Additionally, we present novel extractions of the longitudinal and transverse response functions and compare them with previous experimental analysis and nuclear ab-initio calculations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Physics-based hybrid machine learning for critical heat flux prediction with uncertainty quantification

Critical heat flux (CHF) is a key quantity in nuclear system modeling due to its impact on heat transfer, safety margins, and reactor performance. This study develops and validates an uncertainty-aware hybrid modeling approach that combines machine learning with physics-based models to predict CHF in cases of dryout. The Biasi and Bowring empirical correlations were paired with three ML uncertainty quantification (UQ) techniques: deep neural network (DNN) ensembles, Bayesian neural networks (BNNs), and deep Gaussian processes (DGPs). A pure ML model without a base model was evaluated for comparison. Model performance was assessed under plentiful (7,350 points) and limited (9 points) training data scenarios using parity, uncertainty distributions, and calibration curves. Results show that the Biasi hybrid DNN ensemble achieved the best overall performance, with a mean absolute relative error of 1.846%, and well-calibrated uncertainty estimates. The BNN-based hybrids showed slightly higher error (2.14%) but superior uncertainty calibration. DGP models underperformed, with over 6% error and poor uncertainty calibration. All hybrid models outperformed pure machine learning configurations, demonstrating resistance against data scarcity. These findings indicate that hybrid modeling significantly improves predictive accuracy, interpretability, and resilience to data scarcity. The integration of uncertainty awareness provides actionable confidence in CHF predictions, which is vital for safety-critical decisions in nuclear applications. This hybrid approach offers a viable pathway for deploying ML models in reactor analysis tools while preserving domain knowledge and physical consistency.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Bayesian Entropy Neural Networks for physics-aware prediction

This article addresses the need for deep learning models to integrate well-defined constraints into their outputs, driven by their application in surrogate models, learning with limited data and partial information, and scenarios requiring flexible model behavior to incorporate non-data sample information. We introduce Bayesian Entropy Neural Networks (BENN), a framework grounded in Maximum Entropy (MaxEnt) principles, designed to impose constraints on Bayesian Neural Network (BNN) predictions. BENN is capable of constraining not only the predicted values but also their derivatives and variances, ensuring a more robust and reliable model output. To achieve simultaneous uncertainty quantification and constraint satisfaction, we employ the method of multipliers approach. This allows for the concurrent estimation of neural network parameters and the Lagrangian multipliers associated with the constraints. Our experiments, spanning diverse applications such as beam deflection modeling and microstructure generation, demonstrate the effectiveness of BENN. The results highlight significant improvements over traditional BNNs and showcase competitive performance relative to contemporary constrained deep learning methods.

14 SOLAR ENERGY↗

Bayesian tensorized neural networks with automatic rank selection

Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In order to address this challenge, this paper proposes a low-rank Bayesian tensorized neural network. Our Bayesian method performs automatic model compression via an adaptive tensor rank determination. We also present approaches for posterior density calculation and maximum a posteriori (MAP) estimation for the end-to-end training of our tensorized neural network. Here, we provide experimental validation on a two-layer fully connected neural network, a 6-layer CNN and a 110-layer residual neural network where our work produces 7.4x to 137x more compact neural networks directly from the training while achieving high prediction accuracy.

Low-rank tensor↗

Computer-aided detection using non-convolutional neural network Gaussian processes

Deep convolutional neural networks (CNNs) have in recent years achieved record-breaking performance on many image classification tasks and are therefore well-suited for computer aided detection (CAD). The need for uncertainty quantification for CAD motivates the need for a probabilistic framework for deep learning. The most well-known probabilistic neural network model is the Bayesian neural network (BNN), but BNNs are notoriously difficult to sample for large complex network architectures, and as such their use is restricted to small problems. It is known that the limit of BNNs as their widths increase toward infinity is a Gaussian process (GP), and there has been considerable research interest in these infinitely wide BNNs. Recently, this classic result has been extended to deep architectures in what is termed the neural network Gaussian process (NNGP) model. In this work, we implement an NNGP model and apply it to the ChestXRay14 dataset at the full resolution of 1024x1024 pixels. Even without any convolutional aspects to the network architecture and without any data augmentation, our five layer deep NNGP model outperforms other non-convolutional models and therefore helps to narrow the performance gap between non-convolutional and convolutional models. Our NNGP model is fully Bayesian and therefore offers uncertainty information through its predictive variance that can be used to formulate a predictive confidence measure. We show that the performance of the NNGP model is significantly boosted after low-confidence predictions are rejected, suggesting that convolution is most beneficial only for these low-confidence examples. Finally, our results indicate that an extremely large fully-connected neural network with appropriate regularization could perform as well as the NNGP if not for the computational bottleneck resulting from the large number of model parameters.

Agrawal, Devanshu↗

Python Codebase and Jupyter Notebooks - Applications of Machine Learning Techniques to Geothermal Play Fairway Analysis in the Great Basin Region, Nevada

Git archive containing Python modules and resources used to generate machine-learning models used in the "Applications of Machine Learning Techniques to Geothermal Play Fairway Analysis in the Great Basin Region, Nevada" project. This software is licensed as free to use, modify, and distribute with attribution. Full license details are included within the archive. See "documentation.zip" for setup instructions and file trees annotated with module descriptions.

Brown, Stephen↗

A neural-network-enhanced parameter-varying framework for multi-objective model predictive control applied to buildings

Management of the electrical grid is becoming more complex due to the increased penetration of alternative energy generation technologies and a broadening diversity of electric loads. This complexity creates challenges in balancing demand and generation that can increase the potential for grid instabilities. One effective way to address this issue is to leverage previously unexploited demand flexibility through advanced control strategies. In this work, we propose an advanced control method, called adaptive neural parameter-varying model predictive control (ANPV-MPC), to control the temperature and energy consumption of a building via its Heating, Ventilation, and Air Conditioning system. ANPV-MPC combines key ideas in parameter-varying control, adaptive control, and online learning strategies to bridge the gap between computationally efficient linear model predictive control and more accurate nonlinear model predictive control. The novelty in ANPV-MPC is the use of a physics-inspired Bayesian neural network to estimate the coefficients of the parameter-varying linear control model. The Bayesian neural network additionally provides uncertainty estimates, triggering online training to capture evolving building system conditions. We show that ANPV-MPC can approximate the building system dynamics with a 28.39% higher accuracy than traditional linear model predictive control, resulting in 36.23% better control performance without increasing complexity of the optimal control problem. ANPV-MPC also adapts in real time to previously unseen conditions using online learning, further improving its performance.

24 POWER TRANSMISSION AND DISTRIBUTION↗

eeyore

The software "eeyore" solves the problem of running Markov chain Monte Carlo (MCMC) algorithms on Bayesian neural networks. There is software for MCMC algorithms and software for neural network (such as PyTorch and TensorFlow), but there are limited solutions for combining the two. The main idea is to create a subclass of the 'torch.nn.Module' for defining Bayesian neural networks with the desired functionality required by MCMC taking the form of additional methods in that sub-class. This way, neural network functionality of PyTorch is exposed to the developer and user, allowing to develop MCMC methods on top of it through a simple API. The code implements Langevin Monte Carlo algorithms and population MCMC algorithms, as well as combinations of these, for Bayesian neural networks. This kind of functionality is not available elsewhere, to the best of the authors' knowledge. Moreover, a simple API is available that allows to implement new MCMC algorithms in the future.

Papamarkou, Theodore↗

Uncertainty quantification and propagation in lithium-ion battery electrodes using bayesian convolutional neural networks

The complex nature of manufacturing processes stipulates electrodes to possess high variability with increased heterogeneity during production. X-ray computed tomography imaging has proved to be critical in visualizing the complicated stochastic particle distribution of as-manufactured electrodes in lithium-ion batteries. However, accurate prediction of their electrochemical performance necessitates precise evaluation of kinetic and transport properties from real electrodes. Image segmentation that characterizes voxels to particle/pore phase is often meticulous and fraught with subjectivity owing to a myriad of unconstrained choices and filter algorithms. Here we utilize a Bayesian convolutional neural network to tackle segmentation subjectivity and quantify its pertinent uncertainties. Otsu inter-variance and Blind/Referenceless Imaging Spatial Quality Evaluator are used to assess the relative image quality of grayscale tomograms, thus evaluating the uncertainty in the derived microstructural attributes. We analyze how image uncertainty is correlated with the uncertainties and magnitude of kinetic and transport properties of an electrode, further identifying pathways of uncertainty propagation within microstructural attributes. The coupled effect of spatial heterogeneity and microstructural anisotropy on the uncertainty quantification of transport parameters is also understood. This work demonstrates a novel methodology to extract microstructural descriptors from real electrode images through quantification of associated uncertainties and discerning the relative strength of their propagation, thus facilitating feedback to manufacturing processes from accurate image based electrochemical simulations.

25 ENERGY STORAGE↗

Genetic Algorithm for Optimization of Neural Networks for Bayesian Inference of Model Uncertainty

The objective of this work was to develop a genetic optimization algorithm that can design a neural network capable of producing uncertainty estimates along with predictions. This algorithm is necessary because the inclusion of uncertainty modeling in a neural network greatly complicates the network’s design space, making the development of a converging model extremely difficult and time consuming. The genetic algorithm presented in this work uses a number of value ranges for various configurable neural network parameters to create a randomly generated population of network architectures. The initially generated population is then evolved over the course of several generations, with the best performing models breeding to produce novel network configurations. Mutations are randomly applied to the network designs to facilitate the development of adaptations beneficial to the task being performed. An experiment was conducted to validate the proposed algorithm, in which the genetic optimizer was tasked with producing a neural network capable of predicting the sound pressure level (SPL) resulting from jet-surface interaction (JSI) noise. The data used for this task was generated at the NASA Glenn Research Center in the Aero-Acoustic Propulsion Laboratory. Starting with an initial population size of 35 randomly generated networks, and evolved over the course of 10 generations, the genetic algorithm produced a design able to predict SPL as a result of JSI noise within 0.272 dB, on average.

Genetic algorithm↗

B-PINNs: Bayesian Physics-informal Neural Networks for Forward and Inverse PDE Problems with Noisy Data

We propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framework. Compared with PINNs, in addition to uncertainty quantification, B-PINNs obtain more accurate predictions in scenarios with large noise due to their capability of avoiding overfitting. We conduct a systematic comparison between the two different approaches for the B-PINNs posterior estimation (i.e., HMC or VI), along with dropout used for quantifying uncertainty in deep neural networks. Our experiments show that HMC is more suitable than VI with mean field Gaussian approximation for the B-PINNs posterior estimation, while dropout employed in PINNs can hardly provide accurate predictions with reasonable uncertainty. Finally, we replace the BNN in the prior with a truncated Karhunen-Loève (KL) expansion combined with HMC or a deep normalizing flow (DNF) model as posterior estimators. The KL is as accurate as BNN and much faster but this framework cannot be easily extended to high-dimensional problems unlike the BNN based framework.

Non linear PDEs, Noisy data, Bysian physics inform↗