Search NASA⌕ Search

SEARCH · Search NASA

Results for “Bayesian Deep Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Semi-supervised Bayesian Low-shot Learning

Deep neural networks (NNs) typically outperform traditional machine learning (ML) approaches for complicated, non-linear tasks. It is expected that deep learning (DL) should offer superior performance for the important non-proliferation task of predicting explosive device configuration based upon observed optical signature, a task which human experts struggle with. However, supervised machine learning is difficult to apply in this mission space because most recorded signatures are not associated with the corresponding device description, or “truth labels.” This is challenging for NNs, which traditionally require many samples for strong performance. Semi-supervised learning (SSL), low-shot learning (LSL), and uncertainty quantification (UQ) for NNs are emerging approaches that could bridge the mission gaps of few labels and rare samples of importance. NN explainability techniques are important in gaining insight into the inferential feature importance of such a complex model. In this work, SSL, LSL, and UQ are merged into a single framework, a significant technical hurdle not previously demonstrated. Exponential Average Adversarial Training (EAAT) and Pairwise Neural Networks (PNNs) are chosen as the SSL and LSL methods of choice. Permutation feature importance (PFI) for functional data is used to provide explainability via the Variable importance Explainable Elastic Shape Analysis (VEESA) pipeline. A variety of uncertainty quantification approaches are explored: Bayesian Neural Networks (BNNs), ensemble methods, concrete dropout, and evidential deep learning. Two final approaches, one utilizing ensemble methods and one utilizing evidential learning, are constructed and compared using a well-quantified synthetic 2D dataset along with the DIRSIG Megascene.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Uncovering multiscale structure-property correlations via active learning in scanning tunneling microscopy

Atomic arrangements and local sub-structures fundamentally influence emergent material functionalities. These structures are conventionally probed using spatially resolved studies and the property correlations are deciphered by a researcher based on sequential explorations, thereby limiting the efficiency and scope. Here we demonstrate a multi-scale Bayesian deep-learning based framework that automatically correlates material structure with its electronic properties using scanning tunneling microscopy (STM) measurements in real-time. Its predictions are used to autonomously direct exploration toward regions of the sample that optimize a given material property. This method is deployed on a low-temperature ultra-high vacuum STM to understand the structure-property relationship in a europium-based semimetal, EuZn 2 As 2 , a promising candidate relevant to magnetism-driven topological phenomena. The framework employs a sparse-sampling approach to efficiently construct the scalar-property space using minimal measurements, about 1–10% of the data required in standard hyperspectral methods. Moreover, we formulate the problem hierarchically across length scales, implementing autonomous workflow to locate mesoscopic and atomic structures that correspond to a target material property. This framework offers the choice to design scalar-property from the spectroscopic data to steer sample exploration. Our findings reveal correlations of the electronic properties unique to surface terminations, local defect density, and point defects.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Data-driven assessment of magnetic charged particle confinement parameter scaling in magnetized liner inertial fusion experiments on Z

In magneto-inertial fusion, the ratio of the characteristic fuel length perpendicular to the applied magnetic field R to the α-particle Larmor radius $ϱ_α$ is a critical parameter setting the scale of electron thermal-conduction loss and charged burn-product confinement. Here, using a previously developed deep-learning-based Bayesian inference tool, we obtain the magnetic-field fuel-radius product BR ∝ R / $ϱ_α$ from an ensemble of 16 magnetized liner inertial fusion (MagLIF) experiments. Observations of the trends in BR are consistent with relative trade-offs between compression and flux loss as well as the impact of mix from 1D resistive radiation magneto-hydrodynamics simulations in all but two experiments, for which 3D effects are hypothesized to play a significant role. Finally, we explain the relationship between BR and the generalized Lawson parameter χ. Our results indicate the ability to improve performance in MagLIF through careful tuning of experimental inputs, while also highlighting key risks from mix and 3D effects that must be mitigated in scaling MagLIF to higher currents with a next-generation driver.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Karhunen–Loève deep learning method for surrogate modeling and approximate Bayesian parameter estimation

We evaluate the performance of the Karhunen-Loève Deep Neural Network (KL-DNN) framework for surrogate modeling and approximate Bayesian parameter estimation in partial differential equation models. In the surrogate model, the Karhunen-Loève (KL) expansions are used for the dimensionality reduction of the number of unknown parameters and variables, and a deep neural network is employed to relate the reduced space of parameters to that of the state variables. The KL-DNN surrogate model is used to formulate a maximum-a-posteriori-like least-squares problem, which is randomized to draw samples of the posterior distribution of the parameters. We test the proposed framework for a hypothetical unconfined aquifer via comparison with the forward MODFLOW and inverse PEST++ iterative ensemble smoother (IES) solutions as well as the state-of-the-art Fourier neural operator (FNO) and deep operator networks (DeepONets) operator learning surrogate models. Our results show that the KL-DNN surrogate model outperforms FNO and DeepONet for forward predictions. For solving inverse problems, the randomized algorithm provides the same or more accurate Bayesian predictions of the parameters than IES as evidenced by the higher log-predictive probability of both the estimated parameter field and the forecast hydraulic head. The posterior mean obtained from the randomized algorithm is closer to the reference parameter field than that obtained with FNO as the maximum a posteriori estimate.

Approximate Bayesian inference↗
DOE OSTI2022

Dynamic Role-Based Access Control Policy for Smart Grid Applications: An Offline Deep Reinforcement Learning Approach

Role-based access control (RBAC) is adopted in the information and communication technology domain for authentication purposes. However, due to a very large number of entities within organizational access control (AC) systems, static RBAC management can be inefficient, costly, and can lead to cybersecurity threats. In this paper, a novel hybrid RBAC model is proposed, based on the principles of offline deep reinforcement learning (RL) and Bayesian belief networks. The considered framework utilizes a fully offline RL agent, which models the behavioral history of users as a Bayesian belief-based trust indicator. Thus, the initial static RBAC policy is improved in a dynamic manner through off-policy learning while guaranteeing compliance of the internal users with the security rules of the system. By deploying our implementation within the smart grid domain and specifically within a Distributed Energy Resources (DER) ecosystem, we provide an end-to-end proof of concept of our model. Finally, detailed analysis and evaluation regarding the offline training phase of the RL agent are provided, while the online deployment of the hybrid RL-based RBAC model into the DER ecosystem highlights its key operation features and salient benefits over traditional RBAC models.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

We Know Where We Don't Know: 3D Bayesian CNNs for Credible Geometric Uncertainty

Deep learning has been successfully applied to the segmentation of 3D Computed Tomography (CT) scans. Establishing the credibility of these segmentations requires uncertainty quantification (UQ) to identify untrustworthy predictions. Recent UQ architectures include Monte Carlo dropout networks (MCDNs), which approximate deep Gaussian processes, and Bayesian neural networks (BNNs), which learn the distribution of the weight space. BNNs are advantageous over MCDNs for UQ but are thought to be computationally infeasible in high dimension, and neither architecture has produced interpretable geometric uncertainty maps. We propose a novel 3D Bayesian convolutional neural network (BCNN), the first deep learning method which generates statistically credible geometric uncertainty maps and scales for application to 3D data. We present experimental results on CT scans of graphite electrodes and laser-welded metals and show that our BCNN outperforms an MCDN in recent uncertainty metrics. The geometric uncertainty maps generated by our BCNN capture distributions of sigmoid values that are interpretable as confidence intervals, critical for applications that rely on deep learning for high-consequence decisions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Data-driven prediction of scaling and ignition of inertial confinement fusion experiments

Recent advances in inertial confinement fusion (ICF) at the National Ignition Facility (NIF), including ignition and energy gain, are enabled by a close coupling between experiments and high-fidelity simulations. Neither simulations nor experiments can fully constrain the behavior of ICF implosions on their own, meaning pre- and postshot simulation studies must incorporate experimental data to be reliable. Linking past data with simulations to make predictions for upcoming designs and quantifying the uncertainty in those predictions has been an ongoing challenge in ICF research. We have developed a data-driven approach to prediction and uncertainty quantification that combines large ensembles of simulations with Bayesian inference and deep learning. The approach builds a predictive model for the statistical distribution of key performance parameters, which is jointly informed by past experiments and physics simulations. The prediction distribution captures the impact of experimental uncertainty, expert priors, design changes, and shot-to-shot variations. We have used this new capability to predict a 10× increase in ignition probability between Hybrid-E shots driven with 2.05 MJ compared to 1.9 MJ, and validated our predictions against subsequent experiments. We describe our new Bayesian postshot and prediction capabilities, discuss their application to NIF ignition and validate the results, and finally investigate the impact of data sparsity on our prediction results.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Deep learning-based spatio-temporal fusion for high-fidelity ultra-high-speed X-ray radiography

Full-field ultra-high-speed (UHS) X-ray imaging experiments have been well established to characterize various processes and phenomena. However, the potential of UHS experiments through the joint acquisition of X-ray videos with distinct configurations has not been fully exploited. In this paper, we investigate the use of a deep learning-based spatio-temporal fusion (STF) framework to fuse two complementary sequences of X-ray images and reconstruct the target image sequence with high spatial resolution, high frame rate and high fidelity. We applied a transfer learning strategy to train the model and compared the peak signal-to-noise ratio (PSNR), average absolute difference (AAD) and structural similarity (SSIM) of the proposed framework on two independent X-ray data sets with those obtained from a baseline deep learning model, a Bayesian fusion framework and the bicubic interpolation method. The proposed framework outperformed the other methods with various configurations of the input frame separations and image noise levels. With three subsequent images from the low-resolution (LR) sequence of a four times lower spatial resolution and another two images from the high-resolution (HR) sequence of a 20 times lower frame rate, the proposed approach achieved average PSNRs of 37.57 dB and 35.15 dB, respectively. When coupled with the appropriate combination of high-speed cameras, the proposed approach will enhance the performance and therefore the scientific value of UHS X-ray imaging experiments.

deep learning↗
DOE OSTI2023

Data-Driven Discovery of Linear Molecular Probes with Optimal Selective Affinity for PFAS in Water

Approaches to tackle the wide and growing variety of highly persistent per- and polyfluoroalkyl substances (PFAS) are of pressing global need because of their detrimental human health effects, such as cancer, birth defects, and hormone imbalance. Sensitive, selective, and easy-to-use real-time sensors to monitor and detect PFAS and sorbents to extract them are critical to meeting government-mandated environmental concentrations. In this work, we combine all-atom molecular dynamics simulations, enhanced sampling, deep representational learning, and Bayesian optimization to perform high-throughput virtual screening for highly sensitive and selective molecular probes. Our molecular design space consists of 3850 linear hydrocarbon chains with varying degrees of halogenation with and without amine- and phosphine-based headgroups. By employing a data-driven search process, we efficiently explore the molecular design space to optimize the sensitivity to perfluorooctanesulfonic acid (PFOS) as a prototypical PFAS analyte and selectivity relative to a sodium dodecyl sulfate (SDS) interferent. We calculate 504 Gibbs free energies of probe-analyte and probe-interferent interactions and identify probes with PFOS association free energies of up to (-ΔG PFOS ) = 9.8 ± 0.2 kJ/mol and selectivities relative to SDS of (-ΔΔG PFOS–SDS ) = 3.1 ± 1.5 kJ/mol. A C 11 Br 23 P(CH 3 ) 2 probe containing 11 backbone brominated carbons and a tertiary phosphine headgroup possesses the most sensitive binding constant to PFOS within the defined search space of K b PFOS = 177.4 ± 12.7, and a semibrominated probe C 5 H 11 C 7 Br 14 N(CH 3 ) 2 containing 12 backbone carbons and a tertiary amine headgroup possesses the highest selectivity relative to SDS of K b PFOS /K b SDS = 4.6 ± 1.7. A retrospective analysis of our data to extract interpretable design rules reveals that the sensitivity of linear hydrogenated probes increases by approximately 1 kJ/mol per C–C bond. The addition or removal of halogen atoms and amine or phosphine headgroups produces nonmonotonic changes in both sensitivity and selectivity with changes to the sensitivity of up to 2.5 kJ/mol. Finally, this work places empirical limitations on the performance of a wide range of linear probes for PFOS detection and offers a generic strategy for high-throughput computational screening to promote selective and sensitive binding.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Discovery of Self-Assembling π-Conjugated Peptides by Active Learning-Directed Coarse-Grained Molecular Simulation

Electronically active organic molecules have demonstrated great promise as novel soft materials for energy harvesting and transport. Self-assembled nanoaggregates formed from π-conjugated oligopeptides composed of an aromatic core flanked by oligopeptide wings offer emergent optoelectronic properties within a water-soluble and biocompatible substrate. Nanoaggregate properties can be controlled by tuning core chemistry and peptide composition, but the sequence–structure–function relations remain poorly characterized. Here, we employ coarse-grained molecular dynamics simulations within an active learning protocol employing deep representational learning and Bayesian optimization to efficiently identify molecules capable of assembling pseudo-1D nanoaggregates with good stacking of the electronically active π-cores. We consider the DXXX-OPV3-XXXD oligopeptide family, where D is an Asp residue and OPV3 is an oligophenylenevinylene oligomer (1,4-distyrylbenzene), to identify the top performing XXX tripeptides within all 20 3 = 8000 possible sequences. By direct simulation of only 2.3% of this space, we identify molecules predicted to exhibit superior assembly relative to those reported in prior work. Spectral clustering of the top candidates reveals new design rules governing assembly. This work establishes new understanding of DXXX-OPV3-XXXD assembly, identifies promising new candidates for experimental testing, and presents a computational design platform that can be generically extended to other peptide-based and peptide-like systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Probabilistic Programming for Transportable Source Characterization and Uncertainty Quantification of the North Korean Nuclear Tests 2006–2017

Here, we introduce a transportable technique to determine the yield and depth of burial (DOB) from seismic source spectra of underground nuclear explosions. We demonstrate this technique on the six declared North Korean nuclear tests. This approach derives source spectra in absolute units from regional phase (Pg) amplitudes by correcting the observations for geometric spreading, attenuation, and site amplification. We couple the source spectra and explosion source models with a probabilistic programming framework that integrates deep learning techniques and Bayesian modeling. This approach permits the exchange of information across various data categories to quantify both the data and model uncertainty. This technique stands out as an innovative use of broad‐area propagation models, making it transportable across various geologic settings. This method proves to be effective in scenarios with diverse and/or limited observational data, even when the source depth is unknown. We present new independent estimates of absolute yield and DOB that are consistent with the prior assessments, underscoring the potential of this method in enhancing transportable nuclear explosion monitoring capabilities.

58 GEOSCIENCES↗
DOE OSTI2021

Deep Bayesian local crystallography

Abstract The advent of high-resolution electron and scanning probe microscopy imaging has opened the floodgates for acquiring atomically resolved images of bulk materials, 2D materials, and surfaces. This plethora of data contains an immense volume of information on materials structures, structural distortions, and physical functionalities. Harnessing this knowledge regarding local physical phenomena necessitates the development of the mathematical frameworks for extraction of relevant information. However, the analysis of atomically resolved images is often based on the adaptation of concepts from macroscopic physics, notably translational and point group symmetries and symmetry lowering phenomena. Here, we explore the bottom-up definition of structural units and symmetry in atomically resolved data using a Bayesian framework. We demonstrate the need for a Bayesian definition of symmetry using a simple toy model and demonstrate how this definition can be extended to the experimental data using deep learning networks in a Bayesian setting, namely rotationally invariant variational autoencoders.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Posterior Regularized Bayesian Neural Network incorporating soft and hard knowledge constraints

Neural Networks (NNs) have been widely used in supervised learning due to their ability to model complex nonlinear patterns, often presented in high-dimensional data such as images and text. However, traditional NNs often lack the ability for uncertainty quantification. Bayesian NNs (BNNS) could help measure the uncertainty by considering the distributions of the NN model parameters. Besides, domain knowledge is commonly available and could improve the performance of BNNs if it can be appropriately incorporated. In this work, we propose a novel Posterior-Regularized Bayesian Neural Network (PR-BNN) model by incorporating different types of knowledge constraints, such as the soft and hard constraints, as a posterior regularization term. Furthermore, we propose to combine the augmented Lagrangian method and the existing BNN solvers for efficient inference. Furthermore, the experiments in simulation and two case studies about aviation landing prediction and solar energy output prediction have shown the knowledge constraints and the performance improvement of the proposed model over traditional BNNs without the constraints.

14 SOLAR ENERGY↗
DOE OSTI2022

General-Purpose Bayesian Tensor Learning With Automatic Rank Determination and Uncertainty Quantification

A major challenge in many machine learning tasks is that the model expressive power depends on model size. Low-rank tensor methods are an efficient tool for handling the curse of dimensionality in many large-scale machine learning models. The major challenges in training a tensor learning model include how to process the high-volume data, how to determine the tensor rank automatically, and how to estimate the uncertainty of the results. While existing tensor learning focuses on a specific task, this paper proposes a generic Bayesian framework that can be employed to solve a broad class of tensor learning problems such as tensor completion, tensor regression, and tensorized neural networks. We develop a low-rank tensor prior for automatic rank determination in nonlinear problems. Our method is implemented with both stochastic gradient Hamiltonian Monte Carlo (SGHMC) and Stein Variational Gradient Descent (SVGD). We compare the automatic rank determination and uncertainty quantification of these two solvers. We demonstrate that our proposed method can determine the tensor rank automatically and can quantify the uncertainty of the obtained results. We validate our framework on tensor completion tasks and tensorized neural network training tasks.

Bayesian inference↗
DOE OSTI2022

Variational encoder geostatistical analysis (VEGAS) with an application to large scale riverine bathymetry

Estimation of riverbed profiles, also known as bathymetry, plays a vital role in many applications, such as safe and efficient inland navigation, prediction of bank erosion, land subsidence, and flood risk management. The high cost and complex logistics of direct bathymetry surveys, i.e, depth imaging, have encouraged the use of indirect measurements such as surface flow velocities. However, estimating high-resolution bathymetry from indirect measurements is an inverse problem that can be computationally challenging. Here, we propose a reduced-order model (ROM) based approach that utilizes a variational autoencoder (VAE), a type of deep neural network with a narrow layer in the middle, to compress bathymetry and flow velocity information and accelerate bathymetry inverse problems from flow velocity measurements. In our application, the shallow-water equations (SWE) with appropriate boundary conditions (BCs), e.g., the discharge and/or the free surface elevation, constitute the forward problem, to predict flow velocity. Then, ROMs of the SWEs are constructed on a nonlinear manifold of low dimensionality through a variational encoder and the bathymetry inversion problem is derived on the low-dimensional latent space in a Hierarchical Bayesian setting. Further, the reformulation allows variational inference with a small number (e.g., $\mathscr{O}$ (100) of ROM runs and efficient uncertainty quantification. We have tested our inversion approach on a one-mile reach of the Savannah River, GA, USA. Once the neural network is trained (offline stage), the proposed technique can perform the inversion operation orders of magnitude faster than traditional inversion methods that are commonly based on linear projections, such as principal component analysis (PCA), or the principal component geostatistical approach (PCGA). Furthermore, tests show that the algorithm can estimate the bathymetry with good accuracy even with sparse flow velocity measurements.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Scaling and merging time-resolved pink-beam diffraction with variational inference

Time-resolved x-ray crystallography (TR-X) at synchrotrons and free electron lasers is a promising technique for recording dynamics of molecules at atomic resolution. While experimental methods for TR-X have proliferated and matured, data analysis is often difficult. Extracting small, time-dependent changes in signal is frequently a bottleneck for practitioners. Recent work demonstrated this challenge can be addressed when merging redundant observations by a statistical technique known as variational inference (VI). However, the variational approach to time-resolved data analysis requires identification of successful hyperparameters in order to optimally extract signal. In this case study, we present a successful application of VI to time-resolved changes in an enzyme, DJ-1, upon mixing with a substrate molecule, methylglyoxal. We present a strategy to extract high signal-to-noise changes in electron density from these data. Furthermore, we conduct an ablation study, in which we systematically remove one hyperparameter at a time to demonstrate the impact of each hyperparameter choice on the success of our model. We expect this case study will serve as a practical example for how others may deploy VI in order to analyze their time-resolved diffraction data.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

A Statistician’s Overview of Physics-Informed Neural Networks for Spatio-Temporal Data

The recent success of deep neural network models with physical constraints (so-called, Physics-Informed Neural Networks, PINNs) has led to renewed interest in the incorporation of mechanistic information in predictive models. Statisticians and others have long been interested in this problem, which has led to several practical and innovative solutions dating back decades. In this overview, we focus on the problem of data-driven prediction and inference of dynamic spatio-temporal processes that include mechanistic information, such as would be available from partial differential equations, with a strong focus on the quantification of uncertainty associated with data, process, and parameters. Here, we give a brief review of several paradigms and focus our attention on Bayesian implementations given they naturally accommodate uncertainty quantification. We then show that it is straight-forward to include the Bayesian PINN (B-PINN) within the Bayesian hierarchical model (BHM) framework that has long been considered for modeling dynamic spatio-temporal processes. Such a BHM-PINN is illustrated via a simulation study in which a latent nonlinear Burgers’ equation PDE governs the dynamics of Poisson distributed spatio-temporal data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Bayesian↗