Search NASA⌕ Search

SEARCH · Search NASA

Results for “Bayesian Neural Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

Next-Level Energy Management in Manufacturing: Facility-Level Energy Digital Twin Framework Based on Machine Learning and Automated Data Collection

This research introduces an energy prediction framework at the facility level supported by automated data collection and machine learning models. It investigates whether reducing the prediction time scale allows for applying more complex machine learning techniques and if those techniques improve the prediction accuracy. The primary advantages of this framework lie in its automation of the energy prediction process and its provision of real-time energy data suitable for use in energy dashboards or digital twins. A sitewide dataset was created by combining 15 min energy and daily production data of five shops—assembly, battery, body (electric), body (gas), and paint—from a globally recognized electric vehicle manufacturer. Various machine learning models were evaluated on daily, weekly, and monthly datasets, including, in increasingly complex order: naïve, simple linear regression, net regularized generalized linear regression, principal component regression, k-nearest neighbor, random forest, and Bayesian regularized neural network. Compared to the current state-of-the-art energy consumption prediction for the industrial facility level, this research investigates more complex models and smaller time intervals for higher accuracy. The findings revealed that the more complex monthly models require a minimum of a year and a half of data to operate, while weekly models demand a year of data to achieve improved accuracy. Daily models can operate with only six months of data but exhibit poor performance due to reduced prediction accuracy of production. Key challenges identified include access to reliable, high-quality energy and production data and the initial demand for human labor.

digital twin↗

Machine Learning Techniques Applied to Sensor Data Correction in Building Technologies

Since commercial and residential buildings account for nearly half of the United States' energy consumption, making them more energy-efficient is a vital part of the nation's overall energy strategy. Sensors play an important role in this research by collecting data needed to analyze performance of components, systems, and whole-buildings. Given this reliance on sensors, ensuring that sensor data are valid is a crucial problem. Solutions being researched are machine learning techniques, namely: artificial neural networks and Bayesian Networks. Types of data investigated in this study are: (1) temperature; (2) humidity; (3) refrigerator energy consumption; (4) heat pump liquid pressure; and (5) water flow. These data are taken from Oak Ridge National Laboratory's (ORNL) ZEBRAlliance research project which is composed of four single-family homes in Oak Ridge, TN. Results show that for the temperature, humidity, pressure, and flow sensors, data can mostly be predicted with root-mean-square error (RMSE) of less than 10% of the respective sensor's mean value. Results for the energy sensor are not as good; RMSE are centered about 100% of the mean value and are often well above 200%. Bayesian networks have RSME of less than 5% of the respective sensor's mean value, but took substantially longer to train.

building technologies↗

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗

Uncertainty Quantification for Neutron Shield Using Convolutional Neural Networks

Uncertainty quantification from radiation transport calculations was conducted using a Bayesian inference approach. A surrogate model, using a convolutional neural network, was employed to emulate the neutron fluence, which was simulated with a Monte Carlo radiation transport model. This allowed for a computationally cheap approach to evaluate input parameters and to sample their corresponding posterior probability distributions. Experimental data from the literature were employed to perform uncertainty quantification studies for concrete shields. As a result, the method is a nonintrusive approach that enables studies with multiple input parameters and can be applied to any radiation transport model.

Bayesian inference↗

Search for the Standard Model Higgs boson produced in association with a W Boson in the isolated-track charged-lepton channel using the Collider Detector at Fermilab

The Higgs boson is the only elementary particle predicted by the Standard Model (SM) that has not yet been observed experimentally. If it exists, it explains the spontaneous electroweak symmetry breaking and the origin of mass for gauge bosons and fermions. We test the validity of the SM by performing a search for the associated production of a Higgs boson and a W boson in the channel where the Higgs boson decays to a bottom-antibottom quark pair and the W boson decays to a charged lepton and a neutrino (the WH channel). We study a dataset of proton-antiproton collisions at a centre-of-mass energy √s = 1.96 TeV provided by the Tevatron accelerator, corresponding to an integrated luminosity of 5.7 fb -1 , and recorded using the Collider Detector at Fermilab (CDF).We select events consistent with the signature of exactly one charged lepton (electron or muon), missing transverse energy due to the undetected neutrino (MET) and two collimated streams of particles (jets), at least one of which is required to be identified as originating from a bottom quark. We improve the discrimination of Higgs signal from backgrounds through the use of an artificial neural network. Using a Bayesian statistical inference approach, we set for each hypothetical Higgs boson mass in the range 100-150 GeV/c 2 with 5 GeV/c 2 increments a 95% credibility level (CL) upper limit on the ratio between the Higgs production cross section times branching fraction and the SM prediction. Our main original contributions are the addition of a novel charged lepton reconstruction algorithm with looser requirements (ISOTRK) with respect the electron or muon tight criteria (TIGHT), as well as the introduction of a novel trigger-combination method that allows to maximize the event yield while avoiding trigger correlations and that is used for the ISOTRK category. The ISOTRK candidate is a high-transverse-momentum good-quality track isolated from other activity in the tracking system and not required to match a calorimeter cluster, as for a tight electron candidate, or an energy deposit in the muon detector, as for a tight muon candidate. The ISOTRK category recovers real charged leptons that otherwise would be lost in the non-instrumented regions of the detector. This allows the reconstruction of more W boson candidates, which in turn increases the number of reconstructed WH signal candidate events, and therefore improves the sensitivity of the WH search. For the TIGHT charged lepton categories, we employ charged-lepton-dedicated triggers to improve the rate of WH signal acceptance during data taking. Since there is no ISOTRK-dedicated trigger at CDF, for the ISOTRK charged lepton category we employ three MET-plus-jets-based triggers. For each trigger we first identify the jet selection where the trigger efficiency is flat with respect to jet information (transverse energy and direction of motion in the transverse plane for the two jets in the event) and then we parametrize the trigger efficiency as a function of trigger MET. On an event-by-event basis, for each trigger we compute a trigger efficiency as a function of trigger parametrization, trigger MET, jet information, trigger prescale and information about whether the trigger is defined or not. For the ISOTRK category we combine the three triggers using a novel method, which allows the combination of any number of triggers in order to maximize the event yield while avoiding trigger correlations. On an event-by-event basis, only the trigger with the largest efficiency is used. By avoiding a logical 'OR' between triggers, the loss in the yield of events accepted by the trigger combination is compensated by a smaller and easier-to-compute corresponding systematic uncertainty. The addition of the ISOTRK charged lepton category to the TIGHT category produces an increase of 33% in the WH signal yield and a decrease of 15.5% to 19.0% in the median expected 95% CL cross-section upper limits across the entire studied Higgs mass interval. The improvement in analysis sensitivity is smaller than the improvement in signal yield because the ISOTRK category has a smaller signal over background ratio than the TIGHT category, due to the looser ISOTRK reconstruction criteria. The observed (median expected) 95% CL SM Higgs upper limits on cross section times branching ratio vary between 2.39 x SM (2.73 x SM) for a Higgs mass of 100 GeV/c 2 to 31.1 x SM (31.2 x SM) for a Higgs mass of 150 GeV/c 2 , while the value for a 115 GeV/c 2 Higgs boson is that of 5.08 x SM (3.79 x SM). The novel trigger combination method is already in use by several CDF analyses. It is applicable to any analysis that uses triggers based on MET and jets, such as supersymmetry searches at the ATLAS and CMS experiments at the Large Hadron Collider. In its most general form, the method can be used by any analysis that combines any number of different triggers.

43 PARTICLE ACCELERATORS↗

FastBNNs

FastBNNs is a Python package implementing open-source algorithms for fast Bayesian inference of neural networks.

Schodt, David↗

Application of deep learning to single-shot gas-phase laser-induced breakdown spectroscopy

Single-shot fs laser-induced breakdown spectroscopy (LIBS) has the potential to capture ns-scale electrode desorption phenomena in pulsed power fusion drivers. However, the successful implementation of the diagnostic for this purpose is challenging, as it requires interpreting single-shot measurements collected from low-density gas mixtures. In this work, we demonstrate the efficacy of a Bayesian-optimized convolutional neural network (CNN) to interpret these measurements. We generated 256 distinct measurement conditions at relevant gas pressures ranging from 80–530 mTorr by mixing 100–250 sccm H 2 and 50–200 sccm CH 4 in increments of 10 sccm. Despite the considerable overlap between signals separated by 20 sccm, the CNN is able to predict the H 2 flow rate with a root-mean-square error (RMSE) of 15.9 sccm and the CH 4 flow rate with an RMSE of 12.0 sccm. The average relative prediction error is <9% for each gas and largely remains below or near 10%.

Brown, Nathan Parnell [Sandia National Lab. (SNL-N↗

Tunable Stochasticity in an Artificial Spin Network

Metamaterials present the possibility of artificially generating advanced functionalities through engineering of their internal structure. Artificial spin networks, in which a large number of nanoscale magnetic elements are coupled together, are promising metamaterial candidates that enable the control of collective magnetic behavior through tuning of the local interaction between elements. In this work, the motion of magnetic domain-walls in an artificial spin network leads to a tunable stochastic response of the metamaterial, which can be tailored through an external magnetic field and local lattice modifications. This type of tunable stochastic network produces a controllable random response exploiting intrinsic stochasticity within magnetic domain-wall motion at the nanoscale. An iconic demonstration used to illustrate the control of randomness is the Galton board. In this system, multiple balls fall into an array of pegs to generate a bell-shaped curve that can be modified via the array spacing or the tilt of the board. A nanoscale recreation of this experiment using an artificial spin network is employed to demonstrate tunable stochasticity. Furthermore, this type of tunable stochastic network opens new paths toward post-Von Neumann computing architectures such as Bayesian sensing or random neural networks, in which stochasticity is harnessed to efficiently perform complex computational tasks.

Artificial spin network↗

Multi-variance replica exchange SGMCMC for inverse and forward problems via Bayesian PINN

Physics-informed neural network (PINN) has been successfully applied in solving a variety of nonlinear non-convex forward and inverse problems. However, the training is challenging because of the non-convex loss functions and the multiple optima in the Bayesian inverse problem. In this work, we propose a multi-variance replica exchange stochastic gradient Langevin dynamics method to tackle the challenge of the multiple local optima in the optimization and the challenge of the multiple modal posterior distribution in the inverse problem. Replica exchange methods are capable of escaping from the local traps and accelerating the convergence; two chains with different temperatures are designed where the low temperature chain aims for the local convergence, and the target of the high temperature chain is to travel globally and explore the whole loss function entropy landscape. However, it may not be efficient to solve mathematical inversion problems by using the vanilla replica method directly since the method doubles the computational cost in evaluating the forward solvers (likelihood functions) in the two chains. To address this issue, we propose to make different assumptions on the energy function estimation and this facilities one to use solvers of different fidelities in the likelihood function evaluation. More precisely, one can use a solver with low fidelity in the high temperature chain while using a solver with high fidelity in the low temperature chain. Our proposed method significantly lowers the computational cost in the high temperature chain, meanwhile preserving the accuracy and converging very fast. Here we give an unbiased estimate of the swapping rate and give an estimation of the discretization error of the scheme. To verify our idea, we design and solve four inverse problems which have multiple modes. The proposed method is also employed to train the Bayesian PINN to solve the forward and inverse problems; faster and more accurate convergence has been observed when compared to the stochastic gradient Langevin dynamics (SGLD) method and vanilla replica exchange methods.

97 MATHEMATICS AND COMPUTING↗

Bayesian Optimized Deep Ensemble for Uncertainty Quantification of Deep Neural Networks: a System Safety Case Study on Sodium Fast Reactor Thermal Stratification Modeling

Deep neural networks (DNNs) are increasingly important to scientific computing and engineering system simulations. Accurate uncertainty quantification (UQ) for DNNs is critical in safety-sensitive engineering domains. Traditional Deep Ensemble (DE) methods, while easy to implement, frequently suffer from poorly calibrated uncertainty estimates and limited predictive accuracy due to reliance on fixed architectures with varied weight initializations. To address these issues, we introduce a workflow that combines Bayesian Optimization (BO) and DE. The workflow is modular, scalable, and integrates parallel BO initialized with Sobol sequences to individually optimize the hyperparameters of each ensemble member. This method enhances ensemble diversity, improves predictive accuracy, and provides reliable uncertainty estimates. We evaluate the proposed BODE approach in a sodium fast reactor thermal stratification modeling case study, where we used a densely connected convolutional neural network to predict turbulent viscosity during the reactor transient with consideration of data noise. We benchmark its performance against several optimization approaches, including baseline deep ensemble, evolutionary algorithm-optimized ensemble, ensemble formed via random search combined with greedy selection, and a BO ensemble using random initialization. Here, our results demonstrate superior performance of the developed BODE approach. In noise-free scenarios, BODE notably reduces incorrect aleatoric uncertainty and significantly enhances predictive accuracy. Under conditions of 5% and 10% Gaussian noise, BODE adaptively quantifies uncertainty proportional to data noise, achieving up to an 80% reduction in root mean square error compared to baseline methods and producing well-calibrated prediction intervals.

Bayesian optimization↗

Off-the-shelf deep learning is not enough, and requires parsimony, Bayesianity, and causality

Abstract Deep neural networks (‘deep learning’) have emerged as a technology of choice to tackle problems in speech recognition, computer vision, finance, etc. However, adoption of deep learning in physical domains brings substantial challenges stemming from the correlative nature of deep learning methods compared to the causal, hypothesis driven nature of modern science. We argue that the broad adoption of Bayesian methods incorporating prior knowledge, development of solutions with incorporated physical constraints and parsimonious structural descriptors and generative models, and ultimately adoption of causal models, offers a path forward for fundamental and applied research.

97 MATHEMATICS AND COMPUTING↗

Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems

We explore using neural operators, or neural network representations of nonlinear maps between function spaces, to accelerate infinite-dimensional Bayesian inverse problems (BIPs) with models governed by nonlinear parametric partial differential equations (PDEs). Neural operators have gained significant attention in recent years for their ability to approximate the parameter-to-solution maps defined by PDEs using as training data solutions of PDEs at a limited number of parameter samples. The computational cost of BIPs can be drastically reduced if the large number of PDE solves required for posterior characterization are replaced with evaluations of trained neural operators. However, reducing error in the resulting BIP solutions via reducing the approximation error of the neural operators in training can be challenging and unreliable. We provide an a priori error bound result that implies certain BIPs can be ill-conditioned to the approximation error of neural operators, thus leading to inaccessible accuracy requirements in training. To reliably deploy neural operators in BIPs, we consider a strategy for enhancing the performance of neural operators: correcting the prediction of a trained neural operator by solving a linear variational problem based on the PDE residual. We show that a trained neural operator with error correction can achieve a quadratic reduction of its approximation error, all while retaining substantial computational speedups of posterior sampling when models are governed by highly nonlinear PDEs. The strategy is applied to two numerical examples of BIPs based on a nonlinear reaction–diffusion problem and deformation of hyperelastic materials. We demonstrate that posterior representations of the two BIPs produced using trained neural operators are greatly and consistently enhanced by error correction.

97 MATHEMATICS AND COMPUTING↗

Understanding the role of segmentation on process-structure–property predictions made via machine learning

Here, the present study investigated the effect of porosity surface determination methods on performance of machine learning models used to predict the tensile properties of AlSi10Mg processed by laser powder bed fusion from micro-computed tomography data. Machine learning models applied in this work include support vector machines, neural networks, decision trees, and Bayesian classifiers. The effects of isosurface thresholding and local gradient approaches for porosity segmentation, as well as image filtering schemes, on model precision were evaluated for samples produced under differing levels of global energy density.

36 MATERIALS SCIENCE↗

BMINN: Learning chemical potentials and parameters from voltage data for multi-phase battery modeling

Free-energy landscapes and chemical potentials govern the dynamics of phase transitions, transport, and stability in functional materials, yet they remain experimentally inaccessible under realistic operating conditions. Here we introduce a Bayesian model-integrated neural network (BMINN) that embeds physics-based formulations of non-autonomous partial differential-algebraic equations into probabilistic learning. This approach reconstructs hidden thermodynamics directly from macroscopic current-voltage data, providing quantitative access to metastable states, staging transitions, and energy barriers without synchrotron probes. Demonstrated on lithium-graphite electrodes, BMINN recovers full Gibbs free-energy landscapes with fidelity validated against operando X-ray diffraction. The framework generalizes across dynamical regimes, enabling accurate voltage prediction, internal state estimation, and inference of governing parameters. Beyond batteries, BMINN exemplifies a broadly applicable strategy for learning missing physics in multiphase, non-equilibrium systems, offering a new pathway to uncover hidden thermodynamic functions across condensed matter and materials physics.

25 ENERGY STORAGE↗