Search NASA⌕ Search

SEARCH · Search NASA

Results for “Bayesian learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40

How Much Testing is Needed to Manage Supportability Risks for Beyond-LEO Missions?

Supportability will be a significantly greater driver of cost and risk for future deep-space crewed missions than it has been in the past. Spares requirements and maintenance risk mitigation in particular present an unprecedented challenge for missions beyond Low Earth Orbit (LEO), since, for the first time in human spaceflight history, crews will be weeks or months away from resupply or a safe return to Earth in the event of an abort. Under these conditions, failure rates are a critical parameter that must be well-understood in order to manage logistics and risk effectively. However, failure rates cannot be measured directly, and can only be estimated based on past experience and test results. Previous research has shown that International Space Station (ISS) operational experience has provided significant benefits to future missions by reducing uncertainty and improving accuracy in failure rate estimates, resulting in significant reductions in mass and risk for beyond-LEO missions. This paper updates and expands on that research and quantifies the potential value of continued testing for future mission supportability. Frequentist and Bayesian models for evaluating, validating, and updating failure rate estimates are described, and are combined with supportability models to examine potential impacts of additional operating experience for future missions in terms of logistics mass reduction. The implications of these results for technology development, system design, and program planning are discussed along with lessons learned and recommendations for future system development. In the end, there is no simple answer to the question of how much testing is required, but the models described in this paper provide a way to evaluate the potential impacts of testing in order to inform test planning.

Andrew C Owens↗

Application of deep learning to single-shot gas-phase laser-induced breakdown spectroscopy

Single-shot fs laser-induced breakdown spectroscopy (LIBS) has the potential to capture ns-scale electrode desorption phenomena in pulsed power fusion drivers. However, the successful implementation of the diagnostic for this purpose is challenging, as it requires interpreting single-shot measurements collected from low-density gas mixtures. In this work, we demonstrate the efficacy of a Bayesian-optimized convolutional neural network (CNN) to interpret these measurements. We generated 256 distinct measurement conditions at relevant gas pressures ranging from 80–530 mTorr by mixing 100–250 sccm H 2 and 50–200 sccm CH 4 in increments of 10 sccm. Despite the considerable overlap between signals separated by 20 sccm, the CNN is able to predict the H 2 flow rate with a root-mean-square error (RMSE) of 15.9 sccm and the CH 4 flow rate with an RMSE of 12.0 sccm. The average relative prediction error is <9% for each gas and largely remains below or near 10%.

Brown, Nathan Parnell [Sandia National Lab. (SNL-N↗

Techno-economic implications and cost of forecasting errors in solar PV power production using optimized deep learning models

Accurate solar Photovoltaic (PV) power forecasting is important for enhancing both the performance and economic feasibility of PV systems. This study evaluates several deep learning models, including Dense Neural Networks (DNN), Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN), and a hybrid LSTMCNN model, for predicting PV power production one day in advance. Prior to optimization, the models exhibited relatively high errors, with the best model (DNN) achieving a Root Mean Square Error (RMSE) of 31.13 kW and a coefficient of determination (R 2 ) of 62.15 %. After employing Bayesian optimization, the LSTM-CNN model demonstrated the best performance, with the RMSE reduced to 9.79 kW and R 2 improved to 97.62 %, showcasing significant enhancement in predictive accuracy. Here, the economic evaluation considered three cases: rewards for underestimation (0.08 USD/kWh), no rewards, and penalties for both over-and underestimation (120 % of the utility tariff). In the rewards scenario, the LSTM-CNN model reduced the Levelized Cost of Electricity (LCOE) by 4 %, while in the penalty scenario, a backup diesel generator would have increased the LCOE by 49 %. Additionally, the LSTM-CNN model minimized financial losses, achieving the lowest penalties and maximizing net cash flow compared to other models, demonstrating its overall technical and economic superiority.

Deep learning↗

Data driven drift correction for complex optical systems

To exploit the thousand-fold increase in spectral brightness of modern light sources, increasingly intricate experiments are being conducted that demand extremely precise beam trajectory. Maintaining the optimal trajectory over several hours of an experiment with the needed precision necessitates active drift control. Here, we outline time varying Bayesian optimization (TVBO) as a data driven approach for robust drift correction, and illustrate its application for a split and delay optical system composed of six crystals and twelve input dimensions. Using numerical simulations, we exhibit the application of TVBO for linear drift, non-smooth temporal drift as well as constrained TVBO for multi-objective control settings, representing real-life operating conditions. This approach can be easily adapted to other X-ray beam conditioning and guidance systems, including multi-crystal monochromators and grazing-incidence mirrors, to maintain sub-micrometer and nanoradian beam stability over the course of an experiment spanning several hours.

Bayesian optimization↗

Comparing Automated Posterior Estimation Techniques for Modeling Strong Lenses In Ground-based Survey Data

Current and future ground-based cosmological surveys, such as the Dark Energy Survey (DES), and the Vera Rubin Observatory Legacy Survey of Space and Time (LSST), are predicted to discover thousands to tens of thousands of strong gravitational lenses. The large number of strong lenses discoverable in future surveys will make strong lensing a highly competitive and complementary cosmic probe. However, conventional lens modeling techniques are unable to scale up to the sheer number of lenses that will be discovered through upcoming surveys. Therefore, the use of automated lens analysis techniques is necessary. We demonstrate that machine learning methods can be used to automate the inference of informative model posteriors of strong lensing systems in ground-based surveys with credible uncertainty estimation. We present two Simulation-Based Inference (SBI) approaches for lens parameter estimation of galaxy-galaxy lenses. We demonstrate applications of Neural Posteriors Estima tors (NPEs) and Bayesian Neural Network (BNNs) to automate the inference of a 12-parameter lensing system for DES-like ground-based imaging data. We apply a suite of diagnostics (e.g., posterior coverage and SBC) to validate the performance of our methods. We find that NPEs outperform the BNN, producing posterior distributions that are for the most part both more accurate and more precise; in particular, several source-light model parameters are systematically biased in the BNN implementation.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Operation Optimization Using Reinforcement Learning with Integrated Artificial Reasoning Framework

In large and complex systems, operational decision-making requires a systematic analysis with a vast amount of data from both process parameters and component status monitoring. In this paper, we present an integrated artificial reasoning approach for system state transition models that can help operational decision-making with explainable and traceable reasoning. The integrated artificial reasoning framework is a physics-based approach of defining the system structure in a Bayesian network, so we leveraged it in a Markov decision process (MDP) for finding optimal operational solutions. In our proposed framework, the MDP is implemented on a dynamic Bayesian network (DBN), which represents causalities in a system. The multilevel flow modeling was utilized in order to extract these causalities in a more efficient and objective manner. Since multilevel flow modeling is based on the fundamental energy and mass conservation laws, the target system is decomposed into several mass, energy, and information structures, which serve as the basis for a DBN. The MDP consists of the processes of finding a solution for the Bellman equation, which can be derived from the conditional probability equations of the constructed DBN. System operators can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from the component degradation process or random failures. We analyzed a simplified example system to illustrate finding an optimal operational policy with this approach.

Kim, Junyung↗

Probabilistic Photometric Redshifts in the Era of Petascale Astronomy

With the growth of large photometric surveys, accurately estimating photometric redshifts, preferably as a probability density function (PDF), and fully understanding the implicit systematic uncertainties in this process has become increasingly important. These surveys are expected to obtain images of billions of distinct galaxies. As a result, storing and analyzing all of these photometric redshift PDFs will be non-trivial, and this challenge becomes even more severe if a survey plans to compute and store multiple different PDFs. In this thesis, we have developed an end-to-end framework that will compute accurate and robust photometric redshift PDFs for massive data sets by using two new, state-of-the-art machine learning techniques that are based on a random forest and a random atlas, respectively. By using data from several photometric surveys, we demonstrate the applicability of these new techniques, and we demonstrate that our new approach is among the best techniques currently available. We also show how different techniques can be combined by using novel Bayesian techniques to improve the photometric redshift precision to unprecedented levels while also presenting new approaches to better identify outliers. In addition, our framework provides supplementary information regarding the data being analyzed, including unbiased estimates of the accuracy of the technique without resorting to a validation data set, identification of poor photometric redshift areas within the parameter space occupied by the spectroscopic training data, and a quantification of the relative importance of the variables used during the estimation process. Furthermore, we present a new approach to represent and store photometric redshift PDFs by using a sparse representation with outstanding compression and reconstruction capabilities. We also demonstrate how this framework can also be directly incorporated into cosmological analyses. The new techniques presented in this thesis are crucial to enable the development of precision cosmology in the era of petascale astronomical surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

Tree Classification Software

This paper introduces the IND Tree Package to prospective users. IND does supervised learning using classification trees. This learning task is a basic tool used in the development of diagnosis, monitoring and expert systems. The IND Tree Package was developed as part of a NASA project to semi-automate the development of data analysis and modelling algorithms using artificial intelligence techniques. The IND Tree Package integrates features from CART and C4 with newer Bayesian and minimum encoding methods for growing classification trees and graphs. The IND Tree Package also provides an experimental control suite on top. The newer features give improved probability estimates often required in diagnostic and screening tasks. The package comes with a manual, Unix 'man' entries, and a guide to tree methods and research. The IND Tree Package is implemented in C under Unix and was beta-tested at university and commercial research laboratories in the United States.

Buntine, Wray↗

Unsupervised machine learning discovery of structural units and transformation pathways from imaging data

We show that unsupervised machine learning can be used to learn chemical transformation pathways from observational Scanning Transmission Electron Microscopy (STEM) data. To enable this analysis, we assumed the existence of atoms, a discreteness of atomic classes, and the presence of an explicit relationship between the observed STEM contrast and the presence of atomic units. With only these postulates, we developed a machine learning method leveraging a rotationally invariant variational autoencoder (VAE) that can identify the existing molecular fragments observed within a material. The approach encodes the information contained in STEM image sequences using a small number of latent variables, allowing the exploration of chemical transformation pathways by tracing the evolution of atoms in the latent space of the system. The results suggest that atomically resolved STEM data can be used to derive fundamental physical and chemical mechanisms involved, by providing encodings of the observed structures that act as bottom-up equivalents of structural order parameters. The approach also demonstrates the potential of variational (i.e., Bayesian) methods in the physical sciences and will stimulate the development of more sophisticated ways to encode physical constraints in the encoder–decoder architectures and generative physical laws and causal relationships in the latent space of VAEs.

97 MATHEMATICS AND COMPUTING↗

Structurally flexible cloud microphysics, observationally constrained at all scales via ML-accelerated Bayesian inference

We discuss the challenge of developing observationally informed parameterizations of microphysics for use at a hierarchy of modeling scales. Our proposed approach is applicable to any domain that suffers from a two-fold parameterization problem, where physical processes are not resolved at the model scale (the first problem), and those processes are uncertain at any scale (the second problem). For such problems, a physical approach facilitates modeling across scales, as well as systematic observational inference accelerated by machine learning (ML) surrogate models, and quantification of physical uncertainties.

54 ENVIRONMENTAL SCIENCES↗

Sequential Bayesian Experimental Design for Calibration of Expensive Simulation Models

Simulation models of critical systems often have parameters that need to be calibrated using observed data. For expensive simulation models, calibration is done using an emulator of the simulation model built on simulation output at different parameter settings. Using intelligent and adaptive selection of parameters to build the emulator can drastically improve the efficiency of the calibration process. The article proposes a sequential framework with a novel criterion for parameter selection that targets learning the posterior density of the parameters. The emergent behavior from this criterion is that exploration happens by selecting parameters in uncertain posterior regions while simultaneously exploitation happens by selecting parameters in regions of high posterior density. Furthermore, the advantages of the proposed method are illustrated using several simulation experiments and a nuclear physics reaction model.

97 MATHEMATICS AND COMPUTING↗

Short-term nodal load forecasting based on machine learning techniques

This paper introduces an advanced Short-term Nodal Load Forecasting (STNLF) method that forecasts nodal load profiles for the next day in power systems, based on the combined use of three machine learning techniques. Least Absolute Shrinkage and Selection Operator (LASSO) is employed to reduce the number of features for a single nodal load forecasting. Principal Component Analysis (PCA) is used to capture the features of historical loads in low-dimensional space compared to the original high-dimensional load space where features are barely possible to depict. Additionally, Bayesian Ridge Regression (BRR) is utilized to decide the parameters of the prediction model from a statistics perspective. Tests based on modified PJM load data demonstrate the effectiveness of the proposed STNLF method compared to the state-of-the-art General Regression Neural Network (GRNN) method. Moreover, the reliability of the day-ahead Unit Commitment (UC) solution is shown to have been improved, based on the forecasted load data using the proposed STNLF method.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Differentiable Multiphysics Codes: A Breakthrough Technology for Simulation and Computing

This document summarizes the findings of a strategic planning exercise commissioned by the Weapons Simulation and Computing, Computational Physics (WSC/CP) program at the Lawrence Livermore National Laboratory (LLNL) in FY24. During the year, the committee met with multiple stakeholder communities to gather input, opinions, suggestions and concerns which have been incorporated throughout this document. The key findings from this exercise are summarized: • The development of multiphysics modelling and simulation (mod/sim) codes and software technologies, their deployment on exascale compute platforms, and their broad adoption across the NNSA is a major success of the Advanced Simulation and Computing (ASC) program and the Exascale Computing Project (ECP). Sustained investment in these core technologies is essential. • Today’s state of the art involves running ensembles of O(100K) simulations to perform uncertainty quantification (UQ) and design studies using multiple statistical methods such as Bayesian optimization to understand sensitivities of our models and explore parameterized design spaces. Even with exascale computing, we are practically limited to O(10) parameters in these studies since the number of simulations required to sample the space scales exponentially with the number of design parameters. • The data from these simulation ensembles is increasingly being used to train machine learned (ML) surrogates (or reduced order models, ROMs) which can then be used for optimization or real time design exploration. However, the trained surrogates are still limited in the number of parameters they can represent due to the sampling limitations previously noted. • Augmenting our suite of integrated multiphysics simulation codes, both current and emerging, with the ability to compute gradients (solution derivatives) of arbitrary simulation outputs with respect to (some or all) simulation inputs would be a breakthrough technology, opening the door to a new era of efficient and automated inverse design based on verified and validated mod/sim capabilities. • This capability, which we refer to as differentiable multiphysics codes (DMCs), would revolutionize both UQ and optimization studies by breaking the curse of dimensionality that presently limits our “gradient-free” ensemble based computing approach. A similar breakthrough occurred in the AI/ML community once the ability to compute gradients of arbitrary loss functions using back-propagation became commonplace. Gradient information from the multiphysics codes can also be used to dramatically improve the efficiency and scale of training of ML/ROM surrogates for rapid assessments. • Achieving this in our suite of codes will be a grand challenge, similar to the amount of effort that was required to transition from CPU to GPU computing. It will require buy-in from the entire WSC/CP program and beyond, including all integrated codes, physics and engineering models, third-party library dependencies and performance portability abstractions. It will also require investment in research and development of numerical methods for computing adjoints of coupled physics across multiple adaptively refined moving meshes and of stochastic (Monte Carlo) and mesh free (SPH) methods. • New software and numerical techniques, largely pioneered by the AI/ML community, make this feasible. Chief among these is automatic differentiation (AD), the ability to employ AD at point-wise locations in a physics calculation (instead of traditional black-box approaches) and the ability to perform “back-propagation in time” (or reverse mode AD) for non-linear partial differential equations (PDEs). Fundamentally, the conclusion of this strategic planning exercise is that the time is right to undertake a large scale effort in WSC, centered on the existing integrated codes, to continue the natural evolution of mod/sim in the age of AI/ML. Instead of attempting to replace mod/sim with purely data driven AI/ML models, we believe the key to success is to integrate AI/ML by building on top of the decades of hard-won knowledge and the verified/validated multiphysics modelling capability that is the hallmark of the ASC program.

97 MATHEMATICS AND COMPUTING↗

A Framework for the Analysis of Deep Neural Networks in Autonomous Aerospace Applications using Bayesian Statistics

Deep Neural Networks (DNNs) are considered to be key components in many autonomous systems. Applications range from vision-based obstacle avoidance to intelligent/learning control and planning. Safety-critical applications as found in the aerospace domain require that the behavior of the DNN is validated and tested rigorously for safety of the autonomous system (AUS). In this paper, we present a framework to support testing of DNNs and the analysis of the network structure. Our framework employs techniques from statistical modeling and active learning to effectively generate test cases for DNN safety testing and performance analysis. We will present results of a case study on a physics-based Deep recurrent residual neural network (DR-RNN), which has been trained to emulate the aerodynamics behavior of a fixed-wing aircraft.

Deep Neural networks↗

Bayes_Opt-SWMM: A Gaussian process-based Bayesian optimization tool for real-time flood modeling with SWMM

Real-time flood model plays a pivotal role in averting urban flood damage, particularly when there is minimal lead time for preparatory measures. However, urban flood modeling in real-time often contends with inherent uncertainties arising from input data uncertainty and parameter ambiguities. Here this study introduces a real-time calibration (RTC) tool called Bayes_Opt-SWMM, specifically tailored for real-time urban flood modeling and uncertainty optimization. This tool leverages the Gaussian process-based Bayesian optimization algorithm and interfaces seamlessly with the Stormwater Management Model (SWMM). It integrates real-time model forcing data and flood monitoring collected through sensors and gauges which are strategically placed within critical locations of urban drainage systems. Our approach hinges on the Surrogate Model based Uncertainty Optimization (SMUO) concept, providing an avenue for enhancing real-time flood modeling. Bayes_Opt-SWMM runs the optimization process using a surrogate model called Gaussian Process emulator with two inference methods: (1) the Gaussian Process (GP) model and (2) Markov Chain Monte Carlo (MCMC) algorithm in GP model (GP_MCMC). Furthermore, three acquisition functions, namely Expected Improvement (EI), Maximum Probability of Improvement (MPI), and Lower Confidence Bound (LCB), facilitate optimal parameter fitting within the surrogate models. The efficiency of GP-based surrogate models in learning SWMM model parameters, leads to an improved uncertainty quantification and accelerated real-time flood modeling in urban areas. Overall, Bayes_Opt-SWMM emerges as a cost-effective and valuable tool for real-time flood modeling and monitoring, with significant potential for managing intelligent storm water systems in urban environments.

54 ENVIRONMENTAL SCIENCES↗

Combining lattice QCD and phenomenological inputs on generalised parton distributions at moderate skewness

Abstract We present a systematic study demonstrating the impact of lattice QCD data on the extraction of generalised parton distributions (GPDs). For this purpose, we use a previously developed modelling of GPDs based on machine learning techniques fulfilling the theoretical requirements of polynomiality, a form of positivity constraint and known reduction limits. A special care is given to estimate the uncertainty stemming from the ill-posed character of the connection between GPDs and the experimental processes usually considered to constrain them, like deeply virtual Compton scattering (DVCS). Moke lattice QCD data inputs are included in a Bayesian framework to a prior model based on an Artificial Neural Network. This prior model is fitted to reproduce the most experimentally accessible information of a phenomenological extraction by Goloskokov and Kroll. We highlight the impact of the precision, correlation and kinematic coverage of lattice data on GPD extraction at moderate $$\xi $$ ξ which has only been brushed in the literature so far, paving the way for a joint extraction of GPDs.

Physics↗

Optimization of Water-Alternating-CO2 Injection Field Operations Using a Machine-Learning-Assisted Workflow

Summary This paper will present a robust workflow to address multiobjective optimization (MOO) of carbon dioxide (CO2)-enhanced oil recovery (EOR)-sequestration projects with a large number of operational control parameters. Farnsworth unit (FWU) field, a mature oil reservoir undergoing CO2 alternating water injection (CO2-WAG) EOR, will be used as a field case to validate the proposed optimization protocol. The expected outcome of this work would be a repository of Pareto-optimal solutions of multiple objective functions, including oil recovery, carbon storage volume, and project economics. FWU’s numerical model is used to demonstrate the proposed optimization workflow. Because using MOO requires computationally intensive procedures, machine-learning-based proxies are introduced to substitute for the high-fidelity model, thus reducing the total computation overhead. The vector machine regression combined with the Gaussian kernel (Gaussian-SVR) is used to construct proxies. An iterative self-adjusting process prepares the training knowledge base to develop robust proxies and minimizes computational time. The proxies’ hyperparameters will be optimally designed using Bayesian optimization to achieve better generalization performance. Trained proxies will be coupled with multiobjective particle swarm Optimization (MOPSO) protocol to construct the Pareto-front solution repository. The outcomes of this workflow will be a repository containing Pareto-optimal solutions of multiple objectives considered in the CO2-WAG project. The proposed optimization workflow will be compared with another established methodology using a multilayer neural network (MLNN) to validate its feasibility in handling MOO with a large number of parameters to control. Optimization parameters used include operational variables that might be used to control the CO2-WAG process, such as the duration of the water/gas injection period, producer bottomhole pressure (BHP) control, and water injection rate of each well included in the numerical model. It is proved that the workflow coupling Gaussian-SVR proxies and the iterative self-adjusting protocol is more computationally efficient. The MOO process is made more rapid by squeezing the size of the required training knowledge base while maintaining the high accuracy of the optimized results. The outcomes of the optimization study show promising results in successfully establishing the solution repository considering multiple objective functions. Results are also verified by validating the Pareto fronts with simulation results using obtained optimized control parameters. The outcome from this work could provide field operators an opportunity to design a CO2-WAG project using as many inputs as possible from the reservoir models. The proposed work introduces a novel concept that couples Gaussian-SVR proxies with a self-adjusting protocol to increase the computational efficiency of the proposed workflow and to guarantee the high accuracy of the obtained optimized results. More importantly, the workflow can optimize a large number of control parameters used in a complex CO2-WAG process, which greatly extends its utility in solving large-scale MOO problems in various projects with similar desired outcomes.

Energy & Fuels↗

Measurement of Neutrino Absolute Deep Inelastic Scattering Cross Section in Iron, Lead, Carbon, and Plastic Using MINERvA Detector at $E_{\nu} = 6$ \GeV

A deep inelastic sample is selected from the nuclear charged current inclusive analysis, by requiring reconstructed $Q^{2} \geq 1.0$ (GeV/c)$^{2}$ and $W \geq 2.0$ GeV/c$^{2}$. The neutrino energy range of $5 < E_{\nu} < 50$ previously applied to this analysis in the Low Energy (LE) analysis is removed and the energy range cut is imposed on the muon energy from $2 < E_{\mu} < 50$ GeV. Machine learning vertexing is used to improve the track-based vertexing resulting in higher purity and efficiency of the sample. A local sideband method is utilized to remove the plastic background for the events reconstructed in the target. After plastic backgrounds and non-DIS backgrounds are subtracted, the reconstructed neutrino energy and Bjorken-x are unsmeared to their true quantities via Bayesian unfolding. The unfolded distributions of carbon, iron, lead, and plastic are efficiency corrected and then divided by the flux and the number of nucleons in the target to produce the absolute cr oss section as a function of $E_{\nu}$ and differential cross section as function of $x_{bj}$. The results of the analysis show good agreement between data and MC for $E_{\nu}$ and $x_{bj}$ (see Figures \ref{fig:xsecEnuC} through \ref{fig:xsecXCH}). None of the distributions displayed significant differences with the theoretical models underlying MINERvAs MC.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗