Search NASA⌕ Search

SEARCH · Search NASA

Results for “Bayesian network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

Two papers on feed-forward networks

Connectionist feed-forward networks, trained with back-propagation, can be used both for nonlinear regression and for (discrete one-of-C) classification, depending on the form of training. This report contains two papers on feed-forward networks. The papers can be read independently. They are intended for the theoretically-aware practitioner or algorithm-designer; however, they also contain a review and comparison of several learning theories so they provide a perspective for the theoretician. The first paper works through Bayesian methods to complement back-propagation in the training of feed-forward networks. The second paper addresses a problem raised by the first: how to efficiently calculate second derivatives on feed-forward networks.

Buntine, Wray L.↗

Stopping criteria for ending autonomous, single detector radiological source searches

While the localization of radiological sources has traditionally been handled with statistical algorithms, such a task can be augmented with advanced machine learning methodologies. The combination of deep and reinforcement learning has provided learning-based navigation to autonomous, single-detector, mobile systems. However, these approaches lacked the capacity to terminate a surveying/search task without outside influence of an operator or perfect knowledge of source location (defeating the purpose of such a system). Two stopping criteria are investigated in this work for a machine learning navigated system: one based upon Bayesian and maximum likelihood estimation (MLE) strategies commonly used in source localization, and a second providing the navigational machine learning network with a “stop search” action. A convolutional neural network was trained via reinforcement learning in a 10 m × 10 m simulated environment to navigate a randomly placed detector-agent to a randomly placed source of varied strength (stopping with perfect knowledge during training). The network agent could move in one of four directions (up, down, left, right) after taking a 1 s count measurement at the current location. During testing, the stopping criteria for this navigational algorithm was based upon a Bayesian likelihood estimation technique of source presence, updating this likelihood after each step, and terminating once the confidence of the source being in a single location exceeded 0.9. A second network was trained and tested with similar architecture as the previous but which contained a fifth action: for self-stopping. The accuracy and speed of localization with set detector and source initializations were compared over 50 trials of MLE-Bayesian approach and 1000 trials of the CNN with self-stopping. The statistical stopping condition yielded a median localization error of ~1.41 m and median localization speed of 12 steps. The machine learning stopping condition yielded a median localization error of 0 m and median localization speed of 17 steps. This work demonstrated two stopping criteria available to a machine learning guided, source localization system.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

Microstructural Materials Design Via Deep Adversarial Learning Methodology

Identifying the key microstructure representations is crucial for computational materials design (CMD). However, existing microstructure characterization and reconstruction (MCR) techniques have limitations to be applied for microstructural materials design. Some MCR approaches are not applicable for microstructural materials design because no parameters are available to serve as design variables, while others introduce significant information loss in either microstructure representation and/or dimensionality reduction. In this work, we present a deep adversarial learning methodology that overcomes the limitations of existing MCR techniques. In the proposed methodology, generative adversarial networks (GAN) are trained to learn the mapping between latent variables and microstructures. Thereafter, the low-dimensional latent variables serve as design variables, and a Bayesian optimization framework is applied to obtain microstructures with desired material property. Due to the special design of the network architecture, the proposed methodology is able to identify the latent (design) variables with desired dimensionality, as well as capturing complex material microstructural characteristics. The validity of the proposed methodology is tested numerically on a synthetic microstructure dataset and its effectiveness for microstructural materials design is evaluated through a case study of optimizing optical performance for energy absorption. Additional features, such as scalability and transferability, are also demonstrated in this work. In essence, the proposed methodology provides an end-to-end solution for microstructural materials design, in which GAN reduces information loss and preserves more microstructural characteristics, and the GP-Hedge optimization improves the efficiency of design exploration.

Engineering↗

Combining lattice QCD and phenomenological inputs on generalised parton distributions at moderate skewness

Abstract We present a systematic study demonstrating the impact of lattice QCD data on the extraction of generalised parton distributions (GPDs). For this purpose, we use a previously developed modelling of GPDs based on machine learning techniques fulfilling the theoretical requirements of polynomiality, a form of positivity constraint and known reduction limits. A special care is given to estimate the uncertainty stemming from the ill-posed character of the connection between GPDs and the experimental processes usually considered to constrain them, like deeply virtual Compton scattering (DVCS). Moke lattice QCD data inputs are included in a Bayesian framework to a prior model based on an Artificial Neural Network. This prior model is fitted to reproduce the most experimentally accessible information of a phenomenological extraction by Goloskokov and Kroll. We highlight the impact of the precision, correlation and kinematic coverage of lattice data on GPD extraction at moderate $$\xi $$ ξ which has only been brushed in the literature so far, paving the way for a joint extraction of GPDs.

Physics↗

The public health exposome and pregnancy-related mortality in the United States: a high-dimensional computational analysis

Racial inequities in maternal mortality in the U.S. continue to be stark. The 2015–2018, 4-year total population, county-level, pregnancy-related mortality ratio (PRM; deaths per 100,000 live births; National Center for Health Statistics (NCHS), restricted use mortality file) was linked with the Public Health Exposome (PHE). Using data reduction techniques, 1591 variables were extracted from over 62,000 variables for use in this analysis, providing information on the relationships between PRM and the social, health and health care, natural, and built environments. Graph theoretical algorithms and Bayesian analysis were applied to PHE/PRM linked data to identify latent networks. PHE variables most strongly correlated with total population PRM were years of potential life lost and overall life expectancy. Population-level indicators of PRM were overall poverty, smoking, lack of exercise, heat, and lack of adequate access to food. In this high-dimensional analysis, overall life expectancy, poverty indicators, and health behaviors were found to be the strongest predictors of pregnancy-related mortality. This provides strong evidence that maternal death is part of a broader constellation of both similar and unique health behaviors, social determinants and environmental exposures as other causes of death.

60 APPLIED LIFE SCIENCES↗

Decentralized Signal Processing and Distributed Control of Collaborative Autonomous Networks (17-ERD-101 LDRD Final Report)

Intelligent autonomous sensor networks, often comprised of large numbers of sensors, must be capable of jointly exploiting data collected at each agent in the network, and using that data to optimize their future actions towards multiple mission objectives. Centralized signal processing and optimization solutions process all data and determine all future actions at a single agent, and the resulting information and commands are disseminated back to the network. The communications bandwidth this requires and the single point of failure the central agent represents often make these solutions untenable for national security applications. In this project, several fundamental algorithms for solving both the decentralized signal processing and network optimization were developed, as well as simulation software to validate the results of these algorithms at scale. Specifically, novel algorithms for Bayesian decentralized estimation and decentralized detection and optimization based on the alternating direction method of multipliers (ADMM) were developed for autonomous sensor networks and published in the literature. The first large scale simulation of autonomous sensor networks (1000 agents) was conducted on this project, validating the performance of the developed algorithms. These algorithms and simulation tools are critical components of any decentralized autonomous network and have current and future national security applications, including distributed sensor networks for detection, estimation, and tracking problems, and large decentralized cyber-physical infrastructure such as the power grid.

42 ENGINEERING↗

A Conceptual Approach to Assimilating Remote Sensing Data to Improve Soil Moisture Profile Estimates in a Surface Flux/Hydrology Model: Overview - Part 1

Knowledge of the amount of water in the soil is of great importance to many earth science disciplines. Soil moisture is a key variable in controlling the exchange of water and energy between the land surface and the atmosphere. Thus, soil moisture information is valuable in a wide range of applications including weather and climate, runoff potential and flood control, early warning of droughts, irrigation, crop yield forecasting, soil erosion, reservoir management, geotechnical engineering, and water quality. Despite the importance of soil moisture information, widespread and continuous measurements of soil moisture are not possible today. Although many earth surface conditions can be measured from satellites, we still cannot adequately measure soil moisture from space. Research in soil moisture remote sensing began in the mid 1970s shortly after the surge in satellite development. Recent advances in remote sensing have shown that soil moisture can be measured, at least qualitatively, by several methods. Quantitative measurements of moisture in the soil surface layer have been most successful using both passive and active microwave remote sensing, although complications arise from surface roughness and vegetation type and density. Early attempts to measure soil moisture from space-borne microwave instruments were hindered by what is now considered sub-optimal wavelengths (shorter than 5 cm) and the coarse spatial resolution of the measurements. L-band frequencies between 1 and 3 GHz (10-30 cm) have been deemed optimal for detection of soil moisture in the upper few centimeters of soil. The Electronically Steered Thinned Array Radiometer (ESTAR), an aircraft-based instrument operating a 1,4 GHz, has shown great promise for soil moisture determination. Initiatives are underway to develop a similar instrument for space. Existing space-borne synthetic aperture radars (SARS) operating at C- and L-band have also shown some potential to detect surface wetness. The advantage of radar is its much higher resolution than passive microwave systems, but it is currently hampered by surface roughness effects and the lack of a good algorithm based on a single frequency and single polarization. In addition, its repeat frequency is generally low (about 40 days). In the meantime, two new radiometers offer some hope for remote sensing of soil moisture from space. The Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), launched in November 1997, possesses a 10.65 GHz channel and the Advanced Microwave Scanning Radiometer (AMSR) on both the ADEOS-11 and Earth Observing System AM-1 platforms to be launched in 1999 possesses a 6.9 GHz channel. Aside from issues about interference from vegetation, the coarse resolution of these data will provide considerable challenges pertaining to their application. The resolution of TMI is about 45 km and that of AMSR is about 70 km. These resolutions are grossly inconsistent with the scale of soil moisture processes and the spatial variability of factors that control soil moisture. Scale disparities such as these are forcing us to rethink how we assimilate data of various scales in hydrologic models. Of particular interest is how to assimilate soil moisture data by reconciling the scale disparity between what we can expect from present and future remote sensing measurements of soil moisture and modeling soil moisture processes. It is because of this disparity between the resolution of space-based sensors and the scale of data needed for capturing the spatial variability of soil moisture and related properties that remote sensing of soil moisture has not met with more widespread success. Within a single footprint of current sensors at the wavelengths optimal for this application, in most cases there is enormous heterogeneity in soil moisture created by differences in landcover, soils and topography, as well as variability in antecedent precipitation. It is difficult to interpret the meaning of 'mean' soil moisture under such conditions and even more difficult to apply such a value. Because of the non-linear relationships between near-surface soil moisture and other variables of interest, such as surface energy fluxes and runoff, mean soil moisture has little applicability at such large scales. It is for these reasons that the use of remote sensing in conjunction with a hydrologic model appears to be of benefit in capturing the complete spatial and temporal structure of soil moisture. This paper is Part I of a four-part series describing a method for intermittently assimilating remotely-sensed soil moisture information to improve performance of a distributed land surface hydrology model. The method, summarized in section II, involves the following components, each of which is detailed in the indicated section of the paper or subsequent papers in this series: Forward radiative transfer model methods (section II and Part IV); Use of a Kalman filter to assimilate remotely-sensed soil moisture estimates with the model profile (section II and Part IV); Application of a soil hydrology model to capture the continuous evolution of the soil moisture profile within and below the root zone (section III); Statistical aggregation techniques (section IV and Part II); Disaggregation techniques using a neural network approach (section IV and Part III); and Maximum likelihood and Bayesian algorithms for inversely solving for the soil moisture profile in the upper few cm (Part IV).

Crosson, William L.↗

PINN surrogate of Li-ion battery models for parameter inference, Part I: Implementation and multi-fidelity hierarchies for the single-particle model

To plan and optimize energy storage demands that account for Li-ion battery aging dynamics, techniques need to be developed to diagnose battery internal states accurately and rapidly. Here, this study seeks to reduce the computational resources needed to determine a battery's internal states by replacing physics-based Li-ion battery models - such as the single-particle model (SPM) and the pseudo-2D (P2D) model - with a physics-informed neural network (PINN) surrogate. The surrogate model makes high-throughput techniques, such as Bayesian calibration, tractable to determine battery internal parameters from voltage responses. This manuscript is the first of a two-part series that introduces PINN surrogates of Li-ion battery models for parameter inference (i.e., state-of-health diagnostics). In this first part, a method is presented for constructing a PINN surrogate of the SPM. A multi-fidelity hierarchical training, where several neural nets are trained with multiple physics-loss fidelities is shown to significantly improve the surrogate accuracy when only training on the governing equation residuals. The implementation is made available in a companion repository (https://github.com/NREL/PINNSTRIPES). The techniques used to develop a PINN surrogate of the SPM are extended in Part II for the PINN surrogate for the P2D battery model, and explore the Bayesian calibration capabilities of both surrogates.

25 ENERGY STORAGE↗

Determining the Bayesian optimal sampling strategy in a hierarchical system.

Consider a classic hierarchy tree as a basic model of a 'system-of-systems' network, where each node represents a component system (which may itself consist of a set of sub-systems). For this general composite system, we present a technique for computing the optimal testing strategy, which is based on Bayesian decision analysis. In previous work, we developed a Bayesian approach for computing the distribution of the reliability of a system-of-systems structure that uses test data and prior information. This allows for the determination of both an estimate of the reliability and a quantification of confidence in the estimate. Improving the accuracy of the reliability estimate and increasing the corresponding confidence require the collection of additional data. However, testing all possible sub-systems may not be cost-effective, feasible, or even necessary to achieve an improvement in the reliability estimate. To address this sampling issue, we formulate a Bayesian methodology that systematically determines the optimal sampling strategy under specified constraints and costs that will maximally improve the reliability estimate of the composite system, e.g., by reducing the variance of the reliability distribution. This methodology involves calculating the 'Bayes risk of a decision rule' for each available sampling strategy, where risk quantifies the relative effect that each sampling strategy could have on the reliability estimate. A general numerical algorithm is developed and tested using an example multicomponent system. The results show that the procedure scales linearly with the number of components available for testing.

99 GENERAL AND MISCELLANEOUS↗

Physics-Informed Machine Learning of Dynamical Systems for Efficient Bayesian Inference

Although the no-u-turn sampler (NUTS) is a widely adopted method for performing Bayesian inference, it requires numerous posterior gradients which can be expensive to compute in practice. Recently, there has been a significant interest in physics-based machine learning of dynamical (or Hamiltonian) systems and Hamiltonian neural networks (HNNs) is a noteworthy architecture. But these types of architectures have not been applied to solve Bayesian inference problems efficiently. We propose the use of HNNs for performing Bayesian inference efficiently without requiring numerous posterior gradients. We introduce latent variable outputs to HNNs (L-HNNs) for improved expressivity and reduced integration errors. We integrate L-HNNs in NUTS and further propose an online error monitoring scheme to prevent sampling degeneracy in regions where L-HNNs may have little training data. We demonstrate L-HNNs in NUTS with online error monitoring consider several complex high-dimensional posterior densities and compare its performance to NUTS.

97 MATHEMATICS AND COMPUTING↗

Training Spiking Neural Networks Using Combined Learning Approaches

Spiking neural networks (SNNs), the class of neural networks used in neuromorphic computing, are difficult to train using traditional back-propagation techniques. Spike timingdependent plasticity (STDP) is a biologically inspired learning mechanism that can be used to train SNNs. Evolutionary algorithms have also been demonstrated as a method for training SNNs. In this work, we explore the relationship between these two training methodologies. We evaluate STDP and evolutionary optimization as standalone methods for training networks, and also evaluate a combined approach where STDP weight updates are applied within an evolutionary algorithm. We also apply Bayesian hyperparameter optimization as a meta learner for each of the algorithms. We find that STDP by itself is not an ideal learning rule for randomly connected networks, while the inclusion of STDP within an evolutionary algorithm leads to similar performance, with a few interesting differences. This study suggests future work in understanding the relationship between network topology and learning rules.

Elbrecht, Daniel↗

Statistically-informed deep learning for gravitational wave parameter estimation

We introduce deep learning models to estimate the masses of the binary components of black hole mergers, $(m_1,m_2)$, and three astrophysical properties of the post-merger compact remnant, namely, the final spin, $a_\mathrm f$, and the frequency and damping time of the ringdown oscillations of the fundamental $\ell = m = 2$ bar mode, $(\omega_\mathrm R, \omega_\mathrm I)$. Our neural networks combine a modified WaveNet architecture with contrastive learning and normalizing flow. We validate these models against a Gaussian conjugate prior family whose posterior distribution is described by a closed analytical expression. Upon confirming that our models produce statistically consistent results, we used them to estimate the astrophysical parameters $(m_1,m_2, a_\mathrm f, \omega_\mathrm R, \omega_\mathrm I)$ of five binary black holes: GW150914, GW170104, GW170814, GW190521 and GW190630. We use PyCBC Inference to directly compare traditional Bayesian methodologies for parameter estimation with our deep learning based posterior distributions. Our results show that our neural network models predict posterior distributions that encode physical correlations, and that our data-driven median results and 90% confidence intervals are similar to those produced with gravitational wave Bayesian analyses. This methodology requires a single V100 NVIDIA GPU to produce median values and posterior distributions within two milliseconds for each event. Furthermore, this neural network, and a tutorial for its use, are available at the Data and Learning Hub for Science.

79 ASTRONOMY AND ASTROPHYSICS↗

DECODING SPECTRAL ENERGY DISTRIBUTIONS OF DUST-OBSCURED STARBURST-ACTIVE GALACTIC NUCLEUS

We present BayeSED, a general purpose tool for Bayesian analysis of spectral energy distributions (SEDs) using pre-existing model SED libraries or their linear combinations. The artificial neural networks, principal component analysis, and multimodal-nested sampling (MultiNest) techniques are employed to allow the highly efficient sampling of posterior distribution and the calculation of Bayesian evidence. As a demonstration, we apply this tool to a sample of hyperluminous infrared galaxies (HLIRGs). The Bayesian evidence obtained for a pure starburst, a pure active galactic nucleus (AGN), and a linear combination of starburst+AGN models show that the starburst+AGN model has the highest evidence for all galaxies in this sample. The Bayesian evidence for the three models and the estimated contributions of starbursts and AGNs to infrared luminosity show that HLIRGs can be classified into two groups: one dominated by starbursts and the other dominated by AGNs. Other parameters and corresponding uncertainties about starbursts and AGNs are also estimated using the model with the highest Bayesian evidence. We find that the starburst region of the HLIRGs dominated by starbursts tends to be more compact and has a higher fraction of OB stars than that of HLIRGs dominated by AGNs. Meanwhile, the AGN torus of the HLIRGs dominated by AGNs tends to be more dusty than that of HLIRGs dominated by starbursts. These results are consistent with previous researches, but need to be tested further with larger samples. Overall, we believe that BayeSED could be a reliable and efficient tool for exploring the nature of complex systems such as dust-obscured starburst-AGN composite systems by decoding their SEDs.

79 ASTRONOMY AND ASTROPHYSICS↗

Toward machine-learning-assisted PW-class high-repetition-rate experiments with solid targets

We present progress in utilizing a machine learning (ML) assisted optimization framework to study the trends in a parameter space defined by spectrally shaped, high-intensity, petawatt-class (8 J, 45 fs) laser pulses interacting with solid targets and give the first simulation-based overview of predicted trends. A neural network (NN) incorporating uncertainty quantification is trained to predict the number of hot electrons generated by the laser–target interaction as a function of pulse shaping parameters. The predictions of this NN serve as the basis function for a Bayesian optimization framework to navigate this space. For post-experimental evaluation, we compare two separate neural network (NN) models. One is based solely on data from experiments, and the other is trained only on ensemble particle-in-cell simulations. Reviewing the predicted and observed trends across the experiment-capable laser parameter search space, we find that both ML models predict a maximal increase in hot electron generation at a level of approximately 12%–18%; however, no statistically significant enhancement was observed in experiments. On direct comparison of the NN models, the average discrepancy is 8.5%, with a maximum of 30%. Since shot-to-shot fluctuations in experiments affect the observations, we evaluate the behavior of our optimization framework by performing virtual experiments that vary the number of repeated observations and the noise levels. Here, we discuss the implications of such a framework for future autonomous exploration platforms in high-repetition-rate experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Machine learning for continuous quantum error correction on superconducting qubits

Abstract Continuous quantum error correction has been found to have certain advantages over discrete quantum error correction, such as a reduction in hardware resources and the elimination of error mechanisms introduced by having entangling gates and ancilla qubits. We propose a machine learning algorithm for continuous quantum error correction that is based on the use of a recurrent neural network to identify bit-flip errors from continuous noisy syndrome measurements. The algorithm is designed to operate on measurement signals deviating from the ideal behavior in which the mean value corresponds to a code syndrome value and the measurement has white noise. We analyze continuous measurements taken from a superconducting architecture using three transmon qubits to identify three significant practical examples of non-ideal behavior, namely auto-correlation at temporal short lags, transient syndrome dynamics after each bit-flip, and drift in the steady-state syndrome values over the course of many experiments. Based on these real-world imperfections, we generate synthetic measurement signals from which to train the recurrent neural network, and then test its proficiency when implementing active error correction, comparing this with a traditional double threshold scheme and a discrete Bayesian classifier. The results show that our machine learning protocol is able to outperform the double threshold protocol across all tests, achieving a final state fidelity comparable to the discrete Bayesian classifier.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Thinking Bayesian for plasma physicists

Bayesian statistics offers a powerful technique for plasma physicists to infer knowledge from the heterogeneous data types encountered. To explain this power, a simple example, Gaussian Process Regression, and the application of Bayesian statistics to inverse problems are explained. The likelihood is the key distribution because it contains the data model, or theoretic predictions, of the desired quantities. By using prior knowledge, the distribution of the inferred quantities of interest based on the data given can be inferred. Because it is a distribution of inferred quantities given the data and not a single prediction, uncertainty quantification is a natural consequence of Bayesian statistics. The benefits of machine learning in developing surrogate models for solving inverse problems are discussed, as well as progress in quantitatively understanding the errors that such a model introduces.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

BIhNNs

The code enables to perform Bayesian inference in an efficient manner through the use of Hamiltonian Neural Networks (HNNs), Deep Neural Networks (DNNs), Neural ODEs, and Symplectic Neural Networks (SympNets) used with state-of-the-art sampling schemes like Hamiltonian Monte Carlo (HMC) and the No-U-Turn-Sampler (NUTS).

Dhulipala, Som↗

Localization of a radioactive source in an urban environment using Bayesian Metropolis methods [Bayesian Metropolis methods for source localization in an urban environment]

Here, we present a method for localizing an unknown source of radiation in an urban environment using a distributed detector network. This method employs statistical parameter estimation techniques, relying on an approximation for the response of a detector to the source based on a simplified model of the underlying transport phenomena, combined with a Metropolis-type sampler that is modified to propagate the effect of fixed epistemic uncertainties in the material macroscopic cross sections of objects in the scene. We apply this technique to data collected during a measurement campaign conducted in a realistic analog for an urban scene using a network of six mobile detectors. Our initial results are able to localize the source to within approximately 8 m over a scene of size 300 m 200 m in two independent trials with 30 min count times, including a characterization of the uncertainty associated with the poorly known macroscopic cross sections of objects in the scene. In these measurements, the nearest detectors were between 20 m to 30 m from the source, and recorded count rates between approximately 3 and 13 times background. A few detectors had line-of-sight to the source, while the majority were obscured by objects present in the scene. After extending our model to account for the orientation of the detectors and correcting for anomalies in the measurement data we were able to further improve the localization accuracy to approximately 2 m in both trials.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗