Search NASA⌕ Search

SEARCH · Search NASA

Results for “Bayesian Methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64

Stan : A Probabilistic Programming Language

Stan is a probabilistic programming language for specifying statistical models. A Stan program imperatively defines a log probability function over parameters conditioned on specified data and constants. As of version 2.14.0, Stan provides full Bayesian inference for continuous-variable models through Markov chain Monte Carlo methods such as the No-U-Turn sampler, an adaptive form of Hamiltonian Monte Carlo sampling. Penalized maximum likelihood estimates are calculated using optimization methods such as the limited memory Broyden-Fletcher-Goldfarb-Shanno algorithm. Stan is also a platform for computing log densities and their gradients and Hessians, which can be used in alternative algorithms such as variational Bayes, expectation propagation, and marginal inference using approximate integration. To this end, Stan is set up so that the densities, gradients, and Hessians, along with intermediate quantities of the algorithm such as acceptance probabilities, are easily accessible. Stan can also be called from the command line using the cmdstan package, through R using the rstan package, and through Python using the pystan package. All three interfaces support sampling and optimization-based inference with diagnostics and posterior analysis. rstan and pystan also provide access to log probabilities, gradients, Hessians, parameter transforms, and specialized plotting.

97 MATHEMATICS AND COMPUTING↗

Advanced Semi-Supervised Learning with Uncertainty Estimation for Phase Identification in Distribution Systems

The integration of advanced metering infrastructure (AMI) into power distribution networks generates valuable data for tasks such as phase identification; however, the limited and unreliable availability of labeled data in the form of customer phase connectivity presents challenges. To address this issue, we propose a semi-supervised learning (SSL) framework that effectively leverages labeled and unlabeled data. Our approach incorporates self-training, label spreading, and Bayesian neural networks (BNNs) to enhance phase identification with AMI data. Our method uses an ensemble of multilayer perceptron classifiers in a self-training setup, iteratively adding high-confidence pseudo-labels to improve robustness. We also apply label spread to propagate labels based on data similarity, which enhances generalization across diverse distributions. In addition, we employ a BNNs with uncertainty estimation, boosting confidence in predictions and reducing phase identification errors. In our case study, we achieved approximately 98% +/- 0.08 accuracy with uncertainty using minimal and unreliable labeled data from a real U.S. utility, Duquesne Light Company. Our SSL approach, combined with uncertainty estimation, provides an efficient solution for phase identification in AMI data, ultimately improving the reliability of smart grid applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Advanced Semi-Supervised Learning With Uncertainty Estimation for Phase Identification in Distribution Systems

The integration of advanced metering infrastructure (AMI) into power distribution networks generates valuable data for tasks such as phase identification; however, the limited and unreliable availability of labeled data in the form of customer phase connectivity presents challenges. To address this issue, we propose a semi-supervised learning (SSL) framework that effectively leverages labeled and unlabeled data. Our approach incorporates self-training, label spreading, and Bayesian neural networks (BNNs) to enhance phase identification with AMI data. Our method uses an ensemble of multilayer perceptron classifiers in a self-training setup, iteratively adding high-confidence pseudo-labels to improve robustness. We also apply label spread to propagate labels based on data similarity, which enhances generalization across diverse distributions. In addition, we employ a BNNs with uncertainty estimation, boosting confidence in predictions and reducing phase identification errors. In our case study, we achieved approximately 98% +/- 0.08 accuracy with uncertainty using minimal and unreliable labeled data from a real U.S. utility, Duquesne Light Company. Our SSL approach, combined with uncertainty estimation, provides an efficient solution for phase identification in AMI data, ultimately improving the reliability of smart grid applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Interpreting High-resolution Spectroscopy of Exoplanets using Cross-correlations and Supervised Machine Learning

We present a new method for performing atmospheric retrieval on ground-based, high-resolution data of exoplanets. Our method combines cross-correlation functions with a random forest, a supervised machine-learning technique, to overcome challenges associated with high-resolution data. A series of cross-correlation functions are concatenated to give a “CCF-sequence” for each model atmosphere, which reduces the dimensionality by a factor of ∼100. The random forest, trained on our grid of ∼65,000 models, provides a likelihood-free method of retrieval. The precomputed grid spans 31 values of both temperature and metallicity, and incorporates a realistic noise model. We apply our method to HARPS-N observations of the ultra-hot Jupiter KELT-9b and obtain a metallicity consistent with solar (logM = − 0.2 ± 0.2). Our retrieved transit chord temperature (T=6000{sub −200}{sup +0}K) is unreliable as strong ion lines lie outside of the extent of the training set, which we interpret as being indicative of missing physics in our atmospheric model. We compare our method to traditional nested sampling, as well as other machine-learning techniques, such as Bayesian neural networks. We demonstrate that the likelihood-free aspect of the random forest makes it more robust than nested sampling to different error distributions, and that the Bayesian neural network we tested is unable to reproduce complex posteriors. We also address the claim in Cobb et al. 2019 that our random forest retrieval technique can be overconfident but incorrect. We show that this is an artifact of the training set, rather than of the machine-learning method, and that the posteriors agree with those obtained using nested sampling.

79 ASTRONOMY AND ASTROPHYSICS↗

Making Steppingstones out of Stumbling Blocks: A Bayesian Model Evidence Estimator with Application to Groundwater Transport Model Selection

Bayesian model evidence (BME) is a measure of the average fit of a model to observation data given all the parameter values that the model can assume. By accounting for the trade-off between goodness-of-fit and model complexity, BME is used for model selection and model averaging purposes. For strict Bayesian computation, the theoretically unbiased Monte Carlo based numerical estimators are preferred over semi-analytical solutions. This study examines five BME numerical estimators and asks how accurate estimation of the BME is important for penalizing model complexity. The limiting cases for numerical BME estimators are the prior sampling arithmetic mean estimator (AM) and the posterior sampling harmonic mean (HM) estimator, which are straightforward to implement, yet they result in underestimation and overestimation, respectively. We also consider the path sampling methods of thermodynamic integration (TI) and steppingstone sampling (SS) that sample multiple intermediate distributions that link the prior and the posterior. Although TI and SS are theoretically unbiased estimators, they could have a bias in practice arising from numerical implementation. For example, sampling errors of some intermediate distributions can introduce bias. We propose a variant of SS, namely the multiple one-steppingstone sampling (MOSS) that is less sensitive to sampling errors. We evaluate these five estimators using a groundwater transport model selection problem. SS and MOSS give the least biased BME estimation at an efficient computational cost. If the estimated BME has a bias that covariates with the true BME, this would not be a problem because we are interested in BME ratios and not their absolute values. On the contrary, the results show that BME estimation bias can be a function of model complexity. Thus, biased BME estimation results in inaccurate penalization of more complex models, which changes the model ranking. This was less observed with SS and MOSS as with the three other methods.

54 ENVIRONMENTAL SCIENCES↗

Robust Estimation of Electron Density From Anatomic Magnetic Resonance Imaging of the Brain Using a Unifying Multi-Atlas Approach

Purpose: To develop a reliable method to estimate electron density based on anatomic magnetic resonance imaging (MRI) of the brain. Methods and Materials: We proposed a unifying multi-atlas approach for electron density estimation based on standard T1- and T2-weighted MRI. First, a composite atlas was constructed through a voxelwise matching process using multiple atlases, with the goal of mitigating effects of inherent anatomic variations between patients. Next we computed for each voxel 2 kinds of conditional probabilities: (1) electron density given its image intensity on T1- and T2-weighted MR images; and (2) electron density given its spatial location in a reference anatomy, obtained by deformable image registration. These were combined into a unifying posterior probability density function using the Bayesian formalism, which provided the optimal estimates for electron density. We evaluated the method on 10 patients using leave-one-patient-out cross-validation. Receiver operating characteristic analyses for detecting different tissue types were performed. Results: The proposed method significantly reduced the errors in electron density estimation, with a mean absolute Hounsfield unit error of 119, compared with 140 and 144 (P<.0001) using conventional T1-weighted intensity and geometry-based approaches, respectively. For detection of bony anatomy, the proposed method achieved an 89% area under the curve, 86% sensitivity, 88% specificity, and 90% accuracy, which improved upon intensity and geometry-based approaches (area under the curve: 79% and 80%, respectively). Conclusion: The proposed multi-atlas approach provides robust electron density estimation and bone detection based on anatomic MRI. If validated on a larger population, our work could enable the use of MRI as a primary modality for radiation treatment planning.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗

Nuclear Data Adjustment for Nonlinear Applications in the OECD/NEA WPNCS SG14 Benchmark—A Bayesian Inverse UQ-Based Approach for Data Assimilation

The Organisation for Economic Co-operation and Development Working Party on Nuclear Criticality Safety has proposed a benchmark exercise to assess the performance of current nuclear data adjustment techniques applied to nonlinear applications and experiments with low correlation to applications. This work introduces Bayesian inverse uncertainty quantification (IUQ) employing scientific machine learning surrogate models as a method for nuclear data adjustments in this benchmark, and compares IUQ to the more traditional methods of generalized linear least squares (GLLS) and Monte Carlo Bayes (MOCABA). Posterior predictions from IUQ showed agreement with GLLS and MOCABA for linear applications. Here, when comparing GLLS, MOCABA, and IUQ posterior predictions to computed model responses using adjusted parameters, we observe that the GLLS predictions failed to replicate the computed response distributions for nonlinear applications, while MOCABA showed near agreement, and IUQ used the computed model responses directly. We also discuss observations on why experiments with low correlation to applications can be informative to nuclear data adjustments and identify some properties useful in selecting experiments for inclusion in nuclear data adjustment. Performance in this benchmark indicates potential for Bayesian IUQ in nuclear data adjustments.

Bayesian calibration↗

Integration of seismic-pressure-petrophysics inversion of continuous active-seismic monitoring data for monitoring and quantifying CO 2 plume (Final Report)

The overall objective of this project is to develop and validate an integrated package of joint seismic-pressure-petrophysics inversion (jSPPI) of continuous active-source seismic monitoring dataset capable of providing real-time monitoring of CO 2 plume during geologic carbon sequestration (GCS). The three specific developments include: (a) the methodologies for fast seismic full waveform inversion of continuous active source seismic monitoring, (CASSM) datasets for simultaneously estimating velocity and attenuation, and with data assimilation; (b) joint Bayesian petrophysical inversion of seismic models and pressure data for providing and updating CO 2 saturation models; (c) the methods using multiple datasets including (Crainfield and Frio-II borehole) synthetic, laboratory, and field CASSM datasets. The outcomes of jSPPI include (a) a workflow for processing CASSM data, (b) Bayesian inversion algorithms using CASSM data and pressure response data, and (c) integration with data assimilation algorithms for continuously updating site-specific models used for prediction and reservoir management. The validation of joint FWI will be conducted using synthetic models based on the Cranfield and Frio experiments as well as field CASSM datasets collected as part of the Frio-II pilot injection. To quantify and map the mass and distribution of CO 2 (saturation), we will jointly invert velocity and attenuation measurements from the FWI with a Bayesian approach using a rock physics model for attenuation (e.g., White’s attenuation model with two selected patch sizes (White, 1976; Dutta and Seriff, 1979)). The Bayesian inversion will be applied to each time step in the CASSM survey in an updating scheme, which integrates with an ensemble of reservoir simulations at each step. A more complete experimental validation dataset will be collected as part of a mesoscale (2-3 m) gas-CO 2 injection experiment utilizing a higher frequency version of the CASSM system developed for laboratory studies; the integrated inversion will be demonstrated using this dataset which will provide both a dense geometry as well as more precise secondary confirmation measurements (e.g. saturation) typically not available in the field. The resulting real-time map of CO 2 saturation is able to provide a deeper scientific understanding of the complex, time-varying dynamics of subsurface fluid flow migration path as well as the rapid detection of CO 2 leakage hazards.

25 ENERGY STORAGE↗

Bayesian Treed Calibration: An Application to Carbon Capture With AX Sorbent

In cases where field or experimental measurements are not available, computer models can model real physical or engineering systems to reproduce their outcomes. They are usually calibrated in light of experimental data to create a better representation of the real system. Statistical methods, based on Gaussian processes, for calibration and prediction have been especially important when the computer models are expensive and experimental data limited. In this paper, we develop the Bayesian treed calibration (BTC) as an extension of standard Gaussian process calibration methods to deal with non-stationarity computer models and/or their discrepancy from the field (or experimental) data. Our proposed method partitions both the calibration and observable input space, based on a binary tree partitioning, into sub-regions where existing model calibration methods can be applied to connect a computer model with the real system. The estimation of the parameters in the proposed model is carried out using Markov chain Monte Carlo (MCMC) computational techniques. Different strategies have been applied to improve mixing. We illustrate our method in two artificial examples and a real application that concerns the capture of carbon dioxide with AX amine based sorbents. The source code and the examples analyzed in this paper are available as part of the supplementary materials.

Bayesian tree↗

Improving Weak Lensing Mass Map Reconstructions using Gaussian and Sparsity Priors: Application to DES SV

Mapping the underlying density field, including non-visible dark matter, using weak gravitational lensing measurements is now a standard tool in cosmology. Due to its importance to the science results of current and upcoming surveys, the quality of the convergence reconstruction methods should be well understood. We compare three methods: Kaiser–Squires (KS), Wiener filter, and Glimpse. Kaiser–Squires is a direct inversion, not accounting for survey masks or noise. The Wiener filter is well-motivated for Gaussian density fields in a Bayesian framework. Glimpse uses sparsity, aiming to reconstruct non-linearities in the density field. We compare these methods with several tests using public Dark Energy Survey (DES) Science Verification (SV) data and realistic DES simulations. The Wiener filter and Glimpse offer substantial improvements over smoothed Kaiser–Squires with a range of metrics. Both the Wiener filter and Glimpse convergence reconstructions show a 12 per cent improvement in Pearson correlation with the underlying truth from simulations. To compare the mapping methods’ abilities to find mass peaks, we measure the difference between peak counts from simulated ΛCDM shear catalogues and catalogues with no mass fluctuations (a standard data vector when inferring cosmology from peak statistics); the maximum signal-to-noise of these peak statistics is increased by a factor of 3.5 for the Wiener filter and 9 for Glimpse. With simulations, we measure the reconstruction of the harmonic phases; the phase residuals’ concentration is improved 17 per cent by Glimpse and 18 per cent by the Wiener filter. The correlationbetween reconstructions from data and foreground redMaPPer clusters is increased 18 per cent by the Wiener filter and 32 per cent by Glimpse.

79 ASTRONOMY AND ASTROPHYSICS↗

Descriptor Aided Bayesian Optimization for Many-Level Qualitative Variables With Materials Design Applications

Abstract Engineering design often involves qualitative and quantitative design variables, which requires systematic methods for the exploration of these mixed-variable design spaces. Expensive simulation techniques, such as those required to evaluate optimization objectives in materials design applications, constitute the main portion of the cost of the design process and underline the need for efficient search strategies—Bayesian optimization (BO) being one of the most widely adopted. Although recent developments in mixed-variable Bayesian optimization have shown promise, the effects of dimensionality of qualitative variables have not been well studied. High-dimensional qualitative variables, i.e., with many levels, impose a large design cost as they typically require a larger dataset to quantify the effect of each level on the optimization objective. We address this challenge by leveraging domain knowledge about underlying physical descriptors, which embody the physics of the underlying physical phenomena, to infer the effect of unobserved levels that have not been sampled yet. We show that physical descriptors can be intuitively embedded into the latent variable Gaussian process approach—a mixed-variable GP modeling technique—and used to selectively explore levels of qualitative variables in the Bayesian optimization framework. This physics-informed approach is particularly useful when one or more qualitative variables are high dimensional (many-level) and the modeling dataset is small, containing observations for only a subset of levels. Through a combination of mathematical test functions and materials design applications, our method is shown to be robust to certain types of incomplete domain knowledge and significantly reduces the design cost for problems with high-dimensional qualitative variables.

Engineering↗

Nonparametric Bayesian Modeling for Automated Database Schema Matching

The problem of merging databases arises in many government and commercial applications. Schema matching, a common first step, identifies equivalent fields between databases. We introduce a schema matching framework that builds nonparametric Bayesian models for each field and compares them by computing the probability that a single model could have generated both fields. Our experiments show that our method is more accurate and faster than the existing instance-based matching algorithms in part because of the use of nonparametric Bayesian models.

Bayesian↗

Expectation-propagation for weak radionuclide identification at radiation portal monitors

We propose a sparsity-promoting Bayesian algorithm capable of identifying radionuclide signatures from weak sources in the presence of a high radiation background. The proposed method is relevant to radiation identification for security applications. In such scenarios, the background typically consists of terrestrial, cosmic, and cosmogenic radiation that may cause false positive responses. We evaluate the new Bayesian approach using gamma-ray data and are able to identify weapons-grade plutonium, masked by naturally-occurring radioactive material (NORM), in a measurement time of a few seconds. We demonstrate this identification capability using organic scintillators (stilbene crystals and EJ-309 liquid scintillators), which do not provide direct, high-resolution, source spectroscopic information. Compared to the EJ-309 detector, the stilbene-based detector exhibits a lower identification error, on average, owing to its better energy resolution. Organic scintillators are used within radiation portal monitors to detect gamma rays emitted from conveyances crossing ports of entry. The described method is therefore applicable to radiation portal monitors deployed in the field and could improve their threat discrimination capability by minimizing “nuisance” alarms produced either by NORM-bearing materials found in shipped cargoes, such as ceramics and fertilizers, or radionuclides in recently treated nuclear medicine patients.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

AI-assisted transport of radioactive ion beams

Beams of radioactive heavy ions allow researchers to study rare and unstable atomic nuclei, shedding light into the internal structure of exotic nuclei and on how chemical elements are formed in stars. However, the extraction and transport of radioactive beams rely on time-consuming expert-driven tuning methods, where hundreds of parameters are manually optimized. Here, in this study, we introduce a system that employs Artificial Intelligence (AI), specifically utilizing Bayesian Optimization, to assist in the transport process of radioactive beams. We apply our methodology to real-life scenarios showing advantages when compared with standard tuning methods. This AI-assisted approach can be extended to other radioactive beam facilities around the world to improve operational efficiency and enhance scientific output.

43 PARTICLE ACCELERATORS↗

Efficient Neural Network Approaches for Conditional Optimal Transport with Applications in Bayesian Inference

In this work, we present two neural network approaches that approximate the solutions of static and dynamic conditional optimal transport (COT) problems. Both approaches enable conditional sampling and conditional density estimation, which are core tasks in Bayesian inference—particularly in the simulation-based (“likelihood-free”) setting. Our methods represent the target conditional distribution as a transformation of a tractable reference distribution. Obtaining such a transformation, chosen here to be an approximation of the COT map, is computationally challenging even in moderate dimensions. To improve scalability, our numerical algorithms use neural networks to parameterize candidate maps and further exploit the structure of the COT problem. Our static approach approximates the map as the gradient of a partially input convex neural network. It uses a novel numerical implementation to increase computational efficiency compared to state-of-the-art alternatives. Our dynamic approach approximates the conditional optimal transport via the flow map of a regularized neural ODE; compared to the static approach, it is slower to train but offers more modeling choices and can lead to faster sampling. We demonstrate both algorithms numerically, comparing them with competing state-of-the-art approaches, using benchmark datasets and simulation-based Bayesian inverse problems.

97 MATHEMATICS AND COMPUTING↗

Identifying Bayesian optimal experiments for uncertain biochemical pathway models

Abstract Pharmacodynamic (PD) models are mathematical models of cellular reaction networks that include drug mechanisms of action. These models are useful for studying predictive therapeutic outcomes of novel drug therapies in silico. However, PD models are known to possess significant uncertainty with respect to constituent parameter data, leading to uncertainty in the model predictions. Furthermore, experimental data to calibrate these models is often limited or unavailable for novel pathways. In this study, we present a Bayesian optimal experimental design approach for improving PD model prediction accuracy. We then apply our method using simulated experimental data to account for uncertainty in hypothetical laboratory measurements. This leads to a probabilistic prediction of drug performance and a quantitative measure of which prospective laboratory experiment will optimally reduce prediction uncertainty in the PD model. The methods proposed here provide a way forward for uncertainty quantification and guided experimental design for models of novel biological pathways.

97 MATHEMATICS AND COMPUTING↗

From Optimization to Sampling Through Gradient Flows

Optimization and sampling algorithms play a central role in science and engineering as they enable finding optimal predictions, policies, and recommendations, as well as expected and equilibrium states of complex systems. The notion of “optimality” is formalized by the choice of an objective function, while the notion of an “expected” state is specified by a probabilistic model for the distribution of states. Optimizing rugged objective functions and sampling multimodal distributions is computationally challenging, especially in high-dimensional problems. Here, for this reason, many optimization and sampling methods have been developed by researchers working in disparate fields such as Bayesian statistics, molecular dynamics, genetics, quantum chemistry, machine learning, weather forecasting, econometrics, and medical imaging.

Trillos, N. García↗