Search NASA⌕ Search

SEARCH · Search NASA

Results for “Hamiltonian Monte Carlo”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Constraining neutrino oscillation and interaction parameters with the NOvA Near Detector and Far Detector data using Markov Chain Monte Carlo

This thesis reports a constraint of the neutrino oscillation parameters $\Delta m^{2}_{32}$, $\sin^2 \theta_{23}$, and $\delta_{CP}$ using the NuMI Off-Axis $\nu$ Appearance (NOvA) experiment's Near Detector (ND) data and Far Detector (FD) fake data set simultaneously. This thesis also reports a constraint on NOvA's systematic uncertainty model solely with its Near Detector data. The Hamiltonian Monte Carlo algorithm is used to estimate Bayesian Credible Intervals for the oscillation and interaction parameters. The $1\sigma$ Credible Intervals for $\sin^2 \theta_{23}$ are $(0.44, 0.512)$ $\cup$ $(0.536, 0.56)$, for $\Delta m^{2}_{32}$ $(2.41 \times 10^{-3}$ eV$^2,\ 2.52 \times 10^{-3}$ eV$^2)$, and for $\delta_{CP}$ $(0.74\pi,\ 1.1\pi)$ $\cup$ $(1.38\pi,\ 1.58\pi)$. The statistical power of the ND data constrains NOvA's interaction parameters, while the FD fake data constrains the oscillation parameters. This is the first analysis within NOvA to constrain the ND and FD prediction sim ultaneously, and to investigate the neutrino interaction modeling in the context of constraining the oscillation parameters. To constrain the ND data requires a sophisticated understanding of the neutrino interaction modeling and its uncertainties. The interested reader is advised to focus on Chapters 4 and 6, which discuss the ND selection, uncertainties, and ND-only fits to data. The reader interested in oscillation parameter constraints will find this in Chapter 7.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Quantification and visualization of uncertainties in reconstructed penumbral images of implosions at Omega

Penumbral imaging is a technique used in plasma diagnostics in which a radiation source shines through one or more large apertures onto a detector. To interpret a penumbral image, one must reconstruct it to recover the original source. The inferred source always has some error due to noise in the image and uncertainty in the instrument geometry. Interpreting the inferred source thus requires quantification of that inference’s uncertainty. Markov chain Monte Carlo algorithms have been used to quantify uncertainty for similar problems but have never been used for the inference of the shape of an image. Because of this, there are no commonly accepted ways of visualizing uncertainty in two-dimensional data. This paper demonstrates the application of the Hamiltonian Monte Carlo algorithm to the reconstruction of penumbral images of fusion implosions and presents ways to visualize the uncertainty in the reconstructed source. This methodology enables more rigorous analysis of penumbral images than has been done in the past.

Instruments & Instrumentation↗
DOE OSTI2026

E-PINNs: Epistemic Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) have demonstrated promise as a framework for solving forward and inverse problems involving partial differential equations. Despite recent progress in the field, it remains challenging to quantify uncertainty in these networks. While techniques such as Bayesian PINNs (B-PINNs) provide a principled approach to capturing epistemic uncertainty through Bayesian inference, they can be computationally expensive for large-scale applications. In this work, we propose Epistemic Physics-Informed Neural Networks (E-PINNs), a framework that uses a small network, the epinet, to efficiently quantify epistemic uncertainty in PINNs. The proposed approach works as an add-on to existing, pre-trained PINNs with a small computational overhead. We demonstrate the applicability of the proposed framework in various test cases and compare the results with B-PINNs using Hamiltonian Monte Carlo (HMC) posterior estimation and dropout-equipped PINNs (Dropout-PINNs). In our experiments, E-PINNs achieve calibrated coverage with competitive sharpness at substantially lower cost. We demonstrate that when B-PINNs produce narrower bands, they under-cover in our tests. E-PINNs also show better calibration than Dropout-PINNs in these examples, indicating a favorable accuracy-efficiency trade-off.

AI for Science↗
DOE OSTI2022

Learning functional priors and posteriors from data and physics

In this work, we develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physicsinformed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field using experimental data from the Norwegian Deepwater Programme (NDP). The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In summary, the proposed method is capable of learning flexible functional priors, e.g., both Gaussian and non-Gaussian process, and can be readily extended to big data problems by enabling mini-batch training using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

A Differentiable Hydrology Approach for Modeling With Time-Varying Parameters

In the analysis of observational data with mathematical models, it is often desirable to let one or more model parameters vary with time to account for changing environmental conditions or to offer more flexibility. A standard approach for dynamic models is to allow for the number of parameters to grow as a function of time, leading to a high-dimensional inference problem for long data records. We propose using Hamiltonian Monte Carlo, a gradient-based Markov chain Monte Carlo (MCMC) method, to perform Bayesian inference for time-varying parameters (TVP) in hydrology models. As derivatives of model error functions with regard to parameters are not available in closed form, we implemented the GR4J rainfall-runoff models in Theano/PyMC3, and Jax/NumPyro, allowing for the application of automatic differentiation techniques. A simulation experiment assessing the viability of these methods for recovering underlying temporal variation in parameters indicates that a discrete Gaussian random walk-based prior appears to be best suited across a range of scenarios while the continuous-time Gaussian process compares relatively poorly. In an analysis of 20 years of daily real-world streamflow records from the Model Parameter Estimation Experiment (MOPEX) hydrology data set with increasing data sparsity and at multiple temporal resolutions, we find that TVP inference with gradient-based MCMC is a flexible and reliable approach for analyzing hydrology models with dynamic parameter sets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Ferroelectric phase-transition frustration near a tricritical composition point

Phase transition describes a mutational behavior of matter states at a critical transition temperature or external field. Despite the phase-transition orders are well sorted by classic thermodynamic theory, ambiguous situations interposed between the first- and second-order transitions were exposed one after another. Here, we report discovery of phase-transition frustration near a tricritical composition point in ferroelectric Pb(Zr 1-x Ti x )O 3 . Our multi-scale transmission electron microscopy characterization reveals a number of geometrically frustrated microstructure features such as self-assembled hierarchical domain structure, degeneracy of mesoscale domain tetragonality and decoupled polarization-strain relationship. Associated with deviation from the classic mean-field theory, dielectric critical exponent anomalies and temperature dependent birefringence data unveil that the frustrated transition order stems from intricate competition of short-range polar orders and their decoupling to long-range lattice deformation. With supports from effective Hamiltonian Monte Carlo simulations, our findings point out a potentially universal mechanism to comprehend the abnormal critical phenomena occurring in phase-transition materials.

42 ENGINEERING↗
DOE OSTI2025

A compact x-ray spectrometer for measurements of electron temperature distributions in inertial confinement fusion implosions at OMEGA

The Wedge Range Filter (WRF), commonly used for proton spectroscopy at the OMEGA Laser Facility and National Ignition Facility, is adapted to measure the x-ray continuum spectrum through transmission measurement using a continuous-gradient filter. Continuum x rays emitted from the hotspot of an implosion contain information about the plasma composition and electron temperature. The WRF data are leveraged to probe this distribution, specifically the electron temperature distribution. In this work, the data recorded with the WRF are forward modeled using a temperature distribution model folded with the WRF response function. An uncertainty analysis is conducted through a Bayesian regression algorithm using a Hamiltonian Monte Carlo sampler. This analysis enables the uncertainties in the instrument response to be folded into the uncertainty estimation of the electron temperature and absolute x-ray emission. Data analysis for a series of OMEGA implosions is presented and compared with radiation hydrodynamic simulations.

Lasers↗
DOE OSTI2024

Neural simulation-based inference of the neutron star equation of state directly from telescope spectra

Neutron stars provide a unique opportunity to study strongly interacting matter under extreme density conditions. The intricacies of matter inside neutron stars and their equation of state are not directly visible, but determine bulk properties, such as mass and radius, which affect the star's thermal X-ray emissions. However, the telescope spectra of these emissions are also affected by the stellar distance, hydrogen column, and effective surface temperature, which are not always well-constrained. Uncertainties on these nuisance parameters must be accounted for when making a robust estimation of the equation of state. In this study, we develop a novel methodology that, for the first time, can infer the full posterior distribution of both the equation of state and nuisance parameters directly from telescope observations. This method relies on the use of neural likelihood estimation, in which normalizing flows use samples of simulated telescope data to learn the likelihood of the neutron star spectra as a function of these parameters, coupled with Hamiltonian Monte Carlo methods to efficiently sample from the corresponding posterior distribution. Our approach surpasses the accuracy of previous methods, improves the interpretability of the results by providing access to the full posterior distribution, and naturally scales to a growing number of neutron star observations expected in the coming years.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

The information content of projected galaxy fields

ABSTRACT The power spectrum of the non-linearly evolved large-scale mass distribution recovers only a minority of the information available on the mass fluctuation amplitude. We investigate the recovery of this information in 2D ‘slabs’ of the mass distribution averaged over ≈100 h−1 Mpc along the line of sight, as might be obtained from photometric redshift surveys. We demonstrate a Hamiltonian Monte Carlo method to reconstruct the non-Gaussian mass distribution in slabs, under the assumption that the projected field is a point-transformed Gaussian random field, Poisson-sampled by galaxies. When applied to the Quijote N-body suite at z = 0.5 and at a transverse resolution of 2 h−1 Mpc, the method recovers ∼30 times more information than the 2D power spectrum in the well-sampled limit, recovering the Gaussian limit on information. At a more realistic galaxy sampling density of 0.01 h3 Mpc−3, shot noise reduces the information gain to a factor of 5 improvement over the power spectrum at resolutions of 4 h−1 Mpc or smaller.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Differentiable stochastic halo occupation distribution

ABSTRACT In this work, we demonstrate how differentiable stochastic sampling techniques developed in the context of deep reinforcement learning can be used to perform efficient parameter inference over stochastic, simulation-based, forward models. As a particular example, we focus on the problem of estimating parameters of halo occupation distribution (HOD) models that are used to connect galaxies with their dark matter haloes. Using a combination of continuous relaxation and gradient re-parametrization techniques, we can obtain well-defined gradients with respect to HOD parameters through discrete galaxy catalogue realizations. Having access to these gradients allows us to leverage efficient sampling schemes, such as Hamiltonian Monte Carlo, and greatly speed up parameter inference. We demonstrate our technique on a mock galaxy catalogue generated from the Bolshoi simulation using a standard HOD model and find near-identical posteriors as standard Markov chain Monte Carlo techniques with an increase of ∼8× in convergence efficiency. Our differentiable HOD model also has broad applications in full forward model approaches to cosmic structure and cosmological analysis.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

DAmodel: hierarchical Bayesian modelling of DA white dwarfs for spectrophotometric calibration

We use hierarchical Bayesian modelling to calibrate a network of 32 all-sky faint DA white dwarf (DA WD) spectrophotometric standards (⁠16.5 < V , 19.5⁠) alongside three CALSPEC standards, from 912 Å to 32 μm. The framework is the first of its kind to jointly infer photometric zero points and WD parameters (surface gravity log g⁠, effective temperature T eff ⁠, extinction A V ⁠, dust relation parameter R V ) by simultaneously modelling both photometric and spectroscopic data. We model panchromatic Hubble Space Telescope Wide Field Camera 3 (HST/WFC3) UVIS and IR photometry, HST/STIS UV spectroscopy, and ground-based optical spectroscopy to sub-per cent precision. Photometric residuals for the sample are the lowest yet yielding < 0.004 mag RMS on average from the UV to the NIR, achieved by jointly inferring time-dependent changes in system sensitivity and WFC3/IR count-rate nonlinearity. Our GPU-accelerated implementation enables efficient sampling via Hamiltonian Monte Carlo, critical for exploring the high-dimensional posterior space. The hierarchical nature of the model enables population analysis of intrinsic WD and dust parameters. Inferred spectral energy distributions from this model will be essential for calibrating the James Webb Space Telescope as well as next-generation surveys, including Vera Rubin Observatory’s Legacy Survey of Space and Time and the Nancy Grace Roman Space Telescope.

methods: statistical↗
DOE OSTI2020

Bayesian differential programming for robust systems identification under uncertainty

This paper presents a machine learning framework for Bayesian systems identification from noisy, sparse and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling. This allows an efficient inference of the posterior distributions over plausible models with quantified uncertainty, while the use of sparsity-promoting priors enables the discovery of interpretable and parsimonious representations for the underlying latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed methods, including nonlinear oscillators, predator–prey systems and examples from systems biology. Taken together, our findings put forth a flexible and robust workflow for data-driven model discovery under uncertainty. All codes and data accompanying this article are available at https://bit.ly/34FOJMj .

Science & Technology - Other Topics↗
DOE OSTI2022

Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data

We present a machine learning framework (GP-NODE) for Bayesian model discovery from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. The use of the Finnish Horseshoe as a sparsity-promoting prior for free model parameters also enables the discovery of parsimonious representations for the latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator–prey systems, systems biology and a 50-dimensional human motion dynamical system. This article is part of the theme issue ‘Data-driven prediction in dynamical systems’.

Science & Technology - Other Topics↗
DOE OSTI2022

Marginal unbiased score expansion and application to CMB lensing

Here, we present the marginal unbiased score expansion (MUSE) method, an algorithm for generic high-dimensional hierarchical Bayesian inference. MUSE performs approximate marginalization over arbitrary non-Gaussian latent parameter spaces, yielding Gaussianized asymptotically unbiased and near-optimal constraints on global parameters of interest. It is computationally much cheaper than exact alternatives like Hamiltonian Monte Carlo (HMC), excelling on funnel problems which challenge HMC, and does not require any problem-specific user supervision like other approximate methods such as variational inference or many simulation-based inference methods. MUSE makes possible the first joint Bayesian estimation of the delensed Cosmic Microwave Background (CMB) power spectrum and gravitational lensing potential power spectrum, demonstrated here on a simulated data set as large as the upcoming South Pole Telescope 3G 1500 deg 2 survey, corresponding to a latent dimensionality of ~6 million and of order 100 global bandpower parameters. On a subset of the problem where an exact but more expensive HMC solution is feasible, we verify that MUSE yields nearly optimal results. We also demonstrate that existing spectrum-based forecasting tools which ignore pixel-masking underestimate predicted error bars by only ~10%. This method is a promising path forward for fast lensing and delensing analyses which will be necessary for future CMB experiments such as SPT-3G, Simons Observatory, or CMB-S4, and can complement or supersede existing HMC approaches. The success of MUSE on this challenging problem strengthens its case as a generic procedure for a broad class of high-dimensional inference problems.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Scalable Bayesian Physics-Informed Kolmogorov-Arnold Networks

Uncertainty quantification (UQ) plays a pivotal role in scientific machine learning, especially when surrogate models are used to approximate complex systems. Although multilayer perceptions (MLPs) are commonly employed as surrogates, they often suffer from overfitting due to their large number of parameters. Kolmogorov-Arnold networks (KANs) offer an alternative solution with fewer parameters. However, gradient-based inference methods, such as Hamiltonian Monte Carlo (HMC), may result in computational inefficiency when applied to KANs, especially for large-scale datasets, due to the high cost of back-propagation. To address these challenges, we propose a novel approach, combining the dropout Tikhonov ensemble Kalman inversion (DTEKI) with Chebyshev KANs. This gradient-free method effectively mitigates overfitting and enhances numerical stability. In addition, we incorporate the active subspace method to reduce the parameter-space dimensionality, allowing us to improve the accuracy of predictions and obtain more reliable uncertainty estimates. Extensive experiments demonstrate the efficacy of our approach in various test cases, including scenarios with large datasets and high noise levels. Our results show that the new method achieves comparable or better accuracy, much higher efficiency as well as stability compared to HMC, in addition to scalability. Moreover, by leveraging the low-dimensional parameter subspace, our method preserves prediction accuracy while substantially reducing further the computational cost.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Elsa: enhanced latent spaces for improved collider simulations

Abstract Simulations play a key role for inference in collider physics. We explore various approaches for enhancing the precision of simulations using machine learning, including interventions at the end of the simulation chain (reweighting), at the beginning of the simulation chain (pre-processing), and connections between the end and beginning (latent space refinement). To clearly illustrate our approaches, we use W + jets matrix element surrogate simulations based on normalizing flows as a prototypical example. First, weights in the data space are derived using machine learning classifiers. Then, we pull back the data-space weights to the latent space to produce unweighted examples and employ the Latent Space Refinement ( Laser ) protocol using Hamiltonian Monte Carlo. An alternative approach is an augmented normalizing flow, which allows for different dimensions in the latent and target spaces. These methods are studied for various pre-processing strategies, including a new and general method for massive particles at hadron colliders that is a tweak on the widely-used RamboOnDiet mapping. We find that modified simulations can achieve sub-percent precision across a wide range of phase space.

Nachman, Benjamin (ORCID:0000000310240932)↗
DOE OSTI2020

Algorithms in Diffraction Profile Analysis

This chapter introduces several approaches to crystallographic structure refinement from diffraction profile data including the Rietveld method, global optimization methods, and Bayesian modeling. First, the Rietveld method and related nonlinear least squares approaches to analyzing diffraction profiles are briefly discussed. We then review the most common algorithms within global optimization as alternative methods of structure solution: grid search, simulated annealing, and genetic algorithms. As the Rietveld method is the standard tool for analyzing diffraction profile data, we catalogue its challenges and limitations. A Bayesian approach to diffraction profile analysis is presented as an alternative, along with several sampling algorithms including Gibbs sampling, variants of Metropolis sampling, Hamiltonian Monte Carlo, and approximate Bayesian computing. These sampling algorithms are applied to the fitting of a neutron diffraction profile from a National Institute of Standards and Technology silicon standard reference material, and their results are compared.

Singh, Susheela↗