Search NASA⌕ Search

SEARCH · Search NASA

Results for “intractable likelihood”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

11 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Amortized simulation-based frequentist inference for tractable and intractable likelihoods

Abstract High-fidelity simulators that connect theoretical models with observations are indispensable tools in many sciences. If the likelihood is known, inference can proceed using standard techniques. However, when the likelihood is intractable or unknown, a simulator makes it possible to infer the parameters of a theoretical model directly from real and simulated observations when coupled with machine learning. We introduce an extension of the recently proposed likelihood-free frequentist inference ( LF2I ) approach that makes it possible to construct confidence sets with the p -value function and to use the same function to check the coverage explicitly at any given parameter point. Like LF2I , this extension yields provably valid confidence sets in parameter inference problems for which a high-fidelity simulator is available. The utility of our algorithm is illustrated by applying it to three pedagogically interesting examples: the first is from cosmology, the second from high-energy physics and astronomy, both with tractable likelihoods, while the third, with an intractable likelihood, is from epidemiology 3 3 Code to reproduce all of our results is available on https://github.com/AliAlkadhim/ALFFI . .

Computer Science↗
DOE OSTI2025

Enhancing Gaussian Process Surrogates for Optimization and Posterior Approximation via Random Exploration

This paper proposes novel noise-free Bayesian optimization strategies that rely on a random exploration step to enhance the accuracy of Gaussian process surrogate models. The new algorithms retain the ease of implementation of the classical GP-UCB algorithm, but the additional random exploration step accelerates their convergence, nearly achieving the optimal convergence rate. Furthermore, to facilitate Bayesian inference with intractable likelihoods, we propose to utilize optimization iterates for maximum a posteriori estimation to build a Gaussian process surrogate model for the unnormalized log-posterior density. We provide bounds for the Hellinger distance between the true and the approximate posterior distributions in terms of the number of design points. We demonstrate the effectiveness of our Bayesian optimization algorithms in nonconvex benchmark objective functions, in a machine learning hyperparameter tuning problem, and in a black-box engineering design problem. The effectiveness of our posterior approximation approach is demonstrated in two Bayesian inference problems for parameters of dynamical systems.

Bayesian inference↗
DOE OSTI2023

Simulation Based Inference with Domain Adaptation for Strong Gravitational Lensing

Simulation based inference leverages machine learning to carry out Bayesian inference for systems with intractable likelihoods. However, transitioning a network trained on simulated data to real data runs the risk of encountering domain shift, leading to performance losses. We attempt to implement domain adaptation into the sbi neural posterior estimation framework using the Maximum Mean Discrepancy as an additional network loss. We test two network architectures and use masked autoregressive flow for density estimation. We test the network on a set of 400,000 simulated strong gravitational lensing images generated using deeplenstronomy. The source domain is defined as low noise whereas the target domain has a noise profile sampled from experimentally derived DES survey conditions. We find that, while DA does lead to performance improvements, they are marginal at ~6% less inference error. We also find a similar marginal improvement in uncertainty calibration at around 8%.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Simulation based inference with domain adaptation for strong gravitational lensing

Simulation based inference leverages machine learning to carry out Bayesian inference in systems with intractable likelihoods. However, transitioning a network trained on simulated data to real data runs the risk of encountering domain shift, leading to performance losses. We attempt to implement domain adaptation into the sbi neural posterior estimation framework using the Maximum Mean Discrepancy as an additional network loss, using masked autoregressive fow (MAF) as our density estimator. We test the network on a set of 400,000 simulated strong gravitational lensing images generated using deeplenstronomy. The source domain is defined as low noise whereas the target domain has a noise profile sampled from experimentally derived DES survey conditions. We find that SBI appears robust against small changes in the data with similar performance on source and target. Moreover, while DA does lead to performance improvements, they are marginal at 6% less inference error.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Strong Lensing Cosmology with Population-level Calibrated Neural Ratio Estimation

Strong gravitational lensing contains key information about cosmic acceleration. Modern and next-generation galaxy imaging surveys are expected to provide high-quality data on $\mathcal{O}(10^5)$ galaxy-galaxy lensing systems. The plethora and complexity of the data are likely to present computational challenges for parameter inference methods for fitting high-dimensional likelihoods, which are often analytically intractable. Neural Ratio Estimation (NRE) efficiently computes individual likelihood ratios that can be combined into population-level posteriors. We use simulations to study the capacity of NRE to jointly predict the dark energy equation-of-state parameter $w$ and the total matter density $Ω_{m}$ from lensing images and companion spectroscopic information. We also introduce a post hoc posterior coverage calibration procedure that mitigates the model overconfidence that is typically found in neural density estimation applications. Our experiments show that the errors on both parameters decrease with increasing inference population sizes. In particular, for 100 lenses in a standard $Λ$CDM Universe, our calibrated NRE model achieves median fractional uncertainty of $22.8\%$ in $w$ and $2.9\%$ in $Ω_{m}$. This proof of concept demonstrates a potentially scalable approach for efficient cosmological parameter inference with large populations of galaxy-scale lenses observed in future surveys.

Jarugula, Sreevani [Fermilab] (ORCID:0000000253867↗
DOE OSTI2023

Neural networks for parameter estimation in intractable models

The goal is to use deep learning models to estimate parameters in statistical models when standard likelihood estimation methods are computationally infeasible. For instance, inference for max-stable processes is exceptionally challenging even with small datasets, but simulation is straightforward. Data from model simulations are used to train deep neural networks and learn statistical parameters from max-stable models. The proposed neural network-based method provides a competitive alternative to current approaches, as demonstrated by considerable accuracy and computational time improvements. Finally, it serves as a proof of concept for deep learning in statistical parameter estimation and can be extended to other estimation problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Supercharging simulation-based inference for Bayesian optimal experimental design

Abstract Bayesian optimal experimental design (BOED) seeks to maximize the expected information gain (EIG) of experiments. This requires a likelihood estimate, which in many settings is intractable. Simulation-based inference (SBI) provides powerful tools for this regime. However, existing work explicitly connecting SBI and BOED is restricted to a single contrastive EIG bound. We show that the EIG admits multiple formulations which can directly leverage modern SBI density estimators, encompassing neural posterior, likelihood, and ratio estimation. Building on this perspective, we define a novel EIG estimator using neural likelihood estimation. Further, we identify optimization as a key bottleneck of gradient based EIG maximization and show that a simple multi-start parallel gradient ascent procedure can substantially improve reliability and performance. With these innovations, our SBI-based BOED methods are able to match or outperform by up to 22% existing state-of-the-art approaches across standard BOED benchmarks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Deducing neutron star equation of state from telescope spectra with machine-learning-derived likelihoods

The interiors of neutron stars reach densities and temperatures beyond the limits of terrestrial experiments, providing vital laboratories for probing nuclear physics. While the star's interior is not directly observable, its pressure and density determine the star's macroscopic structure which affects the spectra observed in telescopes. The relationship between the observations and the internal state is complex and partially intractable, presenting difficulties for inference. Previous work has focused on the regression from stellar spectra of parameters describing the internal state. We demonstrate a calculation of the full likelihood of the internal state parameters given observations, accomplished by replacing intractable elements with machine learning models trained on samples of simulated stars. Our machine-learning-derived likelihood allows us to perform maximum a posteriori estimation of the parameters of interest, as well as full scans. We demonstrate the technique by inferring stellar mass and radius from an individual stellar spectrum, as well as equation of state parameters from a set of spectra. Our results are more precise than pure regression models, reducing the width of the parameter residuals by 11.8% in the most realistic scenario. The neural networks will be released as a tool for fast simulation of neutron star properties and observed spectra.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Scalable computations for nonstationary Gaussian processes

Nonstationary Gaussian process models can capture complex spatially varying dependence structures in spatial datasets. However, the large number of observations in modern datasets makes fitting such models computationally intractable with conventional dense linear algebra. In addition, derivative-free or even first-order optimization methods can be very slow to converge when estimating many spatially varying parameters. In this paper, we present a computational framework which couples an algebraic block diagonal plus low-rank covariance matrix approximation with stochastic trace estimation to facilitate the efficient use of second-order solvers for maximum likelihood estimation of Gaussian process models with many parameters. We demonstrate the effectiveness of these methods by simultaneously fitting 192 parameters in the popular nonstationary model of Paciorek and Schervish using 107,600 sea surface temperature anomaly measurements.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Learning epistatic polygenic phenotypes with Boolean interactions

Detecting epistatic drivers of human phenotypes is a considerable challenge. Traditional approaches use regression to sequentially test multiplicative interaction terms involving pairs of genetic variants. For higher-order interactions and genome-wide large-scale data, this strategy is computationally intractable. Moreover, multiplicative terms used in regression modeling may not capture the form of biological interactions. Building on the Predictability, Computability, Stability (PCS) framework, we introduce the epiTree pipeline to extract higher-order interactions from genomic data using tree-based models. The epiTree pipeline first selects a set of variants derived from tissue-specific estimates of gene expression. Next, it uses iterative random forests (iRF) to search training data for candidate Boolean interactions (pairwise and higher-order). We derive significance tests for interactions, based on a stabilized likelihood ratio test, by simulating Boolean tree-structured null (no epistasis) and alternative (epistasis) distributions on hold-out test data. Finally, our pipeline computes PCS epistasis p-values that probabilisticly quantify improvement in prediction accuracy via bootstrap sampling on the test set. We validate the epiTree pipeline in two case studies using data from the UK Biobank: predicting red hair and multiple sclerosis (MS). In the case of predicting red hair, epiTree recovers known epistatic interactions surrounding MC1R and novel interactions, representing non-linearities not captured by logistic regression models. In the case of predicting MS, a more complex phenotype than red hair, epiTree rankings prioritize novel interactions surrounding HLA-DRB1 , a variant previously associated with MS in several populations. Taken together, these results highlight the potential for epiTree rankings to help reduce the design space for follow up experiments.

59 BASIC BIOLOGICAL SCIENCES↗