Search NASA⌕ Search

SEARCH · Search NASA

Results for “Poisson likelihood”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Combined Neyman–Pearson chi-square: An improved approximation to the Poisson-likelihood chi-square

We describe an approximation to the widely-used Poisson-likelihood chi-square using a linear combination of Neyman’s and Pearson’s chi-squares, namely “combined Neyman–Pearson chi-square” ($X^2_{CNP}$). Through analytical derivations and toy model simulations, we show that leads to a significantly smaller bias on the best-fit model parameters compared to those using either Neyman’s or Pearson’s chi-square. When the computational cost of using the Poisson-likelihood chi-square is high, $X^2_{CNP}$ provides a good alternative given its natural connection to the covariance matrix formalism.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Effect of likelihood misspecification in Gaussian process-driven autonomous experimentation

In recent years, several groups have designed Autonomous Experiment (AE) models with the aim of using them as an alternative method for neutron scattering scanning. In an AE, Gaussian processes (GPs) are most frequently used due to their interpretability, their non-parametric nature, their universal approximation, and their closed-form predictive distribution. GPs have two key components, namely, the model for the likelihood of a neutron count knowing the underlying dynamic structure factor and the acquisition function. In this paper, we investigate the impact, on the quality of an AE, of the likelihood and acquisition function choices, in energy scans and (Q, ω) ones, with respect to the signal-over-noise ratio. While we hypothesized that the quality of GP predictions would decrease when the normal to Poisson likelihood approximation breaks down at low count rates, we found that the use of the correct Poisson likelihood does not improve the quality of the data collected, as well as yields very poor results in (Q, ω) scans at low count rates. In fact, the best results are obtained with a combination of normal likelihood, including the observation noise, and the change in variance acquisition function. In addition, we find that the performance, or quality of the predictive distribution, is a misleading measure of efficiency, that is, of the quality of the data collected.

Perryman, David Elliott [Inst. Laue-Langevin (ILL)↗
DOE OSTI2020

A practical way to regularize unfolding of sharply varying spectra with low data statistics

Unfolding is a well-established tool in particle physics. However, a naive application of the standard regularization techniques to unfold the momentum spectrum of protons ejected in the process of negative muon nuclear capture led to a result exhibiting unphysical artifacts. A finite data sample limited the range in which unfolding can be performed, thus introducing a cutoff. A sharply falling “true” distribution led to low data statistics near the cutoff, which exacerbated the regularization bias and produced an unphysical spike in the resulting spectrum. An improved approach has been developed to address these issues and is illustrated using a toy model. The approach uses full Poisson likelihood of data, and produces a continuous, physically plausible, unfolded distribution. Here, the new technique has a broad applicability since sharply falling spectra are common.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

ASCR Workshop Position Paper: Challenges and Opportunities in High Energy Physics

High energy particle physics and cosmology concern themselves with estimating fundamental parameters of nature, such as the masses and interactions of fundamental particles like the Higgs boson and the rate of expansion of the universe. In doing so, they analyze exabyte-scale datasets, some of the largest in all of science, and face many challenges in subsequent data analysis. These challenges are shared between the two disciplines, but we focus on particle physics to highlight one specific domain. In particle physics, the standard method for estimating parameters involves performing Monte Carlo (MC) integration as a function of both parameters of interest and nuisance parameters using an expensive simulator, counting the number of observed collision events (i.i.d. samples) from an experiment in the corresponding integration domains, and forming a Poisson likelihood function. This likelihood function is then used in a Frequentist manner to construct a maximum likelihood point estimate (MLE) and confidence set for the parameters. To sufficiently populate the high-dimensional integration domains, simulators consume billions of CPU-hours annually and produce hundreds of petabytes of intermediate output data. Several techniques have been developed to: optimize definitions of the integration domains so as to be maximally sensitive to a particular subset of parameters, efficiently estimate the integrals, and build robust surrogate models by interpolating between integral evaluations at different parameter points. One can view this whole endeavor as classical Simulation-Based Inference (SBI).

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

PISCES two-detector covariance matrix fit for the NOvA Experiment

NOvA is a long-baseline neutrino oscillation experiment with two functionally identical detectors: a Near Detector (ND) at Fermilab, placed 1 km from the neutrino source, and a Far Detector (FD) located 810 km away from the ND in Minnesota. NOvA's primary physics goals are the precise measurements of neutrino oscillation parameters $\theta_{23}$ and $\Delta m^2_{32}$ , determine the neutrino mass ordering, and constrain the value of $\delta_{CP}$, via the study of muon neutrino to electron neutrino oscillation. In the standard NOvA three-flavor analysis, oscillation parameters are extracted using an extrapolation technique in which the ND data constrain the FD prediction through a ratio method. While this allows for systematic uncertainties sharing the same effects in both detectors to cancel, it remains an FD-only fit and does not fully leverage the constraining power of the high-statistics ND. This analysis proposes a simultaneous ND+FD fit using the PISCES method. PISCES (Parameter Inference with Systematic Covariance and Exact Statistics) is a framework designed to support complex configurations such as a joint ND+FD fit. This allows PISCES to take full advantage of the ND data to directly constrain systematic uncertainties across all samples. In PISCES, systematic uncertainties are encoded in a fractional covariance matrix, and statistical uncertainties are handled with a Poisson likelihood, making the approach well suited for low-statistics samples. For interpretability, we further use a Newton–Raphson + PCA method to recover per-systematic pulls from the covariance formulation. This poster presents the full PISCES joint ND+FD fit for the NOvA three-flavor analysis, describes its implementation and evaluates its performance through extensive robustness tests and fake data studies. It also provides a comparison between the PISCES joint ND+FD results and the standard NOvA extrapolation method.

Rajaoalisoa, Miriama [Cincinnati U.] (ORCID:000000↗
DOE OSTI2026

Measurements of three-flavor neutrino oscillations from a PISCES two-detector fit to the NOvA Experiment data

NOvA is a long-baseline neutrino oscillation experiment with two functionally identical detectors: a Near Detector (ND) at Fermilab, placed 1 km from the neutrino source, and a Far Detector (FD) located 810 km away from the ND in Minnesota. NOvA s primary physics goals are to measure the neutrino oscillation parameters $\theta_{23}$ and $\Delta m^2_{32}$ with high precision, determine the neutrino mass hierarchy, and constrain the value of $\delta_{CP}$, primarily via the study of muon neutrino to electron neutrino oscillation. Extracting values for oscillation parameters from fits to data usually relies on treating systematic uncertainties as nuisance parameters, a strategy that suffers from poor scalability as the number of uncertainties becomes larger. This work introduces PISCES (Parameter Inference with Systematic Covariance and Exact Statistics), a novel method that circumvents this scalability problem by encoding systematic uncertainties into a covariance matrix. PISCES utilizes a nested minimization in which optimal systematic pulls are first computed using the covariance matrix in an inner minimization step, then the oscillation parameters are profiled over in the outer minimization. PISCES also uses a Poisson Likelihood term, making it ideal for the inclusion of low-statistic samples in the fits. PISCES is a flexible framework that also supports complex fits, such as a joint Near and Far detector fit. In the standard NOvA analysis, oscillation parameters are extracted using an extrapolation technique in which the ND data indirectly constrain the FD prediction via a ratio method. PISCES, on the other hand, enables a simultaneous ND+FD fit, allowing the high-statistics ND data to directly constrain systematic uncertainties across all samples. This thesis presents the full PISCES joint ND+FD fit for the NOvA three-flavor analysis, details its implementation, and evaluates its performance through extensive robustness tests and fake data studies. It also provides a comparison between the PISCES joint ND+FD results and the standard NOvA extrapolation method using the full NOvA 10-year data set. The results demonstrate that PISCES can successfully fit NOvA data while incorporating the constraints from the ND detectors consistently, using physically motivated systematic uncertainties to account for data/MC discrepancies.

Rajaoalisoa, Miriama [Cincinnati U.]↗
DOE OSTI2025

A Latent-Variable Formulation of the Poisson Canonical Polyadic Tensor Model: Maximum Likelihood Estimation and Fisher Information

We establish parameter inference for the Poisson canonical polyadic (PCP) tensor model through a latent-variable formulation. Our approach exploits the observation that any random PCP tensor can be derived by marginalizing an unobservable random tensor of one dimension larger. The loglikelihood of this larger dimensional tensor, referred to as the “complete” loglikelihood, is comprised of multiple rank one PCP loglikelihoods. Using this methodology, we first derive maximum likelihood estimators for the PCP model and demonstrate that several existing algorithms for fitting non-negative matrix and tensor factorizations are Expectation-Maximization algorithms. Next, we derive the observed and expected Fisher information matrices for the PCP model. The Fisher information provides us crucial insights into the well-posedness of the tensor model, such as the role that tensor rank plays in identifiability and indeterminacy. For the special case of rank one PCP models, we demonstrate that these results are greatly simplified.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Iterative X-ray spectroscopic ptychography

Spectroscopic ptychography is a powerful technique to determine the chemical composition of a sample with high spatial resolution. In spectro-ptychography, a sample is rastered through a focused X-ray beam with varying photon energy so that a series of phaseless diffraction data are recorded. Each chemical component in the material under investigation has a characteristic absorption and phase contrast as a function of photon energy. Using a dictionary formed by the set of contrast functions of each energy for each chemical component, it is possible to obtain the chemical composition of the material from high-resolution multi-spectral images. This paper presents SPA (spectroscopic ptychography with alternating direction method of multipliers), a novel algorithm to iteratively solve the spectroscopic blind ptychography problem. First, a nonlinear spectro-ptychography model based on Poisson maximum likelihood is designed, and then the proposed method is constructed on the basis of fast iterative splitting operators. SPA can be used to retrieve spectral contrast when considering either a known or an incomplete (partially known) dictionary of reference spectra. By coupling the redundancy across different spectral measurements, the proposed algorithm can achieve higher reconstruction quality when compared with standard state-of-the-art two-step methods. It is demonstrated how SPA can recover accurate chemical maps from Poisson-noised measurements, and its enhanced robustness when reconstructing reduced-redundancy ptychography data using large scanning step sizes is shown.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Reconstructing the Position and Intensity of Multiple Gamma-Ray Point Sources with a Sparse Parametric Algorithm

IEEE We present an experimental demonstration of Additive Point Source Localization (APSL), a sparse parametric imaging algorithm that reconstructs the 3D positions and activities of multiple gamma-ray point sources. Using a handheld gamma-ray detector array and up to four 8 μCi 137 Cs gamma-ray sources, we performed both source-search and source-separation experiments in an indoor laboratory environment. In the majority of the source-search measurements, APSL reconstructed the correct number of sources with position accuracies of ~20 cm and activity accuracies (unsigned) of ~20%, given measurement times of two to three minutes and distances of closest approach (to any source) of ~20 cm. In source-separation measurements where the detector could be moved freely about the environment, APSL was able to resolve two sources separated by 75 cm or more given only ~60 s of measurement time. In these source-separation measurements, APSL produced larger total activity errors of ~40%, but obtained source separation distances accurate to within 15 cm. We also compare our APSL results against traditional Maximum Likelihood-Expectation Maximization (ML-EM) reconstructions, and demonstrate improved image accuracy and interpretability using APSL over ML-EM. These results indicate that APSL is capable of accurately reconstructing gamma-ray source positions and activities using measurements from existing detector hardware.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Regularized Differentiation for Bioburden Density Estimation in Planetary Protection

In this paper, we propose and investigate the performance of two novel shrinkage estimators for bioburden density estimation in planetary protection. The estimators are based on the regularized differentiation of a cumulative count of colony forming units collected throughout the data collecting session or the life cycle of the entire mission. The regularized differentiation recasts the problem of bioburden density estimation as a linear least squares problem. The least squares problem is then solved through regularization techniques, such as truncated singular value decomposition and penalized least squares. The regularization is necessary to avoid noise amplification during the differentiation of noisy data. The two regularization estimators are compared with four other commonly used estimators to simultaneously evaluate the means of multivariable independent Poisson distributions: the maximum likelihood, noninformative Bayes estimator with Jeffreys prior, Empirical Bayes using conjugate gamma-Poisson model with gamma parameters selected by method of moments, and the Clevenson-Zidek estimator. It is shown through computer-simulated data that the regularized differentiation based on ridge regression has the smallest mean-squared error among all estimators. The analysis of shrinkage mechanism implemented by regularized differentiation is performed, and it is shown that the regularized differentiation amounts to performing a weighted averaging of all the samples. The weights are determined by the regularization parameter automatically selected by the L-curve technique. Since the method of least squares makes no distributional assumptions about the data, it presents an attractive technique for bioburden density estimation when there are concerns about the misspecification of the distributional model. The paper concludes with the analysis of the bioburden data collected during InSight mission and directions for future work.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Tensor decompositions for count data that leverage stochastic and deterministic optimization

There is growing interest to extend low-rank matrix decompositions to multi-way arrays, or tensors. One fundamental low-rank tensor decomposition is the canonical polyadic decomposition (CPD). The challenge of fitting a low-rank, nonnegative CPD model to Poisson-distributed count data is of particular interest. Several popular algorithms use local search methods to approximate the maximum likelihood estimator (MLE) of the Poisson CPD model. Here, this work presents two new algorithms that extend state-of-the-art local methods for Poisson CPD. Hybrid GCP-CPAPR combines Generalized Canonical Decomposition (GCP) with stochastic optimization and CP Alternating Poisson Regression (CPAPR), a deterministic algorithm, to increase the probability of converging to the MLE over either method used alone. Restarted CPAPR with SVDrop uses a heuristic based on the singular values of the CPD model unfoldings to identify convergence toward optimizers that are not the MLE and restarts within the feasible domain of the optimization problem, thus reducing overall computational cost when using a multi-start strategy. We provide empirical evidence that indicates our approaches outperform existing methods with respect to converging to the Poisson CPD MLE.

CPAPR↗
DOE OSTI2022

A Compound Poisson Generator Approach to Point-source Inference in Astrophysics

Abstract The identification and description of point sources is one of the oldest problems in astronomy, yet even today the correct statistical treatment for point sources remains one of the field’s hardest problems. For dim or crowded sources, likelihood-based inference methods are required to estimate the uncertainty on the characteristics of the source population. In this work, a new parametric likelihood is constructed for this problem using compound Poisson generator (CPG) functionals that incorporate instrumental effects from first principles. We demonstrate that the CPG approach exhibits a number of advantages over non-Poissonian template fitting (NPTF)—an existing method—in a series of test scenarios in the context of X-ray astronomy. These demonstrations show that the effect of the point-spread function, effective area, and choice of point-source spatial distribution cannot, generally, be factorized as they are in NPTF, while the new CPG construction is validated in these scenarios. Separately, an examination of the diffuse-flux emission limit is used to show that most simple choices of priors on the standard parameterization of the population model can result in unexpected biases: when a model comprising both a point-source population and diffuse component is applied to this limit, nearly all observed flux will be assigned to either the population or to the diffuse component. A new parameterization is presented for these priors that properly estimates the uncertainties in this limit. In this choice of priors, CPG correctly identifies that the fraction of flux assigned to the population model cannot be constrained by the data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

A Hybrid Method for Tensor Decompositions that Leverages Stochastic and Deterministic Optimization

In this paper, we propose a hybrid method that uses stochastic and deterministic search to compute the maximum likelihood estimator of a low-rank count tensor with Poisson loss via state-of-theart local methods. Our approach is inspired by Simulated Annealing for global optimization and allows for fine-grain parameter tuning as well as adaptive updates to algorithm parameters. We present numerical results that indicate our hybrid approach can compute better approximations to the maximum likelihood estimator with less computation than the state-of-the-art methods by themselves.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Poisson-response Tensor-on-Tensor Regression and Applications

We introduce Poisson-response tensor-on-tensor regression (PToTR), a novel regression framework designed to handle tensor responses composed element-wise of random Poisson-distributed counts. Tensors, or multi-dimensional arrays, composed of counts are common data in fields such as inter national relations, social networks, epidemiology, and medical imaging, where events occur across multiple dimensions like time, location, and dyads. PToTR accommodates such tensor responses alongside tensor covariates, providing a versatile tool for multi dimensional data analysis. We propose algorithms for maximum likelihood estimation under a canonical polyadic (CP) structure on the regression coefficient tensor that satisfy the positivity of Poisson parameters and then provide an initial theoretical error analysis for PToTR estimators. We also demonstrate the utility of PToTR through three concrete applications: longitudinal data analysis of the Integrated Crisis Early Warning System database, positron emission tomography (PET) image reconstruction, and change-point detection of communication patterns in longitudinal dyadic data. These applications highlight the versatility of PToTR in addressing complex, structured count data across various domains.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Tensor Decompositions for Count Data that Leverage Stochastic and Deterministic Optimization

There is growing interest to extend low-rank matrix decompositions to multi-way arrays, or tensors. One fundamental low-rank tensor decomposition is the canonical polyadic decomposition (CPD). The challenge of fitting a low-rank, nonnegative CPD model to Poisson-distributed count data is of particular interest. Several popular algorithms use local search methods to approximate the global maximum likelihood estimator from local minima. Simultaneously, a recent trend in theoretical computer science and numerical linear algebra leverages randomization to solve very large, hard problems. The typical approach is to use randomization for a fast approximation and determinism for refinement to yield effective algorithms with theoretical guarantees. Two popular algorithms for Poisson CPD reflect that emergent dichotomy: CP Alternating Poisson Regression is a deterministic algorithm and Generalized Canonical Polyadic decomposition makes use of stochastic algorithms in several variants. This work extends recent work to develop two new methods that leverage randomized and deterministic algorithms for improved accuracy and performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Analysis of overlapping count data

Counts of a specific characteristic were obtained within regions defined on an object that was manufactured in a proprietary setting. The count regions were altered during production and resulted in misaligned or overlapping count data. A closed-formula maximum likelihood estimator (MLE) of the new region means is derived using all of the available count data and an independent Poisson model. The MLE is shown to be preferable to estimators constructed using generalized linear models for the overlapping data setting. This closed-form estimator extends to over-dispersed overlapping count data as the quasi-MLE and also performs well with correlated overlapping count data. Standard errors for the estimator are approximated and are validated with a simulation study. Additionally, the methods are extended to overlapping multinomial data. Illustrative examples of the methods are provided throughout the paper and are reproducible with the supplemental R code. Additionally, proofs of the paper’s results are also included in the supplemental material.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Adversarial autoencoder ensemble for fast and probabilistic reconstructions of few-shot photon correlation functions for solid-state quantum emitters

Second-order photon correlation measurements [g (2) (τ) functions] are widely used to classify single-photon emission purity in quantum emitters or to measure the multiexciton quantum yield of emitters that can simultaneously host multiple excitations – such as quantum dots – by evaluating the value of g (2) (τ = 0). Accumulating enough photons to accurately calculate this value is time consuming and could be accelerated by fitting of few-shot photon correlations. Here, we develop an uncertainty-aware, deep adversarial autoencoder ensemble (AAE) that reconstructs noise-free g (2) (τ) functions from noise-dominated, few-shot inputs. The model is trained with simulated g (2) (τ) functions that are facilely generated by Poisson sampling time bins. The AAE reconstructions are performed orders-of-magnitude faster, with reconstruction errors and estimates of g (2) (τ = 0) that are lower in variance and similar in accuracy compared to Maximum likelihood estimation and Levenberg-Marquardt least-squares fitting approaches, for simulated and experimentally measured few-shot g (2) (τ) functions (~100 two-photon events) of InP/ZnS/ZnSe and CdS/CdSe/CdS quantum dots. The deep-ensemble model comprises eight individual autoencoders, allowing for probabilistic reconstructions of noise-free g (2) (τ) functions, and we show that the predicted variance scales inversely with number of shots, with comparable uncertainties to computationally intensive Markov chain Monte Carlo sampling. Furthermore, this work demonstrates the advantage of machine learning models to perform uncertainty-aware, fast, and accurate reconstructions of simple Poisson-distributed photon correlation functions, allowing for on-the-fly reconstructions and accelerated materials characterization of solid-state quantum emitters.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

Large deviations of ionic currents in dilute electrolytes

Here, we evaluate the exponentially rare fluctuations of the ionic current for a dilute electrolyte by means of macroscopic fluctuation theory. We consider the fluctuating hydrodynamics of a fluid electrolyte described by a stochastic Poisson–Nernst–Planck equation. We derive the Euler–Lagrange equations that dictate the optimal concentration profiles of ions conditioned on exhibiting a given current, whose form determines the likelihood of that current in the long-time limit. For a symmetric electrolyte under small applied voltages, number density fluctuations are small, and ionic current fluctuations are Gaussian with a variance determined by the Nernst–Einstein conductivity. Under large applied potentials, the ionic current distribution is generically non-Gaussian. Its structure is constrained thermodynamically by Gallavotti–Cohen symmetry and the thermodynamic uncertainty principle.

Farhadi, Jafar [University of California, Berkeley↗