Search NASA⌕ Search

SEARCH · Search NASA

Results for “statistical inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Robust cosmological inference from non-linear scales with k -th nearest neighbour statistics

ABSTRACT We present the methodology for deriving accurate and reliable cosmological constraints from non-linear scales ($\lt 50\, h^{-1}$ Mpc) with k-th nearest neighbour (kNN) statistics. We detail our methods for choosing robust minimum scale cuts and validating galaxy–halo connection models. Using cross-validation, we identify the galaxy–halo model that ensures both good fits and unbiased predictions across diverse summary statistics. We demonstrate that we can model kNNs effectively down to transverse scales of $r_{\rm p}\sim 3\, h^{-1}$ Mpc and achieve precise and unbiased constraints on the matter density and clustering amplitude, leading to a 2 per cent constraint on σ8. Our simulation-based model pipeline is resilient to varied model systematics, spanning simulation codes, halo finding, and cosmology priors. We demonstrate the effectiveness of this approach through an application to the Beyond-2p mock challenge. We propose further explorations to test more complex galaxy–halo connection models and tackle potential observational systematics.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

A framework for inferring and analyzing pharmacotherapy treatment patterns

Background: To discover pharmacotherapy prescription patterns and their statistical associations with outcomes through a clinical pathway inference framework applied to real-world data. Methods: We apply machine learning steps in our framework using a 2006 to 2020 cohort of veterans with major depressive disorder (MDD). Outpatient antidepressant pharmacy fills, dispensed inpatient antidepressant medications, emergency department visits, self-harm, and all-cause mortality data were extracted from the Department of Veterans Affairs Corporate Data Warehouse. Results: Our MDD cohort consisted of 252,179 individuals. During the study period there were 98,417 emergency department visits, 1,016 cases of self-harm, and 1,507 deaths from all causes. The top ten prescription patterns accounted for 69.3% of the data for individuals starting antidepressants at the fluoxetine equivalent of 20-39 mg. Additionally, we found associations between outcomes and dosage change. Conclusions: For 252,179 Veterans who served in Iraq and Afghanistan with subsequent MDD noted in their electronic medical records, we documented and described the major pharmacotherapy prescription patterns implemented by Veterans Health Administration providers. Ten patterns accounted for almost 70% of the data. Associations between antidepressant usage and outcomes in observational data may be confounded. The low numbers of adverse events, especially those associated with all-cause mortality, make our calculations imprecise. Furthermore, our outcomes are also indications for both disease and treatment. Despite these limitations, we demonstrate the usefulness of our framework in providing operational insight into clinical practice, and our results underscore the need for increased monitoring during critical points of treatment.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Inferring Quantum Network Topology Using Local Measurements

Statistical correlations that can be generated across the nodes in a quantum network depend crucially on its topology. However, this topological information might not be known a priori, or it may need to be verified. In this paper, we propose an efficient protocol for distinguishing and inferring the topology of a quantum network. We leverage entropic quantities-namely, the von Neumann entropy and the measured mutual information-as well as measurement covariance to uniquely characterize the topology. We show that the entropic quantities are sufficient to distinguish two networks that prepare GHZ states. Moreover, if qubit measurements are available, both entropic quantities and covariance can be used to infer the network topology without state-preparation assumptions. We show that the protocol can be entirely robust to noise and can be implemented via quantum variational optimization. Numerical experiments on both classical simulators and quantum hardware show that covariance is generally more reliable for accurately and efficiently inferring the topology, whereas entropy-based methods are often better at identifying the absence of entanglement in the low-shot regime.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Scaling and merging time-resolved pink-beam diffraction with variational inference

Time-resolved x-ray crystallography (TR-X) at synchrotrons and free electron lasers is a promising technique for recording dynamics of molecules at atomic resolution. While experimental methods for TR-X have proliferated and matured, data analysis is often difficult. Extracting small, time-dependent changes in signal is frequently a bottleneck for practitioners. Recent work demonstrated this challenge can be addressed when merging redundant observations by a statistical technique known as variational inference (VI). However, the variational approach to time-resolved data analysis requires identification of successful hyperparameters in order to optimally extract signal. In this case study, we present a successful application of VI to time-resolved changes in an enzyme, DJ-1, upon mixing with a substrate molecule, methylglyoxal. We present a strategy to extract high signal-to-noise changes in electron density from these data. Furthermore, we conduct an ablation study, in which we systematically remove one hyperparameter at a time to demonstrate the impact of each hyperparameter choice on the success of our model. We expect this case study will serve as a practical example for how others may deploy VI in order to analyze their time-resolved diffraction data.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Trust Your Gut: Comparing Human and Machine Inference from Noisy Visualizations

People commonly utilize visualizations not only to examine a given dataset, but also to draw generalizable conclusions about the underlying models or phenomena. Prior research has compared human visual inference to that of an optimal Bayesian agent, with deviations from rational analysis viewed as problematic. However, human reliance on non-normative heuristics may prove advantageous in certain circumstances. We investigate scenarios where human intuition might surpass idealized statistical rationality. In two experiments, we examine individuals’ accuracy in characterizing the parameters of known data-generating models from bivariate visualizations. Our findings indicate that, although participants generally exhibited lower accuracy compared to statistical models, they frequently outperformed Bayesian agents, particularly when faced with extreme samples. Participants appeared to rely on their internal models to filter out noisy visualizations, thus improving their resilience against spurious data. However, participants displayed overconfidence and struggled with uncertainty estimation. They also exhibited higher variance than statistical machines. Our findings suggest that analyst gut reactions to visualizations may provide an advantage, even when departing from rationality. These results carry implications for designing visual analytics tools, offering new perspectives on how to integrate statistical models and analyst intuition for improved inference and decision-making. The data and materials for this paper are available at https://osf.io/qmfv6

human-machine collaboration↗
DOE OSTI2022

Statistical characterization of twin transmission across grain boundaries in magnesium

Here, a detailed statistical analysis of twin transmission (TT) across grain boundaries is performed in rolled, commercial purity magnesium compressed along the rolling direction. EBSD images are acquired from two different cuts: a section containing the rolling (RD) and normal (ND) directions; and a section at 35° to the ND which contains transverse direction (TD). An automated twinning analysis software, METIS, is used to obtain the statistical correlations between deformation twins and other microstructural features by analyzing EBSD microstructures comprising thousands of grains and twins. This detailed statistical analysis reveals that the TT propensity is sensitive to the grain boundary (GB) misorientation angle but not to the GB misorientation axis. Specifically, TT propensity decreases with increasing GB misorientation angle; however, the decreasing trend is not monotonic. Further, the detailed analysis of TT events combining macroscopic Schmid factor and the geometric measures (m’: accounts for the alignment of both twin plane normal and shear directions; m”: accounts for the alignment of only twin shear directions; and m”’: accounts for the alignment of only twin plane normal) helps in understanding and identifying the process of twin-pair formation, i.e., co-nucleation versus transmission, also to identify the role of local stresses at GBs induced by inter-grain interaction versus twinning shear transformation processes. In addition, Bayesian inference is used to draw statistically meaningful conclusions as to the likelihood of transmission given misorientation angles. Lastly, twin chain frequency as a function of chain length is predicted using the probability distribution obtained from the Bayesian inference and compared with the actual data from the EBSD microstructures.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Inference of bipolar neutrino flavor oscillations near a core-collapse supernova based on multiple measurements at Earth

Neutrinos in compact-object environments, such as core-collapse supernovae, can experience various kinds of collective effects in flavor space, engendered by neutrino-neutrino interactions. These include “bipolar” collective oscillations, which are exhibited by neutrino ensembles where different flavors dominate at different energies. Considering the importance of neutrinos in the dynamics and nucleosynthesis in these environments, it is desirable to ascertain whether an Earth-based detection could contain signatures of bipolar oscillations that occurred within a supernova envelope. To that end, we, in this study, continue examining a cost-function formulation of statistical data assimilation (SDA) to infer solutions to a small-scale model of neutrino flavor transformation. SDA is an inference paradigm designed to optimize a model with sparse data. Our model consists of two monoenergetic neutrino beams with different energies emanating from a source and coherently interacting with each other and with a matter background, with radially varying interaction strengths. We attempt to infer flavor transformation histories of these beams using simulated measurements of the flavor content at locations “in vacuum” (that is, far from the source), which could in principle correspond to Earth-based detectors. Within the scope of this small-scale model, we found that: (i) based on such measurements, the SDA procedure is able to infer whether bipolar oscillations had occurred within the protoneutron star envelope, and (ii) if the measurements sample the full amplitude of the neutrino oscillations in vacuum, then the amplitude of the prior bipolar oscillations is well predicted. This result intimates that the inference paradigm can well complement numerical integration codes, via its ability to infer flavor evolution at physically inaccessible locations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Advanced data analysis in inertial confinement fusion and high energy density physics

Bayesian analysis enables flexible and rigorous definition of statistical model assumptions with well-characterized propagation of uncertainties and resulting inferences for single-shot, repeated, or even cross-platform data. This approach has a strong history of application to a variety of problems in physical sciences ranging from inference of particle mass from multi-source high-energy particle data to analysis of black-hole characteristics from gravitational wave observations. The recent adoption of Bayesian statistics for analysis and design of high-energy density physics (HEDP) and inertial confinement fusion (ICF) experiments has provided invaluable gains in expert understanding and experiment performance. In this Review, we discuss the basic theory and practical application of the Bayesian statistics framework. We highlight a variety of studies from the HEDP and ICF literature, demonstrating the power of this technique. Due to the computational complexity of multi-physics models needed to analyze HEDP and ICF experiments, Bayesian inference is often not computationally tractable. Two sections are devoted to a review of statistical approximations, efficient inference algorithms, and data-driven methods, such as deep-learning and dimensionality reduction, which play a significant role in enabling use of the Bayesian framework. We provide additional discussion of various applications of Bayesian and machine learning methods that appear to be sparse in the HEDP and ICF literature constituting possible next steps for the community. We conclude by highlighting community needs, the resolution of which will improve trust in data-driven methods that have proven critical for accelerating the design and discovery cycle in many application areas.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

An exponential build-up in seismic energy suggests a months-long nucleation of slow slip in Cascadia

Slow slip events result from the spontaneous weakening of the subduction megathrust and bear strong resemblance to earthquakes, only slower. This resemblance allows us to study fundamental aspects of nucleation that remain elusive for classic, fast earthquakes. We rely on machine learning algorithms to infer slow slip timing from statistics of seismic waveforms. We find that patterns in seismic power follow the 14-month slow slip cycle in Cascadia, arguing in favor of the predictability of slow slip rupture. Here, we show that seismic power exponentially increases as the slowly slipping portion of the subduction zone approaches failure, a behavior that shares a striking similarity with the increase in acoustic power observed prior to laboratory slow slip events. Our results suggest that the nucleation phase of Cascadia slow slip events may last from several weeks up to several months.

58 GEOSCIENCES↗
DOE OSTI2024

Consistent and reproducible computation of the glass transition temperature from molecular dynamics simulations

In many fields, from semiconductors for opto-electronic applications to ionic liquids (ILs) for separations, the glass transition temperature (Tg) of a material is a useful gauge for its potential use in practical settings. As a result, there is a great deal of interest in predicting Tg using molecular simulations. However, the uncertainty and variation in the trend shift method, a common approach in simulations to predict Tg, can be high. This is due to the need for human intervention in defining a fitting range for linear fits of density with temperature assumed for the liquid and glass phases across the simulated cooling. The definition of such fitting ranges then defines the estimate for the Tg as the intersection of linear fits. We eliminate this need for human intervention by leveraging the Shapiro–Wilk normality test and proposing an algorithm to define the fitting ranges and, consequently, Tg. Through this integration, we incorporate into our automated methodology that residuals must be normally distributed around zero for any fit, a requirement that must be met for any regression problem. Consequently, fitting ranges for realizing linear fits for each phase are statistically defined rather than visually inferred, obtaining an estimate for Tg without any human intervention. The method is also capable of finding multiple linear regimes across density vs temperature curves. We compare the predictions of our proposed method across multiple IL and semiconductor molecular dynamics simulation results from the literature and compare other proposed methods for automatically detecting Tg from density–temperature data. We believe that our proposed method would allow for more consistent predictions of Tg. We make this methodology available and open source through GitHub.

Chemistry↗
DOE OSTI2020

Gauges, loops, and polynomials for partition functions of graphical models

Graphical models represent multivariate and generally not normalized probability distributions. Computing the normalization factor, called the partition function, is the main inference challenge relevant to multiple statistical and optimization applications. The problem is #P-hard that is of an exponential complexity with respect to the number of variables. Here, aimed at approximating the partition function, we consider multi-graph models where binary variables and multivariable factors are associated with edges and nodes, respectively, of an undirected multi-graph. We suggest a new methodology for analysis and computations that combines the Gauge function technique from Chertkov and Chernyak with the technique developed in Anari and Oveis Gharan 2017 arXiv:1702.02937; Gurvits 2011 arXiv:1106.2844; Straszak and Vishnoi 2017 55th Annual Allerton Conf. on Communication, Control, and Computing, based on the recent progress in the field of real stable polynomials. We show that the Gauge function, representing a single-out term in a finite sum expression for the partition function which achieves extremum at the so-called belief-propagation gauge, has a natural polynomial representation in terms of gauges/variables associated with edges of the multi-graph. Moreover, Gauge function can be used to recover the partition function through a sequence of transformations allowing appealing algebraic and graphical interpretations. Algebraically, one step in the sequence consists of the application of a differential operator over gauges associated with an edge. Graphically, the sequence is interpreted as a repetitive elimination/contraction of edges resulting in multi-graph models on decreasing in size (number of edges) graphs with the same partition function as in the original multi-graph model. Even though the complexity of computing factors in the sequence of the derived multi-graph models and respective Gauge functions grow exponentially with the number of eliminated edges, polynomials associated with the new factors remain bi-stable if the original factors have this property. Moreover, we show that BP estimations in the sequence do not decrease, each low-bounding the partition function.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Forecasting constraints on the mean free path of ionizing photons at z ≥ 5.4 from the Lyman-α forest flux autocorrelation function

ABSTRACT Fluctuations in Lyman-α (Ly α) forest transmission towards high-z quasars are partially sourced from spatial fluctuations in the ultraviolet background, the level of which are set by the mean free path of ionizing photons (λmfp). The autocorrelation function of Ly α forest flux characterizes the strength and scale of transmission fluctuations and, as we show, is thus sensitive to λmfp. Recent measurements at z ∼ 6 suggest a rapid evolution of λmfp at z > 5.0 which would leave a signature in the evolution of the autocorrelation function. For this forecast, we model mock Ly α forest data with properties similar to the XQR-30 extended data set at 5.4 ≤ z ≤ 6.0. At each z, we investigate 100 mock data sets and an ideal case where mock data matches model values of the autocorrelation function. For ideal data with λmfp = 9.0 cMpc at z = 6.0, we recover $\lambda _{\text{mfp}}=12^{+6}_{-3}$ cMpc. This precision is comparable to direct measurements of λmfp from the stacking of quasar spectra beyond the Lyman limit. Hypothetical high-resolution data leads to a $\sim 40~{{\ \rm per\ cent}}$ reduction in the error bars over all z. The distribution of mock values of the autocorrelation function in this work is highly non-Gaussian for high-z, which should caution work with other statistics of the high-z Ly α forest against making this assumption. We use a rigorous statistical method to pass an inference test, however future work on non-Gaussian methods will enable higher precision measurements.

Astronomy & Astrophysics↗
DOE OSTI2023

Effects of baryonic feedback on the cosmic web

Here, we study the effect of baryons on the cosmic web—halos, filaments, walls, and voids. To do so, we apply a modified version of NEXUS, a cosmic web morphological analysis algorithm, to the IllustrisTNG simulations. We find that halos lose more than 10% of their mass due to baryons, mostly to filaments and a small portion to walls and voids. However, the mass transfer does not significantly shift the boundaries of structures, leaving the volume fractions of the cosmic structures largely unaffected. We quantify the effects of baryonic feedback on the power spectrum and the probability density function of the density field for individual cosmic structures. For the power spectrum, most suppression due to feedback can be accounted for by including M ≥ 10 12 M ⊙ /h halos, without considering other cosmic structures. However, when examining the probability density function of the density field, we find nearly 100% suppression of the emptiest regions and 10%-level effects (boost or suppression) in the remaining regions of filaments, walls, and voids. Our results indicate the importance of modeling the effects of baryons in the whole cosmic web, not just halos, for cosmological analysis beyond two-point statistics or field-based inferences.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Masses of Sunyaev-Zel’dovich galaxy clusters detected by the Atacama Cosmology Telescope: Stacked lensing measurements with Subaru HSC year 3 data

We present a stacked lensing analysis of 96 galaxy clusters selected by the thermal Sunyaev-Zel’dovich (SZ) effect in maps of the cosmic microwave background (CMB). We select foreground galaxy clusters with a 5 σ -level SZ threshold in CMB observations from the Atacama Cosmology Telescope, while we define background source galaxies for the lensing analysis with secure photometric redshift cuts in Year 3 data of the Subaru Hyper Suprime Cam survey. We detect the stacked lensing signal in the range of 0.1 < R [ h - 1 Mpc ] < 100 in each of three cluster redshift bins, 0.092 < z ≤ 0.445 , 0.445 < z ≤ 0.695 , and 0.695 < z ≤ 1.180 , with 32 galaxy clusters in each bin. The cumulative signal-to-noise ratios of the lensing signal are 14.6, 12.0, and 6.6, respectively. Using a halo-based forward model, we then constrain statistical relationships between the mass inferred from the SZ observation (i.e. SZ mass) and the total mass derived from our stacked lensing measurements. At the average SZ mass in the cluster sample ( 2.1 - 2.4 × 10 14 h - 1 M ⊙ ), our likelihood analysis shows that the average total mass differs from the SZ counterpart by a factor of 1.3 ± 0.2 , 1.6 ± 0.2 , and 1.6 ± 0.3 (68%) in the aforementioned redshift ranges, respectively. Our limits are consistent with previous lensing measurements, and we find that the cluster modeling choices can introduce a 1 σ -level difference in our parameter inferences.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Leveraging Application Data Constraints to Optimize Database-Backed Web Applications

Exploiting the relationships among data is a classical query optimization technique. As persistent data is increasingly being created and maintained programmatically, prior work that infers data relationships from data statistics misses an important opportunity. We present Coco, the first tool that identifies data relationships by analyzing database-backed applications. Once identified, Coco leverages the constraints to optimize the application's physical design and query execution. Instead of developing a fixed set of predefined rewriting rules, Coco employs an enumerate-test-verify technique to automatically exploit the discovered data constraints to improve query execution. Each resulting rewrite is provably equivalent to the original query. Using 14 real-world web applications, our experiments show that Coco can discover numerous data constraints from code analysis and improve real-world application performance significantly.

Computer Science↗
DOE OSTI2022

A Search of the Full Six Years of the Dark Energy Survey for Outer Solar System Objects

We present a search for outer solar system objects in the 6 yr of data from the Dark Energy Survey (DES). The DES covered a contiguous 5000 deg 2 of the southern sky with ≈80,000 3 deg 2 exposures in the grizY filters between 2013 and 2019. This search yielded 812 trans-Neptunian objects (TNOs), one Centaur and one Oort cloud comet, 458 reported here for the first time. We present methodology that builds upon our previous search on the first 4 yr of data. All images were reprocessed with an optimized detection pipeline that leads to an average completeness gain of 0.47 mag per exposure, as well as improved transient catalog production and algorithms for linkage of detections into orbits. All objects were verified by visual inspection and by the “sub-threshold significance,” the signal-to-noise ratio in the stack of images in which its presence is indicated by the orbit, but no detection was reported. This yields a pure catalog complete to r ≈ 23.8 mag and distances 29 < d < 2500 au. The TNOs have minimum (median) of 7 (12) nights’ detections and arcs of 1.1 (4.2) yr, and will have grizY magnitudes available in a further publication. We present software for simulating our observational biases for comparisons of models to our detections. Initial inferences demonstrating the catalog’s statistical power are: the data are inconsistent with the CFEPS-L7 model for the classical Kuiper Belt; the 16 “extreme” TNOs (a > 150 au, q > 30 au) are consistent with the null hypothesis of azimuthal isotropy; and nonresonant TNOs with q > 38 au, a > 50 au show a significant tendency to be sunward of major mean-motion resonances.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Statistical data analysis of x-ray spectroscopy data enabled by neural network accelerated Bayesian inference

Bayesian inference applied to x-ray spectroscopy data analysis enables uncertainty quantification necessary to rigorously test theoretical models. However, when comparing to data, detailed atomic physics and radiation transfer calculations of x-ray emission from non-uniform plasma conditions are typically too slow to be performed in line with statistical sampling methods, such as Markov Chain Monte Carlo sampling. Furthermore, differences in transition energies and x-ray opacities often make direct comparisons between simulated and measured spectra unreliable. Here, we present a spectral decomposition method that allows for corrections to line positions and bound–bound opacities to best fit experimental data, with the goal of providing quantitative feedback to improve the underlying theoretical models and guide future experiments. In this work, we use a neural network (NN) surrogate model to replace spectral calculations of isobaric hot-spots created in Kr-doped implosions at the National Ignition Facility. The NN was trained on calculations of x-ray spectra using an isobaric hot-spot model post-processed with Cretin, a multi-species atomic kinetics and radiation code. The speedup provided by the NN model to generate x-ray emission spectra enables statistical analysis of parameterized models with sufficient detail to accurately represent the physical system and extract the plasma parameters of interest.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Hierarchical Inference with Bayesian Neural Networks: An Application to Strong Gravitational Lensing

In the past few years, approximate Bayesian Neural Networks (BNNs) have demonstrated the ability to produce statistically consistent posteriors on a wide range of inference problems at unprecedented speed and scale. However, any disconnect between training sets and the distribution of real-world objects can introduce bias when BNNs are applied to data. This is a common challenge in astrophysics and cosmology, where the unknown distribution of objects in our universe is often the science goal. In this work, we incorporate BNNs with flexible posterior parameterizations into a hierarchical inference framework that allows for the reconstruction of population hyperparameters and removes the bias introduced by the training distribution. We focus on the challenge of producing posterior PDFs for strong gravitational lens mass model parameters given Hubble Space Telescope–quality single-filter, lens-subtracted, synthetic imaging data. We show that the posterior PDFs are sufficiently accurate (statistically consistent with the truth) across a wide variety of power-law elliptical lens mass distributions. We then apply our approach to test data sets whose lens parameters are drawn from distributions that are drastically different from the training set. We show that our hierarchical inference framework mitigates the bias introduced by an unrepresentative training set's interim prior. Simultaneously, we can precisely reconstruct the population hyperparameters governing our test distributions. Our full pipeline, from training to hierarchical inference on thousands of lenses, can be run in a day. The framework presented here will allow us to efficiently exploit the full constraining power of future ground- and space-based surveys (https://github.com/swagnercarena/ovejero).

79 ASTRONOMY AND ASTROPHYSICS↗