Search NASA⌕ Search

SEARCH · Search NASA

Results for “STATISTICAL”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Effects of overlapping sources on cosmic shear estimation: Statistical sensitivity and pixel-noise bias

The next generation of dark-energy imaging surveys — so called “Stage-IV” surveys, such as that of the Rubin Observatory Legacy Survey of Space and Time (LSST) — will cross a threshold in the number density of detected sources on the sky that requires qualitatively different image analysis and measurement techniques compared to the current generation of Stage-III surveys. In Stage-IV surveys, a significant amount of the cosmologically useful information is due to sources whose images overlap with those of other sources on the sky. Here, we focus on the weak gravitational lensing probe, for which we expect the largest impact since the cosmic shear signal is primarily encoded in the estimated shapes of observed galaxies and thus directly impacted by overlaps. We introduce a framework based on the Fisher formalism to analyze the effect of the overlapping sources (“blending”) on the estimation of cosmic shear. This method gives concrete predictions for the minimum loss of information due to noise and blending for any choice of “deblending” scheme and shape-measurement algorithm. Our studies account for undetected sources but do not address their full effects and biases they may introduce. We use simulated images and predict this impact of blending for three surveys: the Dark Energy Survey (DES), the Hyper-Suprime Cam Subaru Strategic Program (HSC-SSP), and the Rubin LSST. Our methodology successfully estimates the statistical sensitivity to weak lensing for DES and HSC early results. For LSST, we present the expected loss in statistical sensitivity for the ten-year survey due to blending. We find that for approximately 62% of galaxies that are likely to be detected in full-depth LSST images, at least 1% of the flux in their pixels is from overlapping sources. We also find that the statistical correlations between measures of overlapping galaxies and, to a much lesser extent (0.2%) the higher shot noise level due to their presence, decrease the effective number density of galaxies, N eff , by ~ 18%. We calculate an upper limit on N eff of 39.4 galaxies per arcmin 2 in r band. We study the impact of stars on as a function of stellar density and illustrate the diminishing returns of extending the survey into lower Galactic latitudes. We extend the simulation-based Fisher formalism to predict the expected increase in pixel-noise bias due to blending for maximum-likelihood (ML) shape estimators. We find that noise bias depends sensitively on the particular shape estimator and measure of ensemble-average shape that is used, and properties of the galaxy that include redshift-dependent quantities such as size and luminosity. The source code for these studies is available online.[The documented software developed for the catalog-level studies are available in the open-source LSST DESC github repository https://github.com/LSSTDESC/WeakLensingDeblending. The software for analyzing one or two galaxies with user-defined parameters is in the open-source github repository https://github.com/ismael-mendoza/ShapeMeasurementFisherFormalism.]

79 ASTRONOMY AND ASTROPHYSICS↗

Map-level baryonification: unified treatment of weak lensing two-point and higher-order statistics

Precision cosmology benefits from extracting maximal information from cosmic structures, motivating the use of higher-order statistics (HOS) at small spatial scales. However, predicting how baryonic processes modify matter statistics at these scales has been challenging. The baryonic correction model (BCM) addresses this by modifying dark-matter-only simulations to mimic baryonic effects, providing a flexible, simulation-based framework for predicting both two-point and HOS. We show that a 3-parameter version of the BCM can jointly fit weak lensing maps' two-point statistics, wavelet phase harmonics coefficients, scattering coefficients, and the third and fourth moments to within 2% accuracy across all scales ℓ < 2000 and tomographic bins for a DES-Y3-like redshift distribution ( z ≲ 2), using the FLAMINGO simulations. These results demonstrate the viability of BCM-assisted, simulation-based weak lensing inference of two-point and HOS, paving the way for robust cosmological constraints that fully exploit non-Gaussian information on small spatial scales.

79 ASTRONOMY AND ASTROPHYSICS↗

A study on the statistical significance of mutual information between morphology of a galaxy and its large-scale environment

ABSTRACT A non-zero mutual information between morphology of a galaxy and its large-scale environment is known to exist in Sloan Digital Sky Survey (SDSS) upto a few tens of Mpc. It is important to test the statistical significance of these mutual information if any. We propose three different methods to test the statistical significance of these non-zero mutual information and apply them to SDSS and Millennium run simulation. We randomize the morphological information of SDSS galaxies without affecting their spatial distribution and compare the mutual information in the original and randomized data sets. We also divide the galaxy distribution into smaller subcubes and randomly shuffle them many times keeping the morphological information of galaxies intact. We compare the mutual information in the original SDSS data and its shuffled realizations for different shuffling lengths. Using a t-test, we find that a small but statistically significant (at $99.9{{\ \rm per\ cent}}$ confidence level) mutual information between morphology and environment exists upto the entire length-scale probed. We also conduct another experiment using mock data sets from a semi-analytic galaxy catalogue where we assign morphology to galaxies in a controlled manner based on the density at their locations. The experiment clearly demonstrates that mutual information can effectively capture the physical correlations between morphology and environment. Our analysis suggests that physical association between morphology and environment may extend to much larger length-scales than currently believed, and the information theoretic framework presented here can serve as a sensitive and useful probe of the assembly bias and large-scale environmental dependence of galaxy properties.

Sarkar, Suman↗

Nearest neighbour distributions: New statistical measures for cosmological clustering

ABSTRACT The use of summary statistics beyond the two-point correlation function to analyse the non-Gaussian clustering on small scales, and thereby, increasing the sensitivity to the underlying cosmological parameters, is an active field of research in cosmology. In this paper, we explore a set of new summary statistics – the k-Nearest Neighbour Cumulative Distribution Functions (kNN-CDF). This is the empirical cumulative distribution function of distances from a set of volume-filling, Poisson distributed random points to the k-nearest data points, and is sensitive to all connected N-point correlations in the data. The kNN-CDF can be used to measure counts in cell, void probability distributions, and higher N-point correlation functions, all using the same formalism exploiting fast searches with spatial tree data structures. We demonstrate how it can be computed efficiently from various data sets – both discrete points, and the generalization for continuous fields. We use data from a large suite of N-body simulations to explore the sensitivity of this new statistic to various cosmological parameters, compared to the two-point correlation function, while using the same range of scales. We demonstrate that the use of kNN-CDF improves the constraints on the cosmological parameters by more than a factor of 2 when applied to the clustering of dark matter in the range of scales between 10 and $40\, h^{-1}\, {\rm Mpc}$. We also show that relative improvement is even greater when applied on the same scales to the clustering of haloes in the simulations at a fixed number density, both in real space, as well as in redshift space. Since the kNN-CDF are sensitive to all higher order connected correlation functions in the data, the gains over traditional two-point analyses are expected to grow as progressively smaller scales are included in the analysis of cosmological data, provided the higher order correlation functions are sensitive to cosmology on the scales of interest.

79 ASTRONOMY AND ASTROPHYSICS↗

Dark energy survey year 3 results: likelihood-free, simulation-based w CDM inference with neural compression of weak-lensing map statistics

We present simulation-based cosmological wcold dark matter (wCDM) inference using dark energy survey year 3 weak-lensing maps, via neural data compression of weak-lensing map summary statistics: power spectra, peak counts, and direct map-level compression/inference with convolutional neural networks (CNN). Using simulation-based inference, also known as likelihood-free or implicit inference, we use forward-modelled mock data to estimate posterior probability distributions of unknown parameters. This approach allows all statistical assumptions and uncertainties to be propagated through the forward-modelled mock data; these include sky masks, non-Gaussian shape noise, shape measurement bias, source galaxy clustering, photometric redshift uncertainty, intrinsic galaxy alignments, non-Gaussian density fields, neutrinos, and non-linear summary statistics. We include a series of tests to validate our inference results. This paper also describes the Gower Street simulation suite: 791 full-sky pkdgrav3 dark matter simulations, with cosmological model parameters sampled with a mixed active-learning strategy, from which we construct over 3000 mock dark energy survey lensing data sets. For wCDM inference, for which we allow –1 < w < –$\frac{1}{3}$⁠, our most constraining result uses power spectra combined with map-level (CNN) inference. Using gravitational lensing data only, this map-level combination gives Ω m = 0.283$^{+0.020}_{–0.027}$⁠, S 8 = 0.804$^{+0.025}_{–0.017⁠}$, and w < –0.80 (with a 68 per cent credible interval); compared to the power spectrum inference, this is more than a factor of two improvement in dark energy parameter (Ω⁠ DE , w⁠) precision.

79 ASTRONOMY AND ASTROPHYSICS↗

Cosmology with second- and third-order shear statistics for the Dark Energy Survey: Methods and simulated analysis

We present a new pipeline designed for the robust inference of cosmological parameters using both second- and third-order shear statistics. We build a theoretical model for rapid evaluation of three-point correlations using our fastnc code and integrate it into the cosmosis framework. We measure the two-point functions 𝜉 ± and the full configuration-dependent three-point shear correlation functions across all auto- and cross-redshift bins. We compress the three-point functions into the mass aperture statistic ⟨ℳ$^{3}_{ap}$⟩ for a set of 796 simulated shear maps designed to model the Dark Energy Survey Year 3 data. We estimate from it the full covariance matrix and model the effects of intrinsic alignments, shear calibration biases and photometric redshift uncertainties. We apply scale cuts to minimize the contamination from the baryonic signal as modeled through hydrodynamical simulations. We find a significant improvement of 83% on the figure of merit in the Ω m − 𝑆 8 plane when we add the ⟨ℳ$^{3}_{ap}$⟩ data to 𝜉 ± . Here, we present our findings for all relevant cosmological and systematic uncertainty parameters and discuss the complementarity of third-order and second-order statistics.

79 ASTRONOMY AND ASTROPHYSICS↗

Improving statistical precision in Monte Carlo samples with negative weights via reweighting and uncertainty quantification

High statistical precision is critical for Monte Carlo (MC) samples in high energy physics and is degraded by negatively weighted events. This paper investigates a procedure to learn the relationship between the negative and positive weight distributions of any sample, allowing the reduction of statistical uncertainty by reweighting kinematically equivalent events with the same sign. A robust uncertainty quantification method is required for the practical application of such method. Two methods for the estimation of the reweighting uncertainty are developed: one at the event and another one at the final observable level. The latter method is strongly favored. The gains in statistical precision are then quantified. The method is demonstrated on Sherpa vector boson plus jets samples when using all generated events and when restricted to the signal region of a mock analysis. It is demonstrated to significantly reduce stochastic behavior in sparse MC samples while decreasing the overall uncertainty with a sufficiently well-known reweighting function.

Monte Carlo methods↗

Statistical uncertainties of the v n { 2 k } harmonics from Q cumulants

Analytic formulas to calculate statistical uncertainties of v n { 2 k } harmonics extracted from the Q cumulants are presented. The Q cumulants are multivariate polynomial functions of the weighted means of 2m-particle azimuthal correlations, $\langle\langle 2 m \rangle\rangle$. Variances and covariances of the $\langle\langle 2 m \rangle\rangle$ are included in the analytic formulas of the uncertainties that can be calculated simultaneously with the calculations of the v n { 2 k } harmonics. The calculations are performed using a simple toy model, which roughly simulates elliptic flow azimuthal anisotropy with magnitudes around 0.05. The results are compared with the results obtained by the many data subsets, and by the bootstrapping method. The first one is a common way of estimation of the statistical uncertainties of the v n { 2 k } harmonics in a real experiment. In order to increase precision in the measurement of the v n { 2 k } harmonics, a large number of 15 000 subsets and the same number of the resampling in the bootstrap method is used. Unlike the other ways of the analytic calculation of the v n { 2 k } statistical uncertainties, our proposal that includes the use of squared weights in the calculation of both the variances and covariances, gives the best agreement with the results obtained from the subsets and bootstrap method. Additionally, a recurrence equation between Q cumulants of any order is also presented.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Using kernel-based statistical distance to study the dynamics of charged particle beams in particle-based simulation codes

Measures of discrepancy between probability distributions (statistical distance) are widely used in the fields of artificial intelligence and machine learning. We describe how certain measures of statistical distance can be implemented as numerical diagnostics for simulations involving charged-particle beams. Related measures of statistical dependence are also described. The resulting diagnostics provide sensitive measures of dynamical processes important for beams in nonlinear or high-intensity systems, which are otherwise difficult to characterize. Here, the focus is on kernel-based methods such as maximum mean discrepancy, which have a well-developed mathematical foundation and reasonable computational complexity. Several benchmark problems and examples involving intense beams are discussed. While the focus is on charged-particle beams, these methods may also be applied to other many-body systems such as plasmas or gravitational systems.

47 OTHER INSTRUMENTATION↗

Statistical upscaling of ecosystem CO 2 fluxes across the terrestrial tundra and boreal domain: Regional patterns and uncertainties

Abstract The regional variability in tundra and boreal carbon dioxide (CO 2 ) fluxes can be high, complicating efforts to quantify sink‐source patterns across the entire region. Statistical models are increasingly used to predict (i.e., upscale) CO 2 fluxes across large spatial domains, but the reliability of different modeling techniques, each with different specifications and assumptions, has not been assessed in detail. Here, we compile eddy covariance and chamber measurements of annual and growing season CO 2 fluxes of gross primary productivity (GPP), ecosystem respiration (ER), and net ecosystem exchange (NEE) during 1990–2015 from 148 terrestrial high‐latitude (i.e., tundra and boreal) sites to analyze the spatial patterns and drivers of CO 2 fluxes and test the accuracy and uncertainty of different statistical models. CO 2 fluxes were upscaled at relatively high spatial resolution (1 km 2 ) across the high‐latitude region using five commonly used statistical models and their ensemble, that is, the median of all five models, using climatic, vegetation, and soil predictors. We found the performance of machine learning and ensemble predictions to outperform traditional regression methods. We also found the predictive performance of NEE‐focused models to be low, relative to models predicting GPP and ER. Our data compilation and ensemble predictions showed that CO 2 sink strength was larger in the boreal biome (observed and predicted average annual NEE −46 and −29 g C m −2 yr −1 , respectively) compared to tundra (average annual NEE +10 and −2 g C m −2 yr −1 ). This pattern was associated with large spatial variability, reflecting local heterogeneity in soil organic carbon stocks, climate, and vegetation productivity. The terrestrial ecosystem CO 2 budget, estimated using the annual NEE ensemble prediction, suggests the high‐latitude region was on average an annual CO 2 sink during 1990–2015, although uncertainty remains high.

Virkkala, Anna‐Maria↗

Divide and conquer: using RhizoVision Explorer to aggregate data from multiple root scans using image concatenation and statistical methods

Roots are important in agricultural and natural systems for determining plant productivity and soil carbon inputs. Sometimes, the amount of roots in a sample is too much to fit into a single scanned image, so the sample is divided among several scans, and there is no standard method to aggregate the data. Here, we describe and validate two methods for standardizing measurements across multiple scans: image concatenation and statistical aggregation. We developed a Python script that identifies which images belong to the same sample and returns a single, larger concatenated image. These concatenated images and the original images were processed with RhizoVision Explorer, a free and open-source software. An R script was developed, which identifies rows of data belonging to the same sample and applies correct statistical methods to return a single data row for each sample. These two methods were compared using example images from switchgrass, poplar, and various tree and ericaceous shrub species from a northern peatland and the Arctic. Most root measurements were nearly identical between the two methods except median diameter, which cannot be accurately computed by statistical aggregation. We believe the availability of these methods will be useful to the root biology community.

59 BASIC BIOLOGICAL SCIENCES↗

A Statistical Evaluation of Combining Human Productivity Metrics in the Indoor Environment

The potential of improving human productivity by providing healthy indoor environments has been a consistent interest in the building field for decades. This research field's long-standing challenge is to measure human productivity given the complex nature of office work. Previous studies have diversified productivity metrics, allowing greater flexibility in collecting human data; however, this diversity complicates the ability to combine productivity metrics from disparate studies within a meta-analysis. This study aims to categorize existing productivity metrics and statistically assess which categories show similar behavior when used to measure the impacts of indoor environmental quality. The 106 productivity metrics compiled were grouped into six productivity metric categories: neurobehavioral speed, accuracy, neurobehavioral response time, call handling time, self-reported productivity, and performance score. Then, this study set neurobehavioral speed as the baseline category given its fitness to the efficiency-based definition of productivity (i.e., output versus input) and conducted three statistical analyses with the other categories to evaluate their similarity. As results, the categories of neurobehavioral response time, self-reported productivity, and call handling time were found to have statistical similarity with neurobehavioral speed. This study contributes to creating a constructive research environment for future meta-analyses to understand which human productivity metrics can be combined with each other.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Perspective on Tsallis statistics for nuclear and particle physics

This is a concise introduction to the topic of nonextensive Tsallis statistics meant especially for those interested in its relation to high-energy proton–proton, proton–nucleus and nucleus–nucleus collisions. The three types of Tsallis statistics are reviewed. Only one of them is consistent with the fundamental hypothesis of equilibrium statistical mechanics. The single-particle distributions associated with it, namely Boltzmann, Fermi–Dirac and Bose–Einstein, are derived. These are not equilibrium solutions to the conventional Boltzmann transport equation which must be modified in a rather nonintuitive manner for them to be so. Nevertheless, the Boltzmann limit of the Tsallis distribution is extremely efficient in representing a wide variety of single-particle distributions in high-energy proton–proton, proton–nucleus and nucleus–nucleus collisions with only three parameters, one of them being the so-called nonextensitivity parameter [Formula: see text]. This distribution interpolates between an exponential at low transverse energy, reflecting thermal equilibrium, to a power law at high transverse energy, reflecting the asymptotic freedom of Quantum Chromodynamics (QCD). It should not be viewed as a fundamental new parameter representing nonextensive behavior in these collisions.

Physics↗

Projecting Future Energy Production from Operating Wind Farms in North America. Part II: Statistical Downscaling

Abstract Capacity factors (CFs) derived from daily expected power at 22 operating wind farms in different regions of North America are used as predictands to train statistical downscaling algorithms using output from ERA5. The statistical downscaling models are then used to make CF projections for a suite of CMIP6 Earth System Models (ESMs). Downscaling is performed using a hybrid statistical approach that employs synoptic types derived using k -means clustering applied to sea level pressure fields with variance corrections applied as a function of the pressure gradient intensity. ESMs exhibit marked variability in terms of the skill with which the frequency of synoptic types and pressure gradients are reproduced relative to ERA5, and that differential skill is used to infer differential credibility in the associated CF projections. Projections of median annual mean CF [P50(CF)] in each 20-yr period from 1980 to 2099 show evidence of declines at most wind farms except in parts of the southern Great Plains, although the magnitude of the changes is strongly dependent on the ESM. For example, P50(CF) in 2080–99 deviate from those in 1980–99 by from −3.1 to +0.2 percentage points in the Northeast. The largest-magnitude declines in P50(CF) ranging from −3.9 to −2 percentage points are projected for the southern West Coast. CF trends exhibit marked seasonality and are strongly linked to changes in the relative intensity of future synoptic patterns, with much less impact from shifts in the occurrence of synoptic types over time. Internal climate modes continue to play a significant role in inducing interannual variability in wind power production, even under high radiative forcing scenarios. Significance Statement We describe how future climate changes may affect wind resources and wind power generation. Near-term changes in projected wind power electricity generation potential at operating wind farms over North America are small, but by the end of the current century electricity production is projected to decrease in many areas but may increase in parts of the southern Great Plains. The amount of change in projected wind power production is a strong function of the Earth system model that is downscaled and also depends on the continued presence of internally forced climate variability. An additional dependence on the amount of greenhouse gas–induced global warming indicates the transition of the energy sector to low-carbon sources may assist in maintaining the abundant U.S. wind resource.

Meteorology & Atmospheric Sciences↗

Overestimated prediction using polygenic prediction derived from summary statistics

When polygenic risk score (PRS) is derived from summary statistics, independence between discovery and test sets cannot be monitored. We compared two types of PRS studies derived from raw genetic data (denoted as rPRS) and the summary statistics for IGAP (sPRS). Two variables with the high heritability in UK Biobank, hypertension, and height, are used to derive an exemplary scale effect of PRS. sPRS without APOE is derived from International Genomics of Alzheimer’s Project (IGAP), which records ΔAUC and ΔR 2 of 0.051 ± 0.013 and 0.063 ± 0.015 for Alzheimer’s Disease Sequencing Project (ADSP) and 0.060 and 0.086 for Accelerating Medicine Partnership - Alzheimer’s Disease (AMP-AD). On UK Biobank, rPRS performances for hypertension assuming a similar size of discovery and test sets are 0.0036 ± 0.0027 (ΔAUC) and 0.0032 ± 0.0028 (ΔR 2 ). For height, ΔR 2 is 0.029 ± 0.0037. Considering the high heritability of hypertension and height of UK Biobank and sample size of UK Biobank, sPRS results from AD databases are inflated. Independence between discovery and test sets is a well-known basic requirement for PRS studies. However, a lot of PRS studies cannot follow such requirements because of impossible direct comparisons when using summary statistics. Thus, for sPRS, potential duplications should be carefully considered within the same ethnic group.

97 MATHEMATICS AND COMPUTING↗

Multidimensional scaling informed by F -statistic: Visualizing grouped microbiome data with inference

Multidimensional scaling (MDS) is a widely used dimensionality reduction technique in microbial ecology data analysis that captures the multivariate structure of the data while preserving pairwise distances between samples. While improvements in MDS have enhanced the ability to reveal group-specific data patterns, these MDS-based methods require prior assumptions for inference, limiting their application in general microbiome analysis. Here, in this study, we introduce a new MDS-based ordination method, “F-informed MDS,” which configures the data distribution based on the F-statistic, the ratio of dispersion between groups sharing common and different characteristics. Using semisynthetic datasets, we demonstrate that the proposed method is robust to hyperparameter selection while maintaining statistical significance throughout the ordination process. Various quality metrics for evaluating dimensionality reduction confirm that F-informed MDS is comparable to state-of-the-art methods in preserving both local and global data structures. Its application to a diatom-associated bacterial community suggests the role of this new method in interpreting the community’s response to the host. Our approach offers a well-founded refinement of MDS that aligns with statistical test results, which can be beneficial for broader multidimensional data analyses in microbiology and ecology. This new visualization tool can be incorporated into standard microbiome data analyses.

Biological and medical sciences↗

Hot Droughts and Forest Tree Dynamics in the Amazon - Statistical Models, Scripts, Data, and Outputs

This package contains data, outputs, equations, and R scripts for analyses for manuscript entitled "Hot droughts in the Amazon: A window to a future hypertropical climate" by J. Chambers et al., in particular it contains statistical models and analyses for the INPA BIONTE tree mortality study. The Models folder contains details for all statistical models in PDF files. The Scripts folder contains the R scripts for Bayesian Hierarchical Models (two text files) and SEMs (one text file) are separate and reasonably annotated. All data associated with these scripts are in the data folder. The Data folder contains two of the three CSV files used for the analyses and are called by the R scripts. Two of them are part of published datasets (`BIONTE_mortality-rates.csv` from Lima et al. 2024, DOI:10.15486/ngt/1898910 and `SPEI.csv` from Pastorello et al. 2023 DOI:10.15486/ngt/1958257) and also provided in this package for convenience (please see the corresponding datasets for usage and citation terms). The third dataset (`BIONTE_gapfilled_wd.csv`) contains sensitive information and can be obtained by contacting the manuscript lead author. The Outputs folder contains the two output files that provide extra information about the analyses. The file `figuresFeb2025d.pdf` contains all the figures from the manuscript - captions are in the manuscript. The file `ChambersMS.pdf` contains primary results from Bayesian statistical models, regression analyses, and validation steps applied to the tree mortality data from the INPA experiments. The document includes visual summaries, model diagnostics, and leave-one-out (LOO) validation results. A breakdown of file contents can be found in the README file that is part of this package.

54 ENVIRONMENTAL SCIENCES↗

Publishing statistical models: Getting the most out of particle physics experiments

The statistical models used to derive the results of experimental analyses are of incredible scientific value and are essential information for analysis preservation and reuse. In this paper, we make the scientific case for systematically publishing the full statistical models and discuss the technical developments that make this practical. By means of a variety of physics cases - including parton distribution functions, Higgs boson measurements, effective field theory interpretations, direct searches for new physics, heavy flavor physics, direct dark matter detection, world averages, and beyond the Standard Model global fits - we illustrate how detailed information on the statistical modelling can enhance the short- and long-term impact of experimental results.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗