Search NASA⌕ Search

SEARCH · Search NASA

Results for “Summary statistics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Overestimated prediction using polygenic prediction derived from summary statistics

When polygenic risk score (PRS) is derived from summary statistics, independence between discovery and test sets cannot be monitored. We compared two types of PRS studies derived from raw genetic data (denoted as rPRS) and the summary statistics for IGAP (sPRS). Two variables with the high heritability in UK Biobank, hypertension, and height, are used to derive an exemplary scale effect of PRS. sPRS without APOE is derived from International Genomics of Alzheimer’s Project (IGAP), which records ΔAUC and ΔR 2 of 0.051 ± 0.013 and 0.063 ± 0.015 for Alzheimer’s Disease Sequencing Project (ADSP) and 0.060 and 0.086 for Accelerating Medicine Partnership - Alzheimer’s Disease (AMP-AD). On UK Biobank, rPRS performances for hypertension assuming a similar size of discovery and test sets are 0.0036 ± 0.0027 (ΔAUC) and 0.0032 ± 0.0028 (ΔR 2 ). For height, ΔR 2 is 0.029 ± 0.0037. Considering the high heritability of hypertension and height of UK Biobank and sample size of UK Biobank, sPRS results from AD databases are inflated. Independence between discovery and test sets is a well-known basic requirement for PRS studies. However, a lot of PRS studies cannot follow such requirements because of impossible direct comparisons when using summary statistics. Thus, for sPRS, potential duplications should be carefully considered within the same ethnic group.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Towards testing the theory of gravity with DESI: summary statistics, model predictions and future simulation requirements

Shortly after its discovery, General Relativity (GR) was applied to predict the behavior of our Universe on the largest scales, and later became the foundation of modern cosmology. Its validity has been verified on a range of scales and environments from the Solar system to merging black holes. However, experimental confirmations of GR on cosmological scales have so far lacked the accuracy one would hope for — its applications on those scales being largely based on extrapolation and its validity there sometimes questioned in the shadow of the discovery of the unexpected cosmic acceleration. Future astronomical instruments surveying the distribution and evolution of galaxies over substantial portions of the observable Universe, such as the Dark Energy Spectroscopic Instrument (DESI), will be able to measure the fingerprints of gravity and their statistical power will allow strong constraints on alternatives to GR. In this paper, based on a set of N-body simulations and mock galaxy catalogs, we study the predictions of a number of traditional and novel summary statistics beyond linear redshift distortions in two well-studied modified gravity models — chameleon f(R) gravity and a braneworld model — and the potential of testing these deviations from GR using DESI. These summary statistics employ a wide array of statistical properties of the galaxy and the underlying dark matter field, including two-point and higher-order statistics, environmental dependence, redshift space distortions and weak lensing. We find that they hold promising power for testing GR to unprecedented precision. The major future challenge is to make realistic, simulation-based mock galaxy catalogs for both GR and alternative models to fully exploit the statistic power of the DESI survey (by matching the volumes and galaxy number densities of the mocks to those in the real survey) and to better understand the impact of key systematic effects. Using these, we identify future simulation and analysis needs for gravity tests using DESI.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

National Summary Statistics for Depot-Based Medium- and Heavy-Duty Vehicle Operations

This file provides nationally aggregated summary statistics to characterize the daily operations of medium- and heavy-duty vehicles. It provides typical daily driving distances, domicile dwell durations, and one version of potential normalized electric vehicle depot charging load curves. The methodology to create these data was designed with national representativeness in mind, and the data are suitable for national analysis.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Evaluating MPI resource usage summary statistics

The Message Passing Interface (MPI) remains the dominant programming model for scientific applications running on today’s high-performance computing (HPC) systems. This dominance stems from MPI’s powerful semantics for inter-process communication that has enabled scientists to write applications for simulating important physical phenomena. MPI does not, however, specify how messages and synchronization should be carried out. Those details are typically dependent on low-level architecture details and the message characteristics of the application. Therefore, analyzing an application’s MPI resource usage is critical to tuning MPI’s performance on a particular platform. The result of this analysis is typically a discussion of the mean message sizes, queue search lengths and message arrival times for a workload or set of workloads. While a discussion of the arithmetic mean in MPI resource usage might be the most intuitive summary statistic, it is not always the most accurate in terms of representing the underlying data. In this paper, we analyze MPI resource usage for a number of key MPI workloads using an existing MPI trace collector and discrete-event simulator. Our analysis demonstrates that the average, while easy and efficient to calculate, is a useful metric for characterizing latency and bandwidth measurements, but may not be a good representation of application message sizes, match list search depths, or MPI inter-operation times. Additionally, we show that the median and mode are superior choices in many cases. We also observe that the arithmetic mean is not the best representation of central tendency for data that are drawn from distributions that are multi-modal or have heavy tails. Furthermore, the results and analysis of our work provide valuable guidance on how we, as a community, should discuss and analyze MPI resource usage data for scientific applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Baltimore Social-Environmental Collaborative (BSEC) Doppler Lidar & Derived Products

This repository contains all processed Doppler‐lidar outputs from the PSU lidar deployed for the Baltimore Social‐Environmental Collaborative (BSEC) project. Vertical Stare Scans (fixed‐beam, vertical profiling): 1 Hz backscatter intensity (m⁻¹ sr⁻¹), signal‐to‐noise ratio (unitless), and Doppler vertical‐velocity (m s⁻¹) on ~30 m range gates, stored as CF-compliant NetCDF. Wind Profiles (horizontal‐wind retrieval): daily NetCDF outputs of retrieved horizontal wind speed (m s⁻¹) and direction (degrees), computed from the angled‐scan returns. Profile Statistics (summary statistics on the vertical velocity): 15 min windows (default) of mean, variance, skewness, kurtosis, high-frequency variance, etc., as a function of height; saved as CF-compliant NetCDF files. Boundary Layer Height (BLH) (fuzzy-logic output): 15 min BLH estimates (m), with lower/upper fuzzy bounds (m) and a quality flag (0–4) indicating data status (e.g., no data, good, below range, ran out of signal, cloud-topped). Cloud Base Height (Haar-gradient detection): 15 min estimates of cloud-base height (m) with a cloud-detection quality flag (0–3: none, low, moderate, high). All five product streams are organized by year and date under their own top-level folders (01_Vertical_Stare_Scans/ through 05_Cloud_Height/). Each folder contains a data_ /YYYY/ subdirectory with daily CF-compliant NetCDF outputs (96 windows per day at 15 min intervals). Global attributes in each file include creation history, version (2.0.0), institution, and source. Instrument & MeasurementsThe PSU Doppler Lidar samples aerosol backscatter (m⁻¹ sr⁻¹), signal-to-noise ratio, and radial velocity at ~1 Hz. Vertical stare scans point the beam straight up; after collecting angled scans through multiple elevation angles, the "Wind Profiles" product contains the fully retrieved horizontal wind speed and direction. Data were collected continuously at ~30 m range resolution, with a typical height ceiling of ~12 km. How to Use Open any NetCDF with Python's xarray, MATLAB, or similar CF-compliant tools. Stare scans and angled-scan retrievals (Wind Profiles) are CF-compliant daily NetCDF files. Profile-Statistics, BLH, and Cloud Height files are daily 15 min summaries (96 time steps per file). Inspect the included variables (e.g., vertical_velocity_variance, wind_speed, BLH, cloud_base_height) for your analyses. Use the quality flags (BLH_flag, cloud_flag) to filter out poor-quality retrievals. For more information or questions about processing methods, please contact:Nicholas E. Prince ⟨nec5299@psu.edu⟩Penn State Department of Meteorology & Atmospheric Science

Air Quality↗
DOE OSTI2026

Baltimore Social-Environmental Collaborative (BSEC) Doppler Lidar & Derived Products

This repository contains all processed Doppler‐lidar outputs from the PSU lidar deployed for the Baltimore Social‐Environmental Collaborative (BSEC) project. Vertical Stare Scans (fixed‐beam, vertical profiling): 1 Hz backscatter intensity (m⁻¹ sr⁻¹), signal‐to‐noise ratio (unitless), and Doppler vertical‐velocity (m s⁻¹) on 30 m range gates (and 3 m range gates), stored as CF-compliant NetCDF. Wind Profiles (horizontal‐wind retrieval): daily NetCDF outputs of retrieved horizontal wind speed (m s⁻¹) and direction (degrees), computed from the angled‐scan returns. Profile Statistics (summary statistics on the vertical velocity): 15 min windows (default) of mean, variance, skewness, kurtosis, high-frequency variance, etc., as a function of height; saved as CF-compliant NetCDF files. Boundary Layer Height (BLH) (fuzzy-logic output): 15 min BLH estimates (m), with lower/upper fuzzy bounds (m) and a quality flag (0–4) indicating data status (e.g., no data, good, ran out of signal, below range, cloud-topped). Cloud Base Height (Haar-gradient detection): 10 min estimates of cloud-base height (m). All five product streams are organized by year and date under their own top-level folders (01_Vertical_Stare_Scans/ through 05_Cloud_Height/). Each folder contains a data_ /YYYY/ subdirectory with daily CF-compliant NetCDF outputs (96 windows per day at 15 min intervals). Global attributes in each file include creation history, version (3.0.0), institution, and source. Instrument & MeasurementsThe PSU Doppler Lidar samples aerosol backscatter (m⁻¹ sr⁻¹), signal-to-noise ratio, and radial velocity at ~1 Hz. Vertical stare scans point the beam straight up; after collecting angled scans through multiple elevation angles, the "Wind Profiles" product contains the fully retrieved horizontal wind speed and direction. Data were collected continuously at ~30 m range resolution (and 3 m for the year of 2025), with a typical height ceiling of ~12 km. How to Use Open any NetCDF with Python's xarray, MATLAB, or similar CF-compliant tools. Stare scans and angled-scan retrievals (Wind Profiles) are CF-compliant daily NetCDF files. Profile-Statistics, BLH, and Cloud Height files are daily 15 min (10 min for Cloud Heights) summaries (96 time steps per file). Inspect the included variables (e.g., vertical_velocity_variance, wind_speed, BLH, cloud_base_height) for your analyses. Use the quality flags (BLH_flag, cloud_flag) to filter out poor-quality retrievals. For more information or questions about processing methods, please contact:Nicholas E. Prince ⟨nec5299@psu.edu⟩Penn State Department of Meteorology & Atmospheric Science

Air Quality↗
DOE OSTI2026

Baltimore Social-Environmental Collaborative (BSEC) Doppler Lidar & Derived Products

This repository contains all processed Doppler‐lidar outputs from the PSU lidar deployed for the Baltimore Social‐Environmental Collaborative (BSEC) project. Vertical Stare Scans (fixed‐beam, vertical profiling): 1 Hz backscatter intensity (m⁻¹ sr⁻¹), signal‐to‐noise ratio (unitless), and Doppler vertical‐velocity (m s⁻¹) on 30 m range gates (and 3 m range gates), stored as CF-compliant NetCDF. Wind Profiles (horizontal‐wind retrieval): daily NetCDF outputs of retrieved horizontal wind speed (m s⁻¹) and direction (degrees), computed from the angled‐scan returns. Profile Statistics (summary statistics on the vertical velocity): 15 min windows (default) of mean, variance, skewness, kurtosis, high-frequency variance, etc., as a function of height; saved as CF-compliant NetCDF files. Boundary Layer Height (BLH) (fuzzy-logic output): 15 min BLH estimates (m), with lower/upper fuzzy bounds (m) and a quality flag (0–4) indicating data status (e.g., no data, good, ran out of signal, below range, cloud-topped). Cloud Base Height (Haar-gradient detection): 10 min estimates of cloud-base height (m). All five product streams are organized by year and date under their own top-level folders (01_Vertical_Stare_Scans/ through 05_Cloud_Height/). Each folder contains a data_ /YYYY/ subdirectory with daily CF-compliant NetCDF outputs (96 windows per day at 15 min intervals). Global attributes in each file include creation history, version (3.0.1), institution, and source. Instrument & MeasurementsThe PSU Doppler Lidar samples aerosol backscatter (m⁻¹ sr⁻¹), signal-to-noise ratio, and radial velocity at ~1 Hz. Vertical stare scans point the beam straight up; after collecting angled scans through multiple elevation angles, the "Wind Profiles" product contains the fully retrieved horizontal wind speed and direction. Data were collected continuously at ~30 m range resolution (and 3 m for the year of 2025), with a typical height ceiling of ~12 km. How to Use Open any NetCDF with Python's xarray, MATLAB, or similar CF-compliant tools. Stare scans and angled-scan retrievals (Wind Profiles) are CF-compliant daily NetCDF files. Profile-Statistics, BLH, and Cloud Height files are daily 15 min (10 min for Cloud Heights) summaries (96 time steps per file). Inspect the included variables (e.g., vertical_velocity_variance, wind_speed, BLH, cloud_base_height) for your analyses. Use the quality flags (BLH_flag, cloud_flag) to filter out poor-quality retrievals. For more information or questions about processing methods, please contact:Nicholas E. Prince ⟨nec5299@psu.edu⟩Penn State Department of Meteorology & Atmospheric Science

Air Quality↗
DOE OSTI2023

Full forward model of galaxy clustering statistics with AbacusSummit light cones

ABSTRACT Novel summary statistics beyond the standard 2-point correlation function (2PCF) are necessary to capture the full astrophysical and cosmological information from the small-scale (r < 30h−1Mpc) galaxy clustering. However, the analysis of beyond-2PCF statistics on small scales is challenging because we lack the appropriate treatment of observational systematics for arbitrary summary statistics of the galaxy field. In this paper, we develop a full forward modelling pipeline for a wide range of summary statistics using the large high-fidelity AbacusSummit light cones that account for many systematic effects as well as remain flexible and computationally efficient to enable posterior sampling. We apply our forward model approach to a fully realistic mock galaxy catalog and demonstrate that we can recover unbiased constraints on the underlying galaxy–halo connection model using two separate summary statistics: the standard 2PCF and the novel k-th nearest neighbour (kNN) statistics, which are sensitive to correlation functions of all orders. We will demonstrate its strong constraining power on extended galaxy–halo connection models and cosmology in follow up papers. We expect this to become a powerful approach when applying to upcoming surveys such as DESI where we can leverage a multitude of summary statistics across a wide redshift range to maximally extract information from the non-linear scales.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Parametrizing the detector response with neural networks

In high energy physics, characterizing the response of a detector to radiation is one of the most important and fundamental experimental tasks. In many cases, this task is accomplished by parameterizing summary statistics of the full detector response probability density. The parameterized detector response can then be used for calibration as well as for directly improving physics analysis sensitivity. This paper reports on how to parameterize summary statistics of the detector response using neural networks. Specifically, neural networks are powerful tools for incorporating multidimensional data and the loss function used during training determines which summary statistic is learned. One common summary statistic that has not been combined with deep learning (as far as the authors are aware) is the mode. A neural network-based approach to mode learning is proposed and empirically demonstrated in the context of high energy jet calibrations. Altogether, the neural network-based toolkit for detector response parameterization can enhance the utility of data collected at high energy physics experiments and beyond.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

2D k -th nearest neighbour statistics: a highly informative probe of galaxy clustering

ABSTRACT Beyond standard summary statistics are necessary to summarize the rich information on non-linear scales in the era of precision galaxy clustering measurements. For the first time, we introduce the 2D k-th nearest neighbour (kNN) statistics as a summary statistic for discrete galaxy fields. This is a direct generalization of the standard 1D kNN by disentangling the projected galaxy distribution from the redshift-space distortion signature along the line-of-sight. We further introduce two different flavours of 2D kNNs that trace different aspects of the galaxy field: the standard flavour which tabulates the distances between galaxies and random query points, and a ‘DD’ flavour that tabulates the distances between galaxies and galaxies. We showcase the 2D kNNs’ strong constraining power both through theoretical arguments and by testing on realistic galaxy mocks. Theoretically, we show that 2D kNNs are computationally efficient and directly generate other statistics such as the popular two-point correlation function (2PCF), voids probability function, and counts-in-cell statistics. In a more practical test, we apply the 2D kNN statistics to simulated galaxy mocks that fold in a large range of observational realism and recover parameters of the underlying extended halo occupation distribution (HOD) model that includes velocity bias and galaxy assembly bias. We find unbiased and significantly tighter constraints on all aspects of the HOD model with the 2D kNNs, both compared to the standard 1D kNN, and the classical redshift-space 2PCF.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Dark Energy Survey Year 3 results: Simulation-based cosmological inference with wavelet harmonics, scattering transforms, and moments of weak lensing mass maps. II. cosmological results

Here, we present a simulation-based cosmological analysis using a combination of Gaussian and non-Gaussian statistics of the weak lensing mass (convergence) maps from the first three years of the Dark Energy Survey. We implement the following: (1) second and third moments; (2) wavelet phase harmonics; (3) the scattering transform. Our analysis is fully based on simulations, spans a space of seven 𝑤 Cold Dark Matter (𝑤⁢ CDM) cosmological parameters, and forward models the most relevant sources of systematics inherent in the data: masks, noise variations, clustering of the sources, intrinsic alignments, and shear and redshift calibration. We implement a neural network compression of the summary statistics, and we estimate the parameter posteriors using a simulation-based inference approach. Including and combining different non-Gaussian statistics is a powerful tool that strongly improves constraints over Gaussian statistics (in our case, the second moments); in particular, the figure of merit (𝑆 8 , Ω m ) is improved by 70% (Λ ⁢CDM) and 90% (𝑤 ⁢CDM). When all the summary statistics are combined, we achieve a 2% constraint on the amplitude of fluctuations parameter 𝑆 8 ≡ 𝜎 8 ⁢(Ω m /0.3) 0.5 , obtaining 𝑆 8 = 0.794 ±0.017 (Λ⁢ CDM) and 𝑆 8 = 0.817 ±0.021 (𝑤 ⁢CDM), and a ∼10% constraint on Ω m , obtaining Ω m =0.259 ±0.025 (Λ ⁢CDM) and Ω m = 0.273 ±0.029 (𝑤⁢ CDM). In the context of the 𝑤⁢ CDM scenario, these statistics also strengthen the constraints on the parameter 𝑤, obtaining 𝑤 <−0.72. The constraints from different statistics are shown to be internally consistent (with a 𝑝-value>0.1 for all combinations of statistics examined). We compare our results to other weak lensing results from the first three years of the Dark Energy Survey data, finding good consistency; we also compare with results from external datasets, such as planck constraints from the cosmic microwave background, finding statistical agreement, with discrepancies no greater than <2.2⁢𝜎.

79 ASTRONOMY AND ASTROPHYSICS↗