Search NASA⌕ Search

SEARCH · Search NASA

Results for “Summary statistics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Cluster characterization in atom probe tomography: Machine learning using multiple summary functions

In this work, we develop a machine learning-based method to characterize intracluster concentration (ρ c ), background concentration (ρ b ), clustering radius (r̄), and radius dispersity (δ r ) in simulated atom probe tomography data using multiple spatial statistics summary functions to train a Bayesian regularized neural network. Here, we build upon previous work that utilized Ripley’s K-function by incorporating additional features from nearest-neighbor spatial statistics summary functions to better characterize concentration-based metrics. The addition of nearest-neighbor based features allows for highly accurate estimates of ρ c and ρ b , both with 90% of the predictions within 4.0% of the real value; the root-mean-square errors are reduced by 81.5% and 92.8% from predictions using only K-function based features, respectively. Additionally, including these nearest-neighbor based features improves the ability to differentiate between r̄ and δ r .

36 MATERIALS SCIENCE↗
DOE OSTI2023

A forward modeling approach to analyzing galaxy clustering with S IM BIG

We present cosmological constraints from a simulation-based inference (SBI) analysis of galaxy clustering from the SimBIG forward modeling framework. SimBIG leverages the predictive power of high-fidelity simulations and provides an inference framework that can extract cosmological information on small nonlinear scales. In this work, we apply SimBIG to the Baryon Oscillation Spectroscopic Survey (BOSS) CMASS galaxy sample and analyze the power spectrum, P ℓ (k), to k max = 0.5 h/Mpc. We construct 20,000 simulated galaxy samples using our forward model, which is based on 2,000 high-resolution Quijote N -body simulations and includes detailed survey realism for a more complete treatment of observational systematics. We then conduct SBI by training normalizing flows using the simulated samples and infer the posterior distribution of ΛCDM cosmological parameters: Ω m , Ω b , h, n s , σ 8 . We derive significant constraints on Ω m and σ 8 , which are consistent with previous works. Our constraint on σ 8 is 27% more precise than standard P ℓ analyses because we exploit additional cosmological information on nonlinear scales beyond the limit of current analytic models, k > 0.25 h/Mpc. This improvement is equivalent to the statistical gain expected from a standard P ℓ analysis of galaxy sample ~ 60% larger than CMASS. While we focus on P ℓ in this work for validation and comparison to the literature, SimBIG provides a framework for analyzing galaxy clustering using any summary statistic. We expect further improvements on cosmological constraints from subsequent SimBIG analyses of summary statistics beyond P ℓ .

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Dark Energy Survey Year 3 results: optimized $w$CDM simulation-based inference with weak lensing map-level hybrid statistics

We present cosmological constraints from the Dark Energy Survey Year 3 (DES Y3) weak lensing data using hierarchical hybrid statistics within a Bayesian simulation-based inference framework that is based on the Gower Street simulations. To maximize the precision of the inference, we have developed a new, information-theory based, data compression of the weak lensing maps to just seven highly informative summary statistics. The hybrid scheme exploits the high information content of the power spectrum, compressing both the power spectrum and neural-based summaries that are designed to extract further information. Our simulation-based approach enables principled forward modelling of all major sources of systematic uncertainty and survey properties into realistic mock observations, including the survey mask, photometric redshift uncertainties, intrinsic galaxy alignments, multiplicative shear calibration bias, source galaxy clustering, non-Gaussian shape noise, and non-linear structure formation. The summary statistics are then used in a Bayesian simulation-based inference pipeline. The inference is validated through coverage tests and checks for robustness against baryonic feedback. Assuming a $w$CDM cosmology, our analysis yields $S_8 = 0.808 \pm 0.017$, $Ω_{\rm m} = 0.325 \pm 0.024$, and $w < -0.766$ (marginalized posterior 68 per cent credible intervals). This rigorous combination of information theory, physics- and neural network-based extreme data compression, and principled Bayesian analysis improves the figure of merit for $(Ω_{\rm m}, S_8, w)$ by 60 per cent over the previous state-of-the-art, and by almost a factor of 3 over two-point analyses of the same data. They are the most precise joint constraints on $(Ω_{\rm m}, S_8, w)$ from weak gravitational lensing data alone of any survey to date. We intend to apply this analysis to the more recent DES Y6 data.

Williamson, J. [University Coll. London]↗
NASA NTRS2011

Use of Minute-by-Minute Cardiovascular Measurements During Tilt Tests to Strengthen Inference on the Effect of Long-Duration Space Flight on Orthostatic Hypotension

Typical methodology for evaluating the effects of spaceflight on orthostatic hypotension (OH) has been survival analysis of tolerance times from 80 head-up tilt tests. However when scheduled test durations are short, there may not be enough failures to allow survival analysis to adequately estimate and compare the effects of flight phase (e.g. pre-flight, number of days post-flight), flight duration, and their interaction, as well as interactions with effects of interventions or countermeasures. The problem is exacerbated in the presence of a repeated measures design, in which subjects participate in tilt tests during various flight phases. Here we show how it is possible to dramatically improve the efficiency of statistical inference in this setting by making use of the additional information contained in minute-by-minute observations of cardiovascular parameters thought to be reflective of progression towards presyncope during tilt testing. Methods: We retrospectively examined operational tilt test (OTT; 10 -min 80 head-up tilt) data from 20 International Space Station (ISS) and 66 Shuttle astronauts 10 d before launch (L-10), on landing day (R+0) and during recovery (R+1, R+3, R+6-10) depending on the level of participation. Data from 5 ISS astronauts tested on R+0 or R+1 who used non-standard countermeasures were excluded. In addition to OTT survival time, 8 cardiovascular parameters (CP: heart rate, systolic, diastolic, and mean arterial blood pressure, pulse pressure, stroke volume, cardiac output, and total peripheral resistance) that might be predictive of progression towards presyncope were measured every minute of each OTT. Statistical analysis was predicated on a two ]stage model of causation. In the first stage, flight duration and time from landing affect the astronauts' degree of OH, which is manifested in the time trends and variation of the above CPs during OTTs. In the second stage, the behavior of these parameters directly affects OTT survival time. Actual analysis proceeded in the opposite direction. First we identified those CPs or linear combinations that best predicted OTT survival regardless of what spaceflight conditions led to OTT completion or presyncope. From these, we calculated a summary statistic (one per OTT) that best predicted survival. We then used mixed ]model regression analysis to relate changes in the summary statistic to flight phase and duration. Inference on the effects of phase, duration, and their interaction on OH follows directly from this second analysis. Results: A linear combination (W) of diastolic blood pressure (DBP) and stroke volume (SV) was found to be the best predictor of OTT survival using the complete data set of minute-by-minute observations of CPs for each OTT. Furthermore, the log-transformed standard deviation of W (Z = log SW) was found to be a strong predictor of survival in the reduced data set consisting of one observation per OTT. In other words, this measure of variability of W during an OTT was the best indicator of whether or not the subject could complete the 10-min test, with higher variability (i.e. higher values of Z) being associated with greater probability of failure. In the mixed-model regression analysis where Z was now treated as a outcome with flight phase and duration groups (ISS and STS) as predictors, we found that there was a significantly more variability in W (higher values of Z) for both groups at R+0, but with no evidence of an interaction until R+3, when the ISS group still had inflated variability, but not the STS group. Conclusions: Variability of the cardiovascular index W recovers more slowly after long-compared to short-duration spaceflight. Since high variability of W has also been shown to be predictive of OTT failure, a primary manifestation of OH, a logical conclusion is that recovery from OH also is slower after long-duration compared to short-duration spaceflights.

Feiveson, Alan H.↗
DOE OSTI2022

KaRMMa – kappa reconstruction for mass mapping

ABSTRACT We present KaRMMa, a novel method for performing mass map reconstruction from weak-lensing surveys. We employ a fully Bayesian approach with a physically motivated lognormal prior to sample from the posterior distribution of convergence maps. We test KaRMMa on a suite of dark matter N-body simulations with simulated DES Y1-like shear observations. We show that KaRMMa outperforms the basic Kaiser–Squires mass map reconstruction in two key ways: (1) our best map point estimate has lower residuals compared to Kaiser–Squires; and (2) unlike the Kaiser–Squires reconstruction, the posterior distribution of KaRMMa maps is nearly unbiased in all summary statistics we considered, namely: one-point and two-point functions, and peak/void counts. In particular, KaRMMa successfully captures the non-Gaussian nature of the distribution of κ values in the simulated maps. We further demonstrate that the KaRMMa posteriors correctly characterize the uncertainty in all summary statistics we considered.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Dark Energy Survey Year 3 results: Simulation-based cosmological inference with wavelet harmonics, scattering transforms, and moments of weak lensing mass maps. Validation on simulations

Beyond-two-point statistics contain additional information on cosmological as well as astrophysical and observational (systematics) parameters. In this methodology paper we provide an end-to-end simulationbased analysis of a set of Gaussian and non-Gaussian weak lensing statistics using detailed mock catalogs of the Dark Energy Survey (DES). Here, we implement: 1) second and third moments; 2) wavelet phase harmonics (WPH); 3) the scattering transform (ST). Our analysis is fully based on simulations, it spans a space of seven $νw$CDM cosmological parameters, and it forward models the most relevant sources of systematics of the data (masks, noise variations, clustering of the sources, intrinsic alignments, and shear and redshift calibration). We implement a neural network compression of the summary statistics, and we estimate the parameter posteriors using a likelihood-free-inference approach. We validate the pipeline extensively, and we find that WPH exhibits the strongest performance when combined with second moments, followed by ST, and then by third moments. The combination of all the different statistics further enhances constraints with respect to second moments, up to 25 percent, 15 percent, and 90 percent for S 8 , Ω m , and the figure-of-merit FoM S8;Ωm , respectively. We further find that non-Gaussian statistics improve constraints on w and on the amplitude of intrinsic alignment with respect to second moments constraints. The methodological advances presented here are suitable for application to Stage IV surveys from Euclid, Rubin-LSST, and Roman with additional validation on mock catalogs for each survey. In a companion paper we present an application to DES Year 3 data.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2007

Bootstrap data methodology for sequential hybrid model building

A method for modeling engine operation comprising the steps of: 1. collecting a first plurality of sensory data, 2. partitioning a flight envelope into a plurality of sub-regions, 3. assigning the first plurality of sensory data into the plurality of sub-regions, 4. generating an empirical model of at least one of the plurality of sub-regions, 5. generating a statistical summary model for at least one of the plurality of sub-regions, 6. collecting an additional plurality of sensory data, 7. partitioning the second plurality of sensory data into the plurality of sub-regions, 8. generating a plurality of pseudo-data using the empirical model, and 9. concatenating the plurality of pseudo-data and the additional plurality of sensory data to generate an updated empirical model and an updated statistical summary model for at least one of the plurality of sub-regions.

Volponi, Allan J.↗
DOE OSTI2024

Hawai‘i Supernova Flows: a peculiar velocity survey using over a Thousand Supernovae in the near-infrared

ABSTRACT We introduce the Hawai‘i Supernova Flows project and present summary statistics of the first 1217 astronomical transients observed, 668 of which are spectroscopically classified Type Ia Supernovae (SNe Ia). Our project is designed to obtain systematics-limited distances to SNe Ia while consuming minimal dedicated observational resources. To date, we have performed almost 5000 near-infrared (NIR) observations of astronomical transients and have obtained spectra for over 200 host galaxies lacking published spectroscopic redshifts. In this survey paper, we describe the methodology used to select targets, collect/reduce data, calculate distances, and perform quality cuts. We compare our methods to those used in similar studies, finding general agreement or mild improvement. Our summary statistics include various parametrizations of dispersion in the Hubble diagrams produced using fits to several commonly used SN Ia models. We find the lowest dispersions using the SNooPy package’s EBV_model2, with a root mean square deviation of 0.165 mag and a normalized median absolute deviation of 0.123 mag. The full utility of the Hawai‘i Supernova Flows data set far exceeds the analyses presented in this paper. Our photometry will provide a valuable test bed for models of SN Ia incorporating NIR data. Differential cosmological studies comparing optical samples and combined optical and NIR samples will have increased leverage for constraining chromatic effects like dust extinction. We invite the community to explore our data by making the light curves, fits, and host galaxy redshifts publicly accessible.

Do, Aaron (ORCID:0000000334297845)↗
DOE OSTI2022

Unsupervised Machine Learning for Exploratory Data Analysis of Exoplanet Transmission Spectra

Abstract Transit spectroscopy is a powerful tool for decoding the chemical compositions of the atmospheres of extrasolar planets. In this paper, we focus on unsupervised techniques for analyzing spectral data from transiting exoplanets. After cleaning and validating the data, we demonstrate methods for: (i) initial exploratory data analysis, based on summary statistics (estimates of location and variability); (ii) exploring and quantifying the existing correlations in the data; (iii) preprocessing and linearly transforming the data to its principal components; (iv) dimensionality reduction and manifold learning; (v) clustering and anomaly detection; and (vi) visualization and interpretation of the data. To illustrate the proposed unsupervised methodology, we use a well-known public benchmark data set of synthetic transit spectra. We show that there is a high degree of correlation in the spectral data, which calls for appropriate low-dimensional representations. We explore a number of different techniques for such dimensionality reduction and identify several suitable options in terms of summary statistics, principal components, etc. We uncover interesting structures in the principal component basis, namely well-defined branches corresponding to different chemical regimes of the underlying atmospheres. We demonstrate that those branches can be successfully recovered with a K-means clustering algorithm in a fully unsupervised fashion. We advocate for lower-dimensional representations of the spectroscopic data in terms of the main principal components, in order to reveal the existing structure in the data and quickly characterize the chemical class of a planet.

Matchev, Konstantin T. (ORCID:0000000341829096)↗
NASA NTRS1980

Image-analysis library

MATHPAC image-analysis library is collection of general-purpose mathematical and statistical routines and special-purpose data-analysis and pattern-recognition routines for image analysis. MATHPAC library consists of Linear Algebra, Optimization, Statistical-Summary, Densities and Distribution, Regression, and Statistical-Test packages.

Source record↗
DOE OSTI2022

Differentially Private K -Means Clustering Applied to Meter Data Analysis and Synthesis

The proliferation of smart meters has resulted in a large amount of data being generated. It is increasingly apparent that methods are required for allowing a variety of stakeholders to leverage the data in a manner that preserves the privacy of the consumers. The sector is scrambling to define policies, such as the so called ‘15/15 rule’, to respond to the need. However, the current policies fail to adequately guarantee privacy. Here, in this paper, we address the problem of allowing third parties to apply K-means clustering, obtaining customer labels and centroids for a set of load time series by applying the framework of differential privacy. We leverage the method to design an algorithm that generates differentially private synthetic load data consistent with the labeled data. We test our algorithm’s utility by answering summary statistics such as average daily load profiles for a 2-dimensional synthetic dataset and a real-world power load dataset.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Cosmology from weak lensing peaks and minima with Subaru Hyper Suprime-Cam Survey first-year data

We present cosmological constraints derived from peak counts, minimum counts, and the angular power spectrum of the Subaru Hyper Suprime-Cam first-year (HSC Y1) weak lensing shear catalogue. Weak lensing peak and minimum counts contain non-Gaussian information and hence are complementary to the conventional two-point statistics in constraining cosmology. In this work, we forward-model the three summary statistics and their dependence on cosmology, using a suite of N-body simulations tailored to the HSC Y1 data. We investigate systematic and astrophysical effects including intrinsic alignments, baryon feedback, multiplicative bias, and photometric redshift uncertainties. We mitigate the impact of these systematics by applying cuts on angular scales, smoothing scales, signal-to-noise ratio bins, and tomographic redshift bins. By combining peaks, minima, and the power spectrum, assuming a flat-ΛCDM model, we obtain S 8 ≡ σ 8 $\sqrt{Ω_m/0.3}$ = 0.810$^{+0.022}_{-0.026}$⁠, a 35 per cent tighter constraint than that obtained from the angular power spectrum alone. Our results are in agreement with other studies using HSC weak lensing shear data, as well as with Planck 2018 cosmology and recent CMB lensing constraints from the Atacama Cosmology Telescope and the South Pole Telescope.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Quality Assurance and Quality Control (QA/QC) of Meteorological Time Series Data for Billy Barr, East River, Colorado USA

A comprehensive Quality Assurance (QA) and Quality Control (QC) statistical framework consists of three major phases: Phase 1—Preliminary raw data sets exploration, including time formatting and combining datasets of different lengths and different time intervals; Phase 2—QA of the datasets, including detecting and flagging of duplicates, outliers, and extreme values; and Phase 3—the development of time series of a desired frequency, imputation of missing values, visualization and a final statistical summary. The time series data collected at the Billy Barr meteorological station (East River Watershed, Colorado) were analyzed. The developed statistical framework is suitable for both real-time and post-data-collection QA/QC analysis of meteorological datasets.The files that are in this data package include one excel file, converted to CSV format (Billy_Barr_raw_qaqc.csv) that contains the raw meteorological data, i.e., input data used for the QA/QC analysis. The second CSV file (Billy_Barr_1hr.csv) is the QA/QC and flagged meteorological data, i.e., output data from the QA/QC analysis. The last file (QAQC_Billy_Barr_2021-03-22.R) is a script written in R that implements the QA/QC and flagging process. The purpose of the CSV data files included in this package is to provide input and output files implemented in the R script.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1984

Studies of lightning data in conjunction with geostationary satellite data

The archiving of a more complete data base to perform lightning studies is outlined. This effort includes: (1) continued archiving of bureau of land management LLP, geostationary satellite, and NWS radar data; (2) expansion of the McIDAS real-time LLP access to other networks. Additional processing tools for display and analysis of lightning location data in conjunction with geostationary satellite data are developed which entails adapting existing McIDAS software to allow the production of statistical summaries and contouring of lightning characteristics over user defined areas or storms. The plotting of three dimensional displays of lightning statistics versus satellite and radar data, and the performance of an error analysis of lightning location data using overlapping regions of the BLM network are discussed.

Suchman, D.↗
NASA NTRSDate not supplied

A NASA GISTEMP Observational Uncertainty Ensemble: Regional and Monthly Uncertainty

The historical global temperature record is an essential data product for quantifying the variability and change of the Earth system. In recent years, better characterization of observational uncertainty in global and hemispheric trends has become available, but the methodologies are not necessarily applicable to analyses at smaller regional areas, or monthly means, where station sparsity and other systematic issues contribute to greater uncertainty. This work details a gridded uncertainty ensemble of historical temperature anomalies from the Goddard Institute for Space Studies (GISS) Surface Temperature product (GISTEMP) product. This ensemble characterizes the complex spatial and temporal correlation structure of uncertainty in gridded historical temperature, enabling proper uncertainty propagation for climate and social science at regional and monthly scales. This work details the methodology for generating the uncertainty ensemble, key statistics of the uncertainty evolution over space and time, and provides best practices for using the uncertainty ensemble in future studies. Summary statistics from the uncertainty ensemble are in good agreement with production GISTEMP. Two applications of the uncertainty ensemble are also presented. First, the warmest year on record is shown to most likely be 2016 with a 53.2% chance and 2020 as the second most likely with a 44.4% chance. Second, it is shown that the arctic is warming 2.5 - 5 times faster than the globe, significantly faster than the regularly quoted twice as fast.

GISTEMP↗
DOE OSTI2024

Predicting U.S. federal fleet electric vehicle charging patterns using internal combustion engine vehicle fueling transaction statistics

Utilizing fueling transactions from internal combustion engine vehicles (ICEVs), the authors estimated how frequently midday public charging would be required for U.S. federal fleet battery electric vehicles (BEVs). Fueling transaction summary statistics are more widely available than trip-level telematics data, making this methodology more accessible and transferable to other researchers and fleet managers considering BEV replacements. For example, readers can easily apply a linear model using only the count of back-to-back fueling events at gas stations over 57 straight-line miles apart to predict days exceeding range. This linear regression predicted binned days exceeding 250 miles at 80% accuracy on a hold-out test set from the same fleet as the training data and 66 % accuracy on a new fleet displaying different driving behaviors. The authors additionally provide linear equations for days exceeding 200 and 300 miles as alternative range estimates to account for differences in BEV range and temperature impacts. Beyond the single-feature linear models which readers can apply, the authors tuned and trained other machine learning models on a variety of fueling transaction statistics including consecutive transaction distances, transaction distance from garage, estimated miles traveled from fuel economy and fuel quantity, and transaction periodicity. Utilizing a subset of 1678 light-duty federal fleet vehicles which contained daily vehicle miles traveled (VMT) in addition to fueling statistics, the authors determined which fueling transaction statistics were most relevant in predicting driving days exceeding 250 miles (an approximation of BEV rated driving range). In support of the U.S. federal fleet transition to zero-emission vehicles (ZEVs), the authors used these statistics and machine learning models to predict the frequency of BEV midday charging. After training models on the subset with VMT, the authors predicted days exceeding rated range for 112,902 light-duty vehicles operating in similar circumstances in the federal fleet using a Support Vector Regressor (SVR). In conclusion, they then used the projections as part of the ZEV Planning and Charging (ZPAC) tool to identify optimal candidates for BEVs for the federal fleet. An anonymized version of ZPAC is included in the supplementary materials.

25 ENERGY STORAGE↗
DOE OSTI2022

The DESI N -body Simulation Project – II. Suppressing sample variance with fast simulations

ABSTRACT Dark Energy Spectroscopic Instrument (DESI) will construct a large and precise three-dimensional map of our Universe. The survey effective volume reaches $\sim 20\, h^{-3}\, \mathrm{Gpc}^{3}$. It is a great challenge to prepare high-resolution simulations with a much larger volume for validating the DESI analysis pipelines. AbacusSummit is a suite of high-resolution dark-matter-only simulations designed for this purpose, with $200\, h^{-3}\, \mathrm{Gpc}^{3}$ (10 times DESI volume) for the base cosmology. However, further efforts need to be done to provide a more precise analysis of the data and to cover also other cosmologies. Recently, the CARPool method was proposed to use paired accurate and approximate simulations to achieve high statistical precision with a limited number of high-resolution simulations. Relying on this technique, we propose to use fast quasi-N-body solvers combined with accurate simulations to produce accurate summary statistics. This enables us to obtain 100 times smaller variance than the expected DESI statistical variance at the scales we are interested in, e.g. $k \lt 0.3\, h\, \mathrm{Mpc}^{-1}$ for the halo power spectrum. In addition, it can significantly suppress the sample variance of the halo bispectrum. We further generalize the method for other cosmologies with only one realization in AbacusSummit suite to extend the effective volume ∼20 times. In summary, our proposed strategy of combining high-fidelity simulations with fast approximate gravity solvers and a series of variance suppression techniques sets the path for a robust cosmological analysis of galaxy survey data.

cosmology: theory↗