Search NASA⌕ Search

SEARCH · Search NASA

Results for “Statistical Testing Approach”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Preventing Failures By Dataset Shift Detection in Safety-Critical Graph Applications

Dataset shift refers to the problem where the input data distribution may change over time (e.g., between training and test stages). Since this can be a critical bottleneck in several safety-critical applications such as healthcare, drug-discovery, etc., dataset shift detection has become an important research issue in machine learning. Though several existing efforts have focused on image/video data, applications with graph-structured data have not received sufficient attention. Therefore, in this paper, we investigate the problem of detecting shifts in graph structured data through the lens of statistical hypothesis testing. Specifically, we propose a practical two-sample test based approach for shift detection in large-scale graph structured data. Our approach is very flexible in that it is suitable for both undirected and directed graphs, and eliminates the need for equal sample sizes. Using empirical studies, we demonstrate the effectiveness of the proposed test in detecting dataset shifts. We also corroborate these findings using real-world datasets, characterized by directed graphs and a large number of nodes.

97 MATHEMATICS AND COMPUTING↗

Modeling the Functional Forms of Grid Disturbances

This report introduces a functional form that may be used to quantitatively predict the impacts of new grid tools and changing system qualities on the likelihoods, durations, and depths of various grid disturbances. Each disturbance scenario is modeled to have three component stages—avoidance, reactance and recovery, which together parametrically estimate one disturbance’s impacts. The modeled scenario is then placed and replicated within an analysis period to represent the likelihood or frequency of the scenario and its consequent impacts. Whereas analysts have struggled to define and apply metrics for grid resilience, the functional form introduced by this report shares units of measurement with accepted grid-status measures (e.g., numbers of customers currently experiencing a service outage). Furthermore, the integrated and averaged functional form over an analysis period provides a meaningful normalized performance metric (e.g., customer outage minutes per year) that is ultimately independent of the duration of the period of. The approach may be applied similarly regardless of the severity or frequency of the disturbances that are being analyzed. Because metrics can be chosen to be identical in both the hypothetical future and the actual historical past, the historical past eventually becomes the test of the future predictions, at least in a statistical sense. The authors originally developed this approach to facilitate analysis of the effects of transactive energy (TE) systems effects on electric power grid resilience. TE systems invite energy suppliers and consumers to actively collaborate toward the discovery of, and their responses to, the locational value of energy. The findings from this process are often embodied as energy prices, the dynamics of which indicate the locational value of energy and can further represent important grid service needs. While some academic papers claim to quantify the value of a specific TE system design toward grid resilience, the answer, in general, has been elusive. Not only do multiple and conflicting definitions of resilience and reliability exist, but countless TE systems are being invented. We conclude the following: (1) The ideal analysis should harmonize rather than differentiate resilience and reliability. Therefore, this report uses the more general term disturbance whenever the overloaded terms resilience and reliability can be avoided. (2) The effectiveness of TE systems must be mapped to underlying qualities of a TE system, thereby avoiding presumptions that every TE design offers similar advantages. The authors seek to evaluate the parametric effects of TE system qualities (e.g., spatial granularity, granularity of time steps, length of future prediction horizon) on avoiding, reacting to, and recovering from grid disturbances. Furthermore, any advantages (or disadvantages) must be fairly compared with the many alternative tools, systems, and strategies that might offer comparable benefits.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR-MS) peak intensity normalization for complex mixture analyses

Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR-MS) is a preferred technique for analyzing complex organic mixtures. Currently, there is no consensus normalization approach, nor an objective method for selecting one, for quantitative analyses of FT-ICR-MS data. We investigate a method to evaluate and score the amount of bias various normalization approaches introduce into the data. We evaluate the ability of the Statistical Procedure for the Analysis of Normalization Strategies (SPANS) to guide selection of appropriate normalization approaches for two different FT-ICR-MS datasets. Further, we test the robustness of SPANS results to changes in SPANS parameter values and assess the impact of using various normalization approaches on downstream statistical analyses. The normalization approach identified by SPANS differed for the two datasets. Normalization methods impacted the statistical significance of peaks differently, underscoring the importance of carefully evaluating potential methods. More consistent SPANS scores resulted when at least 120 significant peaks are used, where larger sets of peaks were obtained by increasing the p-value threshold. Interestingly, we show that Total Sum Scaling and highest peak normalization, used in previous studies, underperformed relative to SPANS-recommended normalization approaches. While there is no single, best normalization method for all datasets, SPANS provides a mechanism to identify an appropriate normalization method for analyzing FT-ICR-MS data quantitatively. As a result, the number of peaks used in the background distributions of SPANS contributes more significantly to the reproducibility of results than the p-value thresholds used to obtain those peaks.

47 OTHER INSTRUMENTATION↗

Multidimensional scaling informed by F -statistic: Visualizing grouped microbiome data with inference

Multidimensional scaling (MDS) is a widely used dimensionality reduction technique in microbial ecology data analysis that captures the multivariate structure of the data while preserving pairwise distances between samples. While improvements in MDS have enhanced the ability to reveal group-specific data patterns, these MDS-based methods require prior assumptions for inference, limiting their application in general microbiome analysis. Here, in this study, we introduce a new MDS-based ordination method, “F-informed MDS,” which configures the data distribution based on the F-statistic, the ratio of dispersion between groups sharing common and different characteristics. Using semisynthetic datasets, we demonstrate that the proposed method is robust to hyperparameter selection while maintaining statistical significance throughout the ordination process. Various quality metrics for evaluating dimensionality reduction confirm that F-informed MDS is comparable to state-of-the-art methods in preserving both local and global data structures. Its application to a diatom-associated bacterial community suggests the role of this new method in interpreting the community’s response to the host. Our approach offers a well-founded refinement of MDS that aligns with statistical test results, which can be beneficial for broader multidimensional data analyses in microbiology and ecology. This new visualization tool can be incorporated into standard microbiome data analyses.

Biological and medical sciences↗

To have value, comparisons of high-throughput phenotyping methods need statistical tests of bias and variance

The gap between genomics and phenomics is narrowing. The rate at which it is narrowing, however, is being slowed by improper statistical comparison of methods. Quantification using Pearson’s correlation coefficient ( r ) is commonly used to assess method quality, but it is an often misleading statistic for this purpose as it is unable to provide information about the relative quality of two methods. Using r can both erroneously discount methods that are inherently more precise and validate methods that are less accurate. These errors occur because of logical flaws inherent in the use of r when comparing methods, not as a problem of limited sample size or the unavoidable possibility of a type I error. A popular alternative to using r is to measure the limits of agreement (LOA). However both r and LOA fail to identify which instrument is more or less variable than the other and can lead to incorrect conclusions about method quality. An alternative approach, comparing variances of methods, requires repeated measurements of the same subject, but avoids incorrect conclusions. Variance comparison is arguably the most important component of method validation and, thus, when repeated measurements are possible, variance comparison provides considerable value to these studies. Statistical tests to compare variances presented here are well established, easy to interpret and ubiquitously available. The widespread use of r has potentially led to numerous incorrect conclusions about method quality, hampering development, and the approach described here would be useful to advance high throughput phenotyping methods but can also extend into any branch of science. The adoption of the statistical techniques outlined in this paper will help speed the adoption of new high throughput phenotyping techniques by indicating when one should reject a new method, outright replace an old method or conditionally use a new method.

59 BASIC BIOLOGICAL SCIENCES↗

Reconstruction of a direction-dependent primordial power spectrum from Planck CMB data

We consider the possibility that the primordial curvature perturbation is direction-dependent. To first order this is parameterised by a quadrupolar modulation of the power spectrum and results in statistical anisotropy of the CMB, which can be quantified using `bipolar spherical harmonics'. We compute these for the Planck DR2-2015 SMICA map and estimate the noise covariance from Planck Full Focal Plane 9 simulations. A constant quadrupolar modulation is detected with 2.2$σ$ significance, dropping to 2$σ$ when the primordial power is assumed to scale with wave number k as a power law. Going beyond previous work we now allow the spectrum to have arbitrary scale-dependence. Our non-parametric reconstruction then suggests several spectral features, the most prominent at k ~ 0.006 Mpc -1 . When a constant quadrupolar modulation is fitted to data in the range 0.005 ≤ k/Mpc -1 ≤ 0.008, its preferred directions are found to be related to the cosmic hemispherical asymmetry and the CMB dipole. To determine the significance we apply two test statistics to our reconstructions of the quadrupolar modulation from data, against reconstructions of realisations of noise only. With a test statistic sensitive only to the amplitude of the modulation, the reconstructions from the multipole range 30 ≤ ℓ ≤ 1200 are unusual with 2.1$σ$ significance. With the second test statistic, sensitive also to the direction, the significance rises to 6.9$σ$. Our approach is easily generalised to include other data sets such as polarisation, large-scale structure and forthcoming 21-cm line observations which will enable these anomalies to be investigated further.

79 ASTRONOMY AND ASTROPHYSICS↗

Life Prediction of Large Lithium-Ion Battery Packs with Active and Passive Balancing

Lithium-ion battery packs take a major part of large-scale stationary energy storage systems. One challenge in reducing battery pack cost is to reduce pack size without compromising pack service performance and lifespan. Prognostic life model can be a powerful tool to handle the state of health (SOH) estimate and enable active life balancing strategy to reduce cell imbalance and extend pack life. This work proposed a life model using both empirical and physical-based approaches. The life model described the compounding effect of different degradations on the entire cell with an empirical model. Then its lower-level submodels considered the complex physical links between testing statistics (state of charge level, C-rate level, duty cycles, etc.) and the degradation reaction rates with respect to specific aging mechanisms. The hybrid approach made the life model generic, robust and stable regardless of battery chemistry and application usage. The model was validated with a custom pack with both passive and active balancing systems implemented, which created four different aging paths in the pack. The life model successfully captured the aging trajectories of all four paths. The life model prediction errors on capacity fade and resistance growth were within +/-3% and +/-5% of the experiment measurements.

25 ENERGY STORAGE↗

Bayesian stability and force modeling for uncertain machining processes

Accurately simulating machining operations requires knowledge of the cutting force model and system frequency response. However, this data is collected using specialized instruments in an ex-situ manner. Bayesian statistical methods instead learn the system parameters using cutting test data, but to date, these approaches have only considered milling stability. This paper presents a physics-based Bayesian framework which incorporates both spindle power and milling stability. Initial probabilistic descriptions of the system parameters are propagated through a set of physics functions to form probabilistic predictions about the milling process. The system parameters are then updated using automatically selected cutting tests to reduce parameter uncertainty and identify more productive cutting conditions, where spindle power measurements are used to learn the cutting force model. The framework is demonstrated through both numerical and experimental case studies. Results show that the approach accurately identifies both the system natural frequency and cutting force model.

42 ENGINEERING↗

Detecting differential protein abundance by combining peptide level P -values

The majority of methods for detecting differentially abundant proteins between samples in label-free LC-MS bottom-up proteomics experiments rely on statistically testing inferred protein abundances derived from peptide ionization intensities or averaging peptide level statistics. Here in this paper, we statistically test peptide ionization intensities directly and combine the resulting dependent P-values using the Empirical Brown's Method (EBM), avoiding error introduced through the estimation of protein abundances or summarizing test statistics. We show that on a spike-in proteomics dataset, a peptide level approach using EBM outperforms differential abundance detection using a protein level approach and several analysis workflows, including MSstats. Additionally, we demonstrate the effectiveness of this approach by detecting enriched proteins from an activity-based protein profiling dataset.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Impact of Climate Variabilities and Human Activities on Surface Water Extents in Reservoirs of Yongding River Basin, China, from 1985 to 2016 Based on Landsat Observations and Time Series Analysis

Yongding River is the largest river flowing through Beijing, the capital city of China. In recent years, Yongding River Basin (YDRB) has seen increasing human impacts on water resources, posing serious challenges in hydrological and ecological health. In this study, remote sensing techniques and statistical time series approaches for hydrological studies were combined to characterize the dynamics and driving factors of reservoir water extents in YDRB during 1985–2016. First, 107 Landsat 4, 5, 7 and 8 images were used to extract surface water extents in YDRB during 1985–2016 using a combination of water indices and Otsu threshold algorithm. Significant positive correlation was found between water extents and the annual inflow for the two biggest reservoirs, the downstream Guanting and upstream Cetian reservoirs, proving their representativeness of surface water availability in this basin. Then, statistical time series approaches including trend-free pre-whitening Mann-Kendall trend test, Pettit change-point test and double mass curve method, which are frequently used in hydrological studies, were adopted to quantify the trend of reservoir water extents dynamics and the relative contributions of climate variability and human activities. Results showed that the water extents in both reservoirs exhibited significant downward trend with change point occurring in 2001 and 2005 for Guanting and Cetian, respectively. About 74%~75% of the shrinkage during the post-change period can be attributed to human activities, among which GDP, population, electricity power production, raw coal production, steel and crude iron production, value of agriculture output, and urban area were the major human drivers. Hydrological connectivity between the upstream Cetian and downstream Guanting reservoirs declined during the post-change period. Since 2012, water extents in both reservoirs recovered as a result of various governmental water management policies including the South-to-North Water Diversion Project. The methodology introduced in this study can be used for analyzing the dynamics and driving mechanism of surface water resources, especially for un-gauged or poorly-gauged watersheds.

47 OTHER INSTRUMENTATION↗

Conservative Estimation of Tail Probabilities from Limited Sample Data

Several sparse-sample uncertainty quantification (UQ) methods are compared for conservative but not overly conservative estimation of small tail probabilities involving responses that lay beyond specified thresholds in the tails of probability distributions. Sixteen very differently shaped distributions (or probability density functions, PDFs) and tail probability magnitudes ranging from 10 -5 to 10 -1 are considered in order for the study to be relevant to a wide range of risk analysis and quantification of margins and uncertainty (QMU) problems. The emphasis of the study is on limited data regimes ranging from N = 2 to 20 samples, reflective of most experimental and some expensive computational situations. Relatively simple sparse-sample UQ methods tested for this regime involve statistical tolerance interval "Equivalent Normal and related "Ensemble of Normals" and "Superdistribution (SD) approaches. (The independently derived SD is effectively equivalent to the Bayesian posterior predictive distribution given the assumptions of the derivation.) The performance of the methods was generally improved for N ≥ 5 samples with a generalized Jackknife resampling technique, which determines a tail probability estimate by averaging estimates from smaller sub-samples. Several quantitative metrics for method conservatism and accuracy of tail probability estimation are used to assess and rank the methods' performance over many random trials for each test PDF and probability magnitude. A variant of Bootstrap resampling was also tried, but did not significantly improve tail probability estimates in most cases. Detailed results are presented from over 100-million tests over the above factors that provide useful granular information on which methods or combination of methods perform best in various areas of the factor space.

97 MATHEMATICS AND COMPUTING↗

Atacama Cosmology Telescope: DR6 gravitational lensing and SDSS BOSS cross-correlation measurement and constraints on gravity with the 𝐸 𝐺 statistic

We derive new constraints on the 𝐸 𝐺 statistic as a test of gravity, combining the cosmic microwave background (CMB) lensing map estimated from Data Release 6 (DR6) of the Atacama Cosmology Telescope with Sloan Digital Sky Survey III Baryon Oscillation Spectroscopic Survey (SDSS BOSS) CMASS and LOWZ galaxy data. We develop an analysis pipeline to measure the cross-correlation between CMB lensing maps and galaxy data, following a blinding policy and testing the approach through null and consistency checks. By testing the equivalence of the spatial and temporal gravitational potentials, the 𝐸 𝐺 statistic can distinguish Λ⁢ CDM from alternative models of gravity. We find 𝐸 𝐺 ⁡(𝑧 eff = 0.555) = 0.3⁢1$^{+0.06}_{−0.05}$ for Atacama Cosmology Telescope (ACT) and CMASS data at 68.28% confidence level, and 𝐸 𝐺 ⁡(𝑧 eff = 0.316) = 0.4⁢9$^{+0.14}_{−0.11}$ for the ACT and LOWZ. Systematic errors are estimated to be 3% and 4%, respectively. Including CMB lensing information from Planck PR4 results in 𝐸 𝐺 ⁡(𝑧 eff = 0.555) = 0.3⁢4$^{+0.05}_{−0.05}$ with CMASS and 𝐸 𝐺 ⁡(𝑧 eff = 0.316) = 0.4⁢3$^{+0.11}_{−0.09}$ with LOWZ. These are consistent with predictions for the Λ⁢ CDM model that best fits the Planck CMB anisotropy and SDSS BOSS baryon acoustic oscillations (BAO), where 𝐸$^{GR}_{𝐺⁡}$(𝑧 eff =0.555) =0.401 ± 0.005 for CMB lensing combined with CMASS and 𝐸$^{GR}_{𝐺}$⁡(𝑧 eff = 0.316) = 0.452 ± 0.005 combined with LOWZ. We also find 𝐸 𝐺 to be scale independent, with probability to exceed >5%, as predicted by general relativity. The methods developed in this work are also applicable to improved future analyses with upcoming spectroscopic galaxy samples and CMB lensing measurements.

79 ASTRONOMY AND ASTROPHYSICS↗

Grid-based minimization at scale: Feldman-Cousins corrections for light sterile neutrino search

High Energy Physics (HEP) experiments generally employ sophisticated statistical methods to present results in searches of new physics. In the problem of searching for sterile neutrinos, likelihood ratio tests are applied to short-baseline neutrino oscillation experiments to construct confidence intervals for the parameters of interest. The test statistics of the form Δχ2 is often used to form the confidence intervals, however, this approach can lead to statistical inaccuracies due to the small signal rate in the region-of-interest. In this paper, we present a computational model for the computationally expensive Feldman-Cousins corrections to construct a statistically accurate confidence interval for neutrino oscillation analysis. The program performs a grid-based minimization over oscillation parameters and is written in C++. Our algorithms make use of vectorization through Eigen3, yielding a single-core speed-up of 350 compared to the original implementation, and achieve MPI data parallelism by employing DIY. We demonstrate the strong scaling of the application at High-Performance Computing (HPC) sites. We utilize HDF5 along with HighFive to write the results of the calculation to file.

Wospakrik, Marianette↗

Correlation approach for quality assurance of additive manufactured parts based on optical metrology

Surface topography and surface finish are two significant factors for evaluating the quality of products in additive manufacturing (AM). AM parts are fabricated layer by layer, which is quite different from traditional formative or subtractive methods. Despite rapid progress in additive manufacturing and associated optical metrology for quality control and in-situ monitoring, limited research has been conducted to investigate the reliability of 3D surface measurement data. The surface topologies scanned by multiple optical systems demonstrated significant differences due to varying sampling mechanisms, resolutions, system noises, etc. The 3D datasets should be trustworthy in order to extract parameters for quality assurance or feedback control from 3D surface measurements. In this paper, we set up new standards to evaluate the reliability of 3D surface measurement data and analyze the variation in the topographical profile. In this study, two non-contact optical methods based on Focus Variation Microscopy (FVM) and Structured Light System (SLS) were adopted to measure the surface topography of the target components. The two optical metrology systems generated two entirely different point cloud datasets. Statistical methods were applied to test the difference between the data obtained from the two systems. By using a data analytics approach for comparison, it was found that the surface roughness estimated from the point cloud data sets of FVM and SLS has no significant difference, though the point cloud data sets were completely different. This paper provides standard validation approach to evaluate the plausibility of metrology data from in-situ real-time surface analysis for process planning of AM.

36 MATERIALS SCIENCE↗

Toward In Situ Synchrotron Mapping of Crystal Selection Processes during Crystal Growth

In this work, we present an automated and rapid method for non-destructive mapping of crystal grains in a rod-shaped sample. The approach was designed for application to in situ float-zone crystal growth experiments at an x-ray synchrotron source, but could be useful in other applications. The methods have been tested on a TiO 2 boule grown in an optical float zone furnace. The approach applies a statistical filter to polycrystalline diffraction patterns on 2D detectors to rapidly determine the degree of powder-quality 1of the signal. When larger crystals emerge in the growth, their position, size and shape can be tracked using an automated blob-tracking algorithm that follows individual Bragg peaks as a function of position in a grid-scan, even when multiple crystals are contributing spots to diffraction images. This method is found to be robust as the same crystal shape can be independently reconstructed using different sets of Bragg reflections. Image segmentation methods are then used to map out the polycrystalline grains. We also note that other information about crystal quality, such as mosaicity or strain state, may be inferred and mapped from the intensity variation of the Bragg peaks at different locations within the sample.

36 MATERIALS SCIENCE↗

Statistical Tests for Convergence in Monte Carlo Criticality Calculations

This report describes methods for using statistical tests to automatically determine convergence of Monte Carlo (MC) iterations in criticality calculations. The methods described herein are intended to replace the traditional approach used for the past 60 years: make a trial run; determine convergence based on plots of k-effective and Shannon entropy vs cycle; adjust the input parameters for controlling the iterations; make a final run to obtain results and statistics.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Short-wavelength-sensitive 2 (Sws2) visual photopigment models combined with atomistic molecular simulations to predict spectral peaks of absorbance

For many species, vision is one of the most important sensory modalities for mediating essential tasks that include navigation, predation and foraging, predator avoidance, and numerous social behaviors. The vertebrate visual process begins when photons of the light interact with rod and cone photoreceptors that are present in the neural retina. Vertebrate visual photopigments are housed within these photoreceptor cells and are sensitive to a wide range of wavelengths that peak within the light spectrum, the latter of which is a function of the type of chromophore used and how it interacts with specific amino acid residues found within the opsin protein sequence. Minor differences in the amino acid sequences of the opsins are known to lead to large differences in the spectral peak of absorbance (i.e. the λ max value). In our prior studies, we developed a new approach that combined homology modeling and molecular dynamics simulations to gather structural information associated with chromophore conformation, then used it to generate statistical models for the accurate prediction of λ max values for photopigments derived from Rh1 and Rh2 amino acid sequences. In the present study, we test our novel approach to predict the λ max of phylogenetically distant Sws2 cone opsins. To build a model that can predict the λ max using our approach presented in our prior studies, we selected a spectrally-diverse set of 11 teleost Sws2 photopigments for which both amino acid sequence information and experimentally measured λ max values are known. The final first-order regression model, consisting of three terms associated with chromophore conformation, was sufficient to predict the λ max of Sws2 photopigments with high accuracy. This study further highlights the breadth of our approach in reliably predicting λ max values of Sws2 cone photopigments, evolutionary-more distant from template bovine RH1, and provided mechanistic insights into the role of known spectral tuning sites.

59 BASIC BIOLOGICAL SCIENCES↗

Periodicity significance testing with null-signal templates: reassessment of PTF’s SMBH binary candidates

Periodograms are widely employed for identifying periodicity in time series data, yet they often struggle to accurately quantify the statistical significance of detected periodic signals when the data complexity precludes reliable simulations. We develop a data-driven approach to address this challenge by introducing a null-signal template (NST). The NST is created by carefully randomizing the period of each cycle in the periodogram template, rendering it non-periodic. It has the same frequentist properties as a periodic signal template, and we show with simulations that the distribution of false positives is the same as with the original periodic template, regardless of the underlying data. Thus, performing a periodicity search with the NST acts as an effective simulation of the null (no-signal) hypothesis, without having to simulate the noise properties of the data. We apply the NST method to the supermassive black hole binaries (SMBHB) search in the Palomar Transient Factory (PTF), where Charisi et al. had previously proposed 33 high signal-to-noise candidates utilizing simulations to quantify their significance. Our approach reveals that these simulations do not capture the complexity of the real data. There are no statistically significant periodic signal detections above the non-periodic background. To improve the search sensitivity, we introduce a Gaussian quadrature based algorithm for the Bayes Factor with correlated noise as a test statistic. We show with simulations that this improves sensitivity to true signals by more than an order of magnitude. However, the Bayes Factor approach also results in no statistically significant detections in the PTF data.

79 ASTRONOMY AND ASTROPHYSICS↗