Search NASA⌕ Search

SEARCH · Search NASA

Results for “two-sample testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

14 records
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Exercising Prediction Validation Methods for Analysis of Quiet Supersonic Overflight

En route noise certification procedures for quiet supersonic aircraft are in development under the International Civil Aviation Organization for a proposed standard that could enable civil supersonic overland flight. While these quiet supersonic aircraft do not exist yet, methods are being developed and evaluated using existing N-wave empirical datasets and low-boom simulation datasets. One potential aspect of future certification methods is the ability to use a validated prediction model to predict the certification noise level under reference day weather conditions. Methods to validate predictions, such as the two-sample t-test, Fisher’s combined probability test, two-sample Kolmogorov-Smirnov test, and equivalence testing, have been exercised and modified for this application. While some methods compare mean values, others compare values across the distribution, and a combination of methods can be used. The methods were applied to a subset of an N-wave dataset to demonstrate the applicability and identify potential areas for improvement. Atmospheric turbulence variability was included as a factor in these analyses through KZKFourier propagation simulations with parameters matching actual flights and weather conditions. Results are presented for six noise metrics (PL, ASEL, BSEL, DSEL, ESEL, and ISBAP), and plans to expand the analyses are discussed.

sonic boom↗
DOE OSTI2021

Preventing Failures By Dataset Shift Detection in Safety-Critical Graph Applications

Dataset shift refers to the problem where the input data distribution may change over time (e.g., between training and test stages). Since this can be a critical bottleneck in several safety-critical applications such as healthcare, drug-discovery, etc., dataset shift detection has become an important research issue in machine learning. Though several existing efforts have focused on image/video data, applications with graph-structured data have not received sufficient attention. Therefore, in this paper, we investigate the problem of detecting shifts in graph structured data through the lens of statistical hypothesis testing. Specifically, we propose a practical two-sample test based approach for shift detection in large-scale graph structured data. Our approach is very flexible in that it is suitable for both undirected and directed graphs, and eliminates the need for equal sample sizes. Using empirical studies, we demonstrate the effectiveness of the proposed test in detecting dataset shifts. We also corroborate these findings using real-world datasets, characterized by directed graphs and a large number of nodes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Topological Signatures of Adversaries in Multimodal Alignments

Topological Data Analysis for Adversarial Detection (LANL O4937) - Detects adversarial examples in vision-language models using persistent homology and two-sample testing. Combines TDA features from CLIP embeddings with statistical methods (ME, SCF, SAMMD, C2ST) for robust detection across ImageNet, CIFAR-10/100.

Bhattarai, Manish↗
NASA NTRS1985

Statistical methods for astronomical data with upper limits. I - Univariate distributions

The statistical treatment of univariate censored data is discussed. A heuristic derivation of the Kaplan-Meier maximum-likelihood estimator from first principles is presented which results in an expression amenable to analytic error analysis. Methods for comparing two or more censored samples are given along with simple computational examples, stressing the fact that most astronomical problems involve upper limits while the standard mathematical methods require lower limits. The application of univariate survival analysis to six data sets in the recent astrophysical literature is described, and various aspects of the use of survival analysis in astronomy, such as the limitations of various two-sample tests and the role of parametric modelling, are discussed.

Feigelson, E. D.↗
NASA NTRS1994

Searching for hard X-ray directivity during the rise, peak, and decay phases of solar flares

We have identified 72 large solar flares (peak counting rates more than 1000 counts/s) observed by Hard X-ray Burst Spectroscopy (HXRBS) on-board the Solar Maximum Mission (SMM). Using a database of these flares, we have studied hard X-ray (50-850 keV) spectral center-to-limb variation and its evolution with time. The major results are the following: (1) During the rise phase, the center-to-limb spectral variation is small, with a hardness of delta delta = 0.02 +/- 0.25, and a statistical significance of 0.1 sigma. (2) During the peak phase, the center-to-limb variation is delta delta = 0.13 +/- 0.13, with a statistical significance of 1 sigma. (3) During the decay phase, the center-to-limb variation changes to softening. The softness is relatively large with delta delta = -0.25 +/- 0.21, and a statistical significance of 1.2 sigma. (4) The linear least-squares fits to the spectral center-to-limb variations do not have slopes significantly different from zero during all those three phases. (5) The center events and limb events spectral distributions are shown to be not different by using Kolmogorov-Smirnov two-samples test. (6) The fraction of events detected near the limb is marginally consistent with that expected from isotropically emitting flares. (7) On average, flares evolve as soft-hard-soft. These results suggest that there is no statistically significant evidence for hard X-ray directivity during the rise, peak, and decay phases of solar flares. The hard X-ray radiation pattern at those energies is almost isotropic during all those phases. This lack of directivity (or anisotropy) found in this study is not in agreement with the results discovered by Vestrand et al. (1987) in which they found energetic photon source is anisotropic, using SMM Gamma-Ray Spectrometer (GRS) data at a much higher energy band of 0.3-1 MeV. If we want to interpret the results of Vestrand et al. (1987) and our present results in a self-consistent way, we must conclude that at energies of greater than 50 keV, the Compton backscattering suppresses the hard X-ray anisotropy sufficiently so that no significant directivity can be measured during the rise, peak, and decay phases of solar flares. during the rise, peak, and decay phases of solar flares.

Li, Peng↗
NASA NTRSDate not supplied

Copula-based Method to Generate Consistent Surface Pressures Under Uncertainty

This paper establishes a method to create surface pressure databases that allow for uncertainty quantification. Aerodynamic databases are critical products for launch vehicles and other aeronautical systems, and surface pressure databases are one such database that constains large quantities of data. The focus of this work is the relationship between the integrated force and moment data base and the surface pressure database. In particular, the work attempts to provide a method that maintains consistency between these two databases when accounting for uncertainty. The integrated force and moment database and surface pressure databases are constructed from CFD data which is high-density but low-trust. However, the force and moment database will often also include data from high-trust but low-density sources such as from wind tunnel experiments. This means that the quantified uncertainty of the force and moment database is higher quality as it includes this high fidelity wind tunnel data. This motivates the idea to use the force and moment database uncertainty when constructing the surface pressure database uncertainty. The method utilizes the statistical idea of a copula in order to generate surface pressures that match with uncertain integrated force and moment distributions as well as being consistent with known CFD data. This statistical consistency is quantified by using the Maximum Mean Discrepancy two-sample test. The predictive error of the method is also approximated using leave-one-out error estimation and the good overall performance of the method is presented using probability boxes in a simulated uncertainty scenario.

SLS↗
DOE OSTI2022

Testing and signal identification for two-sample high-dimensional covariances via multi-level thresholding

The paper considers testing and signal identification for covariance matrices from two populations of marginally sub-Gaussian distributed. A multi-level thresholding procedure is proposed for testing the equality of two high-dimensional covariance matrices, which is designed to detect sparse and faint differences between the covariances. A novel -statistic composition is developed to establish the asymptotic distribution of the thresholding statistics in conjunction with the matrix blocking and the coupling techniques. It is shown that the proposed test is more powerful than the existing tests in detecting sparse and weak signals in covariances. Multiple testing procedures are constructed to discover different covariances and the sub-groups of variables with different covariance structures between the two populations. The proposed procedures are based on the multi-level thresholding test, which are able to control the false discovery proportion () with high power. Simulation experiments and a case study on the returns of the S&P 500 stocks before and after the COVID-19 pandemic are conducted to demonstrate and compare the utilities of the proposed methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Automated Identification of Characteristic Droplet Size Distributions in Stratocumulus Clouds Utilizing a Data Clustering Algorithm

Abstract Droplet-level interactions in clouds are often parameterized by a modified gamma fitted to a “global” droplet size distribution. Do “local” droplet size distributions of relevance to microphysical processes look like these average distributions? This paper describes an algorithm to search and classify characteristic size distributions within a cloud. The approach combines hypothesis testing, specifically, the Kolmogorov–Smirnov (KS) test, and a widely used class of machine learning algorithms for identifying clusters of samples with similar properties: density-based spatial clustering of applications with noise (DBSCAN) is used as the specific example for illustration. The two-sample KS test does not presume any specific distribution, is parameter free, and avoids biases from binning. Importantly, the number of clusters is not an input parameter of the DBSCAN-type algorithms but is independently determined in an unsupervised fashion. As implemented, it works on an abstract space from the KS test results, and hence spatial correlation is not required for a cluster. The method is explored using data obtained from the Holographic Detector for Clouds (HOLODEC) deployed during the Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) field campaign. The algorithm identifies evidence of the existence of clusters of nearly identical local size distributions. It is found that cloud segments have as few as one and as many as seven characteristic size distributions. To validate the algorithm’s robustness, it is tested on a synthetic dataset and successfully identifies the predefined distributions at plausible noise levels. The algorithm is general and is expected to be useful in other applications, such as remote sensing of cloud and rain properties. Significance Statement A typical cloud can have billions of drops spread over tens or hundreds of kilometers in space. Keeping track of the sizes, positions, and interactions of all of these droplets is impractical, and, as such, information about the relative abundance of large and small drops is typically quantified with a “size distribution.” Droplets in a cloud interact locally, however, so this work is motivated by the question of whether the cloud droplet size distribution is different in different parts of a cloud. A new method, based on hypothesis testing and machine learning, determines how many different size distributions are contained in a given cloud. This is important because the size distribution describes processes such as cloud droplet growth and light transmission through clouds.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2017

Updated MISR Dark Water Research Aerosol Retrieval Algorithm - Part 1: Coupled 1.1 km Ocean Surface Chlorophyll a Retrievals with Empirical Calibration Corrections

As aerosol amount and type are key factors in the 'atmospheric correction' required for remote-sensing chlorophyll alpha concentration (Chl) retrievals, the Multi-angle Imaging SpectroRadiometer (MISR) can contribute to ocean color analysis despite a lack of spectral channels optimized for this application. Conversely, an improved ocean surface constraint should also improve MISR aerosol-type products, especially spectral single-scattering albedo (SSA) retrievals. We introduce a coupled, self-consistent retrieval of Chl together with aerosol over dark water. There are time-varying MISR radiometric calibration errors that significantly affect key spectral reflectance ratios used in the retrievals. Therefore, we also develop and apply new calibration corrections to the MISR top-of-atmosphere (TOA) reflectance data, based on comparisons with coincident MODIS (Moderate Resolution Imaging Spectroradiometer) observations and trend analysis of the MISR TOA bidirectional reflectance factors (BRFs) over three pseudo-invariant desert sites. We run the MISR research retrieval algorithm (RA) with the corrected MISR reflectances to generate MISR-retrieved Chl and compare the MISR Chl values to a set of 49 coincident SeaBASS (SeaWiFS Bio-optical Archive and Storage System) in situ observations. Where Chl(sub in situ) less than 1.5 mg m(exp -3), the results from our Chl model are expected to be of highest quality, due to algorithmic assumption validity. Comparing MISR RA Chl to the 49 coincident SeaBASS observations, we report a correlation coefficient (r) of 0.86, a root-mean-square error (RMSE) of 0.25, and a median absolute error (MAE) of 0.10. Statistically, a two-sample Kolmogorov- Smirnov test indicates that it is not possible to distinguish between MISR Chl and available SeaBASS in situ Chl values (p greater than 0.1). We also compare MODIS-Terra and MISR RA Chl statistically, over much broader regions. With about 1.5 million MISR-MODIS collocations having MODIS Chl less than 1.5 mg m(exp -3), MISR and MODIS show very good agreement: r = 0.96, MAE = 0.09, and RMSE = 0.15. The new dark water aerosol/Chl RA can retrieve Chl in low-Chl, case I waters, independent of other imagers such as MODIS, via a largely physical algorithm, compared to the commonly applied statistical ones. At a minimum, MISR's multi-angle data should help reduce uncertainties in the MODIS-Terra ocean color retrieval where coincident measurements are made, while also allowing for a more robust retrieval of particle properties such as spectral single-scattering albedo.

Limbacher, James A.↗
DOE OSTI2021

Ensuring statistical reproducibility of ocean model simulations in the age of hybrid computing

Novel high performance computing systems that feature hybrid architectures require large scale code refactoring to unravel underlying exploitable parallelism. Such redesign can often be accompanied with machine-precision changes as the order of computation cannot always be maintained. For chaotic systems like climate models, these round-off level differences can grow rapidly. Systematic errors may also manifest initially as machine-precision differences. Isolating genuine round off level differences from such errors remains a challenge. Here, we apply two-sample equality of distribution tests to evaluate statistical reproducibility of the ocean model component of US Department of Energy's Energy Exascale Earth System Model (E3SM). A 2-year control simulation ensemble is compared to a modified ensemble as a test case - after a known non-bit-for-bit change in a model component is introduced - to evaluate the null hypothesis that the two ensembles are statistically indistinguishable. To quantify the false negative rates of these tests, we conduct a formal power analysis using a targeted suite of short simulation ensembles. The ensemble suite contains several perturbed ensembles, each with a progressively different climate than the baseline ensemble - obtained by perturbing the magnitude of a single model tuning parameter, the Gent and McWilliams κ, in a controlled manner. The null hypothesis is evaluated for each of perturbed ensembles using these tests. The power analysis informs on the detection limits of the tests for given ensemble size allowing model developers to evaluate the impact of an introduced non-bit-for-bit change to the model.

Mahajan, Salil↗
NASA NTRS1994

Two-sample discrimination of Poisson means

This paper presents a statistical test for detecting significant differences between two random count accumulations. The null hypothesis is that the two samples share a common random arrival process with a mean count proportional to each sample's exposure. The model represents the partition of N total events into two counts, A and B, as a sequence of N independent Bernoulli trials whose partition fraction, f, is determined by the ratio of the exposures of A and B. The detection of a significant difference is claimed when the background (null) hypothesis is rejected, which occurs when the observed sample falls in a critical region of (A, B) space. The critical region depends on f and the desired significance level, alpha. The model correctly takes into account the fluctuations in both the signals and the background data, including the important case of small numbers of counts in the signal, the background, or both. The significance can be exactly determined from the cumulative binomial distribution, which in turn can be inverted to determine the critical A(B) or B(A) contour. This paper gives efficient implementations of these tests, based on lookup tables. Applications include the detection of clustering of astronomical objects, the detection of faint emission or absorption lines in photon-limited spectroscopy, the detection of faint emitters or absorbers in photon-limited imaging, and dosimetry.

Lampton, M.↗
NASA NTRS1978

Performance evaluation of the SAO VLG-11 atomic hydrogen masers

A new generation of frequency standards, the VLG-11 hydrogen maser, has been designed and built at the Smithsonian Astrophysical Observatory (SAO). A comprehensive series of environmental and short-term stability tests on three VLG-11 masers has been completed and evaluated. The test results reported here show substantial improvements over previous hydrogen masers in measured sensitivity to variations in ambient temperature, barometric pressure, and magnetic fields. The maser frequency stability, as represented by the two-sample (Allan) variance, has been measured for averaging times ranging from 1 to 100,00 seconds. The variance is lower than 1 x 10 to the -15th for averaging intervals between 1400 and 20,000 seconds. For times shorter than 4000 seconds, the measured stability data agree remarkably well with theoretical values calculated from thermal noise mechanisms and the 'noise within the linewidth' regime is clearly discernable in the data.

Levine, M. W.↗
NASA NTRS1984

The system design of a rubidium maser frequency standard

The Rubidium Maser Frequency Standard is a precision frequency source with excellent short-term stability. A type PBR-II Rb maser frequency standard was developed by the Beijing Institute of Radio Metrology and Measurement (BIRMM). The time-domain frequency stability (two-sample variance) of this frequency standard is less than 1/5 times 10 to the 13th power for t=10ms yields 1.0s, fh=1.0 KHz. Two PBR-II frequency standards were used as reference frequency sources in a frequency stability measurement system. Some important system characteristics for the PBR-II Rb maser frequency standard such as phase noise and frequency stability transfer characteristics are discussed. Furthermore, the design of the frequency standard for optimum frequency stability of the output signal; the choice of a voltage controlled crystal oscillator for the frequency standard; the design of the phase-locked loop; and the frequency stability test results on the PBR-II are discussed.

Xiong, C. X.↗
NASA NTRS1985

Statistical analysis of astronomical data containing upper bounds - General methods and examples drawn from X-ray astronomy

Statistical procedures taken from the field of survival analysis have been adapted to astronomical usage and have been applied to a sample of stars in the B-V color range between 0.1 and 0.5 with measured soft X-ray luminosities and projected equatorial velocities. The two-sample problem and linear regression problem with arbitrarily censored data were studied. A new method for determining the linear regression between two random variables in the presence of arbitrary censoring has been developed which can also be used for a likelihood-ratio test for the independence of two random variables and for principal-component analysis in the presence of arbitrary censoring. The required numerical computations can be carried out straightforwardly and rapidly.

Schmitt, J. H. M. M.↗