Varied statistical problems and tests, volume 2
Multivariate statistical analysis including divergence, probability of misclassification, information theory, and matrix analysis
Source record↗
SEARCH · Search NASA
Results for “test statistics”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Cleanroom certification model
The Cleanroom software development methodology is designed to take the gamble out of product releases for both suppliers and receivers of the software. The ingredients of this procedure are a life cycle of executable product increments, representative statistical testing, and a standard estimate of the MTTF (Mean Time To Failure) of the product at the time of its release. A statistical approach to software product testing using randomly selected samples of test cases is considered. A statistical model is defined for the certification process which uses the timing data recorded during test. A reasonableness argument for this model is provided that uses previously published data on software product execution. Also included is a derivation of the certification model estimators and a comparison of the proposed least squares technique with the more commonly used maximum likelihood estimators.
Currit, P. A.↗
On Determining the Rise, Size, and Duration Classes of a Sunspot Cycle
The behavior of ascent duration, maximum amplitude, and period for cycles 1 to 21 suggests that they are not mutually independent. Analysis of the resultant three-dimensional contingency table for cycles divided according to rise time (ascent duration), size (maximum amplitude), and duration (period) yields a chi-square statistic (= 18.59) that is larger than the test statistic (= 9.49 for 4 degrees-of-freedom at the 5-percent level of significance), thereby, inferring that the null hypothesis (mutual independence) can be rejected. Analysis of individual 2 by 2 contingency tables (based on Fisher's exact test) for these parameters shows that, while ascent duration is strongly related to maximum amplitude in the negative sense (inverse correlation) - the Waldmeier effect, it also is related (marginally) to period, but in the positive sense (direct correlation). No significant (or marginally significant) correlation is found between period and maximum amplitude. Using cycle 22 as a test case, we show that by the 12th month following conventional onset, cycle 22 appeared highly likely to be a fast-rising, larger-than-average-size cycle. Because of the inferred correlation between ascent duration and period, it also seems likely that it will have a period shorter than average length.
Wilson, Robert M.↗
Strong gravitational lensing statistics as a test of cosmogonic scenarios
Gravitational lensing statistics can provide a direct and powerful test of cosmic structure formation theories. Since lensing tests, directly, the magnitude of the nonlinear mass density fluctuations on lines of sight to distant objects, no issues of 'bias' (of mass fluctuations with respect to galaxy density fluctuations) exist here, although lensing observations provide their own ambiguities of interpretation. We develop numerical techniques for generating model density distributions with the very large spatial dynamic range required by lensing considerations and for identifying regions of the simulations capable of multiple image lensing in a conservative and computationally efficient way that should be accurate for splittings significantly larger than 3 seconds. Applying these techniques to existing standard Cold dark matter (CDM) (Omega = 1) and Primeval Baryon Isocurvature (PBI) (Omega = 0.2) simulations (normalized to the Cosmic Background Explorer Satellite (COBE) amplitude), we find that the CDM model predicts large splitting (greater than 8 seconds) lensing events roughly an order-of-magnitude more frequently than the PBI model. Under the reasonable but idealized assumption that lensing structrues can be modeled as singular isothermal spheres (SIS), the predictions can be directly compared to observations of lensing events in quasar samples. Several large splitting (Delta Theta is greater than 8 seconds) cases are predicted in the standard CDM model (the exact number being dependent on the treatment of amplification bias), whereas none is observed. In a formal sense, the comparison excludes the CDM model at high confidence (essentially for the same reason that CDM predicts excessive small-scale cosmic velocity dispersions.) A very rough assessment of low-density but flat CDM model (Omega = 0.3, Lambda/3H(sup 2 sub 0) = 0.7) indicates a far lower and probably acceptable level of lensing. The PBI model is consistent with, but not strongly tested by, the available lensing data, and other open models would presumably do as well as PBI. These preliminary conclusions and the assumptions on which they are based can be tested and the analysis can be applied to other cosmogonic models by straightforward extension of the work presented here.
Cen, Renyue↗
Performance tests of signature extension algorithms
Comparative tests were performed on seven signature extension algorithms to evaluate their effectiveness in correcting for changes in atmospheric haze and sun angle in a Landsat scene. Four of the algorithms were cluster matching, and two were maximum likelihood algorithms. The seventh algorithm determined the haze level in both training and recognition segments and used a set of tables calculated from an atmospheric model to determine the affine transformation that corrects the training signatures for changes in sun angle and haze level. Three of the algorithms were tested on a simulated data set, and all of the algorithms were tested on consecutive-day data. The classification performance on the data sets using the algorithms is presented, along with results of statistical tests on the accuracy and proportion estimates. The three algorithms tested on the simulated data produced significant improvements over the results obtained using untransformed signatures. For the consecutive-day data, the tested algorithms produced improvements in most but not all cases. The tests indicated also that no statistically significant differences were noted among the algorithms.
Abotteen, R.↗
Contrasting Perspectives Of Junior versus Senior NASA ISS Flight Controllers On Leadership And Cultural Issues
NASA flight controllers have always worked in a very demanding environment, but the International Space Station (ISS) poses even more challenges than prior missions. A recent NASA/Ames survey by Parke and Orasanu of NASA/Johnson flight controllers uncovered concerns about communications problems between American personnel and their international counterparts. To better understand these problems, we interviewed 14 senior and 12 junior ISS flight controllers at NASA/Johnson about leadership and cultural challenges they face and strategies for addressing these challenges. The qualitative interview data were coded and tabulated. Here we present quantitative analyses testing for differences between junior and senior controllers. Based on nonparametric statistical tests comparing responses across groups, the senior controllers were significantly more aware of the impact of working in dispersed teams, the context of constant change, and the upcoming multilateral challenges, while junior controllers were more aware of language and cultural issues. We consider our findings in light of other studies of controllers and other known differences between senior and junior controllers. For example, the fact that senior controllers had their formative early experience controlling pre-ISS short-duration Shuttle missions seems to have both positive and negative aspects, which are supported by our data. Our findings may also reflect gender differences, but we cannot unconfound this effect in our data because all the senior respondents were males. Many of the junior-senior differences are not only due to elapsed time on the job, but also due to a cohort effect. The findings of this study should be used for training curricula tailored differently for junior and senior controllers.
Clement, James L.↗
Infusing Statistical Thinking into the NASA Quesst Community Test Campaign
Statistical thinking permeates many important decisions as NASA plans its Quesst mission, which will culminate in a series of community overflights using the X-59 aircraft to demonstrate low-noise supersonic flight. Month-long longitudinal surveys will be deployed to assess human perception and annoyance to this new acoustic phenomenon. NASA works with a large contractor team to develop systems and methodologies to estimate noise doses, to test and field socio-acoustic surveys, and to study the relationship between the two quantities, dose and response, through appropriate choices of statistical models. This latter dose-response relationship will serve as an important tool as national and international noise regulators debate whether overland supersonic flights could be permitted once again within permissible noise limits. In this presentation we highlight several areas where statistical thinking has come into play, including issues of sampling, classification and data fusion, and analysis of longitudinal survey data that are subject to rare events and the consequences of measurement error. We note several operational constraints that shape the appeal or feasibility of some decisions on statistical approaches, and we identify several important remaining questions to be addressed.
Bayesian model↗
FDR doesn't Tell the Whole Story: Joint Influence of Effect Size and Covariance Structure on the Distribution of the False Discovery Proportions
As part of a 2009 Annals of Statistics paper, Gavrilov, Benjamini, and Sarkar report results of simulations that estimated the false discovery rate (FDR) for equally correlated test statistics using a well-known multiple-test procedure. In our study we estimate the distribution of the false discovery proportion (FDP) for the same procedure under a variety of correlation structures among multiple dependent variables in a MANOVA context. Specifically, we study the mean (the FDR), skewness, kurtosis, and percentiles of the FDP distribution in the case of multiple comparisons that give rise to correlated non-central t-statistics when results at several time periods are being compared to baseline. Even if the FDR achieves its nominal value, other aspects of the distribution of the FDP depend on the interaction between signed effect sizes and correlations among variables, proportion of true nulls, and number of dependent variables. We show examples where the mean FDP (the FDR) is 10% as designed, yet there is a surprising probability of having 30% or more false discoveries. Thus, in a real experiment, the proportion of false discoveries could be quite different from the stipulated FDR.
Feiveson, Alan H.↗
The generation of random numbers on the IBM 360/44
A summary is presented of the generation and testing of random numbers on the IBM 360/44 computer. Various generators and statistical tests are considered.
Speed, F. M.↗
Implications for high speed research: The relationship between sonic boom signature distortion and atmospheric turbulence
In this study there were two primary tasks. The first was to develop an algorithm for quantifying the distortion in a sonic boom. Such an algorithm should be somewhat automatic, with minimal human intervention. Once the algorithm was developed, it was used to test the hypothesis that the cause of a sonic boom distortion was due to atmospheric turbulence. This hypothesis testing was the second task. Using readily available sonic boom data, we statistically tested whether there was a correlation between the sonic boom distortion and the distance a boom traveled through atmospheric turbulence.
Sparrow, Victor W.↗
A Statistical Filtering Approach for Gravity Recovery and Climate Experiment (GRACE) Gravity Data
We describe and analyze a statistical filtering approach for GRACE data that uses a parametrized model for the temporal evolution of the GRACE coefficients. After least-squares adjustment, a statistical test is performed to assess the significance of the estimated parameters. If the test is passed, the parameters are used by the filter in the reconstruction of the field; otherwise they are rejected. The test is performed, and the filter is formed, separately for annual components of the model and the trend. This new approach is distinct from Gaussian smoothing since it uses the data themselves to test for specific components of the time-varying gravity field. The statistical filter appears inherently to remove most of the "stripes" present in the GRACE fields, although destriping the fields prior to filtering seems to help the trend recovery. We demonstrate that the statistical filter produces reasonable maps for the annual components and trend. We furthermore assess the statistical filter for the annual components using ground-based GPS data in South America by assuming that the annual component of the gravity signal is associated only with groundwater storage. The un-destriped, statistically filtered field has a X2 value relative to the GPS data consistent with the best result from smoothing. In the space domain, the statistical filters are qualitatively similar to Gaussian smoothing. Unlike Gaussian smoothing, however, the statistical filter has significant sidelobes, including large negative sidelobes on the north-south axis, potentially revealing information on the errors, and the correlations among the errors, for the GRACE coefficients.
Davis. J. L.↗
Characterization of Space Shuttle Reusable Rocket Motor Static Test Stand Thrust Measurements
Space Shuttle Reusable Solid Rocket Motors (RSRM) are static tested at two ATK Thiokol Propulsion facilities in Utah, T-24 and T-97. The newer T-97 static test facility was recently upgraded to allow thrust measurement capability. All previous static test motor thrust measurements have been taken at T-24; data from these tests were used to characterize thrust parameters and requirement limits for flight motors. Validation of the new T-97 thrust measurement system is required prior to use for official RSRM performance assessments. Since thrust cannot be measured on RSRM flight motors, flight motor measured chamber pressure and a nominal thrust-to-pressure relationship (based on static test motor thrust and pressure measurements) are used to reconstruct flight motor performance. Historical static test and flight motor performance data are used in conjunction with production subscale test data to predict RSRM performance. The predicted motor performance is provided to support Space Shuttle trajectory and system loads analyses. Therefore, an accurate nominal thrust-to-pressure (F/P) relationship is critical for accurate RSRM flight motor performance and Space Shuttle analyses. Flight Support Motors (FSM) 7, 8, and 9 provided thrust data for the validation of the T-97 thrust measurement system. The T-97 thrust data were analyzed and compared to thrust previously measured at T-24 to verify measured thrust data and identify any test-stand bias. The T-97 FIP data were consistent and within the T-24 static test statistical family expectation. The FSMs 7-9 thrust data met all NASA contract requirements, and the test stand is now verified for future thrust measurements.
Mart L. Cook↗
"Weekend-effect" evidence for intensification of storms by pollution over U.S.
Aerosols change the way precipitating clouds evolve, due to their effect on condensation and ice formation. The net effect of aerosols on precipitation varies depending on the local environment and synoptic situation, the types of aerosols, and also on how far downstream one looks for effects. Repeated experiments with controlled release of aerosols would be helpful in unraveling how aerosols affect precipitation. The weekly variation in human activity may provide just such a set of experiments. Using the rainfall estimates from the passive microwave instrument on the Tropical Rainfall Measuring Mission (TRMM), which has been orbiting for over eight years and can view areas as far north as about 40 degs, we find that there is indeed a weekly variation in summer (JJA) rainfall over the southeastern U.S. land area, with the average rain rate peaking in the middle of the week (Tue-Thu). Rainfall over nearby oceanic areas, on the other hand, seems to peak on weekends. A bootstrap statistical test indicates that the variations are unlikely to be due to chance. Physically related weekly variations in the statistics of ground-based measurements and model reanalysis data are found. The midweek peak in rainfall over land coincides with the well documented midweek peak in pollution around much of the U.S., and is unlikely to be due to other weekly variations in human influence. The midweek increase is also unlikely to be a consequence of the radiative heating or cooling of the atmosphere by the aerosols.
Bell, T. L.↗
An Assessment of Actual and Potential Building Climate Zone Change and Variability From the Last 30 Years Through 2100 Using NASA's MERRA and CMIP5 Simulations
Background: In the US, residential and commercial building infrastructure combined consumes about 40% of total energy usage and emits about 39% of total CO2 emission (DOE/EIA "Annual Energy Outlook 2013"). Building codes, as used by local and state enforcement entities are typically tied to the dominant climate within an enforcement jurisdiction classified according to various climate zones. These climate zones are based upon a 30-year average of local surface observations and are developed by DOE and ASHRAE. Establishing the current variability and potential changes to future building climate zones is very important for increasing the energy efficiency of buildings and reducing energy costs and emissions in the future. Objectives: This paper demonstrates the usefulness of using NASA's Modern Era Retrospective-analysis for Research and Applications (MERRA) atmospheric data assimilation to derive the DOE/ASHRAE building climate zone maps and then using MERRA to define the last 30 years of variability in climate zones for the Continental US. An atmospheric assimilation is a global atmospheric model optimized to satellite, atmospheric and surface in situ measurements. Using MERRA as a baseline, we then evaluate the latest Climate Model Inter-comparison Project (CMIP) climate model Version 5 runs to assess potential variability in future climate zones under various assumptions. Methods: We derive DOE/ASHRAE building climate zones using surface and temperature data products from MERRA. We assess these zones using the uncertainties derived by comparison to surface measurements. Using statistical tests, we evaluate variability of the climate zones in time and assess areas in the continental US for statistically significant trends by region. CMIP 5 produced a data base of over two dozen detailed climate model runs under various greenhouse gas forcing assumptions. We evaluate the variation in building climate zones for 3 different decades using an ensemble and quartile statistics to provide an assessment of potential building climate zone changes relative to the uncertainties demonstrated using MERRA. Findings and Conclusions: These results show that there is a statistically significant increase in the area covered by warmer climate zones and a tendency for a reduction of area in colder climate zones in some limited regions. The CMIP analysis shows that models vary from relatively little building climate zone change for the least sensitive and conservation assumptions to a warming of at most 3 zones for certain areas, particularly the north central US by the end of the 21st century.
Stackhouse, Paul W., Jr.↗
Detection of Digital Elevation Model Errors Using X-band Weather Radar
Flight in Instrument Meteorological Conditions requires pilots to manipulate flight controls while referring to a Primary Flight Display. The Primary Flight Display indicates aircraft attitude along with, in some cases, many other state variables such as altitude, speed, and guidance cues. Synthetic Vision Systems have been proposed that overlay the traditional information provided on Primary Flight Displays onto a scene depicting the location of terrain and other geo-spatial features.Terrain models used by these displays must have sufficient quality to avoid providing misleading information. This paper describes how X-band radar measurements can be used as part of a monitor, and/or maintenance system, to quantify the integrity of terrain models that are used by systems such as Synthetic Vision. Terrain shadowing effects, as seen by the radar, are compared in a statistical manner against estimated shadow feature elements extracted from the stored terrain model from the perspective of the airborne observer. A test statistic is defined that enables detection of errors as small as the range resolution of the radar. Experimental results obtained from two aircraft platforms hosting certified commercial-off-the-shelf X-band radars test the premise and illustrate its potential.
Young, Steven D.↗
A rule-based software test data generator
Rule-based software test data generation is proposed as an alternative to either path/predicate analysis or random data generation. A prototype rule-based test data generator for Ada programs is constructed and compared to a random test data generator. Four Ada procedures are used in the comparison. Approximately 2000 rule-based test cases and 100,000 randomly generated test cases are automatically generated and executed. The success of the two methods is compared using standard coverage metrics. Simple statistical tests showing that even the primitive rule-based test data generation prototype is significantly better than random data generation are performed. This result demonstrates that rule-based test data generation is feasible and shows great promise in assisting test engineers, especially when the rule base is developed further.
Deason, William H.↗
Safeguarding a Lunar Rover with Wald's Sequential Probability Ratio Test
The virtual bumper is a safeguarding mechanism for autonomous and remotely operated robots. In this paper we take a new approach to the virtual bumper system by using an old statistical test. By using a modified version of Wald's sequential probability ratio test we demonstrate that we can reduce the number of false positive reported by the virtual bumper, thereby saving valuable mission time. We use the concept of sequential probability ratio to control vehicle speed in the presence of possible obstacles in order to increase certainty about whether or not obstacles are present. Our new algorithm reduces the chances of collision by approximately 98 relative to traditional virtual bumper safeguarding without speed control.
safeguarding↗
Simulation of deep one- and two-dimensional redshift surveys
It is shown that slice or pencil-beam redshift surveys of galaxies can be simulated in a box with nonequal sides. This method saves a lot of computer time and memory while providing essentially the same results as from whole-cube simulations. A 2457.6/h Mpc-long rod (out to a redshift z = 0.58 in two opposite directions) is simulated using the standard biased cold dark matter model as an example to mimic the recent deep pencil-beam surveys by Broadhurst et al. (1990). The structures (spikes) seen in these simulated samples occur when the narrow pencil-beam pierces walls, filaments, and clusters appearing randomly along the line-of-sight. A statistical test for goodness of fit to a periodic lattice has been applied to the observations and the simulations. It is found that the statistical significance level (P = 15.4 percent) is not strong enough to reject the null hypothesis that the observations and the simulations were drawn at random from the same set.
Park, Changbom↗