Search NASA⌕ Search

SEARCH · Search NASA

Results for “hypothesis tests”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Banff Challenge 2

Experimental particle physics collaborations constantly seek newer and better ideas for improving the sensitivity of their searches for new particles and phenomena. Statistical techniques are the last step in interpreting the results of an experiment; they are used to make discoveries (hypothesis testing), and to measure parameters (point estimation). They are also used in the first step - experiment and analysis design. Banff Challenge 2 asks participants to test their methods of discovering hidden signals in simulated datasets and of measuring the properties of these signals. The Challenge problems are described, and the performances of the submitted entries is summarized, for datasets with and without simulated signals present.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Vector wind profile gust model

To enable development of a vector wind gust model suitable for orbital flight test operations and trade studies, hypotheses concerning the distributions of gust component variables were verified. Methods for verification of hypotheses that observed gust variables, including gust component magnitude, gust length, u range, and L range, are gamma distributed and presented. Observed gust modulus has been drawn from a bivariate gamma distribution that can be approximated with a Weibull distribution. Zonal and meridional gust components are bivariate gamma distributed. An analytical method for testing for bivariate gamma distributed variables is presented. Two distributions for gust modulus are described and the results of extensive hypothesis testing of one of the distributions are presented. The validity of the gamma distribution for representation of gust component variables is established.

Adelfang, S. I.↗

Individual Response Trends to Urban Air Mobility Noise in A Laboratory Study

Advanced Air Mobility, of which Urban Air Mobility (UAM) is a subset, presents new opportunities for more dynamic aviation transportation systems. It is essential to understand the human response to the noise of these vehicles for sustainable operations. This research aims to investigate the relationship between the noise level and number of events on individual human annoyance to UAM vehicle noise. A “noise and number” laboratory psychoacoustic study was conducted in the Exterior Effects Room at NASA Langley Research Center in January 2023. A total of 38 participants listened to 4-minute audio clips of UAM vehicle flyover noise and provided their annoyance response on an 11-point numerical scale. A test hypothesis was that annoyance to multiple flyovers can be found by adding the annoyance responses to individual flyovers. Results for the overall test subject pool suggest that annoyance grows faster than hypothesized as the number of aircraft events increases. This trend of faster annoyance growth only seems to hold when the individual flyovers are heard as distinct events. Parsing the data down to individuals provides further insight into the mechanisms contributing to the increased annoyance with number of events.

noise and number↗

Sequential Wald Test Employing a Constrained Filter Bank: Application to Spacecraft Conjunctions

A binary Wald sequential probability ratio test that uses the residuals of norm-inequality-constrained Kalman filters for its likelihood ratio may be employed for a class of compound hypothesis tests on non-stationary systems. The hypotheses concern an inequality constraint on the norm of some elements of the system state. Each of two filters minimizes the summed-squares of its estimation errors subject to one or the other direction of the inequality constraint. This solution is motivated by the problem of satellite conjunction assessment, wherein the constraint concerns the close approach distance between two space objects. The outcome of the test can inform decisions concerning risk mitigation maneuvers.

J. Russell Carpenter↗

Satellite Conjunction Assessment Risk Analysis for "Dilution Region" Events: Issues and Operational Approaches

An important activity within Space Traffic Management is the detection and prevention of possible on-orbit collisions between space objects. The principal parameter for assessing collision likelihood is the probability of collision, which is widely accepted among conjunction assessment practitioners; but it possesses a known deficiency in that it can produce a false sense of safety when the orbital position uncertainties for the conjuncting objects are high. The probability of collision is said to be “diluted” in such a situation and to understate the possible risk; certain approaches have been recommended by researchers to provide (largely conservative) risk estimates and remediation methodologies in these cases. The present analysis explores two of the main proposals for quantifying and remediating possible risk in the dilution region and quantifies their operational implications. These implications with regard to imputed additional workload are considerable, especially in anticipating the conjunction event levels expected with the deployment of the USAF Space Fence radar. This effort has been undertaken as part of a larger enterprise that seeks to clarify the philosophical and statistical underpinnings of the conjunction risk assessment process. The analysis presented herein argues that a form of hypothesis testing is implicitly used in conjunction assessment risk analysis, and that there are a number of conceptual and practical reasons for constructing the associated null hypothesis to counsel against a satellite conjunction remediation action. In short, it is concluded that, for the purposes of determining whether a conjunction remediation action should be pursued, dilution-region probabilities of collision should be treated no differently from those produced under other circumstances.

Probability of collision↗

Matched-pair classification

Following an analogous distinction in statistical hypothesis testing, we investigate variants of machine learning where the training set comes in matched pairs. We demonstrate that even conventional classifiers can exhibit improved performance when the input data has a matched-pair structure. Online algorithms, in particular, converge quicker when the data is presented in pairs. In some scenarios (such as the weak signal detection problem), matched pairs can be generated from independent samples, with the effect not only doubling the nominal size of the training set, but of providing the structure that leads to better learning. A family of 'dipole' algorithms is introduced that explicitly takes advantage of matched-pair structure in the input data and leads to further performance gains. Finally, we illustrate the application of matched-pair learning to chemical plume detection in hyperspectral imagery.

97 MATHEMATICS AND COMPUTING↗

A DATA-DRIVEN MODEL FOR SPECTRA: FINDING DOUBLE REDSHIFTS IN THE SLOAN DIGITAL SKY SURVEY

We present a data-driven method-heteroscedastic matrix factorization, a kind of probabilistic factor analysis-for modeling or performing dimensionality reduction on observed spectra or other high-dimensional data with known but non-uniform observational uncertainties. The method uses an iterative inverse-variance-weighted least-squares minimization procedure to generate a best set of basis functions. The method is similar to principal components analysis (PCA), but with the substantial advantage that it uses measurement uncertainties in a responsible way and accounts naturally for poorly measured and missing data; it models the variance in the noise-deconvolved data space. A regularization can be applied, in the form of a smoothness prior (inspired by Gaussian processes) or a non-negative constraint, without making the method prohibitively slow. Because the method optimizes a justified scalar (related to the likelihood), the basis provides a better fit to the data in a probabilistic sense than any PCA basis. We test the method on Sloan Digital Sky Survey (SDSS) spectra, concentrating on spectra known to contain two redshift components: these are spectra of gravitational lens candidates and massive black hole binaries. We apply a hypothesis test to compare one-redshift and two-redshift models for these spectra, utilizing the data-driven model trained on a random subset of all SDSS spectra. This test confirms 129 of the 131 lens candidates in our sample and all of the known binary candidates, and turns up very few false positives.

79 ASTRONOMY AND ASTROPHYSICS↗

Surveillance system and method having an adaptive sequential probability fault detection test

System and method providing surveillance of an asset such as a process and/or apparatus by providing training and surveillance procedures that numerically fit a probability density function to an observed residual error signal distribution that is correlative to normal asset operation and then utilizes the fitted probability density function in a dynamic statistical hypothesis test for providing improved asset surveillance.

Herzog, James P.↗

Surveillance system and method having an adaptive sequential probability fault detection test

System and method providing surveillance of an asset such as a process and/or apparatus by providing training and surveillance procedures that numerically fit a probability density function to an observed residual error signal distribution that is correlative to normal asset operation and then utilizes the fitted probability density function in a dynamic statistical hypothesis test for providing improved asset surveillance.

Bickford, Randall L.↗

Surveillance System and Method having an Adaptive Sequential Probability Fault Detection Test

System and method providing surveillance of an asset such as a process and/or apparatus by providing training and surveillance procedures that numerically fit a probability density function to an observed residual error signal distribution that is correlative to normal asset operation and then utilizes the fitted probability density function in a dynamic statistical hypothesis test for providing improved asset surveillance.

Bickford, Randall L.↗

Evaluation of Smart Grid Technologies Employed for System Reliability Improvement: Pacific Northwest Smart Grid Demonstration Experience

The Pacific Northwest Smart Grid Demonstration (PNWSGD) was a smart grid technology performance evaluation project that included multiple U.S. states and cooperation from multiple electric utilities in the northwest region. One of the local objectives for the project was to achieve improved distribution system reliability. Toward this end, some PNWSGD utilities automated their distribution systems, including the application of fault detection, isolation, and restoration and advanced metering infrastructure. In light of this investment, a major challenge was to establish a correlation between implementation of these smart grid technologies and actual improvements of distribution system reliability. This paper proposes using Welch’s t-test to objectively determine and quantify whether distribution system reliability is improving over time. The proposed methodology is generic, and it can be implemented by any utility after calculation of the standard reliability indices. The effectiveness of the proposed hypothesis testing approach is demonstrated through comprehensive practical results. It is believed that wider adoption of the proposed approach can help utilities to evaluate a realistic long-term performance of smart grid technologies.

Agalgaonkar, Yashodhan P. (ORCID:0000000290061848)↗

Spatially Dependent Multiple Testing Under Model Misspecification, With Application to Detection of Anthropogenic Influence on Extreme Climate Events

The Weather Risk Attribution Forecast (WRAF) is a forecasting tool that uses output from global climate models to make simultaneous attribution statements about whether and how greenhouse gas emissions have contributed to extreme weather across the globe. However, in conducting a large number of simultaneous hypothesis tests, the WRAF is prone to identifying false “discoveries.” A common technique for addressing this multiple testing problem is to adjust the procedure in a way that controls the proportion of true null hypotheses that are incorrectly rejected, or the false discovery rate (FDR). Unfortunately, generic FDR procedures suffer from low power when the hypotheses are dependent, and techniques designed to account for dependence are sensitive to misspecification of the underlying statistical model. In this paper, we develop a Bayesian decision-theoretical approach for dependent multiple testing and a nonparametric hierarchical statistical model that flexibly controls false discovery and is robust to model misspecification. We illustrate the robustness of our procedure to model error with a simulation study, using a framework that accounts for generic spatial dependence and allows the practitioner to flexibly specify the decision criteria. Finally, we apply our procedure to several seasonal forecasts and discuss implementation for the WRAF workflow. Lastly, supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.

54 ENVIRONMENTAL SCIENCES↗

Automated system function allocation and display format: Task information processing requirements

An important consideration when designing the interface to an intelligent system concerns function allocation between the system and the user. The display of information could be held constant, or 'fixed', leaving the user with the task of searching through all of the available information, integrating it, and classifying the data into a known system state. On the other hand, the system, based on its own intelligent diagnosis, could display only relevant information in order to reduce the user's search set. The user would still be left the task of perceiving and integrating the data and classifying it into the appropriate system state. Finally, the system could display the patterns of data. In this scenario, the task of integrating the data is carried out by the system, and the user's information processing load is reduced, leaving only the tasks of perception and classification of the patterns of data. Humans are especially adept at this form of display processing. Although others have examined the relative effectiveness of alphanumeric and graphical display formats, it is interesting to reexamine this issue together with the function allocation problem. Currently, Johnson Space Center is the test site for an intelligent Thermal Control System (TCS), TEXSYS, being tested for use with Space Station Freedom. Expert TCS engineers, as well as novices, were asked to classify several displays of TEXSYS data into various system states (including nominal and anomalous states). Three different display formats were used: fixed, subset, and graphical. The hypothesis tested was that the graphical displays would provide for fewer errors and faster classification times by both experts and novices, regardless of the kind of system state represented within the display. The subset displays were hypothesized to be the second most effective display format/function allocation condition, based on the fact that the search set is reduced in these displays. Both the subset and the graphic display conditions were hypothesized to be processed more efficiently than the fixed display conditions.

Czerwinski, Mary P.↗

Upper bounds on the error probabilities and asymptotic error exponents in quantum multiple state discrimination

We consider the multiple hypothesis testing problem for symmetric quantum state discrimination between r given states σ₁, …, σ{sub r}. By splitting up the overall test into multiple binary tests in various ways we obtain a number of upper bounds on the optimal error probability in terms of the binary error probabilities. These upper bounds allow us to deduce various bounds on the asymptotic error rate, for which it has been hypothesized that it is given by the multi-hypothesis quantum Chernoff bound (or Chernoff divergence) C(σ₁, …, σ{sub r}), as recently introduced by Nussbaum and Szkoła in analogy with Salikhov's classical multi-hypothesis Chernoff bound. This quantity is defined as the minimum of the pairwise binary Chernoff divergences min{sub j<k}C(σ{sub j},σ). It was known already that the optimal asymptotic rate must lie between C/3 and C, and that for certain classes of sets of states the bound is actually achieved. It was known to be achieved, in particular, when the state pair that is closest together in Chernoff divergence is more than 6 times closer than the next closest pair. Our results improve on this in two ways. First, we show that the optimal asymptotic rate must lie between C/2 and C. Second, we show that the Chernoff bound is already achieved when the closest state pair is more than 2 times closer than the next closest pair. We also show that the Chernoff bound is achieved when at least r - 2 of the states are pure, improving on a previous result by Nussbaum and Szkoła. Finally, we indicate a number of potential pathways along which a proof (or disproof) may eventually be found that the multi-hypothesis quantum Chernoff bound is always achieved.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Variational autoencoders for new physics mining at the Large Hadron Collider

Using variational autoencoders trained on known physics processes, we develop a one-sided threshold test to isolate previously unseen processes as outlier events. Since the autoencoder training does not depend on any specific new physics signature, the proposed procedure doesn’t make specific assumptions on the nature of new physics. An event selection based on this algorithm would be complementary to classic LHC searches, typically based on model-dependent hypothesis testing. Such an algorithm would deliver a list of anomalous events, that the experimental collaborations could further scrutinize and even release as a catalog, similarly to what is typically done in other scientific domains. Event topologies repeating in this dataset could inspire new-physics model building and new experimental searches. Running in the trigger system of the LHC experiments, such an application could identify anomalous events that would be otherwise lost, extending the scientific reach of the LHC.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Time-Based CAN IDS Paper Results Code

Modern vehicles are complex cyber-physical systems made of hundreds of electronic control units (ECUs) that communicate over controller area networks (CANs). This inherited complexity has expanded the CAN attack surface which is vulnerable to message injection attacks. These injections change the overall timing characteristics of messages on the bus, and thus, to detect these malicious messages, time-based intrusion detection systems (IDSs) have been proposed. However, time-based IDSs are usually trained and tested on low-fidelity datasets with unrealistic, labeled attacks. This makes difficult the task of evaluating, comparing, and validating IDSs. Here we detail and benchmark four time-based IDSs against the newly published ROAD dataset, the first open CAN IDS dataset with real (non-simulated) stealthy attacks with physically verified effects. We found that methods that perform hypothesis testing by explicitly estimating message timing distributions have lower performance than methods that seek anomalies in a distribution related statistic. In particular, these “distribution-agnostic” based methods outperform “distribution-based” methods by at least 55% in area under the precision-recall curve (AUC-PR). Our results expand the body of knowledge of CAN time-based IDSs by providing details of these methods and reporting their results when tested on datasets with real advanced attacks. Finally, we develop an after-market plug-in detector using lightweight hardware, which can be used to deploy the best performing IDS method on nearly any vehicle.

Moriano, Pablo [Oak Ridge National Lab. (ORNL), Oa↗

Autoregressive modeling with state-space embedding vectors for damage detection under operational and environmental variability

A nonlinear time series approach is presented to detect damage in systems by using a state-space reconstruction to infer the geometrical structure of a deterministic dynamical system from observed time series response at multiple locations. The unique contribution of this approach is using a Multivariate Autoregressive (MAR) model of a baseline condition to predict the state space, where the model encodes the embedding vectors rather than scalar time series. A hypothesis test is established that the MAR model will fail to predict future response if damage is present in the test condition, and this test is investigated for robustness in the context of operational and environmental variability. The applicability of this approach is demonstrated using acceleration time series from a base-excited 3-story frame structure.

42 ENGINEERING↗