Results for “FREQUENTIST METHODS”

NASA NTRS2010

Sequential Probability Ratio Test for Collision Avoidance Maneuver Decisions

When facing a conjunction between space objects, decision makers must chose whether to maneuver for collision avoidance or not. We apply a well-known decision procedure, the sequential probability ratio test, to this problem. We propose two approaches to the problem solution, one based on a frequentist method, and the other on a Bayesian method. The frequentist method does not require any prior knowledge concerning the conjunction, while the Bayesian method assumes knowledge of prior probability densities. Our results show that both methods achieve desired missed detection rates, but the frequentist method's false alarm performance is inferior to the Bayesian method's

Carpenter, J. Russell↗

NASA NTRS2016

Uncertainty Estimates of Psychoacoustic Thresholds Obtained from Group Tests

Adaptive psychoacoustic test methods, in which the next signal level depends on the response to the previous signal, are the most efficient for determining psychoacoustic thresholds of individual subjects. In many tests conducted in the NASA psychoacoustic labs, the goal is to determine thresholds representative of the general population. To do this economically, non-adaptive testing methods are used in which three or four subjects are tested at the same time with predetermined signal levels. This approach requires us to identify techniques for assessing the uncertainty in resulting group-average psychoacoustic thresholds. In this presentation we examine the Delta Method of frequentist statistics, the Generalized Linear Model (GLM), the Nonparametric Bootstrap, a frequentist method, and Markov Chain Monte Carlo Posterior Estimation and a Bayesian approach. Each technique is exercised on a manufactured, theoretical dataset and then on datasets from two psychoacoustics facilities at NASA. The Delta Method is the simplest to implement and accurate for the cases studied. The GLM is found to be the least robust, and the Bootstrap takes the longest to calculate. The Bayesian Posterior Estimate is the most versatile technique examined because it allows the inclusion of prior information.

Rathsam, Jonathan↗

NASA NTRS2022

Comparison of Likelihood Methods for Generalized Linear Mixed Models with Application to Quiet Supersonic Flights 2018 Data

Repeated measurement will be a feature of the survey data collected during the Quesst missionX-59 community response tests (CRT). Since each participant will report his or her categorical level of annoyance in response to multiple events, the responses from any single individual may be correlated with one another. Several models within the class of generalized linear mixed models (GLMM) are pertinent to the analysis of correlated categorical outcomes; the random intercept logistic regression model is one example. Both Bayesian and frequentist methods for fitting these models are available, with frequentist methods relying on some form of approximation (of either an integral or the integrand) that appears in the marginal likelihood function. Given several anticipated similarities of the X-59 CRT data to data collected during a past risk reduction, Quiet Supersonic Flights 2018 (QSF18), this short note is intended to create awareness. It documents an instance in which a reported population average dose-response relationship derived from QSF18 single event data was distorted by the integral approximation applied in likelihood-based methods. We review some of the available literature on the topic, compare the outputs of several different computational approaches implemented in available statistical software, and present simple corrective actions that may be useful during the Quesst mission.

dose-response model↗

NASA NTRS2013

Bayesian Revision of Residual Detection Power

This paper addresses some issues with quality assessment and quality assurance in response surface modeling experiments executed in wind tunnels. The role of data volume on quality assurance for response surface models is reviewed. Specific wind tunnel response surface modeling experiments are considered for which apparent discrepancies exist between fit quality expectations based on implemented quality assurance tactics, and the actual fit quality achieved in those experiments. These discrepancies are resolved by using Bayesian inference to account for certain imperfections in the assessment methodology. Estimates of the fraction of out-of-tolerance model predictions based on traditional frequentist methods are revised to account for uncertainty in the residual assessment process. The number of sites in the design space for which residuals are out of tolerance is seen to exceed the number of sites where the model actually fails to fit the data. A method is presented to estimate how much of the design space in inadequately modeled by low-order polynomial approximations to the true but unknown underlying response function.

DeLoach, Richard↗

NASA NTRS2020

Using Bayesian Estimation to Quantify the Risks of Spaceflight

Complex PRA is the best method and current gold standard, as it allows us to reflect changes (improvements) in individual spacecraft system risks over time. Yet, Bayesian methods – using simple inputs – allowed us to get estimates that are in line with high-complexity PRA. Here Bayesian estimation acts as a “gut check” on PRA (supposing a “reasonable” prior is used). Frequentist calculations can overestimate P(failure), especially when total number of trials are few (e.g., after Challenger). In general, Bayesian probability estimation can be used in any risk assessment situation to integrate what we think we know ahead of time with what we observe over time.

Reynolds, Robert J.↗

NASA NTRS2015

Parameter Estimation for Compact Binaries with Ground-Based Gravitational-Wave Observations Using the LALInference

The Advanced LIGO and Advanced Virgo gravitational wave (GW) detectors will begin operation in the coming years, with compact binary coalescence events a likely source for the first detections. The gravitational waveforms emitted directly encode information about the sources, including the masses and spins of the compact objects. Recovering the physical parameters of the sources from the GW observations is a key analysis task. This work describes the LALInference software library for Bayesian parameter estimation of compact binary signals, which builds on several previous methods to provide a well-tested toolkit which has already been used for several studies. We show that our implementation is able to correctly recover the parameters of compact binary signals from simulated data from the advanced GW detectors. We demonstrate this with a detailed comparison on three compact binary systems: a binary neutron star (BNS), a neutron star - black hole binary (NSBH) and a binary black hole (BBH), where we show a cross-comparison of results obtained using three independent sampling algorithms. These systems were analysed with non-spinning, aligned spin and generic spin configurations respectively, showing that consistent results can be obtained even with the full 15-dimensional parameter space of the generic spin configurations. We also demonstrate statistically that the Bayesian credible intervals we recover correspond to frequentist confidence intervals under correct prior assumptions by analysing a set of 100 signals drawn from the prior. We discuss the computational cost of these algorithms, and describe the general and problem-specific sampling techniques we have used to improve the efficiency of sampling the compact binary coalescence (CBC) parameter space.

gravitational wave↗

NASA NTRSDate not supplied

Effects of Dose Error and Sample Size on Sonic Boom Dose-Response Curves

NASA will soon be collecting noise-annoyance community survey data as the X-59 aircraft flies supersonically over several communities in the USA. Sparse measurements of the X-59 sonic thumps will be used together with physics-based simulations to estimate noise doses at survey participant locations. These dose estimates have associated error that affects the accuracy of modeled dose-response curves, which can result in misestimation of annoyance. The precision in dose-response curves is also a consideration in selecting the number of survey participants. To enable pretest studies of dose error and precision, simulated dose-response data were generated based on NASA’s Quiet Supersonic Flights 2018 test. The data included various degrees of dose error and sample size. Frequentist multilevel logistic regression models were fit to the true and perturbed dose-response data. Simple proportional relationships were identified between the model parameters and the perturbation standard deviation. The summary dose-response curves illustrate the impact on accuracy if dose error is not accounted for in the model. The precision in the dose-response curves is also shown as the number of participants and degree of participation is varied. Finally, sampling variability is illustrated by showing the dose-response curves for several replicates with random draws of participants and errors.

X-59↗

NASA NTRSDate not supplied

Effects of Dose Error and Sample Size on Sonic Boom Dose-response Curves

NASA will soon be collecting noise-annoyance community survey data as the X-59 aircraft flies supersonically over several communities in the USA. Sparse measurements of the X-59 sonic thumps will be used together with physics-based simulations to estimate noise doses at survey participant locations. These dose estimates have associated error that affects the accuracy of modeled dose-response curves, which can result in misestimation of annoyance. The precision in dose-response curves is also a consideration in selecting the number of survey participants. To enable pretest studies of dose error and precision, simulated dose-response data were generated based on NASA’s Quiet Supersonic Flights 2018 test. The data included various degrees of dose error and sample size. Frequentist multilevel logistic regression models were fit to the true and perturbed dose-response data. Simple proportional relationships were identified between the model parameters and the perturbation standard deviation. The summary dose-response curves illustrate the impact on accuracy if dose error is not accounted for in the model. The precision in the dose-response curves is also shown as the number of participants and degree of participation is varied. Finally, sampling variability is illustrated by showing the dose-response curves for several replicates with random draws of participants and errors.

X-59↗