Field Evaluations of Sampling, Interviewing, and Flight Tracking of NASA's Low Boom Flight Demonstrator Aircraft
The first year’s effort identified sampling and interviewing as the principal risks to assessment of prompt reactions to overflights producing low-amplitude sonic booms. It also 1) established the utility of geo-information system-based route planning for LBFD flight missions, 2) developed and demonstrated a prototype of a geographically-distributed, Internet-enabled instrumentation system capable of wide-area tracking of LBFD aircraft in near-real time. The latter system permits synchronizing the conduct of interviews in multiple overflown communities with arrival times of shock waves at interviewing sites; and of measuring, archiving, and processing their acoustic signatures. Means were also recommended for constructing representative, telephone-based samples of eligible respondents living in households within carpet boom corridors adjacent to LBFD flight tracks, and for conducting interviews with cross-sectional (independent) samples of such respondents about their prompt reactions to exposure to low-amplitude sonic booms. A detailed study design was prepared and accepted by NASA for a set of single-contact attempt telephone interviews with a nationally representative sample of households. The study design focused on testing automated and live agent interview completion rates obtainable without callbacks. A minimal (two monitoring station) version of the aircraft tracking system was built and installed near a civil airport in a successful demonstration of the system’s ability to detect and track aircraft movements. The field exercise also demonstrated the ability of the system to capture the acoustic emissions of departing aircraft, and to serve aircraft position and sound level information to remote, geographically-distributed analysts in near-real time. Upon approval of OMB and IRB of the detailed study plan, a stratified, nationally representative sample of landline and wireless telephone-subscribing households was constructed. A total of 12,734 telephone interview contact attempts of the sort required by a straightforward cross-sectional study design were then made. These contact attempts demonstrated the impracticality of conducting a time-critical, cross-sectional study of prompt community response to low-amplitude sonic booms by means of “independent” (single contact attempt per respondent for each LBFD flight mission) telephone samples of respondents. The observed interview completion rates for these single telephone contact attempts were so low (~ 1% to 3% for automated and live agent interviews, respectively) that: 1) the representativeness of collected opinions would be susceptible to intuitive challenge as inadequate, even absent conclusive evidence of non-representativeness. Refuting challenges to representativeness would have to demonstrate that the composition of the actual sample did not differ from that of the target population, a task that is tantamount to proving a negative; 2) the information required to refute allegations of non-representativeness would require a questionnaire considerably lengthier than that required simply to determine the prevalence of boom-induced startle and annoyance. Such a questionnaire would have to inquire about potentially sensitive and intrusive matters, including respondents’ age, gender, education, employment, home ownership, income, ethnicity, family size, and other demographic factors; and 3) unreasonable numbers of attempts would be required to re-contact households with unsuccessful initial contact attempts, given the limited time available for doing so. For example, if about 500 completed interviews were desired in a supersonically overflown community, approximately 50,000 automated interview attempts would have to be made within ten to fifteen minutes of each LBFD overflight. Such large numbers of contact attempts could well exceed the numbers of households available for interview in areas of similar boom exposure levels in some communities near LBFD flight tracks. Such large numbers of interviews could be cost-effectively undertaken only by means of automated (i.e., outgoing interactive voice response) interviewing, a data collection method ill-suited for complex and sensitive questionnaire items. The infeasibility of independent sampling for evaluating prompt responses to LBFD overflights in a cross-sectional study is due in large part to simple non-response: that is, potential respondents – particularly those contacted on wireless telephones – refusing to answer calls with unfamiliar caller IDs. It is also due in part, however, to 1) the lack of time to attempt to contact the same respondent more than once within a few minutes after the arrival of a shock wave at the respondent’s location; and 2) the need to place calls during weekday/daytime hours, when response rates are notably lower than during evenings and weekends. Despite the poor interview completion rates achieved under the above constraints, cross sectional assessments of delayed reactions to LBFD overflights could still be feasible, if multiple attempts could be made to contact respondents during evening and weekend time periods, over extended time periods. Detailed plans for a longitudinal (panel) sample were developed as an alternative to a cross sectional sample design.