Garrett G Sadler | Search NASA

NASA NTRS2021

A Cognitive Walkthrough of Multiple Drone Delivery Operations

Advances of early twenty-first century aviation and transportation technologies provide opportunities for enhanced aerial projects, and the overall integration of unmanned aircraft systems (UAS) into the National Airspace System (NAS) has applications across a wide range of operations. Through these, remote operators have learned to manage several UAS at the same time in a variety of operational environments. The present work details a component piece of an ongoing body of research into multi-UAS operations. Beginning in early 2020, NASA has collaborated with Uber Technologies to design and develop concepts of operations, roles and responsibilities, and ground control station (GCS) concepts to enable food delivery operations via multiple, small UAS (sUAS). A cognitive walkthrough was chosen as the method for data collection. This allowed information to be gathered from UAS subject matter experts (SMEs) that could further mature designs for future human-in-the-loop (HITL) simulations; in addition, it allowed information to be collected remotely during the stringent restrictions of the COVID-19 pandemic. Consequently, the described cognitive walkthrough activity utilized remote data collection protocols mediated through the usage of programs designed for presentation and telecommunications. Scenarios were designed, complete with airspace, contingencies, and remedial actions, to be presented to the SMEs. Information was collected using a combination of rating scales and open-ended questions. Results received from the SMEs revealed expected hazards, workloads, and information concerns inherent in the contingency scenarios. SMEs also provided insight into the design of GCS tools and displays as well as the duties and relationships of human operators (i.e., monitors) and automation (i.e., informers and flight managers). Implications of these findings are discussed.

unmanned aircraft systems↗

NASA NTRS2021

UAS Integration in the NAS Flight Test 6: Full Mission Results

Recent standards development efforts for the integration of Unmanned Aircraft Systems (UAS) into the National Airspace System (NAS) such as those in RTCA Inc. Special Committee 228 (SC-228) have focused on relatively large UAS transitioning to and from Class A airspace. In an effort to expand the range of vehicle classes that can access the NAS, the NASA UAS Integration in the NAS project has investigated Low Size, Weight, and Power (Low SWaP) technologies that would allow smaller UAS to detect-and-avoid (DAA) traffic. Through batch and human in the loop (HITL) simulation studies, the UAS Integration in the NAS DAA subproject have identified candidate performance standards that would contribute to enabling extended Low SWaP, UAS operations under 10,000 feet. These candidate performance standards include minimum field of regard (FOR) values for Low SWaP air surveillance sensors as well as a DAA well-clear (DWC) definition which can be applied to non-cooperative traffic to reduce the required maneuver initiation range. To test the assumptions of the project’s simulation studies and validate the candidate performance standards, a live flight research event was executed at NASA Armstrong Flight Research Center. The UAS Integration in the NAS Project Flight Test 6 Full Mission sought to characterize UAS pilot responses to traffic conflicts using a representative Low SWAP DAA system in an operational NAS environment. To achieve this, live, virtual and constructive distributed environment (LVC-DE) elements were combined to simulate a sector of Oakland center airspace and induce encounters with a live, manned aircraft. A Navmar Applied Sciences Tigershark XP was used as the UAS ownship and was integrated into the test architecture to enable it to be controlled from a Vigilant Spirit Control Station (VSCS) research ground control station. Qualified UAS pilots were recruited to act as subject pilots under test (SPUT) to control the Tigershark XP in a simulated mission while coordinating with a participating air traffic controller in simulated airspace. The intruder speed, intruder equipage and encounter geometry were varied between six scripted encounters per SPUT. Various metrics were collected including pilot reaction time from the onset of DAA alert, ATC coordination rate, probability and severity of losses of DAA well clear, and subjective ratings of system acceptability. The implications of these results on the development of standards for Low SWAP DAA systems will be discussed.

UAV↗

NASA NTRS2020

An Evaluation of UAS Pilot Workload and Acceptability Ratings with Four Simulated Radar Declaration Ranges

Currently, minimum operational performance standards (MOPS) are being developed for a broader range of unmanned aircraft system (UAS) platforms, including smaller UAS that will feature onboard sensors that are low in size, weight, and power, otherwise known as low SWaP. The low SWaP sensors used to detect non-cooperative traffic will have limited declaration ranges compared to those designed for medium-to-large UAS. A human-in-the-loop (HITL) study was conducted examining four possible radar declaration ranges (i.e., 1.5 NM, 2 NM, 2.5 NM, and 3 NM) for a potential low SWaP sensor with a detect and avoid (DAA) system encountering various non-cooperative encounters in Oakland Center airspace. Participants had lower workload, particularly workload associated with temporal demand and effort, in scenarios that featured larger declaration ranges. Furthermore, participants reported better ability to remain DAA well clear within the larger declaration range conditions, specifically with the 2.5 NM and 3 NM conditions.

UAS↗

NASA NTRSDate not supplied

An Examination of Two Non-Cooperative Detect and Avoid Well Clear Definitions

NASA’s Unmanned Aircraft Systems Integration into the National Airspace System (UAS in the NAS) project examines the technical barriers associated with the operation of UAS in civil airspace. The present study explored the differential effects of two candidate non-cooperative Detect-and-Avoid Well Clear (DWC)definitions on pilot and system performance in a human-in-the-loop simulation. Active-duty UAS pilots were recruited to maintain DWC against representative Class 4 encounter types with a low size, weight, and power (SWaP) radar declaration range of 3.5 nautical miles (nmi). Objective performance indicated that pilots could consistently maintain DWC against non-cooperative intruders with either DWC candidate, with negligible differences in response times and separation performance against caution and warning-level threats. While losses of DWC were avoided at rates comparable to Phase 1 findings, pilots uploaded their responses to caution-level alerts over 5 seconds faster in the current setup relative to Phase 1. Encounters with faster closure rates were susceptible to shortened caution-level alert durations, especially when employing the DWC criterion with the additional ‘Tau’ (temporal) component. Consequently, caution-level threats frequently elevated to warning-level status (nearly twice as often with theTau candidate). The variable caution alert durations appeared to impact pilots’ coordination with air traffic control (ATC), as ATC approval rates were lower with the ‘Tau’and ‘Disc’ candidates relative to Phase 1 research. Ultimately, the increased alerting time enabled by the Disc candidate deemed it more suitable for any reductions to the assumed radar declaration range requirement, which was re-evaluated in a follow-on study. Findings from this study will inform Phase 2 Minimum Operational Performance Standards (MOPS)development for UAS with alternative surveillance equipment and performance capabilities.

Kevin J Monk↗

NASA NTRSDate not supplied

Display and Automation Considerations for the Airborne Collision Avoidance System Xu

In this paper we examine several display and automation considerations of a collision avoidance system that is currently under development: the Airborne Collision Avoidance System (ACAS) Xu. This study builds on previous work conducted as part of NASA’s Unmanned Aircraft Systems (UAS) Integration into the National Airspace System (NAS) project. ACAS Xu represents the next-generation successor to the Traffic Alert and Collision Avoidance System (TCAS II), wherein the Xu variant is intended for UAS applications. Whereas TCAS II exclusively issues RAs in the vertical dimension, a major distinction between ACAS Xu and previous collision avoidance (CA) systems is the introduction of horizontal and “blended” RAs (i.e., RAs with both horizontal and vertical components). This present work was conducted as an engineering analysis involving two parts. In Part 1, a two-by-two, within-subjects study was performed that manipulated how RAs were presented to a pilot situated at a UAS ground control station. Five participants experienced four experimental trials in which text and aural alerting characteristics were manipulated. In Part 2, another five participants experienced four trials in which the levels of automation were manipulated with regard to the CA and return-to-course (RTC) tasks. The results for Part 1 found no effect of display or alerting configuration on pilot performance. However, it was discovered that pilot response time to RAs greatly depended on the RA type. In particular, pilots were quicker to respond to vertical RAs (M = 4.52 seconds) than horizontal (M = 7.42 seconds) and blended (M = 9.68 seconds) RAs in which both dimensions were issued simultaneously. For Part 2 of the study, pilots found both auto-CA and auto-RTC functions equally useful. Most pilots were comfortable with the automation, however responses were mixed. Three of five participants indicated high levels of comfort with the auto-CA function, while two rated their comfort as low. Pilots’ comfort for the auto-RTC functionality was slightly higher: four out of five pilots gave high ratings, while one pilot gave a low rating. Overall, pilots ordinally ranked their preference for automated functions as auto-CA together with auto-RTC (when an aural alert announces a change between CA and RTC states), auto-CA, and auto-CA and RTC (without the aural state-change announcement). Recommendations for improving the display of automation are also discussed.

collision avoidance↗

NASA NTRSDate not supplied

An Evaluation of UAS Pilot Workload and Acceptability Ratings with Four Simulated Radar Declaration Ranges

Currently, minimum operational performance standards (MOPS) are being developed for a broader range of unmanned aircraft system (UAS) platforms, including smaller UAS that will feature onboard sensors that are low in size, weight, and power, otherwise known as low SWaP. The low SWaP sensors used to detect non-cooperative traffic will have limited declaration ranges compared to those designed for medium-to-large UAS. A human-in-the-loop (HITL) study was conducted examining four possible radar declaration ranges (i.e., 1.5 NM, 2 NM, 2.5 NM, and 3 NM) for a potential low SWaP sensor with a detect and avoid (DAA) system encountering various non-cooperative encounters in Oakland Center airspace. Participants had lower workload, particularly workload associated with temporal demand and effort, in scenarios that featured larger declaration ranges. Furthermore, participants reported better ability to remain DAA well clear within the larger declaration range conditions, specifically with the 2.5 NM and 3 NM conditions.

UAS↗

NASA NTRSDate not supplied

UAS Pilot Assessments of Display and Alerting for the Next Generation Airborne Collision Avoidance System Xu

Unmanned aircraft systems (UAS) must comply with specific standards to operate in the National Airspace System (NAS). Among the requirements are the detect and avoid (DAA) capabilities, which include display, alerting, and guidance specifications. Previous studies have queried pilots for their subjective feedback on these display elements on earlier systems; the present study sought pilot evaluations with an initial iteration of the unmanned variant of a Next Generation Airborne Collision Avoidance System (ACAS XU). Sixteen participants piloted simulated aircraft with both standalone and integrated DAA displays. Their opinions were gathered using post-block and post-simulation questionnaires as well as guided debriefs. The data showed pilots had better understanding and comfort with the system when using an integrated display. Pilots also rated ACAS Xu alerting and guidance as generally acceptable and effective. Implications for further development of ACAS XU and DAA displays are discussed.

unmanned aircraft systems↗

NASA NTRSDate not supplied

A Cognitive Walkthrough of Multiple Drone Delivery Operations

Advances of early twenty-first century aviation and transportation technologies provide opportunities for enhanced aerial projects, and the overall integration of unmanned aircraft systems (UAS) into the National Airspace System (NAS) has applications across a wide range of operations. Through these, remote operators have learned to manage several UAS at the same time in a variety of operational environments. The present work details a component piece of an ongoing body of research into multi-UAS operations. Beginning in early 2020, NASA has collaborated with Uber Technologies to design and develop concepts of operations, roles and responsibilities, and ground control station (GCS) concepts to enable food delivery operations via multiple, small UAS (sUAS). A cognitive walkthrough was chosen as the method for data collection. This allowed information to be gathered from UAS subject matter experts (SMEs) that could further mature designs for future human-in-the-loop (HITL) simulations; in addition, it allowed information to be collected remotely during the stringent restrictions of the COVID-19 pandemic. Consequently, the described cognitive walkthrough activity utilized remote data collection protocols mediated through the usage of programs designed for presentation and telecommunications. Scenarios were designed, complete with airspace, contingencies, and remedial actions, to be presented to the SMEs. Information was collected using a combination of rating scales and open-ended questions. Results received from the SMEs revealed expected hazards, workloads, and information concerns inherent in the contingency scenarios. SMEs also provided insight into the design of GCS tools and displays as well as the duties and relationships of human operators (i.e., monitors) and automation (i.e., informers and flight managers). Implications of these findings are discussed.

unmanned aircraft systems↗

NASA NTRSDate not supplied

Streamlining Tactical Operator Handoffs During Multi-Vehicle Applications

Increased automation has shifted the operator control paradigm from a single operator controlling a single vehicle, to multiple operators collaborating to control multiple vehicles; this paradigm is known as m:N. Many questions remain unanswered in this new operational paradigm about the division of assets as workload for individual operators varies overtime. This paper explores the management of workload by enabling operators to temporarily handoff vehicles among each other. A study was conducted to explore both a manual and assisted method for performing handoffs during manipulated contingency scenarios. The assisted handoff method allowed subjects to easily choose and group nominal and/or contingency vehicles. The number of contingencies was also manipulated to determine the effect workload had on how pilots utilized the ability to handoff vehicles. Results show subjects performed handoffs more often when there were more contingencies and when the assisted handoff tool was available. In addition, the assisted tool make subjects feel more comfortable, enabling them to feel like they could take longer to resolve contingency situations. Lastly, even during contingencies, subjects were able to successfully complete secondary tasks.

M:N operations↗

NASA NTRSDate not supplied

A Remote, Human-in-the-Loop Evaluation of a Multiple-Drone Delivery Operation

Over time, advances in unmanned aircraft systems (UAS) have enabled a shift in the operational paradigm from one operator managing one aircraft to that of multiple operators working together to manage multiple aircraft. This shift has highlighted the need for effective human-autonomy teaming methods to maintain manageable workload levels for operators as well as high standards of system performance and safety. This paper presents a study aimed at evaluating whether automation can help operators manage workload during small UAS (sUAS) package delivery scenarios featuring contingency situations. These contingency situations, resulting from unplanned UAS Volume Reservations (UVRs), required flight path reroutes for multiple aircraft simultaneously. The study manipulated the number of aircraft affected by the UVRs and the level of automation support. The presence of terrain conflicts was also controlled within each scenario. Due to the COVID-19 pandemic, subjects were not able to gain direct access to the Ground Control System (GCS). Therefore, the study was conducted using a subject-surrogate paradigm that required subjects to relay commands through a verbal protocol from remote locations outside of the lab to a researcher surrogate who had direct control of the GCS interfaces at the lab location. Results show that the automated support condition was associated with faster reroute response times, more efficient reroute maneuvers, and significantly lower levels of perceived workload than the manual reroute condition. However, the automation support level did not significantly impact pilots’ ability to avoid the UVR successfully; pilots were overwhelmingly capable of avoiding the UVR in all conditions. The presence of terrain conflicts primarily impacted pilot performance by leading to multiple uploads per vehicle, which was not typically required when pilots only needed to maneuver laterally. Although subjects did not have direct control over the GCS, subjective ratings indicate that the displays under test provided them with sufficient information to manage their aircraft and promptly respond to the unplanned UVRs. Overall, the objective and subjective data strongly suggest that the verbal protocol and subject-surrogate paradigm were effective methods for collecting data remotely amid the COVID-19 pandemic.

multi-UAS↗

NASA NTRSDate not supplied

Effects of Communication Modality on Pilot-Controller Coordination during a Simulated m:N Operation

The last decade or so has seen growing interest in new control paradigms and concepts of operation for uncrewed aircraft systems (UAS) in which multiple aircraft are piloted remotely by a single or relatively small number of people. Referred to as “one-to-many” and “many-to- many” (alternatively, “multi-operator, multi-vehicle”)—and frequently expressed as the corresponding ratios, 1:N and m:N—such novel configurations of aircraft and the people who manage them are seen as critical to the path to future operations involving UAS. Examples of industry domains interested in these control paradigms are small package delivery services utilizing small UAS and passenger-carrying, short-range “Urban Air Mobility” (UAM) operations. Stakeholders in such operations have identified communication and coordination of flight activity with air traffic controllers (ATC) as a barrier to operations. In contrast to present-day flight operations, in which a pilot communicates with one ATC on one radio frequency for one aircraft, multi-vehicle operations potentially entail a significant increase in pilot task load for management of comms. New concepts, such as UAS Service Suppliers (USSs) and Providers of Services to UAM (PSUs), have been proposed to address the known bottleneck for Air Traffic Management (ATM) presented by multi-vehicle operations. While progress has been steadily made over years developing USSs and PSUs, it is generally expected that initial UAM operations will rely on traditional voice-over-radio communication with ATC for purposes of ATM. The current study was a human-in-the-loop simulation that had participants, each possessing a Private Pilot License, act as the ground-based pilot-in- command for multiple vehicles in a hypothetical UAM service in the San Francisco Bay Area. The experiment utilized a 2-by-3, within-subjects design in which the pilot’s Vehicle Load (4 vs. 12) and Comm System (Voice, Datalink, and a Hybrid) were manipulated. The task given to pilots was to use the Comm System to coordinate flight activity for all aircraft with appropriate controllers, having to obtain departure and arrival clearances at “vertiport” facilities and transition clearances for any intermediate airspaces along the route. Pilots were additionally responsible for compliance with vectoring instructions issued by ATC. Subjective workload questionnaires (NASA-TLX) were administered following each experimental trial. Screen recordings of the pilot’s Ground Control Station (GCS) and audio recordings of trials were subsequently coded to obtain performance metrics: response times and error rates. Presented in this paper are results related to pilot responses to vectoring instructions issued by ATC. Workload was found to be significantly higher in the 12-Vehicle condition compared to the 4-Vehicle condition, nearly maxing out the NASA-TLX overall workload scale. There was no significant difference made by the Comm System on workload ratings. Pilots’ response times to communications were fastest in the Voice condition, although overall “service time” for compliance was shorter in Datalink and Hybrid conditions in most cases. Errors by pilots were frequent in both Vehicle Load conditions, most perniciously when using the Voice system. The results of this study suggest tradeoffs in advantages and disadvantages of the three comm systems. Recommendations for communication system design are provided taking the tradeoffs into account.

Garrett G Sadler↗

NASA NTRSDate not supplied

Effects of Communication Modality on Pilot-Controller Coordination during a Simulated m:N Operation

The last decade or so has seen growing interest in new control paradigms and concepts of operation for uncrewed aircraft systems (UAS) in which multiple aircraft are piloted remotely by a single or relatively small number of people. Referred to as “one-to-many” and “many-to- many” (alternatively, “multi-operator, multi-vehicle”)—and frequently expressed as the corresponding ratios, 1:N and m:N—such novel configurations of aircraft and the people who manage them are seen as critical to the path to future operations involving UAS. Examples of industry domains interested in these control paradigms are small package delivery services utilizing small UAS and passenger-carrying, short-range “Urban Air Mobility” (UAM) operations. Stakeholders in such operations have identified communication and coordination of flight activity with air traffic controllers (ATC) as a barrier to operations. In contrast to present-day flight operations, in which a pilot communicates with one ATC on one radio frequency for one aircraft, multi-vehicle operations potentially entail a significant increase in pilot task load for management of comms. New concepts, such as UAS Service Suppliers (USSs) and Providers of Services to UAM (PSUs), have been proposed to address the known bottleneck for Air Traffic Management (ATM) presented by multi-vehicle operations. While progress has been steadily made over years developing USSs and PSUs, it is generally expected that initial UAM operations will rely on traditional voice-over-radio communication with ATC for purposes of ATM. The current study was a human-in-the-loop simulation that had participants, each possessing a Private Pilot License, act as the ground-based pilot-in- command for multiple vehicles in a hypothetical UAM service in the San Francisco Bay Area. The experiment utilized a 2-by-3, within-subjects design in which the pilot’s Vehicle Load (4 vs. 12) and Comm System (Voice, Datalink, and a Hybrid) were manipulated. The task given to pilots was to use the Comm System to coordinate flight activity for all aircraft with appropriate controllers, having to obtain departure and arrival clearances at “vertiport” facilities and transition clearances for any intermediate airspaces along the route. Pilots were additionally responsible for compliance with vectoring instructions issued by ATC. Subjective workload questionnaires (NASA-TLX) were administered following each experimental trial. Screen recordings of the pilot’s Ground Control Station (GCS) and audio recordings of trials were subsequently coded to obtain performance metrics: response times and error rates. Presented in this paper are results related to pilot responses to vectoring instructions issued by ATC. Workload was found to be significantly higher in the 12-Vehicle condition compared to the 4-Vehicle condition, nearly maxing out the NASA-TLX overall workload scale. There was no significant difference made by the Comm System on workload ratings. Pilots’ response times to communications were fastest in the Voice condition, although overall “service time” for compliance was shorter in Datalink and Hybrid conditions in most cases. Errors by pilots were frequent in both Vehicle Load conditions, most perniciously when using the Voice system. The results of this study suggest tradeoffs in advantages and disadvantages of the three comm systems. Recommendations for communication system design are provided taking the tradeoffs into account.

urban air mobility↗

NASA NTRSDate not supplied

sUAS Ground Control Station Capabilities Impact on Fleet Management

Future Small Uncrewed Aerial Systems (sUAS) missions will require multiple operators to collaborate and manage large fleets of highly automated vehicles. It is critical that these operators maintain the necessary situational awareness to modify vehicle missions if/when unexpected situations arise in the operating environment. It has been suggested that highly automated vehicle capabilities can lead to reducing the capabilities given to operators to modify/direct vehicle missions. This paper explores various Ground Control Station (GCS) capabilities configurations and their impact on sUAS fleet management. Results show that a GCS with a combination of manual and automated capabilities allowed participants to make more effective decisions while maintaining workload and response times similar to that of a GCS with only automated capabilities.

sUAS↗

NASA NTRSDate not supplied

A Queuing Theory Approach to Pilot-Controller Coordination for m:N Operations

In recent years, attention and interest by industry and researchers has grown in a control paradigm for remotely piloted aircraft termed “m:N operations.” In an m:N operation, a team of m remote pilots in command (RIPCs) collaboratively manage the flights of N aircraft. A consequence of an m:N concept of operations is that the RPICs will have to switch attention from one aircraft to another and from one task to another. Previous research in m:N operations has focused on the workload experienced by an RPIC and their level of situation awareness on their flights. Researchers have found that RPIC workload and situation awareness are generally sensitive to increasing N, although NASA’s Multi-Vehicle (m:N) Working Group has suggested that the driver of workload/situation awareness is the number of exceptions requiring human intervention as opposed to the value of N itself. In any case, a natural antecedent of workload is task load. In this paper, queueing theory is applied to a 1:N Urban Air Mobility (UAM) air taxi operation in order to estimate pilot task load for managing radio communications with air traffic controllers (ATCs) under increasing N. An M/M/1 queueing system is used to model the RIPC’s servicing of calls and clearance requests (e.g., departure, arrival, or airspace transition) to ATC for the N aircraft. Important parameters for the queueing model are the task arrival rate and the average service time for task completion. Radio communication times from past human-in-the-loop simulation studies are used to measure service times for a 1:4 and 1:12 UAM operation and to interpolate service times for 4 < N < 12. A Monte Carlo method is then employed, using the measured and interpolated service times, to estimate arrival rate and related queueing statistics. The paper concludes by considering the estimated queuing statistics, particularly the RPIC’s utilization (i.e., proportion of time actively servicing tasks), the length of the task queue over time, and the implications for task-balanced system design.

task load↗