Kevin J Monk | Search NASA

NASA NTRS2021

A Cognitive Walkthrough of Multiple Drone Delivery Operations

Advances of early twenty-first century aviation and transportation technologies provide opportunities for enhanced aerial projects, and the overall integration of unmanned aircraft systems (UAS) into the National Airspace System (NAS) has applications across a wide range of operations. Through these, remote operators have learned to manage several UAS at the same time in a variety of operational environments. The present work details a component piece of an ongoing body of research into multi-UAS operations. Beginning in early 2020, NASA has collaborated with Uber Technologies to design and develop concepts of operations, roles and responsibilities, and ground control station (GCS) concepts to enable food delivery operations via multiple, small UAS (sUAS). A cognitive walkthrough was chosen as the method for data collection. This allowed information to be gathered from UAS subject matter experts (SMEs) that could further mature designs for future human-in-the-loop (HITL) simulations; in addition, it allowed information to be collected remotely during the stringent restrictions of the COVID-19 pandemic. Consequently, the described cognitive walkthrough activity utilized remote data collection protocols mediated through the usage of programs designed for presentation and telecommunications. Scenarios were designed, complete with airspace, contingencies, and remedial actions, to be presented to the SMEs. Information was collected using a combination of rating scales and open-ended questions. Results received from the SMEs revealed expected hazards, workloads, and information concerns inherent in the contingency scenarios. SMEs also provided insight into the design of GCS tools and displays as well as the duties and relationships of human operators (i.e., monitors) and automation (i.e., informers and flight managers). Implications of these findings are discussed.

unmanned aircraft systems↗

NASA NTRS2021

UAS Integration in the NAS Flight Test 6: Full Mission Results

Recent standards development efforts for the integration of Unmanned Aircraft Systems (UAS) into the National Airspace System (NAS) such as those in RTCA Inc. Special Committee 228 (SC-228) have focused on relatively large UAS transitioning to and from Class A airspace. In an effort to expand the range of vehicle classes that can access the NAS, the NASA UAS Integration in the NAS project has investigated Low Size, Weight, and Power (Low SWaP) technologies that would allow smaller UAS to detect-and-avoid (DAA) traffic. Through batch and human in the loop (HITL) simulation studies, the UAS Integration in the NAS DAA subproject have identified candidate performance standards that would contribute to enabling extended Low SWaP, UAS operations under 10,000 feet. These candidate performance standards include minimum field of regard (FOR) values for Low SWaP air surveillance sensors as well as a DAA well-clear (DWC) definition which can be applied to non-cooperative traffic to reduce the required maneuver initiation range. To test the assumptions of the project’s simulation studies and validate the candidate performance standards, a live flight research event was executed at NASA Armstrong Flight Research Center. The UAS Integration in the NAS Project Flight Test 6 Full Mission sought to characterize UAS pilot responses to traffic conflicts using a representative Low SWAP DAA system in an operational NAS environment. To achieve this, live, virtual and constructive distributed environment (LVC-DE) elements were combined to simulate a sector of Oakland center airspace and induce encounters with a live, manned aircraft. A Navmar Applied Sciences Tigershark XP was used as the UAS ownship and was integrated into the test architecture to enable it to be controlled from a Vigilant Spirit Control Station (VSCS) research ground control station. Qualified UAS pilots were recruited to act as subject pilots under test (SPUT) to control the Tigershark XP in a simulated mission while coordinating with a participating air traffic controller in simulated airspace. The intruder speed, intruder equipage and encounter geometry were varied between six scripted encounters per SPUT. Various metrics were collected including pilot reaction time from the onset of DAA alert, ATC coordination rate, probability and severity of losses of DAA well clear, and subjective ratings of system acceptability. The implications of these results on the development of standards for Low SWAP DAA systems will be discussed.

UAV↗

NASA NTRS2020

A Human-in-theLoop Evaluation of ACAS Xu

As part of the Phase 2 UAS DAA MOPS, a Class 3 DAA system has been under development with the potential to resolve many of the limitations of Class 1 and 2 systems. A Class 3 system would combine the DAA and CA functions into a single, unified system, and would also extend the CA capabilities relative to TCAS II. Class 3 is enabled by the Airborne Collision Avoidance System (ACAS) XU, a next generation CA system developed specifically for UAS operations. Unlike Class 2 systems, ACAS XU provides CA protection against both cooperative and non-cooperative traffic. Class 3 systems also expand the CA logic to allow horizontal RAs in addition to vertical RAs. The purpose of the current study was to evaluate ACAS XU in a real-time, HITL simulation. The latest version of ACAS XU was implemented and pilots were tasked with responding to scripted traffic conflicts over the course of four experimental trials. Variables included the location of the ACAS XU guidance information (standalone vs. integrated) and traffic conflict type (DAA or CA threat). Sixteen active UAS pilots participated in the study, with ATC and ‘pseudo’ pilots acting as airspace confederates. Results showed that pilots were able to maintain DWC with ACAS XU at a rate comparable to previous research (~5%). Compliance rates to initial RAs were high (~90%) but dropped significantly when the target heading value issued during horizontal RAs were updated over the course of an encounter (30-70%). The implications of these findings on the display, alerting, and guidance requirements for Class 3 systems will be discussed.

unmanned aircraft systems↗

NASA NTRS2020

UAS Pilot Performance Comparisons with Different Low Size, Weight and Power Sensor Ranges

The present study evaluated the performance of UAS pilots under four simulated low size, weight, and power (SwaP) sensor ranges: 1.5nmi, 2.0nmi, 2.5nmi, and 3.0nmi. Nine active-duty UAS pilots responded to scripted DAA conflicts against non-cooperative intruders while flying a simulated RQ-7 Shadow at varied speeds along a pre-filed flight path in Class E airspace. Findings revealed a linear effect of sensor range on alerting time and separation performance, with nearly every DAA well clear (DWC) violation and all Near Mid-Air Collision (NMAC) events occurring below 2.5nmi. Response time differences at these reduced ranges were negligible due to the high frequency of warning-level alerts that require an immediate response. Since caution alert duration was truncated to some degree by each tested declaration range, pilots were often unable to coordinate their avoidance maneuvers with ATC prior to their uploads. Nonetheless, the 2.5nmi range allowed minimum alerting times that were sufficient for acceptable pilot performance. These findings will inform DAA system requirements for UAS with alternative surveillance equipment and aircraft performance capabilities. Implications on DAA display and sensor requirements are discussed.

unmanned aircraft systems↗

NASA NTRS2020

An Evaluation of UAS Pilot Workload and Acceptability Ratings with Four Simulated Radar Declaration Ranges

Currently, minimum operational performance standards (MOPS) are being developed for a broader range of unmanned aircraft system (UAS) platforms, including smaller UAS that will feature onboard sensors that are low in size, weight, and power, otherwise known as low SWaP. The low SWaP sensors used to detect non-cooperative traffic will have limited declaration ranges compared to those designed for medium-to-large UAS. A human-in-the-loop (HITL) study was conducted examining four possible radar declaration ranges (i.e., 1.5 NM, 2 NM, 2.5 NM, and 3 NM) for a potential low SWaP sensor with a detect and avoid (DAA) system encountering various non-cooperative encounters in Oakland Center airspace. Participants had lower workload, particularly workload associated with temporal demand and effort, in scenarios that featured larger declaration ranges. Furthermore, participants reported better ability to remain DAA well clear within the larger declaration range conditions, specifically with the 2.5 NM and 3 NM conditions.

UAS↗

NASA NTRSDate not supplied

An Examination of Two Non-Cooperative Detect and Avoid Well Clear Definitions

NASA’s Unmanned Aircraft Systems Integration into the National Airspace System (UAS in the NAS) project examines the technical barriers associated with the operation of UAS in civil airspace. The present study explored the differential effects of two candidate non-cooperative Detect-and-Avoid Well Clear (DWC)definitions on pilot and system performance in a human-in-the-loop simulation. Active-duty UAS pilots were recruited to maintain DWC against representative Class 4 encounter types with a low size, weight, and power (SWaP) radar declaration range of 3.5 nautical miles (nmi). Objective performance indicated that pilots could consistently maintain DWC against non-cooperative intruders with either DWC candidate, with negligible differences in response times and separation performance against caution and warning-level threats. While losses of DWC were avoided at rates comparable to Phase 1 findings, pilots uploaded their responses to caution-level alerts over 5 seconds faster in the current setup relative to Phase 1. Encounters with faster closure rates were susceptible to shortened caution-level alert durations, especially when employing the DWC criterion with the additional ‘Tau’ (temporal) component. Consequently, caution-level threats frequently elevated to warning-level status (nearly twice as often with theTau candidate). The variable caution alert durations appeared to impact pilots’ coordination with air traffic control (ATC), as ATC approval rates were lower with the ‘Tau’and ‘Disc’ candidates relative to Phase 1 research. Ultimately, the increased alerting time enabled by the Disc candidate deemed it more suitable for any reductions to the assumed radar declaration range requirement, which was re-evaluated in a follow-on study. Findings from this study will inform Phase 2 Minimum Operational Performance Standards (MOPS)development for UAS with alternative surveillance equipment and performance capabilities.

Kevin J Monk↗

NASA NTRSDate not supplied

Display and Automation Considerations for the Airborne Collision Avoidance System Xu

In this paper we examine several display and automation considerations of a collision avoidance system that is currently under development: the Airborne Collision Avoidance System (ACAS) Xu. This study builds on previous work conducted as part of NASA’s Unmanned Aircraft Systems (UAS) Integration into the National Airspace System (NAS) project. ACAS Xu represents the next-generation successor to the Traffic Alert and Collision Avoidance System (TCAS II), wherein the Xu variant is intended for UAS applications. Whereas TCAS II exclusively issues RAs in the vertical dimension, a major distinction between ACAS Xu and previous collision avoidance (CA) systems is the introduction of horizontal and “blended” RAs (i.e., RAs with both horizontal and vertical components). This present work was conducted as an engineering analysis involving two parts. In Part 1, a two-by-two, within-subjects study was performed that manipulated how RAs were presented to a pilot situated at a UAS ground control station. Five participants experienced four experimental trials in which text and aural alerting characteristics were manipulated. In Part 2, another five participants experienced four trials in which the levels of automation were manipulated with regard to the CA and return-to-course (RTC) tasks. The results for Part 1 found no effect of display or alerting configuration on pilot performance. However, it was discovered that pilot response time to RAs greatly depended on the RA type. In particular, pilots were quicker to respond to vertical RAs (M = 4.52 seconds) than horizontal (M = 7.42 seconds) and blended (M = 9.68 seconds) RAs in which both dimensions were issued simultaneously. For Part 2 of the study, pilots found both auto-CA and auto-RTC functions equally useful. Most pilots were comfortable with the automation, however responses were mixed. Three of five participants indicated high levels of comfort with the auto-CA function, while two rated their comfort as low. Pilots’ comfort for the auto-RTC functionality was slightly higher: four out of five pilots gave high ratings, while one pilot gave a low rating. Overall, pilots ordinally ranked their preference for automated functions as auto-CA together with auto-RTC (when an aural alert announces a change between CA and RTC states), auto-CA, and auto-CA and RTC (without the aural state-change announcement). Recommendations for improving the display of automation are also discussed.

collision avoidance↗

NASA NTRSDate not supplied

UAS Pilot Performance Comparisons with Different Low Size, Weight and Power Sensor Ranges

The present study evaluated the performance of UAS pilots under four simulated low size, weight, and power (SwaP) sensor ranges: 1.5nmi, 2.0nmi, 2.5nmi, and 3.0nmi. Nine active-duty UAS pilots responded to scripted DAA conflicts against non-cooperative intruders while flying a simulated RQ-7 Shadow at varied speeds along a pre-filed flight path in Class E airspace. Findings revealed a linear effect of sensor range on alerting time and separation performance, with nearly every DAA well clear (DWC) violation and all Near Mid-Air Collision (NMAC) events occurring below 2.5nmi. Response time differences at these reduced ranges were negligible due to the high frequency of warning-level alerts that require an immediate response. Since caution alert duration was truncated to some degree by each tested declaration range, pilots were often unable to coordinate their avoidance maneuvers with ATC prior to their uploads. Nonetheless, the 2.5nmi range allowed minimum alerting times that were sufficient for acceptable pilot performance. The study findings will inform DAA system requirements for UAS with alternative surveillance equipment and aircraft performance capabilities. Implications on DAA display and sensor requirements are discussed.

unmanned aircraft systems↗

NASA NTRSDate not supplied

A Human-in-the-Loop Evaluation of ACAS Xu

As part of the Phase 2 UAS DAA MOPS, a Class 3 DAA system has been under development with the potential to resolve many of the limitations of Class 1 and 2 systems. A Class 3 system would combine the DAA and CA functions into a single, unified system, and would also extend the CA capabilities relative to TCAS II. Class 3 is enabled by the Airborne Collision Avoidance System (ACAS) XU, a next generation CA system developed specifically for UAS operations. Unlike Class 2 systems, ACAS XU provides CA protection against both cooperative and non-cooperative traffic. Class 3 systems also expand the CA logic to allow horizontal RAs in addition to vertical RAs. The purpose of the current study was to evaluate ACAS XU in a real-time, HITL simulation. The latest version of ACAS XU was implemented and pilots were tasked with responding to scripted traffic conflicts over the course of four experimental trials. Variables included the location of the ACAS XU guidance information (standalone vs. integrated) and traffic conflict type (DAA or CA threat). Sixteen active UAS pilots participated in the study, with ATC and ‘pseudo’ pilots acting as airspace confederates. Results showed that pilots were able to maintain DWC with ACAS XU at a rate comparable to previous research (~5%). Compliance rates to initial RAs were high (~90%) but dropped significantly when the target heading value issued during horizontal RAs were updated over the course of an encounter (30-70%). The implications of these findings on the display, alerting, and guidance requirements for Class 3 systems will be discussed.

unmanned aircraft systems↗

NASA NTRSDate not supplied

A Human-in-the-Loop Evaluation of ACAS Xu

As part of the Phase 2 UAS DAA MOPS, a Class 3 DAA system has been under development with the potential to resolve many of the limitations of Class 1 and 2 systems. A Class 3 system would combine the DAA and CA functions into a single, unified system, and would also extend the CA capabilities relative to TCAS II. Class 3 is enabled by the Airborne Collision Avoidance System (ACAS) XU, a next generation CA system developed specifically for UAS operations. Unlike Class 2 systems, ACAS XU provides CA protection against both cooperative and non-cooperative traffic. Class 3 systems also expand the CA logic to allow horizontal RAs in addition to vertical RAs. The purpose of the current study was to evaluate ACAS XU in a real-time, HITL simulation. The latest version of ACAS XU was implemented and pilots were tasked with responding to scripted traffic conflicts over the course of four experimental trials. Variables included the location of the ACAS XU guidance information (standalone vs. integrated) and traffic conflict type (DAA or CA threat). Sixteen active UAS pilots participated in the study, with ATC and ‘pseudo’ pilots acting as airspace confederates. Results showed that pilots were able to maintain DWC with ACAS XU at a rate comparable to previous research (~5%). Compliance rates to initial RAs were high (~90%) but dropped significantly when the target heading value issued during horizontal RAs were updated over the course of an encounter (30-70%). The implications of these findings on the display, alerting, and guidance requirements for Class 3 systems will be discussed.

unmanned aircraft systems↗

NASA NTRSDate not supplied

UAS Pilot Performance Comparisons with Different Low Size, Weight and Power Sensor Ranges

The present study evaluated the performance of UAS pilots under four simulated low size, weight, and power (SwaP) sensor ranges: 1.5nmi, 2.0nmi, 2.5nmi, and 3.0nmi. Nine active-duty UAS pilots responded to scripted DAA conflicts against non-cooperative intruders while flying a simulated RQ-7 Shadow at varied speeds along a pre-filed flight path in Class E airspace. Findings revealed a linear effect of sensor range on alerting time and separation performance, with nearly every DAA well clear (DWC) violation and all Near Mid-Air Collision (NMAC) events occurring below 2.5nmi. Response time differences at these reduced ranges were negligible due to the high frequency of warning-level alerts that require an immediate response. Since caution alert duration was truncated to some degree by each tested declaration range, pilots were often unable to coordinate their avoidance maneuvers with ATC prior to their uploads. Nonetheless, the 2.5nmi range allowed minimum alerting times that were sufficient for acceptable pilot performance. These findings will inform DAA system requirements for UAS with alternative surveillance equipment and aircraft performance capabilities. Implications on DAA display and sensor requirements are discussed.

unmanned aircraft systems↗

NASA NTRSDate not supplied

An Evaluation of UAS Pilot Workload and Acceptability Ratings with Four Simulated Radar Declaration Ranges

Currently, minimum operational performance standards (MOPS) are being developed for a broader range of unmanned aircraft system (UAS) platforms, including smaller UAS that will feature onboard sensors that are low in size, weight, and power, otherwise known as low SWaP. The low SWaP sensors used to detect non-cooperative traffic will have limited declaration ranges compared to those designed for medium-to-large UAS. A human-in-the-loop (HITL) study was conducted examining four possible radar declaration ranges (i.e., 1.5 NM, 2 NM, 2.5 NM, and 3 NM) for a potential low SWaP sensor with a detect and avoid (DAA) system encountering various non-cooperative encounters in Oakland Center airspace. Participants had lower workload, particularly workload associated with temporal demand and effort, in scenarios that featured larger declaration ranges. Furthermore, participants reported better ability to remain DAA well clear within the larger declaration range conditions, specifically with the 2.5 NM and 3 NM conditions.

UAS↗

NASA NTRSDate not supplied

UAS Pilot Assessments of Display and Alerting for the Next Generation Airborne Collision Avoidance System Xu

Unmanned aircraft systems (UAS) must comply with specific standards to operate in the National Airspace System (NAS). Among the requirements are the detect and avoid (DAA) capabilities, which include display, alerting, and guidance specifications. Previous studies have queried pilots for their subjective feedback on these display elements on earlier systems; the present study sought pilot evaluations with an initial iteration of the unmanned variant of a Next Generation Airborne Collision Avoidance System (ACAS XU). Sixteen participants piloted simulated aircraft with both standalone and integrated DAA displays. Their opinions were gathered using post-block and post-simulation questionnaires as well as guided debriefs. The data showed pilots had better understanding and comfort with the system when using an integrated display. Pilots also rated ACAS Xu alerting and guidance as generally acceptable and effective. Implications for further development of ACAS XU and DAA displays are discussed.

unmanned aircraft systems↗

NASA NTRSDate not supplied

Detect-and-Avoid Maneuver Planning: Benefits of Including Route Recapture

Development of Detect-and-avoid (DAA) systems’ maneuver guidance requirements at RTCA has centered on tactical maneuvers away from the intruder. Recapturing the flight-plan route the initial maneuver is not directly taken into account by most DAA maneuver guidance algorithms. This work demonstrates potential challenges and inefficiencies that can arise in recapturing the route after resolving a conflict. Horizontal resolution trajectories for a test matrix of encounters with varying aircraft speeds and geometries are computed by minimizing either flight time or deviation to serve as a baseline. Turn directions computed from a reference DAA algorithm coupled with a pilot selection model and a second algorithm called Autoresolver (AR) are compared to these baseline resolution trajectories. Baseline results show that turning into the intruder yields favorable cost for most encounters. Additional analysis of pilot response data shows that only three-fourths of pilots’ horizontal maneuvers turn into intruders, a percentage much lower than the baseline results. Improvement to a DAA guidance algorithm based on findings in this work is discussed.

Route Recapture↗

NASA NTRSDate not supplied

Detect-and-Avoid Maneuver Planning: Benefits of Including Route Recapture

Development of Detect-and-avoid (DAA) systems’ maneuver guidance requirements at RTCA has centered on tactical maneuvers away from the intruder. Recapturing the flight-plan route the initial maneuver is not directly taken into account by most DAA maneuver guidance algorithms. This work demonstrates potential challenges and inefficiencies that can arise in recapturing the route after resolving a conflict. Horizontal resolution trajectories for a test matrix of encounters with varying aircraft speeds and geometries are computed by minimizing either flight time or deviation to serve as a baseline. Turn directions computed from a reference DAA algorithm coupled with a pilot selection model and a second algorithm called Autoresolver (AR) are compared to these baseline resolution trajectories. Baseline results show that turning into the intruder yields favorable cost for most encounters. Additional analysis of pilot response data shows that only three-fourths of pilots’ horizontal maneuvers turn into intruders, a percentage much lower than the baseline results. Improvement to a DAA guidance algorithm based on findings in this work is discussed.

Route Recapture↗

NASA NTRSDate not supplied

A Cognitive Walkthrough of Multiple Drone Delivery Operations

Advances of early twenty-first century aviation and transportation technologies provide opportunities for enhanced aerial projects, and the overall integration of unmanned aircraft systems (UAS) into the National Airspace System (NAS) has applications across a wide range of operations. Through these, remote operators have learned to manage several UAS at the same time in a variety of operational environments. The present work details a component piece of an ongoing body of research into multi-UAS operations. Beginning in early 2020, NASA has collaborated with Uber Technologies to design and develop concepts of operations, roles and responsibilities, and ground control station (GCS) concepts to enable food delivery operations via multiple, small UAS (sUAS). A cognitive walkthrough was chosen as the method for data collection. This allowed information to be gathered from UAS subject matter experts (SMEs) that could further mature designs for future human-in-the-loop (HITL) simulations; in addition, it allowed information to be collected remotely during the stringent restrictions of the COVID-19 pandemic. Consequently, the described cognitive walkthrough activity utilized remote data collection protocols mediated through the usage of programs designed for presentation and telecommunications. Scenarios were designed, complete with airspace, contingencies, and remedial actions, to be presented to the SMEs. Information was collected using a combination of rating scales and open-ended questions. Results received from the SMEs revealed expected hazards, workloads, and information concerns inherent in the contingency scenarios. SMEs also provided insight into the design of GCS tools and displays as well as the duties and relationships of human operators (i.e., monitors) and automation (i.e., informers and flight managers). Implications of these findings are discussed.

unmanned aircraft systems↗

NASA NTRSDate not supplied

Physiological and Subjective Responses of Pilots During Advanced Air Mobility Flight Testing With Automated Systems

Aviation is constantly evolving, mostly due to the integration of automated systems into the National Airspace System. This presents a host of challenges and opportunities. NASA's Advanced Air Mobility (AAM) project has taken a significant leap forward with a research flight test led by the Integration of Automated Systems (IAS) sub-project. This effort focused on assessing automated flight deck algorithms essential for supporting high-density Urban Air Mobility (UAM) operations. Carrying out a two-ship flight test in UAM Maturity Level 4 scenarios, the team evaluated state-of-the-art algorithms, including Hazard Perception and Avoidance (HPA) and flight path management (FPM) systems. This presentation focuses on results of pilots' physiological and subjective responses during the flight scenarios conducted. In collaboration with Lockheed Martin Advanced Technology Laboratories (ATL), we collected and analyzed live-flight biometric data using eye tracking, mobile brain imaging, and heart rate sensors. The data provides insights into pilots' workload and cognitive engagement while navigating automated tools, leveraging both biometric data and post-encounter subjective assessments. The research highlights the interaction between human operators and automated systems, contributing valuable lessons learned about gathering human data in live-flight environments. The knowledge acquired from this endeavor enhances our understanding of human factors in automated flight and informs future studies attempting to undertake similar feats.

air mobility↗