Engineering topics
Shively, Robert J.
Publications and source records attributed to Shively, Robert J..
SAM 3.0 Workshop Alliance SV Lab
Nissan is building a Seamless Autonomy Mobility (SAM) which is a cloud-based system of artificial intelligence that manages a fleet of autonomous vehicles. It pairs artificial intelligence with human intelligence in order to create a seamless solution that allows autonomous drive to be viable and marketable. They have created a Mobility Manager function which is a laborer who executed human-in-the-loop functions within SAM system. Nissan invited the HAT Lab to discuss NASA's lessons learned on remote operator support and to participate in a workshop discussion of one of their use cases. This presentation covers some of the HAT Lab's work on a human-autonomy teaming Agent and previous Reduced Crew Operations ground station development.
Crew Resource Management for Automated Teammates (CRM-A)
Crew Resource Management (CRM) is the application of human factors knowledge and skills to ensure that teams make effective use of all resources. This includes ensuring that pilots bring in opinions of other teammates and utilize their unique capabilities. CRM was originally developed 40 years ago in response to a number of airline accidents in which the crew was found to be at fault. The goal was to improve teamwork among airline cockpit crews. The notion of "team" was later expanded to include cabin crew and ground resources. CRM has also been adopted by other industries, most notably medicine. Automation research now finds itself faced with similar issues to those faced by aviation 40 years ago: how to create a more robust system by making full use of both the automation and its human operators. With advances in machine intelligence, processing speed and cheap and plentiful memory, automation has advanced to the point that it can and should be treated as a teammate to fully take advantage of its capabilities and contributions to the system. This area of research is known as Human-Autonomy Teaming (HAT). Research on HAT has identified reusable patterns that can be applied in a wide range of applications. These patterns include features such as bi-directional communication and working agreements. This paper will explore the synergies between CRM and HAT. We believe that HAT research has much to learn from CRM and that there are benefits to expanding CRM to cover automation.
Unmanned Aerial Systems (UAS) Integration into the National Airspace System (NAS)
NASA's Unmanned Aerial Systems (UAS) integration into the National Air Space (NAS) project has been working closely with the FAA and RTCA Special Committee 228 to identify and break down barriers to UAS integration. A focus of this work is on detect and avoid (DAA) technologies. A pilot has responsibility to see and avoid other aircraft and to remain "well clear," using their best judgment (Federal Aviation Regulations (FAR) Sec. 91.113). For UAS to perform this function, the see function is replaced by sensors to detect the other aircraft. Secondly, the pilot judgment of well clear has to be replaced by a mathematical expression. For Phase 1 of this effort, a well clear violation was defined if all three of these conditions are true: a) the horizontal clearance is less than 4000 ft., and b) the vertical clearance is less than 450 ft., and c) the time to loss of well clear is less than 35 seconds. This definition was developed with a great deal of community input and testing to ensure interoperability with Air Traffic Control (ATC) and pilots of manned aircraft. Appropriate guidance, alerting and displays were developed to allow UAS, with the appropriate sensors, to effectively maintain well clear. This work contributed to FAA Technical Standard Orders: TSO-C211, Detect and Avoid and TSO-C212, ATAR for Traffic Surveillance. Phase 2 of this work extends the operational environment to include the terminal area and lesser capable aircraft that might not have the payload capability to carry the RADAR defined in Phase 1. This session reports on work from Phase 1 and initial work in Phase 2.
2nd Workshop of Human-Automation Interaction Considerations for UAS Integration
NASA's Unmanned Aircraft Systems (UAS) Integration in the National Airspace System (NAS) Project (UAS-NAS) and the UAS Executive Committee (EXCOM) Science and Research Panel (SARP) invite you to attend the 2nd Workshop on Human-Automation Interaction Considerations for UAS Integration. A follow on to the workshop hosted by the National Academies of Science, Engineering and Medicine, this two-day workshop aims to tackle two critical issues for UAS integration in the NAS being addressed by NASA and the SARP: control of multiple UAS by a single, or multiple, operators (multi-UAS), and automatic collision avoidance (auto-CA). Attendees will be asked to generate real human-automation architecture and human machine interface solutions for these problems during interactive breakout sessions. Attendance is limited to select government and academia invitees only. This presentation is outlines the objectives of the workshop.
Contingency Management with Human Autonomy Teaming
Automation is playing an increasingly important role in many operations. It is often cheaper faster and more precise than human operators. However, automation is not perfect. There are many situations in which a human operator must step in. We refer to these instances as contingencies and the act of stepping in contingency management. Here we propose coupling Human Autonomy Teaming (HAT) with contingency management. We describe two aspects to HAT, bi-directional communication, and working agreements (or plays). Bi-directional communication like Crew Resource Management in traditional aviation, allows all parties to contribute to a decision. Working agreements specify roles and responsibilities. Importantly working agreements allow for the possibility of roles and responsibilities changing depending on environmental factors (e.g., situations the automation was not designed for, workload, risk, or trust). This allows for the automation to "automatically" become more autonomous as it becomes more trusted and/or it is updated to deal with a more complete set of possible situations. We present a concrete example using a prototype contingency management station one might find in a future airline operations center. Automation proposes reroutes for aircraft that encounter bad weather or are forced to divert for environmental or systems reasons. If specific conditions are met, these recommendations may be autonomously datalinked to the affected aircraft.
Multi-Operator Multi-UAV (MOMU) Control: Exploring the Influence of Sensor Tools and Playbook Task Delegation
New concepts of operations for Unmanned Aerial Vehicles (UAVs) will require a change from the current 2:1 operator to vehicle crew configuration. One particular control paradigm, largely driven by logistics, manpower, and training burdens, as well as the desire to force multiply, involves a single operator simultaneously managing multiple UAVs. This mode of operations has shown to significantly increase cognitive workload and decrease situation awareness, as operators are required to simultaneously attend to multiple sources of information. One potential way to mitigate potential drawbacks of multi-vehicle control by a single operator is to migrate to a multi-operator multi-UAV (MOMU) crew configuration, whereby M operators control N (> M) vehicles. This type of crew configuration can be organized in several ways to dynamically manage cognitive workload, match operator qualifications and skills to mission requirements, increase utilization of available assets, and thereby achieve maximum force multiplication. The present experiment examined task performance in a simulated MOMU environment and evaluated the potential benefits of sensor management aids ("Tools") as well as integrated sensor and flight automation ("Plays") compared to a fully manual condition ("Manual"). Tools support the operator by facilitating rapid understanding and management of sensor information, while the Plays support the operator by offloading/automating subtasks. Six pairs of participants were recruited for this study and tasked with sharing a pool of UAVs in order to conduct reconnaissance, surveillance, and target acquisition (RSTA) missions in adjacent Areas of Operation (AOs). Participants were given four tasks to accomplish, in order of priority: 1) prosecute High Value Targets (HVTs); 2) identify / track targets (military vehicles); 3) identify / mark civilian vehicles; and 4) respond to chat messages. Performance on the mission tasks was measured in terms of accuracy and reaction time. A composite mission score was also calculated using a payoff matrix that weighted each task according to priority. The results indicate that Playbook demonstrated better performance overall with higher accuracy rates and the highest composite score compared to Tools and Manual. The implications of these results to supporting future MOMU concepts of operations is discussed.
Human-Autonomy Teaming in a Flight Following Task
The NATO HFM-247 Working Group is creating a summary report of the group's activities on human-autonomy teaming. This chapter is a summary of our at NASA Ames work toward developing a framework for human-autonomy teaming (HAT) in aviation. The purpose of this project was to demonstrate and evaluate proposed tenets of HAT. The HAT features were derived from three tenets and were built into an automated recommender system on a ground station. These tenets include bi-directional communication, automation transparency, and operator directed interface. This study focused primarily on interactions with one piece of automation, the Autonomous Constrained Flight Planner (ACFP). The ACFP is designed to support rapid diversion decisions for commercial pilots in off-nominal situations. Much effort has gone into enhancing this tool not only in capability but also in transparency. In this study, participants used the ACFP at a ground station designed to aid dispatchers in a flight following role to reroute aircraft in situations such as inclement weather, system failures and medical emergencies. Participants performed this task both with HAT features enabled and without and provided feedback. We examined subjective and behavioral indicators of HAT collaborations using a proof-of-concept demonstration of HAT tenets. The data collected suggest potential advantages and disadvantages of HAT.
Bi-Directional Communication: A Critical Component of HAT
Known problems with automation include lack of mode awareness, automation brittleness, and risk of miscalibrated trust. Human-Autonomy Teaming (HAT) is essential for improving these problems. This presentation outlines critical components for Human-Autonomy Teaming.
Measured Response for UAS Integration into the National Airspace System
The measured response (MR) is the response time of aircraft to Air Traffic Controller (ATCo) commands and clearances. The overall MR can be broken up into several components, including the pilot verbal latencies (MR1), the time between the end of an ATCo clearance and the beginning of the pilots read back, and the execution initiation latency (MR2), the time between the end of the ATCos clearance and when the pilot begins to execute a maneuver. The MR is a crucial concern for the integration of Unmanned Aircraft Systems (UAS) into the National Airspace System (NAS) due to potentially greater latencies stemming from remote pilot communication and command execution. As a result, it is important to quantify what latencies in verbal responding and command execution are acceptable for safe and efficient operations in the NAS. The present studies begin to address these issues in a series of four simulations supported by the UAS Integration into the NAS program.
Unmanned Aircraft System (UAS) Delegation of Separation in NextGen Airspace
The purpose of this study was to determine the feasibility of unmanned aircraft systems (UAS) performing delegated separation in the national airspace system (NAS). Delegated separation is the transfer of responsibility for maintaining separation between aircraft or vehicles from air navigation service providers to the relevant pilot or flight operator. The effects of delegated separation and traffic display information level were collected through performance, workload, and situation awareness measures. The results of this study show benefits related to the use of conflict detection alerts being shown on the UAS operator's cockpit situation display (CSD), and to the use of full delegation. Overall, changing the level of separation responsibility and adding conflict detection alerts on the CSD was not found to have an adverse effect on performance as shown by the low amounts of losses of separation. The use of conflict detection alerts on the CSD and full delegation responsibilities given to the UAS operator were found to create significantly reduced workload, significantly increased situation awareness and significantly easier communications between the UAS operator and air traffic controller without significantly increasing the amount of losses of separation.
UAS Integration Into the NAS: An Examination of Baseline Compliance in the Current Airspace System
As a result of the FAA Modernization and Reform Act of 2012, Unmanned Aerial Systems (UAS) are expected to be integrated into the National Airspace System (NAS) by 2015. Several human factors challenges need to be addressed before UAS can safely and routinely fly in the NAS with manned aircraft. Perhaps the most significant challenge is for the UAS to be non-disruptive to the air traffic management system. Another human factors challenge is how to provide UAS pilots with intuitive traffic information in order to support situation awareness (SA) of their airspace environment as well as a see-and-avoid capability comparable to manned aircraft so that a UAS pilot could safely maneuver the aircraft to maintain separation and collision avoidance if necessary. A simulation experiment was conducted to examine baseline compliance of UAS operations in the current airspace system. Researchers also examined the effects of introducing a Cockpit Situation Display (CSD) into a UAS Ground Control Station (GCS) on UAS pilot performance, workload and situation awareness while flying in a positively controlled sector. Pilots were tasked with conducting a highway patrol police mission with a Medium Altitude Long Endurance (MALE) UAS in L.A. Center airspace with two mission objectives: 1) to reroute the UAS when issued new instructions from their commander, and 2) to communicate with Air Traffic Control (ATC) to negotiate flight plan changes and respond to vectoring and altitude change instructions. Objective aircraft separation data, workload ratings, SA data, and subjective ratings regarding UAS operations in the NAS were collected. Results indicate that UAS pilots were able to comply appropriately with ATC instructions. In addition, the introduction of the CSD improved pilot SA and reduced workload associated with UAS and ATC interactions.
Unmanned Aircraft Systems (UAS) Integration in the National Airspace System (NAS) Project Subcommittee Final
UAS Integration in the NAS Project overview with details from each of the subprojects. Subprojects include: Communications, Certification, Integrated Test and Evaluation, Human Systems Integration, and Separation Assurance/Sense and Avoid Interoperability.
Pre-flight risk assessment in emergency medical service (EMS) helicopters
A preflight risk assessment system (SAFE) was developed at NASA-Ames Research Center for civil EMS operations to assist pilots in making a decision objectively to accept or decline a mission. The ability of the SAFE system to predict risk profiles was examined at an EMS operator. Results of this field study showed that the usefulness of SAFE was largely dependent on the type of mission flown.
Cobra communications switch integration program
The paper describes a design modification to reduce the visual and manual workload associated with the radio selection and communications tasks in the U.S. Army AH-1 Cobra helicopter. The modification involves the integration of the radio selection and microphone actuating tasks into a single operation controlled by the transmit-intercom switch. Ground-based and flight tests were conducted to evaluate the modified configuration during twelve flight tasks. The results show that the proposed configuration performs twice as fast as the original configuration.
Pilot workload prediction
A predicting model of pilot workload is developed using a time-based algorithm, work-load values from previous research, and experimental data obtained by a group of experienced pilots on a Singer-Link Gat-1 instrument trainer with three degrees of motion (roll, pitch, and yaw). Each pilot performed three experimental flights presented in a counterbalanced order; each flight consisted of short, medium, or long cruise and initial approach segments. Results strongly suggest that pilots were more sensitive to the rate at which work was done than to the total amount of work accomplished. The result of predictions obtained with the model showed that the time-weighted average of the component work-load ratings were able to predict the obtained work-load ratings accurately.
Field study of communication and workload in police helicopters - Implications for AI cockpit design
This paper reports on the work performed by civilian helicopter crews, using audio and video recordings and a variety of workload measures (heart rate and subjective ratings) obtained in a field study of public service helicopter missions. The number and frequency of communications provided a significant source of workload. This is relevant to the design of automated cockpit systems, since many designs presuppose the use of voice I/O systems. Fluency of communications (including pauses, hesitation markers, repetitions, and false starts) furnished an early indication of the effects of fatigue. Three workload measures were correlated to identify high workload segments of flight, and to suggest alternate task allocations between crew members.