HaMLET: Human and Machine Learning Effective Teaming
Explore the source record for details and available documents.
Wisniewski, Kyra Lynn↗
SEARCH · Search NASA
Results for “Human machine teaming”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Defining Collaborative Control Interactions Using Systems Theory
Human teams collaborate by establishing roles, changing functional authorities, maintaining team cognition, coordinating, and mutually helping one another close control loops. These complex inter-actions are inspiring novel concepts to improve human-machine and multi-machine collaboration. However, these new systems face engineering gaps in modeling, analysis, design, and assurance. As such, few have been fielded in safety-critical domains like aerospace. To analyze safety, this paper introduces a system-theoretic framework to describe interactions that are—or are planned to be—used in multi-controller systems. It outlines a taxonomy of seven structural dimensions that influence controller interactions and nine dynamics observed in collaborative control that are defined using Systems Theory. An analyzed set of 101 controller interactions in aerospace systems demonstrates how to apply the framework and that designers are trying to create more sophisticated systems. This framework provides the foundation to extend system-theoretic hazard analysis techniques to systematically find collabora-tive-control causal factors.
Human machine teaming↗
On the Moral Hazard of Autonomy
This paper describes the concept of moral hazard as applied to technologies that incorporate automation and autonomy. Moral hazard is said to exist when a party to a transaction feels more comfortable taking undue risks because another party will bear the costs if things go badly. As opposed to regular physical hazards, a moral hazard comes from within a person. In this paper, we reveal two categories of moral hazards related to autonomy. The first category of moral hazard occurs when the owner of the autonomy introduces an autonomous system without accepting the full responsibility for improper operation thereby shifting the risks from one party to another party. This category of moral hazard is similar to moral hazards experienced in other industries and can often be addressed through appropriate policy and establishing liability for irresponsible behavior. The issue becomes more complicated in cases where the operator of the autonomy may not have a full understanding of the system behavior. In the second category of moral hazard, risks are shifted from people to autonomy. In this category, the humans in proximity to the autonomous system begin to trust its behavior. Their behavior may change in that they may believe they are more insulated from harm and subsequently exhibit more risky behavior towards increasingly autonomous technologies. Mitigating this type of moral hazard may require the autonomy to possess certain design features to discourage this type of harmful human behavior so that humans do not suffer needlessly in their interactions with autonomous systems by placing inappropriate trust where that trust is neither warranted nor deserved.
Cybernetics↗
NASA HRI At the Intersection of Unmanned & Autonomous Systems
Explore the source record for details and available documents.
human robot interaction↗
Resilience Modeling in Complex Engineered Systems with Human-Machine Interactions
In recent times, there has been a growing interest in resilience-based design. Resilience-based design operates on the concept that failures and unexpected events will happen, and when they occur, complex engineered systems should be able to operate within acceptable bounds and recover reasonably. Humans can contribute to the resilience of a system by quickly detecting unforeseen events and taking corrective measures. To this effect, researchers have proposed guidelines and design approaches that can help promote human-system resilience. However, there is no early design stage tool to validate if a system is indeed resilient after applying these guidelines and design methods. In this research, we integrate the Human Error and Functional Failure Reasoning (HEFFR) framework into the fmdtools toolkit to enable designers to model the combined (machine, human, and joint) failures, including their propagation and dynamic effects, during early design stages. This integrated tool also allows designers to model the effects of performance shaping factors, team dynamics, and human-machine interactions in systems of systems. A demonstrative example of a remotely operated rover is explored to demonstrate how this approach can be applied to understand resilience in complex engineered systems with human interactions.
Lukman Irshad↗
Examining the Changing Roles and Responsibilities of Humans in Envisioned Future In-Time Aviation Safety Management Systems
Advances in technology are enabling new concepts of operations that will trans-form aviation including increasingly autonomous capabilities to handle evolving complex dynamic ecosystems like those associated with Advanced Aerial Mobility. A major challenge is how to ensure today’s safety levels are maintained as the system scales for rapid detection and timely mitigation of safety issues. NASA has developed a concept of operation for In-Time Aviation Safety Management Systems (IASMS) that represents a system-of-system perspective on interconnected capabilities needed to proactively reduce risk in complex operational environments where unknown hazards may exist. As a result, NASA research priorities include under-standing how the balance between humans and automation changes in such envisioned systems, which may lead to novel human-machine interaction paradigms and human-autonomy teaming for informed contingency management.
Lawrence Prinzel↗
Autonomous, Context-Sensitive, Task Management Systems and Decision Support Tools I: Human-Autonomy Teaming Fundamentals and State of the Art
Recent advances in artificial intelligence, machine learning, data mining and extraction, and especially in sensor technology have resulted in the availability of a vast amount of digital data and information and the development of advanced automated reasoners. This creates the opportunity for the development of a robust dynamic task manager and decision support tool that is context sensitive and integrates information from a wide array of on-board and off aircraft sourcesa tool that monitors systems and the overall flight situation, anticipates information needs, prioritizes tasks appropriately, keeps pilots well informed, and is nimble and able to adapt to changing circumstances. This is the first of two companion reports exploring issues associated with autonomous, context-sensitive, task management and decision support tools. In the first report, we explore fundamental issues associated with the development of an integrated, dynamic, flight information and automation management system. We discuss human factors issues pertaining to information automation and review the current state of the art of pilot information management and decision support tools. We also explore how effective human-human team behavior and expectations could be extended to teams involving humans and automation or autonomous systems.
context-sensitive↗
Enabling Advanced Air Mobility Operations through Appropriate Trust in Human-Autonomy Teaming: Foundational Research Approaches and Applications
Emerging Advanced Air Mobility(AAM)operations will be enabled by increasingly autonomous systems, requiring technologies to take on more responsibilities and fundamentally altering traditional human-automation interaction paradigms. The growing reliance on higher levels of automation will necessitate research to identify capabilities and principles that facilitate humans and machines working and thinking better together, i.e., human-autonomy teaming (HAT). Trust is an inherent requirement in effective teams because when members work interdependently, those agents (human, automation) must be willing to accept a level of risk to rely upon each other to reach goals and contribute to team tasks. This work provides an initial approach to enabling AAM operations through appropriate trust within HAT. The main contributions of this approach resides in connecting the construct of trust to mental models. Using the outlined mental model approach, we propose novel HAT strategies, such as Adaptive Trust Calibration, and preview planned research activities derived from this approach. Additionally, we propose several practical applications that can currently be employed by AAM development communities.
Eric T Chancey↗
Multimodal Neuroelectric Interface Development
This project aims to improve performance of NASA missions by developing multimodal neuroelectric technologies for augmented human-system interaction. Neuroelectric technologies will add completely new modes of interaction that operate in parallel with keyboards, speech, or other manual controls, thereby increasing the bandwidth of human-system interaction. We recently demonstrated the feasibility of real-time electromyographic (EMG) pattern recognition for a direct neuroelectric human-computer interface. We recorded EMG signals from an elastic sleeve with dry electrodes, while a human subject performed a range of discrete gestures. A machine-teaming algorithm was trained to recognize the EMG patterns associated with the gestures and map them to control signals. Successful applications now include piloting two Class 4 aircraft simulations (F-15 and 757) and entering data with a "virtual" numeric keyboard. Current research focuses on on-line adaptation of EMG sensing and processing and recognition of continuous gestures. We are also extending this on-line pattern recognition methodology to electroencephalographic (EEG) signals. This will allow us to bypass muscle activity and draw control signals directly from the human brain. Our system can reliably detect P-rhythm (a periodic EEG signal from motor cortex in the 10 Hz range) with a lightweight headset containing saline-soaked sponge electrodes. The data show that EEG p-rhythm can be modulated by real and imaginary motions. Current research focuses on using biofeedback to train of human subjects to modulate EEG rhythms on demand, and to examine interactions of EEG-based control with EMG-based and manual control. Viewgraphs on these neuroelectric technologies are also included.
Trejo, Leonard J.↗
Control Room of the Future Testbed Workshop – After-Action Report
The U.S. Department of Energy’s Office of Electricity is supporting a one-year, multi-laboratory effort to define the needs and requirements for a Control Room of the Future testbed, or CROFT. The effort responds to increasing grid complexity driven by large new loads, dynamic generation resources, and the growing adoption of advanced technologies and tools, including artificial intelligence (AI) and machine learning (ML). To support safe, secure, and effective grid modernization, CROFT will focus on how emerging technologies and tools can be rigorously evaluated in realistic operational settings, with attention to human-machine interaction, cognitive load, and workforce readiness. The project team includes Argonne National Laboratory, Idaho National Laboratory, National Laboratory of the Rockies, and Pacific Northwest National Laboratory. As part of the scoping effort, the team conducted two industry-focused workshops: one at DTECH on February 5, 2026, informed by prior industry interviews, and a second on May 4, 2026, adjacent to IEEE T&D. These engagements brought together utilities, vendors, consultants, national laboratories, academia, and government stakeholders to identify and prioritize use cases, barriers, validation needs, data-sharing constraints, and near- and longer-term requirements. This feedback will directly inform CROFT’s architecture and research focus areas, ensuring the testbed is grounded in real-world operational needs and designed to evaluate emerging technologies and tools in realistic control-room environments.
artificial intelligence↗
Designing for Advanced Aerial Mobility: Human-Autonomy Teaming and In-Time System-Wide Safety Assurance
The continued growth of aviation shall require new innovative technologies and operational concepts to meet the ever-increasing demands on air transportation. The NASA Advanced Air Mobility (AAM) project focuses on emerging aviation markets, such as Urban Air Mobility (UAM). UAM is defined as “...a safe and efficient system for air passenger and cargo transportation within an urban area. It is inclusive of small package delivery and other urban unmanned aerial system services and supports a mix of onboard/ground-piloted and increasingly autonomous operations” ([1]). The AAM project emphasizes technology development and validating system-level concepts and solutions in coordination with other NASA Aeronautics Research Mission Directorate (ARMD) projects to enable UAM metro- and micro-plex vertiport and airspace concepts of operations. The NASA AAM research portfolio includes the concepts of Remote Supervisor-in-Command (RSC) and Fleet and Airspace Manager (FAM) as possible human roles for consumer fleet providers. NASA research in RSC is focused on development of guidelines and standards for remote pilots/operators passively and actively controlling a large fleet of autonomous aircraft. For FAM, flight and ground system concepts and technologies to enable high density homogeneous operations at increased scale from vertiport(s), and coordination with other humans in the systems (e.g., UAM urban airspace manager, Air Traffic Control) are key research areas. The envisioned UAM operations are posited to require autonomous systems to enable functions ranging from fleet and resource management to vehicle control. Although automation has become increasingly sophisticated and ubiquitous in civil aviation, autonomy represents a significant evolution in automation, which has generally been limited in functional scope and capability. As autonomy takes on increasing responsibilities, humans and machines will be required to work together in new and different ways [2], rather than traditional design approaches focused on how machines (i.e., autonomy) can do the work of people. The emerging field of human-autonomy teaming (HAT) represents a comprehensive and prioritized research-driven approach to enable the success of future emerging aviation market applications through capabilities and principles that facilitate humans and machine working and thinking better together. The NASA Transformational Tools and Technologies (TTT) Autonomous System (AS) Sub-project was created to assist with the transition into higher levels of autonomy to enable new modes of air transportation, such as UAM. TTT-AS has identified HAT as a key research need to enable UAM while maintaining today’s ultra-safe aviation system safety levels. The latter challenge has been taken up by the NASA System-Wide Safety (SWS) Project, which recognizes that aviation safety, as it evolves, shall require new ways of thinking about safety to include integration of a wide-range of existing and new safety systems and practices, enhanced tools and technologies, increased access to data and data fusion, improved data analysis capabilities, enhanced in-time risk monitoring and detection, hazard prioritization and mitigation, safety assurance decision-support, and in-time integrated system analytics [3].The operational concept of UAM represents a variety of work that has been termed, “work-as-imagined” to characterize the idea that how people think that work is done and how work is actually done are often not the same [4]. To ensure design success and system safety, looking at “work-as-done” provides a comparative approach toward UAM concept and technology design through examination of corresponding analogs found today in aviation (e.g., on-demand operations) and other transportation domains (e.g., port operations). The paper shall discuss various alternative applications with specific focus on airline operation center (AOC) operations, and unmanned aerial system (UAS) command-and-control to inform scaled-versions of FAM and RSC, respectively, and with consideration of the national airspace system contextual environment. The tenets and principles of the HAT field and current NASA research efforts under the TTT-AS sub-project shall also be described. Finally, the SWS sub-project efforts to develop In-Time System-Wide Safety Assurance (ISSA) and In-Time Safety Management Systems (IASMS) are discussed in terms of how “in-time” safety assurance may be conceptualized for the on-demand mobility air taxi “work-as-imagined” operational concept [5]. As part of this effort, concepts from the emerging field of resilience engineering, are being studied. Traditional approaches to aviation safety have focused on what can go wrong and how to prevent it. Another approach to thinking about system safety should reflect not only “avoiding things that go wrong” (protective safety) but also “ensuring that things go right” (productive safety), that enables a system to exhibit the resilient performance [6] necessary for the success of the future aviation system emerging concepts of operations. The paper shall describe efforts focused on how productive safety and resilience may enable a more complete approach to system safety thinking and design of ISSA and IASMS for UAM. Future directions and research needs shall also be discussed.
resilience↗
Workload Measurement in Human Autonomy Teaming: How and Why?
This is an invited talk on autonomy and workload for an AFRL Blue Sky workshop sponsored by the Florida Institute for Human Machine Studies. The presentation reviews various metrics of workload and how to move forward with measuring workload in a human-autonomy teaming environment.
autonomy↗
System-Theoretic Analysis of Unsafe Collaborative Control in Teaming Systems
The interactions that occur in human-teaming are inspiring novel aerospace designs aimed at improving how humans and machines, or multiple machines, work together. Unfortunately, current Systems Engineering processes are ill-equipped to handle these complex relationships and are unable to design and assure the safety for these systems. To close part of this gap, this paper introduces a novel system-theoretic analytical process to identify unsafe collaborative control actions. It is part of a broader set of techniques that extend the state-of-the-art in hazard analysis, System Theoretic Process Analysis (STPA), to systematically address collaboration. The method rigorously expresses the different ways multiple commands may be unsafe together. Using Systems Theory, it employs abstraction to manage the combinatorial complexity in enumerating control contributions from multiple collaborating components. An algorithm integrates these concepts into an end-to-end process and is supported by automation to enumerate, refine, prune, and prioritize unsafe combinations of control actions. The output of the method feeds the specification of system requirements to implement safety-guided design starting early in concept development. The process is demonstrated on a manned-unmanned aircraft teaming case study and finds new causal factors that were not previously found in a past hazard analysis of the same system.
System Safety↗
Modeling Leadership Styles in Human-Robot Team Dynamics
The recent proliferation of robotic systems in our society has placed questions regarding interaction between humans and intelligent machines at the forefront of robotics research. In response, our research attempts to understand the context in which particular types of interaction optimize efficiency in tasks undertaken by human-robot teams. It is our conjecture that applying previous research results regarding leadership paradigms in human organizations will lead us to a greater understanding of the human-robot interaction space. In doing so, we adapt four leadership styles prevalent in human organizations to human-robot teams. By noting which leadership style is more appropriately suited to what situation, as given by previous research, a mapping is created between the adapted leadership styles and human-robot interaction scenarios-a mapping which will presumably maximize efficiency in task completion for a human-robot team. In this research we test this mapping with two adapted leadership styles: directive and transactional. For testing, we have taken a virtual 3D interface and integrated it with a genetic algorithm for use in &le-operation of a physical robot. By developing team efficiency metrics, we can determine whether this mapping indeed prescribes interaction styles that will maximize efficiency in the teleoperation of a robot.
Cruz, Gerardo E.↗
Evaluation of Technology Concepts for Traffic Data Management and Relevant Audio for Datalink in Commercial Airline Flight Decks
Datalink is currently operational for departure clearances and in oceanic environments and is currently being tested in high altitude domestic enroute airspace. Interaction with even simple datalink clearances may create more workload for flight crews than the voice system they replace if not carefully designed. Datalink may also introduce additional complexity for flight crews with hundreds of uplink messages now defined for use. Finally, flight crews may lose airspace awareness and operationally relevant information that they normally pickup from Air Traffic Control (ATC) voice communications with other aircraft (i.e., “party-line” transmissions). Once again, automation may be poised to increase workload on the flight deck for incremental benefit. Datalink implementation to support future air traffic management concepts needs to be carefully considered, understanding human communication norms and especially, the change from voice- to text-based communications modality and its effect on pilot workload and situation awareness. Increasingly autonomous systems, where autonomy is designed to support human-autonomy teaming, may be suited to solve these issues. NASA is conducting research and development of increasingly autonomous systems, utilizing machine-learning algorithms seamlessly integrated with humans whereby task performance of the combined system is significantly greater than the individual components. Increasingly autonomous systems offer the potential for significantly improved levels of performance and safety that are superior to either human or automation alone. Two increasingly autonomous systems concepts - a traffic data manager and a conversational co-pilot - were developed to intelligently address the datalink issues in a complex, future state environment with significant levels of traffic. The system was tested for suitability of datalink usage for terminal airspace. The traffic data manager allowed for automated declutter of the Automatic Dependent Surveillance-Broadcast (ADS-B) display. The system determined relevant traffic for display based on machine learning algorithms trained by experienced human pilot behaviors. The conversational co-pilot provided relevant audio air traffic control messages based on context and proximity to ownship. Both systems made use of the connected aircraft concepts to provide intelligent context to determine relevancy above and beyond proximity to ownship. A human-in-the-loop test was conducted in NASA Langley Research Center’s Integration Flight Deck B-737-800 simulator to evaluate the traffic data manager and the conversational co-pilot. Twelve airline crews flew various normal and non-normal procedures and their actions and performance were recorded in response to the procedural events. This paper details the flight crew performance and evaluation during the events.
Etherington, Timothy↗
Machine Learning for Predicting Team Functioning in HERA Missions
Team functioning is integral to success in future long term space exploration missions. Proactively detecting declines in team functioning can mitigate conflict and ensure mission success. This project developed a speech-based artificial intelligence (AI) system that unobtrusively predicts degradation in team functioning, including performance and cohesion, in the Human Exploration Research Analog (HERA) Campaigns 4 and 5. The AI system conducted automated analysis of the prosodic (tone of voice) and linguistic (language content) components of speech, modeling interpersonal dynamics at both the turn-taking and day-wide levels. We investigated team functioning via observing structured interactions (i.e., multi-mission space exploration vehicle-extra vehicular activity [MMSEV-EVA], team interaction battery [TIB]) and unstructured interactions before the MMSEV-EVA task. We developed machine learning models to predict team functioning (objective task accuracy, self reported team efficacy and self reported team cohesion) by analyzing OpenSmile acoustic features, linguistic descriptors extracted via the linguistic inquiry and word count (LIWC) dictionary, and semantic embeddings. In the TIB, static models using logistic regression and random forests were not able to predict task accuracy, but predicted team efficacy and cohesion during both the decision making and relational tasks to a moderate level (60-70%). Majority voting on the individual turns to predict day long team efficacy further increased accuracies (70-80%). Finally, long short-term memory (LSTM) models showed the best performance across all variables (80-91%), including task performance. In the MMSEV-EVA, static models achieved an accuracy of 60% with majority voting, which increased to 80% through the incorporation of mission day as a variable, accounting for the learning effect. A key finding across both tasks was the "team-dependent" nature of these interactions; models achieved much higher accuracy when trained on prior days of the same team's data rather than attempting to generalize across entirely different teams, with even 1-2 days of prior data per team achieving 5-15% improvement over team-independent models. In addition, the incorporation of pre-task data from the same team also improves model performance, e.g., incorporating data from the decision-making task of the TIB, which preceded the relational task, improved the prediction of team efficacy and cohesion during the latter. We compared model performance when trained on machine-generated data compared to data that had been further corrected by human annotators. Overall, models trained on human-corrected data exhibited a modest improvement in performance, particularly when acoustic features were used. We found no significant correlation between word error rate (WER) and model accuracy (r(55) = -0.08, p = 0.51), but model’s accuracy was significantly higher for medium/high quality transcription (0.74 (SD = 0.48)) compared to the low-quality group (0.64 (SD = 0.36)) (t(63)=2.82, p = 0.006). Based on these, several design recommendation emerge, that could inform Standards at NASA. Models predicting team functioning should incorporate at least one to two days of historical interaction data, include brief pre-task discussions, and explicitly model temporal learning effects, especially for longer operational tasks. Minimum quality standards for automated speech-processing pipelines are needed, given the performance gains observed with manually corrected acoustic data. Finally, systems should leverage both acoustic features and language embeddings in complementary ways, with modality choices and fusion strategies tailored to mission context, task demands, and data quality requirements.
Shrivatsa Mishra↗
Exploring Multimodal Interactions in Human-Autonomy Teaming Using a Natural User Interface
The creation of a multimodal, natural user interface to facilitate multi-agent interaction is essential to establishing trust among human and machine teammates in multi-agent systems. Trust is being researched, along with trustworthiness, as a path to certification of autonomous systems by the Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) project at NASA. The Autonomous Mission Experimental Logistics Interactive Assistant (AMELIA) is a natural user interface that enables multimodal interaction and is designed for rapid mission planning. AMELIA is an intelligent system that considers the user’s preferred communication strategies, as well as the time-critical aspect of the multi-agent system decision-making process. Twenty-four participants planned a multi-agent search and rescue mission, with the aid of an intelligent assistant. The results show that while the combined use of touch and speech was faster than speech alone, the single modality, touch, was still the most efficient. Future research should investigate additional input technologies.
Lisa R Le Vie↗
Exploring Multimodal Interactions in Human-Autonomy Teaming Using a Natural User Interface
The creation of a multimodal, natural user interface to facilitate multi-agent interaction is essential to establishing trust among human and machine teammates in multi-agent systems. Trust is being researched, along with trustworthiness, as a path to certification of autonomous systems by the Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) project at NASA. The Autonomous Mission Experimental Logistics Interactive Assistant (AMELIA) is a natural user interface that enables multimodal interaction and is designed for rapid mission planning. AMELIA is an intelligent system that considers the user’s preferred communication strategies, as well as the time-critical aspect of the multi-agent system decision-making process. Twenty-four participants planned a multi-agent search and rescue mission, with the aid of an intelligent assistant. The results show that while the combined use of touch and speech was faster than speech alone, the single modality, touch, was still the most efficient. Future research should investigate additional input technologies.
Lisa Renee Le Vie↗