Search NASA⌕ Search

SEARCH · Search NASA

Results for “Human-In-The-Loop”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Predictive Workload Model for Air Traffic Controllers during UAM Operations

The effect of airspace factors on air traffic controller (ATC) workload has been an active area of study for almost three decades due to the importance of safety considerations necessary to design and maintain operations. Existing literature has examined several traffic-related (e.g., number of aircraft under control, loss of separation) contributors to ATC workload and proposed mathematical functions to best describe controller response. However, future air traffic continues to increase in complexity with the introduction of urban air mobility (UAM) – or the transportation of humans and cargo using electric vertical takeoff and landing (eVTOL) aircraft. UAM aims to alleviate congestion for existing ground transportation systems and improve mobility within urban centers and other high-demand locations. This shift in the traditional airspace paradigm necessitates an evolved understanding of model use and development for ATC workload prediction. This study aimed to develop an ATC workload forecasting model based on human-in-the-loop (HITL) simulation data for UAM operations at large airports. Data collected from the HITL simulation served as the training and testing data for a Long Short-Term Memory recurrent neural network and enabled time-series forecasting of ATC workload from traffic characteristics. Results demonstrated the potential of LSTM models for forecasting ATC workload 40 minutes into the future and highlighted important considerations for future development.

predictive model↗

Bone Conduction Headphone Research and Testing for xEMU Communications Applications

The new space suit being developed for exploration EVAs on the surface of the Moon and in microgravity environments is referred to as xEMU. A government reference model had been developed and has undergone extensive functional and environmental testing. This suit contains new upgrades from the current EMU on ISS, such as integrated speakers and microphones, eliminating the need for astronauts to wear a Communications Carrier Assembly (CCA) on their heads during spacewalks. However, this design approach results in speaker-to-microphone acoustic coupling and communications echo and feedback. A proposed solution to this issue is to replace the integrated open speakers with an astronaut worn bone conduction headset for audio capabilities while on EVA to receive incoming voice communications from Mission Control and other EVA or IVA crew members. This would eliminate the acoustic coupling and echo effect while being more ergonomically sound than the current CCA and leave the ear open to suit-ambient sound for situational awareness. A human-in-the-loop evaluation was performed, comparing five commercially available bone conduction headsets, to evaluate comfort, fit, and adjustability for long-duration wear. Five engineering test subjects with different head sizes were utilized to wear the headsets for six hours and provide succinct feedback and score the headsets on a variety of factors, to help determine which of the headsets performed the best and could advance to future bone conduction audio testing. Three of the headsets were well-received among the diverse group of subjects, and at least one of these will advance to further testing to be considered for future use under the xEMU helmet for exploration EVAs. Rating results and evaluation methods for this bone conduction headset evaluation will be presented.

Bridget Cavanaugh↗

Overview of an Exploratory, Real-Time, Multi-Pilot Simulation Study of Early eVTOL Operations at Non-Towered Vertiports

This paper provides a report out on an exploratory, multi-aircraft/multi-pilot, real-time simulation study conducted by NASA of early commercial powered-lift, Urban Air Mobility (UAM) operations at a non-towered vertiport. As used in this paper, vertiport refers to the primary ground and airspace elements facilitating the takeoff and landing of electric vertical takeoff and landing (eVTOL) aircraft with central emphasis on a vertipad, i.e. the physical touch-down and lift-off area and surrounding approach , departure, local pattern procedures. The study, known as the Piloted UML-2 ConOps Study (PUCS), had two high-level goals. The first goal was providing preliminary insights and observations relevant to the piloting and flight operations of early, commercial UAM operations aligned with the initial stage of the FAA’s Advanced Air Mobility (AAM) Implementation Plan and the second level NASA’s UAM Maturity Level (UML) scale. The second goal was evaluating a novel, medium-fidelity, extensible, many-pilot, real-time simulation capability known as the UAM Flyers developed by NASA. The Flyers are intended to allow rapid development, screening, evaluation, and demonstrations of potential Concepts of Operation (ConOps) for UAM flight operations and airspace management in a modular and low-cost, real-time, human-in-the-loop rapid simulation prototyping environment. For this study, ten Flyer cockpits were configured to evaluate flight operations through a non-towered vertiport with pilot interfaces and displays (external and in-cockpit) appropriate for operations under visual flight rules (VFR) and employing flight and communication procedures representative of current operations at non-towered airports. The presented results include an achieved operational tempo; durations of individual flight tasks for approaches and departures; off-nominal events and triggers; and pilot comments regarding potential procedural and technology improvements.

Urban Air Mobility↗

Assistive Detect and Avoid Technology in Urban Air Mobility Environments

The use of Assistive Detect and Avoid (Assistive DAA or ADAA) technology in Urban Air Mobility (UAM) environments poses potential benefits as well as challenges. Assistive DAA refers to the leveraged use of DAA technology, originally developed to replace see-and-avoid capabilities for remotely piloted aircraft, in onboard-piloted aircraft to augment (rather than replace) pilots’ see-and-avoid abilities and thus enhance the safety and efficiency of visual flight operations. ADAA is anticipated to be especially safety-enhancing in airspace where traffic density is high or traditional air traffic services are limited, such as in future UAM environments. ADAA may also enable higher-tempo UAM operations than with only see-and-avoid capabilities, while still maintaining acceptable levels of safety. UAM concepts under development by the FAA, NASA, and industry focus on operations moving people and cargo in urban and suburban areas using innovative technologies, operations, and aircraft, including electric vertical takeoff and landing (eVTOL) aircraft. Researchers at NASA Langley Research Center, in collaboration with FAA researchers at the William J. Hughes Technical Center in Atlantic City, NJ, have conducted a series of medium-fidelity, human-in-the-loop research simulations of potential future UAM operations and concepts in both Class C and Class B airspace environments. These simulations have included use of a Langley-developed ADAA research tool called DANTi, which enables configurable ADAA displays to be presented to pilots of simulated eVTOL aircraft participating in higher-density and higher-tempo UAM operations. Experience and observations made during testing of the NASA-developed DANTi ADAA capability in the UAM NFLITE simulation environment will be reported in this paper together with a discussion of airspace integration and regulatory topics.

Detect and Avoid↗

Overview of an Exploratory, Multi-Pilot Simulation Study of Early eVTOL Operations at Non-Towered Vertiports

This paper provides a report out on an exploratory, multi-aircraft/multi-pilot, real-time simulation study conducted by NASA of early commercial powered-lift, Urban Air Mobility (UAM) operations at a non-towered vertiport. As used in this paper, vertiport refers to the primary ground and airspace elements facilitating the takeoff and landing of electric vertical takeoff and landing (eVTOL) aircraft with central emphasis on a vertipad, i.e. the physical touch-down and lift-off area and surrounding approach , departure, local pattern procedures. The study, known as the Piloted UML-2 ConOps Study (PUCS), had two high-level goals. The first goal was providing preliminary insights and observations relevant to the piloting and flight operations of early, commercial UAM operations aligned with the initial stage of the FAA’s Advanced Air Mobility (AAM) Implementation Plan and the second level NASA’s UAM Maturity Level (UML) scale. The second goal was evaluating a novel, medium-fidelity, extensible, many-pilot, real-time simulation capability known as the UAM Flyers developed by NASA. The Flyers are intended to allow rapid development, screening, evaluation, and demonstrations of potential Concepts of Operation (ConOps) for UAM flight operations and airspace management in a modular and low-cost, real-time, human-in-the-loop rapid simulation prototyping environment. For this study, ten Flyer cockpits were configured to evaluate flight operations through a non-towered vertiport with pilot interfaces and displays (external and in-cockpit) appropriate for operations under visual flight rules (VFR) and employing flight and communication procedures representative of current operations at non-towered airports. The presented results include an achieved operational tempo; durations of individual flight tasks for approaches and departures; off-nominal events and triggers; and pilot comments regarding potential procedural and technology improvements.

Urban Air Mobility↗

Predictive Workload Model for Air Traffic Controllers during UAM Operations

The effect of airspace factors on air traffic controller (ATC) workload has been an active area of study for almost three decades due to the importance of safety considerations necessary to design and maintain operations. Existing literature has examined several traffic-related (e.g., number of aircraft under control, loss of separation) contributors to ATC workload and proposed mathematical functions to best describe controller response. However, future air traffic continues to increase in complexity with the introduction of urban air mobility (UAM) – or the transportation of humans and cargo using electric vertical takeoff and landing (eVTOL) aircraft. UAM aims to alleviate congestion for existing ground transportation systems and improve mobility within urban centers and other high-demand locations. This shift in the traditional airspace paradigm necessitates an evolved understanding of model use and development for ATC workload prediction. This study aimed to develop an ATC workload forecasting model based on human-in-the-loop (HITL) simulation data for UAM operations at large airports. Data collected from the HITL simulation served as the training and testing data for a Long Short-Term Memory recurrent neural network and enabled time-series forecasting of ATC workload from traffic characteristics. Results demonstrated the potential of LSTM models for forecasting ATC workload 40 minutes into the future and highlighted important considerations for future development.

predictive model↗

Assistive Detect and Avoid Technology in Urban Air Mobility Environments

The use of Assistive Detect and Avoid (Assistive DAA or ADAA) technology in Urban Air Mobility (UAM) environments poses potential benefits as well as challenges. Assistive DAA refers to the leveraged use of DAA technology, originally developed to replace see-and-avoid capabilities for remotely piloted aircraft, in onboard-piloted aircraft to augment (rather than replace) pilots’ see-and-avoid abilities and thus enhance the safety and efficiency of visual flight operations. ADAA is anticipated to be especially safety-enhancing in airspace where traffic density is high or traditional air traffic services are limited, such as in future UAM environments. ADAA may also enable higher-tempo UAM operations than with only see-and-avoid capabilities, while still maintaining acceptable levels of safety. UAM concepts under development by the FAA, NASA, and industry focus on operations moving people and cargo in urban and suburban areas using innovative technologies, operations, and aircraft, including electric vertical takeoff and landing (eVTOL) aircraft. Researchers at NASA Langley Research Center, in collaboration with FAA researchers at the William J. Hughes Technical Center in Atlantic City, NJ, have conducted a series of medium-fidelity, human-in-the-loop research simulations of potential future UAM operations and concepts in both Class C and Class B airspace environments. These simulations have included use of a Langley-developed ADAA research tool called DANTi, which enables configurable ADAA displays to be presented to pilots of simulated eVTOL aircraft participating in higher-density and higher-tempo UAM operations. Experience and observations made during testing of the NASA-developed DANTi ADAA capability in the UAM NFLITE simulation environment will be reported in this paper together with a discussion of airspace integration and regulatory topics.

Detect and Avoid↗

Deflection of Solar Energetic Protons by an Ion Thruster

While Galactic Cosmic Rays (GCR) represent a long-term cancer risk to deep space human missions, Solar Energetic Particles (SEP) represent a significant acute radiation hazard to human spaceflight. This work investigates wave-particle interactions caused by the incidence of a beam of energetic protons, representing a notional SEP event, onto a cloud of ions representing a notional ion thruster plume. The resulting electromagnetic ion/ion instability is developed to investigate the degree to which energetic protons can be scattered out of the initial beam. This work includes investigations of deflection rate dependences on incident SEP energy and flux, ion thruster power and orientation relative to SEP incidence, plume ion species, and background plasma. Also investigated are the spectrum and angular distribution of electromagnetic radiation resulting from the wave-particle interactions. The primary objective of this investigation is to develop a technique for active mitigation of SEP radiation threats to spacecraft and crew, by exploring the wave-particle interaction of a SEP event with the plasma plume expelled from an ion thruster. Since the cost in expended fuel restricts this technique to intermittent use during interplanetary flights, effective prediction of when a SEP event might impact the mission is still important, and the observation of conditions that might lead to SEP events, human-in-the-loop space weather analyses, and development of SEP prediction models remain essential to crew safety, spacecraft function, and mission success.

Interplanetary Space Flight↗

A Queuing Theory Approach to Pilot-Controller Coordination for m:N Operations

In recent years, attention and interest by industry and researchers has grown in a control paradigm for remotely piloted aircraft termed “m:N operations.” In an m:N operation, a team of m remote pilots in command (RIPCs) collaboratively manage the flights of N aircraft. A consequence of an m:N concept of operations is that the RPICs will have to switch attention from one aircraft to another and from one task to another. Previous research in m:N operations has focused on the workload experienced by an RPIC and their level of situation awareness on their flights. Researchers have found that RPIC workload and situation awareness are generally sensitive to increasing N, although NASA’s Multi-Vehicle (m:N) Working Group has suggested that the driver of workload/situation awareness is the number of exceptions requiring human intervention as opposed to the value of N itself. In any case, a natural antecedent of workload is task load. In this paper, queueing theory is applied to a 1:N Urban Air Mobility (UAM) air taxi operation in order to estimate pilot task load for managing radio communications with air traffic controllers (ATCs) under increasing N. An M/M/1 queueing system is used to model the RIPC’s servicing of calls and clearance requests (e.g., departure, arrival, or airspace transition) to ATC for the N aircraft. Important parameters for the queueing model are the task arrival rate and the average service time for task completion. Radio communication times from past human-in-the-loop simulation studies are used to measure service times for a 1:4 and 1:12 UAM operation and to interpolate service times for 4 < N < 12. A Monte Carlo method is then employed, using the measured and interpolated service times, to estimate arrival rate and related queueing statistics. The paper concludes by considering the estimated queuing statistics, particularly the RPIC’s utilization (i.e., proportion of time actively servicing tasks), the length of the task queue over time, and the implications for task-balanced system design.

task load↗

Pathfinding for Airspace with Autonomous Vehicles (PAAV) ACAS Xr Terminal Area Simulation Overview

This presentation provides an overview of an upcoming human-in-the-loop simulation planned by the Pathfinding for Airspace with Autonomous Vehicles (PAAV) subproject, as part of the Air Traffic Management eXploration project. The study will investigate Detect and Avoid (DAA) concepts in and around the DAA terminal area (DTA). The DTA has received little investigation in real-time, pilot-in-the-loop simulation. This study proposes to use current remotely piloted aircraft system (RPAS) pilots to investigate the effectiveness of current DTA requirements using the FAA's Airborne Collision Avoidance System X for Rotorcraft (ACAS Xr) as the DAA system under test. The findings of this stufy will be used to inform DTA and ACAS Xr standards development. This presentation will be provided to RTCA Special Committee 228 as an overview and as an opportunity to collect feedback from the larger RPAS/DAA community on the study design and key scenarios and metrics. The presentation provides a brief overview of open areas of research with ACAS Xr, the experimental design, the simulation environment and the simulation schedule.

advanced air mobility↗

Interactive Rotated Object Detection for Novel Class Detection in Remotely Sensed Imagery

In this paper we propose IRTR-DETR an Interactive and Real-Time Rotated DEtection TRansformer that extends IRTDETR to predict rotated bounding boxes. IRTR-DETR maintains the Human-In-The-Loop (HIL) workflow of IRTDETR but introduces rotation-aware heads for improved detection of objects with arbitrary orientations. Similarly to IRTDETR IRTR-DETR can be trained with a small labeled sample set in an interactive setting but we show that it can also be pretrained on related but not identical data--such as a building damage dataset--before being applied to tasks like identifying buildings under construction. We demonstrate the efficacy of our approach on the publicly available Tiny-DOTA and xBD dataset as well as two study-cases on proprietary datasets of greenhouses and houses under construction ("waffle homes"). Detecting greenhouses is highly relevant in the context of damage assessment while "waffle homes" aid understanding typical floorplans and building codes in different areas both thereby supporting population modeling emergency response and policy planning. Our method outperforms the state of the art in interactive rotated object detection on the Tiny-DOTA dataset by 5.7 percent and improves upon the non interactive RTDETR by 7.85 to 19.39 percent (depending on the number of provided samples) while maintaining its real-time efficiency.

Burges, Marvin [ORNL] (ORCID:0000000312690769)↗

HEPTAPOD: Orchestrating High Energy Physics Workflows Towards Autonomous Agency

Many workflows in high-energy-physics (HEP) stand to benefit from recent advances in transformer-based large language models (LLMs). While early applications of LLMs focused on text generation and code completion, modern LLMs now support orchestrated agency: the coordinated execution of complex, multi-step tasks through tool use, structured context, and iterative reasoning. We introduce the HEP Toolkit for Agentic Planning, Orchestration, and Deployment (HEPTAPOD), an orchestration framework designed to bring this emerging paradigm to HEP pipelines. The framework enables LLMs to interface with domain-specific tools, construct and manage simulation workflows, and assist in common utility and data analysis tasks through schema-validated operations and run-card-driven configuration. To demonstrate these capabilities, we consider a representative Beyond the Standard Model (BSM) Monte Carlo validation pipeline that spans model generation, event simulation, and downstream analysis within a unified, reproducible workflow. HEPTAPOD provides a structured and auditable layer between human researchers, LLMs, and computational infrastructure, establishing a foundation for transparent, human-in-the-loop systems.

Menzo, Tony [Alabama U.; Fermilab] (ORCID:00000002↗

Explainable AI for Multivariate Time Series Pattern Exploration: Latent Space Visual Analytics With Temporal Fusion Transformer and Variational Autoencoders in Power Grid Event Diagnosis

Detecting and analyzing complex patterns in multivariate time-series data is crucial for decision-making in urban and environmental system operations. However, challenges arise from the high dimensionality, intricate complexity, and interconnected nature of complex patterns, which hinder the understanding of their underlying physical processes. Existing AI methods often face limitations in interpretability, computational efficiency, and scalability, reducing their applicability in real-world scenarios. This paper proposes a novel visual analytics framework that integrates two generative AI models, Temporal Fusion Transformer (TFT) and Variational Autoencoders (VAEs), to reduce complex patterns into lower-dimensional latent spaces and visualize them in 2D using dimensionality reduction techniques such as PCA, t-SNE, and UMAP with DBSCAN. These visualizations, presented through coordinated and interactive views and tailored glyphs, enable intuitive exploration of complex multivariate temporal patterns, identifying patterns’ similarities and uncover their potential correlations for a better interpretability of the AI outputs. The framework is demonstrated through a case study on power grid signal data, where it identifies multi-label grid event signatures, including faults and anomalies with diverse root causes. Additionally, novel metrics and visualizations are introduced to validate the models and assess the performance, efficiency, and consistency of latent maps generated by VAE, which have been utilized in prior studies for latent space cartography and used as a benchmark in this study, and the emerging TFT architecture under various configurations. These analyses provide actionable insights for model parameter tuning and reliability improvements. Comparative results highlight that TFT achieves shorter run times and superior scalability to diverse time-series data shapes compared to VAE. This work advances fault diagnosis in multivariate time series, fostering explainable AI to support critical system operations.

Explainable AI↗

Air Traffic Controller Performance and Acceptability of Multiple UAS in a Simulated NAS Environment

Previously, we showed that air traffic controllers (ATCos) rated UAS pilot verbal response latencies as acceptable when a 1.5 s delay was added to the UAS pilot responses, but a 5 s delay was rated as mostly unacceptable. In the present study we determined whether a 1.5 s added delay in the UAS pilots' verbal communications would affect ATCos interactions with UAS and other conventional aircraft when the number and speed of the UAS were manipulated. Eight radar-certified ATCos participated in this simulation. The ATCos managed a medium altitude sector containing arrival aircraft, en route aircraft, and one to four UAS. The UAS were conducting a surveillance mission and flew at either a "slow" or "fast" speed. We measured both UAS and conventional pilots' verbal communication latencies, and obtained ATCos' acceptability ratings for these latencies. Although the UAS pilot response latencies were longer than those of conventional pilots, the ATCos rated UAS pilot verbal communication latencies to be as acceptable as those of conventional pilots. Because the overall traffic load within the sector was held constant, ATCos only performed slightly worse when multiple UAS were in their sector compared to when only one UAS was in the sector. Implications of these findings for UAS integration in the NAS are discussed.

measured response↗

Subject Matter Expert Evaluation of Multi-Flight Common Route Advisories

Traffic flow management seeks to balance the demand for National Airspace System (NAS) flight resources, such as airspace and airports, with the available supply. When forecasted weather blocks nominal air traffic routes, traffic managers must re-route affected flights for weather avoidance. Depending on the nature and scope of the weather, traffic managers may use pre-coordinated re-routes such as Playbook Routes or Coded Departure Routes, or may design ad hoc local re-routes. The routes of affected flights are modified accordingly. These weather avoidance routes will, of course, be less efficient than the nominal routes due to increased flight time and fuel burn. In current traffic management operations, the transition into a weather avoidance re-routing initiative is typically implemented more aggressively than the transition out of that initiative after the weather has dissipated or moved away. For example, strategic large-scale Playbook re-routes are sometimes left in place (as initially implemented) for many hours before being lifted entirely when the weather dissipates. There is an opportunity to periodically modify the re-routing plan as weather evolves, thereby attenuating its adverse impact on flight time and fuel consumption; this is called delay recovery. Multi-Flight Common Routes (MFCR) is a NASA-developed operational concept and associated decision support tool for delay recovery, designed to assist traffic managers to efficiently update weather avoidance traffic routes after the original re-routes have become stale due to subsequent evolution of the convective weather system. MFCR groups multiple flights to reduce the number of advisories that the traffic manager needs to evaluate, and also merges these flights on a common route segment to provide an orderly flow of re-routed traffic. The advisory is presented to the appropriate traffic manager who evaluates it and has the option to modify it using MFCRs graphical user interface. If the traffic manager finds the advisory to be operationally appropriate, he or she would coordinate with the Area Supervisor(s) of the sectors that currently control the flights in the advisory. When the traffic manager accepts the MFCR advisory via the user interface, the corresponding flight plan amendments would be sent to the displays of the appropriate sector controllers, using the Airborne Re-Routing (ABRR) capability which is scheduled for nationwide operation in 2017. The sector controllers would then offer this time-saving route modification to the pilots of the affected flights via datalink (or voice), and implement the corresponding flight plan amendment if the pilots accept it. MFCR is implemented as an application in the software environment of the Future Air traffic management Concepts Evaluation Tool (FACET). This paper focuses on an initial subject matter expert (SME) evaluation of MFCR. The evaluation covers MFCRs operational concept, algorithm, and user interface.

Human-in-the-loop Evaluation↗

Subject Matter Expert Evaluation of Multi-Flight Common Route Advisories

Traffic flow management seeks to balance the demand for National Airspace System (NAS) flight resources, such as airspace and airports, with the available supply. When forecasted weather blocks nominal air traffic routes, traffic managers must re-route affected flights for weather avoidance. Depending on the nature and scope of the weather, traffic managers may use pre-coordinated re-routes such as Playbook Routes or Coded Departure Routes, or may design ad hoc local re-routes. The routes of affected flights are modified accordingly. These weather avoidance routes will, of course, be less efficient than the nominal routes due to increased flight time and fuel burn. In current traffic management operations, the transition into a weather avoidance re-routing initiative is typically implemented more aggressively than the transition out of that initiative after the weather has dissipated or moved away. For example, strategic large-scale Playbook re-routes are sometimes left in place (as initially implemented) for many hours before being lifted entirely when the weather dissipates. There is an opportunity to periodically modify the re-routing plan as weather evolves, thereby attenuating its adverse impact on flight time and fuel consumption; this is called delay recovery. Multi-Flight Common Routes (MFCR) is a NASA-developed operational concept and associated decision support tool for delay recovery, designed to assist traffic managers to efficiently update weather avoidance traffic routes after the original re-routes have become stale due to subsequent evolution of the convective weather system. MFCR groups multiple flights to reduce the number of advisories that the traffic manager needs to evaluate, and also merges these flights on a common route segment to provide an orderly flow of re-routed traffic. The advisory is presented to the appropriate traffic manager who evaluates it and has the option to modify it using MFCRs graphical user interface. If the traffic manager finds the advisory to be operationally appropriate, he or she would coordinate with the Area Supervisor(s) of the sectors that currently control the flights in the advisory. When the traffic manager accepts the MFCR advisory via the user interface, the corresponding flight plan amendments would be sent to the displays of the appropriate sector controllers, using the Airborne Re-Routing (ABRR) capability which is scheduled for nationwide operation in 2017. The sector controllers would then offer this time-saving route modification to the pilots of the affected flights via datalink (or voice), and implement the corresponding flight plan amendment if the pilots accept it. MFCR is implemented as an application in the software environment of the Future Air traffic management Concepts Evaluation Tool (FACET). This paper focuses on an initial subject matter expert (SME) evaluation of MFCR. The evaluation covers MFCRs operational concept, algorithm, and user interface.

Traffic flow management↗