Search NASA⌕ Search

SEARCH · Search NASA

Results for “Human-In-The-Loop”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

Overview of an Exploratory, Multi-Pilot Simulation Study of Early eVTOL Operations at Non-Towered Vertiports

This paper provides a report out on an exploratory, multi-aircraft/multi-pilot, real-time simulation study conducted by NASA of early commercial powered-lift, Urban Air Mobility (UAM) operations at a non-towered vertiport. As used in this paper, vertiport refers to the primary ground and airspace elements facilitating the takeoff and landing of electric vertical takeoff and landing (eVTOL) aircraft with central emphasis on a vertipad, i.e. the physical touch-down and lift-off area and surrounding approach , departure, local pattern procedures. The study, known as the Piloted UML-2 ConOps Study (PUCS), had two high-level goals. The first goal was providing preliminary insights and observations relevant to the piloting and flight operations of early, commercial UAM operations aligned with the initial stage of the FAA’s Advanced Air Mobility (AAM) Implementation Plan and the second level NASA’s UAM Maturity Level (UML) scale. The second goal was evaluating a novel, medium-fidelity, extensible, many-pilot, real-time simulation capability known as the UAM Flyers developed by NASA. The Flyers are intended to allow rapid development, screening, evaluation, and demonstrations of potential Concepts of Operation (ConOps) for UAM flight operations and airspace management in a modular and low-cost, real-time, human-in-the-loop rapid simulation prototyping environment. For this study, ten Flyer cockpits were configured to evaluate flight operations through a non-towered vertiport with pilot interfaces and displays (external and in-cockpit) appropriate for operations under visual flight rules (VFR) and employing flight and communication procedures representative of current operations at non-towered airports. The presented results include an achieved operational tempo; durations of individual flight tasks for approaches and departures; off-nominal events and triggers; and pilot comments regarding potential procedural and technology improvements.

Urban Air Mobility↗

Predictive Workload Model for Air Traffic Controllers during UAM Operations

The effect of airspace factors on air traffic controller (ATC) workload has been an active area of study for almost three decades due to the importance of safety considerations necessary to design and maintain operations. Existing literature has examined several traffic-related (e.g., number of aircraft under control, loss of separation) contributors to ATC workload and proposed mathematical functions to best describe controller response. However, future air traffic continues to increase in complexity with the introduction of urban air mobility (UAM) – or the transportation of humans and cargo using electric vertical takeoff and landing (eVTOL) aircraft. UAM aims to alleviate congestion for existing ground transportation systems and improve mobility within urban centers and other high-demand locations. This shift in the traditional airspace paradigm necessitates an evolved understanding of model use and development for ATC workload prediction. This study aimed to develop an ATC workload forecasting model based on human-in-the-loop (HITL) simulation data for UAM operations at large airports. Data collected from the HITL simulation served as the training and testing data for a Long Short-Term Memory recurrent neural network and enabled time-series forecasting of ATC workload from traffic characteristics. Results demonstrated the potential of LSTM models for forecasting ATC workload 40 minutes into the future and highlighted important considerations for future development.

predictive model↗

Assistive Detect and Avoid Technology in Urban Air Mobility Environments

The use of Assistive Detect and Avoid (Assistive DAA or ADAA) technology in Urban Air Mobility (UAM) environments poses potential benefits as well as challenges. Assistive DAA refers to the leveraged use of DAA technology, originally developed to replace see-and-avoid capabilities for remotely piloted aircraft, in onboard-piloted aircraft to augment (rather than replace) pilots’ see-and-avoid abilities and thus enhance the safety and efficiency of visual flight operations. ADAA is anticipated to be especially safety-enhancing in airspace where traffic density is high or traditional air traffic services are limited, such as in future UAM environments. ADAA may also enable higher-tempo UAM operations than with only see-and-avoid capabilities, while still maintaining acceptable levels of safety. UAM concepts under development by the FAA, NASA, and industry focus on operations moving people and cargo in urban and suburban areas using innovative technologies, operations, and aircraft, including electric vertical takeoff and landing (eVTOL) aircraft. Researchers at NASA Langley Research Center, in collaboration with FAA researchers at the William J. Hughes Technical Center in Atlantic City, NJ, have conducted a series of medium-fidelity, human-in-the-loop research simulations of potential future UAM operations and concepts in both Class C and Class B airspace environments. These simulations have included use of a Langley-developed ADAA research tool called DANTi, which enables configurable ADAA displays to be presented to pilots of simulated eVTOL aircraft participating in higher-density and higher-tempo UAM operations. Experience and observations made during testing of the NASA-developed DANTi ADAA capability in the UAM NFLITE simulation environment will be reported in this paper together with a discussion of airspace integration and regulatory topics.

Detect and Avoid↗

Deflection of Solar Energetic Protons by an Ion Thruster

While Galactic Cosmic Rays (GCR) represent a long-term cancer risk to deep space human missions, Solar Energetic Particles (SEP) represent a significant acute radiation hazard to human spaceflight. This work investigates wave-particle interactions caused by the incidence of a beam of energetic protons, representing a notional SEP event, onto a cloud of ions representing a notional ion thruster plume. The resulting electromagnetic ion/ion instability is developed to investigate the degree to which energetic protons can be scattered out of the initial beam. This work includes investigations of deflection rate dependences on incident SEP energy and flux, ion thruster power and orientation relative to SEP incidence, plume ion species, and background plasma. Also investigated are the spectrum and angular distribution of electromagnetic radiation resulting from the wave-particle interactions. The primary objective of this investigation is to develop a technique for active mitigation of SEP radiation threats to spacecraft and crew, by exploring the wave-particle interaction of a SEP event with the plasma plume expelled from an ion thruster. Since the cost in expended fuel restricts this technique to intermittent use during interplanetary flights, effective prediction of when a SEP event might impact the mission is still important, and the observation of conditions that might lead to SEP events, human-in-the-loop space weather analyses, and development of SEP prediction models remain essential to crew safety, spacecraft function, and mission success.

Interplanetary Space Flight↗

A Queuing Theory Approach to Pilot-Controller Coordination for m:N Operations

In recent years, attention and interest by industry and researchers has grown in a control paradigm for remotely piloted aircraft termed “m:N operations.” In an m:N operation, a team of m remote pilots in command (RIPCs) collaboratively manage the flights of N aircraft. A consequence of an m:N concept of operations is that the RPICs will have to switch attention from one aircraft to another and from one task to another. Previous research in m:N operations has focused on the workload experienced by an RPIC and their level of situation awareness on their flights. Researchers have found that RPIC workload and situation awareness are generally sensitive to increasing N, although NASA’s Multi-Vehicle (m:N) Working Group has suggested that the driver of workload/situation awareness is the number of exceptions requiring human intervention as opposed to the value of N itself. In any case, a natural antecedent of workload is task load. In this paper, queueing theory is applied to a 1:N Urban Air Mobility (UAM) air taxi operation in order to estimate pilot task load for managing radio communications with air traffic controllers (ATCs) under increasing N. An M/M/1 queueing system is used to model the RIPC’s servicing of calls and clearance requests (e.g., departure, arrival, or airspace transition) to ATC for the N aircraft. Important parameters for the queueing model are the task arrival rate and the average service time for task completion. Radio communication times from past human-in-the-loop simulation studies are used to measure service times for a 1:4 and 1:12 UAM operation and to interpolate service times for 4 < N < 12. A Monte Carlo method is then employed, using the measured and interpolated service times, to estimate arrival rate and related queueing statistics. The paper concludes by considering the estimated queuing statistics, particularly the RPIC’s utilization (i.e., proportion of time actively servicing tasks), the length of the task queue over time, and the implications for task-balanced system design.

task load↗

Pathfinding for Airspace with Autonomous Vehicles (PAAV) ACAS Xr Terminal Area Simulation Overview

This presentation provides an overview of an upcoming human-in-the-loop simulation planned by the Pathfinding for Airspace with Autonomous Vehicles (PAAV) subproject, as part of the Air Traffic Management eXploration project. The study will investigate Detect and Avoid (DAA) concepts in and around the DAA terminal area (DTA). The DTA has received little investigation in real-time, pilot-in-the-loop simulation. This study proposes to use current remotely piloted aircraft system (RPAS) pilots to investigate the effectiveness of current DTA requirements using the FAA's Airborne Collision Avoidance System X for Rotorcraft (ACAS Xr) as the DAA system under test. The findings of this stufy will be used to inform DTA and ACAS Xr standards development. This presentation will be provided to RTCA Special Committee 228 as an overview and as an opportunity to collect feedback from the larger RPAS/DAA community on the study design and key scenarios and metrics. The presentation provides a brief overview of open areas of research with ACAS Xr, the experimental design, the simulation environment and the simulation schedule.

advanced air mobility↗

Precise Motion Control of Hybrid Hydraulic Electric Architecture (HHEA)

Off-highway heavy-duty vehicles have been long-standing users of hydraulic systems for power transmission and control. However, traditional hydraulic systems suffer from significant energy losses which lead to increased operating costs and a larger carbon footprint due to higher CO2 emissions. Improving the efficiency of these mobile machines is crucial not only for reducing their environmental impact but also for saving billions of dollars in operating costs. Currently, the state-of-the-art Load Sensing Architecture uses throttling valves for control, which significantly reduces its efficiency and does not recuperate energy from over-running loads. Researchers have developed several architectures such as Common Pressure Rail systems, Displacement Control, STEAM, and Electrohydraulic Architecture to improve the efficiency of off-road mobile machines. However, each of these architectures has its drawbacks. To increase system efficiency and take advantage of electrification benefits, our research group has developed a novel Hybrid Hydraulic-Electric Architecture (HHEA). The HHEA can significantly improve efficiency, decrease the size of electrical components, and maintain control performance. This new architecture has the potential to revolutionize the off-highway mobile machine industry and lead to a more sustainable future. The HHEA uses a set of common pressure rails to provide the majority of power to the actuators via power-dense hydraulics and uses electric motors for precise control and power modulation. In the context of off-road mobile machines, energy savings are undoubtedly important but it is equally important to consider the machines’ ability to perform tasks with precision and accuracy according to given commands. Therefore, precise motion control is of utmost importance to maintain the utility of Hybrid Hydraulic-Electric Architecture (HHEA). The HHEA presents a unique challenge to motion control due to the discrete pressure changes that occur when the system switches between selected pressure rails. These changes are made to minimize system inefficiencies or to keep the system within the torque capability of the electric motor. Hence, it is important to solve the motion control challenges for HHEA. This thesis aims at developing an effective motion control strategy for HHEA. The dissertation presents a two-tiered control strategy for HHEA, comprising a high-level and a low-level controller. The primary responsibility of the high- level controller is to optimize energy efficiency by making informed pressure rail selections. On the other hand, the low-level controller is focused on achieving precise motion control of the HHEA, which is crucial for realizing the desired reference trajectories. To achieve this, the low-level controller utilizes a passivity-based backstepping integral controller as the nominal control, which handles the motion control between two pressure rail switches. Additionally, a separate least norm controller is utilized as a transition controller to manage motion control during pressure rail transitions. The effectiveness of the combined control strategy is demonstrated through experiments conducted on two hardware-in-the-loop testbeds. Furthermore, the HHEA is installed on the boom and stick actuators of a backhoe arm to build a Human-in-the-Loop system that a human operator can control. A real-time rail switching algorithm is developed to determine pressure rail switching based on present duty cycle information from the operator. Modifications have been made to the human-machine interface to achieve more intuitive control. Modifications include performing control in the task-oriented coordinates, incorporating pressure feedback to enhance control with physical interaction, and using velocity field control to simplify multi-degree-of-freedom tasks and to enable novice operators to perform them with reduced risk, improved efficiency, and productivity. The research in this dissertation makes significant contributions to the field of off-road mobile machine control, providing a novel and effective control strategy for the HHEA, and demonstrating the potential for simplified machine operation.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Towards Agentic AI on Particle Accelerators

As particle accelerators grow in complexity, traditional control methods face increasing challenges in achieving optimal performance. This paper envisions a paradigm shift: a decentralized multi-agent framework for accelerator control, powered by Large Language Models (LLMs) and distributed among autonomous agents. We present a proposition of a self-improving decentralized system where intelligent agents handle high-level tasks and communication and each agent is specialized control individual accelerator components. This approach raises some questions: What are the future applications of AI in particle accelerators? How can we implement an autonomous complex system such as a particle accelerator where agents gradually improve through experience and human feedback? What are the implications of integrating a human-in-the-loop component for labeling operational data and providing expert guidance? We show two examples, where we demonstrate viability of such architecture.

43 PARTICLE ACCELERATORS↗

Interactive Rotated Object Detection for Novel Class Detection in Remotely Sensed Imagery

In this paper we propose IRTR-DETR an Interactive and Real-Time Rotated DEtection TRansformer that extends IRTDETR to predict rotated bounding boxes. IRTR-DETR maintains the Human-In-The-Loop (HIL) workflow of IRTDETR but introduces rotation-aware heads for improved detection of objects with arbitrary orientations. Similarly to IRTDETR IRTR-DETR can be trained with a small labeled sample set in an interactive setting but we show that it can also be pretrained on related but not identical data--such as a building damage dataset--before being applied to tasks like identifying buildings under construction. We demonstrate the efficacy of our approach on the publicly available Tiny-DOTA and xBD dataset as well as two study-cases on proprietary datasets of greenhouses and houses under construction ("waffle homes"). Detecting greenhouses is highly relevant in the context of damage assessment while "waffle homes" aid understanding typical floorplans and building codes in different areas both thereby supporting population modeling emergency response and policy planning. Our method outperforms the state of the art in interactive rotated object detection on the Tiny-DOTA dataset by 5.7 percent and improves upon the non interactive RTDETR by 7.85 to 19.39 percent (depending on the number of provided samples) while maintaining its real-time efficiency.

Burges, Marvin [ORNL] (ORCID:0000000312690769)↗

HEPTAPOD: Orchestrating High Energy Physics Workflows Towards Autonomous Agency

Many workflows in high-energy-physics (HEP) stand to benefit from recent advances in transformer-based large language models (LLMs). While early applications of LLMs focused on text generation and code completion, modern LLMs now support orchestrated agency: the coordinated execution of complex, multi-step tasks through tool use, structured context, and iterative reasoning. We introduce the HEP Toolkit for Agentic Planning, Orchestration, and Deployment (HEPTAPOD), an orchestration framework designed to bring this emerging paradigm to HEP pipelines. The framework enables LLMs to interface with domain-specific tools, construct and manage simulation workflows, and assist in common utility and data analysis tasks through schema-validated operations and run-card-driven configuration. To demonstrate these capabilities, we consider a representative Beyond the Standard Model (BSM) Monte Carlo validation pipeline that spans model generation, event simulation, and downstream analysis within a unified, reproducible workflow. HEPTAPOD provides a structured and auditable layer between human researchers, LLMs, and computational infrastructure, establishing a foundation for transparent, human-in-the-loop systems.

Menzo, Tony [Alabama U.; Fermilab] (ORCID:00000002↗

Explainable AI for Multivariate Time Series Pattern Exploration: Latent Space Visual Analytics With Temporal Fusion Transformer and Variational Autoencoders in Power Grid Event Diagnosis

Detecting and analyzing complex patterns in multivariate time-series data is crucial for decision-making in urban and environmental system operations. However, challenges arise from the high dimensionality, intricate complexity, and interconnected nature of complex patterns, which hinder the understanding of their underlying physical processes. Existing AI methods often face limitations in interpretability, computational efficiency, and scalability, reducing their applicability in real-world scenarios. This paper proposes a novel visual analytics framework that integrates two generative AI models, Temporal Fusion Transformer (TFT) and Variational Autoencoders (VAEs), to reduce complex patterns into lower-dimensional latent spaces and visualize them in 2D using dimensionality reduction techniques such as PCA, t-SNE, and UMAP with DBSCAN. These visualizations, presented through coordinated and interactive views and tailored glyphs, enable intuitive exploration of complex multivariate temporal patterns, identifying patterns’ similarities and uncover their potential correlations for a better interpretability of the AI outputs. The framework is demonstrated through a case study on power grid signal data, where it identifies multi-label grid event signatures, including faults and anomalies with diverse root causes. Additionally, novel metrics and visualizations are introduced to validate the models and assess the performance, efficiency, and consistency of latent maps generated by VAE, which have been utilized in prior studies for latent space cartography and used as a benchmark in this study, and the emerging TFT architecture under various configurations. These analyses provide actionable insights for model parameter tuning and reliability improvements. Comparative results highlight that TFT achieves shorter run times and superior scalability to diverse time-series data shapes compared to VAE. This work advances fault diagnosis in multivariate time series, fostering explainable AI to support critical system operations.

Explainable AI↗

Load Shedding for Voltage Regulation With Probabilistic Agent Compliance

With the increased observability and controllability of distribution systems, the share of behind-the-meter systems is trending upwards rapidly. As a consequence, the impact of human behaviors on system performance can no longer be ignored and should be reflected in the energy management system models. In this paper, we discuss the problem of distribution system voltage control by active power curtailment where the agent compliance of the load curtailment signal is probabilistic. We discuss the modeling of the optimal voltage control problem with probabilistic agent compliance as a chance-constrained optimization problem, its tractable safe approximation using convex restriction, and a scenario-based mixed-integer reformulation as well as the associated solution method based on augmented Lagrangian method. The numerical simulation on IEEE test system validates the effectiveness of the proposed approach in obtaining high-quality feasible load curtailment signal with low computational cost, which makes it a viable tool for real time decision making.

augmented Lagrangian method↗

Air Traffic Controller Performance and Acceptability of Multiple UAS in a Simulated NAS Environment

Previously, we showed that air traffic controllers (ATCos) rated UAS pilot verbal response latencies as acceptable when a 1.5 s delay was added to the UAS pilot responses, but a 5 s delay was rated as mostly unacceptable. In the present study we determined whether a 1.5 s added delay in the UAS pilots' verbal communications would affect ATCos interactions with UAS and other conventional aircraft when the number and speed of the UAS were manipulated. Eight radar-certified ATCos participated in this simulation. The ATCos managed a medium altitude sector containing arrival aircraft, en route aircraft, and one to four UAS. The UAS were conducting a surveillance mission and flew at either a "slow" or "fast" speed. We measured both UAS and conventional pilots' verbal communication latencies, and obtained ATCos' acceptability ratings for these latencies. Although the UAS pilot response latencies were longer than those of conventional pilots, the ATCos rated UAS pilot verbal communication latencies to be as acceptable as those of conventional pilots. Because the overall traffic load within the sector was held constant, ATCos only performed slightly worse when multiple UAS were in their sector compared to when only one UAS was in the sector. Implications of these findings for UAS integration in the NAS are discussed.

measured response↗

Subject Matter Expert Evaluation of Multi-Flight Common Route Advisories

Traffic flow management seeks to balance the demand for National Airspace System (NAS) flight resources, such as airspace and airports, with the available supply. When forecasted weather blocks nominal air traffic routes, traffic managers must re-route affected flights for weather avoidance. Depending on the nature and scope of the weather, traffic managers may use pre-coordinated re-routes such as Playbook Routes or Coded Departure Routes, or may design ad hoc local re-routes. The routes of affected flights are modified accordingly. These weather avoidance routes will, of course, be less efficient than the nominal routes due to increased flight time and fuel burn. In current traffic management operations, the transition into a weather avoidance re-routing initiative is typically implemented more aggressively than the transition out of that initiative after the weather has dissipated or moved away. For example, strategic large-scale Playbook re-routes are sometimes left in place (as initially implemented) for many hours before being lifted entirely when the weather dissipates. There is an opportunity to periodically modify the re-routing plan as weather evolves, thereby attenuating its adverse impact on flight time and fuel consumption; this is called delay recovery. Multi-Flight Common Routes (MFCR) is a NASA-developed operational concept and associated decision support tool for delay recovery, designed to assist traffic managers to efficiently update weather avoidance traffic routes after the original re-routes have become stale due to subsequent evolution of the convective weather system. MFCR groups multiple flights to reduce the number of advisories that the traffic manager needs to evaluate, and also merges these flights on a common route segment to provide an orderly flow of re-routed traffic. The advisory is presented to the appropriate traffic manager who evaluates it and has the option to modify it using MFCRs graphical user interface. If the traffic manager finds the advisory to be operationally appropriate, he or she would coordinate with the Area Supervisor(s) of the sectors that currently control the flights in the advisory. When the traffic manager accepts the MFCR advisory via the user interface, the corresponding flight plan amendments would be sent to the displays of the appropriate sector controllers, using the Airborne Re-Routing (ABRR) capability which is scheduled for nationwide operation in 2017. The sector controllers would then offer this time-saving route modification to the pilots of the affected flights via datalink (or voice), and implement the corresponding flight plan amendment if the pilots accept it. MFCR is implemented as an application in the software environment of the Future Air traffic management Concepts Evaluation Tool (FACET). This paper focuses on an initial subject matter expert (SME) evaluation of MFCR. The evaluation covers MFCRs operational concept, algorithm, and user interface.

Human-in-the-loop Evaluation↗

Subject Matter Expert Evaluation of Multi-Flight Common Route Advisories

Traffic flow management seeks to balance the demand for National Airspace System (NAS) flight resources, such as airspace and airports, with the available supply. When forecasted weather blocks nominal air traffic routes, traffic managers must re-route affected flights for weather avoidance. Depending on the nature and scope of the weather, traffic managers may use pre-coordinated re-routes such as Playbook Routes or Coded Departure Routes, or may design ad hoc local re-routes. The routes of affected flights are modified accordingly. These weather avoidance routes will, of course, be less efficient than the nominal routes due to increased flight time and fuel burn. In current traffic management operations, the transition into a weather avoidance re-routing initiative is typically implemented more aggressively than the transition out of that initiative after the weather has dissipated or moved away. For example, strategic large-scale Playbook re-routes are sometimes left in place (as initially implemented) for many hours before being lifted entirely when the weather dissipates. There is an opportunity to periodically modify the re-routing plan as weather evolves, thereby attenuating its adverse impact on flight time and fuel consumption; this is called delay recovery. Multi-Flight Common Routes (MFCR) is a NASA-developed operational concept and associated decision support tool for delay recovery, designed to assist traffic managers to efficiently update weather avoidance traffic routes after the original re-routes have become stale due to subsequent evolution of the convective weather system. MFCR groups multiple flights to reduce the number of advisories that the traffic manager needs to evaluate, and also merges these flights on a common route segment to provide an orderly flow of re-routed traffic. The advisory is presented to the appropriate traffic manager who evaluates it and has the option to modify it using MFCRs graphical user interface. If the traffic manager finds the advisory to be operationally appropriate, he or she would coordinate with the Area Supervisor(s) of the sectors that currently control the flights in the advisory. When the traffic manager accepts the MFCR advisory via the user interface, the corresponding flight plan amendments would be sent to the displays of the appropriate sector controllers, using the Airborne Re-Routing (ABRR) capability which is scheduled for nationwide operation in 2017. The sector controllers would then offer this time-saving route modification to the pilots of the affected flights via datalink (or voice), and implement the corresponding flight plan amendment if the pilots accept it. MFCR is implemented as an application in the software environment of the Future Air traffic management Concepts Evaluation Tool (FACET). This paper focuses on an initial subject matter expert (SME) evaluation of MFCR. The evaluation covers MFCRs operational concept, algorithm, and user interface.

Traffic flow management↗

Desert Research and Technology Studies (D-RATS) 2022 Test Report

The primary purpose of the 2022 Desert Research and Technology Studies (D-RATS 22) analog mission field test was to provide data and recommendations regarding how pressurized rover (PR) design, cabin configuration, driving modes, timeline constraints, and mission operations can be acceptably implemented to support future Artemis missions that include a PR. The goal of these evaluations was to provide first order feedback and data on design concepts currently being traded as options for future pressurized rover flight prototypes. The Japan Aerospace Exploration Agency (JAXA) may provide a PR for future Artemis missions. One of the primary objectives of D-RATS 2022 was to share with the JAXA PR team the processes and protocols by which NASA has learned to successfully evaluate rover concepts through integrated HITL evaluations over the past 20+ years, so that JAXA could apply any applicable lessons learned to their PR design, development, testing, and evaluation.

analog↗