Search NASA⌕ Search

SEARCH · Search NASA

Results for “Human-In-The-Loop”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Phenomena Portal: Large- Scale Visual Exploration of Atmospheric Phenomena

The Earth science community is experiencing a high influx of remote sensing data due to recent advancements in sensor technology. This enables the community to extend their research on a larger scale than ever before. Unfortunately, traditional data processing techniques do not scale well to these new, high volume data sources. State-of-the-art machine learning (ML) pipelines have been proven to overcome these burdens in various other fields but are underexploited within the physical sciences community. Moreover, ML is reliant on labeled data, which is currently sparsely available, owing to the fact that ML adoption is still in the early stages within the Earth and atmospheric science communities. To address these issues, we developed the Phenomena Portal, a visual exploration tool that uses ML to detect various atmospheric phenomena on a global scale. This allows the Earth and atmospheric science communities to view trends of occurrences of phenomena, identify potential relationships between them, and analyze spatiotemporal patterns over time. These detections can also serve as initial labeled data for ML research pertaining to the respective phenomena. The tool also incorporates feedback from subject matter experts to further improve the model detection accuracy, thereby facilitating human-in-the-loop. This presentation will provide an overview of the ML model development and cloud deployment. We also discuss the capabilities of the user interface for displaying the detections.

Muthukumaran Ramasubramanian↗
NASA NTRS2001

Monte-carlo maneuver analysis for the microwave anisotropy probe

The results of this trajectory replication are presented, followed by the results of monte-carlo maneuver simulations. The results are subject to two important assumptions: that linearization is valid (as the software used for the monte-carlo simulation, LAMBIC, is based on a linearization of the trajectory about the nominal) and that the planned human-in-the-loop lunar targeting may be approximated by targeting to a fixed aimpoint.

monte-carlo↗
NASA NTRS2018

Do You See What I See? Interactive Visualization of Mission Design and Navigation

Mission Design and Navigation (MDNav) is an intensive process requiring advanced computational resources, expert human intuition, and many successive human-in-the-loop iterations to converge on acceptable trajectory designs or navigation solutions. The current bottleneck in MDNav is not the underlying computational algorithms but the human cognitive capacity to prune through a multitude of simulated results to select high-value candidates. One approach to alleviate this burden is through the judicious application of visualizations that allow humans to interactively filter data in multiple dimensions to reveal salient patterns and highlight divergences. When designed efficiently, such interactive visualizations should aid human operators to get familiar with data faster, visually observe correlations, and communicate findings more effortlessly. In this work, we present three visualization case studies that have the potential to increase human operator efficiency in MDNav. While identifying the most critical “pain points” that operators face, and also working on potential solutions, we followed a human-centered design approach. We started with a series of interviews with potential users, and then rapidly created prototypes for alternative solutions, validated outcomes with feedback from users through out development of these proof of concept visualizations. With this survey of our current efforts, we demonstrate the transformative capability of interactive data visualizations for improving mission development and operations, enabling operators to grow intuition, and communicating key concepts across diverse mission teams.

Arora, Nitin↗
NASA NTRSDate not supplied

Artemis Lunar Surface VR/ARGOS Trainer

This proposal aims to provide insight by identifying potential risks and unknowns of lander egress and surface operations through a Mixed Reality (MR) planning, training, and analysis capability that integrates Virtual Reality (VR) simulations and the Active Response Gravity Offload System (ARGOS) in support of Artemis missions to the moon. The VR simulation will incorporate lunar digital elevation map data and imagery to provide accurate terrain of the south pole and Shackleton Crater. Date specific ephemerides will used to simulate the extreme lighting environment. Virtual representations of a lunar lander vehicle will be represented with a physical mockup of the porch and ladder assembly. Human-in-the-loop engineering test runs within ARGOS will be used to refine performance of the Mixed Reality interface with the mockup platform and define procedures for training.

Lee K Bingham↗
NASA NTRSDate not supplied

AIRNOISEUAM: An Urban Air Mobility Noise-Exposure Prediction Tool

A new software tool called AIRNOISEUAM is introduced that models the noise exposure of Urban Air Mobility (UAM) operations. Given relevant UAM aircraft performance models, mission profiles, and Noise-Power-Distance data, AIRNOISEUAM predicts the noise exposure footprint for receptors on the ground. The performance of AIRNOISEUAM using a Robinson R66 helicopter model and a six-passenger quadrotor model, and a diverse set of scenarios from NASA’s UAM human-in-the-loop simulations is compared to that of the industry-standard tool with the same inputs. The predicted noise exposure results from both tools are found to be nearly identical. AIRNOISEUAM offers a fast-time, flexible interface and modular design to facilitate the dynamic requirements of the aviation research community.

Urban air mobility↗
NASA NTRSDate not supplied

AIRNOISEUAM: An Urban Air Mobility Noise-Exposure Prediction Tool

A new software tool called AIRNOISEUAM is introduced that models the noise exposure of Urban Air Mobility (UAM) operations. Given relevant UAM aircraft performance models, mission profiles, and Noise-Power-Distance data, AIRNOISEUAM predicts the noise exposure footprint for receptors on the ground. The performance of AIRNOISEUAM using a Robinson R66 helicopter model and a six-passenger quadrotor model, and a diverse set of scenarios from NASA’s UAM human-in-the-loop simulations is compared to that of the industry-standard tool with the same inputs. The predicted noise exposure results from both tools are found to be nearly identical. AIRNOISEUAM offers a fast-time, flexible interface and modular design to facilitate the dynamic requirements of the aviation research community.

Urban Air Mobility↗
NASA NTRSDate not supplied

A Cognitive Walkthrough of Multiple Drone Delivery Operations

Advances of early twenty-first century aviation and transportation technologies provide opportunities for enhanced aerial projects, and the overall integration of unmanned aircraft systems (UAS) into the National Airspace System (NAS) has applications across a wide range of operations. Through these, remote operators have learned to manage several UAS at the same time in a variety of operational environments. The present work details a component piece of an ongoing body of research into multi-UAS operations. Beginning in early 2020, NASA has collaborated with Uber Technologies to design and develop concepts of operations, roles and responsibilities, and ground control station (GCS) concepts to enable food delivery operations via multiple, small UAS (sUAS). A cognitive walkthrough was chosen as the method for data collection. This allowed information to be gathered from UAS subject matter experts (SMEs) that could further mature designs for future human-in-the-loop (HITL) simulations; in addition, it allowed information to be collected remotely during the stringent restrictions of the COVID-19 pandemic. Consequently, the described cognitive walkthrough activity utilized remote data collection protocols mediated through the usage of programs designed for presentation and telecommunications. Scenarios were designed, complete with airspace, contingencies, and remedial actions, to be presented to the SMEs. Information was collected using a combination of rating scales and open-ended questions. Results received from the SMEs revealed expected hazards, workloads, and information concerns inherent in the contingency scenarios. SMEs also provided insight into the design of GCS tools and displays as well as the duties and relationships of human operators (i.e., monitors) and automation (i.e., informers and flight managers). Implications of these findings are discussed.

unmanned aircraft systems↗
NASA NTRS2021

A Cognitive Walkthrough of Multiple Drone Delivery Operations

Advances of early twenty-first century aviation and transportation technologies provide opportunities for enhanced aerial projects, and the overall integration of unmanned aircraft systems (UAS) into the National Airspace System (NAS) has applications across a wide range of operations. Through these, remote operators have learned to manage several UAS at the same time in a variety of operational environments. The present work details a component piece of an ongoing body of research into multi-UAS operations. Beginning in early 2020, NASA has collaborated with Uber Technologies to design and develop concepts of operations, roles and responsibilities, and ground control station (GCS) concepts to enable food delivery operations via multiple, small UAS (sUAS). A cognitive walkthrough was chosen as the method for data collection. This allowed information to be gathered from UAS subject matter experts (SMEs) that could further mature designs for future human-in-the-loop (HITL) simulations; in addition, it allowed information to be collected remotely during the stringent restrictions of the COVID-19 pandemic. Consequently, the described cognitive walkthrough activity utilized remote data collection protocols mediated through the usage of programs designed for presentation and telecommunications. Scenarios were designed, complete with airspace, contingencies, and remedial actions, to be presented to the SMEs. Information was collected using a combination of rating scales and open-ended questions. Results received from the SMEs revealed expected hazards, workloads, and information concerns inherent in the contingency scenarios. SMEs also provided insight into the design of GCS tools and displays as well as the duties and relationships of human operators (i.e., monitors) and automation (i.e., informers and flight managers). Implications of these findings are discussed.

unmanned aircraft systems↗
NASA NTRS2021

Integrated Demand Management Techport Closeout

Over the period 2016 to 2020, Integrated Demand Management (IDM) concept, procedures, and tools have been developed - first for clear-weather and a single airport constraint and then expanded to handle a multi-constraint problem during convective weather at Newark (EWR), LaGuardia (LGA) and Philadelphia (PHL) airports. The concept was evaluated in a series of human-in-the-loop simulations to confirm concept benefits in terms of better schedule predictability, reduction of delays, and increased throughput, especially during convective weather. In addition, other conditions with mixed participation of multi-trajectories from the airlines were evaluated that showed significant benefits to the individual airlines as well as the overall traffic flow. The IDM concept was initiated in the SMART-NAS project, was completed under the ATM-X project, and was developed collaboratively with the FAA and airline partners. Over the course of IDM development, NASA researchers produced 19 conference papers and publications. Outside supporting organizations, funded by IDM, produced 20 additional conference papers and publications, in which they advanced fundamental research on topics such as better stochastic traffic demand prediction, and application of machine learning techniques for modeling traffic management initiatives. Final concept procedures and tool specifications have been transferred to the FAA Air Traffic Organization Operational Concepts, Validation, and Requirements group.

William N Chan↗
NASA NTRSDate not supplied

ACAS Xr Part Task Sim, Preliminary Experiment Design

In early 2022, the Human Autonomy Teaming Lab (NASA Ames Research Center) will conduct a manned, human-in-the-loop (HITL) simulation. This part task HITL will begin the lab’s Hazard Perception and Avoidance (HPA) technical work under NASA’s Advanced Air Mobility (AAM), Automated Flight and Contingency Management (AFCM) Sub-Project. The goals will be to assess levels of automation for manned, electric vertical takeoff and landing (eVTOL) aircraft. This simulation will test manual and automated Resolution Advisory (RA) responses and return-to-course (RTC) maneuvers for the first version of the Airborne Collision Avoidance System’s (ACAS) rotary-wing (Xr) variant. This will be conducted on a fixed-based simulator designed to fly eVTOL aircraft while maneuvering for intruding traffic. Variables for this study include levels of autonomy (i.e., manual and automated) as well as the types of alerts at the onset of conflicts (i.e., Corrective and RA). The data collected will include response times, losses of well clear, and maneuver sizes and durations as well as subjective ratings like acceptability, perceived workload, and meaningful human control. Additional details and future anticipations are also discussed.

air taxis↗
DOE OSTI2025

Interactive Rotated Object Detection for Novel Class Detection in Remotely Sensed Imagery

In this paper we propose IRTR-DETR an Interactive and Real-Time Rotated DEtection TRansformer that extends IRTDETR to predict rotated bounding boxes. IRTR-DETR maintains the Human-In-The-Loop (HIL) workflow of IRTDETR but introduces rotation-aware heads for improved detection of objects with arbitrary orientations. Similarly to IRTDETR IRTR-DETR can be trained with a small labeled sample set in an interactive setting but we show that it can also be pretrained on related but not identical data--such as a building damage dataset--before being applied to tasks like identifying buildings under construction. We demonstrate the efficacy of our approach on the publicly available Tiny-DOTA and xBD dataset as well as two study-cases on proprietary datasets of greenhouses and houses under construction ("waffle homes"). Detecting greenhouses is highly relevant in the context of damage assessment while "waffle homes" aid understanding typical floorplans and building codes in different areas both thereby supporting population modeling emergency response and policy planning. Our method outperforms the state of the art in interactive rotated object detection on the Tiny-DOTA dataset by 5.7 percent and improves upon the non interactive RTDETR by 7.85 to 19.39 percent (depending on the number of provided samples) while maintaining its real-time efficiency.

Burges, Marvin [ORNL] (ORCID:0000000312690769)↗
DOE OSTI2025

HEPTAPOD: Orchestrating High Energy Physics Workflows Towards Autonomous Agency

Many workflows in high-energy-physics (HEP) stand to benefit from recent advances in transformer-based large language models (LLMs). While early applications of LLMs focused on text generation and code completion, modern LLMs now support orchestrated agency: the coordinated execution of complex, multi-step tasks through tool use, structured context, and iterative reasoning. We introduce the HEP Toolkit for Agentic Planning, Orchestration, and Deployment (HEPTAPOD), an orchestration framework designed to bring this emerging paradigm to HEP pipelines. The framework enables LLMs to interface with domain-specific tools, construct and manage simulation workflows, and assist in common utility and data analysis tasks through schema-validated operations and run-card-driven configuration. To demonstrate these capabilities, we consider a representative Beyond the Standard Model (BSM) Monte Carlo validation pipeline that spans model generation, event simulation, and downstream analysis within a unified, reproducible workflow. HEPTAPOD provides a structured and auditable layer between human researchers, LLMs, and computational infrastructure, establishing a foundation for transparent, human-in-the-loop systems.

Menzo, Tony [Alabama U.; Fermilab] (ORCID:00000002↗
DOE OSTI2025

Explainable AI for Multivariate Time Series Pattern Exploration: Latent Space Visual Analytics With Temporal Fusion Transformer and Variational Autoencoders in Power Grid Event Diagnosis

Detecting and analyzing complex patterns in multivariate time-series data is crucial for decision-making in urban and environmental system operations. However, challenges arise from the high dimensionality, intricate complexity, and interconnected nature of complex patterns, which hinder the understanding of their underlying physical processes. Existing AI methods often face limitations in interpretability, computational efficiency, and scalability, reducing their applicability in real-world scenarios. This paper proposes a novel visual analytics framework that integrates two generative AI models, Temporal Fusion Transformer (TFT) and Variational Autoencoders (VAEs), to reduce complex patterns into lower-dimensional latent spaces and visualize them in 2D using dimensionality reduction techniques such as PCA, t-SNE, and UMAP with DBSCAN. These visualizations, presented through coordinated and interactive views and tailored glyphs, enable intuitive exploration of complex multivariate temporal patterns, identifying patterns’ similarities and uncover their potential correlations for a better interpretability of the AI outputs. The framework is demonstrated through a case study on power grid signal data, where it identifies multi-label grid event signatures, including faults and anomalies with diverse root causes. Additionally, novel metrics and visualizations are introduced to validate the models and assess the performance, efficiency, and consistency of latent maps generated by VAE, which have been utilized in prior studies for latent space cartography and used as a benchmark in this study, and the emerging TFT architecture under various configurations. These analyses provide actionable insights for model parameter tuning and reliability improvements. Comparative results highlight that TFT achieves shorter run times and superior scalability to diverse time-series data shapes compared to VAE. This work advances fault diagnosis in multivariate time series, fostering explainable AI to support critical system operations.

Explainable AI↗
NASA NTRS2014

Air Traffic Controller Performance and Acceptability of Multiple UAS in a Simulated NAS Environment

Previously, we showed that air traffic controllers (ATCos) rated UAS pilot verbal response latencies as acceptable when a 1.5 s delay was added to the UAS pilot responses, but a 5 s delay was rated as mostly unacceptable. In the present study we determined whether a 1.5 s added delay in the UAS pilots' verbal communications would affect ATCos interactions with UAS and other conventional aircraft when the number and speed of the UAS were manipulated. Eight radar-certified ATCos participated in this simulation. The ATCos managed a medium altitude sector containing arrival aircraft, en route aircraft, and one to four UAS. The UAS were conducting a surveillance mission and flew at either a "slow" or "fast" speed. We measured both UAS and conventional pilots' verbal communication latencies, and obtained ATCos' acceptability ratings for these latencies. Although the UAS pilot response latencies were longer than those of conventional pilots, the ATCos rated UAS pilot verbal communication latencies to be as acceptable as those of conventional pilots. Because the overall traffic load within the sector was held constant, ATCos only performed slightly worse when multiple UAS were in their sector compared to when only one UAS was in the sector. Implications of these findings for UAS integration in the NAS are discussed.

measured response↗
NASA NTRS2017

Subject Matter Expert Evaluation of Multi-Flight Common Route Advisories

Traffic flow management seeks to balance the demand for National Airspace System (NAS) flight resources, such as airspace and airports, with the available supply. When forecasted weather blocks nominal air traffic routes, traffic managers must re-route affected flights for weather avoidance. Depending on the nature and scope of the weather, traffic managers may use pre-coordinated re-routes such as Playbook Routes or Coded Departure Routes, or may design ad hoc local re-routes. The routes of affected flights are modified accordingly. These weather avoidance routes will, of course, be less efficient than the nominal routes due to increased flight time and fuel burn. In current traffic management operations, the transition into a weather avoidance re-routing initiative is typically implemented more aggressively than the transition out of that initiative after the weather has dissipated or moved away. For example, strategic large-scale Playbook re-routes are sometimes left in place (as initially implemented) for many hours before being lifted entirely when the weather dissipates. There is an opportunity to periodically modify the re-routing plan as weather evolves, thereby attenuating its adverse impact on flight time and fuel consumption; this is called delay recovery. Multi-Flight Common Routes (MFCR) is a NASA-developed operational concept and associated decision support tool for delay recovery, designed to assist traffic managers to efficiently update weather avoidance traffic routes after the original re-routes have become stale due to subsequent evolution of the convective weather system. MFCR groups multiple flights to reduce the number of advisories that the traffic manager needs to evaluate, and also merges these flights on a common route segment to provide an orderly flow of re-routed traffic. The advisory is presented to the appropriate traffic manager who evaluates it and has the option to modify it using MFCRs graphical user interface. If the traffic manager finds the advisory to be operationally appropriate, he or she would coordinate with the Area Supervisor(s) of the sectors that currently control the flights in the advisory. When the traffic manager accepts the MFCR advisory via the user interface, the corresponding flight plan amendments would be sent to the displays of the appropriate sector controllers, using the Airborne Re-Routing (ABRR) capability which is scheduled for nationwide operation in 2017. The sector controllers would then offer this time-saving route modification to the pilots of the affected flights via datalink (or voice), and implement the corresponding flight plan amendment if the pilots accept it. MFCR is implemented as an application in the software environment of the Future Air traffic management Concepts Evaluation Tool (FACET). This paper focuses on an initial subject matter expert (SME) evaluation of MFCR. The evaluation covers MFCRs operational concept, algorithm, and user interface.

Human-in-the-loop Evaluation↗
NASA NTRS2017

Subject Matter Expert Evaluation of Multi-Flight Common Route Advisories

Traffic flow management seeks to balance the demand for National Airspace System (NAS) flight resources, such as airspace and airports, with the available supply. When forecasted weather blocks nominal air traffic routes, traffic managers must re-route affected flights for weather avoidance. Depending on the nature and scope of the weather, traffic managers may use pre-coordinated re-routes such as Playbook Routes or Coded Departure Routes, or may design ad hoc local re-routes. The routes of affected flights are modified accordingly. These weather avoidance routes will, of course, be less efficient than the nominal routes due to increased flight time and fuel burn. In current traffic management operations, the transition into a weather avoidance re-routing initiative is typically implemented more aggressively than the transition out of that initiative after the weather has dissipated or moved away. For example, strategic large-scale Playbook re-routes are sometimes left in place (as initially implemented) for many hours before being lifted entirely when the weather dissipates. There is an opportunity to periodically modify the re-routing plan as weather evolves, thereby attenuating its adverse impact on flight time and fuel consumption; this is called delay recovery. Multi-Flight Common Routes (MFCR) is a NASA-developed operational concept and associated decision support tool for delay recovery, designed to assist traffic managers to efficiently update weather avoidance traffic routes after the original re-routes have become stale due to subsequent evolution of the convective weather system. MFCR groups multiple flights to reduce the number of advisories that the traffic manager needs to evaluate, and also merges these flights on a common route segment to provide an orderly flow of re-routed traffic. The advisory is presented to the appropriate traffic manager who evaluates it and has the option to modify it using MFCRs graphical user interface. If the traffic manager finds the advisory to be operationally appropriate, he or she would coordinate with the Area Supervisor(s) of the sectors that currently control the flights in the advisory. When the traffic manager accepts the MFCR advisory via the user interface, the corresponding flight plan amendments would be sent to the displays of the appropriate sector controllers, using the Airborne Re-Routing (ABRR) capability which is scheduled for nationwide operation in 2017. The sector controllers would then offer this time-saving route modification to the pilots of the affected flights via datalink (or voice), and implement the corresponding flight plan amendment if the pilots accept it. MFCR is implemented as an application in the software environment of the Future Air traffic management Concepts Evaluation Tool (FACET). This paper focuses on an initial subject matter expert (SME) evaluation of MFCR. The evaluation covers MFCRs operational concept, algorithm, and user interface.

Traffic flow management↗