Search NASA⌕ Search

SEARCH · Search NASA

Results for “Human Interfaces”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
NASA NTRS1993

An operator interface design for a telerobotic inspection system

The operator interface has recently emerged as an important element for efficient and safe interactions between human operators and telerobotics. Advances in graphical user interface and graphics technologies enable us to produce very efficient operator interface designs. This paper describes an efficient graphical operator interface design newly developed for remote surface inspection at NASA-JPL. The interface, designed so that remote surface inspection can be performed by a single operator with an integrated robot control and image inspection capability, supports three inspection strategies of teleoperated human visual inspection, human visual inspection with automated scanning, and machine-vision-based automated inspection.

Kim, Won S.↗
DOE OSTI2026

From natural language to control signals: a conceptual framework for semantic channel finding in complex experimental infrastructure

Modern experimental platforms such as particle accelerators, fusion devices, telescopes, and industrial process control systems expose tens to hundreds of thousands of control and diagnostic channels, accumulated over decades of hardware evolution. Operators and AI systems alike depend on informal expert knowledge, inconsistent naming conventions, and scattered documentation to locate the signals required for monitoring, troubleshooting, and automated control, creating a persistent bottleneck for reliability, scalability, and emerging language-model-driven interfaces. We formalize semantic channel finding, the task of mapping natural-language intent to concrete control-system signals, as a general problem in complex experimental infrastructure, and introduce a four-paradigm conceptual framework to guide architecture selection based on facility-specific data regimes. The paradigms span (i) direct in-context lookup over small, curated channel dictionaries, (ii) constrained hierarchical navigation through structured trees, (iii) interactive agent exploration using iterative reasoning and tool-based database queries, and (iv) ontology-grounded semantic search that decouples channel meaning from facility-specific naming conventions. We demonstrate the practical feasibility of each paradigm through proof-of-concept implementations at four operational facilities spanning two orders of magnitude in scale: from compact free-electron lasers to large synchrotron light sources, operating under diverse control-system architectures ranging from clean hierarchical naming schemes to legacy environments with decades of heterogeneous conventions. Where evaluated against expert-curated operational queries, these instantiations achieve 90%–97% accuracy, validating the framework’s applicability across real-world deployment scenarios. To accelerate adoption across the broader scientific and industrial control-system community, we release open-source, plug-and-play implementations of all three interactive paradigms-direct lookup, hierarchical navigation, and middle-layer exploration-within the Osprey framework, together with tools for channel database generation, interactive testing, and minimal-configuration deployment. This work establishes semantic channel finding as a foundational capability for human-centric and agentic AI interfaces at large-scale facilities, providing both a systematic framework for architecture design and practical resources to enable adoption without building custom infrastructure from scratch.

channel finding↗
NASA NTRS2012

Teleoperation of Robonaut Using Finger Tracking

With the advent of new finger tracking systems, the idea of a more expressive and intuitive user interface is being explored and implemented. One practical application for this new kind of interface is that of teleoperating a robot. For humanoid robots, a finger tracking interface is required due to the level of complexity in a human-like hand, where a joystick isn't accurate. Moreover, for some tasks, using one's own hands allows the user to communicate their intentions more effectively than other input. The purpose of this project was to develop a natural user interface for someone to teleoperate a robot that is elsewhere. Specifically, this was designed to control Robonaut on the international space station to do tasks too dangerous and/or too trivial for human astronauts. This interface was developed by integrating and modifying 3Gear's software, which includes a library of gestures and the ability to track hands. The end result is an interface in which the user can manipulate objects in real time in the user interface. then, the information is relayed to a simulator, the stand in for Robonaut, at a slight delay.

robotics↗
NASA NTRS2005

Systems Engineering and Integration for Advanced Life Support System and HST

Systems engineering (SE) discipline has revolutionized the way engineers and managers think about solving issues related to design of complex systems: With continued development of state-of-the-art technologies, systems are becoming more complex and therefore, a systematic approach is essential to control and manage their integrated design and development. This complexity is driven from integration issues. In this case, subsystems must interact with one another in order to achieve integration objectives, and also achieve the overall system's required performance. Systems engineering process addresses these issues at multiple levels. It is a technology and management process dedicated to controlling all aspects of system life cycle to assure integration at all levels. The Advanced Integration Matrix (AIM) project serves as the systems engineering and integration function for the Human Support Technology (HST) program. AIM provides means for integrated test facilities and personnel for performance trade studies, analyses, integrated models, test results, and validated requirements of the integration of HST. The goal of AIM is to address systems-level integration issues for exploration missions. It will use an incremental systems integration approach to yield technologies, baselines for further development, and possible breakthrough concepts in the areas of technological and organizational interfaces, total information flow, system wide controls, technical synergism, mission operations protocols and procedures, and human-machine interfaces.

Kamarani, Ali K.↗
NASA NTRS1989

Reflections on human error - Matters of life and death

The last two decades have witnessed a rapid growth in the introduction of automatic devices into aircraft cockpits, and eleswhere in human-machine systems. This was motivated in part by the assumption that when human functioning is replaced by machine functioning, human error is eliminated. Experience to date shows that this is far from true, and that automation does not replace humans, but changes their role in the system, as well as the types and severity of the errors they make. This altered role may lead to fewer, but more critical errors. Intervention strategies to prevent these errors, or ameliorate their consequences include basic human factors engineering of the interface, enhanced warning and alerting systems, and more intelligent interfaces that understand the strategic intent of the crew and can detect and trap inconsistent or erroneous input before it affects the system.

Wiener, Earl L.↗
NASA NTRS1992

Experiments in Nonlinear Adaptive Control of Multi-Manipulator, Free-Flying Space Robots

Sophisticated robots can greatly enhance the role of humans in space by relieving astronauts of low level, tedious assembly and maintenance chores and allowing them to concentrate on higher level tasks. Robots and astronauts can work together efficiently, as a team; but the robot must be capable of accomplishing complex operations and yet be easy to use. Multiple cooperating manipulators are essential to dexterity and can broaden greatly the types of activities the robot can achieve; adding adaptive control can ease greatly robot usage by allowing the robot to change its own controller actions, without human intervention, in response to changes in its environment. Previous work in the Aerospace Robotics Laboratory (ARL) have shown the usefulness of a space robot with cooperating manipulators. The research presented in this dissertation extends that work by adding adaptive control. To help achieve this high level of robot sophistication, this research made several advances to the field of nonlinear adaptive control of robotic systems. A nonlinear adaptive control algorithm developed originally for control of robots, but requiring joint positions as inputs, was extended here to handle the much more general case of manipulator endpoint-position commands. A new system modelling technique, called system concatenation was developed to simplify the generation of a system model for complicated systems, such as a free-flying multiple-manipulator robot system. Finally, the task-space concept was introduced wherein the operator's inputs specify only the robot's task. The robot's subsequent autonomous performance of each task still involves, of course, endpoint positions and joint configurations as subsets. The combination of these developments resulted in a new adaptive control framework that is capable of continuously providing full adaptation capability to the complex space-robot system in all modes of operation. The new adaptive control algorithm easily handles free-flying systems with multiple, interacting manipulators, and extends naturally to even larger systems. The new adaptive controller was experimentally demonstrated on an ideal testbed in the ARL-A first-ever experimental model of a multi-manipulator, free-flying space robot that is capable of capturing and manipulating free-floating objects without requiring human assistance. A graphical user interface enhanced the robot usability: it enabled an operator situated at a remote location to issue high-level task description commands to the robot, and to monitor robot activities as it then carried out each assignment autonomously.

Chen, Vincent Wei-Kang↗
NASA NTRS1996

A dynamical model for reflex activated head movements in the horizontal plane

We present a controls systems model of horizontal-plane head movements during perturbations of the trunk, which for the first time interfaces a model of the human head with neural feedback controllers representing the vestibulocollic (VCR) and the cervicocollic (CCR) reflexes. This model is homeomorphic such that model structure and parameters are drawn directly from anthropomorphic, biomechanical and physiological studies. Using control theory we analyzed the system model in the time and frequency domains, simulating neck movement responses to input perturbations of the trunk. Without reflex control, the head and neck system produced a second-order underdamped response with a 5.2 dB resonant peak at 2.1 Hz. Adding the CCR component to the system dampened the response by approximately 7%. Adding the VCR component dampened head oscillations by 75%. The VCR also improved low-frequency compensation by increasing the gain and phase lag, creating a phase minimum at 0.1 Hz and a phase peak at 1.1 Hz. Combining all three components (mechanics, VCR and CCR) linearly in the head and neck system reduced the amplitude of the resonant peak to 1.1 dB and increased the resonant frequency to 2.9 Hz. The closed loop results closely fit human data, and explain quantitatively the characteristic phase peak often observed.

NASA Discipline Neuroscience↗
NASA NTRS2003

Software for Simulating a Complex Robot

RoboSim (Robot Simulation) is a computer program that simulates the poses and motions of the Robonaut a developmental anthropomorphic robot that has a complex system of joints with 43 degrees of freedom and multiple modes of operation and control. RoboSim performs a full kinematic simulation of all degrees of freedom. It also includes interface components that duplicate the functionality of the real Robonaut interface with control software and human operators. Basically, users see no difference between the real Robonaut and the simulation. Consequently, new control algorithms can be tested by computational simulation, without risk to the Robonaut hardware, and without using excessive Robonaut-hardware experimental time, which is always at a premium. Previously developed software incorporated into RoboSim includes Enigma (for graphical displays), OSCAR (for kinematical computations), and NDDS (for communication between the Robonaut and external software). In addition, RoboSim incorporates unique inverse-kinematical algorithms for chains of joints that have fewer than six degrees of freedom (e.g., finger joints). In comparison with the algorithms of OSCAR, these algorithms are more readily adaptable and provide better results when using equivalent sets of data.

Goza, S. Michael↗
NASA NTRS2018

Small Unmanned Aerial System (UAS) Flight Testing of Enabling Vehicle Technologies for the UAS Traffic Management Project

Small unmanned aerial systems (sUAS) have been studied and results indicate that there is a large array of highly-beneficial applications. These applications are too numerous to list, but include search and rescue, fire spotting, precision agriculture, etc. to name a few. Typically sUAS vehicles weigh less than 55 pounds and will be performing flight operations in the presence of manned aircraft and other sUAS. Certain sUAS applications, such as package delivery, will include operations in the close proximity of the general public. The full benefit from sUAS is contingent upon the resolution of several technological areas to enable free and widespread use of these vehicles. Technological areas in question include, but are not limited to: autonomous sense and avoid and deconfliction of sUAS from other sUAS and manned aircraft, communications and interfaces between the vehicle and human operators, and high-reliability autonomous systems. The NASA UAS Traffic Management (UTM) project is endeavoring to develop a traffic management system and concept of operations for these types of vehicles. An extensive sUAS flight test effort was performed to partially address vehicle-related technological areas and to shape an understanding of future developmental and test efforts for vehicles intended to use the UTM traffic management system. The flight testing described herein had the following objectives: 1) Install and test Dedicated Short Range Communications (DSRC) systems developed for the automotive industry for potential sense and avoid sUAS applications; 2) Evaluate the use of cellular 4G systems to provide vehicle control; 3) Obtain high-resolution video imagery in support of image-based optical detection sense and avoid systems; 4) Acquire data in fixed-wing flight to support validation and maturation of an autonomous range containment system known as Safeguard in fixed-wing flight. A total of 53 flights were performed over 12 operational days at Beaver Dam Airpark in Elberon, VA. This work was sponsored by the UTM project that is part of the Aviation Operations and Safety Program (AOSP) at NASA.

Glaab, Louis J.↗
NASA NTRS2018

Safe2Ditch Autonomous Crash Management System for Small Unmanned Aerial Systems: Concept Definition and Flight Test Results

Small unmanned aerial systems (sUAS) have the potential for a large array of highly-beneficial applications. These applications are too numerous to comprehensively list, but include search and rescue, fire spotting, precision agriculture, etc. to name a few. Typically sUAS vehicles weigh less than 55 lbs and will be performing flight operations in the National Air Space (NAS). Certain sUAS applications, such as package delivery, will include operations in the close proximity of the general public. The full benefit from sUAS is contingent upon the resolution of several technological areas in order to provide an acceptable level of risk for widespread sUAS operations. Operations of sUAS vehicles pose risks to people and property on the ground as well as manned aviation. Several of the more significant sUAS technological areas include, but are not limited to: autonomous sense and avoid and deconfliction of sUAS from other sUAS and manned aircraft, communications and interfaces between the vehicle and human operators, and the overall reliability of the sUAS and constituent subsystems. While all of the technological areas listed contribute significantly to the safe execution of the sUAS flight operations, contingency or emergency systems can greatly contribute to sUAS risk mitigations to manage situations where the vehicle is in distress. The Safe2Ditch (S2D) system is an autonomous crash management system for sUAS. Its function is to enable sUAS to execute emergency landings and avoid injuring people on the ground, damaging property, and lastly preserving the sUAS and payload. A sUAS flight test effort was performed to test the integration of sub-elements of the S2D system with a representative sUAS multi-rotor.

Glaab, Louis J.↗
NASA NTRS2023

Ch. 12. A Theoretical Approach to Management of Limited Attentional Resources to Support the m:N Operation in Advanced Air Mobility Ecosystem

Advanced air mobility (AAM) technologies incorporate increasingly autonomous systems that allow fully remote, independent, and intelligent operation of air vehicles to support the transportation of goods and passengers within and across urban and rural areas. With a myriad of automated technologies enabling the AAM ecosystem, the human operator’s role will likely be a passive supervisory monitor of the air vehicles, involving increasingly fewer humans (m) that manage many more autonomous systems (N), or m:N operations. Unfortunately, the human performance literature suggests that human operators will exhibit poor supervision of numerous autonomous agents due to the limits of attentional resources in the operators. In the general human information-processing model, a human operator exercises a limited pool of attentional resources to engage various information-processing stages including detecting, perceiving, comprehending, and predicting objects around them. Yamani and Horrey (2018) expanded the human information-processing model to characterize a tradeoff between information-processing demand and resource relief that automation brings in the context of automated driving. In their model, a driver interacting with an automated driving system is assumed to reallocate resources “freed” by automation to support other information-processing stages required for successful task performance. A future AAM ecosystem enabled by an orchestration of advanced automated systems, however, requires a single operator to interact with more than one air vehicle with varying levels and degrees of automated systems, making the traditional framework of human-automation interaction insufficient. To address this gap, we provide a review of the literature on situation assessment and trust, two constructs identified as critical for a fuller understanding of intimate and intricate interactions between a human operator and multiple air vehicles equipped with increasingly autonomous systems. Then, we propose an expansion of Yamani and Horrey’s (2018) model to motivate systematic research on the human operator’s role, identify factors that influence resource allocation and guide human-centered design of an interface supporting the m:N operation in the AAM environment.

Advanced Air Mobility↗
NASA NTRSDate not supplied

A Web-Based Negotiation Tool for Conflict Resolution in Upper Class E Traffic Management

In Upper Class E airspace 60,000 feet (or Flight level / FL600), vehicles such as High Altitude Long Endurance (HALE) balloons and slow fixed-wing gliders have diverse vehicle characteristics and limited maneuverability. Due to these unique characteristics of high-altitude operations, a new type of strategic negotiation-based conflict resolution method has been proposed to avoid potential conflict between vehicles in Upper Class E airspace, well in advance of the conflict point and with ample time for bilateral negotiation. This paper introduces the first real-time web-based negotiation tool for high-altitude operations. To facilitate negotiation, this tool incorporates a bilateral negotiation model with an algorithm to assess conflict risks and generate new flight trajectories with a given flight path deviation. This tool allows users to make decisions during negotiation manually and automatically while enforcing the needed constraints for negotiation to be successful. Operators can make decisions at every step while they interact with each other on separate devices. To conduct sensitivity analysis, a mechanism that automates human inputs to the user interface is also developed for this web-based negotiation tool, such that fast-time simulations can be constructed and used to explore various scenarios to gain insights into this web-based negotiation model. Using this tool, a study was conducted to evaluate the impact of different negotiation strategies, vehicle types, vehicle crossing angles, and operator response times during the negotiations on metrics such as total negotiation completion time, number of negotiation rounds, and extra flight distance to avoid the conflict due to negotiation, compared to ones without negotiation. The results suggest that operators benefit from using negotiated flight paths with quick response times, across various crossing angles and vehicle types. Various Negotiation Strategies are investigated to simulate the behaviors associated with different types of negotiations. The findings demonstrate that, in comparison to the conventional method where a single operator assumes full responsibility, negotiation-based strategic deconfliction reduces the total extra flight distance by an average of 24% - 27%. On an individual basis, each operator may be able to save an average of 65% of their extra flight distance.

Upper Class E Traffic Management (ETM)↗
DOE OSTI2025

Data Centers and Digital Assurance Workshop 2 – Prioritizing Digital Assurance Challenges, Session 2

The second session of the TADA (Technical Assistance for Digital Assurance) Data Centers Cohort, held on November 10, 2025, focused on prioritizing digital assurance challenges at the intersection of data centers and the electric grid. Building on the foundational concepts introduced in Workshop 1, this session deepened the application of the Threat–Vulnerability–Consequence (TVC) framework and emphasized the urgency of addressing cybersecurity, supply chain integrity, and operational reliability. Participants explored the growing convergence of digital and physical systems, the expanding attack surface due to global supply chain dependencies, and the implications of AI-driven load behavior. Real-world incidents—including the Volt Typhoon campaign and vulnerabilities in Solarman and Deye platforms—were analyzed to illustrate the risks of unpatched systems, insecure APIs, and inadequate vendor oversight. Key themes included architecture and interface weaknesses, governance gaps, and human and procedural shortcomings. The workshop also examined the evolving regulatory landscape, highlighting new federal mandates around Foreign Entity of Concern (FEOC) compliance and large-load reliability standards. Through interactive exercises, stakeholders ranked and mapped digital assurance risks from their respective perspectives—utilities, operators, and vendors—laying the groundwork for mitigation strategies and shared accountability models to be developed in Workshop 3. Session 2 of 3.

24 - POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1987

Mixed-initiative control of intelligent systems

Mixed-initiative user interfaces provide a means by which a human operator and an intelligent system may collectively share the task of deciding what to do next. Such interfaces are important to the effective utilization of real-time expert systems as assistants in the execution of critical tasks. Presented here is the Incremental Inference algorithm, a symbolic reasoning mechanism based on propositional logic and suited to the construction of mixed-initiative interfaces. The algorithm is similar in some respects to the Truth Maintenance System, but replaces the notion of 'justifications' with a notion of recency, allowing newer values to override older values yet permitting various interested parties to refresh these values as they become older and thus more vulnerable to change. A simple example is given of the use of the Incremental Inference algorithm plus an overview of the integration of this mechanism within the SPECTRUM expert system for geological interpretation of imaging spectrometer data.

Borchardt, G. C.↗
NASA NTRS2004

Robots and humans: synergy in planetary exploration

How will humans and robots cooperate in future planetary exploration? Are humans and robots fundamentally separate modes of exploration, or can humans and robots work together to synergistically explore the solar system? It is proposed that humans and robots can work together in exploring the planets by use of telerobotic operation to expand the function and usefulness of human explorers, and to extend the range of human exploration to hostile environments. Published by Elsevier Ltd.

Robotics↗
NASA NTRSDate not supplied

Enabling a Voice Management System for Space Applications

The sustainable missions beyond Low Earth Orbit (LEO) envisioned for NASA’s Artemis program will require autonomous capabilities. Moreover, Artemis mission crews will need a means to efficiently interact with a spacecraft’s autonomous systems. This interaction can be facilitated by voice and speech communications because voice-based controls enable users to interact hands- and eyes-free, allowing the user to better focus on critical tasks. The goal of our project was to explore the knowledge and technology needed to successfully design effective Voice User Interfaces (VUIs) for autonomous systems utilizing Human Centered Design (HCD) principles. The focus of the human factors’ aspect of engineering, pays close attention to psychological and physiological principles in the development of autonomous crew operation systems. A main objective was to understand how a crew member, through voice interaction, could efficiently and intuitively communicate with a notional autonomous vehicle system manager. This project was a part of the NASA Moon to Mars eXploration Systems and Habitation (M2M X-Hab) 2020 Academic Innovation Challenge. The work from the BLiSS Team, at the University of Michigan, resulted in the design of a system persona, Diego, to which an astronaut may quickly build trust with autonomous systems, to alleviate known stressors on mental health expected during long duration space missions. Optimal software to facilitate integration of the system persona into a reference Lunar orbiting Gateway station was defined. Additionally, a Speech to Text (STT) system and a Graphical User Interface (GUI) that could be implemented in future missions was developed on an Internet of Things (IOT) platform. The Voice User Interface (VUI) design for the M2M X-Hab 2020 project leveraged previous technology developed by the BLiSS team to incorporate a voice-based interface into NASA’s Platform for Autonomous Systems (NPAS) software. This required technologies to convert voice to text, conduct semantic interpretations, and convert responses from the autonomous system to text and to speech; additionally, the spacecraft background noise environment was assessed, a noise mitigation technique was developed, and a relatable personality for the autonomous system was developed in order to facilitate human-like conversations. The success of our effort was largely due to the diversity of the team that included expertise in Space Systems Engineering, Human Computer Interaction, Aerospace Engineering, Computer Science, Biomedical Engineering, and Applied Physics. The diverse perspectives fostered elaborate discussions, resulting in the conception of three main subsystems: (1) User-System, (2) NPAS-System, and (3) Environment-System. The VUI was unique and had to be efficient and intuitive. For this project, 5 subteams were formed, each with a separate objective, Voice Design team, Background Noise Mitigation team, Software Integration team and Graphical User Interface team. The BLiSS team crafted a personality for the VUI to enable human-like conversation and drive user adoption and trust. User surveys were completed and used to help determine the required VUI system personality traits by capturing perspectives and expectations of prospective “Artemis Generation Astronauts”. To further simulate human-like conversations, the system had to be able to quickly interpret user speech and be able to integrate with NASA’s NPAS platform for quick and reliable information transfer. The outcomes of our research were: (1) a working prototype user interface, that is compatible with NASA’s NPAS platform; (2) software that demonstrates the ability of the VUI system to interpret user requests and respond appropriately; (3) the capability to implement fully expanded conversations between user and system using intuitive communication in four request categories; and (4) software and hardware recommendations that optimize the system’s ability to operate in a noisy environment. Our research has laid the foundation for the development of VUI’s for autonomy, and provides a baseline for future VUI developments.

Voice user interface↗
DOE OSTI2024

Synergizing human expertise and AI efficiency with language model for microscopy operation and automated experiment design

With the advent of large language models (LLMs), in both the open source and proprietary domains, attention is turning to how to exploit such artificial intelligence (AI) systems in assisting complex scientific tasks, such as material synthesis, characterization, analysis and discovery. Here, we explore the utility of LLMs, particularly ChatGPT4, in combination with application program interfaces (APIs) in tasks of experimental design, programming workflows, and data analysis in scanning probe microscopy, using both in-house developed APIs and APIs given by a commercial vendor for instrument control. We find that the LLM can be especially useful in converting ideations of experimental workflows to executable code on microscope APIs. Beyond code generation, we find that the GPT4 is capable of analyzing microscopy images in a generic sense. At the same time, we find that GPT4 suffers from an inability to extend beyond basic analyses for more in-depth technical experimental design. We argue that an LLM specifically fine-tuned for individual scientific domains can potentially be a better language interface for converting scientific ideations from human experts to executable workflows. Such a synergy between human expertise and LLM efficiency in experimentation can open new doors for accelerating scientific research, enabling effective experimental protocols sharing in the scientific community.

97 MATHEMATICS AND COMPUTING↗