SEARCH · Search NASA

Results for “methods for gesture based robot control,”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

3 records

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Teleoperation of Construction Robots Using Human Interaction: Case Study Examples: Preprint

The integration of advanced teleoperation methods in construction robotics has the potential to enhance productivity and safety in demanding environments. Among different methods, hand gestures have emerged as an intuitive approach, enabling seamless communication between human operators and robotic systems. This study demonstrates the use of hand gestures for controlling a model robotic excavator and a virtual robotic arm in construction applications. Two case study examples are presented. In the first case study, a series of hand gestures are performed to control a model robotic excavator's bucket and arm movements. The second case study focuses on using hand gestures to control a virtual robotic arm to perform a material handling task in a simulated environment. In both cases, the performed hand gestures are all successfully captured and interpreted by the gesture-based teleoperation method to conduct the corresponding tasks, which demonstrates the application of the gesture-based control method for various types of construction robots. Also, the limitations of the gesture-based teleoperation method (e.g., recognition delay, sensitivity to motion outliers) are discussed. Future work will focus on developing a digital twin system to support gesture-based teleoperation for a wider range of construction robots.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS2013

Gesture-Based Robot Control with Variable Autonomy from the JPL Biosleeve

This paper presents a new gesture-based human interface for natural robot control. Detailed activity of the user's hand and arm is acquired via a novel device, called the BioSleeve, which packages dry-contact surface electromyography (EMG) and an inertial measurement unit (IMU) into a sleeve worn on the forearm. The BioSleeve's accompanying algorithms can reliably decode as many as sixteen discrete hand gestures and estimate the continuous orientation of the forearm. These gestures and positions are mapped to robot commands that, to varying degrees, integrate with the robot's perception of its environment and its ability to complete tasks autonomously. This flexible approach enables, for example, supervisory point-to-goal commands, virtual joystick for guarded teleoperation, and high degree of freedom mimicked manipulation, all from a single device. The BioSleeve is meant for portable field use; unlike other gesture recognition systems, use of the BioSleeve for robot control is invariant to lighting conditions, occlusions, and the human-robot spatial relationship and does not encumber the user's hands. The BioSleeve control approach has been implemented on three robot types, and we present proof-of-principle demonstrations with mobile ground robots, manipulation robots, and prosthetic hands.

BioSleeve↗

NASA NTRS2004

Human-Robot Planetary Exploration Teams

The EVA Robotic Assistant (ERA) project at NASA Johnson Space Center studies human-robot interaction and robotic assistance for future human planetary exploration. Over the past four years, the ERA project has been performing field tests with one or more four-wheeled robotic platforms and one or more space-suited humans. These tests have provided experience in how robots can assist humans, how robots and humans can communicate in remote environments, and what combination of humans and robots works best for different scenarios. The most efficient way to understand what tasks human explorers will actually perform, and how robots can best assist them, is to have human explorers and scientists go and explore in an outdoor, planetary-relevant environment, with robots to demonstrate what they are capable of, and roboticists to observe the results. It can be difficult to have a human expert itemize all the needed tasks required for exploration while sitting in a lab: humans do not always remember all the details, and experts in one arena may not even recognize that the lower level tasks they take for granted may be essential for a roboticist to know about. Field tests thus create conditions that more accurately reveal missing components and invalid assumptions, as well as allow tests and comparisons of new approaches and demonstrations of working systems. We have performed field tests in our local rock yard, in several locations in the Arizona desert, and in the Utah desert. We have tested multiple exploration scenarios, such as geological traverses, cable or solar panel deployments, and science instrument deployments. The configuration of our robot can be changed, based on what equipment is needed for a given scenario, and the sensor mast can even be placed on one of two robot bases, each with different motion capabilities. The software architecture of our robot is also designed to be as modular as possible, to allow for hardware and configuration changes. Two focus areas of our research are safety and crew time efficiency. For safety, our work involves enabling humans to reliably communicate with a robot while moving in the same workspace, and enabling robots to monitor and advise humans of potential problems. Voice, gesture, remote computer control, and enhanced robot intelligence are methods we are studying. For crew time efficiency, we are investigating the effects of assigning different roles to humans and robots in collaborative exploration scenarios.

Tyree, Kimberly↗