Search NASA⌕ Search

SEARCH · Search NASA

Results for “teleoperation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Teleoperation of Construction Robots Using Human Interaction: Case Study Examples: Preprint

The integration of advanced teleoperation methods in construction robotics has the potential to enhance productivity and safety in demanding environments. Among different methods, hand gestures have emerged as an intuitive approach, enabling seamless communication between human operators and robotic systems. This study demonstrates the use of hand gestures for controlling a model robotic excavator and a virtual robotic arm in construction applications. Two case study examples are presented. In the first case study, a series of hand gestures are performed to control a model robotic excavator's bucket and arm movements. The second case study focuses on using hand gestures to control a virtual robotic arm to perform a material handling task in a simulated environment. In both cases, the performed hand gestures are all successfully captured and interpreted by the gesture-based teleoperation method to conduct the corresponding tasks, which demonstrates the application of the gesture-based control method for various types of construction robots. Also, the limitations of the gesture-based teleoperation method (e.g., recognition delay, sensitivity to motion outliers) are discussed. Future work will focus on developing a digital twin system to support gesture-based teleoperation for a wider range of construction robots.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2018

Teleoperation for Nuclear Applications Considering Communication Delay and Uncertainty - 18205

Teleoperation in remote environment is of great interest in robotics. Advancement in communication technology has enabled safe execution of complex and life threatening tasks in network environment in teleoperation. Using bilateral teleoperation, the operator receives force feedback from the slave which provides an interactive feel of forces and smoother coordination. However, time delay across the communication link in sending the command signal from the master to the slave may cause instability to the system. In this research, we use the continuous pole placement method to address the time delay problem in teleoperation. Compared to approximation based and prediction based methods, this method shows improved robustness. Simulations and experiments show a stable and well synchronized control. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2018

Soft Robotic Glove System for Wearable Haptic Teleoperation - 18373

Managing nuclear waste is a difficult, dangerous, and costly task. Teleoperation of robotic operators is an important functionality for this application, allowing robotic devices to operate in dangerous environments while still leveraging the existing abilities and technical knowledge of human workers. One difficulty in implementing effective teleoperation systems is that human operators rely on more senses beside sight: in particular, touch. The sense of touch could provide significant benefits for teleoperation, preventing environmental mishaps that could cost millions of dollars. To help remedy this problem, we present a soft haptic feedback glove system. The primary functionality of this system is to provide tactile and force feedback on the user's hands. This was achieved using custom cylindrical pneumatic actuators. These actuators consist of plastic sheeting folded into a hollow-walled cylinder which is connected to a pressure source. When pressurized, the cylinder walls expand and the cylinder straightens, with the force being related to the pressure. The glove system has one of these actuators over each of the user's three main fingers (index, middle, and thumb), where pressurizing the actuator applies a reaction force to the finger of the user, mimicking the resistance of a grasped object. The expansion of these actuators also causes them to physically press against the user's fingers, giving them the feeling of contact. To demonstrate the usability of this system, we implemented a virtual environment in V-REP simulating the interactions between a robotic Jaco Arm and a floating ball. This simulation receives hand and figure positions from the haptic glove and used them to drive the state of the simulated Jaco Arm. In response to the interactions between the Jaco and the floating ball, the simulation returns commands to the glove, instructing it to pressurize or depressurize its actuators as appropriate, allowing the user to feel the ball in their hand. We tested users in their ability to prevent the ball from floating away using this system with and without the haptic feedback. Results show that the addition of haptic feedback significantly improved the ability of users to interact precisely with the environment. We also used the finger positions of the haptic glove to control a physical robotic hand, and demonstrated the system's usability for control of real robots, though the lack of sensors on the robot made providing the tactile feedback impractical. However, there is more to grasping and holding an object than just the feeling of contact and the resistance to closing one's hand. The weight of the object will apply forces to the user's other joints as they hold it. To this end, we added a haptic feedback device for the user's wrist, a technique that can be generalized to the user's other joints. This is made up of reverse Pneumatic Artificial Muscles (rPAMs) stretching across the wrist between mounts on the hand and forearm. These rPAM actuators are tubes of silicone wrapped in thread such that they extend when pressurized. This extension can apply torque on the user's wrist, mimicking the feeling of gravity, or allowing the simulation of additional contact forces (such as the user's hand impacting an object). We tested this wrist device independently using a simulated path-following experiment. Users showed a significant improvement in performance when aided by haptic feedback during sinusoidal path experiments, demonstrating the usability of this approach. We then integrated the two systems, further demonstrating the usability of this approach. This project represents the first step towards a more effective robotic teleoperation system, allowing skilled human workers to perform tasks more effectively while remaining out of danger. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Performance Evaluation of Augmented Teleoperation of Contact Manipulation Tasks

At Argonne National Laboratory, an enhanced telerobotic concept has been suggested by providing augmented virtual fixtures for precise and dexterous manipulation. The proposed technology incorporated a human-robot interface that can assist personnel on the processes of Deactivation and Decommissioning in sites of high radiation. It has been implemented on a dual-arm robot system, known as Baxter, by integrating the Robotic Operating System (ROS) with 3D sensors and visual-haptic operator interface. However, what remained before its practical utilization is the capability for precisely placing the virtual fixtures. To this end, the objectives involved camera calibration, which required hardware fabrication and setup, as well as modification and implementation of ROS. The goal of this Research and Development is to develop an augmented-reality- based operator interface for human-inclusive, intelligent, and autonomous robotic system for the Deactivation and Decommissioning (D and D) of radioactive sites. The objectives are to develop: Collaborative Robot; Multi-modal Mixed-Reality Human-Robot Interface; Intelligent and Autonomous Behaviors. In order to achieve this, Baxter's Head-Kinect camera and Left-Hand camera need to be calibrated to overlap the real environment with the simulated environment. Camera calibration, which is crucial for precisely placing the virtual fixtures, was successful after running the simulations. The real and simulated environments overlapped, permitting the utilization of augmented teleoperation and the dexterous manipulation of Baxter for D and D applications in nuclear sites. Future steps: Performance evaluation of augmented teleoperation of contact manipulation tasks: Multiple-Camera 3D Reconstruction and Automation. Oculus Interface: Augmented reality and machine learning for enhanced telerobotic operation (Augmented Tele-autonomy): Machine-learning techniques for complex and flexible autonomous robotic behaviors. Interactive remote operation of robotic systems. Mesh cutting with simulation and real robot.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

A Passivity-Shortage Based Control Design for Teleoperation With Time-Varying Delays

This letter investigates the effect of time-varying delays in bilateral teleoperation, with respect to stability and performance, using the properties of passivity-shortage. Until recently, the desired stability and performance characteristics were achieved using the concept of passivity. However, passivity is limited to systems of relative degree zero or one, while passivity-shortage includes systems of higher relative degrees and possibly of non-minimum phase. Passivity-shortage also arises naturally when data transmission is subject to delays, either constant or time-varying. In this letter, the properties of passivity-shortage are employed to design a simple negative feedback controller. We show that the proposed method provides a faster responding solution and improved performance compared to the existing approaches. The performance improvements include improved steady-state error convergence, and robustness against environmental disturbances, even in the presence of varying delays.

42 ENGINEERING↗
DOE OSTI2018

Remotely Teleoperating a Humanoid Robot to Perform Fine Motor Tasks with Virtual Reality - 18446

In this paper, we describe our ongoing work to develop cooperative control of NASA's R5 Valkyrie humanoid robot for performing dexterous manipulation tasks inside gloveboxes commonly found in many nuclear facilities. These tasks can be physically demanding and provide some element of risk to the operator when done by a person in situ. For example, if a glove is ruptured, the operator could be exposed to radioactive material. In many cases, the operator has low visibility and is unable to reach the entire task space, requiring the use of additional tools located inside the glovebox. Such tasks include cleaning particulate from inside the glovebox via sweeping or vacuuming, separating a specific amount of a compound to be weighed on a scale, or grasping and manipulating objects inside the glovebox. There is potential to move the operator to a nearby, safe location and instead place a humanoid robot in the potentially hazardous environment. However, teleoperating a humanoid robot to perform dexterous tasks at a comparable level to a human hand remains a difficult problem. Previous work for controlling humanoid robots often involves one or more operators using a standard 2D display with a mouse and keyboard as controllers. Successful interfaces use sensor fusion to provide information to the operator for increased situation awareness, but these designs have limitations. Gaining proper situation awareness by visualizing 3D information on a 2D screen requires time and increases the cognitive load on the user. Instead, if the operator is able to visualize and control the robot properly in three dimensions, it can increase situation and task awareness, reduce task time, reduce the chance of mistakes, and increase the likelihood of overall task success. We propose a two-part system that combines an HTC Vive virtual reality headset with either the Vive handheld controllers, or the Manus VR wearable gloves as the primary control. The operator wears the headset in a remote location and can visualize a reconstruction of the glovebox, created live by sensor scans from the robot and with sensors located inside the glovebox for a perspective traditionally unavailable to operators. By using the controllers or gloves to control the humanoid robots hands directly, they can plan actions in the virtual reconstruction. When the operator is satisfied with the plan, the actions are sent to the real robot. To test this system we have created a mockup of a glovebox that is accessible by Valkyrie, as well as several tasks that are a subsample of the tasks that might be required when working in a real glovebox. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2018

3D Reconstruction, Augmented Reality, and Telerobotic Operation - 18297

For nuclear facility decommissioning and waste cleanup, it is necessary to deploy simple and robust robots in remote sites performing precise and dexterous manipulation under teleoperation. To meet the challenge, this paper presents a new remote operation method based on augmented reality for intuitive and efficient teleoperation. The enhanced teleoperation system incorporates three technology components - 3D sensing and reconstruction, augmented reality, and teleoperation. The 3D reconstruction technology utilizes 3D cameras and constructs a 3D geometric model of the environment. Compared to the state-of-the- art technology, the presented 3D reconstruction has greater spatial precision and dynamic object tracking capability, which enhances the precision of manipulation tasks in a dynamic environment. The augmented reality technology entails generation of virtual surfaces of various geometries and dynamic characteristics, as well as consistent perceptual overlay in camera, robot and operator hand control spaces. Teleoperation is demonstrated with a dual-arm robot system operated under Robot Operating System (ROS). (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2026

A Modular Accelerator Robotics Framework for AD Robotics

Accelerator tunnels, such as the ones at Fermilab, remain highly radioactive after beam shutoff due to induced radiation from the beam. This residual radiation creates a hazardous environment for manual inspection and repair of beamline components. To minimize worker radiation dose and reduce beam downtime, the AD Robotics Initiative previously built a fleet of low-cost custom mobile robots. However, the custom Python sockets server-client architecture lacked standardization, causing development delays and complicating the integration of new sensors and actuators. Here, we developed a modular system using ROS2 and Docker to standardize the teleoperation and control interfaces. This system was validated by implementing a teleoperation controller with real-time, low-latency, and high-definition video feedback. The aim of this framework is for a new feature or even a robot to be integrated into the system simply by documenting the hardware configuration. Current integration of LiDAR, Odometry, and Depth Cameras provides the foundation for Simultaneous Localization and Mapping (SLAM) tasks. Finally, future work involves integration into the accelerator control system and the attachment of a 6 degree-of-freedom robotic arm for telemanipulation.

Rayyan Khan, M. [Fermilab; Rensselaer Poly.; Unlis↗
DOE OSTI2018

Remote Aerial Inspection Vehicle Design and Evaluation - 18428

This work details the system integration of an aerial vehicle with advanced capabilities for precise inspection in confined, unstructured environments. High levels of uncertainty and varying environmental conditions are significant challenges that degrade the performance of robot subsystems, including state estimation and navigation. The methods presented in this paper outline three methodologies to minimize uncertainty and maintain fast, safe navigation in cluttered, enclosed environments: (i) a state estimation framework that performs sensor fusion of laser odometry and localization using an Unscented Kalman Filter (ii) Robust Experience-driven Predictive Control, an adaptive control strategy to bound state uncertainty and guarantee safety in the presence of model perturbations (iii) an adaptive teleoperation scheme to generate smooth control inputs and infer user intent for an enhanced teleoperation interface. These methodologies are evaluated individually and implemented on a micro-aerial vehicle for testing with varying levels of noise and state uncertainty representing the conditions within the target domain. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2018

TeMoto 2.0: Source Agnostic Command-to-Task Architecture Enabling Increased Autonomy in Remote Systems - 18438

Tele-operated robots provide safe means for executing tasks in hazardous environments such as underground mines, space, disaster and legacy nuclear facility decommissioning sites, etc. The desire to use these systems in increasingly complex and dynamic environments presents a variety of technical and educational challenges including (i) insufficient field knowledge of system developers, (ii) restricted data bandwidth, and (iii) insufficient operator situational awareness. Lack of field knowledge is commonly addressed by actively educating robotics engineers in the perspective areas. The Nuclear Robotics Group (NRG) at the University of Texas at Austin trains students to have the necessary skill and knowledge in both the robotic and nuclear domains. To address the second two deficiencies, the NRG has developed TeMoto: a speech and hand gesture based supervisory teleoperation software. TeMoto addresses several chronic technical issues associated with most teleoperation systems including the operator's situational awareness, communication delays and interface intuitiveness. This paper reviews recent efforts to update and extend the TeMoto architecture (TeMoto 2.0) to support more complex semi-autonomous behaviors and eliminate the unnecessary and confining concept of a human interface as a separate component from the rest of the robot's sensors and actuators. This paper outlines the key architectural changes and demonstrations using a dual-arm mobile manipulator to complete semi-autonomous tasks. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2026

Modular Accelerator Robotics Framework Implementation For Accelerator Inspection

Accelerator tunnels, such as the ones at Fermilab, remain highly radioactive after beam shutoff due to induced radiation from the beam. This residual radiation creates a hazardous environment for manual inspection and repair of beamline components. To minimize worker radiation dose and reduce beam downtime, the AD Robotics Initiative previously built a fleet of low-cost custom mobile robots. However, the custom Python sockets server-client architecture lacked standardization, causing development delays and complicating the integration of new sensors and actuators. Here, we developed a modular system using ROS2 and Docker to standardize the teleoperation and control interfaces. This system was validated by implementing a teleoperation controller with real-time, low-latency, and high-definition video feedback. The aim of this framework is for a new feature or even a robot to be integrated into the system simply by documenting the hardware configuration. Current integration of LiDAR, Odometry, and Depth Cameras provides the foundation for Simultaneous Localization and Mapping (SLAM) tasks. Finally, future work involves integration into the accelerator control system and the attachment of a 6 degree-of-freedom robotic arm for telemanipulation.

Rayyan Khana, M. [Unlisted, US, IL] (ORCID:0009000↗
DOE OSTI2026

Toward Fully Soft and Multifunctional Shape Sensing via Optical Waveguide Arrays

For soft bodies, surface deformation and pressure provide proprioceptive and exteroceptive information, including body configuration, body compliance, and external forces. We develop a sheet sensor with a fully soft sensing surface that provides surface shape reconstruction using optical waveguide arrays. The waveguides are fabricated to achieve a tunable linear response to bi‐directional bending curvature, and the waveguide arrays are configured to differentiate between ambiguous shapes. We characterize the waveguide performance, relating curvature sensitivity to the core's surface roughness. Synergy of waveguide responses reduces the number of sensing elements required and achieves damage resilience. Using waveguide sensitivity to pressure, we also demonstrate feasibility for exteroception. We demonstrate the multifunctional sensing capability by wrapping the sheet around an upper arm, showcasing joint motion and external force sensing. Integrated into or applied onto surfaces of robotic or living systems, this design can be implemented in applications such as virtual reality, teleoperation, physical therapy, and soft robotics.

Yu, Qifan [Department of Mechanical Engineering Ma↗
DOE OSTI2021

Remote Target Engagement System (RTES) Generation 4 ("Gen 4")

RTES is designed to provide teleoperation of remote weapon platform capability at defensive physical protection locations requiring active lethal capabilities for final access denial. Multiple operator consoles allow for coordinated engagement of many weapon platforms by one or more operator consoles. This results in faster response times, increased effectiveness, and increased survivability from removing response force from harm's way increasing survivability. This software is developed against DoD requirements and has been safety and functionally reviewed and approved for operational use. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-1935 O

Klingler, KristopherR.↗
DOE OSTI2023

Exploiting Task Tolerances in Mimicry-based Telemanipulation

We explore task tolerances, i.e., allowable position or rotation inaccuracy, as an important resource to facilitate smooth and effective telemanipulation. Task tolerances provide a robot flexibility to generate smooth and feasible motions; however, in teleoperation, this flexibility may make the user’s control less direct. In this work, we implemented a telema nipulation system that allows a robot to autonomously adjust its configuration within task tolerances. We conducted a user study comparing a telemanipulation paradigm that exploits task tolerances (functional mimicry) to a paradigm that requires the robot to exactly mimic its human operator (exact mimicry), and assess how the choice in paradigm shapes user experience and task performance. Our results show that autonomous adjustments within task tolerances can lead to performance improvements without sacrificing perceived control of the robot. Additionally, we find that users perceive the robot to be more under control, predictable, fluent, and trustworthy in functional mimicry than in exact mimicry.

42 ENGINEERING↗
DOE OSTI2025

Testing, Calibration, and UxS Integration of the Kromek GR1 Plus CZT Gamma Spectrometer

Collecting radiation measurements can be a hazardous task, especially in the presence of highly active sources. Various scenarios necessitate source search, classification, and quantification, often with limited or no a priori information. Activities such as disaster mitigation, emergency response, environmental monitoring, and site remediation may involve dangerous radioactive sources. In these situations, there is a pressing need for remote monitoring capabilities that protect human operators from potential harm and enhance adherence to the principle of “As Low as Reasonably Achievable” (ALARA) for radiation doses. The first step toward achieving remote radiation measurement capabilities is the remote operation of a radiation sensor. Once this milestone is reached, the next challenge is to integrate this remote sensing capability into suitable actuation agents, collectively referred to as uncrewed systems (UxS). These systems include familiar platforms such as robotic quadrupeds, aerial multirotor vehicles, and ground vehicles, any of which may be teleoperated, act autonomously, or utilize a combination of both. A critical factor in achieving remote radiation sensing is the availability of data from the appropriate sensor. Many commercially available radiation sensors have closed-source documentation for their communication protocols. Typical end-user products are often self-contained, handheld devices designed for manual measurement scenarios. While there are commercial off-the-shelf (COTS) integrations of radiation sensors with UxS available for purchase, these solutions are typically tailored for specific use cases and may not meet the requirements of different applications. This paper discusses efforts to remotely acquire radiation measurements from a small form-factor CZT gamma spectrometer. Sandia National Laboratories has successfully demonstrated the initial capability to integrate low size, weight, and power (SWaP) gamma spectroscopy into various UxS, alongside co-located GPS data logging and sensor calibration and qualification. With remote gamma spectroscopy achieved, the stage is set for UxS integration of this capability.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2018

Integrating Risk in Humanoid Robot Control for Applications in the Nuclear Industry - 18230

This paper discuss the integration of risk into a robot control framework for decommissioning applications in the nuclear industry. Our overall objective is to allow the robot to evaluate a risk associated with several methods of completing the same task by combining a set of action sequences. If the environment is known and in the absence of sensing errors each set of actions would successfully complete the task. In this paper, instead of attempting to model the errors associated with each sensing system in order to compute an exact solution, a set of solutions are obtained along with a prescribed risk index. The risk associated with each set of actions can then be compared to possible payoffs or rewards, for instance task completion time or power consumption. This information is then sent to a high level decision planner, for instance a human teleoperator, who can then make a more informed decision regarding the robots actions. In order to illustrate the concept, we introduce three specific risk measures, namely, the collision risk and the risk of toppling and failure risk associated with grasping an object. We demonstrate the results from this foundational study of risk-aware compositional robot autonomy in simulation using NASA's Valkyrie humanoid robot, and the grasping simulator HAPTIX. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗