Search NASA⌕ Search

SEARCH · Search NASA

Results for “visual odometry”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

UAV Research at NASA Langley: Towards Safe, Reliable, and Autonomous Operations

Unmanned Aerial Vehicles (UAV) are fundamental components in several aspects of research at NASA Langley, such as flight dynamics, mission-driven airframe design, airspace integration demonstrations, atmospheric science projects, and more. In particular, NASA Langley Research Center (Langley) is using UAVs to develop and demonstrate innovative capabilities that meet the autonomy and robotics challenges that are anticipated in science, space exploration, and aeronautics. These capabilities will enable new NASA missions such as asteroid rendezvous and retrieval (ARRM), Mars exploration, in-situ resource utilization (ISRU), pollution measurements in historically inaccessible areas, and the integration of UAVs into our everyday lives all missions of increasing complexity, distance, pace, and/or accessibility. Building on decades of NASA experience and success in the design, fabrication, and integration of robust and reliable automated systems for space and aeronautics, Langley Autonomy Incubator seeks to bridge the gap between automation and autonomy by enabling safe autonomous operations via onboard sensing and perception systems in both data-rich and data-deprived environments. The Autonomy Incubator is focused on the challenge of mobility and manipulation in dynamic and unstructured environments by integrating technologies such as computer vision, visual odometry, real-time mapping, path planning, object detection and avoidance, object classification, adaptive control, sensor fusion, machine learning, and natural human-machine teaming. These technologies are implemented in an architectural framework developed in-house for easy integration and interoperability of cutting-edge hardware and software.

Davila, Carlos G.↗
NASA NTRS2019

Planetary Rover Simulation for Lunar Exploration Missions

When planning planetary rover missions it is useful to develop intuition and skills driving in, quite literally, alien environments before incurring the cost of reaching said locales. Simulators make it possible to operate in environments that have the physical characteristics of target locations without the expense and overhead of extensive physical tests. To that end, NASA Ames and Open Robotics collaborated on a Lunar rover driving simulator based on the open source Gazebo simulation platform and leveraging ROS (Robotic Operating System) components. The simulator was integrated with research and mission software for rover driving, system monitoring, and science instrument simulation to constitute an end-to-end Lunar mission simulation capability. Although we expect our simulator to be applicable to arbitrary Lunar regions, we designed to a reference mission of prospecting in polar regions. The harsh lighting and low illumination angles at the Lunar poles combine with the unique reflectance properties of Lunar regolith to present a challenging visual environment for both human and computer perception. Our simulator placed an emphasis on high fidelity visual simulation in order to produce synthetic imagery suitable for evaluating human rover drivers with navigation tasks, as well as providing test data for computer vision software development.In this paper, we describe the software used to construct the simulated Lunar environment and the components of the driving simulation. Our synthetic terrain generation software artificially increases the resolution of Lunar digital elevation maps by fractal synthesis and inserts craters and rocks based on Lunar size-frequency distribution models. We describe the necessary enhancements to import large scale, high resolution terrains into Gazebo, as well as our approach to modeling the visual environment of the Lunar surface. An overview of the mission software system is provided, along with how ROS was used to emulate flight software components that had not been developed yet. Finally, we discuss the effect of using the high-fidelity synthetic Lunar images for visual odometry. We also characterize the wheel slip model, and find some inconsistencies in the produced wheel slip behaviour.

Allan, Mark↗
NASA NTRS2019

Demonstration of Stereo Vision for Deorbit Descent and Landing

Planetary landers need to reduce velocity at low altitude for soft landing. Traditionally, estimating velocity and altitude has been performed with radar sensors whose performance meets the specific mission needs. There are not very many options for these sensors and they are difficult to include in a flight system either due to obsolescence, prohibitive cost or difficulty in accommodation. Recently, alternative sensing modalities are being pursued including Doppler LiDAR and vision. This paper describes results from a recent helicopter field test of a binocular stereo vision system for deorbit descent and landing applications. The system consisted of two 18.6 ̊ field of view cameras mounted 1.7m apart. Post processing of the images showed ranging accuracy better than 1% up to 500mand 17 cm/s velocimetry accuracy at 37m. For a flight system these images could be input into an FPGA-based processor which processes dense stereo and visual odometry in less than 1 second to achieve the stereo ranging frame rates required for soft landing. When coupled with vision based Terrain Relative Navigation this stereo system enables landing accuracies on the order of 10m.

Sternberg, David C.↗
NASA NTRS2017

Gaussian Mixture Models for Temporal Depth Fusion

Sensing the 3D environment of a moving robot is essential for collision avoidance. Most 3D sensors produce dense depth maps, which are subject to imperfections due to various environmental factors. Temporal fusion of depth maps is crucial to overcome those. Temporal fusion is traditionally done in 3D space with voxel data structures, but it can be approached by temporal fusion in image space, with potential benefits in reduced memory and computational cost for applications like reactive collision avoidance for micro air vehicles. In this paper, we present an efficient Gaussian Mixture Models based depth map fusion approach, introducing an online update scheme for dense representations. The environment is modeled from an ego-centric point of view, where each pixel is represented by a mixture of Gaussian inverse-depth models. Consecutive frames are related to each other by transformations obtained from visual odometry. This approach achieves better accuracy than alternative image space depth map fusion techniques at lower computational cost.

Matthies, Larry↗
NASA NTRS2019

Demonstration of stereo vision for deorbit descent and landing

This paper describes results from a recent helicopter field test of a binocular stereo vision system for deorbit descent and landing applications. The system consisted of two 18.6˚ field of view cameras mounted 1.7m apart. Post processing of the images showed ranging accuracy better than 1% up to 500m and 17 cm/s velocimetry accuracy at 37m. For a flight system these images could be input into an FPGA-based processor which processes dense stereo and visual odometry in less than 1 second to achieve the stereo ranging frame rates required for soft landing. When coupled with vision based Terrain Relative Navigation this stereo system enables landing accuracies on the order of 10m.

Johnson, Andrew E.↗
NASA NTRS2021

Online Photometric Calibration of Automatic Gain Thermal Infrared Cameras

Thermal infrared cameras are increasingly being used in various applications such as robot vision, industrial inspection and medical imaging, thanks to their improved resolution and portability. However, the performance of traditional computer vision techniques developed for electro-optical imagery does not directly translate to the thermal domain due to two major reasons: these algorithms require photometric assumptions to hold, and methods for photometric calibration of RGB cameras cannot be applied to thermal-infrared cameras due to difference in data acquisition and sensor phenomenology. In this paper, we take a step in this direction, and introduce a novel algorithm for online photometric calibration of thermalinfrared cameras. Our proposed method does not require any specific driver/hardware support and hence can be applied to any commercial off-the-shelf thermal IR camera. We present this in the context of visual odometry and SLAM algorithms, and demonstrate the efficacy of our proposed system through extensive experiments for both standard benchmark datasets, and real-world field tests with a thermal-infrared camera in natural outdoor environments.

Daftry, Shreyansh↗
NASA NTRS2022

Performance Analysis of Terrain Relative Navigation Using Blue Origin New Shepard Suborbital Flight Telemetry

As part of a NASA Tipping Point Partnership with Blue Origin to mature precision lunar landing technologies, two test flights of the Blue Origin New Shepard vehicle carrying a NASA-developed sensor suite were conducted on 10/13/2020 and 08/26/2021 at the West Texas Launch Site (LS-1). Part of the acquired datasets, comprising data from an inertial measurement unit and a downward facing camera, was postprocessed through a JPL-developed prototype Visual Odometry and Map Relative Localization software (TRNVOSIM), and compared against ground truth acquired by the host vehicle navigation system. In this paper, we provide a description of the algorithms, the test setup, and the processed results.

Pedrotty, Samuel M.↗
NASA NTRS2021

Multi-Resolution Elevation Mapping and Safe Landing Site Detection with Applications to Planetary Rotorcraft

In this paper, we propose a resource-efficient approach to provide an autonomous UAV with an on-board perception method to detect safe, hazard-free landing sites during flights over complex 3D terrain. We aggregate 3D measurements acquired from a sequence of monocular images by a Structure-from-Motion approach into a local, robot-centric, multi-resolution elevation map of the overflown terrain, which fuses depth measurements according to their lateral surface resolution (pixel-footprint) in a probabilistic framework based on the concept of dynamic Level of Detail. Map aggregation only requires depth maps and the associated poses, which are obtained from an on-board Visual Odometry algorithm. An efficient landing site detection method then exploits the features of the underlying multi-resolution map to detect safe landing sites based on the slope, roughness, and quality of the reconstructed terrain surface. The evaluation of the performance of the mapping and landing site detection modules are analyzed independently and jointly in simulated and realworld experiments in order to establish the efficacy of the proposed approach.

Brockers, Roland↗
NASA NTRS2022

How to Build a Rover: An Overview of the Mars 2020 Mission’s Vehicle System Testbed

While NASA’s Mars rover Perseverance continues to make groundbreaking achievements on the Red Planet, its twin is hard at work here on Earth. The Operational Perseverance Twin for the Integration of Mechanisms and Instruments Sent to Mars, or OPTIMISM, is the Mars 2020 Vehicle System Testbed (VSTB) rover operated by NASA Jet Propulsion Laboratory (JPL) in Pasadena, California. OPTIMISM’s home is the JPL Mars Yard; an outdoor field with red soil that simulates the terrain encountered by Perseverance. The VSTB is a full-scale engineering model of the flight rover, serving a number of functions to ensure mission operations can continue smoothly and on schedule. The VSTB possesses instrumentation, computers, mechanisms, cameras, and a Mobility subsystem that are nearly identical to its extraterrestrial twin. Its high fidelity allows the rover to be a highly effective tool to fully test system functionality and performance prior to commanding the flight rover. The early stages of building OPTIMISM began a few months prior to Perseverance departing JPL for Cape Canaveral, FL in early 2020. Electrical integration of the flight system avionics, and compatibility checkouts of the electrical ground support equipment ensured that the foundation of the electrical system was operational and in place. Next, the internal harnessing was installed and compatibility checks of the rover instrumentation and mechanisms were performed to confirm the system was prepared for full buildup. Finally, mechanical assembly of the rover chassis with its external components completed the integration of the system before it was moved to the Mars Yard for its initial phase of testing to perform verification & validation (V&V) of the Mobility subsystem requirements. By the time Perseverance landed at Jezero Crater in February 2021, the first phase of VSTB operations was underway. Surface guidance, navigation, and control (SGNC) testing for the Mobility subsystem ensured functionality and performance requirements were met for various capabilities such as visual odometry (VO), mapping, and automatic navigation (AutoNav). Subsequent integration of the robotic arm (RA) onto the VSTB enabled the V&V campaign for surface sampling operations (SSO) to commence. As the mission’s engineering operations (EO) have gotten underway, the VSTB has been utilized for an array of purposes including troubleshooting software anomalies, and performing dry-runs for first time activities (FTAs) prior to sending the commands to Perseverance. OPTIMISM will continue to serve mission critical functions as long as Perseverance is roving the Red Planet.

Rojas, Jose Trujillo↗
NASA NTRS2021

Mid-Air Range-Visual-Inertial Estimator Initialization for Micro Air Vehicles

Monocular Visual-Inertial Odometry (VIO) has become ubiquitous for navigation of autonomous Micro Air Vehicles (MAVs). Yet, state-of-the-art VIO is still very failure- prone, which can have dramatic consequences. To prevent this, VIO must be able to re-initialize in mid-air, either during a free fall or on a constant horizontal velocity trajectory after attitude control has been re-established. However, for both of these trajectories, the visual scale cannot be observed with VIO batch initializers because of the absence of acceleration change. We propose to use a small and lightweight laser range finder and a scene facet model to initialize vision-based navigation at the right scale under any motion condition and over any scene structure. This new range constraint is integrated into a visual-inertial bundle adjustment initializer. We demonstrate in simulation and real data how this approach can address mid-air state estimation failure in real-time and analyze the sensitivity of estimators to initial state errors.

Brockers, Roland↗
NASA NTRS2005

Slip compensation for a Mars Rover

This paper describes the design, implementation, and experimental results of an integrated system that enables Mars rover to navigate in a high slip environment.

navigation↗
NASA NTRS2004

Real-time detection of moving objects from moving vehicles using dense stereo and optical flow

Dynamic scene perception is very important for autonomous vehicles operating around other moving vehicles and humans. Most work on real-time object tracking from moving platforms has used sparse features or assumed flat scene structures. We have recently extended a real-time, dense stereo system to include real-time, dense optical flow, enabling more comprehensive dynamic scene analysis. We describe algorithms to robustly estimate 6-DOF robot egomotion in the presence of moving objects using dense flow and dense stereo. We then use dense stereo and egomotion estimates to identity other moving objects while the robot itself is moving. We present results showing accurate egomotion estimation and detection of moving people and vehicles under general 6-DOF motion of the robot and independently moving objects. The system runs at 18.3 Hz on a 1.4 GHz Pentium M laptop, computing 160x120 disparity maps and optical flow fields, egomotion, and moving object segmentation. We believe this is a significant step toward general unconstrained dynamic scene analysis for mobile robots, as well as for improved position estimation where GPS is unavailable.

egomotion↗
NASA NTRS2004

Real-time detection of moving objects from moving vehicles using dense stereo and optical flow

Dynamic scene perception is very important for autonomous vehicles operating around other moving vehicles and humans. Most work on real-time object tracking from moving platforms has used sparse features or assumed flat scene structures. We have recently extended a real-time, dense stereo system to include realtime, dense optical flow, enabling more comprehensive dynamic scene analysis. We describe algorithms to robustly estimate 6-DOF robot egomotion in the presence of moving objects using dense flow and dense stereo. We then use dense stereo and egomotion estimates to identify & other moving objects while the robot itself is moving. We present results showing accurate egomotion estimation and detection of moving people and vehicles under general 6-DOF motion of the robot and independently moving objects. The system runs at 18.3 Hz on a 1.4 GHz Pentium M laptop, computing 160x120 disparity maps and optical flow fields, egomotion, and moving object segmentation. We believe this is a significant step toward general unconstrained dynamic scene analysis for mobile robots, as well as for improved position estimation where GPS is unavailable.

egomotion↗
NASA NTRS2004

Real-time Detection of Moving Objects from Moving Vehicles Using Dense Stereo and Optical Flow

Dynamic scene perception is very important for autonomous vehicles operating around other moving vehicles and humans. Most work on real-time object tracking from moving platforms has used sparse features or assumed flat scene structures. We have recently extended a real-time. dense stereo system to include realtime. dense optical flow, enabling more comprehensive dynamic scene analysis. We describe algorithms to robustly estimate 6-DOF robot egomotion in the presence of moving objects using dense flow and dense stereo. We then use dense stereo and egomotion estimates to identify other moving objects while the robot itself is moving. We present results showing accurate egomotion estimation and detection of moving people and vehicles under general 6DOF motion of the robot and independently moving objects. The system runs at 18.3 Hz on a 1.4 GHz Pentium M laptop. computing 160x120 disparity maps and optical flow fields, egomotion, and moving object segmentation. We believe this is a significant step toward general unconstrained dynamic scene analysis for mobile robots, as well as for improved position estimation where GPS is unavailable.

moving object detection↗
NASA NTRS2021

Dense 3D-Reconstruction from Monocular Image Sequences for Computationally Constrained UAS

The ability to find safe landing sites over complex 3D terrain is an essential safety feature for fully autonomous small unmanned aerial systems (UAS), which requires on-board perception for 3D reconstruction and terrain analysis if the overflown terrain is unknown. This is a challenge for UAS that are limited in size, weight and computational power, such as small rotorcrafts executing autonomous missions on Earth, or in planetary applications such as the Mars Helicopter. For such a computationally constraint system, we propose a structure from motion approach that uses inputs from a single downward facing camera to produce dense point clouds of the overflown terrain in real time. In contrast to existing approaches, our method uses metric pose information from a visual-inertial odometry algorithm as camera pose priors, which allows deploying a fast pose refinement step to align camera frames such that a conventional stereo algorithm can be used for dense 3D reconstruction. We validate the performance of our approach with extensive evaluations in simulation, and demonstrate the feasibility with data from UAS flights.

Brockers, Roland↗
NASA NTRS2011

High-Performance 3D Articulated Robot Display

In the domain of telerobotic operations, the primary challenge facing the operator is to understand the state of the robotic platform. One key aspect of understanding the state is to visualize the physical location and configuration of the platform. As there is a wide variety of mobile robots, the requirements for visualizing their configurations vary diversely across different platforms. There can also be diversity in the mechanical mobility, such as wheeled, tracked, or legged mobility over surfaces. Adaptable 3D articulated robot visualization software can accommodate a wide variety of robotic platforms and environments. The visualization has been used for surface, aerial, space, and water robotic vehicle visualization during field testing. It has been used to enable operations of wheeled and legged surface vehicles, and can be readily adapted to facilitate other mechanical mobility solutions. The 3D visualization can render an articulated 3D model of a robotic platform for any environment. Given the model, the software receives real-time telemetry from the avionics system onboard the vehicle and animates the robot visualization to reflect the telemetered physical state. This is used to track the position and attitude in real time to monitor the progress of the vehicle as it traverses its environment. It is also used to monitor the state of any or all articulated elements of the vehicle, such as arms, legs, or control surfaces. The visualization can also render other sorts of telemetered states visually, such as stress or strains that are measured by the avionics. Such data can be used to color or annotate the virtual vehicle to indicate nominal or off-nominal states during operation. The visualization is also able to render the simulated environment where the vehicle is operating. For surface and aerial vehicles, it can render the terrain under the vehicle as the avionics sends it location information (GPS, odometry, or star tracking), and locate the vehicle over or on the terrain correctly. For long traverses over terrain, the visualization can stream in terrain piecewise in order to maintain the current area of interest for the operator without incurring unreasonable resource constraints on the computing platform. The visualization software is designed to run on laptops that can operate in field-testing environments without Internet access, which is a frequently encountered situation when testing in remote locations that simulate planetary environments such as Mars and other planetary bodies.

Powell, Mark W.↗