Search NASA⌕ Search

SEARCH · Search NASA

Results for “Vision based Navigation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

A Minimal State Augmentation Algorithm for Vision-Based Navigation without Using Mapped Landmarks

This paper describes MAVeN (Minimal State Augmentation Algorithm for Vision-Based Navigation), which is a new algorithm for vision-based navigation that has only 21 states, yet is able to track features in successive camera images and use them to propagate estimates of the spacecraft position and velocity. The filter dimension drops to 12 if attitude information is already available. The low filter dimension makes MAVeN a very reliable and practical algorithm for real-time flight implementation. The main idea is to project observed features onto a rough shape model of the ground surface, which are then used by the filter as pseudo-landmarks. The shape model is assumed to be known beforehand, as would be obtained from prior surveillance of the landing site from orbit. MAVeN does not require pre-mapped landmarks, so it is able to navigate terrain that has not been previously observed up close. This property is especially important for close proximity operations in small body missions where ground surface features are being seen for the first time at close range. MAVeN is also able to hover motionless above the ground without position error growth, which is unusual for this class of vision-based navigation algorithms.

San Martin, A. Miguel↗
NASA NTRS1994

Low computation vision-based navigation for a Martian rover

Construction and design details of the Mobot Vision System, a small, self-contained, mobile vision system, are presented. This system uses the view from the top of a small, roving, robotic vehicle to supply data that is processed in real-time to safely navigate the surface of Mars. A simple, low-computation algorithm for constructing a 3-D navigational map of the Martian environment to be used by the rover is discussed.

Gavin, Andrew S.↗
NASA NTRSDate not supplied

Improving Sim-to-Real Transfer in Vision-Based Robot Navigation Via Instance-Level GAN-Based Data Augmentation

Achieving robust vision-based robotic tasks requires large amounts of data, which are often difficult to obtain in real-world scenarios. Simulators and synthetic data offer a cost-effective alternative, but the visual gap between simulation and reality hinders the performance of models when deployed in real-world environments. In this paper, we present a data augmentation pipeline that integrates a foundation model (Segment Anything Model) with an unsupervised image-to-image translation model (CycleGAN) for instance-level domain transfer from simulation to reality. This pipeline enables the generation of realistic labeled data from synthetic images for training supervised machine learning models in vision-based navigation tasks. We evaluate our approach on real-world data for ego-vehicle pose estimation, a critical autonomous navigation task involving the prediction of cross-track position and heading angle relative to road center line markings. The results of our tests show that our GAN-based data augmentation pipeline significantly outperforms models trained solely on simulation data or on data processed with standard image augmentation methods for sim-to-real transfer, enhancing model robustness and generalizability in real-world scenarios. Our method provides a scalable and flexible data augmentation tool for leveraging large synthetic datasets to enhance vision-based robotic navigation tasks.

artificial intelligence↗
NASA NTRSDate not supplied

Vision-Based Distributed Sensing at Vertiports for Advanced Air Mobility and Urban Air Mobility Approach and Landing

Advanced Air Mobility (AAM) encompasses a broad vision for air transportation, including Urban Air Mobility (UAM) as a subset. AAM aims to create a more connected and efficient transportation network across various geographical settings. However, navigating AAM aircraft in GPS-denied or degraded environments during approach and landing is challenging. Traditional vision aids like glideslopes and localizers are limited in vertiport environments due to narrow beam constraints and reduced landing angle options. This paper addresses the need for accurate navigation solutions at vertiports by proposing a vision-based distributed sensing (VIDIS) system utilizing cameras with bundle adjustment to assist incoming AAM aircraft during approach and landing while monitoring surface movements to enhance safety and efficiency. Key focus areas for current and future vertiport developers include identifying suitable sensor types and infrastructure standards to support AAM operations and including vertiport markings as vision-based navigation aids. The proposed system offers a novel approach to overcoming navigation challenges in AAM operations, particularly in urban settings where traditional aids may be insufficient. Preliminary simulation results with distributed cameras demonstrate promising outcomes for implementing bundle adjustment techniques to enhance vision-based navigation solutions at vertiports. Generating waypoint-based trajectories via waypoint integration using explicit guidance synthesis (WINGS) creates smooth AAM trajectories for landing at vertiports by using the current waypoint's terminal conditions as the initial conditions for the next waypoint. Combining bundle adjustment's ground-based solution of vertiport features with WINGS, Coplanar Pose from Orthography and Scaling with Iterations (COPOSIT), and an extended Kalman filter (EKF) estimates the state of an incoming aircraft during approach and landing at vertiports. Future work includes testing VIDIS in a high-fidelity simulation and with real-world data.

Distributed sensing↗
NASA NTRSDate not supplied

Vision-Based Distributed Sensing at Vertiports for Advanced Air Mobility and Urban Air Mobility Approach and Landing

Advanced Air Mobility (AAM) encompasses a broad vision for air transportation, including Urban Air Mobility (UAM) as a subset. AAM aims to create a more connected and efficient transportation network across various geographical settings. However, navigating AAM aircraft in GPS-denied or degraded environments during approach and landing is challenging. Traditional vision aids like glideslopes and localizers are limited in vertiport environments due to narrow beam constraints and reduced landing angle options. This paper addresses the need for accurate navigation solutions at vertiports by proposing a vision-based distributed sensing (VIDIS) system utilizing cameras with bundle adjustment to assist incoming AAM aircraft during approach and landing while monitoring surface movements to enhance safety and efficiency. Key focus areas for current and future vertiport developers include identifying suitable sensor types and infrastructure standards to support AAM operations and including vertiport markings as vision-based navigation aids. The proposed system offers a novel approach to overcoming navigation challenges in AAM operations, particularly in urban settings where traditional aids may be insufficient. Preliminary simulation results with distributed cameras demonstrate promising outcomes for implementing bundle adjustment techniques to enhance vision-based navigation solutions at vertiports. Generating waypoint-based trajectories via waypoint integration using explicit guidance synthesis (WINGS) creates smooth AAM trajectories for landing at vertiports by using the current waypoint's terminal conditions as the initial conditions for the next waypoint. Combining bundle adjustment's ground-based solution of vertiport features with WINGS, Coplanar Pose from Orthography and Scaling with Iterations (COPOSIT), and an extended Kalman filter (EKF) estimates the state of an incoming aircraft during approach and landing at vertiports. Future work includes testing VIDIS in a high-fidelity simulation and with real-world data.

Distributed sensing↗
NASA NTRSDate not supplied

Building Maps for Terrain Relative Navigation Using Blender: An Open-Source Approach

A persistent challenge for vision-based navigation systems that compare imagery to a reference map is generating high quality maps with similar lighting conditions. Image rendering software can be used to apply variable lighting to reference maps or to generate synthetic imagery for test trajectories. While many image rendering software packages are available, with several developed specifically for spaceflight applications, there are often limitations due to cost, image fidelity, or flexibility. In this paper, we demonstrate the use of an open-source image rendering software, Blender, for use in Terrain Relative Navigation (TRN) applications. A scene in Blender was generated based on elevation data and satellite imagery of the region of West Texas used by Blue Origin for the operation of their New Shepard suborbital rocket. The Blender scene was validated by reproducing imagery collected during a flight of New Shepard in October 2020 and was further used to generate reference maps for use by a TRN algorithm on a subsequent New Shepard flight in August 2021. The work was performed under the NASA Safe and Precise Landing Integrated Capabilities Evolution (SPLICE) project, which is focused on technology advancement for precision landing and hazard avoidance. This work aims to lower the cost of entry and generally promote the adoption and advancement of vision-based navigation technologies.

Kyle W Smith↗
NASA NTRS1990

Vision-based planetary rover navigation

NASA and JPL have developed a testbed 'planetary rover' vehicle with sufficient power supplies, sensors, and computational resources for the demonstration of semiautonomous navigation. Attention is presently given to this vehicle's vision-based navigation techniques. The proposed design and its variants allow advantage to be taken of enormous quantities of both spatial and temporal information that are normally wasted, by sampling very fine detail over the full focal plane area to precisely determine those parts of the image that are accurately at the focus range of the pinhole array used. This should generate accurate and reliable real-time range information in a wide variety of natural scenes, with little or no computation.

Wilcox, Brian H.↗
NASA NTRSDate not supplied

Autonomous Nanosatellite Swarming (ANS) Using Radio Frequency and Optical Navigation

The Autonomous Nanosatellite Swarming (ANS) subsystem is a dynamics, guidance, navigation, and control software unit based on commercial-off-the-shelf avionics, which is distributed and deployed on a cluster of cooperative nanosatellites to enable future exploration missions to near-Earth objects (NEOs). ANS allows multiple nanosatellites to act together as a single large spacecraft through radio-frequency (RF) cross-links, optical navigation, and low-thrust control. In particular, ANS-equipped nanosatellites can utilize images of a NEO taken from multiple viewpoints to achieve stereo-vision capabilities and characterize the target body’s shape, gravity field, and dynamical properties. All this is enabled by the fusion of major advances to the state of the art in the astrodynamics of relative motion, RF and vision-based navigation, as well as low-thrust control.

Simone D'Amico↗
NASA NTRS2018

Towards Vision-based Safe Autonomous Landing of MAVs at Night

This paper is about vision-based navigation of Micro Aerial Vehicles (MAVs) at night. Despite it being dark almost half of the time, most of the work to date has addressed only daytime operations. Enabling autonomous night-time operation of MAVs with low SWaP on-board sensing capabilities is still an open problem in current robotics research. In this work, we take a step in this direction and study the problem in the context of vision-based autonomous landing at night using thermal-infrared cameras.

Delaune, Jeff↗
NASA NTRS2012

Localization Using Visual Odometry and a Single Downward-Pointing Camera

Stereo imaging is a technique commonly employed for vision-based navigation. For such applications, two images are acquired from different vantage points and then compared using transformations to extract depth information. The technique is commonly used in robotics for obstacle avoidance or for Simultaneous Localization And Mapping, (SLAM). Yet, the process requires a number of image processing steps and therefore tends to be CPU-intensive, which limits the real-time data rate and use in power-limited applications. Evaluated here is a technique where a monocular camera is used for vision-based odometry. In this work, an optical flow technique with feature recognition is performed to generate odometry measurements. The visual odometry sensor measurements are intended to be used as control inputs or measurements in a sensor fusion algorithm using low-cost MEMS based inertial sensors to provide improved localization information. Presented here are visual odometry results which demonstrate the challenges associated with using ground-pointing cameras for visual odometry. The focus is for rover-based robotic applications for localization within GPS-denied environments.

Swank, Aaron J.↗
NASA NTRS2016

Visual Odometry for Autonomous Deep-Space Navigation Project

Autonomous rendezvous and docking (AR&D) is a critical need for manned spaceflight, especially in deep space where communication delays essentially leave crews on their own for critical operations like docking. Previously developed AR&D sensors have been large, heavy, power-hungry, and may still require further development (e.g. Flash LiDAR). Other approaches to vision-based navigation are not computationally efficient enough to operate quickly on slower, flight-like computers. The key technical challenge for visual odometry is to adapt it from the current terrestrial applications it was designed for to function in the harsh lighting conditions of space. This effort leveraged Draper Laboratory’s considerable prior development and expertise, benefitting both parties. The algorithm Draper has created is unique from other pose estimation efforts as it has a comparatively small computational footprint (suitable for use onboard a spacecraft, unlike alternatives) and potentially offers accuracy and precision needed for docking. This presents a solution to the AR&D problem that only requires a camera, which is much smaller, lighter, and requires far less power than competing AR&D sensors. We have demonstrated the algorithm’s performance and ability to process ‘flight-like’ imagery formats with a ‘flight-like’ trajectory, positioning ourselves to easily process flight data from the upcoming ‘ISS Selfie’ activity and then compare the algorithm’s quantified performance to the simulated imagery. This will bring visual odometry beyond TRL 5, proving its readiness to be demonstrated as part of an integrated system.Once beyond TRL 5, visual odometry will be poised to be demonstrated as part of a system in an in-space demo where relative pose is critical, like Orion AR&D, ISS robotic operations, asteroid proximity operations, and more.

Robinson, Shane↗
NASA NTRS2016

Visual Odometry for Autonomous Deep-Space Navigation Project

Autonomous rendezvous and docking (AR&D) is a critical need for manned spaceflight, especially in deep space where communication delays essentially leave crews on their own for critical operations like docking. Previously developed AR&D sensors have been large, heavy, power-hungry, and may still require further development (e.g. Flash LiDAR). Other approaches to vision-based navigation are not computationally efficient enough to operate quickly on slower, flight-like computers. The key technical challenge for visual odometry is to adapt it from the current terrestrial applications it was designed for to function in the harsh lighting conditions of space. This effort leveraged Draper Laboratory's considerable prior development and expertise, benefitting both parties. The algorithm Draper has created is unique from other pose estimation efforts as it has a comparatively small computational footprint (suitable for use onboard a spacecraft, unlike alternatives) and potentially offers accuracy and precision needed for docking. This presents a solution to the AR&D problem that only requires a camera, which is much smaller, lighter, and requires far less power than competing AR&D sensors. We have demonstrated the algorithm's performance and ability to process 'flight-like' imagery formats with a 'flight-like' trajectory, positioning ourselves to easily process flight data from the upcoming 'ISS Selfie' activity and then compare the algorithm's quantified performance to the simulated imagery. This will bring visual odometry beyond TRL 5, proving its readiness to be demonstrated as part of an integrated system. Once beyond TRL 5, visual odometry will be poised to be demonstrated as part of a system in an in-space demo where relative pose is critical, like Orion AR&D, ISS robotic operations, asteroid proximity operations, and more.

Robinson, Shane↗
NASA NTRS1991

Image-based ranging and guidance for rotorcraft

This report documents the research carried out under NASA Cooperative Agreement No. NCC2-575 during the period Oct. 1988 - Dec. 1991. Primary emphasis of this effort was on the development of vision based navigation methods for rotorcraft nap-of-the-earth flight regime. A family of field-based ranging algorithms were developed during this research period. These ranging schemes are capable of handling both stereo and motion image sequences, and permits both translational and rotational camera motion. The algorithms require minimal computational effort and appear to be implementable in real time. A series of papers were presented on these ranging schemes, some of which are included in this report. A small part of the research effort was expended on synthesizing a rotorcraft guidance law that directly uses the vision-based ranging data. This work is discussed in the last section.

Menon, P. K. A.↗
NASA NTRS2021

Mid-Air Range-Visual-Inertial Estimator Initialization for Micro Air Vehicles

Monocular Visual-Inertial Odometry (VIO) has become ubiquitous for navigation of autonomous Micro Air Vehicles (MAVs). Yet, state-of-the-art VIO is still very failure- prone, which can have dramatic consequences. To prevent this, VIO must be able to re-initialize in mid-air, either during a free fall or on a constant horizontal velocity trajectory after attitude control has been re-established. However, for both of these trajectories, the visual scale cannot be observed with VIO batch initializers because of the absence of acceleration change. We propose to use a small and lightweight laser range finder and a scene facet model to initialize vision-based navigation at the right scale under any motion condition and over any scene structure. This new range constraint is integrated into a visual-inertial bundle adjustment initializer. We demonstrate in simulation and real data how this approach can address mid-air state estimation failure in real-time and analyze the sensitivity of estimators to initial state errors.

Brockers, Roland↗
NASA NTRS2019

Fusion of Visible and Thermal-Infrared Imagery for SLAM for Landing on Icy Moons

This paper addresses the problem of localization for landing on the surface of icy moons, like Europa or Enceladus. Due to the possibility of specular reflection as well as high bulk albedo, icy surfaces present new challenges that make traditional vision-based navigation systems relying on visible imagery unreliable. We propose augmenting visible light cameras with a thermal-infrared camera using inverse-depth parameterized monocular EKF-SLAM to address problems arising from the appearance of icy moons. Results were obtained from a novel procedural Europa surface simulation which models the appearance and the thermal properties simultaneously from physically-based methods. In this framework, we show that thermal features improve localization by 23% on average when compared to a visible camera. Moreover, fusing both sensing modalities increases the improvement in localization to 31% on average, compared to using a visible light camera alone.

thermal vision↗