Search NASA⌕ Search

SEARCH · Search NASA

Results for “Vision based Navigation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Fusion of Visible and Thermal-Infrared Imagery for SLAM for Landing on Icy Moons

This paper addresses the problem of localization for landing on the surface of icy moons, like Europa or Enceladus. Due to the possibility of specular reflection as well as high bulk albedo, icy surfaces present new challenges that make traditional vision-based navigation systems relying on visible imagery unreliable. We propose augmenting visible light cameras with a thermal-infrared camera using inverse-depth parameterized monocular EKF-SLAM to address problems arising from the appearance of icy moons. Results were obtained from a novel procedural Europa surface simulation which models the appearance and the thermal properties simultaneously from physically-based methods. In this framework, we show that thermal features improve localization by 23% on average when compared to a visible camera. Moreover, fusing both sensing modalities increases the improvement in localization to 31% on average, compared to using a visible light camera alone.

thermal vision↗
NASA NTRS2005

Micro-Inspector Spacecraft for Space Exploration Missions

NASA is seeking to embark on a new set of human and robotic exploration missions back to the Moon, to Mars, and destinations beyond. Key strategic technical challenges will need to be addressed to realize this new vision for space exploration, including improvements in safety and reliability to improve robustness of space operations. Under sponsorship by NASA's Exploration Systems Mission, the Jet Propulsion Laboratory (JPL), together with its partners in government (NASA Johnson Space Center) and industry (Boeing, Vacco Industries, Ashwin-Ushas Inc.) is developing an ultra-low mass (<3.0 kg) free-flying micro-inspector spacecraft in an effort to enhance safety and reduce risk in future human and exploration missions. The micro-inspector will provide remote vehicle inspections to ensure safety and reliability, or to provide monitoring of in-space assembly. The micro-inspector spacecraft represents an inherently modular system addition that can improve safety and support multiple host vehicles in multiple applications. On human missions, it may help extend the reach of human explorers, decreasing human EVA time to reduce mission cost and risk. The micro-inspector development is the continuation of an effort begun under NASA's Office of Aerospace Technology Enabling Concepts and Technology (ECT) program. The micro-inspector uses miniaturized celestial sensors; relies on a combination of solar power and batteries (allowing for unlimited operation in the sun and up to 4 hours in the shade); utilizes a low-pressure, low-leakage liquid butane propellant system for added safety; and includes multi-functional structure for high system-level integration and miniaturization. Versions of this system to be designed and developed under the H&RT program will include additional capabilities for on-board, vision-based navigation, spacecraft inspection, and collision avoidance, and will be demonstrated in a ground-based, space-related environment. These features make the micro-inspector design unique in its ability to serve crewed as well as robotic spacecraft, well beyond Earth-orbit and into arenas such as robotic missions, where human teleoperation capability is not locally available.

microspacecraft↗
NASA NTRS2014

Development and Testing of the Smart SPHERES Telerobotic Free-Flyer

Smart SPHERES is a free-flying robot that can be remotely operated by astronauts in space, or by mission controllers on the ground. We are developing Smart SPHERES to perform a variety of intravehicular activities (IVA) inside the International Space Station (ISS). These IVA tasks include interior environmental surveys, inventory, and mobile camera work. During 2013, we tested crew-controlled remote operations and wireless network-based localization with Smart SPHERES on ISS. In 2014, we plan to test 3D vision-based navigation.

Fong, Terrence↗
NASA NTRS2024

Starling Formation-Flying Optical Experiment: Initial Operations and Flight Results

This paper presents initial flight results for angles-only navigation of a swarm of small spacecraft, conducted during the Starling Formation-Flying Optical Experiment (StarFOX). StarFOX is one of four experiments aboard the NASA Starling mission, which consists of four CubeSats launched in July 2023. Angles-only methods apply inter-satellite bearing angles obtained by on-board cameras for navigation, increasing satellite autonomy and enabling new mission concepts. Nevertheless, prior flight demonstrations have only featured one observer and target and have relied upon a-priori target orbit knowledge for initialization, translational maneuvers to resolve target range, and external absolute orbit updates to maintain convergence. StarFOX overcomes these limitations by applying the angles-only Absolute and Relative Trajectory Measurement System (ARTMS), which integrates three novel algorithms. Image Processing detects and tracks multiple targets in images, using multi-hypothesis methods and domain-specific kinematic modeling, and computes target bearing angles. Batch Orbit Determination computes initial swarm orbit estimates from bearing angle batches, via iterative batch least squares and sampling of the weakly observable target range. Sequential Orbit Determination leverages an adaptive, efficient unscented Kalman filter with nonlinear models to refine swarm state estimates over time. Multi-observer measurements shared over an intersatellite link are seamlessly fused to enable robust absolute and relative orbit determination. Initial StarFOX experiments are the first demonstrations of autonomous angles-only navigation for a satellite swarm, including multi-target and multi-observer relative navigation; autonomous initialization of navigation for unknown targets; and simultaneous absolute and relative orbit determination. Relative navigation accuracy of 1% (single observer) and 0.2% (multi-observer) of the inter-satellite range is achieved under challenging measurement conditions. Results demonstrate promising performance with regards to ongoing StarFOX campaigns and the application of angles-only navigation to future distributed missions.

Relative Navigation↗
NASA NTRS1991

Experiments in teleoperator and autonomous control of space robotic vehicles

A program of research embracing teleoperator and automatic navigational control of freely flying satellite robots is presented. Current research goals include: (1) developing visual operator interfaces for improved vehicle teleoperation; (2) determining the effects of different visual interface system designs on operator performance; and (3) achieving autonomous vision-based vehicle navigation and control. This research program combines virtual-environment teleoperation studies and neutral-buoyancy experiments using a space-robot simulator vehicle currently under development. Visual-interface design options under investigation include monoscopic versus stereoscopic displays and cameras, helmet-mounted versus panel-mounted display monitors, head-tracking versus fixed or manually steerable remote cameras, and the provision of vehicle-fixed visual cues, or markers, in the remote scene for improved sensing of vehicle position, orientation, and motion.

Alexander, Harold L.↗
NASA NTRS2018

Starling1 - Mission Technologies Overview

The Starling series of demonstration missions will test technologies required to achieve affordable, distributed spacecraft ("swarm") missions that: are scalable to at least 100 spacecraft for applications that include synchronized multipoint measurements; involve closely coordinated ensembles of two or more spacecraft operating as a single unit for interferometric, synthetic aperture, or similar sensor architectures; or use autonomous or semi-autonomous operation of multiple spacecraft functioning as a unit to achieve science or other mission objectives with low-cost small spacecraft.Starling1 will focus on developing technologies that enable scalability and deep space application. The mission goals include the demonstration of a Mobile Ad-hoc NETwork (MANET) through an in-space communication experiment, vision based relative navigation through the Starling Formation-flying Optical eXperiment (StarFOX), and demonstration of autonomous spacecraft reconfiguration using technologies developed by the Distributed System Autonomy (DSA) project.

Cannon, Howard↗
NASA NTRS2022

Robust Vision-based Multi-spacecraft Guidance Navigation and Control using CNN-based Pose Estimation

In this paper, we present an end-to-end simulation framework for tracking an uncooperative Target spacecraft in Low Earth Orbit using a CubeSat-class Ego spacecraft outfitted with a camera. Currently, capturing high-fidelity realistic images in space for this scenario is difficult and exorbitantly expensive. Therefore, we developed a framework to simulate the spacecraft orbits in Basilisk software and generate high-fidelity realistic images of spacecraft in Unreal Engine, including the effects from Sun, Earth, Moon and stars. The Ego spacecraft uses cameras to capture images of the uncooperative Target and estimates its position and attitude using a CNN based 6DOF pose estimation pipeline, eliminating need for large SWAP-C(Size, Weight, Power and Cost) sensors like LIDAR or reliance on inter-spacecraft communication, This CNN, which is motivated by ESA’s Pose Estimation challenge of 2019, is trained using simulated data from our end-to-end simulation framework. We compare the performance of two distinct CNNbased algorithms for pose estimation along a nominal trajectory. In presence of non-Gaussian modeling uncertainties, the statedependent estimation error is characterized with a quadratic upper-bound. The quadratically-bounded error can be used by a robust controller to maneuver

Rahmani, Amir↗
NASA NTRSDate not supplied

Mars 2020 – Landing a 1-ton rover and helicopter in an ancient Martian Lake

The Mars 2020 spacecraft launched in July 2020 and landed the Perseverance rover and Ingenuity helicopter successfully in Jezero crater on Feb. 18, 2021. Mars 2020 is the first stage of the Mars Sample Return campaign that will bring back the first samples from another planet to Earth. The entry, descent, and landing (EDL) sequence of the Mars 2020 spacecraft largely leveraged the previous Mars Science Laboratory (MSL) mission from 2012. Mars 2020 retained most of the EDL sequences of MSL, including active maneuvering during hypersonic flight to accurately target the landing site and use of the Skycrane descent stage that slowly lowered the rover while hovering above the ground. But Mars 2020 also added Terrain Relative Navigation, a machine vision-based system that allowed the spacecraft to navigate using an on-board camera that mapped ground landmarks to an on-board map, allowing the spacecraft to safely land in locations that were too hazardous for any previous Martian mission. Come hear about the “Seven Minutes of Terror” and the eight years of effort that went into the engineering behind the spacecraft.

Soumyo Dutta↗
NASA NTRSDate not supplied

Simulated Vision-based Approach and Landing System for Advanced Air Mobility

Advanced Air Mobility (AAM) aircraft require precision approach and landing systems (PALS) in several environments, such as urban, suburban, and rural. It is challenging to implement current state-of-the-art methods approved for automated approach and landing for AAM operations with challenges such as GPS degradation in urban environments and visual navigation aids like the glideslope and localizer being narrow and not allowing alternative incoming landing angles at vertiports. However, existing technology and systems, i.e., the instrument landing system (ILS) with glideslope and localizer indicators that use vision, IR, radar, or GPS methods, provide baseline perception and sensing requirements for AAM aircraft approach and landing. This paper focuses on vision-based PAL and computer vision feature correspondence methods to demonstrate a baseline navigation system while adhering to the Federal Aviation Administration requirements and regulations about heliport design (FAA AC 150/5390-2C), which is one of the closest references for vertiport requirements and regulations. The coplanar pose from orthography and scaling with iterations (COPOSIT) algorithm determines pose estimation, which feeds into an Extended Kalman filter that combines IMU with vision to create a vision-based approach and landing (VAL) sensor fusion navigation solution for GPS-denied environments. The VAL navigation solution provides promising simulation results for AAM PALS with Hough circle detection and feature correspondence, which demonstrate robustness to false positives. This paper incorporates moderately high- fidelity simulations with computer graphics rendering to show a distributed sensor network to track an AAM aircraft during approach and landing to compare with the aircraft’s onboard vision-based navigation solution.

distributed sensing↗
NASA NTRS1992

Knowledge based imaging for terrain analysis

A planetary rover will have various vision based requirements for navigation, terrain characterization, and geological sample analysis. In this paper we describe a knowledge-based controller and sensor development system for terrain analysis. The sensor system consists of a laser ranger and a CCD camera. The controller, under the input of high-level commands, performs such functions as multisensor data gathering, data quality monitoring, and automatic extraction of sample images meeting various criteria. In addition to large scale terrain analysis, the system's ability to extract useful geological information from rock samples is illustrated. Image and data compression strategies are also discussed in light of the requirements of earth bound investigators.

Holben, Rick↗
NASA NTRS2019

Demonstration of Stereo Vision for Deorbit Descent and Landing

Planetary landers need to reduce velocity at low altitude for soft landing. Traditionally, estimating velocity and altitude has been performed with radar sensors whose performance meets the specific mission needs. There are not very many options for these sensors and they are difficult to include in a flight system either due to obsolescence, prohibitive cost or difficulty in accommodation. Recently, alternative sensing modalities are being pursued including Doppler LiDAR and vision. This paper describes results from a recent helicopter field test of a binocular stereo vision system for deorbit descent and landing applications. The system consisted of two 18.6 ̊ field of view cameras mounted 1.7m apart. Post processing of the images showed ranging accuracy better than 1% up to 500mand 17 cm/s velocimetry accuracy at 37m. For a flight system these images could be input into an FPGA-based processor which processes dense stereo and visual odometry in less than 1 second to achieve the stereo ranging frame rates required for soft landing. When coupled with vision based Terrain Relative Navigation this stereo system enables landing accuracies on the order of 10m.

Sternberg, David C.↗
NASA NTRS2019

Demonstration of stereo vision for deorbit descent and landing

This paper describes results from a recent helicopter field test of a binocular stereo vision system for deorbit descent and landing applications. The system consisted of two 18.6˚ field of view cameras mounted 1.7m apart. Post processing of the images showed ranging accuracy better than 1% up to 500m and 17 cm/s velocimetry accuracy at 37m. For a flight system these images could be input into an FPGA-based processor which processes dense stereo and visual odometry in less than 1 second to achieve the stereo ranging frame rates required for soft landing. When coupled with vision based Terrain Relative Navigation this stereo system enables landing accuracies on the order of 10m.

Johnson, Andrew E.↗
NASA NTRS2009

Stereo Vision Based Terrain Mapping for Off-Road Autonomous Navigation

Successful off-road autonomous navigation by an unmanned ground vehicle (UGV) requires reliable perception and representation of natural terrain. While perception algorithms are used to detect driving hazards, terrain mapping algorithms are used to represent the detected hazards in a world model a UGV can use to plan safe paths. There are two primary ways to detect driving hazards with perception sensors mounted to a UGV: binary obstacle detection and traversability cost analysis. Binary obstacle detectors label terrain as either traversable or non-traversable, whereas, traversability cost analysis assigns a cost to driving over a discrete patch of terrain. In uncluttered environments where the non-obstacle terrain is equally traversable, binary obstacle detection is sufficient. However, in cluttered environments, some form of traversability cost analysis is necessary. The Jet Propulsion Laboratory (JPL) has explored both approaches using stereo vision systems. A set of binary detectors has been implemented that detect positive obstacles, negative obstacles, tree trunks, tree lines, excessive slope, low overhangs, and water bodies. A compact terrain map is built from each frame of stereo images. The mapping algorithm labels cells that contain obstacles as no-go regions, and encodes terrain elevation, terrain classification, terrain roughness, traversability cost, and a confidence value. The single frame maps are merged into a world map where temporal filtering is applied. In previous papers, we have described our perception algorithms that perform binary obstacle detection. In this paper, we summarize the terrain mapping capabilities that JPL has implemented during several UGV programs over the last decade and discuss some challenges to building terrain maps with stereo range data.

passive perception↗
NASA NTRSDate not supplied

VSLAM and Vision-based Approach and Landing for Advanced Air Mobility

Advanced Air Mobility (AAM) aircraft have many challenges in landing accurately and safely in urban, suburban, and rural environments. Localization in large and open rural environments could utilize GPS, but AAM aircraft in urban environments will encounter GPS degradation. Another challenge involves flight operation time, i.e., flying during the day or at night. There are different guidelines, landmarks, and landing light configurations at runways, heliports, and vertiports for daytime and nighttime applications. Tailoring feature detection methods for AAM approach and landing during the day and night pose different issues and challenges. It is easier to detect edges, lines, and other runway markers during the day than at night. Conversely, it is easier to see landing light configurations and patterns at nighttime than daytime. Consequently, utilizing the same feature detector for daytime and nighttime operations may not be feasible. This paper focuses on a vision-based precision approach and landing (PAL) by comparing ORB SLAM 2, a Vision Simultaneous Localization and Mapping (VSLAM) algorithm, and a novel EKF that combines onboard IMU measurements with coplanar pose from orthography and scaling with iterations (COPOSIT). Conducting unmanned aerial system (UAS) flight tests at NASA Armstrong Flight Research Center (AFRC) with landmarks and fiducials distributed around the landing zone provides a simulated AAM approach and landing data to test vision-based PAL methods to provide Alternative Position, Navigation, and Timing (APNT) solutions for AAM PAL applications. The novel vision-based PAL EKF with IMU and COPOSIT provides accurate state estimation when distributed landmarks and fiducials are in the field of view.

distributed sensing↗
NASA NTRS2007

Dilution-of-Precision-Based Lunar Surface Navigation System Analysis Utilizing Earth-Based Assets

The NASA Vision for Space Exploration is focused on the return of astronauts to the Moon. Although navigation systems have already been proven in the Apollo missions to the Moon, the current exploration campaign will involve more extensive and extended missions requiring new concepts for lunar navigation. In contrast to Apollo missions, which were limited to the near-side equatorial region of the Moon, those under the Exploration Systems Initiative will require navigation on the Moon's limb and far side. These regions are known to have poor Earth visibility, but unknown is the extent to which a navigation system comprised solely of Earth-based tracking stations will provide adequate navigation solutions in these areas. This report presents a dilution-of-precision (DoP)-based analysis of the performance of a network of Earth-based assets. This analysis extends a previous analysis of a lunar network (LN) of navigation satellites by providing an assessment of the capability associated with a variety of assumptions. These assumptions pertain to the minimum provider elevation angle, nadir and zenith beam widths, and a total single failure in one of the Earth-based assets. The assessment is accomplished by making appropriately formed estimates of DoP. Different adaptations of DoP, such as geometrical DoP and positional DoP (GDoP and PDoP), are associated with a different set of assumptions regarding augmentations to the navigation receiver or transceiver.

Welch, Bryan W.↗
NASA NTRS1994

Vision based object pose estimation for mobile robots

Mobile robot navigation using visual sensors requires that a robot be able to detect landmarks and obtain pose information from a camera image. This paper presents a vision system for finding man-made markers of known size and calculating the pose of these markers. The algorithm detects and identifies the markers using a weighted pattern matching template. Geometric constraints are then used to calculate the position of the markers relative to the robot. The selection of geometric constraints comes from the typical pose of most man-made signs, such as the sign standing vertical and the dimensions of known size. This system has been tested successfully on a wide range of real images. Marker detection is reliable, even in cluttered environments, and under certain marker orientations, estimation of the orientation has proven accurate to within 2 degrees, and distance estimation to within 0.3 meters.

Wu, Annie↗
NASA NTRS2006

Dilution of Precision-Based Lunar Navigation Assessment for Dynamic Position Fixing

The NASA Vision for Space Exploration is focused on the return of astronauts to the Moon. While navigation systems have already been proven in the Apollo missions to the moon, the current exploration campaign will involve more extensive and extended missions requiring new concepts for lunar navigation. In contrast to Apollo missions, which were limited to the near-side equatorial region of the moon, missions under the Exploration Systems Initiative will require navigation on the moon's limb and far-side. As these regions have poor Earth visibility, a navigation system comprised solely of Earth-based tracking stations will not provide adequate navigation solutions in these areas. In this paper, a Dilution of Precision (DoP) based analysis of the performance of a network of Moon orbiting satellites is provided. The analysis extends previous analysis of a Lunar Network (LN) of navigation satellites by providing an assessment of the capability associated with a variety of assumptions. These assumptions are with regard to the navigation receiver and satellite visibility. The assessment is accomplished by making appropriately formed estimates of DoP. Different adaptations of DoP (i.e., GDoP, PDoP, etc.) are associated with a different set of assumptions regarding augmentations to the navigation receiver or transceiver.

Sands, Obed S.↗