Search NASA⌕ Search

SEARCH · Search NASA

Results for “pose estimation algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1992

Development of a machine vision system for automated structural assembly

Research is being conducted at the LaRC to develop a telerobotic assembly system designed to construct large space truss structures. This research program was initiated within the past several years, and a ground-based test-bed was developed to evaluate and expand the state of the art. Test-bed operations currently use predetermined ('taught') points for truss structural assembly. Total dependence on the use of taught points for joint receptacle capture and strut installation is neither robust nor reliable enough for space operations. Therefore, a machine vision sensor guidance system is being developed to locate and guide the robot to a passive target mounted on the truss joint receptacle. The vision system hardware includes a miniature video camera, passive targets mounted on the joint receptacles, target illumination hardware, and an image processing system. Discrimination of the target from background clutter is accomplished through standard digital processing techniques. Once the target is identified, a pose estimation algorithm is invoked to determine the location, in three-dimensional space, of the target relative to the robots end-effector. Preliminary test results of the vision system in the Automated Structural Assembly Laboratory with a range of lighting and background conditions indicate that it is fully capable of successfully identifying joint receptacle targets throughout the required operational range. Controlled optical bench test results indicate that the system can also provide the pose estimation accuracy to define the target position.

Sydow, P. Daniel↗
NASA NTRS1992

Development of a machine vision guidance system for automated assembly of space structures

The topics are presented in viewgraph form and include: automated structural assembly robot vision; machine vision requirements; vision targets and hardware; reflective efficiency; target identification; pose estimation algorithms; triangle constraints; truss node with joint receptacle targets; end-effector mounted camera and light assembly; vision system results from optical bench tests; and future work.

Eric G Cooper↗
NASA NTRS2021

Collaborative Pose Estimation of An Unknown Target Using Multiple Spacecraft

A reliable method for pose estimation of an unknown and uncooperative space target using monocular vision remains an open problem. Vision-based pose determination can be challenging in case of unfavorable illumination, time-varying conditions due to rotational motion and relative orbit, and scale ambiguity resolution. To address these challenges, we propose a novel collaborative pose determination algorithm called Multi- Spacecraft Simultaneous Estimation of Pose and Shape algorithm or M-SEPS.Within M-SEPS, a team of chaser spacecraft, each equipped with a monocular camera, exchange information over a local network to jointly estimate the relative kinematic state of the target and its sparse shape landmarks. In this approach, each spacecraft processes its own images and observes particular target landmarks in parallel and in a distributed fashion. Then, the local network is exploited by the spacecraft to share their consensus proposals and aggregate them to achieve the joint estimate. We validate our algorithm using simulations of relative orbits and observations, captured by each chaser spacecraft. To the best of the authors’ knowledge, this is the first cooperative, vision-based algorithm for estimating the pose and shape of a space object for an arbitrary number of spacecraft.

Chung, Soon-Jo↗
NASA NTRS2022

Robust Vision-based Multi-spacecraft Guidance Navigation and Control using CNN-based Pose Estimation

In this paper, we present an end-to-end simulation framework for tracking an uncooperative Target spacecraft in Low Earth Orbit using a CubeSat-class Ego spacecraft outfitted with a camera. Currently, capturing high-fidelity realistic images in space for this scenario is difficult and exorbitantly expensive. Therefore, we developed a framework to simulate the spacecraft orbits in Basilisk software and generate high-fidelity realistic images of spacecraft in Unreal Engine, including the effects from Sun, Earth, Moon and stars. The Ego spacecraft uses cameras to capture images of the uncooperative Target and estimates its position and attitude using a CNN based 6DOF pose estimation pipeline, eliminating need for large SWAP-C(Size, Weight, Power and Cost) sensors like LIDAR or reliance on inter-spacecraft communication, This CNN, which is motivated by ESA’s Pose Estimation challenge of 2019, is trained using simulated data from our end-to-end simulation framework. We compare the performance of two distinct CNNbased algorithms for pose estimation along a nominal trajectory. In presence of non-Gaussian modeling uncertainties, the statedependent estimation error is characterized with a quadratic upper-bound. The quadratically-bounded error can be used by a robust controller to maneuver

Rahmani, Amir↗
NASA NTRS2012

Ground Simulation of an Autonomous Satellite Rendezvous and Tracking System Using Dual Robotic Systems

A hardware-in-the-loop ground system was developed for simulating a robotic servicer spacecraft tracking a target satellite at short range. A relative navigation sensor package "Argon" is mounted on the end-effector of a Fanuc 430 manipulator, which functions as the base platform of the robotic spacecraft servicer. Machine vision algorithms estimate the pose of the target spacecraft, mounted on a Rotopod R-2000 platform, relay the solution to a simulation of the servicer spacecraft running in "Freespace", which performs guidance, navigation and control functions, integrates dynamics, and issues motion commands to a Fanuc platform controller so that it tracks the simulated servicer spacecraft. Results will be reviewed for several satellite motion scenarios at different ranges. Key words: robotics, satellite, servicing, guidance, navigation, tracking, control, docking.

Trube, Matthew J.↗
NASA NTRS2022

Vision-Based Precision Approach and Landing for Advanced Air Mobility

Advanced Air Mobility (AAM) aircraft require perception systems for precision approach and landing systems (PALS) in urban, suburban, rural, and regional environments. The current state-of-the-art methods approved for automated approach and landing will be difficult to utilize in support of AAM operational concepts. However, there are technology and systems from other applications and lower-TRL research that use vision, IR, radar, and GPS methods to provide baseline perception and sensing requirements for AAM aircraft approach and landing. This paper focuses on vision-based PAL to demonstrate a closed-loop baseline controller while adhering to the Federal Aviation Administration requirements and regulations. The coplanar algorithm determines pose estimation, which feeds into an Extended Kalman filter. Combining IMU with vision creates a sensor fusion navigation solution for GPS-denied environments. The state estimate leads to glideslope and localizer error computations, which will be pertinent for designing and deriving guidance laws and control laws for AAM PALS. The IMU and vision navigation solution provides promising simulation results for AAM PALS, and higher fidelity simulations will include computer graphics rendering and feature correspondence.

Evan Kawamura↗
NASA NTRSDate not supplied

Distributed Sensing and Computer Vision Methods for Advanced Air Mobility Approach and Landing

Advanced Air Mobility (AAM) aircraft require precision approach and landing systems (PALS) in several types of environments such as urban, suburban, and rural. It is difficult to implement current state-of-the-art methods approved for automated approach and landing for AAM operations. However, existing technology and systems that use vision, IR, radar, and GPS methods provide baseline perception and sensing requirements for AAM aircraft approach and landing. This paper focuses on vision-based PAL and computer vision feature correspondence methods to demonstrate a baseline navigation system while adhering to the Federal Aviation Administration requirements and regulations. The coplanar algorithm determines pose estimation, which feeds into an Extended Kalman filter that combines IMU with vision to create a sensor fusion navigation solution for GPS-denied environments. The state estimate leads to glideslope and localizer error computations, which will be pertinent for designing and deriving guidance laws and control laws for AAM PALS. The IMU and vision navigation solution provides promising simulation results for AAM PALS. This paper builds on previous work by incorporating high fidelity simulations with computer graphics rendering to demonstrate a distributed sensor network to track an AAM aircraft during approach and landing to compare with the aircraft's onboard navigation solution.

Evan Kawamura↗
NASA NTRSDate not supplied

Simulated Vision-based Approach and Landing System for Advanced Air Mobility

Advanced Air Mobility (AAM) aircraft require precision approach and landing systems (PALS) in several environments, such as urban, suburban, and rural. It is challenging to implement current state-of-the-art methods approved for automated approach and landing for AAM operations with challenges such as GPS degradation in urban environments and visual navigation aids like the glideslope and localizer being narrow and not allowing alternative incoming landing angles at vertiports. However, existing technology and systems, i.e., the instrument landing system (ILS) with glideslope and localizer indicators that use vision, IR, radar, or GPS methods, provide baseline perception and sensing requirements for AAM aircraft approach and landing. This paper focuses on vision-based PAL and computer vision feature correspondence methods to demonstrate a baseline navigation system while adhering to the Federal Aviation Administration requirements and regulations about heliport design (FAA AC 150/5390-2C), which is one of the closest references for vertiport requirements and regulations. The coplanar pose from orthography and scaling with iterations (COPOSIT) algorithm determines pose estimation, which feeds into an Extended Kalman filter that combines IMU with vision to create a vision-based approach and landing (VAL) sensor fusion navigation solution for GPS-denied environments. The VAL navigation solution provides promising simulation results for AAM PALS with Hough circle detection and feature correspondence, which demonstrate robustness to false positives. This paper incorporates moderately high- fidelity simulations with computer graphics rendering to show a distributed sensor network to track an AAM aircraft during approach and landing to compare with the aircraft’s onboard vision-based navigation solution.

distributed sensing↗
NASA NTRS1991

Autonomous proximity operations using machine vision for trajectory control and pose estimation

A machine vision algorithm was developed which permits guidance control to be maintained during autonomous proximity operations. At present this algorithm exists as a simulation, running upon an 80386 based personal computer, using a ModelMATE CAD package to render the target vehicle. However, the algorithm is sufficiently simple, so that following off-line training on a known target vehicle, it should run in real time with existing vision hardware. The basis of the algorithm is a sequence of single camera images of the target vehicle, upon which radial transforms were performed. Selected points of the resulting radial signatures are fed through a decision tree, to determine whether the signature matches that of the known reference signatures for a particular view of the target. Based upon recognized scenes, the position of the maneuvering vehicle with respect to the target vehicles can be calculated, and adjustments made in the former's trajectory. In addition, the pose and spin rates of the target satellite can be estimated using this method.

Cleghorn, Timothy F.↗
NASA NTRS2002

Linear pose estimation from points or lines

We present a general framework which allows for a novel set of linear solutions to the pose estimation problem for both n points and n lines. We present a number of simulations which compare our results to two other recent linear algorithm as well as to iterative approaches.

pose estimation algorithms↗
NASA NTRS1994

Object recognition and pose estimation of planar objects from range data

The Extravehicular Activity Helper/Retriever (EVAHR) is a robotic device currently under development at the NASA Johnson Space Center that is designed to fetch objects or to assist in retrieving an astronaut who may have become inadvertently de-tethered. The EVAHR will be required to exhibit a high degree of intelligent autonomous operation and will base much of its reasoning upon information obtained from one or more three-dimensional sensors that it will carry and control. At the highest level of visual cognition and reasoning, the EVAHR will be required to detect objects, recognize them, and estimate their spatial orientation and location. The recognition phase and estimation of spatial pose will depend on the ability of the vision system to reliably extract geometric features of the objects such as whether the surface topologies observed are planar or curved and the spatial relationships between the component surfaces. In order to achieve these tasks, three-dimensional sensing of the operational environment and objects in the environment will therefore be essential. One of the sensors being considered to provide image data for object recognition and pose estimation is a phase-shift laser scanner. The characteristics of the data provided by this scanner have been studied and algorithms have been developed for segmenting range images into planar surfaces, extracting basic features such as surface area, and recognizing the object based on the characteristics of extracted features. Also, an approach has been developed for estimating the spatial orientation and location of the recognized object based on orientations of extracted planes and their intersection points. This paper presents some of the algorithms that have been developed for the purpose of recognizing and estimating the pose of objects as viewed by the laser scanner, and characterizes the desirability and utility of these algorithms within the context of the scanner itself, considering data quality and noise.

Pendleton, Thomas W.↗
NASA NTRSDate not supplied

Evaluation of Markerless Motion Capture for Monitoring Sensorimotor Performance

BACKGROUND Astronauts returning from long-duration exposure to microgravity frequently exhibit alterations in sensorimotor function leading to postural imbalance, impaired locomotion, and operational challenges to manual control. Mission duration and individual responses often influence both the severity of performance decrements and the variability in adaptation timelines. Postflight disruptions during functional tasks are often detected through body-worn inertial measurement unit (IMU) devices. While IMU sensors are relatively compact, the long-term wear may lead to discomfort, displacement of the sensors on the body, and restrictions in movement or crew behavior. Although IMU data offers valuable insights from a research standpoint, interpreting changes in pre- and post-flight measures can be difficult for crew support personnel beyond the research domain, which can hinder the application for medical assessments and rehabilitation. Finally, the availability of inertial sensors in-flight is limited. There is a need for unobtrusive monitoring tools to improve our ability to monitor adaptation following gravitational transitions in various postflight evaluations and rehabilitation settings. Markerless motion capture (MMC) is an evolving unobtrusive technology that builds upon decades of research with marker-based motion capture systems to provide 3D human pose estimation from multiple synchronized 2D camera views using deep learning algorithms. Markerless technology can revolutionize how data is captured pre- and post-flight and potentially in-flight during intravehicular activity by enabling pose estimation of multiple crew members from onboard camera hardware. METHODS The following presents the initial evaluation of a state-of-the-art commercial-off-the-shelf MMC system, Theia Markerless, compared to IMU devices during various ground-based functional tasks and environmental conditions. The featured functional tasks include assessments from Human Research Program (HRP) funded studies such as Sensorimotor Standard Measures and Sensorimotor Assessments. Synchronous data collected using both motion capture and IMUs are analyzed for six male and female subjects of varying anthropometry. The analysis includes limited assessments of clothing, capture volume configurations, and the tool's sensitivity to detecting performance changes after a spaceflight analog centrifuge exposure. The development of visualization tools to enhance the application of the pose estimation output is also presented. RESULTS Initial results demonstrate comparable root mean square error (RMSE) to existing literature evaluating markerless and marker-based motion capture systems. Considering the relative functional range of motion of the cervical spine, normalized error values for the markerless system’s accuracy of the head was 0.032 in pitch, 0.025 in roll, and 0.018 in yaw plane of motion across a subset of functional tasks. The raw RMSE values were 3.49, 2.25, and 2.81 degrees respectively. For the torso, results suggest normalized errors of 0.469 in pitch, 0.191 in roll, and 0.307 in yaw planes of motion and raw RMSE values of 3.52, 1.53, and 3.07 degrees respectively. The data suggests the functional demands of a particular task influences the estimation accuracy of the MMC system where more dynamic motion and cases where subjects are not upright may introduce diminished tracking accuracy. DISCUSSION The following work lays the foundation for future implementations leveraging markerless motion capture to assess the time course of recovery and provide insight for rehabilitation protocols to enhance crew readiness for the resumption of daily activities. These tools offer effective methods for anonymizing sensitive crew data, facilitating numerous applications across research, medical, and rehabilitation groups. Collaborations with the Anthropometry and Biomechanics Facility will provide further comparisons of the Markerless system to a marker-based system. ACKNOWLEDGEMENT</ This work is supported by NASA’s Exploration Systems Development Mission Directorate Mars Campaign Office Crew Health Countermeasures.

Hannah M. Weiss↗
NASA NTRS2017

Smoothing-Based Relative Navigation and Coded Aperture Imaging

This project will develop an efficient smoothing software for incremental estimation of the relative poses and velocities between multiple, small spacecraft in a formation, and a small, long range depth sensor based on coded aperture imaging that is capable of identifying other spacecraft in the formation. The smoothing algorithm will obtain the maximum a posteriori estimate of the relative poses between the spacecraft by using all available sensor information in the spacecraft formation.This algorithm will be portable between different satellite platforms that possess different sensor suites and computational capabilities, and will be adaptable in the case that one or more satellites in the formation become inoperable. It will obtain a solution that will approach an exact solution, as opposed to one with linearization approximation that is typical of filtering algorithms. Thus, the algorithms developed and demonstrated as part of this program will enhance the applicability of small spacecraft to multi-platform operations, such as precisely aligned constellations and fractionated satellite systems.

Relative positioning↗
NASA NTRS2016

Visual Odometry for Autonomous Deep-Space Navigation Project

Autonomous rendezvous and docking (AR&D) is a critical need for manned spaceflight, especially in deep space where communication delays essentially leave crews on their own for critical operations like docking. Previously developed AR&D sensors have been large, heavy, power-hungry, and may still require further development (e.g. Flash LiDAR). Other approaches to vision-based navigation are not computationally efficient enough to operate quickly on slower, flight-like computers. The key technical challenge for visual odometry is to adapt it from the current terrestrial applications it was designed for to function in the harsh lighting conditions of space. This effort leveraged Draper Laboratory’s considerable prior development and expertise, benefitting both parties. The algorithm Draper has created is unique from other pose estimation efforts as it has a comparatively small computational footprint (suitable for use onboard a spacecraft, unlike alternatives) and potentially offers accuracy and precision needed for docking. This presents a solution to the AR&D problem that only requires a camera, which is much smaller, lighter, and requires far less power than competing AR&D sensors. We have demonstrated the algorithm’s performance and ability to process ‘flight-like’ imagery formats with a ‘flight-like’ trajectory, positioning ourselves to easily process flight data from the upcoming ‘ISS Selfie’ activity and then compare the algorithm’s quantified performance to the simulated imagery. This will bring visual odometry beyond TRL 5, proving its readiness to be demonstrated as part of an integrated system.Once beyond TRL 5, visual odometry will be poised to be demonstrated as part of a system in an in-space demo where relative pose is critical, like Orion AR&D, ISS robotic operations, asteroid proximity operations, and more.

Robinson, Shane↗
NASA NTRS2016

Visual Odometry for Autonomous Deep-Space Navigation Project

Autonomous rendezvous and docking (AR&D) is a critical need for manned spaceflight, especially in deep space where communication delays essentially leave crews on their own for critical operations like docking. Previously developed AR&D sensors have been large, heavy, power-hungry, and may still require further development (e.g. Flash LiDAR). Other approaches to vision-based navigation are not computationally efficient enough to operate quickly on slower, flight-like computers. The key technical challenge for visual odometry is to adapt it from the current terrestrial applications it was designed for to function in the harsh lighting conditions of space. This effort leveraged Draper Laboratory's considerable prior development and expertise, benefitting both parties. The algorithm Draper has created is unique from other pose estimation efforts as it has a comparatively small computational footprint (suitable for use onboard a spacecraft, unlike alternatives) and potentially offers accuracy and precision needed for docking. This presents a solution to the AR&D problem that only requires a camera, which is much smaller, lighter, and requires far less power than competing AR&D sensors. We have demonstrated the algorithm's performance and ability to process 'flight-like' imagery formats with a 'flight-like' trajectory, positioning ourselves to easily process flight data from the upcoming 'ISS Selfie' activity and then compare the algorithm's quantified performance to the simulated imagery. This will bring visual odometry beyond TRL 5, proving its readiness to be demonstrated as part of an integrated system. Once beyond TRL 5, visual odometry will be poised to be demonstrated as part of a system in an in-space demo where relative pose is critical, like Orion AR&D, ISS robotic operations, asteroid proximity operations, and more.

Robinson, Shane↗
NASA NTRS2013

Autonomous Vision-Based Tethered-Assisted Rover Docking

Many intriguing science discoveries on planetary surfaces, such as the seasonal flows on crater walls and skylight entrances to lava tubes, are at sites that are currently inaccessible to state-of-the-art rovers. The in situ exploration of such sites is likely to require a tethered platform both for mechanical support and for providing power and communication. Mother/daughter architectures have been investigated where a mother deploys a tethered daughter into extreme terrains. Deploying and retracting a tethered daughter requires undocking and re-docking of the daughter to the mother, with the latter being the challenging part. In this paper, we describe a vision-based tether-assisted algorithm for the autonomous re-docking of a daughter to its mother following an extreme terrain excursion. The algorithm uses fiducials mounted on the mother to improve the reliability and accuracy of estimating the pose of the mother relative to the daughter. The tether that is anchored by the mother helps the docking process and increases the system's tolerance to pose uncertainties by mechanically aligning the mating parts in the final docking phase. A preliminary version of the algorithm was developed and field-tested on the Axel rover in the JPL Mars Yard. The algorithm achieved an 80% success rate in 40 experiments in both firm and loose soils and starting from up to 6 m away at up to 40 deg radial angle and 20 deg relative heading. The algorithm does not rely on an initial estimate of the relative pose. The preliminary results are promising and help retire the risk associated with the autonomous docking process enabling consideration in future martian and lunar missions.

Axel docking↗
NASA NTRS2012

Fully Self-Contained Vision-Aided Navigation and Landing of a Micro Air Vehicle Independent from External Sensor Inputs

Direct-lift micro air vehicles have important applications in reconnaissance. In order to conduct persistent surveillance in urban environments, it is essential that these systems can perform autonomous landing maneuvers on elevated surfaces that provide high vantage points without the help of any external sensor and with a fully contained on-board software solution. In this paper, we present a micro air vehicle that uses vision feedback from a single down looking camera to navigate autonomously and detect an elevated landing platform as a surrogate for a roof top. Our method requires no special preparation (labels or markers) of the landing location. Rather, leveraging the planar character of urban structure, the landing platform detection system uses a planar homography decomposition to detect landing targets and produce approach waypoints for autonomous landing. The vehicle control algorithm uses a Kalman filter based approach for pose estimation to fuse visual SLAM (PTAM) position estimates with IMU data to correct for high latency SLAM inputs and to increase the position estimate update rate in order to improve control stability. Scale recovery is achieved using inputs from a sonar altimeter. In experimental runs, we demonstrate a real-time implementation running on-board a micro aerial vehicle that is fully self-contained and independent from any external sensor information. With this method, the vehicle is able to search autonomously for a landing location and perform precision landing maneuvers on the detected targets.

autonomous landing↗
NASA NTRS2006

2D/3D Visual Tracker for Rover Mast

A visual-tracker computer program controls an articulated mast on a Mars rover to keep a designated feature (a target) in view while the rover drives toward the target, avoiding obstacles. Several prior visual-tracker programs have been tested on rover platforms; most require very small and well-estimated motion between consecutive image frames a requirement that is not realistic for a rover on rough terrain. The present visual-tracker program is designed to handle large image motions that lead to significant changes in feature geometry and photometry between frames. When a point is selected in one of the images acquired from stereoscopic cameras on the mast, a stereo triangulation algorithm computes a three-dimensional (3D) location for the target. As the rover moves, its body-mounted cameras feed images to a visual-odometry algorithm, which tracks two-dimensional (2D) corner features and computes their old and new 3D locations. The algorithm rejects points, the 3D motions of which are inconsistent with a rigid-world constraint, and then computes the apparent change in the rover pose (i.e., translation and rotation). The mast pan and tilt angles needed to keep the target centered in the field-of-view of the cameras (thereby minimizing the area over which the 2D-tracking algorithm must operate) are computed from the estimated change in the rover pose, the 3D position of the target feature, and a model of kinematics of the mast. If the motion between the consecutive frames is still large (i.e., 3D tracking was unsuccessful), an adaptive view-based matching technique is applied to the new image. This technique uses correlation-based template matching, in which a feature template is scaled by the ratio between the depth in the original template and the depth of pixels in the new image. This is repeated over the entire search window and the best correlation results indicate the appropriate match. The program could be a core for building application programs for systems that require coordination of vision and robotic motion.

Bajracharya, Max↗