Search NASA⌕ Search

SEARCH · Search NASA

Results for “pose estimation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Distributed Sensing and Computer Vision Methods for Advanced Air Mobility Approach and Landing

Advanced Air Mobility (AAM) aircraft require precision approach and landing systems (PALS) in several types of environments such as urban, suburban, and rural. It is difficult to implement current state-of-the-art methods approved for automated approach and landing for AAM operations. However, existing technology and systems that use vision, IR, radar, and GPS methods provide baseline perception and sensing requirements for AAM aircraft approach and landing. This paper focuses on vision-based PAL and computer vision feature correspondence methods to demonstrate a baseline navigation system while adhering to the Federal Aviation Administration requirements and regulations. The coplanar algorithm determines pose estimation, which feeds into an Extended Kalman filter that combines IMU with vision to create a sensor fusion navigation solution for GPS-denied environments. The state estimate leads to glideslope and localizer error computations, which will be pertinent for designing and deriving guidance laws and control laws for AAM PALS. The IMU and vision navigation solution provides promising simulation results for AAM PALS. This paper builds on previous work by incorporating high fidelity simulations with computer graphics rendering to demonstrate a distributed sensor network to track an AAM aircraft during approach and landing to compare with the aircraft's onboard navigation solution.

Evan Kawamura↗
NASA NTRSDate not supplied

Simulated Vision-based Approach and Landing System for Advanced Air Mobility

Advanced Air Mobility (AAM) aircraft require precision approach and landing systems (PALS) in several environments, such as urban, suburban, and rural. It is challenging to implement current state-of-the-art methods approved for automated approach and landing for AAM operations with challenges such as GPS degradation in urban environments and visual navigation aids like the glideslope and localizer being narrow and not allowing alternative incoming landing angles at vertiports. However, existing technology and systems, i.e., the instrument landing system (ILS) with glideslope and localizer indicators that use vision, IR, radar, or GPS methods, provide baseline perception and sensing requirements for AAM aircraft approach and landing. This paper focuses on vision-based PAL and computer vision feature correspondence methods to demonstrate a baseline navigation system while adhering to the Federal Aviation Administration requirements and regulations about heliport design (FAA AC 150/5390-2C), which is one of the closest references for vertiport requirements and regulations. The coplanar pose from orthography and scaling with iterations (COPOSIT) algorithm determines pose estimation, which feeds into an Extended Kalman filter that combines IMU with vision to create a vision-based approach and landing (VAL) sensor fusion navigation solution for GPS-denied environments. The VAL navigation solution provides promising simulation results for AAM PALS with Hough circle detection and feature correspondence, which demonstrate robustness to false positives. This paper incorporates moderately high- fidelity simulations with computer graphics rendering to show a distributed sensor network to track an AAM aircraft during approach and landing to compare with the aircraft’s onboard vision-based navigation solution.

distributed sensing↗
NASA NTRSDate not supplied

Improving Sim-to-Real Transfer in Vision-Based Robot Navigation Via Instance-Level GAN-Based Data Augmentation

Achieving robust vision-based robotic tasks requires large amounts of data, which are often difficult to obtain in real-world scenarios. Simulators and synthetic data offer a cost-effective alternative, but the visual gap between simulation and reality hinders the performance of models when deployed in real-world environments. In this paper, we present a data augmentation pipeline that integrates a foundation model (Segment Anything Model) with an unsupervised image-to-image translation model (CycleGAN) for instance-level domain transfer from simulation to reality. This pipeline enables the generation of realistic labeled data from synthetic images for training supervised machine learning models in vision-based navigation tasks. We evaluate our approach on real-world data for ego-vehicle pose estimation, a critical autonomous navigation task involving the prediction of cross-track position and heading angle relative to road center line markings. The results of our tests show that our GAN-based data augmentation pipeline significantly outperforms models trained solely on simulation data or on data processed with standard image augmentation methods for sim-to-real transfer, enhancing model robustness and generalizability in real-world scenarios. Our method provides a scalable and flexible data augmentation tool for leveraging large synthetic datasets to enhance vision-based robotic navigation tasks.

artificial intelligence↗
NASA NTRS2013

Autonomous Vision-Based Tethered-Assisted Rover Docking

Many intriguing science discoveries on planetary surfaces, such as the seasonal flows on crater walls and skylight entrances to lava tubes, are at sites that are currently inaccessible to state-of-the-art rovers. The in situ exploration of such sites is likely to require a tethered platform both for mechanical support and for providing power and communication. Mother/daughter architectures have been investigated where a mother deploys a tethered daughter into extreme terrains. Deploying and retracting a tethered daughter requires undocking and re-docking of the daughter to the mother, with the latter being the challenging part. In this paper, we describe a vision-based tether-assisted algorithm for the autonomous re-docking of a daughter to its mother following an extreme terrain excursion. The algorithm uses fiducials mounted on the mother to improve the reliability and accuracy of estimating the pose of the mother relative to the daughter. The tether that is anchored by the mother helps the docking process and increases the system's tolerance to pose uncertainties by mechanically aligning the mating parts in the final docking phase. A preliminary version of the algorithm was developed and field-tested on the Axel rover in the JPL Mars Yard. The algorithm achieved an 80% success rate in 40 experiments in both firm and loose soils and starting from up to 6 m away at up to 40 deg radial angle and 20 deg relative heading. The algorithm does not rely on an initial estimate of the relative pose. The preliminary results are promising and help retire the risk associated with the autonomous docking process enabling consideration in future martian and lunar missions.

Axel docking↗
NASA NTRS2019

Identification of Bare-Airframe Dynamics from Closed-Loop Data Using Multisine Inputs and Frequency Responses

Amethod is presented for computing multiple-input multiple-output frequency responses of bare-airframe dynamics for systems excited using orthogonal phase-optimized multisines and including correlated data arising from control mixing or feedback control. The estimation was posed as the solution to an underdetermined system of linear equations, for which additional information was supplied using interpolation of the frequency responses. A simulation model of the NASA T-2 aircraft having two inputs and two outputs was used to investigate the method in the open-loop configuration and under closed-loop control. The method was also applied to flight test data from the X-56A aeroelastic demonstrator having five inputs and ten outputs and flying under closed-loop control with additional control allocation mixing. Results demonstrated that the proposed method accurately estimates the bare airframe frequency responses in the presence of correlated data from control mixing and feedback control. Results also agreed with estimates obtained using different methods that are less sensitive to correlated inputs.

Grauer, Jared A.↗
NASA NTRS1983

Identification of helicopter rotor dynamic models

A recursive, extended Kalman-filter approach is applied to the identifiction of rotor damping levels of representative helicopter dynamic systems. The general formulation of the approach is presented in the context of a typically posed stochastic estimation problem, and the method is analytically applied to determining the damping levels of a coupled rotor-body system. The identified damping covergence characteristics are studied for sensitivity to both constant-coefficient and periodic-coefficient measurement models, process-noise covariance levels, and specified initial estimates of the rotor-system damping. A second application of the method to identifying the plant model for a highly damped, isolated flapping blade with a constant-coefficient state model (hover) and a periodic-coefficient state model (forward flight) is also investigated. The parameter-identification capability is evaluated for the effect of periodicity on the plant model coefficients and the influence of different measurement noise levels.

Molusis, J. A.↗
NASA NTRS2003

Human-Robot Control Strategies for the NASA/DARPA Robonaut

The Robotic Systems Technology Branch at the NASA Johnson Space Center (JSC) is currently developing robot systems to reduce the Extra-Vehicular Activity (EVA) and planetary exploration burden on astronauts. One such system, Robonaut, is capable of interfacing with external Space Station systems that currently have only human interfaces. Robonaut is human scale, anthropomorphic, and designed to approach the dexterity of a space-suited astronaut. Robonaut can perform numerous human rated tasks, including actuating tether hooks, manipulating flexible materials, soldering wires, grasping handrails to move along space station mockups, and mating connectors. More recently, developments in autonomous control and perception for Robonaut have enabled dexterous, real-time man-machine interaction. Robonaut is now capable of acting as a practical autonomous assistant to the human, providing and accepting tools by reacting to body language. A versatile, vision-based algorithm for matching range silhouettes is used for monitoring human activity as well as estimating tool pose.

Diftler, M. A.↗
NASA NTRS2012

Ground Simulation of an Autonomous Satellite Rendezvous and Tracking System Using Dual Robotic Systems

A hardware-in-the-loop ground system was developed for simulating a robotic servicer spacecraft tracking a target satellite at short range. A relative navigation sensor package "Argon" is mounted on the end-effector of a Fanuc 430 manipulator, which functions as the base platform of the robotic spacecraft servicer. Machine vision algorithms estimate the pose of the target spacecraft, mounted on a Rotopod R-2000 platform, relay the solution to a simulation of the servicer spacecraft running in "Freespace", which performs guidance, navigation and control functions, integrates dynamics, and issues motion commands to a Fanuc platform controller so that it tracks the simulated servicer spacecraft. Results will be reviewed for several satellite motion scenarios at different ranges. Key words: robotics, satellite, servicing, guidance, navigation, tracking, control, docking.

Trube, Matthew J.↗
NASA NTRSDate not supplied

Entry, Descent, and Landing GN&C System Evaluation via Cable-Driven Emulation Robotics

An innovative cable-driven parallel robot named the Six Degree-of-Freedom Tendon Actu- ated Robot (STAR), recently developed by researchers at NASA Johnson Space Center, was built to enable entry, descent, and landing (EDL) emulation experiments. Upon functional testing and system verification, STAR employed several simulated EDL trajectories to validate velocimeter LIDAR-based sensor models for cooperative terrain relative navigation applica- tions for its first testing campaign. The STAR lab also hosted researchers from Astrobotic to evaluate the real-time performance of a newly developed sensor payload, UltraNav, through hardware in-the-loop-testing (HWIL). This work introduces the STAR system along with its accompanying testing utilities and capabilities, and deliver the experimental methodology and results of test campaigns performed with the STAR facility to date.

navigation↗
NASA NTRSDate not supplied

Spaceborne Passive-Microwave and Visible/IR Observations of Severe Weather: Leveraging Multiple Perspectives for Detection, Validation, and Climatologies

Severe weather phenomena not only are responsible for damages to property, infrastructure, and agriculture, they are also difficult to measure in-situ and have been associated with large errors and uncertainties in precipitation estimation that pose challenges to establishing the current climatology of severe storms and their future trend in a changing Earth system. Severe convection exhibits distinct signatures in remote-sensing datasets, where it is manifested as textured or overshooting cloud tops in visible/IR imagery, or resulting a prominent brightness temperature depression in passive-microwave imagery. These distinct deep convective signatures in spaceborne datasets have been leveraged to analyze severe thunderstorms, create climatologies, improve prediction, and provide a method of detection around the globe where traditional ground-based data may be inconsistent or unavailable. Visible/IR and passive-microwave instruments are powerful tools for detecting severe thunderstorms, however, these datasets exhibit key limitations when examined individually: passive-microwave channels have large footprint and exhibit non-uniform beam filling. The visible/IR instruments have fine horizontal resolution but are limited by their insensitivity to processes occurring below cloud top. We investigate the nearly simultaneous and colocated MODIS (visible/IR) and AMSR-E (passive-microwave) onboard the Aqua satellite to leverage both datasets together and assess the extent to which these datasets can be combined to improve severe thunderstorm detection. We pair passive microwave, IR, and visible signatures of severe convection with ground-based weather radar, severe weather reports, and environmental parameters defined by the MERRA-2 reanalysis to characterize potentially severe convective storms observed by Aqua MODIS and AMSR-E. We will discuss how MODIS and AMSR-E signatures may be used together to diagnose storm properties and processes, and how the interrelationships between the signatures varies seasonally and geographically.

Sarah D. Bang↗
NASA NTRSDate not supplied

Spacecraft Operation Emulation Via Six DOF Tendon-Actuated Robot

NASA Johnson Space Center’s (JSC) Simulation, Emulation, Navigation, Sensors and STAR (SENSS) Laboratory has developed a large-scale six degree of freedom tendon-actuated robot (STAR) for emulation of spacecraft proximity operations and Entry, Descent, and Landing approach trajectories to aid in the rapid development and testing of a wide range of guidance, navigation, and control algorithms at low cost.

navigation↗
NASA NTRSDate not supplied

Guidance, Navigation, and Control for Laser-Based Non-Cooperative Target Detumbling

Orbital debris is a well known and growing problem presenting risks to the future of spaceflight. Many pieces of debris are defunct satellites or rocket upper stages. These large objects could generate an exponentially growing cloud of debris should they be involved in a collision, i.e., Kessler Syndrome. Future orbital debris remediation efforts often propose perturbing the orbit of these large targets to mitigate risk. These plans often involve non-cooperative rendezvous. In the absence of active control, the targets are expected to be tumbling which makes rendezvous a dangerous prospect. In this work we describe a guidance, navigation, and control system for a contactless non-cooperative target detumbling methodology which will significantly increase the safety of non-cooperative rendezvous. An onboard laser generates thrust on the target by ablating its surface material. This thrust is used to control target attitude. Our guidance, navigation, and control system is able to estimate relative pose, target the laser, and maintain formation flight during the detumbling process. The system is validated with Monte Carlo simulation in a variety of scenarios.

Guidance↗
NASA NTRS2011

Visual Object Recognition and Tracking of Tools

A method has been created to automatically build an algorithm off-line, using computer-aided design (CAD) models, and to apply this at runtime. The object type is discriminated, and the position and orientation are identified. This system can work with a single image and can provide improved performance using multiple images provided from videos. The spatial processing unit uses three stages: (1) segmentation; (2) initial type, pose, and geometry (ITPG) estimation; and (3) refined type, pose, and geometry (RTPG) calculation. The image segmentation module files all the tools in an image and isolates them from the background. For this, the system uses edge-detection and thresholding to find the pixels that are part of a tool. After the pixels are identified, nearby pixels are grouped into blobs. These blobs represent the potential tools in the image and are the product of the segmentation algorithm. The second module uses matched filtering (or template matching). This approach is used for condensing synthetic images using an image subspace that captures key information. Three degrees of orientation, three degrees of position, and any number of degrees of freedom in geometry change are included. To do this, a template-matching framework is applied. This framework uses an off-line system for calculating template images, measurement images, and the measurements of the template images. These results are used online to match segmented tools against the templates. The final module is the RTPG processor. Its role is to find the exact states of the tools given initial conditions provided by the ITPG module. The requirement that the initial conditions exist allows this module to make use of a local search (whereas the ITPG module had global scope). To perform the local search, 3D model matching is used, where a synthetic image of the object is created and compared to the sensed data. The availability of low-cost PC graphics hardware allows rapid creation of synthetic images. In this approach, a function of orientation, distance, and articulation is defined as a metric on the difference between the captured image and a synthetic image with an object in the given orientation, distance, and articulation. The synthetic image is created using a model that is looked up in an object-model database. A composable software architecture is used for implementation. Video is first preprocessed to remove sensor anomalies (like dead pixels), and then is processed sequentially by a prioritized list of tracker-identifiers.

English, James↗
NASA NTRS2023

EVA Task and 3D Pose Recognition from Video

Extravehicular Activity (EVA) has been known to involve potential risks of biomechanical stresses and injuries to crewmembers. Gathering of EVA motion patterns is necessary for risk analysis and mitigation. However, many existing techniques, such as motion capture systems, are not only cost-prohibitive but are impractical for retrospective analysis of past missions. In this work, a software tool was developed, which can estimate the 3D poses of a spacesuit from photographs or videos, without using special sensors or equipment. The tool is based on the state-of-the-art artificial intelligence and machine learning (AI/ML) system, which was trained by studying and capturing motion patterns of past and current spacesuit test data. The AI/ML tool was further enhanced using synthetically generated data, in which the suit postures, backgrounds, camera angles and illumination conditions were parametrically adjusted and rendered for training. The tool, incorporated the methodologies of Convolutional Neural Network (CNN), was trained, and tested in the cloud computing environment. The trained model was then applied on new imagery and video to extract estimated joint positions and suit outlines. The joint positions were further processed to capture activity (“digging”), pose labels (“bending”), and other useful downstream information. The model performance on new imagery and video was successfully assessed for accuracy and reliability. This AI/ML based posture recognition tool thus allows for the quantification of injury risk and task performance characterization for both current and past missions and training, which can immensely help to improve EVA task and suit design.

Kyung Han Kim↗
NASA NTRS1990

Joint transform correlation with the synthetic estimation filter

The synthetic estimation filter (SEF) combines a set of input images into a much smaller set of filters. Those input images correspond to views of a known object taken with a variety of values of the pose parameter to be estimated. The response of the filter to variations in the pose parameter is tailored so that estimation can be done to a precision considerably finer than the intervals between filters in the pose space. The technique is extended to the joint transform correlator; as a reference, the composite image from which the SEF is conventionally calculated is used. Precautions are necessary for practical reasons. For example, the SEF algorithm may call for negative values in the image, an unrealizable condition with amplitude encoding, whereas with phase-mostly filtering in the SEF, negative image values are easily accommodated.

Monroe, Stanley E., Jr.↗
NASA NTRS2017

Smoothing-Based Relative Navigation and Coded Aperture Imaging

This project will develop an efficient smoothing software for incremental estimation of the relative poses and velocities between multiple, small spacecraft in a formation, and a small, long range depth sensor based on coded aperture imaging that is capable of identifying other spacecraft in the formation. The smoothing algorithm will obtain the maximum a posteriori estimate of the relative poses between the spacecraft by using all available sensor information in the spacecraft formation.This algorithm will be portable between different satellite platforms that possess different sensor suites and computational capabilities, and will be adaptable in the case that one or more satellites in the formation become inoperable. It will obtain a solution that will approach an exact solution, as opposed to one with linearization approximation that is typical of filtering algorithms. Thus, the algorithms developed and demonstrated as part of this program will enhance the applicability of small spacecraft to multi-platform operations, such as precisely aligned constellations and fractionated satellite systems.

Relative positioning↗
NASA NTRS1990

Hybrid vision activities at NASA Johnson Space Center

NASA's Johnson Space Center in Houston, Texas, is active in several aspects of hybrid image processing. (The term hybrid image processing refers to a system that combines digital and photonic processing). The major thrusts are autonomous space operations such as planetary landing, servicing, and rendezvous and docking. By processing images in non-Cartesian geometries to achieve shift invariance to canonical distortions, researchers use certain aspects of the human visual system for machine vision. That technology flow is bidirectional; researchers are investigating the possible utility of video-rate coordinate transformations for human low-vision patients. Man-in-the-loop teleoperations are also supported by the use of video-rate image-coordinate transformations, as researchers plan to use bandwidth compression tailored to the varying spatial acuity of the human operator. Technological elements being developed in the program include upgraded spatial light modulators, real-time coordinate transformations in video imagery, synthetic filters that robustly allow estimation of object pose parameters, convolutionally blurred filters that have continuously selectable invariance to such image changes as magnification and rotation, and optimization of optical correlation done with spatial light modulators that have limited range and couple both phase and amplitude in their response.

Juday, Richard D.↗
NASA NTRS1989

Optical pattern recognition; Proceedings of the Meeting, Los Angeles, CA, Jan. 17, 18, 1989

Papers on optical pattern recognition are presented, covering topics such as the estimation of satellite pose and motion parameters using a neural net tracker, associative memory, optical implmentation of programmable neural networks, optoelectronic neural networks, dynamic autoassociative neural memory, heteroassociative memory, bilinear pattern recognition processors, optical processing of optical correlation plane data, and a synthetic discriminant function-based nonlinear optical correlator. Other topics include an interactive optical-digital image processor, geometric transformations for video compression and human teleoperator display, quasiconformal remapping for compensation of human visual field defects, hybrid vision for automated spacecraft landing, advanced symbolic and inference optical correlation filters, and a rotationally invariant holographic tracking system. Additional topics include the detection of rotational and scale-varying objects with a programmable joint transform correlator, a single spatial light modulator binary nonlinear optical correlator, optical joint transform correlation, linear phase coefficient composite filters, and binary phase-only filters.

Liu, Hua-Kuang↗