Search NASA⌕ Search

SEARCH · Search NASA

Results for “stereo vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

Close Range Photogrammetry in Space - Measuring the On-Orbit Clearance between Hardware on the International Space Station

When photogrammetrists read an article entitled "Photogrammetry in Space" they immediately think of terrestrial mapping using satellite imagery. However in the last 19 years the roll of close range photogrammetry in support of the manned space flight program has grown exponentially. Management and engineers have repeatedly entrusted the safety of the vehicles and their crews to the results of photogrammetric analysis. In February 2010, the Node 3 module was attached to the port side Common Berthing Mechanism (CBM) of the International Space Station (ISS). Since this was not the location at which the module was originally designed to be located on the ISS, coolant lines containing liquid ammonia, were installed externally from the US Lab to Node 3 during a spacewalk. During mission preparation I had developed a plan and a set of procedures to have the astronauts acquire stereo imagery of these coolant lines at the conclusion of the spacewalk to enable us to map their as-installed location relative to the rest of the space station. Unfortunately, the actual installation of the coolant lines took longer than expected and in an effort to wrap up the spacewalk on time, the mission director made a real-time call to drop the photography. My efforts to reschedule the photography on a later spacewalk never materialized, so rather than having an as-installed model for the location of coolant lines, the master ISS CAD database continued to display an as-designed model of the coolant lines. Fast forward to the summer of 2015, the ISS program planned to berth a Japanese cargo module to the nadir Common Berthing Mechanism (CBM), immediately adjacent to the Node 3 module. A CAD based clearance analysis revealed a negative four inch clearance between the ammonia lines and a thruster nozzle on the port side of the cargo vehicle. Recognizing that the model of the ammonia line used in the clearance analysis was "as-designed" rather than "as-installed", I was asked to determine the real clearance between the ammonia lines and expected position of the thruster bell using existing on-orbit imagery. Imagery of the area of interest, taken several years earlier from the Space Shuttle during a fly-around of the ISS, was found and used to set a stereo pair. Space Vision System Targets and Handrail bolts measured in the ISS analytical coordinate system (ISSACS) prior to launch, were used to obtain an absolute orientation so all photogrammetric measurement's would be in the ISSACS coordinate system. Coordinates for the design location of the edges of the thruster bell, when the cargo vehicle was fully berthed to the ISS, were displayed in 3-D relative to the as-installed ammonia lines. This immediately revealed a positive clearance, which was later quantified to be a minimum of 10" +/0.5". The analysis was completed over a single weekend by a single analyst. Using updated imagery, acquired from the station's robotic arm, a complete as-installed model of the coolant lines was generated from stereo photography and replaced the design model in the master ISS CAD database.

Liddle, Donn↗
NASA NTRS1990

Telepresence and Space Station Freedom workstation operations

The Space Station Freedom workstation system is a distributed network of computer based workstations that provides the man-machine interfaces for controlling space station systems. This includes control of external manipulator, robotic and free flyer devices by crewmembers in the space station's pressurized shirt-sleeve environment. These remotely controlled devices help minimize the requirement for costly crew extravehicular activity (EVA) time for such tasks as station assembly and payload support. Direct window views may be used for controlling some of the systems, but many activities will be remote or require levels of detail not possible by direct observation. Since controlling remote devices becomes more difficult when direct views are inadequate or unavailable, many performance enhancing techniques have been considered for representing information about remote activities to the operator. Described here are the telepresence techniques under consideration to support operations and training. This includes video enhancements (e.g., graphic and text overlays and stereo viewing), machine vision systems, remote activity animation, and force reflection representation.

Jensen, Dean G.↗
NASA NTRS2005

Vision Based Localization in Urban Environments

As part of DARPA's MARS2020 program, the Jet Propulsion Laboratory developed a vision-based system for localization in urban environments that requires neither GPS nor active sensors. System hardware consists of a pair of small FireWire cameras and a standard Pentium-based computer. The inputs to the software system consist of: 1) a crude grid-based map describing the positions of buildings, 2) an initial estimate of robot location and 3) the video streams produced by each camera. At each step during the traverse the system: captures new image data, finds image features hypothesized to lie on the outside of a building, computes the range to those features, determines an estimate of the robot's motion since the previous step and combines that data with the map to update a probabilistic representation of the robot's location. This probabilistic representation allows the system to simultaneously represent multiple possible locations, For our testing, we have derived the a priori map manually using non-orthorectified overhead imagery, although this process could be automated. The software system consists of two primary components. The first is the vision system which uses binocular stereo ranging together with a set of heuristics to identify features likely to be part of building exteriors and to compute an estimate of the robot's motion since the previous step. The resulting visual features and the associated range measurements are software component, a particle-filter based localization system. This system uses the map and the then fed to the second primary most recent results from the vision system to update the estimate of the robot's location. This report summarizes the design of both the hardware and software and will include the results of applying the system to the global localization of a robot over an approximately half-kilometer traverse across JPL'S Pasadena campus.

unmanned-ground vehicles↗
NASA NTRS1996

Field Geology/Processes

The field geology/process group examined the basic operations of a terrestrial field geologist and the manner in which these operations could be transferred to a planetary lander. Four basic requirements for robotic field geology were determined: geologic content; surface vision; mobility; and manipulation. Geologic content requires a combination of orbital and descent imaging. Surface vision requirements include range, resolution, stereo, and multispectral imaging. The minimum mobility for useful field geology depends on the scale of orbital imagery. Manipulation requirements include exposing unweathered surfaces, screening samples, and bringing samples in contact with analytical instruments. To support these requirements, several advanced capabilities for future development are recommended. Capabilities include near-infrared reflectance spectroscopy, hyper-spectral imaging, multispectral microscopy, artificial intelligence in support of imaging, x ray diffraction, x ray fluorescence, and rock chipping.

Allen, Carlton↗
NASA NTRS1994

Topography from shading and stereo

Methods exploiting photometric information in images that have been developed in machine vision can be applied to planetary imagery. Integrating shape from shading, binocular stereo, and photometric stereo yields a robust system for recovering detailed surface shape and surface reflectance information. Such a system is useful in producing quantitative information from the vast volume of imagery being received, as well as in helping visualize the underlying surface.

Horn, Berthold K. P.↗
NASA NTRS2002

Stereo transparency and the disparity gradient limit

Several studies (Vision Research 15 (1975) 583; Perception 9 (1980) 671) have shown that binocular fusion is limited by the disparity gradient (disparity/distance) separating image points, rather than by their absolute disparity values. Points separated by a gradient >1 appear diplopic. These results are sometimes interpreted as a constraint on human stereo matching, rather than a constraint on fusion. Here we have used psychophysical measurements on stereo transparency to show that human stereo matching is not constrained by a gradient of 1. We created transparent surfaces composed of many pairs of dots, in which each member of a pair was assigned a disparity equal and opposite to the disparity of the other member. For example, each pair could be composed of one dot with a crossed disparity of 6' and the other with uncrossed disparity of 6', vertically separated by a parametrically varied distance. When the vertical separation between the paired dots was small, the disparity gradient for each pair was very steep. Nevertheless, these opponent-disparity dot pairs produced a striking appearance of two transparent surfaces for disparity gradients ranging between 0.5 and 3. The apparent depth separating the two transparent planes was correctly matched to an equivalent disparity defined by two opaque surfaces. A test target presented between the two transparent planes was easily detected, indicating robust segregation of the disparities associated with the paired dots into two transparent surfaces with few mismatches in the target plane. Our simulations using the Tsai-Victor model show that the response profiles produced by scaled disparity-energy mechanisms can account for many of our results on the transparency generated by steep gradients.

NASA Discipline Space Human Factors↗
NASA NTRS1992

Visual specification of robot motion

The authors describe the use of stereo pairs of images to specify robot motion. The experimental setup includes a SUN workstation, a PUMA 560 robot, and an Imaging 151 vision system. An X-window environment displays stereo images of the work scene. Image processing is performed to extract linear edge segments from the images and the results are displayed on screen. Using a pointing device, the user selects a group of edges from the object relevant to the task. The 3D structure of this group of features is found by stereo triangulation and they can be displayed in 3D from any point of view. A viewpoint orthogonal to the plane defined by these 3D edges is used to specify the robot position relative to object position. The actual robot will then be moved to the specified position.

Shiu, Y. C.↗
NASA NTRS1992

Topography from shading and stereo

Methods exploiting photometric information in images that have been developed in machine vision can be applied to planetary imagery. Present techniques, however, focus on one visual cue, such as shading or binocular stereo, and produce results that are either not very accurate in an absolute sense or provide information only at few points on the surface. We plan to integrate shape from shading, binocular stereo and photometric stereo to yield a robust system for recovering detailed surface shape and surface reflectance information. Such a system will be useful in producing quantitative information from the vast volume of imagery being received, as well as in helping visualize the underlying surface. The work will be carried out on a popular computing platform so that it will be easily accessible to other workers.

Horn, Berthold P.↗
NASA NTRS1993

Topography from shading and stereo

Methods exploiting photometric information in images that have been developed in machine vision can be applied to planetary imagery. Present techniques, however, focus on one visual cue, such as shading or binocular stereo, and produce results that are either not very accurate in an absolute sense or provide information only at few points on the surface. We plan to integrate shape from shading, binocular stereo and photometric stereo to yield a robust system for recovering detailed surface shape and surface reflectance information. Such a system will be useful in producing quantitative information from the vast volume of imagery being received, as well as in helping visualize the underlying surface. The work will be carried out on a popular computing platform so that it will be easily accessible to other workers.

Horn, Berthold P.↗
NASA NTRS1987

Computing 3-D structure of rigid objects using stereo and motion

Work performed as a step toward an intelligent automatic machine vision system for 3-D imaging is discussed. The problem considered is the quantitative 3-D reconstruction of rigid objects. Motion and stereo are the two clues considered in this system. The system basically consists of three processes: the low level process to extract image features, the middle level process to establish the correspondence in the stereo (spatial) and motion (temporal) modalities, and the high level process to compute the 3-D coordinates of the corner points by integrating the spatial and temporal correspondences.

Nguyen, Thinh V.↗
NASA NTRS2009

Robot Vision Library

The JPL Robot Vision Library (JPLV) provides real-time robot vision algorithms for developers who are not vision specialists. The package includes algorithms for stereo ranging, visual odometry and unsurveyed camera calibration, and has unique support for very wideangle lenses

Howard, Andrew B.↗
NASA NTRS1981

Visual tracking in stereo

A method is described for visual object tracking by a computer vision system using TV cameras and special low-level image processing hardware. The tracker maintains an internal model of the location, orientation, and velocity of the object in three-dimensional space. This model is used to predict where features of the object will lie on the two-dimensional images produced by stereo TV cameras. The differences in the locations of features in the two-dimensional images as predicted by the internal model and as actually seen create an error signal in the two-dimensional representation. This is multiplied by a generalized inverse Jacobian matrix to deduce the error in the internal model. The procedure repeats to update the internal model of the object's location, orientation and velocity continuously.

Saund, E.↗
NASA NTRS2014

Robonaut 2 - IVA Experiments On-Board ISS and Development Towards EVA Capability

Robonaut 2 (R2) has completed its fixed base activities on-board the ISS and is scheduled to receive its climbing legs in early 2014. In its continuing line of firsts, the R2 torso finished up its on-orbit activities on its stanchion with the manipulation of space blanket materials and performed multiple tasks under teleoperation control by IVA astronauts. The successful completion of these two IVA experiments is a key step in Robonaut's progression towards an EVA capability. Integration with the legs and climbing inside the ISS will provide another important part of the experience that R2 will need prior to performing tasks on the outside of ISS. In support of these on-orbit activities, R2 has been traversing across handrails in simulated zero-g environments and working with EVA tools and equipment on the ground to determine manipulation strategies for an EVA Robonaut. R2 made significant advances in robotic manipulation of deformable materials in space while working with its softgoods task panel. This panel features quarter turn latches that secure a space blanket to the task panel structure. The space blanket covers two cloth cubes that are attached with Velcro to the structure. R2 was able to open and close the latches, pull back the blanket, and remove the cube underneath. R2 simulated cleaning up an EVA worksite as well, by replacing the cube and reattaching the blanket. In order to interact with the softgoods panel, R2 has both autonomously and with a human in the loop identified and localized these deformable objects. Using stereo color cameras, R2 identified characteristic elements on the softgoods panel then extracted the location and orientation of the object in its field of view using stereo disparity and kinematic transforms. R2 used both vision processing and supervisory control to successfully accomplish this important task. Teleoperation is a key capability for Robonaut's effectiveness as an EVA system. To build proficiency, crewmembers have attempted increasingly difficult tasks using R2 inside the Station. After donning motion capture equipment and a virtual reality visor, Expedition 34/35 flight engineer Tom Marshburn began operations with simple hand movements. Having gained confidence, Marshburn guided R2's arms in a leader-follower exercise with crewmate Chris Cassidy. He was also able to use the hand to grab a tumbling roll of tape, a task only demonstrable in microgravity. Later efforts saw Cassidy handle softgoods through shared control with ground operators, mimicking an activity previously achieved using only autonomy. Robotic climbing through the ISS on handrails requires both precision motion and compliant grasps in order to both position grippers on handrails/seat track and prevent large internal forces. R2 climbs using actively controlled compliance and torque limiting to meet both the precision and softness requirements. During a step, the attached leg is controlled to be strong and stiff in order to maintain precision trajectory tracking. The swing leg is controlled to be stiff but weak to minimize unintentional impact forces while maintaining precision. During a simulated dual limb grasp (as shown in Figure 1), the R2 controller maintains one limb rigid and one limb soft to prevent large internal forces from building up. R2's grippers also use a form of force control to limit grip force while not fully closed on either a handrail or seat track thus limiting unintentional forces on cables/objects that may be present in R2's translational path. The on-board torso R2 safety system relies on a single end-effector velocity limit to prevent potential impact forces from exceeding Station maximum load requirements. R2's mobile configuration required modifications to the velocity limiting safety function due to its large, dynamic inertia. R2's legs maneuver the robot's mass creating configuration dependent, joint-relative inertias. A single all-encompassing velocity limit to cover worst case inertia is prohibitively low. The upgraded R2 control and safety systems solve this problem using momentum limiting, momentum control, and kinetic energy minimization. Momentum and kinetic energy take the robot mass into account relieving low velocity restrictions on low inertia end-effectors while ensuring that the overall mass of R2 is limited from hazardous velocities. The momentum of R2's five safety nodes (each of the four end-effectors and the body) is monitored and compared to a single momentum limit. If any of the five nodes exceeds the safety limit, the motor power is removed and the robot comes to a stop. Momentum control/limiting also provides a simple, reliable method to integrate hand held tools into the safety system by providing the tool mass to the control system thus automatically reducing the allowable velocity of the end-effector with the tool. Work on the ground continues to build the skill set for an EVA Robonaut. Recent experiments (Figure 2) demonstrate how a teleoperator can use R2 to manipulate a tether hook, an important safety precaution on spacewalks. Another task displayed Robonaut's ability to pull back a protective jacket over a hose and search for damage, as well as inspect a quick-disconnect fitting for debris. Demonstrations such as these are indicative of EVA work done on ISS, specifically seen during a series of spacewalks over 2012 and 2013 where astronauts searched for an ammonia leak in one of the external cooling loops. Through experiments both on ISS and on the ground, R2 is evolving and providing the information needed to plan out the upgrades that will make an EVA Robonaut an effective tool. With the addition of legs, R2 will start climbing inside the space station and supply invaluable information on how the climbing strategies and task stabilization techniques must be refined. Ground R2 systems will continue to work with additional EVA tools and equipment in preparation for onboard IVA testing and future EVA applications.

Diftler, Myron↗
NASA NTRS1976

Implementation of a stereofluoroscopic system

Clinical applications of a 3-D video imaging technique developed by NASA for observation and control of remote manipulators are discussed. Incorporation of this technique in a stereo fluoroscopic system provides reduced radiation dosage and greater vision and mobility of the user.

Rivers, D. B.↗
NASA NTRS2012

Localization Using Visual Odometry and a Single Downward-Pointing Camera

Stereo imaging is a technique commonly employed for vision-based navigation. For such applications, two images are acquired from different vantage points and then compared using transformations to extract depth information. The technique is commonly used in robotics for obstacle avoidance or for Simultaneous Localization And Mapping, (SLAM). Yet, the process requires a number of image processing steps and therefore tends to be CPU-intensive, which limits the real-time data rate and use in power-limited applications. Evaluated here is a technique where a monocular camera is used for vision-based odometry. In this work, an optical flow technique with feature recognition is performed to generate odometry measurements. The visual odometry sensor measurements are intended to be used as control inputs or measurements in a sensor fusion algorithm using low-cost MEMS based inertial sensors to provide improved localization information. Presented here are visual odometry results which demonstrate the challenges associated with using ground-pointing cameras for visual odometry. The focus is for rover-based robotic applications for localization within GPS-denied environments.

Swank, Aaron J.↗
NASA NTRS2001

Robonaut: A Robotic Astronaut Assistant

NASA's latest anthropomorphic robot, Robonaut, has reached a milestone in its capability. This highly dexterous robot, designed to assist astronauts in space, is now performing complex tasks at the Johnson Space Center that could previously only be carried out by humans. With 43 degrees of freedom, Robonaut is the first humanoid built for space and incorporates technology advances in dexterous hands, modular manipulators, lightweight materials, and telepresence control systems. Robonaut is human size, has a three degree of freedom (DOF) articulated waist, and two, seven DOF arms, giving it an impressive work space for interacting with its environment. Its two, five fingered hands allow manipulation of a wide range of tools. A pan/tilt head with multiple stereo camera systems provides data for both teleoperators and computer vision systems.

Ambrose, Robert O.↗