SEARCH · Search NASA
Results for “stereo vision”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Stereo-vision thermal imaging system for tracking flying animals in wind farm areas (CRADA #763) Abstract
CRADA 763: abstract ThermalTracker-3D (TT3D) is a stereo vision thermal imaging system that provides 3D flight information on detected birds, bats, and other flying targets. The system was initially developed for use in the siting and monitoring of offshore wind projects to establish pre-construction and operation collision risk data but can be applied to terrestrial wind energy projects as well as national security monitoring. This technology will reduce monitoring cost, decrease processing time, and provide more accurate data for wind energy developers/operators and regulatory agencies. While the current technology is at a high level of readiness, Technology Readiness Level (TRL) 7, there remain several barriers to commercialization, particularly around ease-of-use, that result in a low Adoption Readiness Level (ARL). The proposed work will advance commercialization readiness by streamlining calibration methods for built systems. This work will:1. 1. develop a software package for factory and dynamic calibration processes 2. test that package with existing prototype TT3D systems, and 3. conduct outreach with industry end-users.
ThermalTracker 3D
ThermalTracker-3D is a stereo-vision solution for evaluating flight tracks of birds and bats around offshore wind turbines. Using a pair of thermal video cameras, the technology remotely senses movement of animals and objects, day and night, near critical assets. It generates motion tracks by collapsing a sequence of video frames from each camera into a single image that contains an entire flight track and then applies stereo-vision processing to transform the flight track into three dimensions. The approach allows tracking in near real time and automatically identifies moving objects based on features from the motion track and object size.
Three-Dimensional Tracking of High-Speed Tumbling Projectiles Using Stroboscopic Imaging
Creating images of high-speed projectiles has been a topic of interest for almost a century. Historically, ballistics ranges have used air-gap flash photography or high-speed video cameras to capture this type of data. Air-gap flash photography provides a single image at each camera station. Using modern high-speed imagers provides accurate data but is cost prohibitive for a long-distance range. This paper presents a camera system capable of capturing the three-dimensional data of high speed projectiles over a long distance. The system uses relatively low-cost cameras which are set up in a stereo vision configuration and uses high-speed strobe lights to create multi exposure images. Each pulse of light captures the position of the projectile as it passes the camera. For each position captured in the image, the three-dimensional position of the projectile is found using triangulation geometry. The linear velocity of the projectile is calculated by combining the position of the projectile with timing data. Two test series were conducted. The first test series compares different cameras and backdrops for the camera system. The second test series captured position data for two different shapes of high-speed tumbling projectiles.
Autonomous thermal tracking reveals spatiotemporal patterns of seabird activity relevant to interactions with floating offshore wind facilities
Planning is underway for placement of infrastructure needed to begin offshore wind (OSW) energy generation along the West Coast of the United States and elsewhere in the Pacific Ocean. In contrast to the primarily nearshore windfarms currently in the North Atlantic, the seabird communities inhabiting Pacific Wind Energy Areas (WEAs) include significant populations of species that fly by dynamic soaring, a behavior dependent on wind and in which flight height increases steeply with wind speed. Therefore, a more precise and detailed assessment of their 3D airspace use is needed to better understand the potential collision risks that OSW turbines may present to these seabirds. Toward this end, a novel technology called the ThermalTracker-3D (TT3D), which uses thermal imaging and stereo vision, was developed to render high-resolution (on average within ±5 m) flight tracks and related behavior of seabirds. The technology was developed and deployed on a wind-profiling LiDAR buoy in the Humboldt WEA, located 34 to 57 km off California’s coast. During the at-sea deployment between 24 May and 13 August 2021, the TT3D successfully tracked birds moving between 10 and 500 m from the device, around the clock, and in all weather conditions; a total of 1407 detections and their corresponding 3D flight trajectories were recorded. Mean altitudes of detections ranged 6-295 m above sea level (asl). Considering the degree of overlap with anticipated rotor swept zones (RSZ), which extend 25-260 m asl, 79% of detected birds (per m 3 of airspace) moved below the RSZ, 21% moved at heights overlapping the RSZ, and another 0.04% occurred at heights exceeding the RSZ. The high-resolution tracks provided valuable insight into seabird space use, especially at heights that make them vulnerable to collision during various environmental conditions (e.g., darkness, strong winds). Observations made by the TT3D will be useful in filling critical knowledge gaps related to estimating collision and avoidance between seabirds and OSW facilities in the Pacific and elsewhere. Future research will focus on enhancing the TT3D’s identification capabilities to the lowest taxon through validation studies and artificial intelligence, further contributing to seabird conservation efforts associated with OSW.
Autonomous 3D and Radiation Mapping in Tunnel Environments Using Aerial Robots - 18156
In this paper we present the algorithms and systems developed to enable autonomous 3D and radiation mapping, as well as radiological source localization in tunnel environments using aerial robots. The developed technologies aim to address the need of autonomous robotic inspection of nuclear sites such as facilities in the process of being decommissioned. As a motivational example, the PUREX tunnels at the Hanford Site are considered and the robot is designed to be able to conduct multi-modal 3D/radiation characterization of the tunnel environment without any prior knowledge or map provided beforehand. In order to enable the necessary GPS-denied simultaneous localization and mapping in dark, visually-degraded tunnel environments, a multi-modal mapping unit was developed which fuses the data from stereo vision cameras synchronized with flashing LEDs, inertial sensors, and three time-of-flight 3D depth sensors. The robot's pose and surrounding map are estimated at 20 Hz through a depth-enhanced visual- inertial localization and mapping framework. Provided this ability to estimate the robot's pose and develop a dense map in real-time, a receding horizon sampling-based autonomous exploration path planning algorithm was utilized and equips the robot with the ability to explore its environment given no prior map. The aforementioned key robotic functionalities are further extended through the integration of miniaturized fully solid-state scintillation detectors, utilizing advanced scintillators and Silicon Photomultipliers. The derived sensor data is employed for radiation mapping, which when fused with the 3D maps and the spectroscopic source localization methods, allow the aerial robotic platform to localize the radiation source and identify the specific source isotope. The source location is estimated through a geometric algorithm which can localize a source given only three measurements and short dwell times. The developed systems were tested using two small aerial robot platforms of the F450 and F550 frame classes with weights of 2.6 kg and 2.9 kg respectively. All localization and planning as well as high-level control algorithms are executed onboard. The scintillation detector is integrated on the robot and its data is collected through a dedicated microcontroller. A set of field experiments were conducted inside the dark and GPS-denied environment of a 4.25 m wide, 7.5 m tall, and 137 m long railroad tunnel. A Cs-137 source was placed within the tunnel, the associated 3D/radiation maps were derived, and the source location was estimated. Through these efforts, the goal is to develop the robotic technology that would enable the autonomous systematic inspection or emergency response in nuclear waste decommissioning facilities. (authors)
Remote Control of Sensor-less Manipulator for Decommissioning Work - 18560
A remote control system using external sensors is proposed for a sensor-less manipulator. The manipulator poses are estimated from camera images obtained by a stereo vision system. Based on the estimated poses, the sensor-less manipulator is controlled in real-time. This paper describes details of the proposed manipulator pose estimation system, the remote control system and results of a fundamental experiment using the sensor-less manipulator. (authors)
An AI-Based 3D Bat Movement Tracking System at Wind Energy Facilities Using Multi-Thermal Video Cameras
The talk at the NAWEA Wind Tech 2024 conference discusses how to leverage the potential of real-time thermal-imaging methodologies in quantifying nocturnal bat activities at wind turbines, using 3D computer vision techniques within a deep learning framework. This innovation enables the automatic detection and classification of bats, birds, and insects in thermal-imaging videos captured at wind turbine sites, facilitating efficient and accurate data analysis for enhanced understanding and mitigation of bat-wind turbine interactions.
An AI-Based 3D Bat Movement Tracking System at Wind Energy Facilities Using Multi-Thermal Video Cameras
The poster at the 15th Wind Wildlife Research Meeting discusses how to leverage the potential of real-time thermal-imaging methodologies in quantifying nocturnal bat activities at wind turbines, using 3D computer vision techniques within a deep learning framework. This innovation enables the automatic detection and classification of bats, birds, and insects in thermal-imaging videos captured at wind turbine sites, facilitating efficient and accurate data analysis for enhanced understanding and mitigation of bat-wind turbine interactions.
Development of an Energy Efficient and Cost Effective Autonomous Vehicle Research Platform
Commercialization of autonomous vehicle technology is a major goal of the automotive industry, thus research in this space is rapidly expanding across the world. However, despite this high level of research activity, literature detailing a straightforward and cost-effective approach to the development of an AV research platform is sparse. To address this need, we present the methodology and results regarding the AV instrumentation and controls of a 2019 Kia Niro which was developed for a local AV pilot program. This platform includes a drive-by-wire actuation kit, Aptiv electronically scanning radar, stereo camera, MobilEye computer vision system, LiDAR, inertial measurement unit, two global positioning system receivers to provide heading information, and an in-vehicle computer for driving environment perception and path planning. Robotic Operating System software is used as the system middleware between the instruments and the autonomous application algorithms. After selection, installation, and integration of these components, our results show successful utilization of all sensors, drive-by-wire functionality, a total additional power* consumption of 242.8 Watts (*Typical), and an overall cost of $118,189 USD, which is a significant saving compared to other commercially available systems with similar functionality. This vehicle continues to serve as our primary AV research and development platform.
Real-Time Omnidirectional Stereo Rendering: Generating 360° Surround-View Panoramic Images for Comfortable Immersive Viewing
Surround-view panoramic images and videos have become a popular form of media for interactive viewing on mobile devices and virtual reality headsets. Viewing such media provides a sense of immersion by allowing users to control their view direction and experience an entire environment. When using a virtual reality headset, the level of immersion can be improved by leveraging stereoscopic capabilities. Stereoscopic images are generated in pairs, one for the left eye and one for the right eye, and result in providing an important depth cue for the human visual system. For computer generated imagery, rendering proper stereo pairs is well known for a fixed view. However, it is much more difficult to create omnidirectional stereo pairs for a surround-view projection that work well when looking in any direction. One major drawback of traditional omnidirectional stereo images is that they suffer from binocular misalignment in the peripheral vision as a user's view direction approaches the zenith / nadir (north / south pole) of the projection sphere. This paper presents a real-time geometry-based approach for omnidirectional stereo rendering that fits into the standard rendering pipeline. Our approach includes tunable parameters that enable pole merging - a reduction in the stereo effect near the poles that can minimize binocular misalignment. Results from a user study indicate that pole merging reduces visual fatigue and discomfort associated with binocular misalignment without inhibiting depth perception.
Reconstruction and analysis of exploding wire particle trajectories via automatic calibration of stereo images
Quantitative understanding of the physics of dust or granular matter transport significantly impacts several aspects of burning plasma science and technology. Here, this work takes machine vision techniques popular in robotics and self-driving cars and applies them to identification and analysis of microparticles generated from exploding wires. Using only the image frames and knowledge of the intrinsic properties of the cameras, a Python code was written to identify the particles, automatically calibrate the relative image positions, and extract trajectory data. After identifying approximately 50 particles based on the timing of secondary particle explosions, the eight point and random sample consensus algorithms were used to determine the geometric correlation between the cameras. Over 100 particle matches were found between the two camera views. These correlated trajectories were used in subsequent 3D track reconstruction and analysis of the physics behind the particle motion. The 3D reconstruction resulted in accurate positioning of the particles with respect to the experimental setup. The particle motion was consistent with the effects of a 1 g gravitational field modified by drag forces. Lastly, the methods and analyses presented here can be used in many facets of high temperature plasma diagnostics.
Demonstration of a viable quantitative theory for interplanetary type II radio bursts
Between 29 November and 1 December 2013 the two widely separated spacecraft STEREO A and B observed a long lasting, intermittent, type II radio burst for the extended frequency range ≈ 4 MHz to 30 kHz, including an intensification when the shock wave of the associated coronal mass ejection (CME) reached STEREO A. We demonstrate for the first time our ability to quantitatively and accurately simulate the fundamental (F) and harmonic (H) emission of type II bursts from the higher corona (near 11 solar radii) to 1 AU. Our modeling requires the combination of data-driven three-dimensional magnetohydrodynamic simulations for the CME and plasma background, carried out with the BATS-R-US code, with an analytic quantitative kinetic model for both F and H radio emission, including the electron reflection at the shock, growth of Langmuir waves and radio waves, and the radiations propagation to an arbitrary observer. The intensities and frequencies of the observed radio emissions vary hugely by factors ≈ 10{sup 6} and ≈ 10{sup 3}, respectively; the theoretical predictions are impressively accurate, being typically in error by less than a factor of 10 and 20 %, for both STEREO A and B. We also obtain accurate predictions for the timing and characteristics of the shock and local radio onsets at STEREO A, the lack of such onsets at STEREO B, and the z-component of the magnetic field at STEREO A ahead of the shock, and in the sheath. Very strong support is provided by these multiple agreements for the theory, the efficacy of the BATS-R-US code, and the vision of using type IIs and associated data-theory iterations to predict whether a CME will impact Earth’s magnetosphere and drive space weather events.
Pseudo-automatic Determination of Coronal Mass Ejections’ Kinematics in 3D
Coronal mass ejection (CME) events are among the main drivers of geomagnetic disturbances, and hence play a central role in the Sun–Earth system. Their monitoring and, in particular, the determination of their speed and direction of propagation are key issues for the forecasting of space weather near to Earth. We have implemented a method to track CME events in three dimensions by combining triangulation and tie-pointing analysis with a supervised computer vision algorithm. This novel approach does not rely on any geometric constraint, and eliminates the need for visual identification of the CME boundaries. We applied our method to 17 CME events observed simultaneously by the twin Solar Terrestrial Relations Observatory ( STEREO ) COR2 coronagraph imagers from 2008 December to 2011 November in order to obtain their 3D kinematical characterization (i.e., the velocity vector) along with their morphological properties. About ten of these events have already been analyzed using other methodologies. In these cases, we carried out a thorough comparison with our results and found that, in spite of the different nature and spatial coverage range of the other methods with respect to CORSET3D, the majority of the results agree. We found, however, that three events exhibited discrepancies in the magnitude of the velocity vector, four in the longitudinal direction of propagation, and in only one case was there a discrepancy in latitude. The discrepancies appeared in those cases where quasi-simultaneous, quasi-co-located events were observed in the coronagraphs’ fields of view.
A new method for increasing accuracy of distance measurement based on single visual camera
Camera has been widely used in computer vision and mobile robotics applications because the sensor provides complementary information. However, the chief drawback of camera is that it only can provide 2D image information without depth information. There are lots of people studying the distance measurement with single visual camera, however it is difficult to achieve high precision. Usually we use two cameras to form stereo image and measure distance. Obviously it is complex. In this paper, we study the reason why distance measurement with single camera can hardly achieve high precision and propose a new method for improving the accuracy. We evaluate the proposed method and compare with existing methods to demonstrate the efficiency and high precision.
Thermographic phosphor digital image correlation
The invention advances the ratio method of phosphor thermography using two machine vision cameras for full-field temperature measurements of a solid surface. As an example, algorithms from digital image correlation (DIC) can be used to determine the stereoscopic imaging system intrinsic and extrinsic parameters, and accurately register material points on the sample to subpixel locations in each image with 0.07 px or better accuracy. A phosphor calibration sample fabricated using aerosol deposition can be used for in situ determination of the temperature-versus-intensity ratio relationship. The full calibration methodology and several improvements on two-color phosphor thermography open the door for full-field temperature measurements in dynamic tests with deforming test specimens. In particular, thermographic phosphors can be combined with stereo digital image correlation in a novel diagnostic, TP+DIC, to measure time-resolved full-field surface strains and temperatures simultaneously, thereby enabling testing in combined thermo-mechanical environments.
Motion correction for passive radiation imaging of small vessels in ship-to-ship inspections
Passive radiation detection remains one of the most acceptable means of ascertaining the presence of illicit nuclear materials. In maritime applications it is most effective against small to moderately sized vessels, where attenuation in the target vessel is of less concern. Unfortunately, imaging methods that can remove source confusion, localize a source, and avoid other systematic detection issues cannot be easily applied in ship-to-ship inspections because relative motion of the vessels blurs the results over many pixels, significantly reducing system sensitivity. This is particularly true for the smaller watercraft, where passive inspections are most valuable. We have developed a combined gamma-ray, stereo visible-light imaging system that addresses this problem. Data from the stereo imager are used to track the relative location and orientation of the target vessel in the field of view of a coded-aperture gamma-ray imager. Using this information, short-exposure gamma-ray images are projected onto the target vessel using simple tomographic back-projection techniques, revealing the location of any sources within the target. Here,the complex autonomous tracking and image reconstruction system runs in real time on a 48-core workstation that deploys with the system.
Key Residues for Catalytic Function and Metal Coordination in a Carotenoid Cleavage Dioxygenase
Carotenoid cleavage dioxygenases (CCDs) are non-heme iron-containing enzymes found in all domains of life that generate biologically important apocarotenoids. Prior studies have revealed a critical role for a conserved 4-His motif in forming the CCD iron center. By contrast, the roles of other active site residues in catalytic function, including maintenance of the stringent regio- and stereo-selective cleavage activity, typically exhibited by these enzymes have not been thoroughly investigated. Here, we examined the functional and structural importance of active site residues in an apocarotenoid-cleaving oxygenase (ACO) from Synechocystis. Most active site substitutions variably lowered maximal catalytic activity without markedly affecting the K m value for the all-trans-8'-apocarotenol substrate. Native C15-C15' cleavage activity was retained in all ACO variants examined suggesting that multiple active site residues contribute to the enzyme's regioselectivity. Crystallographic analysis of a nearly inactive W149A-substituted ACO revealed marked disruption of the active site structure, including loss of iron coordination by His-238 apparently from an altered conformation of the conserved second sphere Glu-150 residue. Gln- and Asp-150-substituted versions of ACO further confirmed the structural/functional requirement for a Glu side chain at this position, which is homologous to Glu-148 in RPE65, a site in which substitution to Asp has been associated with loss of enzymatic function in Leber congenital amaurosis. The novel links shown here between ACO active site structure and catalytic activity could be broadly applicable to other CCD members and provide insights into the molecular pathogenesis of vision loss associated with an RPE65 point mutation.