SEARCH · Search NASA

Results for “computer vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Tracking Volumetric Units in Modular Factories for Automated Progress Monitoring Using Computer Vision

The construction industry is increasingly adopting off-site and prefabricated methods due to advantages offered in safety, quality, and lead time. Applying industrialized methods for plant management in offsite construction factories requires the collection of large volumes of production process data, which is a tedious task when performed manually. Recent attempts to automate this process have relied on sensor-based data collection methods which are susceptible to noise, expensive, and difficult to validate. Computer vision methods, however, enable process data collection from videos without the limitations of the other sensor-based methods. This technology has not been applied for offsite construction except in very few instances and therefore, this study proposes a novel method to reliably collect the production process data using computer vision method in near real-time from widely used surveillance cameras in offsite construction. The proposed method allows the user to annotate the workstations of interest on the video as ground truths and process these areas throughout the entire video to track the units entering and leaving stations, while continuously updating a near real-time schedule of the production line. This framework was validated by implementing on the surveillance videos of the production process of modular home manufacturing in a factory. The results consistently provided 100% accuracy, after denoising, for all the videos processed including 60 h of work for a station. The developed method enables real-time tracking of station performance, which can enable continuous improvement methods for factory management and resource allocation.

computer vision↗

NASA NTRS1984

JPL Robotics Laboratory computer vision software library

The past ten years of research on computer vision have matured into a powerful real time system comprised of standardized commercial hardware, computers, and pipeline processing laboratory prototypes, supported by anextensive set of image processing algorithms. The software system was constructed to be transportable via the choice of a popular high level language (PASCAL) and a widely used computer (VAX-11/750), it comprises a whole realm of low level and high level processing software that has proven to be versatile for applications ranging from factory automation to space satellite tracking and grappling.

Cunningham, R.↗

DOE OSTI2021

Automatic identification and quantification of dense microcracks in high-performance fiber-reinforced cementitious composites through deep learning-based computer vision

Highlights: • A method is presented to detect, locate, quantify, and visualize dense microcracks in HPFRCC. • Quantification of dense microcracks is realized using deep learning method for the first time. • The presented method uses deep learning models that are trained using a realistic dataset size. • The presented method has a high computation efficiency for identifying and quantifying cracks. • The presented method provides crack width with errors up to 50 μm and a R{sup 2} value of 0.984. High-performance fiber-reinforced cementitious composites (HPFRCCs) feature high mechanical strengths, crack resistance, and durability. Under excessive loading, HPFRCCs demonstrate dense microcracks that are difficult to identify using existing methods. This study presents a computer vision method for identification, quantification, and visualization of microcracks in HPFRCCs based on deep learning. The presented method integrates multiple deep learning models and computer vision techniques in a hierarchical architecture. The crack pattern (e.g., number, width, and spacing of cracks) are automatically determined from pictures without human intervention. This study shows that the presented method achieves an accuracy of 0.992 for crack detection and an accuracy finer than 50 μm (R{sup 2} > 0.984) for quantification of crack width when deep learning models are trained using only 200 pictures of HPFRCCs and 200 pictures of conventional concrete with incorporation of data augmentation. The presented method is expected to be also applicable to other materials featuring complex cracks.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Considerations regarding the Use of Computer Vision Machine Learning in Safety-Related or Risk-Significant Applications in Nuclear Power Plants

With the advancements made to date in the field of artificial intelligence (AI), significant potential exists to utilize AI capabilities for nuclear power plant (NPP) applications. AI can replicate human decision making and it is usually faster and more accurate than humans. For implementations that impact critical NPP applications (e.g., safety-related or non-safety systems that potentially affect overall plant risk), a deeper safety analysis of the AI methods is necessary. AI applied to NPP operations could resemble the use of digital I&C (DI&C) because such applications involve digital computer hardware and custom-designed software that input plant data, execute complex software algorithms, and output the results to a system or licensed human operator to potentially provoke an action. For AI methods to be compliant with current safety requirements for DI&C, AI compatibility must be evaluated, and AI-related gaps may exist that prevent the prompt deployment of AI in NPPs. This effort aims to evaluate how example AI technologies align with the DI&C safety framework, and discusses how they could be analyzed, modeled, tested, and validated in a manner similar to typical DI&C technologies. Because AI is a broad field that encompasses areas such as machine learning (ML), natural language processing, and computer vision, this research focused on a subset of methods categorized as the computer vision ML (CVML) methods. This report explores two CVML use cases, gauge reading and fire watch, considered relevant to the DI&C standards, as they could play a safety-critical role. For the gauge reading use case, a CVML-enabled technology that can read gauges at oblique angles is utilized. For the fire watch use case, a CVML-enabled technology is utilized that migrates fire watch from a manual (human) approach to automated fire detection. These use cases are mainly intended to give context to the CVML system discussion. This effort assumes the worst-case scenario, with the CVML system being used to replace a safety-related or risk-significant system, thus requiring evaluation. Evaluating CVML against most of the relevant safety requirements for DI&C yielded several CVML-specific considerations due to the uniqueness of its characteristics in comparison with typical DI&C systems. For example, CVML models often employ commonly used (open-source) datasets, and it is not always possible to determine the level of overlap among open-source datasets. Therefore, the independence of the developed CVML models when demonstrating diversity is questionable, therefore creating vulnerability to common cause failure (CCF). The design verification process is also impacted since the data overlap could result in overestimation of the software validation and verification (V&V) performance results. Section 2 of this report evaluates a list of the identified CVML-specific characteristics and discusses the resulting considerations and potential solutions in the context of each referenced requirement. A summation is provided in Section 3. This report is not to be used as a guideline. It was developed to identify and consider issues in the implementation of ML technologies used to augment activities that may have a bearing on plant operation. The report draws parallels to the use of DI&C technologies, for which many standards are available to guide their use in nuclear plant operation. It considers the technologies and some of the potential implications of their use in safety-related applications but is not intended to address regulatory or licensing related issues.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

NASA NTRSDate not supplied

Unreal Engine Testbed for Computer Vision of Tall Lunar Tower Assembly

The Tall Lunar Tower project at the NASA Langley Research Center is focused on the design, modeling, fabrication, and testing of a supervised autonomously assembly engineering development unit for tall lunar towers. The lunar south pole environment poses many challenges for robotic assembly of the tall tower, particularly to computer vision camera systems due to a high-contrast lighting environment. This paper will present an Unreal Engine 5 video game engine Lunar South Pole Lighting Testbed to simulate realistic lunar lighting conditions for synthetic image generation. The fidelity of the simulation environment is investigated by comparing the accuracy of computer vision models trained using synthetic image data and trained from real image data collected in a lunar analog environment.

Unreal Engine↗

NASA NTRSDate not supplied

Unreal Engine Testbed for Computer Vision of Tall Lunar Tower Assembly

The Tall Lunar Tower project at the NASA Langley Research Center is focused on the design, modeling, fabrication, and testing of a supervised autonomously assembled engineering development unit for tall lunar towers. The lunar south pole environment poses many challenges for robotic assembly of the tall tower, particularly to computer vision camera systems due to a high-contrast lighting environment. This paper will present an Unreal Engine 5 video game engine Lunar South Pole Lighting Testbed to simulate realistic lunar lighting conditions for synthetic image generation. The fidelity of the simulation environment is investigated by comparing the accuracy of computer vision models trained using synthetic image data and trained from real image data collected in a lunar analog environment.

Unreal Engine↗

NASA NTRS2023

Unreal Engine Testbed for Computer Vision of Tall Lunar Tower Assembly

Unreal Engine↗

DOE OSTI2025

Integrating Machine-learning-assisted Computer Vision with RICH System

Developments in artificial intelligence have vastly expanded the capabilities of robots. Currently, the Spallation Neutron Source (SNS) beamlines at Oak Ridge National Lab (ORNL) have robotic sample loaders to increase the efficiency of running experiments. However, they require retraining if anything about the situation changes, e.g., where the samples are, and cannot notice if errors occur. So, the viability of using computer vision and machine learning to enhance these sample loaders’ functionality was investigated. In this project, the RICH system with a Dobot CR3 6-axis robot present at the VULCAN beamline assisted by an Intel Realsense D435i camera, a unique camera that enables convenient translation of 2D pixel coordinates to 3D world points, was programmed to load ceramic crucibles into a thermogravimetric analyzer (TGA) furnace. An algorithm was constructed in Python with three major phases planned: (1) obtaining a sample, (2) moving it to the target location, and then (3) bringing the sample back to its original location once the experiment finished. In the first phase, the algorithm would dynamically detect sample locations using ArUco markers to recognize the samples’ general location and a custom-trained yolov5 object detection model to locate the crucibles’ centers. Afterward, the robot would be directed to pick up samples based on the crucibles’ calculated positions. In the second phase, the robot would move the sample to a secondary point, reorient its grip, and place the sample at the target location. In the final phase, the robot would determine whether the sample was intact and would bring it back to its original place if it was or raise an alarm. Using this algorithm, the robot was able to pick up different types of crucibles at varying positions. These results indicate that integrating machine-learning-assisted computer vision with robotic sample loaders can result in effective autonomous detection of samples.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Uncertainty quantification of fireball features extracted from nuclear test films using computer vision

Films from the US’s historic nuclear testing era comprise the only extensive collection of imagery depicting high-yield detonations. These films offer unique insights into the characteristics of flows occurring on scales that are difficult to replicate experimentally, and they are a valuable source of data for the validation of models used to describe nuclear detonations. In recent work, we implemented modern computer vision and machine learning techniques to extract features of the fireball following nuclear detonation. With a training dataset of fireball films, we fine-tuned a You Only Look Once 11 (YOLO11) model to detect and track the fireball. Applied to a video, the outer bounding box produced in each frame by YOLO11 is used as an input prompt to Meta’s Segment Anything Model 2 (SAM2), which is shown to accurately predict the boundary of the fireball over time with high resolution. These state-of-the-art computer vision foundation models exhibit impressive visual accuracy in their results but lack an output of values that robustly quantify uncertainty in scientific applications. In this paper, we develop procedures for uncertainty quantification of extracted fireball features. We outline the application of a parallel attention mechanism to calculate uncertainty ranges that complement and better pose model validation data. This higher quality fireball validation data may serve to improve prognostic models describing nuclear detonations in support of nuclear forensic and emergency response activities.

Khristy, Joel [ORNL] (ORCID:0000000209963060)↗

NASA NTRS1992

Computer vision in microstructural analysis

The following is a laboratory experiment designed to be performed by advanced-high school and beginning-college students. It is hoped that this experiment will create an interest in and further understanding of materials science. The objective of this experiment is to demonstrate that the microstructure of engineered materials is affected by the processing conditions in manufacture, and that it is possible to characterize the microstructure using image analysis with a computer. The principle of computer vision will first be introduced followed by the description of the system developed at Texas A&M University. This in turn will be followed by the description of the experiment to obtain differences in microstructure and the characterization of the microstructure using computer vision.

Srinivasan, Malur N.↗

DOE OSTI2022

Computer-Vision-Based Vibration Tracking Using a Digital Camera: A Sparse-Optical-Flow-Based Target Tracking Method

Computer-vision-based target tracking is a technology applied to a wide range of research areas, including structural vibration monitoring. However, current target tracking methods suffer from noise in digital image processing. In this paper, a new target tracking method based on the sparse optical flow technique is introduced for improving the accuracy in tracking the target, especially when the target has a large displacement. The proposed method utilizes the Oriented FAST and Rotated BRIEF (ORB) technique which is based on FAST (Features from Accelerated Segment Test), a feature detector, and BRIEF (Binary Robust Independent Elementary Features), a binary descriptor. ORB maintains a variety of keypoints and combines the multi-level strategy with an optical flow algorithm to search the keypoints with a large motion vector for tracking. Then, an outlier removal method based on Hamming distance and interquartile range (IQR) score is introduced to minimize the error. The proposed target tracking method is verified through a lab experiment—a three-story shear building structure subjected to various harmonic excitations. It is compared with existing sparse-optical-flow-based target tracking methods and target tracking methods based on three other types of techniques, i.e., feature matching, dense optical flow, and template matching. The results show that the performance of target tracking is greatly improved through the use of a multi-level strategy and the proposed outlier removal method. The proposed sparse-optical-flow-based target tracking method achieves the best accuracy compared to other existing target tracking methods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2021

Individual Wave Detection and Tracking within a Rotating Detonation Engine through Computer Vision Object Detection applied to High-Speed Images

Known for their simplistic design and continuous detonation, rotating detonation engines (RDEs) constitute a majority of current pressure gain combustion (PGC) research efforts. Experimental RDE operation times have been continuously extended through the use of rig cooling techniques. As the window of observable behavior is expanded, and as the technology matures toward eventual integration within gas turbines, monitoring techniques must evolve to better match industrial diagnostics. High-speed image analysis techniques prove useful to capture and evaluate the unsteady detonation behavior within the RDE. Traditional image analysis techniques, however, require extensive processing times which prohibit simultaneous monitoring. To better address this problem, a computer vision object detection methodology is proposed to quickly detect individual detonation waves within a single down-axis image. Detonation waves are detected in individual images by the implemented computer vision method You Only Look Once (YOLO) object detection network. In order to detect detonation waves, the network must first be trained using RDE images of interest, for which each required phase of network development is outlined. Detection of waves is improved through proper treatment of the collected image set, variation of Intersection over Union (IoU) and confidence thresholding, and through a parametric study of annotation dimensions. Each detected wave is described by its location and rotational direction, and locations are tracked to calculate wave velocity across each frame, leading to a timestep resolution of 20 µs. Wave velocities are also calculated through a series of frames, leading to a suitable average velocity estimation using as few as 10 frames. Uncertainty analysis accounting for variation in camera framerate, pixel width and annotation centroid locations estimates a total uncertainty of ±4.3% for velocity calculations, using the smallest annotation boxes. This method offers great reductions in processing times, as a step toward real-time monitoring of detonation waves within an RDE. Improving on previous studies, this technique is impartial to wave modes not included in the original training set and calculates wave velocities independent of high-speed pressure data. The ability to isolate waves within predicted bounding boxes will likely facilitate analysis of pixel intensity variation as an estimation of wave strength in future work.

Johnson, Kristyn↗

DOE OSTI2024

Hydra: Computer Vision for Online Data Quality Monitoring

Hydra is a system utilizing computer vision for near real-time data quality monitoring. Currently operational across all of Jefferson Lab’s experimental halls, it reduces the workload of shift takers by autonomously monitoring diagnostic plots during experiments. Hydra uses "off-the-shelf" supervised learning technologies and is supported by a comprehensive MySQL database. To simplify access, web apps have been developed to facilitate both labeling and monitoring of Hydra’s inferences. Hydra can connect with the alarm system and incorporates complete historical tracking, enabling it to identify issues that shift takers could miss. When issues are detected, a natural first question is: "Why does Hydra think there is a problem?" To answer, Hydra employs Gradient-weighted Class Activation Maps (GradCAM) to identify regions of the image that are important for the specific classification. This interpretive layer enhances transparency and trustworthiness, which is essential for integration with experiment workflows and operation. The Hydra system, results, and sociological considerations for deployment will be discussed.

Jeske, Torri↗

NASA NTRSDate not supplied

Computer Vision Dataset for Aircraft Taxi Operations

The development and democratization of computer vision algorithms are contingent on the availability of high-quality datasets. In this paper, we introduce a database of forward-facing videos from taxiing aircraft as well as the time-correlated flight data at approximately 1 to10 Hz containing aircraft state data and environmental conditions. The video data is sourced from the National Aeronautics and Space Administration Airborne Science Program archive and includes over 33 hours of 4k, 1080p, and 720p video from twenty-two airports around the world. This paper describes the method of the database construction and a brief analysis of its contents.

Ryan Horn↗

NASA NTRS2016

Utilizing Commercial Hardware and Open Source Computer Vision Software to Perform Motion Capture for Reduced Gravity Flight

Long duration space travel to Mars or to an asteroid will expose astronauts to extended periods of reduced gravity. Since gravity is not present to aid loading, astronauts will use resistive and aerobic exercise regimes for the duration of the space flight to minimize the loss of bone density, muscle mass and aerobic capacity that occurs during exposure to a reduced gravity environment. Unlike the International Space Station (ISS), the area available for an exercise device in the next generation of spacecraft is limited. Therefore, compact resistance exercise device prototypes are being developed. The NASA Digital Astronaut Project (DAP) is supporting the Advanced Exercise Concepts (AEC) Project, Exercise Physiology and Countermeasures (ExPC) project and the National Space Biomedical Research Institute (NSBRI) funded researchers by developing computational models of exercising with these new advanced exercise device concepts. To perform validation of these models and to support the Advanced Exercise Concepts Project, several candidate devices have been flown onboard NASAs Reduced Gravity Aircraft. In terrestrial laboratories, researchers typically have available to them motion capture systems for the measurement of subject kinematics. Onboard the parabolic flight aircraft it is not practical to utilize the traditional motion capture systems due to the large working volume they require and their relatively high replacement cost if damaged. To support measuring kinematics on board parabolic aircraft, a motion capture system is being developed utilizing open source computer vision code with commercial off the shelf (COTS) video camera hardware. While the systems accuracy is lower than lab setups, it provides a means to produce quantitative comparison motion capture kinematic data. Additionally, data such as required exercise volume for small spaces such as the Orion capsule can be determined. METHODS: OpenCV is an open source computer vision library that provides the ability to perform multi-camera 3 dimensional reconstruction. Utilizing OpenCV, via the Python programming language, a set of tools has been developed to perform motion capture in confined spaces using commercial cameras. Four Sony Video Cameras were intrinsically calibrated prior to flight. Intrinsic calibration provides a set of camera specific parameters to remove geometric distortion of the lens and sensor (specific to each individual camera). A set of high contrast markers were placed on the exercising subject (safety also necessitated that they be soft in case they become detached during parabolic flight); small yarn balls were used. Extrinsic calibration, the determination of camera location and orientation parameters, is performed using fixed landmark markers shared by the camera scenes. Additionally a wand calibration, the sweeping of the camera scenes simultaneously, was also performed. Techniques have been developed to perform intrinsic calibration, extrinsic calibration, isolation of the markers in the scene, calculation of marker 2D centroids, and 3D reconstruction from multiple cameras. These methods have been tested in the laboratory side-by-side comparison to a traditional motion capture system and also on a parabolic flight.

Biodynamics↗

NASA NTRS1981

Computational vision

The range of fundamental computational principles underlying human vision that equally apply to artificial and natural systems is surveyed. There emerges from research a view of the structuring of vision systems as a sequence of levels of representation, with the initial levels being primarily iconic (edges, regions, gradients) and the highest symbolic (surfaces, objects, scenes). Intermediate levels are constrained by information made available by preceding levels and information required by subsequent levels. In particular, it appears that physical and three-dimensional surface characteristics provide a critical transition from iconic to symbolic representations. A plausible vision system design incorporating these principles is outlined, and its key computational processes are elaborated.

Barrow, H. G.↗

DOE OSTI2026

Improvements in Mirror Surface Measurement with Reflected Computer Vision Targets

Over the last several years, NREL has been developing a system to measure large optical surfaces of heliostat mirrors by reflecting computer vision targets. An advantage of this system, called Reflected Target Nonintrusive Assessment (ReTNA), is that it lends itself well to stitching together many images, each reflecting only part of a larger heliostat. In the last few months, this was taken to a new extreme, with a small target (<5m2) being used to measure a >25m2 long focal length heliostat. These measurements were compared with traditional fringe deflectometry methods, which require a >50m2 target, and photogrammetry. The strengths, weaknesses and limitations of ReTNA are discussed. An estimated uncertainty in this new measurement is presented, along with software improvements and a new wireless data collection system. A bill of materials for this measurement system is presented, which has been designed to use all low-cost, off-the-shelf components. Finally, the next steps for future ReTNA development are presented. Overall, ReTNA can be a valuable optics measurement system, complimentary to existing measurement techniques available for large reflective surfaces.

14 SOLAR ENERGY↗

DOE OSTI2025

Automated Waterbox Inspection for Nuclear Power Plants Using Computer Vision - Based Change Detection

Nuclear power plant waterboxes require regular inspection for leaks, missing components, and structural damage during maintenance outages. Traditional manual inspection is time-consuming and poses safety risks from confined space entry. We developed an automated computer vision system for drone-based waterbox inspection in partnership with Florida Light and Power. Our approach uses feature detection and matching to identify critical changes between baseline and current inspection images, automatically flagging additions (leaks/debris), removals (missing plugs), and translations (displaced components) while compensating for drone movement and environmental variations. We systematically evaluated six feature matching methods, from classical approaches (SIFT+BF) to state-of-the-art neural networks (SuperPoint+SuperGlue), using both standard benchmarks (HPatches) and waterbox-specific validation with real-world augmentations. SuperPoint+SuperGlue achieved superior performance with 7.82 pixels RMSE and 100% success rate—2.8x better accuracy than our baseline. While the pre-trained model has commercial licensing restrictions for nuclear deployment, our findings validate this architecture for custom training. We implemented a real-time GUI demonstrating the SIFT+BF approach for immediate deployment, processing drone feeds at 30 FPS with color-coded change visualization. Future work includes training a custom SuperPoint+SuperGlue model on waterbox data and integrating Vision-Language Models for automated reporting and maintenance guidance.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗