Search NASA⌕ Search

SEARCH · Search NASA

Results for “Computer Vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2014

Using computer-extracted image features for modeling of error-making patterns in detection of mammographic masses among radiology residents

Purpose: Mammography is the most widely accepted and utilized screening modality for early breast cancer detection. Providing high quality mammography education to radiology trainees is essential, since excellent interpretation skills are needed to ensure the highest benefit of screening mammography for patients. The authors have previously proposed a computer-aided education system based on trainee models. Those models relate human-assessed image characteristics to trainee error. In this study, the authors propose to build trainee models that utilize features automatically extracted from images using computer vision algorithms to predict likelihood of missing each mass by the trainee. This computer vision-based approach to trainee modeling will allow for automatically searching large databases of mammograms in order to identify challenging cases for each trainee. Methods: The authors’ algorithm for predicting the likelihood of missing a mass consists of three steps. First, a mammogram is segmented into air, pectoral muscle, fatty tissue, dense tissue, and mass using automated segmentation algorithms. Second, 43 features are extracted using computer vision algorithms for each abnormality identified by experts. Third, error-making models (classifiers) are applied to predict the likelihood of trainees missing the abnormality based on the extracted features. The models are developed individually for each trainee using his/her previous reading data. The authors evaluated the predictive performance of the proposed algorithm using data from a reader study in which 10 subjects (7 residents and 3 novices) and 3 experts read 100 mammographic cases. Receiver operating characteristic (ROC) methodology was applied for the evaluation. Results: The average area under the ROC curve (AUC) of the error-making models for the task of predicting which masses will be detected and which will be missed was 0.607 (95% CI,0.564-0.650). This value was statistically significantly different from 0.5 (p < 0.0001). For the 7 residents only, the AUC performance of the models was 0.590 (95% CI,0.537-0.642) and was also significantly higher than 0.5 (p = 0.0009). Therefore, generally the authors’ models were able to predict which masses were detected and which were missed better than chance. Conclusions: The authors proposed an algorithm that was able to predict which masses will be detected and which will be missed by each individual trainee. This confirms existence of error-making patterns in the detection of masses among radiology trainees. Furthermore, the proposed methodology will allow for the optimized selection of difficult cases for the trainees in an automatic and efficient manner.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Diffractive optical computing in free space

Abstract Structured optical materials create new computing paradigms using photons, with transformative impact on various fields, including machine learning, computer vision, imaging, telecommunications, and sensing. This Perspective sheds light on the potential of free-space optical systems based on engineered surfaces for advancing optical computing. Manipulating light in unprecedented ways, emerging structured surfaces enable all-optical implementation of various mathematical functions and machine learning tasks. Diffractive networks, in particular, bring deep-learning principles into the design and operation of free-space optical systems to create new functionalities. Metasurfaces consisting of deeply subwavelength units are achieving exotic optical responses that provide independent control over different properties of light and can bring major advances in computational throughput and data-transfer bandwidth of free-space optical processors. Unlike integrated photonics-based optoelectronic systems that demand preprocessed inputs, free-space optical processors have direct access to all the optical degrees of freedom that carry information about an input scene/object without needing digital recovery or preprocessing of information. To realize the full potential of free-space optical computing architectures, diffractive surfaces and metasurfaces need to advance symbiotically and co-evolve in their designs, 3D fabrication/integration, cascadability, and computing accuracy to serve the needs of next-generation machine vision, computational imaging, mathematical computing, and telecommunication technologies.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Vision Transformers Explained Series

Since their introduction in 2017 with Attention is All You Need¹, transformers have established themselves as the state of the art for natural language processing (NLP). In 2021, An Image is Worth 16x16 Words successfully adapted transformers for computer vision tasks. Since then, numerous transformer-based architectures have been proposed for computer vision. This article walks through the Vision Transformer (ViT) as laid out in An Image is Worth 16x16 Words.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

PlantCV v2: Image analysis software for high-throughput plant phenotyping

Systems for collecting image data in conjunction with computer vision techniques are a powerful tool for increasing the temporal resolution at which plant phenotypes can be measured non-destructively. Computational tools that are flexible and extendable are needed to address the diversity of plant phenotyping problems. We previously described the Plant Computer Vision (PlantCV) software package, which is an image processing toolkit for plant phenotyping analysis. The goal of the PlantCV project is to develop a set of modular, reusable, and repurposable tools for plant image analysis that are open-source and community-developed. Here in this paper we present the details and rationale for major developments in the second major release of PlantCV. In addition to overall improvements in the organization of the PlantCV project, new functionality includes a set of new image processing and normalization tools, support for analyzing images that include multiple plants, leaf segmentation, landmark identification tools for morphometrics, and modules for machine learning.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1994

Research on an autonomous vision-guided helicopter

Integration of computer vision with on-board sensors to autonomously fly helicopters was researched. The key components developed were custom designed vision processing hardware and an indoor testbed. The custom designed hardware provided flexible integration of on-board sensors with real-time image processing resulting in a significant improvement in vision-based state estimation. The indoor testbed provided convenient calibrated experimentation in constructing real autonomous systems.

Amidi, Omead↗
DOE OSTI2026

YOLO11 to SAM2 pipeline for feature extraction from nuclear test films

The response to the effects of nuclear detonations is supported by models that describe the evolution of the nuclear fireball and cloud and the associated transport of active debris. Validation of those descriptions relies on data from the nuclear test operations. Video records of those events offer a rich source of information that was exploited to a limited extent in historic analyses. Computer vision and machine learning techniques are powerful tools that can be used to increase the number of measurements that can be obtained from those films. In this work, we apply computer vision techniques to automatically track the temporal evolution of the nuclear fireball. In particular, we apply You Only Look Once 11 (YOLO11) and Segment Anything Model 2 (SAM2) in combination with minimal human intervention to digitized versions of the original nuclear test films. As part of the proposed workflow, the YOLO11 model is applied to films to determine bounding boxes for the fireball within each frame. These are then used as inputs to SAM2, which uses image segmentation to determine the fireball boundaries and their temporal evolution. We assess the accuracy of our approach by using it to determine the energy released during the Trinity nuclear test and comparing the results with previous analyses based on manual measurements.

Van Exel, Kimberly [ORNL] (ORCID:0009000877463894)↗
NASA NTRS1988

Understanding of and applications for robot vision guidance at KSC

The primary thrust of robotics at KSC is for the servicing of Space Shuttle remote umbilical docking functions. In order for this to occur, robots performing servicing operations must be capable of tracking a swaying Orbiter in Six Degrees of Freedom (6-DOF). Currently, in NASA KSC's Robotic Applications Development Laboratory (RADL), an ASEA IRB-90 industrial robot is being equipped with a real-time computer vision (hardware and software) system to allow it to track a simulated Orbiter interface (target) in 6-DOF. The real-time computer vision system effectively becomes the eyes for the lab robot, guiding it through a closed loop visual feedback system to move with the simulated Orbiter interface. This paper will address an understanding of this vision guidance system and how it will be applied to remote umbilical servicing at KSC. In addition, other current and future applications will be addressed.

Shawaga, Lawrence M.↗
NASA NTRSDate not supplied

Visual and Inertial Datasets for an eVTOL Aircraft Approach and Landing Scenario

A National Aeronautics and Space Administration (NASA) project developing computer vision algorithms for autonomous flight is producing real-world datasets with cameras mounted on aircraft. In related domains, such as autonomous driving, open datasets are key to innovation and advancement in computer vision and autonomous perception for future Advanced Air Mobility (AAM) operations. Few vision datasets, however, are publicly available in the aviation context. This paper introduces preliminary datasets containing several examples of approach and landing scenarios. The platform aircraft include a multirotor small unmanned aerial system (sUAS) and a crewed helicopter as surrogates for future electric vertical take-off and landing (eVTOL) aircraft. The dataset provides video imagery with associated inertial navigation system-global positioning system (INS-GPS) position and attitude estimates and other sensors. Surveyed locations of the visual features of the landing area are included. This dataset is the first to be released in an ongoing effort to collect and share large, diverse datasets relevant to autonomous aviation; community critique that can inform and improve future flight campaigns is welcome.

Nelson Brown↗
NASA NTRS2017

Object and Facial Recognition in Augmented and Virtual Reality: Investigation into Software, Hardware and Potential Uses

As augmented and virtual reality grows in popularity, and more researchers focus on its development, other fields of technology have grown in the hopes of integrating with the up-and-coming hardware currently on the market. Namely, there has been a focus on how to make an intuitive, hands-free human-computer interaction (HCI) utilizing AR and VR that allows users to control their technology with little to no physical interaction with hardware. Computer vision, which is utilized in devices such as the Microsoft Kinect, webcams and other similar hardware has shown potential in assisting with the development of a HCI system that requires next to no human interaction with computing hardware and software. Object and facial recognition are two subsets of computer vision, both of which can be applied to HCI systems in the fields of medicine, security, industrial development and other similar areas.

object recognition↗
DOE OSTI2026

Pre-training Vision Models for the Classification of Alerts from Wide-field Time-domain Surveys

Modern wide-field time-domain surveys facilitate the study of transient, variable and moving phenomena by conducting image differencing and relaying alerts to their communities. Machine learning tools have been used on data from these surveys and their precursors for more than a decade, and convolutional neural networks (CNNs), which make predictions directly from input images, saw particularly broad adoption through the 2010s. Since then, continually rapid advances in computer vision have transformed the standard practices around using such models. It is now commonplace to use standardized architectures pre-trained on large corpora of everyday images (e.g., ImageNet). In contrast, time-domain astronomy studies still typically design custom CNN architectures and train them from scratch. Here, we explore the effects of adopting various pre-training regimens and standardized model architectures on the performance of alert classification. We find that the resulting models match or outperform a custom, specialized CNN like what is typically used for filtering alerts. Moreover, our results show that pre-training on galaxy images from Galaxy Zoo tends to yield better performance than pre-training on ImageNet or training from scratch. We observe that the design of standardized architectures are much better optimized than the custom CNN baseline, requiring significantly less time and memory for inference despite having more trainable parameters. On the eve of the Legacy Survey of Space and Time and other image-differencing surveys, these findings advocate for a paradigm shift in the creation of vision models for alerts, demonstrating that greater performance and efficiency, in time and in data, can be achieved by adopting the latest practices from the computer vision field.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1994

Low computation vision-based navigation for a Martian rover

Construction and design details of the Mobot Vision System, a small, self-contained, mobile vision system, are presented. This system uses the view from the top of a small, roving, robotic vehicle to supply data that is processed in real-time to safely navigate the surface of Mars. A simple, low-computation algorithm for constructing a 3-D navigational map of the Martian environment to be used by the rover is discussed.

Gavin, Andrew S.↗
NASA NTRS1991

A vision-based end-point control for a two-link flexible manipulator

The measurement and control of the end-effector position of a large two-link flexible manipulator are investigated. The system implementation is described and an initial algorithm for static end-point positioning is discussed. Most existing robots are controlled through independent joint controllers, while the end-effector position is estimated from the joint positions using a kinematic relation. End-point position feedback can be used to compensate for uncertainty and structural deflections. Such feedback is especially important for flexible robots. Computer vision is utilized to obtain end-point position measurements. A look-and-move control structure alleviates the disadvantages of the slow and variable computer vision sampling frequency. This control structure consists of an inner joint-based loop and an outer vision-based loop. A static positioning algorithm was implemented and experimentally verified. This algorithm utilizes the manipulator Jacobian to transform a tip position error to a joint error. The joint error is then used to give a new reference input to the joint controller. The convergence of the algorithm is demonstrated experimentally under payload variation. A Landmark Tracking System (Dickerson, et al 1990) is used for vision-based end-point measurements. This system was modified and tested. A real-time control system was implemented on a PC and interfaced with the vision system and the robot.

Obergfell, Klaus↗
DOE OSTI2024

Introducing SpaceNet 9 - Cross-Modal Satellite Imagery Registration for Natural Disaster Responses

Computer vision algorithms are increasingly leveraged to accelerate geospatial analysis for disaster response and recovery. As the diversity of remote sensing imagery grows with optical, SAR, and other modalities, a perquisite for analytics is cross-modal image registration. There is a high potential to harness computer vision for this pre-processing requirement toward enabling downstream analytics such as heterogeneous change detection, automated feature extraction, and data fusion. Advancement in these areas has the potential to simplify data wrangling tasks and further accelerate disaster response timelines. The SpaceNet 9 challenge (launching in mid-2024) focuses on addressing the cross-modal image registration problem and demonstrating the utility of such modules on earthquake impacted scenarios. This paper describes the motivation for the SpaceNet 9 and provides a first overview of the dataset, the baseline algorithm, and implications for seeking cross-modal image registration in Earth observation. Code is available at https://github.com/SpaceNetChallenge/SpaceNet9.

Hansch, Ronny↗
NASA NTRS1991

Experimental validation of docking and capture using space robotics testbeds

This presentation describes the application of robotic and computer vision systems to validate docking and capture operations for space cargo transfer vehicles. Three applications are discussed: (1) air bearing systems in two dimensions that yield high quality free-flying, flexible, and contact dynamics; (2) validation of docking mechanisms with misalignment and target dynamics; and (3) computer vision technology for target location and real-time tracking. All the testbeds are supported by a network of engineering workstations for dynamic and controls analyses. Dynamic simulation of multibody rigid and elastic systems are performed with the TREETOPS code. MATRIXx/System-Build and PRO-MATLAB/Simulab are the tools for control design and analysis using classical and modern techniques such as H-infinity and LQG/LTR. SANDY is a general design tool to optimize numerically a multivariable robust compensator with a user-defined structure. Mathematica and Macsyma are used to derive symbolically dynamic and kinematic equations.

Spofford, John↗
DOE OSTI2019

Robotic Non-destructive Test of Concrete Structures with GPR, Impact Echo and 3D Vision - 19540

Structural members come across with various internal defects that cannot be evaluated visually. Instead, hand-held nondestructive sensors are used to detect such defects. In many situations, the hand-held NDT is not possible in high-risk structures because of limited access or safety issues. Therefore, Robotic NDT plays major role in assessing such structures. Robot Operating Systems (ROS) is a powerful open source tool, which is used to program and control robots and other devices enabling one to use sensors, devices and new functionalities such as autonomous navigation, visual perception. For the research presented, ROS is used to program the Sawyer robot arm manufactured by Rethink Robotics. The control system for the robot ensures the ability to adapt to different weights of integrated measuring devices and measured force applied against the wall with a wheeled measuring device while making a remote sweep. Twelve reinforced concrete (RC) beams built with characterized defects are evaluated with the robotic arm integrated with the measuring devices. The RC beams are characterized by voids, corrosion and debonding problems of rebars. The Impact Echo and Ground Penetrating Radar (GPR) devices attached to the robot arm are used to measure the material properties of the concrete beams. The robot arm ensures consistency and uniformity of measurement even in obscure locations. Infrared cameras are used to obtain 3D images of the surfaces of the concrete structures. Roughness is analyzed using computer vision techniques prior to the use of other sensors that are in touch with the surfaces. The computer vision is also used to handle the images collected from GPR sensor. Hand-held NDE is also performed to verify the results obtained by robotic examination. (authors)

42 ENGINEERING↗
DOE OSTI2018

Automatic Imagery Data Analysis for Proactive Computer-Based Workflow Management during Nuclear Power Plant Outages

This report is being submitted for the task “Final Report” of DOE NEUP Project 15-8121 “Automatic Imagery Data Analysis for Proactive Computer-Based Workflow Management during Nuclear Power Plant Outages.” Typical nuclear power plant (NPP) outages always involve thousands of maintenance and refueling activities and a large number of workers in limited workspaces, while having tight schedules and zero-tolerance for accidents. During an outage, thousands of workers will be working in various workspaces across the NPP. High outage costs and expensive delays (approximately 1.5 million dollars of loss per day of delay) in NPP maintenance demand tight outage schedules. In packed workspaces, an automatic system that monitors human behaviors in real-time and provides insights about current and pending schedule deviations from the plan is critical for ensuring: 1) effective collaboration among workers and worker teams from different trades; 2) less waste of time and resources due to the lack of situational awareness; and 3) proactive outage project control. The overall goal of this project is to test the hypothesis that real-time imagery-based object tracking and spatial analysis, as well as human behavior modeling of outage participants, will significantly improve the efficiency of outage control while lowering the rates of accidents and incidents. Three objectives of this project are: 1) Establish real-time object tracking and spatiotemporal analysis methods that automatically assess the productivity of field activities and detect anomalous spatiotemporal relationships among activities that cause inefficiencies and risks; 2) Establish realtime human tracking and human factor modeling methods for automatically diagnosing unexpected actions of and interactions between outage participants, those which cause inefficient collaborations between Advanced Outage Control Center (AOCC), satellite outage centers, NPP workers, and maintenance service providers; and 3) Test the proposed automated object tracking, human behavior modeling, and spatiotemporal analysis methods in outage control case studies in order to characterize the effectiveness of automated imagery-data-driven methods in proactively improving the efficiency and safety of workflows in outage coordination and risk management. Recent studies of detailed human behavior monitoring on construction sites have examined the potential of applying advanced computer vision algorithms in detecting and tracking anomalous workers (i.e., workers who do not wear hard hats or safety vests) for ensuring job site safety. Some studies of human factor studies revealed the importance of modeling detailed interactions between individuals within and across teams in better understanding the impact of human in proactive project control. Other studies in the construction domain have developed computational simulation frameworks that formalize detailed spatiotemporal interactions between tasks in order to simulate the impacts of individual tasks on the performance of workflows. While integrated analyses that combine human factor assessment, as well as image processing and simulation, are in demand for effective decision-making, limited studies have examined the potential of such integrated analyses in NPP outage control. This research project has examined an automatic outage monitoring and control system that integrates human factor analysis, computer vision techniques, and simulation methods in order to enable engineers to better understand the interactions between humans, resources, and workflows during outage processes. The project aims at providing NPP maintenance agencies insights into more efficient use of limited resources in extending the life of a nuclear plant, as well as reducing waste while ensuring sufficient generation of electricity. This study is significant for the safety of nuclear plants, sustainable electricity generation for livable communities, and cost savings for maintaining electricity infrastructures in the United States.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Multisource Mobile Transfer Learning Algorithm Based on Dynamic Model Compression

With the development of the Internet of Things, the application of computer vision on mobile phones is becoming more and more extensive and people have higher and higher requirements for the timeliness of the recognition results returned and the processing capabilities of the mobile phone for image recognition. However, the processing capability and storage capability of the user terminal equipment cannot meet the needs of identifying and storing a large number of pictures, and the data transmission process will cause high energy consumption of the terminal equipment. At the same time, multisource deep transfer learning has outstanding performance in computer vision and image classification. However, due to the huge amount of calculation of the deep network model, it is impossible to use the existing excellent network model to realize image recognition and classification on the mobile terminal. In order to solve the abovementioned problems, we propose a multisource mobile transfer learning algorithm based on dynamic model compression, this algorithm considers the realization of multisource transfer learning computing in the case of multiple mobile device computing source domains, and the method also guarantees data privacy and security for each device (origin domain). Meanwhile, extensive experiments show that our method can achieve remarkable results in popular image classification datasets.

Gao, Peng↗