Search NASA⌕ Search

SEARCH · Search NASA

Results for “COMPUTER VISION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

2021 Smoky Mountains Conference Data Challenge Synthetic-to-Real Domain Adaptation for Autonomous Driving Dataset

The dataset is comprised of both real and synthetic images from a vehicle's forward-facing camera. Each camera image is accompanied by a corresponding pixel-level semantic segmentation image (all files are .png files). In total, the dataset contains 5600 images in the training/validation set and 1400 images in the testing set. The training dataset contains mostly synthetic RGB images collected with a wide range of weather and lighting conditions using the CARLA simulator [1]. In addition, the training data also includes a small pre-selected subset of data from the Cityscapes training dataset – which is comprised of RGB-segmentation image pairs from driving scenarios in various European cities [2]. The testing data is split into three sets. The first set contains synthetic CARLA images with weather/lighting conditions that were not present in the training set. The second set is a subset of the Cityscapes testing dataset. Finally, the third set is an unknown testing set which will not be revealed to the participants until after the submission deadline. [1] Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, October). CARLA: An open urban driving simulator. In Conference on robot learning (pp. 1-16). PMLR. [2] Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., ... and Schiele, B. (2016). The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3213-3223).

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2023

Trustworthy Physics-Informed Deep Learning for Predictive Scientific Computing

This project has developed powerful trustworthy physics-informed deep learning (TPiDL) models and methods to fundamentally enhance the scale and power of computational modeling in the scientific and engineering domains. Deep learning (DL) has radically advanced the state-of-the-art in machine learning, computer vision, natural language processing, and also scientific computing. Nevertheless, progress has been driven almost entirely by empirical observations, hacks, and tricks. Under the support of this project, the graph operator learning tools and advanced trustworthy physical informed neural networks have been developed. In addition, stochastic gradient replica-exchange Markov Chain Monte Carlo (MCMC) sampling algorithms have been designed to quantify the uncertainties and speed up the training of large-scale neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Permeability Prediction Using Vision Transformers

Accurate permeability predictions remain pivotal for understanding fluid flow in porous media, influencing crucial operations across petroleum engineering, hydrogeology, and related fields. Traditional approaches, while robust, often grapple with the inherent heterogeneity of reservoir rocks. With the advent of deep learning, convolutional neural networks (CNNs) have emerged as potent tools in image-based permeability estimation, capitalizing on micro-CT scans and digital rock imagery. This paper introduces a novel paradigm, employing vision transformers (ViTs)—a recent advancement in computer vision—for this crucial task. ViTs, which segment images into fixed-sized patches and process them through transformer architectures, present a promising alternative to CNNs. We present a methodology for implementing ViTs for permeability prediction, its results on diverse rock samples, and a comparison against conventional CNNs. The prediction results suggest that, with adequate training data, ViTs can match or surpass the predictive accuracy of CNNs, especially in rocks exhibiting significant heterogeneity. This study underscores the potential of ViTs as an innovative tool in permeability prediction, paving the way for further research and integration into mainstream reservoir characterization workflows.

58 GEOSCIENCES↗
DOE OSTI2026

RU-net for automatic characterization of TRISO fuel cross sections

During irradiation, phenomena such as kernel swelling and buffer densification may impact the performance of tristructural isotropic (TRISO) particle fuel. Post-irradiation microscopy is often used to identify these irradiation-induced morphologic changes. However, each fuel compact generally contains thousands of TRISO particles. Manually performing the work to get statistical information on these phenomena is cumbersome and subjective. Here, to reduce the subjectivity inherent in that process and to accelerate data analysis, we used convolutional neural networks (CNNs) to automatically segment cross-sectional images of microscopic TRISO layers. CNNs are a class of machine-learning algorithms specifically designed for processing structured grid data. They have gained popularity in recent years due to their remarkable performance in various computer vision tasks, including image classification, object detection, and image segmentation. In this research, we generated a large irradiated TRISO layer dataset with more than 2,000 microscopic images of cross-sectional TRISO particles and the corresponding annotated images. Based on these annotated images, we used different CNNs to automatically segment different TRISO layers. These CNNs include RU-Net (developed in this study), as well as three existing architectures: U-Net, Residual Network (ResNet), and Attention U-Net. The preliminary results show that the model based on RU-Net performs best in terms of Intersection over Union (IoU). Using CNN models, we can expedite the analysis of TRISO particle cross sections, significantly reducing the manual labor involved and improving the objectivity of the segmentation results.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

YOLO for Radio Frequency Signal Classification

Radio frequency signal classification plays a pivotal role in various applications, including spectrum management, wireless security, and cognitive radio. Extant signal classification methods require significant data throughput and are not multilabel. We propose a novel approach to radio frequency signal classification by leveraging the You Only Look Once (YOLO) object detection method. YOLO is a state-of-the-art deep learning model renowned for its real-time object detection capabilities in computer vision applications. We adapt YOLO for signal classification to enable the automatic and efficient identification of various signal types within a power spectral density image. Index Terms—radio-frequency analysis, object detection, neural networks, machine learning, deep learning.

42 ENGINEERING↗
DOE OSTI2024

Coaxial color channel focus evaluation to estimate standoff height in directed energy deposition additive manufacturing

Directed energy deposition (DED) is an additive manufacturing process that is being rapidly adopted by industry and is well suited for the fabrication of complex components in a variety of metal alloys. In laser cladding systems such as DED, powder is blown in a stream to a metal substrate coincident with a laser necessary to deposit molten metal with 3D spatial control. The focus of both the laser and the powder stream are crucial, and best deposition occurs at a predetermined standoff height between the build surface and the print head. Generally, no monitoring of this distance is implemented in commercial DED systems. Due to potential over or under building, the standoff height often changes over time but tends to self-correct. However, inexpensive and minimally intrusive methods to identify optimal standoff are required to provide real-time control to maintain the optimal distance. The present work explores the quantification of the focus of the three-color channels of a coaxial camera to determine the standoff height. An experiment was performed in which a 254 mm wall is built and the standoff height, initially 5.0 mm below the optimal position, was then intentionally increased every 25.4 mm of wall length by an amount of 1.0 mm to a final position 7.0 mm above optimal. Computer vision is demonstrated to monitor the amount of focus in each color band and estimate standoff distance. Finally, a response can be calculated in under 40 ms using simple hardware and can work in most laser-based DED systems.

36 MATERIALS SCIENCE↗
DOE OSTI2023

A New Reflected Target Optical Assessment System - Stage 1 Development Results: Preprint

NREL has completed stage 1 development of an indoor optical measurement tool for fully assembled heliostats and single facets. This tool began as an indoor version of NREL's NIO technique. It uses similar techniques to other available tools (deflectometry, photogrammetry, etc.), but is designed require very little infrastructure, labor, and time to set up and collect surface slope and canting measurements on fully assembled heliostats and parabolic trough facets, making it a valuable tool for quality assurance and laboratory measurement of helio-stat optics. It accomplishes this by using computer vision, photogrammetry, and multiple images stitched together to minimize the printed target size and required setup precision. This adaptable setup is useful for taking measurements at a variety of heliostat pointing angles, and for measuring fully assembled heliostats on the assembly line. The measurement system's methodology and an analysis of its uncertainty, sensitivity and comparison with established optical measurement systems is described below.

canting↗
DOE OSTI2021

Time-Resolved Geometric Feature Tracking Elucidates Laser-Induced Keyhole Dynamics

In this report, during laser melting of metals, localized metal evaporation resulting in the formation of a keyhole shaped cavity can occur if processing conditions are chosen with high power density. An unstable keyhole can have deleterious effects in certain applications (e.g., laser powder bed fusion) as it increases the likelihood of producing defects such as porosity. In this work, we propose a pipeline that enables complete segmentation and extraction of various geometric features in keyholing conditions. In situ synchrotron high-speed X-ray visualization at the Advanced Photon Source provides large datasets of experimental images with a high spatio-temporal resolution across a range of laser parameters for Ti-6Al- 4 V. Computer vision image processing techniques were used to extract time-resolved quantitative geometric features (e.g., depth, width, front wall angle) throughout keyhole evolution which were subsequently analyzed to understand the relationship between the variation of local keyhole geometry and processing conditions. This analysis is the first to employ a data-driven approach to further our understanding of the keyholing process regime.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Automatic recognition system for document digitization in nuclear power plants

With the increasing number of data-driven models in nuclear applications, large volumes of numerical data are required to accurately model and predict the health status of a plant component. However, many historical operation logs that contain useful information are not fully utilized due to the lack of a systematic approach of digitization. To overcome this issue, this study proposes an automatic pipeline for extracting information from handwritten tabular documents collected from nuclear power plants. In our pipeline, we first denoise scanned documents with morphological operations, and then extract relevant parts from individual pages using both traditional computer vision and neural network methods. Handwriting recognition is applied to obtain text and numbers. As the most challenging step is how to crop only relevant information, the main focus of our paper is to detect tables and cells from scanned handwritten documents. Here we evaluate the efficiency and accuracy of our proposed method on handwritten operational reports obtained from a real-world case study. The results demonstrate the high accuracy and practicality of our proposed method.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2025

Unveiling the Hidden Evolution of Crystal Defects and Disorder in Energy Materials

Control of point defects and disorder in functional thin films and 2D materials is critical to realizing their full potential in applications ranging from energy storage to advanced electronics. However, these phenomena are often poorly understood, difficult to characterize, and challenging to direct with precision. This presentation explores emerging multi-modal computer vision to decipher and predict order in materials across multiple length scales in the electron microscope, from the atomic to the nanoscale. By fusing data from diverse sources, these powerful models provide unprecedented insights into materials' lifecycles, enabling the control of defects and their associated properties at a fundamental level. This capability promises to transform materials design and accelerate the development of next-generation technologies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Automated phase segmentation and quantification of high-resolution TEM image for alloy design

In the alloy design and development process, a wealth of atomically resolved structural high-resolution transmission electron microscopy (HRTEM) images are produced. Identifying the different nano-precipitate phases and tracking their evolution under various compositions and during manufacturing or post-processing requires hundreds of HRTEM images and thousands of precipitates. The nanoscopic phase information labeling and analysis purely relies on humans are prohibitively costly and time-consuming, sometimes not reliable because of the lack of authoritative knowledge. Here, in this work, we develop a novel unsupervised machine learning approach coupled with adaptive computer vision techniques with features in the Fourier space to automatically determine the number of phases and segment/quantify the phases with nanoscale resolution, allowing for quantitative correlation between nanostructure formation, processing and functional properties. To automate the phase extraction/quantification and ascertain its applicability, we have applied the developed framework to the HRTEM images from several alloy systems, processing conditions, image magnifications, and phase types and morphologies (precipitates, nano-twins, stacking faults, crystalline matrix, and amorphous structures) for verification. This study paves the road for compression, visualization, and translation of raw image structural data into physically relevant information in real-time with minimal human supervision. It shows the promise of enabling high-throughput materials characterization for the acceleration of alloy manufacturing and design.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Identifying Genomic Islands with Deep Neural Networks

Background Horizontal gene transfer is the main source of adaptability for bacteria, through which genes are obtained from different sources including bacteria, archaea, viruses, and eukaryotes. This process promotes the rapid spread of genetic information across lineages, typically in the form of clusters of genes referred to as genomic islands (GIs). Different types of GIs exist, and are often classified by the content of their cargo genes or their means of integration and mobility. While various computational methods have been devised to detect different types of GIs, no single method is capable of detecting all types. Results We propose a method, which we call Shutter Island, that uses a deep learning model (Inception V3, widely used in computer vision) to detect genomic islands. The intrinsic value of deep learning methods lies in their ability to generalize. Via a technique called transfer learning, the model is pre-trained on a large generic dataset and then re-trained on images that we generate to represent genomic fragments. We demonstrate that this image-based approach generalizes better than the existing tools. Conclusions We used a deep neural network and an image-based approach to detect the most out of the correct GI predictions made by other tools, in addition to making novel GI predictions. The fact that the deep neural network was re-trained on only a limited number of GI datasets and then successfully generalized indicates that this approach could be applied to other problems in the field where data is still lacking or hard to curate.

Computer Vision↗
DOE OSTI2020

Mapping of Benthic Habitats at Marine Renewable Energy Sites Using Multibeam Echosounder and Sediment Profile Imaging Technologies

Objectives/Scope: The goal of this work was to develop a consistent and semi-automated seafloor survey method for generating high-resolution, benthic habitat maps for environmental assessments and monitoring of marine renewable energy sites. Sediment profile and plan view imaging (SPI/PV) technology was combined with multibeam bathymetry and acoustic backscatter methods to demonstrate a rapid, cost-effective benthic mapping protocol. A key technical innovation of this project was the development of an image processing platform that automatically measures key features from the images. Methods: Multibeam echosounder (MBES) acoustic and SPI/PV surveys were conducted at three coastal areas off the U.S. west coast, including the PacWave South energy test site off of Newport, Oregon. Point data on physical and biological sediment conditions obtained from the SPI/PV imagery were used to efficiently ground-truth the high-resolution MBES bathymetry and backscatter mosaics. The SPI camera is an optical corer that obtains an undisturbed 21 by 15 cm, high-resolution, cross-sectional image of the sediment–water interface and upper sediment column. The plan view camera attached to the SPI camera frame captures a downward looking view of the seabed immediately before the SPI image is obtained. As part of this project, we developed a computer vision image processing platform (iSPI) that uses deep convolutional neural networks and other approaches to automatically identify and measure key features in the images, such as grain size and surface relief. Results: Detailed benthic habitat maps were generated using this mapping approach for three different marine settings. The sites mapped included a silt-dominated embayment; a sloping, nearshore, transitional very fine to coarse sand bottom; and a medium, sand-dominated continental shelf marine energy test site. In each case, acoustic and imaging surveys were completed in less than week, and detailed benthic habitat maps were generated within 60 days. The results were placed within the Coastal and Marine Ecological Classification Standard (CMECS) habitat mapping framework, and various combinations of the bathymetry, backscatter, and SPI and PV data were used to generate CMECS component maps. Novel Information: This project developed and demonstrated a repeatable and cost-effective approach for efficiently mapping benthic habitat conditions over broad areas of the seafloor by combining state-of-the-art acoustic and imaging techniques. The primary survey tools used in this project have been used previously in both the offshore renewable (wind) and oil and gas sectors to map and monitor benthic environments. This project’s innovations include 1) the focused use of high-resolution SPI/PV imagery to ground-truth acoustic mosaics, and 2) the development of a computer automated image analysis processing tool that both streamlines and standardizes the generation of data from the imagery and makes the data extraction process more cost-effective and repeatable.

13 HYDRO ENERGY↗
DOE OSTI2022

SAFE-OCC: A novelty detection framework for Convolutional Neural Network sensors and its application in process control

Herein we present a novelty detection framework for Convolutional Neural Network (CNN) sensors that we call Sensor-Activated Feature Extraction One-Class Classification (SAFE-OCC). We show that this framework enables the safe use of computer vision sensors in process control architectures. Emergent control applications use CNN models to map visual data to a state signal that can be interpreted by the controller. Incorporating such sensors introduces a significant system operation vulnerability because CNN sensors can exhibit high prediction errors when exposed to novel (abnormal) visual data. Unfortunately, identifying such novelties in real-time is nontrivial. To address this issue, the SAFE-OCC framework leverages the convolutional blocks of the CNN to create an effective feature space to conduct novelty detection using a desired one-class classification technique. This approach engenders a feature space that directly corresponds to that used by the CNN sensor and avoids the need to derive an independent latent space. We demonstrate the effectiveness of SAFE-OCC via simulated control environments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Microscopy modality transfer of steel microstructures: Inferring scanning electron micrographs from optical microscopy using generative AI

Scanning electron microscopy (SEM) is resource intensive, which limits its throughput in some applications. As an alternative, we propose applying computer vision and machine learning to generate high-quality synthetic SEM micrographs from micrographs obtained using light optical microscopy (LOM). Working with a correlated LOM/SEM dataset of dual-phase steel images, we test generative models of various architectures, including encoder-decoder networks, generative adversarial networks (GANs), and diffusion-based models. We find that the diffusion models significantly outperform other methods on both qualitative and quantitative assessments, while preserving key metallurgical meaning. This work establishes diffusion as the state-of-the-art for microscopy modality transfer and demonstrates the potential of AI-powered microscopy to enhance LOM with micron scale structural recreation.

Computer vision↗
DOE OSTI2025

Effective Defect Detection Using Instance Segmentation for NDI

Ultrasonic testing is a common Non-Destructive Inspection (NDI) method used in aerospace manufacturing. However, the complexity and size of the ultrasonic scans make it challenging to identify defects through visual inspection or machine learning models. Using computer vision techniques to identify defects from ultrasonic scans is an evolving research area. In this study, we used instance segmentation to identify the presence of defects in the ultrasonic scan images of composite panels that are representative of real components manufactured in aerospace. We used two models based on Mask- RCNN (Detectron 2) and YOLO 11 respectively. Additionally, we implemented a simple statistical pre-processing technique that reduces the burden of requiring custom-tailored pre-processing techniques. Our study demonstrates the feasibility and effectiveness of using instance segmentation in the NDI pipeline by significantly reducing data pre-processing time, inspection time, and overall costs.

computer vision techniques↗
DOE OSTI2020

Automated identification of deformation twin systems in Mg WE43 from SEM DIC

In this study, the application of machine learning and computer vision approaches to microscale deformation data for the automated identification of deformation twinning systems, and their associated twin area, is introduced. Deformation data was obtained during in-situ SEM compression testing of a WE43 Mg alloy using digital image correlation (DIC) modified for use with electron microscopy. A 5.7 mm × 3.4 mm area of interest was analyzed, generating ~100 million data points of deformation. Experimental twin trace directions were determined by applying k-means clustering, morphological thinning, and Hough transforms to the deformation data. The identification of deformation twinning was achieved by consideration of the twin trace direction, its strain value, and its evolution. The deformation map was divided into areas corresponding to individual active twin systems, enabling the analysis of microstructure dependence of deformation behavior and twinning activity. The performance of the proposed twinning identification approach was evaluated by accuracy, precision, and sensitivity metrics. The effect of the tuning parameters used in the algorithm on the performance is also discussed.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Instance Segmentation for Direct Measurements of Satellites in Metal Powders and Automated Microstructural Characterization from Image Data

In this work, we propose instance segmentation as a useful tool for image analysis in materials science. Instance segmentation is an advanced technique in computer vision which generates individual segmentation masks for every object of interest that is recognized in an image. Using an out-of-the-box implementation of Mask R-CNN, instance segmentation is applied to images of metal powder particles produced through gas atomization. Leveraging transfer learning allows for the analysis to be conducted with a very small training set of labeled images. As well as providing another method for measuring the particle size distribution, we demonstrate the first direct measurements of the satellite content in powder samples. After analyzing the results for the labeled data dataset, the trained model was used to generate measurements for a much larger set of unlabeled images. The resulting particle size measurements showed reasonable agreement with laser scattering measurements. The satellite measurements were self-consistent and showed good agreement with the expected trends for different samples. Finally, we present a small case study showing how instance segmentation can be used to measure spheroidite content in the UltraHigh Carbon Steel DataBase, demonstrating the flexibility of the technique.

36 MATERIALS SCIENCE↗