Search NASA⌕ Search

SEARCH · Search NASA

Results for “Instance segmentation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

PlantSegNet: 3D point cloud instance segmentation of nearby plant organs with identical semantics

In this study, we introduce PlantSegNet, a novel neural network model for instance segmentation of nearby objects with similar geometric structures. Our work addresses the challenges of instance segmentation of plant point clouds, including the difficulty of annotating and labeling point clouds, the loss of local structural information in neural network components, and the generation of large numbers of incorrect small clusters due to poor choices of the loss function. One of the key contributions of our approach is a digital twin of sorghum, i.e., a procedural sorghum model, which was used to generate point clouds of sorghum fields. This allowed us to create a large-scale, annotated, synthetic dataset of sorghum plants that we used to train our PlantSegNet model. We demonstrated the effectiveness of our method in segmenting instances of sorghum leaves grown in outdoor field settings. To the best of our knowledge, this is the first study to address this specific instance segmentation problem for plants grown in such a setting. We compared our proposed method with other state-of-the-art methods for indoor settings, including SGPN and TreePartNet, on both synthetic and real data. Furthermore, our results show that PlantSegNet outperforms these methods regarding accuracy, robustness, and efficiency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Instance Segmentation for Direct Measurements of Satellites in Metal Powders and Automated Microstructural Characterization from Image Data

In this work, we propose instance segmentation as a useful tool for image analysis in materials science. Instance segmentation is an advanced technique in computer vision which generates individual segmentation masks for every object of interest that is recognized in an image. Using an out-of-the-box implementation of Mask R-CNN, instance segmentation is applied to images of metal powder particles produced through gas atomization. Leveraging transfer learning allows for the analysis to be conducted with a very small training set of labeled images. As well as providing another method for measuring the particle size distribution, we demonstrate the first direct measurements of the satellite content in powder samples. After analyzing the results for the labeled data dataset, the trained model was used to generate measurements for a much larger set of unlabeled images. The resulting particle size measurements showed reasonable agreement with laser scattering measurements. The satellite measurements were self-consistent and showed good agreement with the expected trends for different samples. Finally, we present a small case study showing how instance segmentation can be used to measure spheroidite content in the UltraHigh Carbon Steel DataBase, demonstrating the flexibility of the technique.

36 MATERIALS SCIENCE↗
DOE OSTI2024

YOLO2U-Net: Detection-guided 3D instance segmentation for microscopy

Microscopy imaging techniques are instrumental for characterization and analysis of biological structures. As these techniques typically render 3D visualization of cells by stacking 2D projections, issues such as out-of-plane excitation and low resolution in the z-axis may pose challenges (even for human experts) to detect individual cells in 3D volumes as these non-overlapping cells may appear as overlapping. In this paper a comprehensive method for accurate 3D instance segmentation of cells in the brain tissue is introduced. The proposed method combines the 2D YOLO detection method with a multi-view fusion algorithm to construct a 3D localization of the cells. Next, the 3D bounding boxes along with the data volume are input to a 3D U-Net network that is designed to segment the primary cell in each 3D bounding box, and in turn, to carry out instance segmentation of cells in the entire volume. The promising performance of the proposed method is shown in comparison with current deep learning-based 3D instance segmentation methods.

3D instance segmentation↗
DOE OSTI2022

An intensity-based post-processing tool for 3D instance segmentation of organelles in soft X-ray tomograms

Investigating the 3D structures and rearrangements of organelles within a single cell is critical for better characterizing cellular function. Imaging approaches such as soft X-ray tomography have been widely applied to reveal a complex subcellular organization involving multiple inter-organelle interactions. However, 3D segmentation of organelle instances has been challenging despite its importance in organelle characterization. Here we propose an intensity-based post-processing tool to identify and separate organelle instances. Our tool separates sphere-like (insulin vesicle) and columnar-shaped organelle instances (mitochondrion) based on the intensity of raw tomograms, semantic segmentation masks, and organelle morphology. We validate our tool using synthetic tomograms of organelles and experimental tomograms of pancreatic β -cells to separate insulin vesicle and mitochondria instances. As compared to the commonly used connected regions labeling, watershed, and watershed + Gaussian filter methods, our tool results in improved accuracy in identifying organelles in the synthetic tomograms and an improved description of organelle structures in β -cell tomograms. In addition, under different experimental treatment conditions, significant changes in volumes and intensities of both insulin vesicle and mitochondrion are observed in our instance results, revealing their potential roles in maintaining normal β -cell function. Our tool is expected to be applicable for improving the instance segmentation of other images obtained from different cell types using multiple imaging modalities.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Effective Defect Detection Using Instance Segmentation for NDI

Ultrasonic testing is a common Non-Destructive Inspection (NDI) method used in aerospace manufacturing. However, the complexity and size of the ultrasonic scans make it challenging to identify defects through visual inspection or machine learning models. Using computer vision techniques to identify defects from ultrasonic scans is an evolving research area. In this study, we used instance segmentation to identify the presence of defects in the ultrasonic scan images of composite panels that are representative of real components manufactured in aerospace. We used two models based on Mask- RCNN (Detectron 2) and YOLO 11 respectively. Additionally, we implemented a simple statistical pre-processing technique that reduces the burden of requiring custom-tailored pre-processing techniques. Our study demonstrates the feasibility and effectiveness of using instance segmentation in the NDI pipeline by significantly reducing data pre-processing time, inspection time, and overall costs.

computer vision techniques↗
DOE OSTI2024

Automated Assembly Progress Monitoring in Modular Construction Factories Using Computer Vision-Based Instance Segmentation

Modular construction has recently gained interest as a transformative construction method. In this method, a large portion of the construction is performed inside factories, where processes are fast-paced and interdependent; therefore, any deviation from the schedule can delay the production. Such deviations are frequent in modular factories due to the labor-intensive nature of the tasks. This propagation of delays can be mitigated by continuously monitoring each process; however, current manual monitoring methods are laborious, and recently proposed contact sensor-based methods are intrusive to the work. In addition, recent computer vision-based monitoring methods inside factories are limited to detection algorithms that fail to provide the pixel-level accuracy required for assembly progress monitoring in highly occluded factory scenes, and they require a large number of manual annotations. Therefore, this paper proposes a method to monitor the installation of subassemblies in modular construction factories using mask R-CNN instance segmentation and improves the data efficiency of the model using a copy-paste augmentation method. This method was validated on the CCTV videos captured from a modular construction factory in the US, resulting in a 9% mAP improvement in segmentation.

computer vision↗
DOE OSTI2023

An efficient instance segmentation approach for studying fission gas bubbles in irradiated metallic nuclear fuel

Abstract Gaseous fission products from nuclear fission reactions tend to form fission gas bubbles of various shapes and sizes inside nuclear fuel. The behavior of fission gas bubbles dictates nuclear fuel performances, such as fission gas release, grain growth, swelling, and fuel cladding mechanical interaction. Although mechanical understanding of the overall evolution behavior of fission gas bubbles is well known, lacking the quantitative data and high-level correlation between burnup/temperature and microstructure evolution blocks the development of predictive models and reduces the possibility of accelerating the qualification for new fuel forms. Historical characterization of fission gas bubbles in irradiated nuclear fuel relied on a simple threshold method working on low-resolution optical microscopy images. Advanced characterization of fission gas bubbles using scanning electron microscopic images reveals unprecedented details and extensive morphological data, which strains the effectiveness of conventional methods. This paper proposes a hybrid framework, based on digital image processing and deep learning models, to efficiently detect and classify fission gas bubbles from scanning electron microscopic images. The developed bubble annotation tool used a multitask deep learning network that integrates U-Net and ResNet to accomplish instance-level bubble segmentation. With limited annotated data, the model achieves a recall ratio of more than 90%, a leap forward compared to the threshold method. The model has the capability to identify fission gas bubbles with and without lanthanides to better understand the movement of lanthanide fission products and fuel cladding chemical interaction. Lastly, the deep learning model is versatile and applicable to the micro-structure segmentation of similar materials.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Segmentation method comparison for residual fiber length measurement across tiled microscopy images

Fiber length distribution (FLD), in part, governs mechanical properties in discontinuous fiber composites, yet manual measurement methods limit the high-throughput characterization needed for materials design optimization. This study compares deep learning segmentation approaches for automated FLD measurement in large-field microscopy, evaluating how method choice affects the microstructural descriptors used in structure-property-processing relationships. A critical challenge is that high-resolution microscopy images (10,000×10,000 pixels) must be tiled for deep learning analysis, fragmenting fibers at boundaries. We demonstrate that segmentation method proves crucial for measurement accuracy. For example, instance segmentation with Slicing Aided Hyper Inference (SAHI) preserves individual fiber integrity across tiles while semantic segmentation prioritizes speed. Comparing against manual measurement of extracted carbon fibers, YOLOv11-SAHI matched manual ground truth (238 μm weighted mean) with 40x speedup (4.5 vs 167 minutes per image). U-Net provides rapid quantification although it is at the cost of reduced accuracy due only reliably measuring stand-alone fibers. Our comparative analysis reveals that instance segmentation with SAHI better preserves length measurements while semantic segmentation prioritizes speed, providing empirical guidance for method selection. The characterization provides essential inputs for mechanical property prediction models and inverse design workflows, accelerating composite materials development cycles.

Additive manufacturing↗
DOE OSTI2025

Crack Identification and Characterization in Deformed Nb3Sn Rutherford Cable Stacks Using Machine Learning

An investigation of instance segmentation of cracks in Nb3Sn 4-stack 40-strand Rutherford cables using machine learning is presented. Three samples were uniaxially and biaxially loaded before metallographic inspections were performed. The Mask R-CNN model was used in the Detectron2 framework with pre-trained weights but fine-tuned to detect and segment cracks. The model detected cracks with bounding box and mask average precisions (AP) of 42.8 and 27.9, respectively, and was used for instance segmentation of all cracks in the three samples. More cracks were found in the sample pre-loaded along the z-axis (i.e., along the cable length). Pre-loading along the x-axis (i.e., on the cables edges) reduced the number of cracks and changed the crack orientation distribution, away from being highly aligned with the y-axis (i.e., normal to the cables broad faces), i.e., the direction with the highest applied load. Fine-tuning of the Segment Anything Model (SAM) was also studied but performed poorly without human-provided prompts. However, the zero-shot capability of SAM showed high promises to accelerate the image annotation process for applications beyond this study.

Croteau, Jean-Francois↗
DOE OSTI2025

Concavity-based local erosion and sphere-size-based local dilation applied to lithium-ion battery electrode microstructures for particle identification

Performance metrics of lithium-ion batteries can be extracted from the analysis of electrode microstructures nanoscale imaging. The characterization workflow can involve a challenging particle identification, or instance segmentation, step. In this work, we propose a new identification method based on an original transformation: a sphere-size-based local dilation followed by a concavity-based local erosion, that is local morphology closing. The new transformation is much more efficient than the global morphology closing, with correct identification achieved with only 1.7 % dilation volume and 2.6 % erosion volume on a test geometry, compared to 39.2 % and more than 50 %, respectively, with its global counterpart. The new method has been then benchmarked versus other identification algorithms (watershed and pseudo coulomb repulsive field) on a real electrode microstructure with equal or better segmentation achieved.

25 ENERGY STORAGE↗
DOE OSTI2025

A comparative analysis of YOLOv8 and U-Net image segmentation approaches for transmission electron micrographs of polycrystalline thin films

Metallic thin films offer a platform to experimentally study the dynamics of microstructural evolution, but the required transmission electron microscopy (TEM)-based imaging generates complex images that are challenging to segment and quantify. This work provides a comparative analysis of a new YOLOv8 model and an established U-Net model for bright-field TEM images of polycrystals, employing a framework leveraging physical observables to evaluate performance against two hand-traced benchmark datasets. This methodology obviates the comparison of large, diversely structured, and manually labeled datasets that are required to assess performance on a per-image/per-pixel basis. It is found that the YOLOv8 model, adapted for real-time instance segmentation, has up to 43× faster inferencing (NVIDIA GeForce RTX 4090) compared to U-Net and reconstructs hand-traced grain size distributions (GSDs) with excellent fidelity, finding mean diameter within 3% for grains near an optimal magnification; for grains that deviate from the optimal pixel-diameter, the size of small- (large)-diameter grains is systematically over- (under)-estimated. This is partially mitigated by including scale-aware augmentations during training. Moreover, when the bias is corrected post-inference by a rigid shift in distribution, the YOLOv8 model reproduces ground truth GSDs with exceptional fidelity, with statistical tests indicating <5% probability that the distributions are distinct. Based on ground truth data, calibration curves pertaining to this shift can be constructed for a given model. This issue is not present in the U-Net model’s results, indicating that for quantitative measurements where the true size of objects is of interest, special procedures must be implemented for YOLO-based models.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Automated Bacterial Identification and Morphological Feature Analysis in Low‐Dose Cryo‐EM Using YOLOv11

Bacteria rapidly adapt to environmental cues through morphological and ultrastructural changes that correlate with physiology and behavior. Cryogenic transmission electron microscopy (cryo‐TEM) can capture these phenotypic changes in near‐native, vitrified states, but manual analysis of low‐dose micrographs is labor intensive and limits throughput. Here, we present an end‐to‐end workflow that combines low‐dose cryo‐TEM imaging with a YOLOv11‐based instance‐segmentation model to automatically identify bacteria and quantify key structural features directly from the micrographs. This workflow enables (i) robust bacterial localization and counting from low‐magnification atlas/montage images, (ii) automated measurements of cell‐envelope (outer–inner membrane) thickness and anisotropy from higher‐magnification views, and (iii) detection and quantification of bacteria–flagella interactions, including overlap length and curvature metrics for interacting versus noninteracting flagella. Using Pantoea sp. YR343 grown under distinct media conditions, we show that the automated measurements agree with manual annotations while substantially reducing analysis time. Together, these tools provide a practical framework for scalable bacterial identification and quantitative phenotyping in low‐dose cryo‐TEM datasets and establish a foundation for extending cryo‐TEM image analysis toward higher‐throughput studies of microbial heterogeneity and biointerfaces.

YOLOv11↗
DOE OSTI2024

Semi-automatic image annotation using 3D LiDAR projections and depth camera data

Efficient image annotation is necessary to utilize deep learning object recognition neural networks in nuclear safeguards, such as for the detection and localization of target objects like nuclear material containers (NMCs). This capability can help automate the inventory accounting of different types of NMCs within nuclear storage facilities. The conventional manual annotation process is labor-intensive and time-consuming, hindering the rapid deployment of deep learning models for NMC identifications. This paper introduces a novel semi-automatic method for annotating 2D images of nuclear material containers (NMCs) by combining 3D light detection and ranging (LiDAR) data with color and depth camera images collected from a handheld scan system. The annotation pipeline involves an operator manually marking new target objects on a LiDAR-generated map, and projecting these 3D locations to images, thereby automatically creating annotations from the projections. The semi-automatic approach significantly reduces manual efforts and the expertise in image annotation that is required to perform the task, allowing deep learning models to be trained on-site within a few hours. The paper compares the performance of models trained on datasets annotated through various methods, including semi-automatic, manual, and commercial annotation services. The evaluation demonstrates that the semi-automatic annotation method achieves comparable or superior results, with a mean average precision (mAP) above 0.9, showcasing its efficiency in training object recognition models. Additionally, the paper explores the application of the proposed method to instance segmentation, achieving promising results in detecting multiple types of NMCs in various formations.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2023

A deep learning model for automatic analysis of cavities in irradiated materials

Transmission electron microscopy (TEM) is a commonly used technique in materials science for defect investigation. Quantitative analysis of defects is important for understanding the properties of a material, but manual analysis of TEM micrographs can be time-consuming and prone to error, especially when the defects have irregular shapes rather than spherical shapes. Many existing methods or deep learning models do not handle a wide range of sizes for the same object type within a single image. In this work, we present a framework that enables users to train an instance segmentation model called Mask R- CNN on any microstructure dataset, perform multi-detection on the same image at different scales, and obtain properties (e.g., size, area) of the objects based on the desired shape (e.g., circle, ellipse, rectangle). Additionally, we have developed a parallel detection module that uses multiple GPUs to increase the efficiency of the object detection process. We demonstrate the capabilities of our framework using a set of TEM images of cavities with different shapes, size distributions, and background contrasts. Finally, we show that the performance of our model in terms of density, size, and swelling of the cavities is comparable to the human average and that our model achieves the highest recall value compared to existing methods due to the use of image multi-rescaling.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Deep learning for morphological identification of extended radio galaxies using weak labels

Abstract The present work discusses the use of a weakly-supervised deep learning algorithm that reduces the cost of labelling pixel-level masks for complex radio galaxies with multiple components. The algorithm is trained on weak class-level labels of radio galaxies to get class activation maps (CAMs). The CAMs are further refined using an inter-pixel relations network (IRNet) to get instance segmentation masks over radio galaxies and the positions of their infrared hosts. We use data from the Australian Square Kilometre Array Pathfinder (ASKAP) telescope, specifically the Evolutionary Map of the Universe (EMU) Pilot Survey, which covered a sky area of 270 square degrees with an RMS sensitivity of 25–35 $\mu$ Jy beam $^{-1}$ . We demonstrate that weakly-supervised deep learning algorithms can achieve high accuracy in predicting pixel-level information, including masks for the extended radio emission encapsulating all galaxy components and the positions of the infrared host galaxies. We evaluate the performance of our method using mean Average Precision (mAP) across multiple classes at a standard intersection over union (IoU) threshold of 0.5. We show that the model achieves a mAP $_{50}$ of 67.5% and 76.8% for radio masks and infrared host positions, respectively. The network architecture can be found at the following link: https://github.com/Nikhel1/Gal-CAM

Astronomy & Astrophysics↗
DOE OSTI2023

Autonomous Molecular Structure Imaging with High-Resolution Atomic Force Microscopy for Molecular Mixture Discovery

Due to its single-molecule sensitivity, high-resolution atomic force microscopy (HR-AFM) has proved to be a valuable and uniquely advantageous tool to study complex molecular mixtures, which hold promise for developing clean energy and achieving environmental sustainability. However, significant challenges remain to achieve the full potential of the sophisticated and time-consuming experiments. Automation combined with machine learning (ML) and artificial intelligence (AI) is key to overcoming these challenges. Here we present Auto-HR-AFM, an AI tool to automatically collect HR-AFM images of petroleum-based mixtures. In this study, we trained an instance segmentation model to teach Auto-HR-AFM how to recognize features in HR-AFM images. Auto-HR-AFM then uses that information to optimize the imaging by adjusting the probe-molecule distance for each molecule in the run. Auto-HR-AFM is the initial tool that will lead to fully automated scanning probe microscopy (SPM) experiments, from start to finish. This automation will allow SPM to become a mainstream characterization technique for complex mixtures, an otherwise unattainable target.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Towards Geospatial Knowledge Graph Infused Neuro-Symbolic AI for Remote Sensing Scene Understanding

Deep learning has proven its effectiveness in numerous tasks for remote sensing scene understanding. However there is an increasing interest to explore fusion of domain-specific background information to the deep neural network to further improve its performance. Remote sensing researchers are also working towards developing models that generalize and adapt to multiple applications. Generalization challenges coupled with the scarcity of large corpora of high-quality noise-free labelled data, have together fueled an interest for leveraging background information. Knowledge graphs serve as excellent choice to represent domain-specific information in a structured, standardized and extensible manner. Integrating symbolic knowledge representations in the form of Knowledge Graph Embedding (KGE) to perform neuro-symbolic reasoning is an emerging research direction promising significant impacts. This vision paper seeks to position ideas and provoke early thoughts toward advancing neuro-symbolic artificial intelligence in the context of geospatial challenges. Specifically, it conceptualizes and elaborates on an architecture for infusing geospatial knowledge from knowledge graph in a deep neural network pipeline. As guiding case studies - land-use land-cover classification, object detection and instance segmentation can benefit from infusing spatio-contextual information with remote sensing imagery. The discussion further reflects on and articulates the challenges and explainable AI opportunities anticipated when scaling and maintaining large-scale geospatial knowledge graphs.

Potnis, Abhishek↗