Search NASA⌕ Search

SEARCH · Search NASA

Results for “3D instance segmentation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

YOLO2U-Net: Detection-guided 3D instance segmentation for microscopy

Microscopy imaging techniques are instrumental for characterization and analysis of biological structures. As these techniques typically render 3D visualization of cells by stacking 2D projections, issues such as out-of-plane excitation and low resolution in the z-axis may pose challenges (even for human experts) to detect individual cells in 3D volumes as these non-overlapping cells may appear as overlapping. In this paper a comprehensive method for accurate 3D instance segmentation of cells in the brain tissue is introduced. The proposed method combines the 2D YOLO detection method with a multi-view fusion algorithm to construct a 3D localization of the cells. Next, the 3D bounding boxes along with the data volume are input to a 3D U-Net network that is designed to segment the primary cell in each 3D bounding box, and in turn, to carry out instance segmentation of cells in the entire volume. The promising performance of the proposed method is shown in comparison with current deep learning-based 3D instance segmentation methods.

3D instance segmentation↗

An intensity-based post-processing tool for 3D instance segmentation of organelles in soft X-ray tomograms

Investigating the 3D structures and rearrangements of organelles within a single cell is critical for better characterizing cellular function. Imaging approaches such as soft X-ray tomography have been widely applied to reveal a complex subcellular organization involving multiple inter-organelle interactions. However, 3D segmentation of organelle instances has been challenging despite its importance in organelle characterization. Here we propose an intensity-based post-processing tool to identify and separate organelle instances. Our tool separates sphere-like (insulin vesicle) and columnar-shaped organelle instances (mitochondrion) based on the intensity of raw tomograms, semantic segmentation masks, and organelle morphology. We validate our tool using synthetic tomograms of organelles and experimental tomograms of pancreatic β -cells to separate insulin vesicle and mitochondria instances. As compared to the commonly used connected regions labeling, watershed, and watershed + Gaussian filter methods, our tool results in improved accuracy in identifying organelles in the synthetic tomograms and an improved description of organelle structures in β -cell tomograms. In addition, under different experimental treatment conditions, significant changes in volumes and intensities of both insulin vesicle and mitochondrion are observed in our instance results, revealing their potential roles in maintaining normal β -cell function. Our tool is expected to be applicable for improving the instance segmentation of other images obtained from different cell types using multiple imaging modalities.

59 BASIC BIOLOGICAL SCIENCES↗

PlantSegNet: 3D point cloud instance segmentation of nearby plant organs with identical semantics

In this study, we introduce PlantSegNet, a novel neural network model for instance segmentation of nearby objects with similar geometric structures. Our work addresses the challenges of instance segmentation of plant point clouds, including the difficulty of annotating and labeling point clouds, the loss of local structural information in neural network components, and the generation of large numbers of incorrect small clusters due to poor choices of the loss function. One of the key contributions of our approach is a digital twin of sorghum, i.e., a procedural sorghum model, which was used to generate point clouds of sorghum fields. This allowed us to create a large-scale, annotated, synthetic dataset of sorghum plants that we used to train our PlantSegNet model. We demonstrated the effectiveness of our method in segmenting instances of sorghum leaves grown in outdoor field settings. To the best of our knowledge, this is the first study to address this specific instance segmentation problem for plants grown in such a setting. We compared our proposed method with other state-of-the-art methods for indoor settings, including SGPN and TreePartNet, on both synthetic and real data. Furthermore, our results show that PlantSegNet outperforms these methods regarding accuracy, robustness, and efficiency.

97 MATHEMATICS AND COMPUTING↗

SwinCell: a 3D transformer and flow-based framework for improved cell segmentation

Segmentation of three-dimensional (3D) cellular images is fundamental for studying and understanding cell structure and function. However, 3D cellular segmentation is challenging, particularly for dense cells and tissues. This challenge arises mainly from the complex contextual information within 3D images, anisotropic properties, and the sensitivity to internal cellular structures, which often lead to incorrect segmentation. In this work, we introduce SwinCell, a 3D transformer-based framework that leverages Swin-transformer to predict flow and differentiate individual cell instances. We demonstrate SwinCell’s utility in the segmentation of nuclei, colon tissue cells, and densely cultured cells. SwinCell strikes a balance between maintaining detailed local feature recognition and understanding broader contextual information. Through extensive testing with both public and in-house 3D cell imaging datasets, SwinCell shows utility in segmenting dense cells, making it a valuable tool for 3D segmentation in cellular analysis that could expedite research in cell biology and tissue engineering.

59 BASIC BIOLOGICAL SCIENCES↗

Progress Update on Segmentation of Neutron Tomographic Images, 2018

The present report documents the techniques developed to segment fast-neutron tomographic images of objects consisting of assemblies of machined parts with distinct boundaries. This report satisfies the fiscal year 2018 technical deliverable for project OR16-3DTomography-PD3Jb, “3D Tomography and Image Processing Using Fast Neutrons,” to report on the extension of geometric shape-finding algorithms to three dimensions. The project has two overall goals. The first of these goals is to extend associated-particle fast-neutron transmission and, particularly, induced-reaction tomographic imaging algorithms to three dimensions. This aspect of the project is beyond the scope of this report. The second goal is to automatically segment the resultant tomographic images into constituent parts and then extract information about the parts, such as the class of shape and potentially the shape parameters. This report describes progress for the image segmentation part of the project. Imaging techniques have been developed as high-confidence methods for confirming the presence and configuration of special nuclear materials (SNM). This high confidence is achieved at the cost of revealing considerable information that may be undesirable to share with the operator of the equipment. One potential way to make use of the high-confidence of imaging methods without revealing imaging data is to employ automated analysis that can extract meaningful attributes of the SNM without showing imaging data to the operator. An essential step of this automated analysis is the segmentation of the image into its constituent parts. For example, a three-dimensional (3D) fast-neutron tomographic image of an assembly containing uranium could be reconstructed and then segmented into discrete parts. The properties of a constituent part identified as uranium would be of interest and could be extracted. For instance, the volume and density of the part would together determine its mass. Moreover, its shape might be expected to fall into a particular geometric class, such as a cube, cylinder, or sphere, and the parameters of the shape (such as side length, height, or diameter) would be expected to fall in a given range. In such an analysis, automated identification of the boundaries and properties of the uranium part is an essential step. A key goal of the present work is to infer the boundaries of objects in fast-neutron tomographic images via the general method of image segmentation. Development of this capability can be broken into more manageable steps, including 1. operator-guided segmentation of two-dimensional (2D) fast-neutron tomographic images, 2. operator-guided segmentation of 3D fast-neutron tomographic images, 3. development of automated segmentation algorithms, and 4. extraction of shape parameters from constituent volumes in 3D. In the first year, the project team reported progress on the first step. Last year, progress on the second step, the extension into 3D, was reported with application to simulated data. The present report describes additional progress on the second step, and progress toward the third step, automated segmentation algorithms. In particular, a multiphase level set approach for image segmentation in three dimensions has been applied to measured 3D data for the first time. Activities related to the fourth step will also be reported, but it will focus on estimation of 3D volumes rather than shape parameters. In this report, the steps before image segmentation will be described, and the multiphase level sets approach to image segmentation will be briefly reviewed. This will be followed by a discussion of the issues encountered when applying the level set technique to measured 3D transmission images, and then a description of the inclusion of induced-reaction data will be provided. Finally, segmentation results for various measured target assemblies will be presented and the image segmentation portion of this project will be summarized and future directions will be discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Focused Ion Beam Tomography of Alloy 617 Corroded in Molten Chloride Salt

Materials qualification of reactor structural materials is a critical step in rapid implementation of advanced nuclear reactor technologies, particularly to assess the corrosion performance in these designs. Accelerated qualification of reactor structural materials requires incorporating powerful computational toolsets, such as phase field modelling in the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework, to predict the evolution of structural materials due to corrosion. Accordingly, computational toolsets will require experimental data generated at appropriate length scales to validate accuracy. Focused ion beam (FIB) provides a high degree of control over manipulation of materials for analytical purposes, including capturing data on the evolution in the microstructure and elemental composition of materials at the mesoscale, an appropriate length scale for phase field modelling of intergranular diffusion phenomena using the MOOSE framework. For instance, the FEI Helios G4 UX dual beam plasma FIB microscope at the Irradiated Materials Characterization Laboratory (IMCL) is capable of backscatter diffraction (EBSD) and energy-dispersive x-ray spectroscopy (EDS) documenting the evolution in the microstructure and elemental composition, respectively. The Helios can perform EDS and EBSD three-dimensionally (3D) using tomography, which is then combined using different software packages to visualize 3D volumes correlating elemental composition to microstructural data. The purpose of this investigation was to develop a streamlined characterization and data processing workflow for 3D tomography studies on the FEI Helios G4 plasma FIB. The investigation is segmented into three parts: 1) Optimizing the data collection workflow, 2) identifying appropriate data processing and visualization software (i.e. DREAM.3D, MIPAR, and VGStudioMax), and 3) establishing an infrastructure for public release. The optimization of the data collection workflow is in collaboration with members of the U220 department to setup formal training on the tomography operation of the G4, through ThermoFisher Scientific, and exploring DREAM.3D, MIPAR, and VGStudioMax data processing/visualization software packages. VGStudioMax currently demonstrates the most promise for future use. Optimization of the data collection and processing workflow is still ongoing. A collaboration with INL High Performance Computing (HPC) established an open-source license for expediting the public release of FIB tomography datasets through HPC. FIB tomography data generated by the G4 will provide comprehensive data for validating 3D phase field mesoscale modelling tools within the MOOSE framework for accelerated qualification of reactor structural materials.

Copeland-Johnson, Trishelle↗

Semi-automatic image annotation using 3D LiDAR projections and depth camera data

Efficient image annotation is necessary to utilize deep learning object recognition neural networks in nuclear safeguards, such as for the detection and localization of target objects like nuclear material containers (NMCs). This capability can help automate the inventory accounting of different types of NMCs within nuclear storage facilities. The conventional manual annotation process is labor-intensive and time-consuming, hindering the rapid deployment of deep learning models for NMC identifications. This paper introduces a novel semi-automatic method for annotating 2D images of nuclear material containers (NMCs) by combining 3D light detection and ranging (LiDAR) data with color and depth camera images collected from a handheld scan system. The annotation pipeline involves an operator manually marking new target objects on a LiDAR-generated map, and projecting these 3D locations to images, thereby automatically creating annotations from the projections. The semi-automatic approach significantly reduces manual efforts and the expertise in image annotation that is required to perform the task, allowing deep learning models to be trained on-site within a few hours. The paper compares the performance of models trained on datasets annotated through various methods, including semi-automatic, manual, and commercial annotation services. The evaluation demonstrates that the semi-automatic annotation method achieves comparable or superior results, with a mean average precision (mAP) above 0.9, showcasing its efficiency in training object recognition models. Additionally, the paper explores the application of the proposed method to instance segmentation, achieving promising results in detecting multiple types of NMCs in various formations.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗