Search NASA⌕ Search

SEARCH · Search NASA

Results for “Image segmentation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Batch Active Learning for Multispectral and Hyperspectral Image Segmentation Using Similarity Graphs

Abstract Graph learning, when used as a semi-supervised learning (SSL) method, performs well for classification tasks with a low label rate. We provide a graph-based batch active learning pipeline for pixel/patch neighborhood multi- or hyperspectral image segmentation. Our batch active learning approach selects a collection of unlabeled pixels that satisfy a graph local maximum constraint for the active learning acquisition function that determines the relative importance of each pixel to the classification. This work builds on recent advances in the design of novel active learning acquisition functions (e.g., the Model Change approach in arXiv:2110.07739) while adding important further developments including patch-neighborhood image analysis and batch active learning methods to further increase the accuracy and greatly increase the computational efficiency of these methods. In addition to improvements in the accuracy, our approach can greatly reduce the number of labeled pixels needed to achieve the same level of the accuracy based on randomly selected labeled pixels.

97 MATHEMATICS AND COMPUTING↗

CryoSegNet: accurate cryo-EM protein particle picking by integrating the foundational AI image segmentation model and attention-gated U-Net

Picking protein particles in cryo-electron microscopy (cryo-EM) micrographs is a crucial step in the cryo-EM-based structure determination. However, existing methods trained on a limited amount of cryo-EM data still cannot accurately pick protein particles from noisy cryo-EM images. The general foundational artificial intelligence–based image segmentation model such as Meta’s Segment Anything Model (SAM) cannot segment protein particles well because their training data do not include cryo-EM images. Here, we present a novel approach (CryoSegNet) of integrating an attention-gated U-shape network (U-Net) specially designed and trained for cryo-EM particle picking and the SAM. The U-Net is first trained on a large cryo-EM image dataset and then used to generate input from original cryo-EM images for SAM to make particle pickings. CryoSegNet shows both high precision and recall in segmenting protein particles from cryo-EM micrographs, irrespective of protein type, shape and size. On several independent datasets of various protein types, CryoSegNet outperforms two top machine learning particle pickers crYOLO and Topaz as well as SAM itself. The average resolution of density maps reconstructed from the particles picked by CryoSegNet is 3.33 Å, 7% better than 3.58 Å of Topaz and 14% better than 3.87 Å of crYOLO. It is publicly available at https://github.com/jianlin-cheng/CryoSegNet

59 BASIC BIOLOGICAL SCIENCES↗

ExtremeMETA: High-speed Lightweight Image Segmentation Model by Remodeling Multi-channel Metamaterial Imagers

Deep neural networks (DNNs) have heavily relied on traditional computational units, such as CPUs and GPUs. However, this conventional approach brings significant computational burden, latency issues, and high power consumption, limiting their effectiveness. This has sparked the need for lightweight networks such as ExtremeC3Net. Meanwhile, there have been notable advancements in optical computational units, particularly with metamaterials, offering the exciting prospect of energy-efficient neural networks operating at the speed of light. Yet, the digital design of metamaterial neural networks (MNNs) faces precision, noise, and bandwidth challenges, limiting their application to intuitive tasks and low-resolution images. In this study, we proposed a large kernel lightweight segmentation model, ExtremeMETA. Based on ExtremeC3Net, our proposed model, ExtremeMETA maximized the ability of the first convolution layer by exploring a larger convolution kernel and multiple processing paths. With the large kernel convolution model, we extended the optic neural network application boundary to the segmentation task. To further lighten the computation burden of the digital processing part, a set of model compression methods was applied to improve model efficiency in the inference stage. The experimental results on three publicly available datasets demonstrated that the optimized efficient design improved segmentation performance from 92.45 to 95.97 on mIoU while reducing computational FLOPs from 461.07 MMacs to 166.03 MMacs. The large kernel lightweight model ExtremeMETA showcased the hybrid design’s ability on complex tasks.

large convolution kernel↗

Parallel Markov Random Fields for Image Segmentation (PMRF-IS) v1.0

This software performs multi-label segmentation of 2D/3D images. It is highly accurate, results rely on use of an Markov random field (MRF) formulation. It runs in both shared and distributed memory parallel modes. MRF algorithms are powerful tools in image analysis to explore contextual information of data.

Heinemann, Colleen↗

SU-E-I-71: Quality Assessment of Surrogate Metrics in Multi-Atlas-Based Image Segmentation

Purpose: With the ever-growing data of heterogeneous quality, relevance assessment of atlases becomes increasingly critical for multi-atlas-based image segmentation. However, there is no universally recognized best relevance metric and even a standard to compare amongst candidates remains elusive. This study, for the first time, designs a quantification to assess relevance metrics’ quality, based on a novel perspective of the metric as surrogate for inferring the inaccessible oracle geometric agreement. Methods: We first develop an inference model to relate surrogate metrics in image space to the underlying oracle relevance metric in segmentation label space, with a monotonically non-decreasing function subject to random perturbations. Subsequently, we investigate model parameters to reveal key contributing factors to surrogates’ ability in prognosticating the oracle relevance value, for the specific task of atlas selection. Finally, we design an effective contract-to-noise ratio (eCNR) to quantify surrogates’ quality based on insights from these analyses and empirical observations. Results: The inference model was specialized to a linear function with normally distributed perturbations, with surrogate metric exemplified by several widely-used image similarity metrics, i.e., MSD/NCC/(N)MI. Surrogates’ behaviors in selecting the most relevant atlases were assessed under varying eCNR, showing that surrogates with high eCNR dominated those with low eCNR in retaining the most relevant atlases. In an end-to-end validation, NCC/(N)MI with eCNR of 0.12 compared to MSD with eCNR of 0.10 resulted in statistically better segmentation with mean DSC of about 0.85 and the first and third quartiles of (0.83, 0.89), compared to MSD with mean DSC of 0.84 and the first and third quartiles of (0.81, 0.89). Conclusion: The designed eCNR is capable of characterizing surrogate metrics’ quality in prognosticating the oracle relevance value. It has been demonstrated to be correlated with the performance of relevant atlas selection and ultimate label fusion.

60 APPLIED LIFE SCIENCES↗

Reinforcement Learning as a Parsimonious Alternative to Prediction Cascades: A Case Study on Image Segmentation

Deep learning architectures have achieved state-of-the-art (SOTA) performance on computer vision tasks such as object detection and image segmentation. This may be attributed to the use of over-parameterized, monolithic deep learning architectures executed on large datasets. Although such large architectures lead to increased accuracy, this is usually accompanied by a larger increase in computation and memory requirements during inference. While this is a non-issue in traditional machine learning (ML) pipelines, the recent confluence of machine learning and fields like the Internet of Things (IoT) has rendered such large architectures infeasible for execution in low-resource settings. For some datasets, large monolithic pipelines may be overkill for simpler inputs. To address this problem, previous efforts have proposed decision cascades where inputs are passed through models of increasing complexity until the desired performance is achieved. However, we argue that cascaded prediction leads to sub-optimal throughput and increased computational cost due to wasteful intermediate computations. To address this, we propose PaSeR (Parsimonious Segmentation with Reinforcement Learning) a non-cascading, cost-aware learning pipeline as an efficient alternative to cascaded decision architectures. Through experimental evaluation on both real-world and standard datasets, we demonstrate that PaSeR achieves better accuracy while minimizing computational cost relative to cascaded models. Further, we introduce a new metric IoU/GigaFlop to evaluate the balance between cost and performance. On the real-world task of battery material phase segmentation, PaSeR yields 179% improvement over SOTA MatPhase model and a 196% improvement over IDK Cascades under the IoU/GigaFlop metric. We also demonstrate PaSeR’s adaptability to complementary models trained on a noisy MNIST dataset, where it outperforms all baselines on IoU/GigaFlop by an average of 44%.

Srikshan, Bharat↗

Image segmentation for analyzing galaxy-galaxy strong lensing systems

The goal of this Letter is to develop a machine learning model to analyze the main gravitational lens and detect dark substructure (subhalos) within simulated images of strongly lensed galaxies. Using the technique of image segmentation, we turn the task of identifying subhalos into a classification problem, where we label each pixel in an image as coming from the main lens, a subhalo within a binned mass range, or neither. Our network is only trained on images with a single smooth lens and either zero or one subhalo near the Einstein ring. On an independent test set with lenses with large ellipticities, quadrupole and octopole moments, and for source apparent magnitudes between 17–25, the area of the main lens is recovered accurately. On average, only 1.3% of the true area is missed and 1.2% of the true area is added to another part of the lens. In addition, subhalos as light as 10 8.5 M ⊙ can be detected if they lie in bright pixels along the Einstein ring. Furthermore, the model is able to generalize to new contexts it has not been trained on, such as locating multiple subhalos with varying masses or more than one large smooth lens.

79 ASTRONOMY AND ASTROPHYSICS↗

SU-E-J-142: Performance Study of Automatic Image-Segmentation Algorithms in Motion Tracking Via MR-IGRT

Purpose: Evaluate commonly used segmentation algorithms on a commercially available real-time MR image guided radiotherapy (MR-IGRT) system (ViewRay), compare the strengths and weaknesses of each method, with the purpose of improving motion tracking for more accurate radiotherapy. Methods: MR motion images of bladder, kidney, duodenum, and liver tumor were acquired for three patients using a commercial on-board MR imaging system and an imaging protocol used during MR-IGRT. A series of 40 frames were selected for each case to cover at least 3 respiratory cycles. Thresholding, Canny edge detection, fuzzy k-means (FKM), k-harmonic means (KHM), and reaction-diffusion level set evolution (RD-LSE), along with the ViewRay treatment planning and delivery system (TPDS) were included in the comparisons. To evaluate the segmentation results, an expert manual contouring of the organs or tumor from a physician was used as a ground-truth. Metrics value of sensitivity, specificity, Jaccard similarity, and Dice coefficient were computed for comparison. Results: In the segmentation of single image frame, all methods successfully segmented the bladder and kidney, but only FKM, KHM and TPDS were able to segment the liver tumor and the duodenum. For segmenting motion image series, the TPDS method had the highest sensitivity, Jarccard, and Dice coefficients in segmenting bladder and kidney, while FKM and KHM had a slightly higher specificity. A similar pattern was observed when segmenting the liver tumor and the duodenum. The Canny method is not suitable for consistently segmenting motion frames in an automated process, while thresholding and RD-LSE cannot consistently segment a liver tumor and the duodenum. Conclusion: The study compared six different segmentation methods and showed the effectiveness of the ViewRay TPDS algorithm in segmenting motion images during MR-IGRT. Future studies include a selection of conformal segmentation methods based on image/organ-specific information, different filtering methods and their influences on the segmentation results. Parag Parikh receives research grant from ViewRay. Sasa Mutic has consulting and research agreements with ViewRay. Yanle Hu receives travel reimbursement from ViewRay. Iwan Kawrakow and James Dempsey are ViewRay employees.

60 APPLIED LIFE SCIENCES↗

Volume of Lytic Vertebral Body Metastatic Disease Quantified Using Computed Tomography–Based Image Segmentation Predicts Fracture Risk After Spine Stereotactic Body Radiation Therapy

Purpose: To determine a threshold of vertebral body (VB) osteolytic or osteoblastic tumor involvement that would predict vertebral compression fracture (VCF) risk after stereotactic body radiation therapy (SBRT), using volumetric image-segmentation software. Methods and Materials: A computational semiautomated skeletal metastasis segmentation process refined in our laboratory was applied to the pretreatment planning CT scan of 100 vertebral segments in 55 patients treated with spine SBRT. Each VB was segmented and the percentage of lytic and/or blastic disease by volume determined. Results: The cumulative incidence of VCF at 3 and 12 months was 14.1% and 17.3%, respectively. The median follow-up was 7.3 months (range, 0.6-67.6 months). In all, 56% of segments were determined lytic, 23% blastic, and 21% mixed, according to clinical radiologic determination. Within these 3 clinical cohorts, the segmentation-determined mean percentages of lytic and blastic tumor were 8.9% and 6.0%, 0.2% and 26.9%, and 3.4% and 15.8% by volume, respectively. On the basis of the entire cohort (n=100), a significant association was observed for the osteolytic percentage measures and the occurrence of VCF (P<.001) but not for the osteoblastic measures. The most significant lytic disease threshold was observed at ≥11.6% (odds ratio 37.4, 95% confidence interval 9.4-148.9). On multivariable analysis, ≥11.6% lytic disease (P<.001), baseline VCF (P<.001), and SBRT with ≥20 Gy per fraction (P=.014) were predictive. Conclusions: Pretreatment lytic VB disease volumetric measures, independent of the blastic component, predict for SBRT-induced VCF. Larger-scale trials evaluating our software are planned to validate the results.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Machine learning-based multidomain processing for texture-based image segmentation and analysis

Atomic and molecular resolved atomic force microscopy (AFM) images offer unique insights into materials' properties such as local ordering, molecular orientation, and topological defects, which can be used to pinpoint physical and chemical interactions occurring at the surface. Utilizing machine learning for extracting underlying physical parameters increases the throughput of AFM data processing and eliminates inconsistencies intrinsic to manual image analysis, thus enabling the creation of reliable frameworks for qualitative and quantitative evaluation of experimental data. Here, we present a robust and scalable approach to the segmentation of AFM images based on flexible pre-selected classification criteria. The usage of supervised learning and feature extraction allows us to retain the consideration of specific problem-dependent features (such as types of periodical structures observed in the images and the associated numerical parameters: spacing, orientation, etc.). We highlight the applicability of this approach for the segmentation of molecular resolved AFM images based on the crystal orientation of the observed domains, automated selection of boundaries, and collection of relevant statistics. Finally, we outline a general strategy for machine learning-enabled analysis of nanoscale systems exhibiting periodic order that could be applied to any analytical imaging technique

36 MATERIALS SCIENCE↗

Deep Learning Image Segmentation for Atmospheric Rivers

Abstract The identification of atmospheric rivers (ARs) is crucial for weather and climate predictions as they are often associated with severe storm systems and extreme precipitation, which can cause large impacts on society. This study presents a deep learning model, termed ARDetect, for image segmentation of ARs using ERA5 data from 1960 to 2020 with labels obtained from the TempestExtremes tracking algorithm. ARDetect is a convolutional neural network (CNN)-based U-Net model, with its structure having been optimized using automatic hyperparameter tuning. Inputs to ARDetect were selected to be the integrated water vapor transport (IVT) and total column water (TCW) fields, as well as the AR mask from TempestExtremes from the previous time step to the one being considered. ARDetect achieved a mean intersection-over-union (mIoU) rate of 89.04% for ARs, indicating its high accuracy in identifying these weather patterns and a superior performance than most deep learning–based models for AR detection. In addition, ARDetect can be executed faster than the TempestExtremes method (seconds vs minutes) for the same period. This provides a significant benefit for online AR detection, especially for high-resolution global models. An ensemble of 10 models, each trained on the same dataset but having different starting weights, was used to further improve on the performance produced by ARDetect, thus demonstrating the importance of model diversity in improving performance. ARDetect provides an effective and fast deep learning–based model for researchers and weather forecasters to better detect and understand ARs, which have significant impacts on weather-related events such as floods and droughts.

Galea, Daniel↗

3D-SIFT-Flow for atlas-based CT liver image segmentation

Purpose: In this paper, the authors proposed a new 3D registration algorithm, 3D-scale invariant feature transform (SIFT)-Flow, for multiatlas-based liver segmentation in computed tomography (CT) images. Methods: In the registration work, the authors developed a new registration method that takes advantage of dense correspondence using the informative and robust SIFT feature. The authors computed the dense SIFT features for the source image and the target image and designed an objective function to obtain the correspondence between these two images. Labeling of the source image was then mapped to the target image according to the former correspondence, resulting in accurate segmentation. In the fusion work, the 2D-based nonparametric label transfer method was extended to 3D for fusing the registered 3D atlases. Results: Compared with existing registration algorithms, 3D-SIFT-Flow has its particular advantage in matching anatomical structures (such as the liver) that observe large variation/deformation. The authors observed consistent improvement over widely adopted state-of-the-art registration methods such as ELASTIX, ANTS, and multiatlas fusion methods such as joint label fusion. Experimental results of liver segmentation on the MICCAI 2007 Grand Challenge are encouraging, e.g., Dice overlap ratio 96.27% ± 0.96% by our method compared with the previous state-of-the-art result of 94.90% ± 2.86%. Conclusions: Experimental results show that 3D-SIFT-Flow is robust for segmenting the liver from CT images, which has large tissue deformation and blurry boundary, and 3D label transfer is effective and efficient for improving the registration accuracy.

60 APPLIED LIFE SCIENCES↗

Designing Antifouling and Antimicrobial Interfaces: Structural Characterization using CryoEM, Automated Microscopy, and AI Image Segmentation

The design of functionalized surfaces for interactions with biological systems is critical across sectors such as healthcare, energy, and agriculture. Tailoring materials for specific applications, such as antifouling and antimicrobial surfaces, demands a comprehensive understanding of topology and chemistry across multiple length and time scales on both biological and materials systems. This work presents the development and characterization of nanostructured surfaces with controlled topographies and chemistries that enhance bacterial membrane disruption, reduce biofilm formation, and improve antimicrobial and antifouling capabilities. Two specific use cases will be presented - the use of cellulose nanocrystals (CNCs) for bacterial growth inhibition and the development of antifouling surfaces to prevent protein and bacterial adsorption [1-4]. By leveraging large language models (LLMs) for image segmentation and training [5], we enable automated analysis of terabyte-scale cryogenic electron microscopy (cryoEM) datasets. This analysis provides statistical insights into the biotic/abiotic interface and facilitates automated electron microscopy experiments to mitigate time and dose. The integration of cryogenic electron tomography (cryoET) and cryogenic focused ion beam (cryoFIB) milling enables high-resolution, near-native-state imaging and 3D reconstructions of bio/material interfaces [6]. Orthogonal characterization techniques and computational modeling further enhances our understanding, offering a robust platform for the design and optimization of next-generation functional surfaces [7].

Williams, Alexis [ORNL] (ORCID:0000000252835822)↗

Adaptive Patching for High-resolution Image Segmentation with Transformers

Attention-based models are proliferating in the space of image analytics, including segmentation. The standard method of feeding images to transformer encoders is to divide the images into patches and then feed the patches to the model as a linear sequence of tokens. For high-resolution images, e.g. microscopic pathology images, the quadratic compute and memory cost prohibits the use of an attention-based model, if we are to use smaller patch sizes that are favorable in segmentation. The solution is to either use custom complex multi-resolution models or approximate attention schemes. We take inspiration from Adapative Mesh Refinement (AMR) methods in HPC by adaptively patching the images, as a pre-processing step, based on the image details to reduce the number of patches being fed to the model, by orders of magnitude. This method has a negligible overhead, and works seamlessly with any attention-based model, i.e. it is a pre-processing step that can be adopted by any attention-based model without friction. We demonstrate superior segmentation quality over SoTA segmentation models for realworld pathology datasets while gaining a geomean speedup of 6.9× for resolutions up to 64K2, on up to 2, 048 GPUs.

Zhang, Enzhi↗

Vision 20/20: Perspectives on automated image segmentation for radiotherapy

Due to rapid advances in radiation therapy (RT), especially image guidance and treatment adaptation, a fast and accurate segmentation of medical images is a very important part of the treatment. Manual delineation of target volumes and organs at risk is still the standard routine for most clinics, even though it is time consuming and prone to intra- and interobserver variations. Automated segmentation methods seek to reduce delineation workload and unify the organ boundary definition. In this paper, the authors review the current autosegmentation methods particularly relevant for applications in RT. The authors outline the methods’ strengths and limitations and propose strategies that could lead to wider acceptance of autosegmentation in routine clinical practice. The authors conclude that currently, autosegmentation technology in RT planning is an efficient tool for the clinicians to provide them with a good starting point for review and adjustment. Modern hardware platforms including GPUs allow most of the autosegmentation tasks to be done in a range of a few minutes. In the nearest future, improvements in CT-based autosegmentation tools will be achieved through standardization of imaging and contouring protocols. In the longer term, the authors expect a wider use of multimodality approaches and better understanding of correlation of imaging with biology and pathology.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Algorithm sensitivity analysis and parameter tuning for tissue image segmentation pipelines

Motivation: Sensitivity analysis and parameter tuning are important processes in large-scale image analysis. They are very costly because the image analysis workflows are required to be executed several times to systematically correlate output variations with parameter changes or to tune parameters. An integrated solution with minimum user interaction that uses effective methodologies and high performance computing is required to scale these studies to large imaging datasets and expensive analysis workflows. Results: The experiments with two segmentation workflows show that the proposed approach can (i) quickly identify and prune parameters that are non-influential; (ii) search a small fraction (about 100 points) of the parameter search space with billions to trillions of points and improve the quality of segmentation results (Dice and Jaccard metrics) by as much as 1.42 x compared to the results from the default parameters; (iii) attain good scalability on a high performance cluster with several effective optimizations. Conclusions: Our work demonstrates the feasibility of performing sensitivity analyses, parameter studies and auto-tuning with large datasets. The proposed framework can enable the quantification of error estimations and output variations in image segmentation pipelines.

Biochemistry & Molecular Biology↗

Generalizable Image Segmentation for Microstructure Characterization Through Integrated SEM and EBSD Analysis

We demonstrate generalizable semantic segmentation using minimal ground truth data. Correlated scanning electron microscopy (SEM) images and electron backscatter diffraction (EBSD) measurements of frictionstir processed 316L stainless steel plates were used to train deep learning models for grain boundary segmentation. Secondary electron (SE) imaging taken at an accelerating voltage of 10 keV correlated to EBSD-derived grain boundaries produced the best performing model. Notably, an ensemble of three models trained on a single SE image produced accurate segmentation over a series of BSE images of samples manufactured under different processing parameters, with a resultant mean absolute error in grain size of 0.34 µm. The striking generalizability of the models likely results from the similar escape depths of the SE training input and the EBSD training output and the reduced probability of dislocation artifacts appearing in the image. This finding highlights the importance of considering the physical principles behind imaging in the development of robust segmentation models for microstructure characterization.

Taufique, Mohammad Fuad Nur↗