Search NASA⌕ Search

SEARCH · Search NASA

Results for “U-net segmentation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

11 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Microstructure Segmentation with Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures, including VGG, Inception, and ResNet, were trained on over 100,000 labelled microscopy images from 54 classes. These pre-trained encoders were then embedded into multiple segmentation architectures including U-Net and DeepLabV3+ to evaluate segmentation performance on newly created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2 percent reduction in relative segmentation error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗
NASA NTRSDate not supplied

Quantum-Compatible Variational Segmentation for Image-to-Image Wildfire Detection Using Satellite Data

Wildfire occurrences have been increasing for the past decade, leaving devastating traces across the world. In the recent efforts, remote sensing and airborne missions have been utilized to better understand and manage wildfires. This has resulted in an exponential increase in volume of remote sensing data, which has pushed the need for intelligent automation of data extraction for wildfire studies. Machine learning offers accurate automation in detecting such natural anomalies and enable decision-makers to take actions in a timely manner. Recent advances in machine learning algorithms, namely probabilistic generative methods, allow researchers and decisionmakers to step beyond detection and study “what-if” scenarios for wildfire occurrences. Additionally, they offer better imitations to the stochastic behavior of nature, and wildfire events. However, optimizing the performance of these probabilistic generative models is a computationally expensive process, specially using digital computers. On the other hand, quantum computers have recently shown a promise to reduce computationally costly training of such models and provide performance improvements. There is a body of research investigating the potential for improved machine learning methods in which key operations are performed on a quantum computer. In this study, we propose a probabilistic image-toimage segmentation approach combining a very well-known segmentation method, U-NET, with a Conditional Variational Auto-Encoder (CVAE) to not only detect wildfires but also describe the stochasticity of the phenomenon and be capable of running “what-if” scenarios. Our proposed model is compatible with training on quantum computers, which results in a quantum-assisted image-to-image segmentation approach and can be used to benchmark the potential benefit of quantum computing over the classical one.

quantum↗
NASA NTRSDate not supplied

Quantum-Assisted Variational Segmentation for Image-to-Image Wildfire Detection Using Satellite Data

The quantum computing community has been searching for suitable applications to demonstrate the potential of near-term quantum devices. Quantum machine learning is a potential candidate, particularly using models that cannot be efficiently simulated with classical computers [1, 2]. This work focuses on a transition phase of quantum computers where the quantum machine learning model is still simulable classically but projected not to be simulable as the size of the model grows. Ultimately quantum computers may have advantages for high-dimensional real-world problems. Due to the limited number of qubits in current noisy intermediate-scale quantum (NISQ) devices, the direct application of quantum computers in high dimensional data is not feasible. To remedy this problem, an encoder-decoder architecture can be utilized. The encoder model would transform the high-dimensional data into a compact representation, to a level that small quantum computers can be used today (or in the near future), and the decoder would take the quantum processed outputs back to the high-dimensional space. Addressing the two challenges of quantum machine learning, this work investigates a hybrid supervised generative model with a quantum Ising Born machine embedded as the latent distribution. The model contains four main parts (Figure 1.a.): (1) a U-NET architecture responsible for learning segmentation flow, (2) a Prior network responsible for learning an encoded latent distribution of the input data, (3) a Born machine which represents the latent distribution, and (4) a Posterior network in charge of learning the joint encoded latent distribution of inputs and target data. The initial model, proposed by [3], is optimized by (1) maximizing the overlap of the prior and posterior latent distributions, and (2) minimizing the segmentation loss. The proposed model is designed to be investigated in a simulation environment applied to the real-world application of wildfire segmentation. Specifically, the model is designed to solve the patchy wildfire segmentations of Moderate Resolution Imaging Spectroradiometer (MODIS) by taking the MODIS observations and using Visible Infrared Imaging Radiometer Suite’s (VIIRS) consistent wildfire product as the target. The model solves patchy wildfire segmentations and provides insight into the epistemic errors sourced from model variation. The model utilizes the Born machine as a QUBO solver to represent the latent space as a Bernoulli distribution. The proposed configuration allows the variational segmentation model to leverage the true quantum probabilistic nature and derive a more expressive latent configuration, increasing the model performance in describing wildfire segmentations. The quantum probabilistic information of the Born machine is directly incorporated in the Kullback-Leibler divergence loss in the prior and posterior distributions, forcing the Bernoulli latent distribution to maximize the overlap of input and joint input-target distributions. The proposed model is then trained and compared with a baseline only consisting of direct Bernoulli latent distribution with no Born machine representing the latent space. The models are evaluated based on the segmentation metrics, such as precision, recall, intersect of union, with uncertainty boundaries accounting for the stochastic nature of the model. Our findings show that even in low latent-dimensional space (due to the limit in computational power of the classical quantum simulator), we are able to effectively capture the latent representation and hence the model performs better than the baseline. The findings are a projection for scaling the model into higher dimensional latent space with the Born machine surpassing the baseline performance. Figure 1. Sub-figure (a) demonstrates the architecture for the training phase. The model consists of a Prior and Posterior network that encode inputs and joint input-target data into compact representations, respectively. The Born machine represents the latent distribution, and the U-NET branch learns the segmentation patterns of the data. The stochasticity is introduced to the U-NET through its last layer to create meaningful but stochastic segmentations. Sub-figure (b) represents the inference phase where the model takes the stochastic behavior from the prior network and injects that into the U-NET. Each attempt of inference will generate different but similar segmentations from the same distribution of the wildfire event. REFERENCES [1] Coyle, B., Mills, D., Danos, V., & Kashefi, E. (2020). The Born supremacy: quantum advantage and training of an Ising Born machine. npj Quantum Information, 6(1), 1-11. [2] Liu, J. G., & Wang, L. (2018). Differentiable learning of quantum circuit born machines. Physical Review A, 98(6), 062324. [3] Kohl, S., Romera-Paredes, B., Meyer, C., De Fauw, J., Ledsam, J. R., Maier-Hein, K., ... & Ronneberger, O. (2018). A probabilistic u-net for segmentation of ambiguous images. Advances in neural information processing systems, 31.

quantum machine learning↗
NASA NTRS2019

Image Processing Methods for Coronal Hole Segmentation, Matching, and Map Classification

The paper presents the results from a multi-year effort to develop and validate image processing methods forselecting the best physical models based on solar image observations. The approach consists of selecting the physical models based on their agreement with coronal holes extracted from the images. Ultimately, the goal is to use physical models to predict geomagnetic storms. We decompose the problem into three subproblems: (i) coronal hole segmentation based on physical constraints, (ii) matching clusters of coronal holes between different maps, and (iii) physical map classification. For segmenting coronal holes, we develop a multi-modal method that uses segmentation maps from three different methods to initialize a level-set method that evolves the initial coronal hole segmentation to the magnetic boundary. Then, we introducea new method based on Linear Programming for matchingclusters of coronal holes. The final matching is then performedusing Random Forests. The methods were carefully validatedusing consensus maps derived from multiple readers, manualclustering, manual map classification, and method validation for50 maps. The proposed multi-modal segmentation method significantly outperformed SegNet, U-net, Henney-Harvey, and FCNby providing accurate boundary detection. Overall, the methodgave a 95.5% map classification accuracy.

Jatla, Venkatesh↗
NASA NTRSDate not supplied

TPSAS-NF1676L-35370-DND

The above anvil cirrus plume (AACP) is a weather phenomenon that signifies an intense tropopause penetrating updraft which can inject cirrus clouds several kilometers into the stratosphere. Storms that have such intense updrafts are often supercells which generate severe weather such as tornadoes, high winds, and hail. In addition, AACPs moisten the stratosphere and influence the Earth's radiative balance. Though an AACP can be identified by the human eye, no automated AACP detection methods currently exist. Lack of detection inhibits understanding of where and how often AACPs occur, and how these storms influence stratospheric air composition. Previous work involved synthesis of multiple remote sensing and severe weather report/warning data sources to identify AACPs in Geostationary Operational Environmental Satellite system (GOES) satellite imagery and better understand their weather impacts (Bedka et al. (Wea. Forecasting, 2018)). This current study demonstrates an automated AACP identification method based on the application of a deep learning segmentation model known as a U-net. This study documents the development of a U-net model capable of identifying emergent AACPs using only satellite infrared (IR) and visible reflected sunlight imagery. The performance of a U-net is quantitatively benchmarked with human AACP identifications and qualitatively assessed through animations of detections generated from GOES-16 1-minute temporal resolution imagery.

Charles Liles↗
NASA NTRS2020

Flood Mapping Using UAVSAR and Convolutional Neural Networks

We have mapped flooded areas in data collected by the NASA/JPL Uninhabited Aerial Vehicle Synthetic Aperture Radar (UAVSAR) using two convolutional neural network (CNN) image classifier architectures: U-Net and SegNet. Our study area was a region around Houston, TX, USA affected by widespread flooding in 2017 due to Hurricane Harvey. To train and test the classifiers, we manually labelled over 10000 image segments in two flight lines. Both U-Net and SegNet yielded higher accuracy than a previous non-machine learning classifier we used as a baseline. U-Net had slightly higher accuracy than SegNet. The classifiers performed better in areas with more homogeneous land cover. To independently validate the classifier accuracy we used NOAA aerial imagery, with overall accuracy around 80%. Future work includes assessing the classifier robustness in other study areas, assessing the classifier dependence on UAVSAR incidence angle, particularly for open water and bare ground, and collecting more training data, particularly in urban areas. This study demonstrates the potential of CNN image classifiers for mapping flooded areas in airborne polarimetric SAR imagery, and for land cover classification of polarimetric SAR imagery more generally.

Denbina, Michael W↗
NASA NTRS2018

Automated Semantic Segmentation for Volumetric Cardiovascular Feature Quantification and Pathology Assessment

We present a pipeline method that curtails the expense and observer bias of manual cardiac evaluation by combining semantic segmentation and disease classification as a fully automatic processing pipeline. The initial element consists of a 2D U-Net convolutional neural network architecture for voxel-wise segmentation of the myocardium and ventricular cavities. The results of the segmentation were used to compute a comprehensive volumetric feature matrix that captured diagnostic clinical procedure data and that was used to model a cardiac pathology classifier.Our approach evaluated anonymized parasternal MRI cardiac images from a database of 100 patients (4 pathology groups, 1 healthy group, 20 patients per group) examined at the University Hospital of Dijon. We achieved top average Dice index scores of 0.939, 0.849, 0.886 for structure segmentation of the left ventricle (LV), right ventricle (RV) and myocardium respectively. A 5-ary pathology classification accuracy of 90% was recorded on an independent test set using our trained model.

Lindsey, Tony↗
NASA NTRSDate not supplied

Satellite-Derived Imagery and Transfer Learning: A Novel Technique for Land Cover Classification

Land cover classification is a continuing research topic due to its relevance to land use and land cover changes from impacts such as climate change, agriculture, urbanization, and hazardous weather. Simple access to frequently changing land cover classifications could provide knowledge and decision support to various researchers and agencies across the globe for each of above-mentioned and related influences. This research aims to provide a novel technique for land cover classification of remote sensing imagery; harnessing Artificial Neural Networks and transfer learning (TL). Knowledge sharing techniques within machine learning are typically utilized when training datasets are sparse or transitions between data modalities is required. In this case, two data modalities, multi and hyperspectral data, are considered for knowledge transfer. The large number of continuous spectral bands available from hyperspectral sensors typically provide increased sophisticated land classification capability compared to more traditional multispectral imagery with limited discrete spectral bands. However, large-scale, frequent access to hyperspectral imagery is relatively limited. The proposed classification technique would therefore prove useful in regions in which hyperspectral data are not readily available for classification but multispectral data are. Image segmentation models are trained on each multi and hyperspectral datasets. Knowledge sharing techniques are then applied to each model to understand what knowledge, if any, is gained when moving between the data modalities. The datasets utilized for training and testing a U-Net model include the European Space Agency’s multispectral imager Sentinel-2 and the hyperspectral German Aerospace Center’s Earth Sensing Imaging Spectrometer (DESIS). Initially, only two classes, land and water, are classified for simplicity. However, more complex classes can be added if knowledge sharing is successful. The workflow for image classification through supervised image segmentation with a U-Net model will be discussed along with metrics calculated before and after TL for both Sentinel-2 and DESIS data are applied. Additionally, future steps to advance the sophistication of this technique as well as other applicable methodologies will be explored.

Emily Foshee↗
NASA NTRS2021

Semantic Segmentation of High-Resolution Satellite Imagery using Generative Adversarial Networks with Progressive Growing

With increase in urbanization and Earth Sciences research into urban areas, the need to quickly and accurately segment urban rooftop maps has never been greater. Cur-rent machine learning techniques struggle to produce high accuracy maps in dense urban zones where there is high image noise and foot print overlap. In this paper, we evaluate a training methodology for pixel-wise segmentation for high resolution satellite imagery using progressive growing of generative adversarial networks as a solution. We apply our model to segmenting building rooftops and compare these results to conventional methods for rooftop segmentation. We evaluate our approach using the SpaceNet version 2 and xView datasets. Our experiments show that for SpaceNet, progressive Generative Adversarial Network (GAN) training achieved a test accuracy of 93% compared to 89% for traditional GAN training and 87% for U-Net architecture, while for xView, we achieved 71% accuracy using progressive GAN training compared to 69% through traditional GAN training and 65% using U-Net.

Semantic↗
NASA NTRS2020

Automated Cardiovascular Pathology Assessment using Semantic Segmentation and Ensemble Learning

Cardiac magnetic resonance imaging provides high spatial resolution, enabling improved extraction of important functional and morphological features for cardiovascular disease staging. Segmentation of ventricular cavities and myocardium in cardiac cine sequencing provides a basis to quantify cardiac measures such as ejection fraction. A method is presented that curtails the expense and observer bias of manual cardiac evaluation by combining semantic segmentation and disease classification into a fully automatic processing pipeline. The initial processing element consists of a robust dilated convolutional neural network architecture for voxel-wise segmentation of the myocardium and ventricular cavities. The resulting comprehensive volumetric feature matrix captures diagnostic clinical procedure data and is utilized by the final processing element to model a cardiac pathology classifier. Our approach evaluated anonymized cardiac images from a training data set of 100 patients (4 pathology groups, 1 healthy group, 20 patients per group) examined at the University Hospital of Dijon. The top average Dice index scores achieved were 0.940, 0.886, 0.849 for structure segmentation of the left ventricle (LV), myocardium and right ventricle (RV) respectively. A 5-ary pathology classification accuracy of 90% was recorded on an independent test set using the trained model. Performance results demonstrate potential for advanced machine learning methods to deliver accurate, efficient and reproducible cardiac pathological assessment.

Semantic Segmentation↗
NASA NTRSDate not supplied

Machine-Learning-Based Adaptive Thinning of CrIS Radiances to Improve Global Tropical Cyclone Analysis and Forecasts

This work is focused on optimizing the assimilation of hyperspectral infrared (IR) radiances from the Cross-track Infrared Sounder (CrIS) with the goal of improving the representation of tropical cyclones (TCs) in global analyses and forecasts. Current operational assimilation systems rely on subsampling IR radiances on a regular thinning grid. A new and improved adaptive methodology based on machine learning (ML) recognizes TCs from geostationary satellite imagery and is implemented in the Goddard Earth Observing System (GEOS) model and data assimilation framework. The ML methodology is extensively trained on existing TC data sets and creates for each TC a dynamic mask, based on the evolving shape and life cycle of that specific event. Once a TC mask is created, a switch is then activated in the data assimilation system to alter the thinning, ingesting more CrIS radiances within the moving mask, thus increasing the TC sampling. After the TC dissipates, the assimilation of CrIS radiances reverts to normal data density. Results of TC segmentation provided by a state-of-the-art generative machine learning model known as the Denoising Diffusion Probabilistic Model (DDPM) are compared to the previously used U-Net model. The new approach surpasses the performance of the previously developed one. The methodology is applied to both clear-sky and cloud-cleared radiances. Benefits from the latter methodology, particularly in improving the structure of TCs and the intensity forecasts, are presented.

Oreste Reale↗