Engineering topics

Crawford, Melba M.

Publications and source records attributed to Crawford, Melba M..

7 records

DOE OSTI2024

Integrating multi-modal remote sensing, deep learning, and attention mechanisms for yield prediction in plant breeding experiments

In both plant breeding and crop management, interpretability plays a crucial role in instilling trust in AI-driven approaches and enabling the provision of actionable insights. The primary objective of this research is to explore and evaluate the potential contributions of deep learning network architectures that employ stacked LSTM for end-of-season maize grain yield prediction. A secondary aim is to expand the capabilities of these networks by adapting them to better accommodate and leverage the multi-modality properties of remote sensing data. In this study, a multi-modal deep learning architecture that assimilates inputs from heterogeneous data streams, including high-resolution hyperspectral imagery, LiDAR point clouds, and environmental data, is proposed to forecast maize crop yields. The architecture includes attention mechanisms that assign varying levels of importance to different modalities and temporal features that, reflect the dynamics of plant growth and environmental interactions. The interpretability of the attention weights is investigated in multi-modal networks that seek to both improve predictions and attribute crop yield outcomes to genetic and environmental variables. This approach also contributes to increased interpretability of the model's predictions. The temporal attention weight distributions highlighted relevant factors and critical growth stages that contribute to the predictions. The results of this study affirm that the attention weights are consistent with recognized biological growth stages, thereby substantiating the network's capability to learn biologically interpretable features. Accuracies of the model's predictions of yield ranged from 0.82-0.93 R 2 ref in this genetics-focused study, further highlighting the potential of attention-based models. Further, this research facilitates understanding of how multi-modality remote sensing aligns with the physiological stages of maize. The proposed architecture shows promise in improving predictions and offering interpretable insights into the factors affecting maize crop yields, while demonstrating the impact of data collection by different modalities through the growing season. By identifying relevant factors and critical growth stages, the model's attention weights provide valuable information that can be used in both plant breeding and crop management. The consistency of attention weights with biological growth stages reinforces the potential of deep learning networks in agricultural applications, particularly in leveraging remote sensing data for yield prediction. To the best of our knowledge, this is the first study that investigates the use of hyperspectral and LiDAR UAV time series data for explaining/interpreting plant growth stages within deep learning networks and forecasting plot-level maize grain yield using late fusion modalities with attention mechanisms.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS2018

Advances in Hyperspectral Image Classification Methods for Vegetation and Agricultural Cropland Studies

Hyperspectral data are becoming more widely available via sensors on airborne and unmanned aerial vehicle (UAV) platforms, as well as proximal platforms. While space-based hyperspectral data continue to be limited in availability, multiple spaceborne Earth-observing missions on traditional platforms are scheduled for launch, and companies are experimenting with small satellites for constellations to observe the Earth, as well as for planetary missions. Land cover mapping via classification is one of the most important applications of hyperspectral remote sensing and will increase in significance as time series of imagery are more readily available. However, while the narrow bands of hyperspectral data provide new opportunities for chemistry-based modeling and mapping, challenges remain. Hyperspectral data are high dimensional, and many bands are highly correlated or irrelevant for a given classification problem. For supervised classification methods, the quantity of training data is typically limited relative to the dimension of the input space. The resulting Hughes phenomenon, often referred to as the curse of dimensionality, increases potential for unstable parameter estimates, overfitting, and poor generalization of classifiers. This is particularly problematic for parametric approaches such as Gaussian maximum likelihood–based classifiers that have been the backbone of pixel-based multispectral classification methods. This issue has motivated investigation of alternatives, including regularization of the class covariance matrices, ensembles of weak classifiers, development of feature selection and extraction methods, adoption of nonparametric classifiers, and exploration of methods to exploit unlabeled samples via semi-supervised and active learning. Data sets are also quite large, motivating computationally efficient algorithms and implementations. This chapter provides an overview of the recent advances in classification methods for mapping vegetation using hyperspectral data. Three data sets that are used in the hyperspectral classification literature (e.g., Botswana Hyperion satellite data and AVIRIS airborne data over both Kennedy Space Center and Indian Pines) are described in Section 3.2 and used to illustrate methods described in the chapter. An additional high-resolution hyperspectral data set acquired by a SpecTIR sensor on an airborne platform over the Indian Pines area is included to exemplify the use of new deep learning approaches, and a multiplatform example of airborne hyperspectral data is provided to demonstrate transfer learning in hyperspectral image classification. Classical approaches for supervised and unsupervised feature selection and extraction are reviewed in Section 3.3. In particular, nonlinearities exhibited in hyperspectral imagery have motivated development of nonlinear feature extraction methods in manifold learning, which are outlined in Section 3.3.1.4. Spatial context is also important in classification of both natural vegetation with complex textural patterns and large agricultural fields with significant local variability within fields. Approaches to exploit spatial features at both the pixel level (e.g., co-occurrence–based texture and extended morphological attribute profiles [EMAPs]) and integration of segmentation approaches (e.g., HSeg) are discussed in this context in Section 3.3.2. Recently, classification methods that leverage nonparametric methods originating in the machine learning community have grown in popularity. An overview of both widely used and newly emerging approaches, including support vector machines (SVMs), Gaussian mixture models, and deep learning based on convolutional neural networks is provided in Section 3.4. Strategies to exploit unlabeled samples, including active learning and metric learning, which combine feature extraction and augmentation of the pool of training samples in an active learning framework, are outlined in Section 3.5. Integration of image segmentation with classification to accommodate spatial coherence typically observed in vegetation is also explored, including as an integrated active learning system. Exploitation of multisensor strategies for augmenting the pool of training samples is investigated via a transfer learning framework in Section 3.5.1.2. Finally, we look to the future, considering opportunities soon to be provided by new paradigms, as hyperspectral sensing is becoming common at multiple scales from ground-based and airborne autonomous vehicles to manned aircraft and space-based platforms.

Pasolli, Edoardo↗

NASA NTRS2015

An Active Learning Framework for Hyperspectral Image Classification Using Hierarchical Segmentation

Augmenting spectral data with spatial information for image classification has recently gained significant attention, as classification accuracy can often be improved by extracting spatial information from neighboring pixels. In this paper, we propose a new framework in which active learning (AL) and hierarchical segmentation (HSeg) are combined for spectral-spatial classification of hyperspectral images. The spatial information is extracted from a best segmentation obtained by pruning the HSeg tree using a new supervised strategy. The best segmentation is updated at each iteration of the AL process, thus taking advantage of informative labeled samples provided by the user. The proposed strategy incorporates spatial information in two ways: 1) concatenating the extracted spatial features and the original spectral features into a stacked vector and 2) extending the training set using a self-learning-based semi-supervised learning (SSL) approach. Finally, the two strategies are combined within an AL framework. The proposed framework is validated with two benchmark hyperspectral datasets. Higher classification accuracies are obtained by the proposed framework with respect to five other state-of-the-art spectral-spatial classification approaches. Moreover, the effectiveness of the proposed pruning strategy is also demonstrated relative to the approaches based on a fixed segmentation.

classification↗

NASA NTRS2014

A Framework for Land Cover Classification Using Discrete Return LiDAR Data: Adopting Pseudo-Waveform and Hierarchical Segmentation

Acquiring current, accurate land-use information is critical for monitoring and understanding the impact of anthropogenic activities on natural environments.Remote sensing technologies are of increasing importance because of their capability to acquire information for large areas in a timely manner, enabling decision makers to be more effective in complex environments. Although optical imagery has demonstrated to be successful for land cover classification, active sensors, such as light detection and ranging (LiDAR), have distinct capabilities that can be exploited to improve classification results. However, utilization of LiDAR data for land cover classification has not been fully exploited. Moreover, spatial-spectral classification has recently gained significant attention since classification accuracy can be improved by extracting additional information from the neighboring pixels. Although spatial information has been widely used for spectral data, less attention has been given to LiDARdata. In this work, a new framework for land cover classification using discrete return LiDAR data is proposed. Pseudo-waveforms are generated from the LiDAR data and processed by hierarchical segmentation. Spatial featuresare extracted in a region-based way using a new unsupervised strategy for multiple pruning of the segmentation hierarchy. The proposed framework is validated experimentally on a real dataset acquired in an urban area. Better classification results are exhibited by the proposed framework compared to the cases in which basic LiDAR products such as digital surface model and intensity image are used. Moreover, the proposed region-based feature extraction strategy results in improved classification accuracies in comparison with a more traditional window-based approach.

Light Detection & Ranging (LIDAR)↗

NASA NTRS1997

Detecting Small-Scale Topographic Changes and Relict Geomorphic Features on Barrier Islands Using SAR

The shapes and elevations of barrier islands may change dramatically over a short period of time during a storm. Coastal scientists and engineers, however, are currently unable to measure these changes occurring over an entire barrier island at once. This three-year project, which is funded by NASA and jointly conducted by the Bureau of Economic Geology and the Center for Space Research at The University of Texas at Austin, is designed to overcome this problem by developing the use of interferometry from airborne synthetic aperture radar (AIRSAR) to measure coastal topography and to detect storm-induced changes in topography. Surrogate measures of topography observed in multiband, fully polarimetric AIRSAR (This type of data are now referred to as POLSAR data.) are also being investigated. Digital elevation models (DEM) of Galveston Island and Bolivar Peninsula, Texas obtained with Topographic SAR (TOPSAR) are compared with measurements by Global Positioning System (GPS) ground surveys and electronic total station surveys. In addition to topographic mapping, this project is evaluating the use of POLSAR to detect old features such as storm scarps, storm channels, former tidal inlets, and beach ridges that have been obscured by vegetation, erosion, deposition, and artificial filling. We have also expanded the work from the original proposal to include the mapping of coastal wetland vegetation and depositional environments. Methods developed during this project will provide coastal geologists with an unprecedented tool for monitoring and understanding barrier island systems. This understanding will improve overall coastal management policies and will help reduce the effects of natural and man-induced coastal hazards. This report summarizes our accomplishments during the second year of the study. Also included is a discussion of our planned activities for year 3 and a revised budget.

Gibeaut, James C.↗

NASA NTRS1996

Detecting Small-Scale topographic Changes and Relict Geomorphic Features on Barrier Islands using SAR

The shapes and elevations of barrier islands may change dramatically over a short period of time such as during a storm. Even between storms, sediment is constantly shifting to and from these islands and between different areas of the islands at varying rates and in varying amounts. Coastal scientists and engineers, however, are currently unable to measure these changes occurring over an entire barrier island at once. This three-year project, which is funded by NASA and jointly conducted by the Bureau of Economic Geology and the Center for Space Research at The University of Texas at Austin, is designed to overcome this problem by developing the use of interferometry from airborne synthetic aperture radar (AIRSAR) to detect changes in coastal topography. Surrogate measures of topography observed in fully polarimetric AIRSAR are also being investigated.

Gibeaut, James C.↗

NASA NTRS1996

Modeling Wetland Vegetation using Polarimetric SAR

A three-year project to study small-scale topographic changes and relict geomorphic features on barrier islands using synthetic aperture radar (SAR) is described. A study area on the Texas coast consisting of Galveston Island and Bolivar Peninsula was overflown by the NASA/JPL DC 8 AIRSAR in April 1995. Data was acquired in the fully polarimetric mode using C-, L-, and P-bands and in the TOPSAR configuration with C- and L-bands in interferometric mode. The study area will be overflown again in late spring 1996. The data will be registered to global positioning system (GPS) surveyed points to form high resolution digital elevation models (DEM) and then analyzed to investigate possible topographic changes.

Slatton, K. Clint↗