One-shot gas detection with transformer paired neural networks in Mako collected longwave infrared hyperspectral imagery
Explore the source record for details and available documents.
Benham, Kevin Andrew↗
SEARCH · Search NASA
Results for “Hyperspectral imagery”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Chemical signature characterization with hyperspectral imagery: novel deep learning model architectures and physically-motivated data augmentation techniques
The high spectral resolution afforded by Hyperspectral Imaging (HSI) sensors is poised to bring unprecedented advancements to signature characterization applications. Thus far, much of the research in the machine learning field devoted to HSI applications has focused on a few specific tasks like land-use land-cover classification. In land classification tasks, spatial information is very important, and model architectures are often designed to leverage spatial contexts. However, it is unclear how well these spatially-tuned models will translate to tasks where spectral information is critical, like the detection and characterization of chemicals. In this work, we compare spectral models (inputs are 1D spectra) and spatial-spectral models (inputs are 3D cubes) in the context of predicting chemical concentration maps. We find that spatial-spectral models perform the best, though we find a wide range in performance across the different architectures tested. Additionally, we find that model performance is impacted by the availability of training data, particularly in scenarios where the training data doesn't fully capture the true variance of real-world conditions. We find that data augmentation can help mitigate sparse coverage of observed parameter space (e.g., seasonal or geographic variability in ground cover), and present augmentation strategies that are tailored to hyperspectral data.
• Artificial intelligence (AI) / machine learning ↗
One-shot gas detection with transformer paired neural networks in Mako collected longwave infrared hyperspectral imagery
To date, careful data treatment workflows and statistical detectors are used to perform hyperspectral image (HSI) detection of any gas contained in a spectral library, which is often expanded with physics models to incorporate different spectral characteristics. In general, surrounding evidence or known gas-release parameters are used to provide confidence in or confirm detection capability, respectively. This makes quantifying detection performance difficult as it is nearly impossible to develop an absolute ground truth for gas target pixel presence in collected HSI. Consequently, developing and comparing new detection methods, especially machine learning (ML)-based methods, is susceptible to subjectivity in derived detection map quality. Here, in this work, we demonstrate the first use of transformer-based paired neural networks (PNNs) for one-shot gas target detection for multiple gases while providing quantitative classification and detection metrics for their use on labeled data. Terabytes of training data are generated from a database of long-wave infrared HSI obtained from historical Mako sensor campaigns over Los Angeles. By incorporating labels, singular signature representations, and a model development pipeline, we can tune and select PNNs to detect multiple gas targets that are not seen in training on a quantitative basis. We additionally assess our test set detections using interpretability techniques widely employed with ML-based predictors, but less common with detection methods relying on learned latent spaces.
Hyperspectral imaging↗
Monitoring of Terrestrial Aquatic Ecosystems Through Analysis of Time-Variant Hyperspectral Imagery
Time-variant spectral analysis of plants can classify stress-based perturbations (e.g., dry soil or increased salination) and quantify mechanistic attributes associated with plant photosynthesis, respiration, hormone function, and plant morphology. This project will investigate the utility of hyperspectral cameras for detecting and monitoring these perturbations.
59 BASIC BIOLOGICAL SCIENCES↗
Low-shot, Semi-supervised, Uncertainty Quantified Learningwith Hyperspectral Imagery Data.
Abstract not provided.
Gray, Kathryn↗
FastACE
SAND2024-01893O The Fast Adaptive Cosine Estimator (FastACE) algorithm modifies the well-known Adaptive Cosine Estimator for target detection in hyperspectral imagery. Specifically, FastACE modifies the computation of the background precision matrix (C_b^-1) under a Vecchia approximation. That is, each spectral band, conditioned on a local neighborhood of bands around that band, is independent of the other spectral bands. The FastACE algorithm leverages a parameterizable, auto-regressive neighborhood for the conditional independence assumption. The underlying math used for computing detection scores is equivalent to ACE, but it is implemented within FastACE. The software implements a target detection algorithm and associated utilities for target detection in hyperspectral imagery. Provided with a hyperspectral image and corresponding target signature, it produces relevant background statistics and the corresponding detection scores for the target signature in the image. The software is designed to integrate into other end-user applications or processing. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.
VanderLaan, John↗
Maps of growing season gross primary production and net ecosystem exchange for Council Road Mile Marker 71, Seward Peninsula, Alaska, [2017-2023]
This data archive is in support of the Next-Generation Ecosystem Experiments in the Arctic (NGEE Arctic) publication "Integrating Characteristic Arctic Vegetation in a Land Surface Model Improves Representation of Carbon Dynamics Across a Tundra Landscape", by Murphy et al. (2025a). Murphy et al. (2025a) evaluated whether incorporating observed Arctic vegetation heterogeneity into ELM, the land model of the Department of Energy’s Energy Exascale Earth System Model (E3SM), improved simulations of tundra carbon cycling. The associated model archive can be found at Murphy et al. (2025b). The study focused on the spatial patterns and net landscape-level growing season productivity and carbon uptake. As part of this evaluation, observationally derived maps of average growing season (June–August) net ecosystem exchange (NEE) and gross primary production (GPP) were developed for the same domain. These maps, which form the dataset described here, integrate eddy covariance flux tower, remote sensing, and vegetation community data to provide spatially explicit benchmarks for model evaluation. The maps provide spatially explicit estimates of average growing season NEE and GPP across 13 tundra vegetation communities within the study domain. By combining flux tower observations with Airborne Visible-Infrared Imaging Spectrometer-Next Generation (AVIRIS-NG) hyperspectral imagery and drone-based normalized difference vegetation index (NDVI), these maps capture the heterogeneity of carbon fluxes associated with different Arctic vegetation types. While they represent average seasonal conditions rather than interannual variability, the maps provide a unique dataset for evaluating model performance, comparing vegetation community contributions to landscape-scale carbon cycling, and supporting regional analyses of Arctic carbon dynamics. This data archive contains 5 m resolution maps of vegetation communities, vegetation community average growing season GPP, and vegetation community average growing season NEE (three *.tif files), a User’s Guide (*pdf file), and Table 1 of the User’s Guide displaying vegetation community coverage and average growing season NEE and GPP values (*.csv file).
Murphy, Bailey [ORNL] (ORCID:0000000203995221)↗
Leaf traits and canopy structure together explain canopy functional diversity: an airborne remote sensing approach
Plant functional diversity is strongly connected to photosynthetic carbon assimilation in terrestrial ecosystems. However, many of the plant functional traits that regulate photosynthetic capacity, including foliar nitrogen concentration and leaf mass per area, vary significantly between and within plant functional types and vertically through forest canopies, resulting in considerable landscape-scale heterogeneity in three dimensions. Hyperspectral imagery has been used extensively to quantify functional traits across a range of ecosystems but is generally limited to providing information for top-of-canopy leaves only. On the other hand, lidar data can be used to retrieve the vertical structure of forest canopies. Because these data are rarely collected at the same time, there are unanswered questions about the effect of forest structure on the 3-D spatial patterns of functional traits across ecosystems. In the United States, the National Ecological Observatory Network's Airborne Observation Platform (NEON AOP) provides an opportunity to address this structure-function relationship by collecting lidar and hyperspectral data together across a variety of ecoregions. With a fusion of hyperspectral and lidar data from the NEON AOP and field-collected foliar trait data, we assessed the impacts of forest structure on spatial patterns of N. In addition, we examine the influence of abiotic gradients and management regimes on top of canopy %N and total canopy N (i.e. the total amount of N (g/m 2 ) within a forest canopy) at a NEON site consisting of a mosaic of open longleaf pine and dense broadleaf deciduous forests. Here, our resulting maps suggest that in contrast with top-of-canopy values, total canopy N variation is dampened across this landscape resulting in relatively homogeneous spatial patterns. At the same time, we found that leaf functional diversity and canopy structural diversity showed distinct dendritic patterns related to the spatial distribution of plant functional types.
54 ENVIRONMENTAL SCIENCES↗
Applications Of Machine Learning to Gas Plume Analysis In Longwave Infrared Hyperspectral Images
Longwave infrared hyperspectral images can be used for gas plume analysis, as many gases exhibit distinct absorption features in this portion of the electromagnetic spectrum. In practice, accurately identifying weak gas signatures is difficult because the observed radiance is dominated by background radiance, which varies with material, temperature, and viewing conditions. Many gas plume analysis pipelines operate on single images, limiting the ability to leverage spatial and multi-view information that could enhance the analysis. The goal of this dissertation is to explore how machine learning and deep learning methods can complement classical approaches to improve gas plume identification in longwave infrared hyperspectral imagery, and to investigate the use of neural radiance fields for hyperspectral scene reconstruction.
3D Scene Reconstruction↗
HyperKube: A Kubernetes Based System for the Automation of Processing and Analysis of Hyperspectral Data Obtained from Multiple Hyperspectral Imaging Systems
Hyperspectral imagery is an emerging field of technology that has enormous potential for remote and proximal sensing in numerous areas of research. The plant phenotyping community is applying this technology to advance the throughput and accuracy of plant phenotypes based on airborne and lab-based hyperspectral imaging technology. Here we report an automated processing and analysis pipeline for four different hyperspectral imaging platforms, discuss the data issues involved, and present a strategy for computing and data architecture to handle hyperspectral data.
47 OTHER INSTRUMENTATION↗
Cross-scale sensing of field-level crop residue cover: Integrating field photos, airborne hyperspectral imaging, and satellite data
Conservation tillage practices can bring benefits to agricultural sustainability. Accurate spatial and temporal resolved information of field-scale crop residue cover, which reflects tillage intensity, is highly valuable for evaluating the outcomes of government conservation programs and voluntary ecosystem service markets, as well as facilitating agroecosystem modeling to quantify cropland biogeochemical processes. Remote sensing has the potential to cost-effectively detect crop residue cover, however, existing regional-scale studies were limited by insufficient ground truth data, scale mismatch between coarse satellite pixels and ground data, and the lack of key spectral data for detecting crop residues. Therefore, this study developed an innovative cross-sensing framework to integrate proximal sensing, airborne hyperspectral imaging, and satellite Earth Observation through deep learning to quantify field-level crop residue cover fractions at the regional scale. Specifically, we have collected intensive ground orthographic photos and conducted airborne hyperspectral surveys at corn and soybean fields of Champaign and nearby counties in Illinois, the heartland of the U.S. Corn Belt. Through semi-automatic labeling aided by ResNet-50 and superpixel image segmentation, we obtained 6719 records of ground residue fractions. With these ground data, we developed the 1-dimensional convolution neural network (CNN) model using airborne hyperspectral reflectance, which has 0.5m spatial resolution and 3–5 nm spectral resolution from 400 to 2400 nm, to predict residue fractions. By applying the CNN model to airborne pixels, we augmented “ground truth” data of crop residues and further combined them with Harmonized Landsat and Sentinel-2 (HLS) satellite data to quantify regional residue fractions at 30 m resolution. Results show that airborne hyperspectral imagery with CNN can accurately detect residue fractions (R 2 = 0.82, relative RMSE = 11.73%) to effectively generate quasi “ground truth” data to support satellite upscaling to all fields. With independent ground data for testing, we found that the ground-airborne-satellite integrative framework achieved better predictions in estimating crop residue cover (R 2 = 0.67, relative RMSE = 17.53%) than the conventional ground-satellite upscaling (R 2 = 0.22, relative RMSE = 32.09%). Here we also found that the shortwave infrared wavelengths, particularly 2100–2300 nm, are vital for predicting crop residue cover. Sentinel-2 and Landsat-8 data have a comparable capability to track residue fractions due to similar shortwave infrared wavelengths. This study highlights the high accuracy of hyperspectral imaging to detect agroecosystem tillage management practices and the advantages of cross-scale sensing to cost-effectively integrate multi-source data to quantify field-level agroecosystem variables across scales.
60 APPLIED LIFE SCIENCES↗
Integrating multi-modal remote sensing, deep learning, and attention mechanisms for yield prediction in plant breeding experiments
In both plant breeding and crop management, interpretability plays a crucial role in instilling trust in AI-driven approaches and enabling the provision of actionable insights. The primary objective of this research is to explore and evaluate the potential contributions of deep learning network architectures that employ stacked LSTM for end-of-season maize grain yield prediction. A secondary aim is to expand the capabilities of these networks by adapting them to better accommodate and leverage the multi-modality properties of remote sensing data. In this study, a multi-modal deep learning architecture that assimilates inputs from heterogeneous data streams, including high-resolution hyperspectral imagery, LiDAR point clouds, and environmental data, is proposed to forecast maize crop yields. The architecture includes attention mechanisms that assign varying levels of importance to different modalities and temporal features that, reflect the dynamics of plant growth and environmental interactions. The interpretability of the attention weights is investigated in multi-modal networks that seek to both improve predictions and attribute crop yield outcomes to genetic and environmental variables. This approach also contributes to increased interpretability of the model's predictions. The temporal attention weight distributions highlighted relevant factors and critical growth stages that contribute to the predictions. The results of this study affirm that the attention weights are consistent with recognized biological growth stages, thereby substantiating the network's capability to learn biologically interpretable features. Accuracies of the model's predictions of yield ranged from 0.82-0.93 R 2 ref in this genetics-focused study, further highlighting the potential of attention-based models. Further, this research facilitates understanding of how multi-modality remote sensing aligns with the physiological stages of maize. The proposed architecture shows promise in improving predictions and offering interpretable insights into the factors affecting maize crop yields, while demonstrating the impact of data collection by different modalities through the growing season. By identifying relevant factors and critical growth stages, the model's attention weights provide valuable information that can be used in both plant breeding and crop management. The consistency of attention weights with biological growth stages reinforces the potential of deep learning networks in agricultural applications, particularly in leveraging remote sensing data for yield prediction. To the best of our knowledge, this is the first study that investigates the use of hyperspectral and LiDAR UAV time series data for explaining/interpreting plant growth stages within deep learning networks and forecasting plot-level maize grain yield using late fusion modalities with attention mechanisms.
59 BASIC BIOLOGICAL SCIENCES↗
Generalized Quantum Convolution for Multidimensional Data
The convolution operation plays a vital role in a wide range of critical algorithms across various domains, such as digital image processing, convolutional neural networks, and quantum machine learning. In existing implementations, particularly in quantum neural networks, convolution operations are usually approximated by the application of filters with data strides that are equal to the filter window sizes. One challenge with these implementations is preserving the spatial and temporal localities of the input features, specifically for data with higher dimensions. In addition, the deep circuits required to perform quantum convolution with a unity stride, especially for multidimensional data, increase the risk of violating decoherence constraints. In this work, we propose depth-optimized circuits for performing generalized multidimensional quantum convolution operations with unity stride targeting applications that process data with high dimensions, such as hyperspectral imagery and remote sensing. We experimentally evaluate and demonstrate the applicability of the proposed techniques by using real-world, high-resolution, multidimensional image data on a state-of-the-art quantum simulator from IBM Quantum.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Early Detection of Drought Stress in Durum Wheat Using Hyperspectral Imaging and Photosystem Sensing
Wheat, being the third largest U.S. crop and the principal food grain, faces significant risks from climate extremes such as drought. This necessitates identifying and developing methods for early water-stress detection to prevent yield loss and improve water-use efficiency. This study investigates the potential of hyperspectral imaging to detect the early stages of drought stress in wheat. The goal is to utilize this technology as a tool for screening and selecting drought-tolerant wheat genotypes in breeding programs. Additionally, this research aims to systematically evaluate the effectiveness of various existing sensors and methods for detecting early stages of water stress. The experiment was conducted in a durum wheat experimental field trial in Maricopa, Arizona, in the spring of 2019 and included well-watered and water-limited treatments of a panel of 224 replicated durum wheat genotypes. Spectral indices derived from hyperspectral imagery were compared against other plant-level indicators of water stress such as Photosystem II (PSII) and relative water content (RWC) data derived from proximal sensors. Our findings showed a 12% drop in photosynthetic activity in the most affected genotypes when compared to the least affected. The Leaf Water Vegetation Index 1 (LWVI1) highlighted differences between drought-resistant and drought-susceptible genotypes. Drought-resistant genotypes retained 43.36% more water in leaves under well-watered conditions compared to water-limited conditions, while drought-susceptible genotypes retained only 15.69% more. The LWVI1 and LWVI2 indices, aligned with the RWC measurements, revealed a strong inverse correlation in the susceptible genotypes, underscoring their heightened sensitivity to water stress in earlier stages. Several genotypes previously classified based on their drought resistance showed spectral indices deviating from expectations. Results from this research can aid farmers in improving crop yields by informing early management practices. Moreover, this research offers wheat breeders insights into the selection of drought-tolerant genotypes, a requirement that is becoming increasingly important as weather patterns continue to change.
Roy, Bishal (ORCID:0000000199125505)↗
Data Fusion of Very High Resolution Hyperspectral and Polarimetric SAR Imagery for Terrain Classification
Performing terrain classification with data from heterogeneous imaging modalities is a very challenging problem. The challenge is further compounded by very high spatial resolution. (In this paper we consider very high spatial resolution to be much less than a meter.) At very high resolution many additional complications arise, such as geometric differences in imaging modalities and heightened pixel-by-pixel variability due to inhomogeneity within terrain classes. In this paper we consider the fusion of very high resolution hyperspectral imaging (HSI) and polarimetric synthetic aperture radar (PolSAR) data. We introduce a framework that utilizes the probabilistic feature fusion (PFF) one-class classifier for data fusion and demonstrate the effect of making pixelwise, superpixel, and pixelwise voting (within a superpixel) terrain classification decisions. We show that fusing imaging modality data sets, combined with pixelwise voting within the spatial extent of superpixels, gives a robust terrain classification framework that gives a good balance between quantitative and qualitative results.
47 OTHER INSTRUMENTATION↗
Absorptive Weak Plume Detection on Gaussian and Non-Gaussian Background Clutter
For additive signals on Gaussian clutter, the optimal detector is a linear matched filter that is adapted to the known signal and the covariance of the background. This adaptive matched filter is widely used for gas-phase plume detection, even though the effect of the plume on the background is not strictly additive. Here, a derivation of the matched filter for a strictly absorptive plume produces, in the weak plume limit, a quadratic filter. This quadratic matched filter is extended in two ways: an elliptically-contoured multivariate t distribution is used to generalize the Gaussian background clutter, and a generalized likelihood ratio test detector is derived to extend applicability to stronger plumes. In addition to detectors whose purpose is to identify presence versus absence of a plume, expressions are also derived for estimating plume strength. The performance of these various detectors is evaluated by implanting simulated plume into background images that are either real hyperspectral images or simulated images based on different (Gaussian, multivariate t, and lognormal) clutter distributions.
79 ASTRONOMY AND ASTROPHYSICS↗
Towards mapping biodiversity from above: Can fusing lidar and hyperspectral remote sensing predict taxonomic, functional, and phylogenetic tree diversity in temperate forests?
Abstract Aim Rapid global change is impacting the diversity of tree species and essential ecosystem functions and services of forests. It is therefore critical to understand and predict how the diversity of tree species is spatially distributed within and among forest biomes. Satellite remote sensing platforms have been used for decades to map forest structure and function but are limited in their capacity to monitor change by their relatively coarse spatial resolution and the complexity of scales at which different dimensions of biodiversity are observed in the field. Recently, airborne remote sensing platforms making use of passive high spectral resolution (i.e., hyperspectral) and active lidar data have been operationalized, providing an opportunity to disentangle how biodiversity patterns vary across space and time from field observations to larger scales. Most studies to date have focused on single sites and/or one sensor type; here we ask how multiple sensor types from the National Ecological Observatory Network’s Airborne Observation Platform (NEON AOP) perform across multiple sites in a single biome at the NEON field plot scale (i.e., 40 m × 40 m). Location Eastern USA. Time period 2017–2018. Taxa studied Trees. Methods With a fusion of hyperspectral and lidar data from the NEON AOP, we assess the ability of high resolution remotely sensed metrics to measure biodiversity variation across eastern US temperate forests. We examine how taxonomic, functional, and phylogenetic measures of alpha diversity vary spatially and assess to what degree remotely sensed metrics correlate with in situ biodiversity metrics. Results Models using estimates of forest function, canopy structure, and topographic diversity performed better than models containing each category alone. Our results show that canopy structural diversity, and not just spectral reflectance, is critical to predicting biodiversity. Main conclusions We found that an approach that jointly leverages spectral properties related to leaf and canopy functional traits and forest health, lidar derived estimates of forest structure, fine‐resolution topographic diversity, and careful consideration of biogeographical differences within and among biomes is needed to accurately map biodiversity variation from above.
54 ENVIRONMENTAL SCIENCES↗
More chemical detection through less sampling: amplifying chemical signals in hyperspectral data cubes through compressive sensing
Compressive sensing (CS) is a method of sampling which permits some classes of signals to be reconstructed with high accuracy even when they were sampled at sub-Nyquist rates. In this paper we explore a phenomenon in which bandwise CS sampling of a hyperspectral data cube followed by reconstruction can actually result in amplification of chemical signals contained in the cube. Perhaps most surprisingly, chemical signal amplification generally seems to increase as the level of sampling decreases. In some examples, the chemical signal is significantly stronger in a data cube reconstructed from 10% CS sampling than it is in the raw, 100% sampled data cube. We explore this phenomenon in two real-world datasets including the Physical Sciences Inc. Fabry-Perot interferometer sensor multispectral dataset and the Johns Hopkins Applied Physics Lab FTIR-based longwave infrared sensor hyperspectral dataset. Each of these datasets contains the release of a chemical simulant, such as glacial acetic acid, triethyl phospate, and sulfur hexafluoride, and in all cases we use the adaptive coherence estimator (ACE) to detect a target signal in the hyperspectral data cube. We end the paper by suggesting some theoretical justifications for why chemical signals would be amplified in CS sampled and reconstructed hyperspectral data cubes and discuss some practical implications.
Hyperspectral imagery, Compressive Sensing, single↗