Search NASA⌕ Search

SEARCH · Search NASA

Results for “multispectral imaging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

ResSR: A Computationally Efficient Residual Approach to Super-Resolving Multispectral Images

Multispectral imaging (MSI) plays a critical role in material classification, environmental monitoring, and remote sensing. However, MSI sensors typically have wavelength-dependent resolution, which limits downstream analysis. MSI super-resolution (MSI-SR) methods address this limitation by reconstructing all bands at a common high spatial resolution. Existing methods can achieve high reconstruction quality but often rely on spatially-coupled optimization or large learning-based models, leading to significant computational cost and limiting their use in large-scale or time-critical settings. In this paper, we introduce ResSR, a computationally efficient, model-based MSI-SR method that achieves high-quality reconstruction without supervised training or spatially-coupled optimization. Notably, ResSR decouples spectral and spatial processing into two sequential steps. ResSR first computes a spectrally-informed high-resolution estimate of the MSI using singular value decomposition together with a spatially-decoupled approximate forward model. It then applies a residual correction step to restore low-frequency spatial consistency while preserving high-frequency detail recovered by the spectral reconstruction. ResSR achieves comparable or improved reconstruction quality relative to existing MSI-SR methods while being

Sullivan, Haley [ORNL] (ORCID:0000000274069217)↗

NDVI and Multispectral Images, UAS Flights, Council Road Site, Seward Peninsula, Alaska, June 2022

The dataset contains Normalized Difference Vegetation Index (NDVI) and multispectral images collected on June 18, 2022 using an UAS (Unmanned Aerial System) Yuneec H520, Mapir Survey 3, and Parrot Sequoia cameras. This data was created to accompany the following datasets:- NGEE Arctic CO2, CH4 and Energy Eddy-Covariance (EC) Flux Tower Auxiliary Measurements, Council Road Mile Marker 71, Seward Peninsula, Alaska (https://doi.org/10.5440/1526749)- Sun-induced chlorophyll fluorescence/eddy covariance measurements at NGEE Arctic Council site, Seward Peninsula, Alaska, 2022 (https://doi.org/10.15485/2466175)- NGEE Arctic Meteorological Data from Instrumented PCDC Stations, Council Site, Seward Peninsula, Alaska, 2018 - 2023 (https://doi.org/10.5440/1529604)Survey 3 is RGN (Red, Greeen, Near Infra Red) camera with 87° Horizontal Field of View (HFOV) (19mm) f/2.8 Aperture, 550nm/660nm/850nm (Red+Green+NIR) filter transmission and produced geo referenced jpg and raw format images. Parrot Sequoia is a camera with 16 megapixel RGB (Red, Green, Blue) sensor and 1.2 megapixel monochrome sensors in discrete spectral bands: Green (550nm - 40Nm Bandwidth), Red (660nm - 40nm Bandwidth), Red Edge (735nm - 10nm Bandwidth) and Near Infrared (790nm - 40nm Bandwidth).The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Recurrent and convolutional neural networks for sequential multispectral optoacoustic tomography ( MSOT ) imaging

Abstract Multispectral optoacoustic tomography (MSOT) is a beneficial technique for diagnosing and analyzing biological samples since it provides meticulous details in anatomy and physiology. However, acquiring high through‐plane resolution volumetric MSOT is time‐consuming. Here, we propose a deep learning model based on hybrid recurrent and convolutional neural networks to generate sequential cross‐sectional images for an MSOT system. This system provides three modalities (MSOT, ultrasound, and optoacoustic imaging of a specific exogenous contrast agent) in a single scan. This study used ICG‐conjugated nanoworms particles (NWs‐ICG) as the contrast agent. Instead of acquiring seven images with a step size of 0.1 mm, we can receive two images with a step size of 0.6 mm as input for the proposed deep learning model. The deep learning model can generate five other images with a step size of 0.1 mm between these two input images meaning we can reduce acquisition time by approximately 71%.

Juhong, Aniwat↗

Estimating Switchgrass Biomass Yield and Lignocellulose Composition from UAV-Based Indices

Innovative methods for estimating commercial-scale switchgrass yields and feedstock quality are essential to optimize harvest logistics and biorefinery efficiency for sustainable aviation fuel production. This study utilized vegetation indices (VIs) derived from multispectral images to predict biomass yield and lignocellulose concentrations of advanced bioenergy-type switchgrass cultivars (“Liberty” and “Independence”) under two N rates (28 and 56 kg N ha –1 ). Field-scale plots were arranged in a randomized complete block design (RCBD) and replicated three times at Urbana, IL. Multispectral images captured during the 2021–2023 growing seasons were used to extract VIs. The results show that linear and exponential models outperformed partial least square and random forest models, with mid-August imagery providing the best predictions for biomass, cellulose, and hemicellulose. The green normalized difference vegetation index (GNDVI) was the best univariate predictor for biomass yield (R 2 = 0.86), while a multivariate combination of the GNDVI and normalized difference red-edge index (NDRE) enhanced prediction accuracy (R 2 = 0.88). Cellulose was best predicted using the NDRE (R 2 = 0.53), whereas hemicellulose prediction was most effective with a multivariate model combining the GNDVI, NDRE, NDVI, and green ratio vegetation index (GRVI) (R 2 = 0.44). These findings demonstrate the potential of UAV-based VIs for the in-season estimation of biomass yield and cellulose concentration.

09 BIOMASS FUELS↗

Integrating LEO and GEO Observations: Toward Optimal Summertime Satellite Precipitation Retrieval

Abstract Reliable quantitative precipitation estimation with a rich spatiotemporal resolution is vital for understanding the Earth’s hydrological cycle. Precipitation estimation over land and coastal regions is necessary for addressing the high degree of spatial heterogeneity of water availability and demand, and for resolving the extremes that modulate and amplify hazards such as flooding and landslides. Advancements in computation power along with unique high spatiotemporal and spectral resolution data streams from passive meteorological sensors aboard geosynchronous Earth-orbiting (GEO) and low Earth-orbiting (LEO) satellites offer exciting opportunities to retrieve information about surface precipitation phenomena using data-driven machine learning techniques. In this study, the capabilities of U-Net–like architecture are investigated to map instantaneous, summertime surface precipitation intensity at the spatial resolution of 2 km. The calibrated brightness temperature products from the Global Precipitation Measurement (GPM) Microwave Imager (GMI) radiometer are combined with multispectral images (visible, near-infrared, and infrared bands) from the Advanced Baseline Imager (ABI) aboard the GOES-R satellites as main inputs to the U-Net–like precipitation algorithm. Total precipitable water and 2-m temperature from the Global Forecast System (GFS) model are also used as auxiliary inputs to the model. The results show that the U-Net–like algorithm can capture fine-scale patterns and intensity of surface precipitation at high spatial resolution over stratiform and convective precipitation regimes. The evaluations reveal the potential of extracting relevant, high spatial features over complex surface types such as mountainous regions and coastlines. The algorithm allows users to interpret the inputs’ importance and can serve as a starting point for further exploration of precipitation systems within the field of hydrometeorology.

Meteorology & Atmospheric Sciences↗

Multispectral Quantitative Phase Imaging Using a Diffractive Optical Network

As a label‐free imaging technique, quantitative phase imaging (QPI) provides optical path length information of transparent specimens for various applications in biology, materials science, and engineering. Multispectral QPI measures quantitative phase information across multiple spectral bands, permitting the examination of wavelength‐specific phase and dispersion characteristics of samples. Herein, the design of a diffractive processor is presented that can all‐optically perform multispectral quantitative phase imaging of transparent phase‐only objects within a snapshot. The design utilizes spatially engineered diffractive layers, optimized through deep learning, to encode the phase profile of the input object at a predetermined set of wavelengths into spatial intensity variations at the output plane, allowing multispectral QPI using a monochrome focal plane array. Through numerical simulations, diffractive multispectral processors are demonstrated to simultaneously perform quantitative phase imaging at 9 and 16 target spectral bands in the visible spectrum. The generalization of these diffractive processor designs is validated through numerical tests on unseen objects, including thin Pap smear images. Due to its all‐optical processing capability using passive dielectric diffractive materials, this diffractive multispectral QPI processor offers a compact and power‐efficient solution for high‐throughput quantitative phase microscopy and spectroscopy.

36 MATERIALS SCIENCE↗

Relationships between summertime surface albedo and melt pond fraction in the central Arctic Ocean: The aggregate scale of albedo obtained on the MOSAiC floe

As part of the Multidisciplinary drifting Observatory for the Study of Arctic Climate (MOSAiC), the HELiX uncrewed aircraft system (UAS) was deployed over the sea ice in the central Arctic Ocean during summer 2020. Albedo measurements were obtained with stabilized pyranometers, and melt pond fraction was calculated from orthomosaic imagery from a surface-imaging multispectral camera. This study analyzed HELiX flight data to provide insights on the temporal and spatial evolution of albedo and melt pond fraction of the MOSAiC floe during the melt season as it drifted south through Fram Strait. The surface albedo distributions showed peak values changing from high albedo (0.55–0.6) to lower values (0.3) as the season advanced. Inspired by methods developed for satellite data, an algorithm was established to retrieve melt pond fraction from the orthomosaic images. We demonstrate that the near-surface observations of melt pond fraction were highly dependent on sample area, offering insight into the influence of subgrid scale features and spatial heterogeneity in satellite observations. Vertical observations conducted with the HELiX were used to quantify the influence of melt pond scales on observed surface albedo as a function of sensor footprint. These scaling results were used to link surface-based measurements collected during MOSAiC to broader-scale satellite data to investigate the influence of surface features on observed albedo. Albedo values blend underlying features within the sensor footprint, as determined by the melt pond size and concentration. This study framed the downscaling (upscaling) problem related to the airborne (surface) observations of surface albedo across a variety of spatial scales.

54 ENVIRONMENTAL SCIENCES↗

hdsullivan/ResSR

This is the official implementation of ResSR [1]. ResSR is a computationally efficient MSI-SR method that achieves high-quality reconstructions by using a closed-form spectral decomposition along with a spatial residual correction. ResSR applies singular value decomposition to identify correlations across spectral bands, uses pixel-wise computation to upsample the MSI, and then applies a residual correction process to correct the high-spatial frequency components of the upsampled bands. While ResSR is formulated as the solution to a spatially-coupled optimization problem, we use pixel-wise regularization and derive an approximate closed-form solution, resulting in a pixel-wise algorithm with a dramatic reduction in computation that achieves state-of-the-art reconstructions. [1] Duba-Sullivan, H., Reid, E. J., Voisin, S., Bouman, C. A., & Buzzard, G. T. (2024). ResSR: A Computationally Efficient Residual Approach to Super-Resolving Multispectral Images. arXiv preprint arXiv:2408.13225.

Duba-Sullivan, Haley [Oak Ridge National Laborator↗

Batch Active Learning for Multispectral and Hyperspectral Image Segmentation Using Similarity Graphs

Abstract Graph learning, when used as a semi-supervised learning (SSL) method, performs well for classification tasks with a low label rate. We provide a graph-based batch active learning pipeline for pixel/patch neighborhood multi- or hyperspectral image segmentation. Our batch active learning approach selects a collection of unlabeled pixels that satisfy a graph local maximum constraint for the active learning acquisition function that determines the relative importance of each pixel to the classification. This work builds on recent advances in the design of novel active learning acquisition functions (e.g., the Model Change approach in arXiv:2110.07739) while adding important further developments including patch-neighborhood image analysis and batch active learning methods to further increase the accuracy and greatly increase the computational efficiency of these methods. In addition to improvements in the accuracy, our approach can greatly reduce the number of labeled pixels needed to achieve the same level of the accuracy based on randomly selected labeled pixels.

97 MATHEMATICS AND COMPUTING↗

High-Resolution Image Products Acquired from Mid-Sized Uncrewed Aerial Systems for Land–Atmosphere Studies

We assess the viability of deploying commercially available multispectral and thermal imagers designed for integration on small uncrewed aerial systems (sUASs, <25 kg) on a mid-size Group-3-classification UAS (weight: 25–600 kg, maximum altitude: 5486 m MSL, maximum speed: 128 m/s) for the purpose of collecting a higher spatial resolution dataset that can be used for evaluating the surface energy budget and effects of surface heterogeneity on atmospheric processes than those datasets traditionally collected by instrumentation deployed on satellites and eddy covariance towers. A MicaSense Altum multispectral imager was deployed on two very similar mid-sized UASs operated by the Atmospheric Radiation Measurement (ARM) Aviation Facility. This paper evaluates the effects of flight on imaging systems mounted on UASs flying at higher altitudes and faster speeds for extended durations. We assess optimal calibration methods, acquisition rates, and flight plans for maximizing land surface area measurements. We developed, in-house, an automated workflow to correct the raw image frames and produce final data products, which we assess against known spectral ground targets and independent sources. We intend this manuscript to be used as a reference for collecting similar datasets in the future and for the datasets described within this manuscript to be used as launching points for future research.

47 OTHER INSTRUMENTATION↗

DeepAndes: A Self-Supervised Vision Foundation Model for Multispectral Remote Sensing Imagery of the Andes

By mapping sites at large scales usingremotely sensed data, archaeologists can generate unique insights into long-term demographic trends, interregional social networks, and human adaptations in the past. Remote sensing surveys complement field-based approaches, and their reach can be especially great when combined with deep learning and computer vision techniques. However, conventional supervised deep learning methods face challenges in annotating fine-grained archaeological features at scale. In addition, while recent vision foundation models have shown remarkable success in learning large-scale remote sensing data with minimal annotations, most off-the-shelf solutions are designed for RGB images rather than multispectral satellite imagery, such as the eight-band data used in our study. In this article, we introduce DeepAndes, a transformer-based vision foundation model trained on three million multispectral satellite images, specifically tailored for Andean archaeology. DeepAndes incorporates a customized DINOv2 self-supervised learning algorithm optimized for eight-band multispectral imagery, marking the first foundation model designed explicitly for the Andes region. We evaluate its image understanding performance through imbalanced image classification, image instance retrieval, and pixel-level semantic segmentation tasks. Our experiments show that DeepAndes achieves superior F1 scores, mean average precision, and Dice scores in few-shot learning scenarios, significantly outperforming models trained from scratch or pretrained on smaller datasets. This underscores the effectiveness of large-scale self-supervised pretraining in archaeological remote sensing.

Guo, Junlin [Vanderbilt Univ., Nashville, TN (Unit↗

UAS remote sensing (3DR SOLO platform): multispectral reflectance, canopy height model, normalized difference vegetation index, Seward Peninsula, Alaska, 2021

Airborne remote sensing data collected using a Parrot Sequoia+ multispectral sensor installed on a 3DR SOLO unoccupied aerial system (UAS) - operated by the Terrestrial Ecosystem Science Technology group at Brookhaven National Laboratory. This package includes data from 10 flights flown over the NGEE-Arctic Council Mile Marker (MM) 71, Kougarok MM64, and Teller MM27 sites on the Seward Peninsula, Alaska, in August 2021. Derived image products include point cloud, ortho-mosaiced multispectral image, ortho-mosaiced RGB image, a digital surface model (DSM) using the structure from motion (SfM) technique, a canopy height model (CHM), and a normalized difference vegetation index (NDVI) map. Unprocessed and processed data products are included in this package (processing levels 0-2). Data and metadata are provided as text (*.txt, *.json, *hdr,), tabular (*.dat, *.csv), point cloud (*.laz), Cloud Optimized GeoTIFF (COG, *.tif), and image (*.jpg, *.tif, *png) formats. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

UAS remote sensing (3DR SOLO platform): multispectral reflectance and normalized difference vegetation index, Seward Peninsula, Alaska, 2022

Airborne remote sensing data collected using a Parrot Sequoia+ multispectral sensor installed on a 3DR SOLO unoccupied aerial system (UAS) – operated by the Terrestrial Ecosystem Science & Technology group https://www.bnl.gov/envsci/testgroup/ at Brookhaven National Laboratory. This package includes data from 19 flights flown over the NGEE-Arctic, Kougarok Mile Marker (MM) 80, Kougarok Fire Complex (KFC) and Teller MM 27 sites in July 2022. Derived image products include point cloud, ortho-mosaiced multispectral image, a digital surface model (DSM) using the structure from motion (SfM) technique, and a normalized difference vegetation index (NDVI) map. Unprocessed and processed data products are included in this package (processing levels 0-2). Data and metadata are provided as text (*.txt, *.json, *hdr,), tabular (*.dat, *.csv), point cloud (*.laz), Cloud Optimized GeoTIFF (COG, *.tif), and image (*.jpg, *.tif, *png) formats.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Multispectral and thermal surface imagery and surface elevation mosaics - Pendleton Feb 2023

This data set contains high-resolution image products (orthomosaics) acquired from midsized uncrewed aerial systems that have been processed for value-added quality. The instrument Altum multispectral imager by Micasense, captures in six bands (blue - 475nm, green - 560nm, red - 668nm, red edge - 717nm, near-infrared - 840 and LWIR/thermal - 11000nm. The optical bands are converted to reflectance via custom code using the instantaneous band horizontal irradiance ratio to the radiance of the pixel. The code used to develop these images first uses tools from the Micasense Python library to apply dark level corrections, row gradient corrections, and radiometric corrections. Next, it uses the processing API from Agisoft Metashape software to align and mosaic the processed imagery, following the processes developed by the USGS' structure from motion workflow documentation. Captures from different altitudes are used to produce an orthomosaic at each height. A tif image containing information related to the six spectral bands, and a digital elevation model (DEM), a tif image containing information related to the elevation of the surveyed terrain. 1 https://www.arm.gov/publications/tech_reports/handbooks/doe-sc-arm-tr-281.pdf 2 https://micasense.github.io/imageprocessing/MicaSense%20Image%20Processing%20Setup.html 3 https://pubs.usgs.gov/of/2021/1039/ofr20211039.pdf

54 ENVIRONMENTAL SCIENCES↗

Comparison of CNN-Based Image Classification Approaches for Implementation of Low-Cost Multispectral Arcing Detection

Camera-based sensing has benefited in recent years from developments in machine learning data processing methods, as well as improved data collection options such as Unmanned Aerial Vehicles (UAV) mounted sensors. However, cost considerations, both for the initial purchase of sensors as well as updates, maintenance, or potential replacement if damaged, can limit adoption of more expensive sensing options for some applications. To evaluate more affordable options with less expensive, more available, and more easily replaceable hardware, we examine the use of machine learning-based image classification with custom datasets, utilizing deep learning based-image classification and the use of ensemble models for sensor fusion. Utilizing the same models for each camera to reduce technical overhead, we showed that for a very representative training dataset, camera-based detection can be successful for detection of electrical arcing. We also use multiple validation datasets, based on conditions expected to be of varying difficulty, to evaluate custom data. These results show that ensemble models of different data sources can mitigate risks from gaps in training data, though the system will be less redundant for those cases unless other precautions are taken. We found that with good quality custom datasets, data fusion models can be utilized without specialization in design to the specific cameras utilized, allowing for less specialized, more accessible equipment to be utilized as multispectral camera components. This approach can provide an alternative to expensive sensing equipment for applications in which lower-cost or more easily replaceable sensing equipment is desirable.

convolutional neural networks↗

Quantifying the robustness of deep multispectral segmentation models against natural perturbations and data poisoning

In overhead image segmentation tasks, including additional spectral bands beyond the traditional RGB channels can improve model performance. However, it is still unclear how incorporating this additional data impacts model robustness to adversarial attacks and natural perturbations. For adversarial robustness, the additional in-formation could improve the model’s ability to distinguish malicious inputs, or simply provide new attack avenues and vulnerabilities. For natural perturbations, the additional information could better inform model decisions and weaken perturbation effects or have no significant influence at all. In this work, we seek to characterize the performance and robustness of a multispectral (RGB and near infrared) image segmentation model subjected to adversarial attacks and natural perturbations. While existing adversarial and natural robustness research has focused primarily on digital perturbations, we prioritize on creating realistic perturbations designed with physical world conditions in mind. For adversarial robustness, we focus on data poisoning attacks whereas for natural robustness, we focus on extending ImageNet-C common corruptions for fog and snow that coherently and self-consistently perturbs the input data. Overall, we find both RGB and multispectral models are vulnerable to data poisoning attacks regardless of input or fusion architectures and that while physically-realizable natural perturbations still degrade model performance, the impact differs based on fusion architecture and input data.

Deep learning, multispectral images, multimodal fu↗