Results for “structure from motion (SFM)” — Page 2

DOE OSTI2023

UAS remote sensing (3DR SOLO platform): multispectral reflectance and normalized difference vegetation index, Seward Peninsula, Alaska, 2022

Airborne remote sensing data collected using a Parrot Sequoia+ multispectral sensor installed on a 3DR SOLO unoccupied aerial system (UAS) – operated by the Terrestrial Ecosystem Science & Technology group https://www.bnl.gov/envsci/testgroup/ at Brookhaven National Laboratory. This package includes data from 19 flights flown over the NGEE-Arctic, Kougarok Mile Marker (MM) 80, Kougarok Fire Complex (KFC) and Teller MM 27 sites in July 2022. Derived image products include point cloud, ortho-mosaiced multispectral image, a digital surface model (DSM) using the structure from motion (SfM) technique, and a normalized difference vegetation index (NDVI) map. Unprocessed and processed data products are included in this package (processing levels 0-2). Data and metadata are provided as text (*.txt, *.json, *hdr,), tabular (*.dat, *.csv), point cloud (*.laz), Cloud Optimized GeoTIFF (COG, *.tif), and image (*.jpg, *.tif, *png) formats.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

UAS remote sensing (3DR SOLO platform): multispectral reflectance, canopy height model, normalized difference vegetation index, Seward Peninsula, Alaska, 2021

Airborne remote sensing data collected using a Parrot Sequoia+ multispectral sensor installed on a 3DR SOLO unoccupied aerial system (UAS) - operated by the Terrestrial Ecosystem Science Technology group at Brookhaven National Laboratory. This package includes data from 10 flights flown over the NGEE-Arctic Council Mile Marker (MM) 71, Kougarok MM64, and Teller MM27 sites on the Seward Peninsula, Alaska, in August 2021. Derived image products include point cloud, ortho-mosaiced multispectral image, ortho-mosaiced RGB image, a digital surface model (DSM) using the structure from motion (SfM) technique, a canopy height model (CHM), and a normalized difference vegetation index (NDVI) map. Unprocessed and processed data products are included in this package (processing levels 0-2). Data and metadata are provided as text (*.txt, *.json, *hdr,), tabular (*.dat, *.csv), point cloud (*.laz), Cloud Optimized GeoTIFF (COG, *.tif), and image (*.jpg, *.tif, *png) formats. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2004

Realtime Absolute Orientation with Outlier Rejection for Visual Odometry and Mesh Merging

This paper addresses a high level spatio-temporal problem, namely "absolute orientation", which arises in visual-odometry (using stereo), or registering two models created by different Structure from Motion (SFM) reconstructions. We compare the very popular method due to Horn using quaternions and our own independently derive method using the orthonormal rotation matrix R. We also introduce a novel approach for outlier rejection using spectral clustering.

Bandari, Esfandiar↗

NASA NTRS2016

Automated Segmentation and Classification of Coral using Fluid Lensing from Unmanned Airborne Platforms

In recent years, there has been a growing interest among biologists in monitoring the short and long term health of the world's coral reefs. The environmental impact of climate change poses a growing threat to these biologically diverse and fragile ecosystems, prompting scientists to use remote sensing platforms and computer vision algorithms to analyze shallow marine systems. In this study, we present a novel method for performing coral segmentation and classification from aerial data collected from small unmanned aerial vehicles (sUAV). Our method uses Fluid Lensing algorithms to remove and exploit strong optical distortions created along the air-fluid boundary to produce cm-scale resolution imagery of the ocean floor at depths up to 5 meters. A 3D model of the reef is reconstructed using structure from motion (SFM) algorithms, and the associated depth information is combined with multidimensional maximum a posteriori (MAP) estimation to separate organic from inorganic material and classify coral morphologies in the Fluid-Lensed transects. In this study, MAP estimation is performed using a set of manually classified 100 x 100 pixel training images to determine the most probable coral classification within an interrogated region of interest. Aerial footage of a coral reef was captured off the coast of American Samoa and used to test our proposed method. 90 x 20 meter transects of the Samoan coastline undergo automated classification and are manually segmented by a marine biologist for comparison, leading to success rates as high as 85%. This method has broad applications for coastal remote sensing, and will provide marine biologists access to large swaths of high resolution, segmented coral imagery.

algorithms segments classifications↗

NASA NTRS2017

Structure from Motion Photogrammetry and Micro X-Ray Computed Tomography 3-D Reconstruction Data Fusion for Non-Destructive Conservation Documentation of Lunar Samples

Our team is developing a modern, cross-disciplinary approach to documentation and preservation of astromaterials, specifically lunar and meteorite samples stored at the Johnson Space Center (JSC) Lunar Sample Laboratory Facility. Apollo Lunar Sample 60639, collected as part of rake sample 60610 during the 3rd Extra-Vehicular Activity of the Apollo 16 mission in 1972, served as the first NASA-preserved lunar sample to be examined by our team in the development of a novel approach to internal and external sample visualization. Apollo Sample 60639 is classified as a breccia with a glass-coated side and pristine mare basalt and anorthosite clasts. The aim was to accurately register a 3-dimensional Micro X-Ray Computed Tomography (XCT)-derived internal composition data set and a Structure-From-Motion (SFM) Photogrammetry-derived high-fidelity, textured external polygonal model of Apollo Sample 60639. The developed process provided the means for accurate, comprehensive, non-destructive visualization of NASA's heritage lunar samples. The data products, to be ultimately served via an end-user web interface, will allow researchers and the public to interact with the unique heritage samples, providing a platform to "slice through" a photo-realistic rendering of a sample to analyze both its external visual and internal composition simultaneously.

Beaulieu, K. R.↗

NASA NTRS2015

Automated Segmentation and Classification of Coral Using Fluid Lensing from Unmanned Airborne Platforms

In recent years, there has been a growing interest among biologists in monitoring the short and long term health of the worlds coral reefs. The environmental impact of climate change poses a growing threat to these biologically diverse and fragile ecosystems, prompting scientists to use remote sensing platforms and computer vision algorithms to analyze shallow marine systems. In this study, we present a novel method for performing coral segmentation and classification from aerial data collected from small unmanned aerial vehicles (sUAV). Our method uses Fluid Lensing algorithms to remove and exploit strong optical distortions created along the air-fluid boundary to produce cm-scale resolution imagery of the ocean floor at depths up to 5 meters. A 3D model of the reef is reconstructed using structure from motion (SFM) algorithms, and the associated depth information is combined with multidimensional maximum a posteriori (MAP) estimation to separate organic from inorganic material and classify coral morphologies in the Fluid-Lensed transects. In this study, MAP estimation is performed using a set of manually classified 100 x 100 pixel training images to determine the most probable coral classification within an interrogated region of interest. Aerial footage of a coral reef was captured off the coast of American Samoa and used to test our proposed method. 90 x 20 meter transects of the Samoan coastline undergo automated classification and are manually segmented by a marine biologist for comparison, leading to success rates as high as 85. This method has broad applications for coastal remote sensing, and will provide marine biologists access to large swaths of high resolution, segmented coral imagery.

algorithms segments classifications↗

NASA NTRSDate not supplied

Using Machine Learning to Infer Material Properties of Debris Fragments from X-ray Images in the DebriSat Project

The DebriSat project is a collaboration effort with the NASA Orbital Debris Program Office, the U.S. Space Force Space Systems Command Center, The Aerospace Corporation, and the University of Florida. To date, over 200,000 fragments from this ground-based, hypervelocity impact experiment have been collected, and processing is underway to determine their physical characteristics, such as material, shape, color, characteristic length, and average cross-sectional area. The x-ray process is primarily used to identify the location of the fragments and estimated size for extraction, so that these physical characteristics can be assessed. This paper proposes a machine learning-based approach to characterize materials from x-ray images of debris fragments embedded in soft-catch foam used in the DebriSat project. The novel methodology discussed in this paper will highlight the use of x-ray imagery data to characterize these fragments without extraction or a human-in-the-loop. Both supervised and unsupervised machine learning techniques are utilized with this approach to infer the physical parameters of the fragments embedded in the soft-catch foam panels used in the impact experiment based on x-ray images of the foam panels. Additionally, 3D reconstructions of the extracted fragments are created with images taken from two different angles using the structure from motion (SfM) method. The characteristic lengths and shape from the 3D reconstruction, alongside the physical characteristics of the debris, are used in the inference of the material type. To develop and test the approach, a dataset of x-ray images of debris fragments of varying sizes and materials is collected. Supervised learning methods such as convolutional neural networks (CNNs), support vector machines (SVM), decision trees, and random forest classifiers are used due to the high-dimensional feature spaces of the debris and nonlinear decision boundaries for material categorization. Given the limited pre-labeled data of embedded debris materials smaller than 10 mm, unsupervised machine learning techniques such as clustering algorithms and autoencoders are used, in addition to supervised learning methods. The clustering algorithms group similar fragments together based on their physical properties, and autoencoders reduce the dimensionality of the x ray images and extract relevant features. The performance of the proposed approach's is analyzed using a range of statistical methods, including confusion matrices, receiver operating characteristic curves, and precision-recall curves. The results are compared with those obtained using a baseline approach that relies on manual identification and classification of debris fragments. To evaluate the effectiveness of different machine learning methods, statistical tests such as t-tests, ANOVA, and cross-validation are performed, comparing the performance of CNNs, SVMs, clustering algorithms, and autoencoders. Additional analysis needs to be conducted to identify any sources of bias or variability that may affect the results, such as variations in imaging conditions or fragmentation patterns. Other topics explored are limitations, refinements, and the potential use of semi-supervised learning techniques, such as self-training to label unlabeled datasets and co-training using x-ray images taken from two different angles as two different models.

Saik Anam Siam↗

NASA NTRS2023

Using Machine Learning to Infer Material Properties of Debris Fragments from X-ray Images in the DebriSat Project

The DebriSat project is a collaboration effort with the NASA Orbital Debris Program Office, the U.S. Space Force Space Systems Command Center, The Aerospace Corporation, and the University of Florida. To date, over 200,000 fragments from this ground-based, hypervelocity impact experiment have been collected, and processing is underway to determine their physical characteristics, such as material, shape, color, characteristic length, and average cross-sectional area. The x-ray process is primarily used to identify the location of the fragments and estimated size for extraction, so that these physical characteristics can be assessed. This paper proposes a machine learning-based approach to characterize materials from x-ray images of debris fragments embedded in soft-catch foam used in the DebriSat project. The novel methodology discussed in this paper will highlight the use of x-ray imagery data to characterize these fragments without extraction or a human-in-the-loop. Both supervised and unsupervised machine learning techniques are utilized with this approach to infer the physical parameters of the fragments embedded in the soft-catch foam panels used in the impact experiment based on x-ray images of the foam panels. Additionally, 3D reconstructions of the extracted fragments are created with images taken from two different angles using the structure from motion (SfM) method. The characteristic lengths and shape from the 3D reconstruction, alongside the physical characteristics of the debris, are used in the inference of the material type. To develop and test the approach, a dataset of x-ray images of debris fragments of varying sizes and materials is collected. Supervised learning methods such as convolutional neural networks (CNNs), support vector machines (SVM), decision trees, and random forest classifiers are used due to the high-dimensional feature spaces of the debris and nonlinear decision boundaries for material categorization. Given the limited pre-labeled data of embedded debris materials smaller than 10 mm, unsupervised machine learning techniques such as clustering algorithms and autoencoders are used, in addition to supervised learning methods. The clustering algorithms group similar fragments together based on their physical properties, and autoencoders reduce the dimensionality of the x ray images and extract relevant features. The performance of the proposed approach's is analyzed using a range of statistical methods, including confusion matrices, receiver operating characteristic curves, and precision-recall curves. The results are compared with those obtained using a baseline approach that relies on manual identification and classification of debris fragments. To evaluate the effectiveness of different machine learning methods, statistical tests such as t-tests, ANOVA, and cross-validation are performed, comparing the performance of CNNs, SVMs, clustering algorithms, and autoencoders. Additional analysis needs to be conducted to identify any sources of bias or variability that may affect the results, such as variations in imaging conditions or fragmentation patterns. Other topics explored are limitations, refinements, and the potential use of semi-supervised learning techniques, such as self-training to label unlabeled datasets and co-training using x-ray images taken from two different angles as two different models.

Saik Anam Siam↗

DOE OSTI2021

A New Method for High Resolution Surface Change Detection: Data Collection and Validation of Measurements from UAS at the Nevada National Security Site, Nevada, USA

The use of uncrewed aerial systems (UAS) increases the opportunities for detecting surface changes in remote areas and in challenging terrain. Detecting surface topographic changes offers an important constraint for understanding earthquake damage, groundwater depletion, effects of mining, and other events. For these purposes, changes on the order of 5–10 cm are readily detected, but sometimes it is necessary to detect smaller changes. An example is the surface changes that result from underground explosions, which can be as small as 3 cm. Previous studies that described change detection methodologies were generally not aimed at detecting sub-5-cm changes. Additionally, studies focused on high-fidelity accuracy were either computationally modeled or did not fully provide the necessary examples to highlight the usability of these workflows. Detecting changes at this threshold may be critical in certain applications, such as global security research and monitoring for high-consequence natural hazards, including landslides. Here we provide a detailed description of the methodology we used to detect 2–3 cm changes in an important applied research setting—surface changes related to underground explosions. This methodology improves the accuracy of change detection data collection and analysis through the optimization of pre-field planning, surveying, flight operations, and post-processing the collected data, all of which are critical to obtaining the highest output data resolution possible. We applied this methodology to a field study location, collecting 1.4 Tb of images over the course of 30 flights, and location data for 239 ground control points (GCPs). We independently verified changes with orthoimagery, and found that structure-from-motion, software-reported root mean square errors (RMSEs) for both control and check points underestimated the actual error. We found that 3 cm changes are detectable with this methodology, thereby improving our knowledge of a rock’s response to underground explosions.

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

Mapping snow depth and volume at the alpine watershed scale from aerial imagery using Structure from Motion

Time series mapping of snow volume in the mountains at global scales and at resolutions needed for water resource management is an unsolved challenge to date. Snow depth mapping by differencing surface elevations from airborne lidar is a mature measurement approach filling the observation gap operationally in a few regions, primarily in mountain headwaters in the Western United States. The same concept for snow depth retrieval from stereo- or multi-view photogrammetry has been demonstrated, but these previous studies had limited ability to determine the uncertainties of photogrammetric snow depth at the basin scale. For example, assessments used non-coincident or discrete points for reference, masked out vegetation, or compared a subset of the fully snow-covered study domain. Here, using a unique data set with simultaneously collected airborne data, we compare snow depth mapped from multi-view Structure from Motion photogrammetry to that mapped by lidar at multiple resolutions over an entire mountain basin (300 km 2 ). After excluding reconstruction errors (negative depths), SfM had lower snow-covered area (~27%) and snow volume (~16%) compared to lidar. The reconstruction errors were primarily in areas with vegetation, shallow snow (< 1 m), and steep slopes (> 60°C). Across the overlapping snow extent, snow depths compared well to lidar with similar mean values (< 0.03 m difference) and snow volume (± 5%) for output resolutions of 3 m and 50 m, and with a normalized median absolute deviation of 0.19 m. Our results indicate that photogrammetry from aerial images can be applied in the mountains but would perform best for deeper snowpacks above tree line.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

Multi-resolution Arctic Shrub Cover Dataset Derived from UAS and Airborne SfM and LiDAR (2013-2025)

We synthesized 177 unoccupied aerial system flights and 77 airborne flights across the Arctic and created a multi-resolution benchmark data of low-to-tall shrub fractional cover leveraging Structure-from-Motion and Light Detection and Ranging. The resulting dataset covered a total of 1899 km2 across Alaska, Western Canada, Sweden, and Siberian Arctic, including key sites from the Oro Arctic to the High Arctic. The dataset is organized into 6 primary data collection directories (“Abisko,” “AWI,” “ERE,” “Fairbanks,” “NGEE,” “Toolik”), each containing site and flight subdirectories. Flight directories include shrub cover rasters (*.tifs) at 1 m, 5 m, and 30 m resolution, the canopy height model at 1 m resolution (*.tifs), and a bounding box *.kml file. For the AWI, Abisko, NGEE, and Fairbanks collections, we also include the GCC raster at 1 m resolution (*.tif). Files are organized by Collection > Site > Flight Name > Data Files. Flight rasters are in the local UTM zone and the .kml files are in the geographic coordinate system EPSG 4326. We also include a .csv file that details the source datasets for every flight. The Next-Generation Ecosystem Experiments in the Arctic (NGEE Arctic) project is a research effort to reduce uncertainty in the Department of Energy’s Energy Exascale Earth System Model (E3SM) by developing a predictive understanding of Arctic tundra ecosystems underlain by permafrost and to quantify feedbacks from the Arctic tundra to the Earth system. NGEE Arctic is supported by the Department of Energy's Office of Biological and Environmental Research. Over Phases 1–3, observations made by the NGEE Arctic team across a gradient of permafrost landscapes in Arctic Alaska improved the representation of tundra processes in the land surface component of E3SM (the E3SM Land Model, ELM). Model improvements emphasized unique aspects of permafrost environments and explored reductions in model complexity while retaining predictive power. The Arctic-informed ELM developed by NGEE Arctic has been used to make novel predictions on processes ranging from permafrost thaw to soil biogeochemical cycling to Earth system feedbacks associated with the unique characteristics of tundra plants. In Phase 4, the NGEE Arctic team is evaluating our new predictive understanding under novel conditions across the Arctic domain. In collaboration with partners at long-term pan-Arctic research sites we are examining whether an Arctic-informed ELM can faithfully simulate interactions among surface and subsurface processes at site, regional, and pan-Arctic scales. In turn, we are using variety of tools to dynamically extend and evaluate ELM inference, with an emphasis on data synthesis and pan-Arctic model evaluation, reintegration of code with an evolving E3SM, scaling across heterogeneous Arctic landscapes, and the appropriate representation of the impacts of increasingly frequent Arctic disturbances.

canopy height model↗