Search NASA⌕ Search

Engineering topics

Bowman, Jordan

Publications and source records attributed to Bowman, Jordan.

OReole-FM: successes and challenges toward billion-parameter foundation models for high-resolution satellite imagery

While the pretraining of Foundation Models (FMs) for remote sensing (RS) imagery is on the rise, models remain restricted to a few hundred million parameters. Scaling models to billions of parameters has been shown to yield unprecedented benefits including emergent abilities, but requires data scaling and computing resources typically not available outside industry R&D labs. In this work, we pair high-performance computing resources including Frontier supercomputer, America's first exascale system, and high-resolution optical RS data to pretrain billion-scale FMs. Our study assesses performance of different pretrained variants of vision Transformers across image classification, semantic segmentation and object detection benchmarks, which highlight the importance of data scaling for effective model scaling. Moreover, we discuss construction of a novel TIU pretraining dataset, model initialization, with data and pretrained models intended for public release. By discussing technical challenges and details often lacking in the related literature, this work is intended to offer best practices to the geospatial community toward efficient training and benchmarking of larger FMs.

Ambrozio Dias, Philipe↗

A Science Gateway for the Repeatable Analysis of Machine Learning Predicted Gravity Anomalies

In recent years, deep learning has become an increasingly popular alternative for modeling in geoscience applications due to its scalability and efficiency. However, the interpretability, compute, data volume, and hyperparameter tuning requirements of deep learning models make development and monitoring difficult. Furthermore, model explainability and communicating results obtained by these models to users or domain experts is a challenge, as domain experts in geoscience also need to have a deep understanding of how those models function in order to support their scientific works. Here, we describe a science gateway and machine learning pipeline for predicting gravity anomalies from geophysical data. The gateway, built on open-source technologies, provides a holistic view of the pipeline through interactive visualizations aimed at enabling efficient exploratory data analysis. The repeatability, reproducibility, and monitoring capabilities of this overall system allow us to iterate and analyze at scale. Using this pipeline and gateway, we can repeatedly produce accurate high-resolution gravity anomaly datasets. By describing the underlying technologies, implementation, and results, here we provide a foundation for the broader adoption of science gateways into cross-cutting geoscience and machine learning research projects as a means to improve the scientific discovery and collaboration in the geophysics and computational sciences community.

58 GEOSCIENCES↗

Towards Geospatial Knowledge Graph Infused Neuro-Symbolic AI for Remote Sensing Scene Understanding

Deep learning has proven its effectiveness in numerous tasks for remote sensing scene understanding. However there is an increasing interest to explore fusion of domain-specific background information to the deep neural network to further improve its performance. Remote sensing researchers are also working towards developing models that generalize and adapt to multiple applications. Generalization challenges coupled with the scarcity of large corpora of high-quality noise-free labelled data, have together fueled an interest for leveraging background information. Knowledge graphs serve as excellent choice to represent domain-specific information in a structured, standardized and extensible manner. Integrating symbolic knowledge representations in the form of Knowledge Graph Embedding (KGE) to perform neuro-symbolic reasoning is an emerging research direction promising significant impacts. This vision paper seeks to position ideas and provoke early thoughts toward advancing neuro-symbolic artificial intelligence in the context of geospatial challenges. Specifically, it conceptualizes and elaborates on an architecture for infusing geospatial knowledge from knowledge graph in a deep neural network pipeline. As guiding case studies - land-use land-cover classification, object detection and instance segmentation can benefit from infusing spatio-contextual information with remote sensing imagery. The discussion further reflects on and articulates the challenges and explainable AI opportunities anticipated when scaling and maintaining large-scale geospatial knowledge graphs.

Potnis, Abhishek↗

Scaling Automatic Vector Data Alignment to Satellite Imagery

Given the tremendous volume of accessible Earth Observation (EO) data, there is a need to develop scalable Geospatial Artificial Intelligence (GeoAI) solutions for time-sensitive applications. Scalability in this context refers to rapidly processing large-scale EO data using high performance computing resources. Accurate mapping of the built environment from remote sensing (RS) imagery has been one of the crucial components in GeoAI workflows for a wide spectrum of humanitarian applications. Derived vector data of built environment is often leveraged for disaster preparedness and response activities. However, factors such as differences in ortho-rectification, atmospheric conditions and human error, results in spatial misalignment between vector data and the timely available RS imagery. Model training for downstream tasks such as object detection, change analysis, etc., is negatively impacted due to such spatial misalignment. Although there has been progress towards automatic alignment of vector data, the lack of scalability remains an open research challenge. This paper proposes to leverage parallel computing to optimize an automatic vector data alignment workflow. It further employs CPU-level multi-core parallelism for improving the performance of the workflow for scalable built environment mapping. We report observations and discuss findings from the preliminary experiments performed on the Summit Supercomputer.

Potnis, Abhishek↗

UAS Edge Computing of Energy Infrastructure Damage Assessment

Energy infrastructure assessments are needed within 72 hours of natural disasters, and previous data collection methods have proven too slow. We demonstrate a scalable end-to-end solution using a prototype unmanned aerial system that performs on-the-edge detection, classification (i.e., damaged or undamaged), and geo-location of utility poles. The prototype is suitable for disaster response because it requires no local communication infrastructure and is capable of autonomous missions. Collections before, during, and after Hurricane Ida in 2021 were used to test the system. The system delivered an F1 score of 0.65 operating with a 2.7 s/frame processing speed with the YOLOv5 large model and an F1 score of 0.55 with a 0.48 s/frame with the YOLOv5 small model. Geo-location uncertainty in the bottom half of the frame was ~8 m, mostly driven by error in camera pointing measurement. With additional training data to improve performance and detect additional types of features, a fleet of similar drones could autonomously collect actionable post-disaster data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Post-Invasion Damage Assessment: Ukraine’s Crop Storage Infrastructure

The Yale School of Public Health's (YSPH) Humanitarian Research Lab (HRL), working in collaboration with Oak Ridge National Labs (ORNL), concludes with high confidence that approximately 21.5% of Ukraine’s estimated 57 million tonnes of crop storage capacity has been impacted by conflict since Russia’s full-scale invasion of Ukraine in February 2022. Intentional and indiscriminate targeting of crop storage infrastructure can constitute a war crime and a crime against humanity under international law.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Few-shot Learning for Post-disaster Structure Damage Assessment

Automating post-disaster damage assessment with remote sensing data is critical for faster surveys of structures impacted by natural disasters. One significant obstacle to training state-of-the-art deep neural networks to support this automation is that large quantities of labelled data are often required. However, obtaining those labels is particularly unrealistic to support post-disaster damage assessment in a timely manner. Few-shot learning methods could help to mitigate this by reducing the amount of labelled data required to successfully train a model while achieving satisfactory results. To this end, we explore a feature reweighting method to the YOLOv3 object detection architecture to achieve few-shot learning of damage assessment models on the xBD dataset. Our results show that the feature reweighting approach yield improved mAP over the baseline with significantly fewer labelled samples. In addition, we use t-SNE to analyze the class-specific reweighting vectors generated by the reweighting module in order to evaluate their inter-class and intra-class similarity. We find that the vectors form clusters based on class, and that these clusters overlap with visually similar classes. Those results show the potential to employ this few-shot learning strategy for rapid damage assessment with post-event remote sensing images.

Bowman, Jordan↗