Engineering topics

Medeiros, Henry

Publications and source records attributed to Medeiros, Henry.

3 records

DOE OSTI2023

An Agenda for Multimodal Foundation Models for Earth Observation

Archives of remote sensing (RS) data are increasing swiftly as new sensing modalities with enhanced spatiotemporal resolution become operational. While promising new breakthroughs, the sheer volume of RS archives stretches the limits of human analysts and existing AI tools, as most models are: i) limited to single data modalities; ii) task-specific; iii) heavily reliant on labeled data. The emerging Foundation Models (FMs) have the potential to address these limitations. Trained on vast unlabeled datasets through self-supervised learning, FMs enable generic feature extraction that facilitate specialization to a wide variety of downstream tasks. This paper describes a vision towards an FM for multimodal Earth Observation data (FM4EO), discussing key building blocks and open challenges. We put particular emphasis on multimodal reasoning, a topic underexplored in EO. Our ultimate goal is a practical path toward FM4EO with capacity to unlock breakthroughs in few-shot learning scenarios, multimodal geographic knowledge integration, synthesis, and hypothesis generation.

Ambrozio Dias, Philipe↗

DOE OSTI2022

Semi-automated Design of Artificial Intelligence Earth Science Models

Prediction and observation of water cycles involve not only patterns isolated in space and time, but rather modeling complex spatio-temporal relationships across multiple sources of data and domains. For instance, Evapotranspiration (ET) and Leaf Area Indexes (LAI) are two critical components in DOE’s Energy Exascale Earth System Model (E3SM). Accurate assessments of ET and LAI are critical for understanding hydrological processes, deforestation, crop yield, and irrigation impacts. However, current ET estimates for global simulations are available at very coarse spatial resolution. They are usually derived from satellite data based on broad plant functional types (PFT), which fail to capture the fine-scale variations due to change in vegetation type across the globe. Within this context and in light of the data-model integration challenges highlighted in the EESSD Strategic Plan, the new era of AI model development for geosciences calls for data-driven methods that provide domain scientists with estimations of parameters such as PFT and LAI in an efficient, interpretable, and easy-to-operate manner.

Ambrozio Dias, Philipe↗

DOE OSTI2021

Semi-automated Design of Artificial Intelligence Earth Systems Models

Prediction and observation of water cycles at various scales involve not only patterns isolated in space and time, but also modeling of complex spatio-temporal relationships across multiple domains. For instance, evapotranspiration (ET) and leaf area indexes (LAI) are two parameters that are needed to accurately model and understand land-atmosphere processes. Accurate assessments of ET and LAI are critical for understanding hydrological processes, deforestation, crop yield, and irrigation impacts. However, ET estimates for global simulations are available at very coarse spatial resolution. They are usually derived from satellite data based on broad plant functional types (PFTs), which fail to capture fine-scale variations because of changes in vegetation type across the globe. Similarly LAI estimates have typically been derived from vegetation indices at global scales or estimated locally using physical models, both of which suffer from a range of uncertainties that impact model sensitivity. The new era of AI model development for Earth systems (ES) calls for data-driven methods that provide domain scientists with uncertainty-aware estimations of biophysical parameters such as ET and LAI in a generalizable, interpretable, and discoverable manner.

54 ENVIRONMENTAL SCIENCES↗