Hybrid physics-AI outperforms numerical weather prediction for extreme precipitation nowcasting
Not provided.
Meteorology & Atmospheric Sciences↗
Engineering topics
Vandal, Thomas
Publications and source records attributed to Vandal, Thomas.
Transfer Learning to Generate True Color Images from GOES-16
Along with scientific applications, Geostationary imagery is often used to learn about weather patterns through true color visualizations. NOAA/NASA's GOES-R series of satellites uses the advanced baseline imager with 16-bands which, unlike previous generations, does not include the green wavelength (500-565 nm) and hence cannot directly generate true color images. However, Himawari, Japan's geostationary satellite, uses a similar 16-band advanced Himawari imager that does include a green band (but missing cirrus). In this work, we show how transfer learning with convolutional neural networks can be applied across satellites to generate "virtual sensors". We apply this approach to generate a green band for GOES-16 and present near true color images.
transfer learning↗
Optical Flow for Intermediate Frame Interpolation of Multispectral Geostationary Satellite Data
Applications in areas such as weather tracking and modeling, ecosystem monitoring, wildfire detection, and land-cover change are heavily dependent on spatial and temporal resolutions of satellite observations. However, there are typically trade-offs between spatial and temporal resolutions in dataset selection. For instance, geostationary weather tracking satellites are designed to take snapshots many times throughout the day but sensor hardware limits data collection. In this work we tackle this limitation, developing a method for temporal upsampling of multi-spectral satellite imagery using optical flow video interpolation deep convolutional neural networks. The presented model, extends Super SloMo (SSM) from single optical flow estimates to multichannel where flows are computed per band. We apply this technique on 8 multi-spectral bands of NOAA/NASA's GOES-16 mesoscale dataset to temporally enhance full disk hemispheric snapshots from 15 minutes to 1 minute. Through extensive experimentation, we show SSM vastly outperforms the linear interpolation baseline and that multichannel optical flows improves performance on GOES-16. A visual analysis of optical flow vectors clearly identifies hurricanes and large-scale atmospheric dynamics. Furthermore, we discuss challenges and open questions related to optical flow and temporal interpolation of multispectral geostationary satellite imagery.
Optical Flow↗
Deep Learning and Uncertainty Quantification for Climate Resilience
Modeling and monitoring of earth’s processes through physical models and satellite observations at high resolutions is crucial for ensuring society’s ability to adapt to climate change. Deep learning (DL) has been shown to be a valuable tool for generating high resolution data, emulating physical models, and detecting weather patterns which can then be used to inform stakeholders and decision makers. However, both the data and model parameters contain substantial uncertainties that may alter users’ decisions. In this work we present two DL applications on high-resolution climate and satellite datasets using Bayesian neural networks to generate well calibrated uncertainty estimates.
Vandal, Thomas↗
GEONEX: Progressive Conditional Generative Adversarial Training Using Transfer learning
Obtaining accurate segmentation on large scale images is an open problem in deep learning. The main problem is the amount of labeled data that exists for large scale images. Traditionally, the common solution to this problem is to crop the large images into smaller images to increase the amount of available data and train a Conditional Generative Adversarial Network (CGAN). CGANs are currently the state of the art in image to image translation and provide better accuracy than the traditional method of training a encoder based conv-net architecture to minimize the loss at each pixel. This method can produce noisy and discontinuous images with inaccurate results. We seek to solve this problem by utilizing the concepts of transfer learning and progressive training to create a CGAN that can segment large scale images with a limited amount of labeled data. In transfer learning we recognize that many learned features are applicable to many classes from multiple domains. This introduces the concept of feature reusability, which is the basis for finetuning. Progressive training got its start in training models on the same images at different resolutions. In this work we instead train a GAN on increasing image scales by transferring the weights from the smaller scales to the larger scales. The learned features at the smaller scales are continually reused and applied to larger scales to create a CGAN that can perform accurate segmentation on large scale images. We apply this method to detect building footprints on very high-resolution overhead imagery (e.g Digital Globe and high resolution airborne platforms).
GEONEX↗