SEARCH · Search NASA
Results for “CLOUD MASK”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Cloud Mask Intercomparison eXercise (CMIX): An evaluation of cloud masking algorithms for Landsat 8 and Sentinel-2
Cloud cover is a major limiting factor in exploiting time-series data acquired by optical spaceborne remote sensing sensors. Multiple methods have been developed to address the problem of cloud detection in satellite imagery and a number of cloud masking algorithms have been developed for optical sensors but very few studies have carried out quantitative intercomparison of state-of-the-art methods in this domain. This paper summarizes results of the first Cloud Masking Intercomparison eXercise (CMIX) conducted within the Committee Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). CEOS is the forum for space agency coordination and cooperation on Earth observations, with activities organized under working groups. CMIX, as one such activity, is an international collaborative effort aimed at intercomparing cloud detection algorithms for moderate-spatial resolution (10–30 m) spaceborne optical sensors. The focus of CMIX is on open and free imagery acquired by the Landsat 8 (NASA/USGS) and Sentinel-2 (ESA) missions. Ten algorithms developed by nine teams from fourteen different organizations representing universities, research centers and industry, as well as space agencies (CNES, ESA, DLR, and NASA), are evaluated within the CMIX. Those algorithms vary in their approach and concepts utilized which were based on various spectral properties, spatial and temporal features, as well as machine learning methods. Algorithm outputs are evaluated against existing reference cloud mask datasets. Those datasets vary in sampling methods, geographical distribution, sample unit (points, polygons, full image labels), and generation approaches (experts, machine learning, sky images). Overall, the performance of algorithms varied depending on the reference dataset, which can be attributed to differences in how the reference datasets were produced. The algorithms were in good agreement for thick cloud detection, which were opaque and had lower uncertainties in their identification, in contrast to thin/semi-transparent clouds detection. Not only did CMIX allow identification of strengths and weaknesses of existing algorithms and potential areas of improvements, but also the problems associated with the existing reference datasets. The paper concludes with recommendations on generating new reference datasets, metrics, and an analysis framework to be further exploited and additional input datasets to be considered by future CMIX activities.
Toward Physics-informed Neural Networks for 3D Multi-layer Cloud Mask Reconstruction
Three-dimensional (3D) cloud retrievals are critical for understanding their impact on climate and other applications such as aviation safety, weather prediction, and remote sensing. However, obtaining high-resolution and accurate vertical representation of clouds remains unsolved due to the limitations imposed by satellite instrumentation, viewing conditions, and the complexity of cloud dynamics. Cloud masks are essential for comprehending various cloud vertical properties, but deriving accurate 3D cloud masks from 2D satellite imagery data is a challenging task. To tackle these challenges, we introduce a physics-informed loss function for training deep learning models that can extend 2D cloud images into 3D cloud masks. The proposed loss, called CloudMask Loss, is composed of two domain knowledge-informed loss terms: one for evaluating cloud position and thickness, and the other for measuring the number of layers. By combining these loss terms, we improve the trainability of the deep learning models for more accurate and meaningful results. We apply the proposed loss function to different neural networks and demonstrate significant improvements in multi-layer cloud mask reconstruction. Utilizing the same neural network architecture, our proposed loss outperforms standard binary crossentropy loss in terms of multi-layer cloud classification accuracy, number of layers accuracy, and thickness mean absolute error (MAE). The proposed loss function can be readily integrated into various neural network architectures, resulting in substantial performance gains in 3D cloud mask generation.
Improved Thin Cirrus and Terminator Cloud Detection in CERES Cloud Mask
Thin cirrus clouds account for about 20-30% of the total cloud coverage and affect the global radiation budget by increasing the Earth's albedo and reducing infrared emissions. Thin cirrus, however, are often underestimated by traditional satellite cloud detection algorithms. This difficulty is caused by the lack of spectral contrast between optically thin cirrus and the surface in techniques that use visible (0.65 micron ) and infrared (11 micron ) channels. In the Clouds and the Earth s Radiant Energy System (CERES) Aqua Edition 1 (AEd1) and Terra Edition 3 (TEd3) Cloud Masks, thin cirrus detection is significantly improved over both land and ocean using a technique that combines MODIS high-resolution measurements from the 1.38 and 11 micron channels and brightness temperature differences (BTDs) of 11-12, 8.5-11, and 3.7-11 micron channels. To account for humidity and view angle dependencies, empirical relationships were derived with observations from the 1.38 micron reflectance and the 11-12 and 8.5-11 micron BTDs using 70 granules of MODIS data in 2002 and 2003. Another challenge in global cloud detection algorithms occurs near the day/night terminator where information from the visible 0.65 micron channel and the estimated solar component of 3.7 micron channel becomes less reliable. As a result, clouds are often underestimated or misidentified near the terminator over land and ocean. Comparisons between the CLAVR-x (Clouds from Advanced Very High Resolution Radiometer [AVHRR]) cloud coverage and Geoscience Laser Altimeter System (GLAS) measurements north of 60 N indicate significant amounts of missing clouds from CLAVR-x because this part of the world was near the day/night terminator viewed by AVHRR. Comparisons between MODIS cloud products (MOD06) and GLAS in the same region also show similar difficulties with MODIS cloud retrievals. The consistent detection of clouds through out the day is needed to provide reliable cloud and radiation products for CERES and other research efforts involving the modeling of clouds and their interaction with the radiation budget.
Development of a Consistent Cross-Platform GEO-Satellite Cloud Mask to Support CERES
Geostationary satellites provide continuous cloud and meteorological observations over a fixed portion of the Earth’s surface, allowing them to monitor the movement of storm systems and their diurnal variation. For climate studies, geostationary observations provide valuable insight of cloud formation and evolution and how they influence the Earth’s radiation budget. The Geo-Satellite Edition 4 cloud mask (GEO Ed4) is used operationally in NASA’s Cloud and Earth’s Radiant Energy System (CERES) project to help account for diurnal variations in cloudiness on Earth’s radiation budget. The Ed4 cloud mask was applied to imager data on MSG (MeteoSat Second Generation), Himawari, GOES-West, and GOES-East satellites using as much spectral information as possible. That approach led to some discontinuities when a more modern satellite replaced an older satellite with less spectral information. A different strategy is being investigated for the CERES GEO Ed5 cloud mask that only uses spectral channels common on every satellite. Thus, a 3-channel (0.6, 3.9, 11 µm) algorithm for daytime cloud detection, and a 2-channels (3.9 and 11 µm) algorithm for nighttime cloud detection have been implemented and tested for Ed5. The goal of Ed5 cloud mask is to achieve global cloud amount consistency across five geo-satellites, and a smooth transition from an old satellite to a new satellite over each geo-location. This paper compares cloud mask results between Ed4 and a preliminary Ed5 version over the GOES-East region, using GOES-8, GOES-13, and GOES-16 satellites, each with a different spectral channel complement. Instantaneous and monthly regional mean inter-comparisons are performed and evaluated with CALIPSO cloud products to assess the accuracy and consistency of the Ed5 approach relative to that taken in Ed4. Results will be presented and discussed at the conference.
New Neural Network Cloud Mask Algorithm Based on Radiative Transfer Simulations
Cloud detection and screening constitute critically important first steps required to derive many satellite data products. Traditional threshold-based cloud mask algorithms require a complicated design process and fine tuning for each sensor, and they have difficulties over areas partially covered with snow/ice. Exploiting advances in machine learning techniques and radiative transfer modeling of coupled environmental systems, we have developed a new, threshold-free cloud mask algorithm based on a neural network classifier driven by extensive radiative transfer simulations. Statistical validation results obtained by using collocated CALIOP and MODIS data show that its performance is consistent over different ecosystems and significantly better than the MODIS Cloud Mask (MOD35 C6) during the winter seasons over snow-covered areas in the mid-latitudes. Simulations using a reduced number of satellite channels also show satisfactory results, indicating its flexibility to be configured for different sensors. Comparedto threshold-based methods and previous machine-learning approaches, this new cloud mask (i) does not rely on thresholds, (ii) needs fewer satellite channels, (iii) has superior performance during winter seasons in mid-latitude areas, and (iv) can easily be applied to different sensors.
The EOS CERES Global Cloud Mask
To detect long-term climate trends, it is essential to produce long-term and consistent data sets from a variety of different satellite platforms. With current global cloud climatology data sets, such as the International Satellite Cloud Climatology Experiment (ISCCP) or CLAVR (Clouds from Advanced Very High Resolution Radiometer), one of the first processing steps is to determine whether an imager pixel is obstructed between the satellite and the surface, i.e., determine a cloud 'mask.' A cloud mask is essential to studies monitoring changes over ocean, land, or snow-covered surfaces. As part of the Earth Observing System (EOS) program, a series of platforms will be flown beginning in 1997 with the Tropical Rainfall Measurement Mission (TRMM) and subsequently the EOS-AM and EOS-PM platforms in following years. The cloud imager on TRMM is the Visible/Infrared Sensor (VIRS), while the Moderate Resolution Imaging Spectroradiometer (MODIS) is the imager on the EOS platforms. To be useful for long term studies, a cloud masking algorithm should produce consistent results between existing (AVHRR) data, and future VIRS and MODIS data. The present work outlines both existing and proposed approaches to detecting cloud using multispectral narrowband radiance data. Clouds generally are characterized by higher albedos and lower temperatures than the underlying surface. However, there are numerous conditions when this characterization is inappropriate, most notably over snow and ice of the cloud types, cirrus, stratocumulus and cumulus are the most difficult to detect. Other problems arise when analyzing data from sun-glint areas over oceans or lakes over deserts or over regions containing numerous fires and smoke. The cloud mask effort builds upon operational experience of several groups that will now be discussed.
Progress Towards a Common CERES Cloud Mask Algorithm for MODIS and VIIRS: Evaluation using CALIOP Data
For over two decades The Clouds and the Earth’s Radiant Energy System (CERES) project has endeavored to produce a long-term global climate data record for detecting changes in the Earth’s radiation budget and to improve understanding of how clouds contribute to those changes. CERES incorporates cloud information derived from passive narrowband satellite imaging radiometers, and over the course of the project different instruments have contributed to this effort, namely the Moderate Resolution Imaging Spectroradiometer (MODIS) and the Visible Infrared Imaging Radiometer Suite (VIIRS).Cloud properties have been derived from MODIS on the Aqua satellite for CERES since 2002,and VIIRS on the NOAA-20 satellite launched in 2017 will continue the cloud record once Aqua-MODIS reaches the end of its operational lifetime. However, MODIS and VIIRS have different spectral capabilities and characteristics which complicates our ability to seamlessly transition the record from MODIS to VIIRS. This study evaluates some of the recent algorithm modifications incorporated by the CERES Cloud Working Group towards developing a unified cloud mask algorithm for MODIS and VIIRS that utilizes a set of spectral bands common to both instruments. Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) data products are used to evaluate MODIS and VIIRS regional cloud fraction estimates and identify areas where improvements can be made. The cloud mask algorithm relies on computed estimates of cloud-free top-of-atmosphere radiances to differentiate cloudy and cloud-free imager pixels. The unified cloud mask algorithm currently under development uses a reduced number of spectral bands common to both MODIS and VIIRS and so relies on the accuracy of these simulated cloud-free radiances more heavily than predecessor algorithms, i.e., the CERES Edition 4 cloud mask, which use as much spectral information as available on each imager with less consideration for cross-platform consistency. Our validation strategy using CALIOP data is also described. The CALIOP observations are critical for assessing cloud detection accuracies and for independently confirming cloud-free conditions which enables a more robust evaluation of the simulated cloud-free radiances for biases due to factors such as view angle and the spectral dependence of water vapor absorption. These effects are known to differ for the two satellite instruments. Comparisons of MODIS and VIIRS cloud fractions are presented in context with estimates from CALIOP and from the CERES Edition 4 cloud mask algorithm to gauge current progress in developing accurate and consistent MODIS and VIIRS cloud properties for CERES.
Implementation on Landsat Data of a Simple Cloud Mask Algorithm Developed for MODIS Land Bands
This letter assesses the performance on Landsat-7 images of a modified version of a cloud masking algorithm originally developed for clear-sky compositing of Moderate Resolution Imaging Spectroradiometer (MODIS) images at northern mid-latitudes. While data from recent Landsat missions include measurements at thermal wavelengths, and such measurements are also planned for the next mission, thermal tests are not included in the suggested algorithm in its present form to maintain greater versatility and ease of use. To evaluate the masking algorithm we take advantage of the availability of manual (visual) cloud masks developed at USGS for the collection of Landsat scenes used here. As part of our evaluation we also include the Automated Cloud Cover Assesment (ACCA) algorithm that includes thermal tests and is used operationally by the Landsat-7 mission to provide scene cloud fractions, but no cloud masks. We show that the suggested algorithm can perform about as well as ACCA both in terms of scene cloud fraction and pixel-level cloud identification. Specifically, we find that the algorithm gives an error of 1.3% for the scene cloud fraction of 156 scenes, and a root mean square error of 7.2%, while it agrees with the manual mask for 93% of the pixels, figures very similar to those from ACCA (1.2%, 7.1%, 93.7%).
An Automatic Cloud Mask Algorithm Based on Time Series of MODIS Measurements
Quality of aerosol retrievals and atmospheric correction depends strongly on accuracy of the cloud mask (CM) algorithm. The heritage CM algorithms developed for AVHRR and MODIS use the latest sensor measurements of spectral reflectance and brightness temperature and perform processing at the pixel level. The algorithms are threshold-based and empirically tuned. They don't explicitly address the classical problem of cloud search, wherein the baseline clear-skies scene is defined for comparison. Here, we report on a new CM algorithm which explicitly builds and maintains a reference clear-skies image of the surface (refcm) using a time series of MODIS measurements. The new algorithm, developed as part of the Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm for MODIS, relies on fact that clear-skies images of the same surface area have a common textural pattern, defined by the surface topography, boundaries of rivers and lakes, distribution of soils and vegetation etc. This pattern changes slowly given the daily rate of global Earth observations, whereas clouds introduce high-frequency random disturbances. Under clear skies, consecutive gridded images of the same surface area have a high covariance, whereas in presence of clouds covariance is usually low. This idea is central to initialization of refcm which is used to derive cloud mask in combination with spectral and brightness temperature tests. The refcm is continuously updated with the latest clear-skies MODIS measurements, thus adapting to seasonal and rapid surface changes. The algorithm is enhanced by an internal dynamic land-water-snow classification coupled with a surface change mask. An initial comparison shows that the new algorithm offers the potential to perform better than the MODIS MOD35 cloud mask in situations where the land surface is changing rapidly, and over Earth regions covered by snow and ice.
HyspIRI Cloud Mask Detection Algorithm Theoretical Basis Document
The Hyperspectral Infrared Imager (HyspIRI) satellite includes a thermal infrared (TIR) multispectral scanner with seven spectral bands in the thermal infrared (TIR) between 7 and 12 μm and one band in the mid-infrared between 3 and 5 μm designed to measure hot targets. The TIR bands have a NEΔT of <0.2 K at 300K and all bands have a spatial scale of 60 m. The two primary Level-2 products that will be generated by HyspIRI TIR data are land surface temperature and emissivity. A critical aspect of minimizing uncertainties in these products and in generating global Level-3 products is an accurate and reliable cloud mask algorithm. This document describes the methodology behind developing the HyspIRI Cloud Mask Algorithm (HyCMA), and challenges associated with cloud detection. The HyCMA will derive its heritage from previously well established cloud mask algorithms such as the Landsat ACCAA, MODIS MOD35 and AVHRR.
A Cloud Mask for AIRS
With the preliminary release of AIRS Level 1 and 2 data to the scientific community, there is a growing need for an accurate AIRS cloud mask for data assimilation studies and in producing products derived from cloud free radiances. Current cloud information provided with the AIRS data are limited or based on simplified threshold tests. A multispectral cloud detection approach has been developed for AIRS that utilizes the hyper-spectral capabilities to detect clouds based on specific cloud signatures across the short wave and long wave infrared window regions. This new AIRS cloud mask has been validated against the existing AIRS Level 2 cloud product and cloud information derived from MODIS. Preliminary results for both day and night applications over the continental U.S. are encouraging. Details of the cloud detection approach and validation results will be presented at the conference.
Forcing, Cloud Feedbacks, Cloud Masking, and Internal Variability in the Cloud Radiative Effect Satellite Record
Explore the source record for details and available documents.
Early Evaluation of the VIIRS Calibration, Cloud Mask and Surface Reflectance Earth Data Records
Surface reflectance is one of the key products fromVIIRS and as withMODIS, is used in developing several higherorder land products. The VIIRS Surface Reflectance (SR) Intermediate Product (IP) is based on the heritageMODIS Collection 5 product (Vermote, El Saleous, & Justice, 2002). The quality and character of surface reflectance depend on the accuracy of the VIIRS Cloud Mask (VCM), the aerosol algorithms and the adequate calibration of the sensor. The focus of this paper is the early evaluation of the VIIRS SR product in the context of the maturity of the operational processing system, the Interface Data Processing System (IDPS). After a brief introduction, the paper presents the calibration performance and the role of the surface reflectance in calibration monitoring. The analysis of the performance of the cloud mask with a focus on vegetation monitoring (no snow conditions) shows typical problems over bright surfaces and high elevation sites. Also discussed is the performance of the aerosol input used in the atmospheric correction and in particular the artifacts generated by the use of the Navy Aerosol Analysis and Prediction System. Early quantitative results of the performance of the SR product over the AERONET sites showthatwith the fewadjustments recommended, the accuracy iswithin the threshold specifications. The analysis of the adequacy of the SR product (Land PEATE adjusted version) in applications of societal benefits is then presented. We conclude with a set of recommendations to ensure consistency and continuity of the JPSS mission with the MODIS Land Climate Data Record.
Improvements in Near-Terminator and Nocturnal Cloud Masks using Satellite Image Data over the Atmospheric Radiation Measurement Sites
Cloud detection using satellite measurements presents a big challenge near the terminator where the visible (VIS; 0.65 (micro)m) channel becomes less reliable and the reflected solar component of the solar infrared 3.9-(micro)m channel reaches very low signal-to-noise ratio levels. As a result, clouds are underestimated near the terminator and at night over land and ocean in previous Atmospheric Radiation Measurement (ARM) Program cloud retrievals using Geostationary Operational Environmental Satellite (GOES) imager data. Cloud detection near the terminator has always been a challenge. For example, comparisons between the CLAVR-x (Clouds from Advanced Very High Resolution Radiometer (AVHRR)) cloud coverage and Geoscience Laser Altimeter System (GLAS) measurements north of 60 degrees N indicate significant amounts of missing clouds from AVHRR because this part of the world was near the day/night terminator viewed by AVHRR. Comparisons between MODIS cloud products and GLAS at the same regions also shows the same difficulty in the MODIS cloud retrieval (Pavolonis and Heidinger 2005). Consistent detection of clouds at all times of day is needed to provide reliable cloud and radiation products for ARM and other research efforts involving the modeling of clouds and their interaction with the radiation budget. To minimize inconsistencies between daytime and nighttime retrievals, this paper develops an improved twilight and nighttime cloud mask using GOES-9, 10, and 12 imager data over the ARM sites and the continental United States (CONUS).
DOE TRACER CSAPR2 Cell Tracking Cloud Mask
C-Band Scanning ARM Precipitation Radar, CF/Radial Formatted, Quality Controlled Tobac Tracking Cloud Mask
3D Cloud Masking Across A Broad Swath Using Multi-Angle Polarimetry and Deep Learning
Understanding the 3-dimensional structure of clouds is of crucial importance to modeling our changing climate. Active sensors, such as radar and lidar, provide accurate vertical cloud profiles, but are mostly restricted to along-track sampling. Passive sensors can capture a wide swath, but struggle to see beneath cloud tops. In essence, both types of products are restricted to two dimensions: as a cross-section in the active case, and an image in the passive case. However, multi-angle sensor configurations contain implicit information about 3D structure, due to parallax and atmospheric path differences. Extracting that implicit information can be challenging, requiring computationally expensive radiative transfer techniques. Machine learning, as an alternative, may be able to capture some of the complexity of a full 3D radiative transfer solution with significantly less computational expense. In this work, we make three contributions towards understanding 3D cloud structure from multi-angle polarimetry. First, we introduce a large-scale, publicly available dataset that fuses existing cloud products into a format more amenable to machine learning. This dataset treats multi-angle polarimetry as an input, and radar-based vertical cloud profiles as an output. Second, we describe and evaluate strong baseline machine learning models based that predict these profiles from the passive imagery. Notably, these models are trained only on center-swath labels, but can predict cloud profiles over the entire passive imagery swath. Third, we leverage the information-theoretic nature of machine learning to draw conclusions about the relative utility of various sensor configurations, including spectral channels, viewing angles, and polarimetry. These findings have implications for Earth-observing missions such as NASA’s Plankton, Aerosol, Cloud-ocean Ecosystem (PACE) and Atmosphere Observing System (AOS) missions, as well as in informing future applications of computer vision to atmospheric remote sensing.
Cloud Masking and Surface Temperature Distribution in the Polar Regions Using AVHRR and other Satellite Data
Surface temperature is one of the key variables associated with weather and climate. Accurate measurements of surface air temperatures are routinely made in meteorological stations around the world. Also, satellite data have been used to produce synoptic global temperature distributions. However, not much attention has been paid on temperature distributions in the polar regions. In the polar regions, the number of stations is very sparse. Because of adverse weather conditions and general inaccessibility, surface field measurements are also limited. Furthermore, accurate retrievals from satellite data in the region have been difficult to make because of persistent cloudiness and ambiguities in the discrimination of clouds from snow or ice. Surface temperature observations are required in the polar regions for air-sea-ice interaction studies, especially in the calculation of heat, salinity, and humidity fluxes. They are also useful in identifying areas of melt or meltponding within the sea ice pack and the ice sheets and in the calculation of emissivities of these surfaces. Moreover, the polar regions are unique in that they are the sites of temperature extremes, the location of which is difficult to identify without a global monitoring system. Furthermore, the regions may provide an early signal to a potential climate change because such signal is expected to be amplified in the region due to feedback effects. In cloud free areas, the thermal channels from infrared systems provide surface temperatures at relatively good accuracies. Previous capabilities include the use of the Temperature Humidity Infrared Radiometer (THIR) onboard the Nimbus-7 satellite which was launched in 1978. Current capabilities include the use of the Advance Very High Resolution Radiometer (AVHRR) aboard NOAA satellites. Together, these two systems cover a span of 16 years of thermal infrared data. Techniques for retrieving surface temperatures with these sensors in the polar regions have been developed. Errors have been estimated to range from 1K to 5K mainly due to cloud masking problems. With many additional channels available, it is expected that the EOS-Moderate Resolution Imaging Spectroradiometer (MODIS) will provide an improved characterization of clouds and a good discrimination of clouds from snow or ice surfaces.