SEARCH · Search NASA
Results for “Cloud mask”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Cloud Mask Intercomparison eXercise (CMIX): An evaluation of cloud masking algorithms for Landsat 8 and Sentinel-2
Cloud cover is a major limiting factor in exploiting time-series data acquired by optical spaceborne remote sensing sensors. Multiple methods have been developed to address the problem of cloud detection in satellite imagery and a number of cloud masking algorithms have been developed for optical sensors but very few studies have carried out quantitative intercomparison of state-of-the-art methods in this domain. This paper summarizes results of the first Cloud Masking Intercomparison eXercise (CMIX) conducted within the Committee Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). CEOS is the forum for space agency coordination and cooperation on Earth observations, with activities organized under working groups. CMIX, as one such activity, is an international collaborative effort aimed at intercomparing cloud detection algorithms for moderate-spatial resolution (10–30 m) spaceborne optical sensors. The focus of CMIX is on open and free imagery acquired by the Landsat 8 (NASA/USGS) and Sentinel-2 (ESA) missions. Ten algorithms developed by nine teams from fourteen different organizations representing universities, research centers and industry, as well as space agencies (CNES, ESA, DLR, and NASA), are evaluated within the CMIX. Those algorithms vary in their approach and concepts utilized which were based on various spectral properties, spatial and temporal features, as well as machine learning methods. Algorithm outputs are evaluated against existing reference cloud mask datasets. Those datasets vary in sampling methods, geographical distribution, sample unit (points, polygons, full image labels), and generation approaches (experts, machine learning, sky images). Overall, the performance of algorithms varied depending on the reference dataset, which can be attributed to differences in how the reference datasets were produced. The algorithms were in good agreement for thick cloud detection, which were opaque and had lower uncertainties in their identification, in contrast to thin/semi-transparent clouds detection. Not only did CMIX allow identification of strengths and weaknesses of existing algorithms and potential areas of improvements, but also the problems associated with the existing reference datasets. The paper concludes with recommendations on generating new reference datasets, metrics, and an analysis framework to be further exploited and additional input datasets to be considered by future CMIX activities.
Using High-Resolution NSRDB Data to Evaluate Cloud Mask Forecast from WRF-Solar EPS
Validating spatiotemporal distributions of cloud forecasts using numerical weather prediction (NWP) models is difficult as this requires high-quality cloud-property at significantly high spatial and temporal resolution over extended periods of time. Observations of cloud properties, such as cloud mask, cloud optical thickness, and cloud type, are vital for assessing the capability of NWP models to forecast various types of clouds. Using the National Solar Radiation Database (NSRDB), this research evaluates ensemble cloud-mask predictions from the WRF-Solar ensemble prediction system (WRF-Solar EPS). From the WRF-Solar EPS, day-ahead solar forecasts for the contiguous United States (CONUS) for 2018 are simulated. Given the NSRDB data is accessible at a resolution of 2 km, we can calculate the cloud fraction across the 9-km grid of WRF-Solar EPS. This allows us to spatially assess the cloud-mask forecasts using two methods against the high-resolution NSRDB: (a) considering all 2-km NSRDB clouds in the forecast domain (EMAll), and (b) using a minimum cloud fraction threshold of 50% to designate a pixel as cloudy (EMP50). The low-resolution cloud masks from WRF-Solar EPS are evaluated directly against the cloud-resolving scale gridded observations from NSRDB using EMAll. With EMP50, we presume that scenes with less than 50% cloud cover from the 2-km NSRDB are clear. Thus, this assessment approach allows for a fair comparison with WRF-Solar EPS resolved to a 9-km grid. A method of point-by-point verification is used to evaluate dichotomous (yes/no) cloud mask predictions against the NSRDB. For each pixel of model extent, cloud frequency and traditional metrics (e.g., probability of detection, false alarm rate, and hit rate, etc.) are computed and compared with satellite-derived data sets. Mismatched cloud frequency (MCF) is computed to measure the present capability of WRF-Solar EPS in representing various types of clouds, which are categorized using three levels of cloud top height (CTH) and cloud optical depth (COD) across entire CONUS. Preliminary results show that the WRF-Solar EPS provides MCF values ranging from 9% to 46%, 16% to 33%, and 8% to 27% for low-level, middle-level, and high-level clouds, respectively, for three CTHs. The model produces MCFs ranging from 27% to 46%, 13% to 34%, and 8% to 19% for thin, medium-thickness, and thick clouds, respectively, for three CODs. The presentation will include a detailed description of the current outcomes as well as potential future extensions. The evaluation approach established in this study is readily extensible to the evaluation of cloud predictions from different ensemble NWP models. In addition, the findings of the suggested evaluation technique aid in identifying model weaknesses and will ultimately lead to advances in WRF-Solar EPS's skill in predicting clouds and solar irradiance.
Toward Physics-informed Neural Networks for 3D Multi-layer Cloud Mask Reconstruction
Three-dimensional (3D) cloud retrievals are critical for understanding their impact on climate and other applications such as aviation safety, weather prediction, and remote sensing. However, obtaining high-resolution and accurate vertical representation of clouds remains unsolved due to the limitations imposed by satellite instrumentation, viewing conditions, and the complexity of cloud dynamics. Cloud masks are essential for comprehending various cloud vertical properties, but deriving accurate 3D cloud masks from 2D satellite imagery data is a challenging task. To tackle these challenges, we introduce a physics-informed loss function for training deep learning models that can extend 2D cloud images into 3D cloud masks. The proposed loss, called CloudMask Loss, is composed of two domain knowledge-informed loss terms: one for evaluating cloud position and thickness, and the other for measuring the number of layers. By combining these loss terms, we improve the trainability of the deep learning models for more accurate and meaningful results. We apply the proposed loss function to different neural networks and demonstrate significant improvements in multi-layer cloud mask reconstruction. Utilizing the same neural network architecture, our proposed loss outperforms standard binary crossentropy loss in terms of multi-layer cloud classification accuracy, number of layers accuracy, and thickness mean absolute error (MAE). The proposed loss function can be readily integrated into various neural network architectures, resulting in substantial performance gains in 3D cloud mask generation.
Improved Thin Cirrus and Terminator Cloud Detection in CERES Cloud Mask
Thin cirrus clouds account for about 20-30% of the total cloud coverage and affect the global radiation budget by increasing the Earth's albedo and reducing infrared emissions. Thin cirrus, however, are often underestimated by traditional satellite cloud detection algorithms. This difficulty is caused by the lack of spectral contrast between optically thin cirrus and the surface in techniques that use visible (0.65 micron ) and infrared (11 micron ) channels. In the Clouds and the Earth s Radiant Energy System (CERES) Aqua Edition 1 (AEd1) and Terra Edition 3 (TEd3) Cloud Masks, thin cirrus detection is significantly improved over both land and ocean using a technique that combines MODIS high-resolution measurements from the 1.38 and 11 micron channels and brightness temperature differences (BTDs) of 11-12, 8.5-11, and 3.7-11 micron channels. To account for humidity and view angle dependencies, empirical relationships were derived with observations from the 1.38 micron reflectance and the 11-12 and 8.5-11 micron BTDs using 70 granules of MODIS data in 2002 and 2003. Another challenge in global cloud detection algorithms occurs near the day/night terminator where information from the visible 0.65 micron channel and the estimated solar component of 3.7 micron channel becomes less reliable. As a result, clouds are often underestimated or misidentified near the terminator over land and ocean. Comparisons between the CLAVR-x (Clouds from Advanced Very High Resolution Radiometer [AVHRR]) cloud coverage and Geoscience Laser Altimeter System (GLAS) measurements north of 60 N indicate significant amounts of missing clouds from CLAVR-x because this part of the world was near the day/night terminator viewed by AVHRR. Comparisons between MODIS cloud products (MOD06) and GLAS in the same region also show similar difficulties with MODIS cloud retrievals. The consistent detection of clouds through out the day is needed to provide reliable cloud and radiation products for CERES and other research efforts involving the modeling of clouds and their interaction with the radiation budget.
Development of a Consistent Cross-Platform GEO-Satellite Cloud Mask to Support CERES
Geostationary satellites provide continuous cloud and meteorological observations over a fixed portion of the Earth’s surface, allowing them to monitor the movement of storm systems and their diurnal variation. For climate studies, geostationary observations provide valuable insight of cloud formation and evolution and how they influence the Earth’s radiation budget. The Geo-Satellite Edition 4 cloud mask (GEO Ed4) is used operationally in NASA’s Cloud and Earth’s Radiant Energy System (CERES) project to help account for diurnal variations in cloudiness on Earth’s radiation budget. The Ed4 cloud mask was applied to imager data on MSG (MeteoSat Second Generation), Himawari, GOES-West, and GOES-East satellites using as much spectral information as possible. That approach led to some discontinuities when a more modern satellite replaced an older satellite with less spectral information. A different strategy is being investigated for the CERES GEO Ed5 cloud mask that only uses spectral channels common on every satellite. Thus, a 3-channel (0.6, 3.9, 11 µm) algorithm for daytime cloud detection, and a 2-channels (3.9 and 11 µm) algorithm for nighttime cloud detection have been implemented and tested for Ed5. The goal of Ed5 cloud mask is to achieve global cloud amount consistency across five geo-satellites, and a smooth transition from an old satellite to a new satellite over each geo-location. This paper compares cloud mask results between Ed4 and a preliminary Ed5 version over the GOES-East region, using GOES-8, GOES-13, and GOES-16 satellites, each with a different spectral channel complement. Instantaneous and monthly regional mean inter-comparisons are performed and evaluated with CALIPSO cloud products to assess the accuracy and consistency of the Ed5 approach relative to that taken in Ed4. Results will be presented and discussed at the conference.
New Neural Network Cloud Mask Algorithm Based on Radiative Transfer Simulations
Cloud detection and screening constitute critically important first steps required to derive many satellite data products. Traditional threshold-based cloud mask algorithms require a complicated design process and fine tuning for each sensor, and they have difficulties over areas partially covered with snow/ice. Exploiting advances in machine learning techniques and radiative transfer modeling of coupled environmental systems, we have developed a new, threshold-free cloud mask algorithm based on a neural network classifier driven by extensive radiative transfer simulations. Statistical validation results obtained by using collocated CALIOP and MODIS data show that its performance is consistent over different ecosystems and significantly better than the MODIS Cloud Mask (MOD35 C6) during the winter seasons over snow-covered areas in the mid-latitudes. Simulations using a reduced number of satellite channels also show satisfactory results, indicating its flexibility to be configured for different sensors. Comparedto threshold-based methods and previous machine-learning approaches, this new cloud mask (i) does not rely on thresholds, (ii) needs fewer satellite channels, (iii) has superior performance during winter seasons in mid-latitude areas, and (iv) can easily be applied to different sensors.
The EOS CERES Global Cloud Mask
To detect long-term climate trends, it is essential to produce long-term and consistent data sets from a variety of different satellite platforms. With current global cloud climatology data sets, such as the International Satellite Cloud Climatology Experiment (ISCCP) or CLAVR (Clouds from Advanced Very High Resolution Radiometer), one of the first processing steps is to determine whether an imager pixel is obstructed between the satellite and the surface, i.e., determine a cloud 'mask.' A cloud mask is essential to studies monitoring changes over ocean, land, or snow-covered surfaces. As part of the Earth Observing System (EOS) program, a series of platforms will be flown beginning in 1997 with the Tropical Rainfall Measurement Mission (TRMM) and subsequently the EOS-AM and EOS-PM platforms in following years. The cloud imager on TRMM is the Visible/Infrared Sensor (VIRS), while the Moderate Resolution Imaging Spectroradiometer (MODIS) is the imager on the EOS platforms. To be useful for long term studies, a cloud masking algorithm should produce consistent results between existing (AVHRR) data, and future VIRS and MODIS data. The present work outlines both existing and proposed approaches to detecting cloud using multispectral narrowband radiance data. Clouds generally are characterized by higher albedos and lower temperatures than the underlying surface. However, there are numerous conditions when this characterization is inappropriate, most notably over snow and ice of the cloud types, cirrus, stratocumulus and cumulus are the most difficult to detect. Other problems arise when analyzing data from sun-glint areas over oceans or lakes over deserts or over regions containing numerous fires and smoke. The cloud mask effort builds upon operational experience of several groups that will now be discussed.
Progress Towards a Common CERES Cloud Mask Algorithm for MODIS and VIIRS: Evaluation using CALIOP Data
For over two decades The Clouds and the Earth’s Radiant Energy System (CERES) project has endeavored to produce a long-term global climate data record for detecting changes in the Earth’s radiation budget and to improve understanding of how clouds contribute to those changes. CERES incorporates cloud information derived from passive narrowband satellite imaging radiometers, and over the course of the project different instruments have contributed to this effort, namely the Moderate Resolution Imaging Spectroradiometer (MODIS) and the Visible Infrared Imaging Radiometer Suite (VIIRS).Cloud properties have been derived from MODIS on the Aqua satellite for CERES since 2002,and VIIRS on the NOAA-20 satellite launched in 2017 will continue the cloud record once Aqua-MODIS reaches the end of its operational lifetime. However, MODIS and VIIRS have different spectral capabilities and characteristics which complicates our ability to seamlessly transition the record from MODIS to VIIRS. This study evaluates some of the recent algorithm modifications incorporated by the CERES Cloud Working Group towards developing a unified cloud mask algorithm for MODIS and VIIRS that utilizes a set of spectral bands common to both instruments. Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) data products are used to evaluate MODIS and VIIRS regional cloud fraction estimates and identify areas where improvements can be made. The cloud mask algorithm relies on computed estimates of cloud-free top-of-atmosphere radiances to differentiate cloudy and cloud-free imager pixels. The unified cloud mask algorithm currently under development uses a reduced number of spectral bands common to both MODIS and VIIRS and so relies on the accuracy of these simulated cloud-free radiances more heavily than predecessor algorithms, i.e., the CERES Edition 4 cloud mask, which use as much spectral information as available on each imager with less consideration for cross-platform consistency. Our validation strategy using CALIOP data is also described. The CALIOP observations are critical for assessing cloud detection accuracies and for independently confirming cloud-free conditions which enables a more robust evaluation of the simulated cloud-free radiances for biases due to factors such as view angle and the spectral dependence of water vapor absorption. These effects are known to differ for the two satellite instruments. Comparisons of MODIS and VIIRS cloud fractions are presented in context with estimates from CALIOP and from the CERES Edition 4 cloud mask algorithm to gauge current progress in developing accurate and consistent MODIS and VIIRS cloud properties for CERES.
Evaluating WRF-Solar EPS cloud mask forecast using the NSRDB
Improving the accuracy of day-ahead solar forecasts using numerical weather prediction models requires improving the forecasting of cloud occurrence and properties. Validating cloud forecasts is challenging because this requires evaluating different types of clouds over a wide variety of regions. This study analyzes the cloud occurrence (or cloud mask) over the contiguous United States (CONUS), predicted by the Weather Research and Forecasting-Solar Ensemble Prediction System (WRF-Solar EPS), to identify the strengths and limitations of the model in reproducing cloud fields. To enable the in-depth analysis of cloud mask forecasts covering CONUS, we use satellite observations from the National Solar Radiation Database (NSRDB). Two evaluation methods are implemented to consider all clouds and partially removed clouds from the 2-km NSRDB in evaluating the 9-km WRF-Solar EPS cloud mask. Cloud detection metrics as well as the frequency of cloud occurrence are used to quantify the monthly performance of WRF-Solar EPS. Mismatched cloud frequency (MCF) is used to assess the model's capability to predict different types of clouds, which are classified using three levels of cloud optical depth (COD) and cloud top height (CTH). The day-ahead forecasts covering the full year of 2018 demonstrate that WRF-Solar EPS produces MCFs ranging from 27%-46%, 13%-34%, and 8%-19% for thin, mid-thickness, and thick clouds, respectively. For three CTH levels, the model shows MCFs ranging from 19%-46%, 16%-33%, and 8%-27% for low-level, middle-level, and high-level clouds, respectively. This comprehensive characterization of model performance helps identify model weakness and will eventually lead to improvements in cloud and solar radiation forecasting.
Implementation on Landsat Data of a Simple Cloud Mask Algorithm Developed for MODIS Land Bands
This letter assesses the performance on Landsat-7 images of a modified version of a cloud masking algorithm originally developed for clear-sky compositing of Moderate Resolution Imaging Spectroradiometer (MODIS) images at northern mid-latitudes. While data from recent Landsat missions include measurements at thermal wavelengths, and such measurements are also planned for the next mission, thermal tests are not included in the suggested algorithm in its present form to maintain greater versatility and ease of use. To evaluate the masking algorithm we take advantage of the availability of manual (visual) cloud masks developed at USGS for the collection of Landsat scenes used here. As part of our evaluation we also include the Automated Cloud Cover Assesment (ACCA) algorithm that includes thermal tests and is used operationally by the Landsat-7 mission to provide scene cloud fractions, but no cloud masks. We show that the suggested algorithm can perform about as well as ACCA both in terms of scene cloud fraction and pixel-level cloud identification. Specifically, we find that the algorithm gives an error of 1.3% for the scene cloud fraction of 156 scenes, and a root mean square error of 7.2%, while it agrees with the manual mask for 93% of the pixels, figures very similar to those from ACCA (1.2%, 7.1%, 93.7%).
An Automatic Cloud Mask Algorithm Based on Time Series of MODIS Measurements
Quality of aerosol retrievals and atmospheric correction depends strongly on accuracy of the cloud mask (CM) algorithm. The heritage CM algorithms developed for AVHRR and MODIS use the latest sensor measurements of spectral reflectance and brightness temperature and perform processing at the pixel level. The algorithms are threshold-based and empirically tuned. They don't explicitly address the classical problem of cloud search, wherein the baseline clear-skies scene is defined for comparison. Here, we report on a new CM algorithm which explicitly builds and maintains a reference clear-skies image of the surface (refcm) using a time series of MODIS measurements. The new algorithm, developed as part of the Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm for MODIS, relies on fact that clear-skies images of the same surface area have a common textural pattern, defined by the surface topography, boundaries of rivers and lakes, distribution of soils and vegetation etc. This pattern changes slowly given the daily rate of global Earth observations, whereas clouds introduce high-frequency random disturbances. Under clear skies, consecutive gridded images of the same surface area have a high covariance, whereas in presence of clouds covariance is usually low. This idea is central to initialization of refcm which is used to derive cloud mask in combination with spectral and brightness temperature tests. The refcm is continuously updated with the latest clear-skies MODIS measurements, thus adapting to seasonal and rapid surface changes. The algorithm is enhanced by an internal dynamic land-water-snow classification coupled with a surface change mask. An initial comparison shows that the new algorithm offers the potential to perform better than the MODIS MOD35 cloud mask in situations where the land surface is changing rapidly, and over Earth regions covered by snow and ice.
HyspIRI Cloud Mask Detection Algorithm Theoretical Basis Document
The Hyperspectral Infrared Imager (HyspIRI) satellite includes a thermal infrared (TIR) multispectral scanner with seven spectral bands in the thermal infrared (TIR) between 7 and 12 μm and one band in the mid-infrared between 3 and 5 μm designed to measure hot targets. The TIR bands have a NEΔT of <0.2 K at 300K and all bands have a spatial scale of 60 m. The two primary Level-2 products that will be generated by HyspIRI TIR data are land surface temperature and emissivity. A critical aspect of minimizing uncertainties in these products and in generating global Level-3 products is an accurate and reliable cloud mask algorithm. This document describes the methodology behind developing the HyspIRI Cloud Mask Algorithm (HyCMA), and challenges associated with cloud detection. The HyCMA will derive its heritage from previously well established cloud mask algorithms such as the Landsat ACCAA, MODIS MOD35 and AVHRR.
A Cloud Mask for AIRS
With the preliminary release of AIRS Level 1 and 2 data to the scientific community, there is a growing need for an accurate AIRS cloud mask for data assimilation studies and in producing products derived from cloud free radiances. Current cloud information provided with the AIRS data are limited or based on simplified threshold tests. A multispectral cloud detection approach has been developed for AIRS that utilizes the hyper-spectral capabilities to detect clouds based on specific cloud signatures across the short wave and long wave infrared window regions. This new AIRS cloud mask has been validated against the existing AIRS Level 2 cloud product and cloud information derived from MODIS. Preliminary results for both day and night applications over the continental U.S. are encouraging. Details of the cloud detection approach and validation results will be presented at the conference.
Micropulse Lidar Cloud Mask Machine-Learning Value-Added Product Report
Cloud detection algorithms of various techniques have been developed and applied to atmospheric ground-based lidar data to identify cloud boundaries and produce clouds masks. While these algorithms are able to identify a wide variety of cloud types and conditions, it is often observed that the algorithms can still fail to accurately detect clouds that are readily discernible when inspecting the lidar imagery. Based on this observation, an alternative approach for cloud detection is to take advantage of machine-learning capabilities and the trained human eye as an interpreter of lidar images, and in turn, to train a neural network to recognize the desired features in the lidar data.
Micropulse Lidar Cloud Mask (MPLCMASK) Value-Added Product for the Fast-Switching Polarized Micropulse Lidar Technical Report
Lidar backscattered signal is a useful tool for identifying vertical cloud structure in the atmosphere in optically thin clouds. Cloud boundaries derived from lidar signals are a necessary input for popular ARM data products, such as the Active Remote Sensing of Clouds (ARSCL) product. An operational cloud boundary algorithm (Wang and Sassen 2001) has been implemented for use with the ARM Micropulse Lidar (MPL) systems. In addition to retrieving cloud boundaries above 500 m, the value-added product (VAP) named Micropulse Lidar Cloud Mask (MPLCMASK) applies lidar-specific corrections (i.e., range-square, background, deadtime, and overlap) as described in Campbell et al. (2002) to the measured backscattered lidar. Depolarization ratio is computed using the methodology developed by Flynn et al. (2007) for polarization-capable MPL systems. The cloud boundaries output from MPLCMASK will be the primary lidar cloud mask for input to the ARSCL product and will be applied to all MPL systems, including historical data sets.
Forcing, Cloud Feedbacks, Cloud Masking, and Internal Variability in the Cloud Radiative Effect Satellite Record
Explore the source record for details and available documents.
Early Evaluation of the VIIRS Calibration, Cloud Mask and Surface Reflectance Earth Data Records
Surface reflectance is one of the key products fromVIIRS and as withMODIS, is used in developing several higherorder land products. The VIIRS Surface Reflectance (SR) Intermediate Product (IP) is based on the heritageMODIS Collection 5 product (Vermote, El Saleous, & Justice, 2002). The quality and character of surface reflectance depend on the accuracy of the VIIRS Cloud Mask (VCM), the aerosol algorithms and the adequate calibration of the sensor. The focus of this paper is the early evaluation of the VIIRS SR product in the context of the maturity of the operational processing system, the Interface Data Processing System (IDPS). After a brief introduction, the paper presents the calibration performance and the role of the surface reflectance in calibration monitoring. The analysis of the performance of the cloud mask with a focus on vegetation monitoring (no snow conditions) shows typical problems over bright surfaces and high elevation sites. Also discussed is the performance of the aerosol input used in the atmospheric correction and in particular the artifacts generated by the use of the Navy Aerosol Analysis and Prediction System. Early quantitative results of the performance of the SR product over the AERONET sites showthatwith the fewadjustments recommended, the accuracy iswithin the threshold specifications. The analysis of the adequacy of the SR product (Land PEATE adjusted version) in applications of societal benefits is then presented. We conclude with a set of recommendations to ensure consistency and continuity of the JPSS mission with the MODIS Land Climate Data Record.