Search NASA⌕ Search

Engineering topics

Collis, Scott M.

Publications and source records attributed to Collis, Scott M..

At least 19 records

OpenCRUMS USA: An Open Machine Learning Framework for Characterizing Variability in Aerosol Reanalysis Data

Advances in artificial intelligence (AI) have called for exploring how these techniques can be used for exploring patterns in large climate datasets. To that regard, the U.S. Department of Energy AI for Earth System Predictability (AI4ESP) supported a pilot initiative called the Open Classification of Regimes in the Southeast USA (OpenCRUMS USA) project to explore how AI can be used to characterize modes of spatial variability in large climate datasets. For this study, we focus on comparing two methods for characterizing the modes of spatial variability of surface aerosol concentration over the Houston region: empirical orthogonal functions (EOFs) and layerwise relevance propagation (LRP) applied to a convolutional neural network (CNN) classifier. We show that EOF analysis typically attributes spatial variability modes that span all of southeast Texas, prohibiting the attribution of spatial variability to localized regions. However, using LRP on the CNN classifier resolves the explanatory parameters at a finer spatial resolution than EOFs. This allows for the attribution of the spatial variability of surface aerosols to local regions of organic carbon which was not possible using EOFs. In addition, the LRP analysis also suggests that synoptic-scale transport of dust is most prevalent during anticyclonic and pretrough synoptic conditions as categorized by self-organizing maps.

54 ENVIRONMENTAL SCIENCES↗

Extracted Radar Columns and In Situ Sensors (RadCLss) Value-Added Product Report

In order to validate precipitation, in 2010 the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility procured 3- and 5-cm wavelength radars for documenting the macrophysical, microphysical, and dynamical structure of precipitating systems. To maximize the scientific impact, ARM supported the development of an application chain to correct for various phenomena in order to retrieve the “point” values of moments of the radar spectrum and polarimetric measurements. In estimation from ARM radars, a workflow was created to directly compare radar “point” values with various in situ observations at the surface.

54 ENVIRONMENTAL SCIENCES↗

tobac v1.5: introducing fast 3D tracking, splits and mergers, and other enhancements for identifying and analysing meteorological phenomena

There is a continuously increasing need for reliable feature detection and tracking tools based on objective analysis principles for use with meteorological data. Many tools have been developed over the previous 2 decades that attempt to address this need but most have limitations on the type of data they can be used with, feature computational and/or memory expenses that make them unwieldy with larger datasets, or require some form of data reduction prior to use that limits the tool's utility. The Tracking and Object-Based Analysis of Clouds (tobac) Python package is a modular, open-source tool that improves on the overall generality and utility of past tools. A number of scientific improvements (three spatial dimensions, splits and mergers of features, an internal spectral filtering tool) and procedural enhancements (increased computational efficiency, internal regridding of data, and treatments for periodic boundary conditions) have been included in tobac as a part of the tobac v1.5 update. These improvements have made tobac one of the most robust, powerful, and flexible identification and tracking tools in our field to date and expand its potential use in other fields. Future plans for tobac v2 are also discussed.

54 ENVIRONMENTAL SCIENCES↗

ARMing the Edge: Designing Edge Computing–Capable Machine Learning Algorithms to Target ARM Doppler Lidar Processing

Abstract There is a need for long-term observations of cloud and precipitation fall speeds in validating and improving rainfall forecasts from climate models. To this end, the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility Southern Great Plains (SGP) site at Lamont, Oklahoma, hosts five ARM Doppler lidars that can measure cloud and aerosol properties. In particular, the ARM Doppler lidars record Doppler spectra that contain information about the fall speeds of cloud and precipitation particles. However, due to bandwidth and storage constraints, the Doppler spectra are not routinely stored. This calls for the automation of cloud and rain detection in ARM Doppler lidar data so that the spectral data in clouds can be selectively saved and further analyzed. During the ARMing the Edge field experiment, a Waggle node capable of performing machine learning applications in situ was deployed at the ARM SGP site for this purpose. In this paper, we develop and test four algorithms for the Waggle node to automatically classify ARM Doppler lidar data. We demonstrate that supervised learning using a ResNet50-based classifier will classify 97.6% of the clear-air images and 94.7% of cloudy images correctly, outperforming traditional peak detection methods. We also show that a convolutional autoencoder paired with k -means clustering identifies 10 clusters in the ARM Doppler lidar data. Three clusters correspond to mostly clear conditions with scattered high clouds, and seven others correspond to cloudy conditions with varying cloud-base heights.

54 ENVIRONMENTAL SCIENCES↗

SAIL Field Campaign X-Band Precipitation Radar Surface Quantitative Precipitation Estimation (SQUIRE) Value-Added Product Report

In 2010, the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility procured 3- and 5-cm wavelength radars for documenting the macrophysical, microphysical, and dynamical structure of precipitating systems. In order to maximize the scientific impact, ARM supported the development of an application chain to correct for various phenomena in order to retrieve the lowest retrieved value on a Cartesian grid. This report details the motivation, science, and progress to date, as well as charting a path forward.

54 ENVIRONMENTAL SCIENCES↗

Optimizing cloud motion estimation on the edge with phase correlation and optical flow

Abstract. Phase correlation (PC) is a well-known method for estimating cloud motion vectors (CMVs) from infrared and visible spectrum images. Commonly, phase shift is computed in the small blocks of the images using the fast Fourier transform. In this study, we investigate the performance and the stability of the blockwise PC method by changing the block size, the frame interval, and combinations of red, green, and blue (RGB) channels from the total sky imager (TSI) at the United States Atmospheric Radiation Measurement user facility's Southern Great Plains site. We find that shorter frame intervals, followed by larger block sizes, are responsible for stable estimates of the CMV, as suggested by the higher autocorrelations. The choice of RGB channels has a limited effect on the quality of CMVs, and the red and the grayscale images are marginally more reliable than the other combinations during rapidly evolving low-level clouds. The stability of CMVs was tested at different image resolutions with an implementation of the optimized algorithm on the Sage cyberinfrastructure test bed. We find that doubling the frame rate outperforms quadrupling the image resolution in achieving CMV stability. The correlations of CMVs with the wind data are significant in the range of 0.38–0.59 with a 95 % confidence interval, despite the uncertainties and limitations of both datasets. A comparison of the PC method with constructed data and the optical flow method suggests that the post-processing of the vector field has a significant effect on the quality of the CMV. The raindrop-contaminated images can be identified by the rotation of the TSI mirror in the motion field. The results of this study are critical to optimizing algorithms for edge-computing sensor systems.

54 ENVIRONMENTAL SCIENCES↗

Let’s Unleash the Network Judgment: A Self-Supervised Approach for Cloud Image Analysis

Accurate cloud type identification and coverage analysis are crucial in understanding the Earth’s radiative budget. Traditional computer vision methods rely on low-level visual features of clouds for estimating cloud coverage or sky conditions. Several handcrafted approaches have been proposed; however, scope for improvement still exists. Newer deep neural networks (DNNs) have demonstrated superior performance for cloud segmentation and categorization. These methods, however, need expert engineering intervention in the preprocessing steps—in the traditional methods—or human assistance in assigning cloud or clear sky labels to a pixel for training DNNs. Such human mediation imposes considerable time and labor costs. We present the application of a new self-supervised learning approach to autonomously extract relevant features from sky images captured by ground-based cameras, for the classification and segmentation of clouds. We evaluate a joint embedding architecture that uses self-knowledge distillation plus regularization. We use two datasets to demonstrate the network’s ability to classify and segment sky images—one with ~85,000 images collected from our ground-based camera and another with 400 labeled images from the WSISEG database. We find that this approach can discriminate full-sky images based on cloud coverage, diurnal variation, and cloud base height. Additionally, it semantically segments the cloud areas without labels. The approach shows competitive performance in all tested tasks, suggesting a new alternative for cloud characterization.

54 ENVIRONMENTAL SCIENCES↗

Corrected Moments in Antenna Coordinates (CMAC) X-SAPR Technical Report

In 2010 the Atmospheric Radiation Measurement user facility procured 3- and 5-cm wavelength radars for documenting the macrophysical, microphysical, and dynamical structure of precipitating systems. In order to maximize the scientific impact, the program supported the development of an application chain to correct for various phenomena in order to retrieve the “point” values of moments of the radar spectrum and polarimetric measurements. This report details the motivation, science, and progress to date as well as charting a path forward.

54 ENVIRONMENTAL SCIENCES↗

An Adaptive Tracking Algorithm for Convection in Simulated and Remote Sensing Data

A robust and computationally efficient object tracking algorithm is developed by incorporating various tracking techniques. Physical properties of the objects, such as brightness temperature or reflectivity, are not considered. Therefore, the algorithm is adaptable for tracking convection-like features in simulated data and remotely sensed two-dimensional images. In this algorithm, a first guess of the motion, estimated using the Fourier phase shift, is used to predict the candidates for matching. A disparity score is computed for each target–candidate pair. The disparity also incorporates overlapping criteria in the case of large objects. Then the Hungarian method is applied to identify the best pairs by minimizing the global disparity. The high-disparity pairs are unmatched, and their target and candidate are declared expired and newly initiated objects, respectively. They are tested for merger and split on the basis of their size and overlap with the other objects. The sensitivity of track duration is shown for different disparity and size thresholds. The report highlights the algorithm’s ability to study convective life cycles using radar and simulated data over Darwin, Australia. The algorithm skillfully tracks individual convective cells (a few pixels in size) and large convective systems. The duration of tracks and cell size are found to be lognormally distributed over Darwin. The evolution of size and precipitation types of isolated convective cells is presented in the Lagrangian perspective. This algorithm is part of a vision for a modular platform [viz., TINT is not TITAN (TINT) and Tracking and Object-Based Analysis of Clouds (tobac)] that will evolve into a sustainable choice to analyze atmospheric features.

54 ENVIRONMENTAL SCIENCES↗

Prediction of Solar Irradiance and Photovoltaic Solar Energy Product Based on Cloud Coverage Estimation Using Machine Learning Methods

Cloud cover estimation from images taken by sky-facing cameras can be an important input for analyzing current weather conditions and estimating photovoltaic power generation. The constant change in position, shape, and density of clouds, however, makes the development of a robust computational method for cloud cover estimation challenging. Accurately determining the edge of clouds and hence the separation between clouds and clear sky is difficult and often impossible. Toward determining cloud cover for estimating photovoltaic output, we propose using machine learning methods for cloud segmentation. We compare several methods including a classical regression model, deep learning methods, and boosting methods that combine results from the other machine learning models. To train each of the machine learning models with various sky conditions, we supplemented the existing Singapore whole sky imaging segmentation database with hazy and overcast images collected by a camera-equipped Waggle sensor node. We found that the U-Net architecture, one of the deep neural networks we utilized, segmented cloud pixels most accurately. However, the accuracy of segmenting cloud pixels did not guarantee high accuracy of estimating solar irradiance. We confirmed that the cloud cover ratio is directly related to solar irradiance. Additionally, we confirmed that solar irradiance and solar power output are closely related; hence, by predicting solar irradiance, we can estimate solar power output. This study demonstrates that sky-facing cameras with machine learning methods can be used to estimate solar power output. This ground-based approach provides an inexpensive way to understand solar irradiance and estimate production from photovoltaic solar facilities.

14 SOLAR ENERGY↗

UNRAVEL: A Robust Modular Velocity Dealiasing Technique for Doppler Radar

Unfold Radar Velocity (UNRAVEL) is an open-source modular Doppler velocity dealiasing algorithm for weather radars. UNRAVEL is an algorithm that does not need external reference velocity data, making it easily applicable. The proposed algorithm includes 11 core modules and 2 dealiasing strategies. UNRAVEL is an iterative algorithm. The goal is to build the dealiasing results starting with the strictest possible continuity tests in azimuth and range and, after each step, relaxing the parameters to include more results from a progressively growing number of reference points. UNRAVEL also has modules that perform 3D continuity checks. Thanks to this modular design, the number of dealiasing strategies can be expanded in order to optimize the dealiasing results. While the first driver dealiases Doppler velocity from each tilt independently from one another, the second driver also performs a three-dimensional continuity check of the velocity using successive elevations. The proposed dealiasing algorithm is tested using severe weather data from an S-band Doppler radar that have been aliased to mimic aliased radial velocity patterns that would be observed by a C-band Doppler radar. Artificially aliasing S-band data permits creation of a reference to which the performance of various dealiasing techniques can be compared. Comparisons show that UNRAVEL consistently outperforms other established dealiasing algorithms for the test period selected in this work.

47 OTHER INSTRUMENTATION↗

An Integrated Approach to Weather Radar Calibration and Monitoring Using Ground Clutter and Satellite Comparisons

Abstract The stability and accuracy of weather radar reflectivity calibration are imperative for quantitative applications, such as rainfall estimation, severe weather monitoring and nowcasting, and assimilation in numerical weather prediction models. Various radar calibration and monitoring techniques have been developed, but only recently have integrated approaches been proposed, that is, using different calibration techniques in combination. In this paper the following three techniques are used: 1) ground clutter monitoring, 2) comparisons with spaceborne radars, and 3) the self-consistency of polarimetric variables. These techniques are applied to a C-band polarimetric radar (CPOL) located in the Australian tropics since 1998. The ground clutter monitoring technique is applied to each radar volumetric scan and provides a means to reliably detect changes in calibration, relative to a baseline. It is remarkably stable to within a standard deviation of 0.1 dB. To obtain an absolute calibration value, CPOL observations are compared to spaceborne radars on board TRMM and GPM using a volume-matching technique. Using an iterative procedure and stable calibration periods identified by the ground echoes technique, we improve the accuracy of this technique to about 1 dB. Finally, we review the self-consistency technique and constrain its assumptions using results from the hybrid TRMM–GPM and ground echo technique. Small changes in the self-consistency parameterization can lead to 5 dB of variation in the reflectivity calibration. We find that the drop-shape model of Brandes et al. with a standard deviation of the canting angle of 12° best matches our dataset.

54 ENVIRONMENTAL SCIENCES↗

An Integrated Approach to Weather Radar Calibration and Monitoring Using Ground Clutter and Satellite Comparisons

The stability and accuracy of weather radar reflectivity calibration are imperative for quantitative applications, such as rainfall estimation, severe weather monitoring and nowcasting, and assimilation in numerical weather prediction models. Various radar calibration and monitoring techniques have been developed, but only recently have integrated approaches been proposed, that is, using different calibration techniques in combination. In this paper the following three techniques are used: 1) ground clutter monitoring, 2) comparisons with spaceborne radars, and 3) the self-consistency of polarimetric variables. These techniques are applied to a C-band polarimetric radar (CPOL) located in the Australian tropics since 1998. The ground clutter monitoring technique is applied to each radar volumetric scan and provides a means to reliably detect changes in calibration, relative to a baseline. It is remarkably stable to within a standard deviation of 0.1 dB (decibels). To obtain an absolute calibration value, CPOL observations are compared to spaceborne radars on board TRMM (Tropical Rainfall Measuring Mission) and GPM (Global Precipitation Measurement) using a volume-matching technique. Using an iterative procedure and stable calibration periods identified by the ground echoes technique, we improve the accuracy of this technique to about 1 dB. Finally, we review the self-consistency technique and constrain its assumptions using results from the hybrid TRMM-GPM and ground echo technique. Small changes in the self-consistency parameterization can lead to 5 dB of variation in the reflectivity calibration. We find that the drop-shape model of Brandes et al. with a standard deviation of the canting angle of 12 degrees best matches our dataset.

Louf, Valentin↗

A 17 year climatology of the macrophysical properties of convection in Darwin

Abstract. The validation of convective processes in global climate models (GCMs) could benefit from the use of large datasets that provide long-term climatologies of the spatial statistics of convection. To that regard, echo top heights (ETHs), convective areas, and frequencies of mesoscale convective systems (MCSs) from 17 years of data from a C-band polarization (CPOL) radar are analyzed in varying phases of the Madden–Julian Oscillation (MJO) and northern Australian monsoon in order to provide ample validation statistics for GCM validation. The ETHs calculated using velocity texture and reflectivity provide similar results, showing that the ETHs are insensitive to various techniques that can be used. Retrieved ETHs are correlated with those from cloud top heights retrieved by Multifunctional Transport Satellites (MTSATs), showing that the ETHs capture the relative variability in cloud top heights over seasonal scales. Bimodal distributions of ETH, likely attributable to the cumulus congestus clouds and mature stages of convection, are more commonly observed when the active phase of the MJO is over Australia due to greater mid-level moisture during the active phase of the MJO. The presence of a convectively stable layer at around 5 km altitude over Darwin inhibiting convection past this level can explain the position of the modes at around 2–4 km and 7–9 km. Larger cells were observed during break conditions compared to monsoon conditions, but only during the inactive phase of the MJO. The spatial distributions show that Hector, a deep convective system that occurs almost daily during the wet season over the Tiwi Islands, and sea-breeze convergence lines are likely more common in break conditions. Oceanic MCSs are more common during the night over Darwin. Convective areas were generally smaller and MCSs more frequent during active monsoon conditions. In general, the MJO is a greater control on the ETHs in the deep convective mode observed over Darwin, with higher distributions of ETH when the MJO is active over Darwin.

54 ENVIRONMENTAL SCIENCES↗

Vertical air motion retrievals in deep convective clouds using the ARM scanning radar network in Oklahoma during MC3E

The US Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) program's Southern Great Plains (SGP) site includes a heterogeneous distributed scanning Doppler radar network suitable for collecting coordinated Doppler velocity measurements in deep convective clouds. The surrounding National Weather Service (NWS) Next Generation Weather Surveillance Radar 1988 Doppler (NEXRAD WSR-88D) further supplements this network. Radar velocity measurements are assimilated in a three-dimensional variational (3DVAR) algorithm that retrieves horizontal and vertical air motions over a large analysis domain (100 km × 100 km) at storm-scale resolutions (250 m). For the first time, direct evaluation of retrieved vertical air velocities with those from collocated 915 MHz radar wind profilers is performed. Mean absolute and root-mean-square differences between the two sources are of the order of 1 and 2 m s -1 , respectively, and time–height correlations are of the order of 0.5. An empirical sensitivity analysis is done to determine a range of 3DVAR constraint weights that adequately satisfy the velocity observations and anelastic mass continuity. It is shown that the vertical velocity spread over this range is of the order of 1 m s -1 . The 3DVAR retrievals are also compared to those obtained from an iterative upwards integration technique. Lastly, the results suggest that the 3DVAR technique provides a robust, stable solution for cases in which integration techniques have difficulty satisfying velocity observations and mass continuity simultaneously.

54 ENVIRONMENTAL SCIENCES↗

The Midlatitude Continental Convective Clouds Experiment (MC3E)

The Midlatitude Continental Convective Clouds Experiment (MC3E), a field program jointly led by the U.S. Department of Energy’s Atmospheric Radiation Measurement program and the NASA Global Precipitation Measurement (GPM) Mission, was conducted in south-central Oklahoma during April – May 2011. MC3E science objectives were motivated by the need to improve understanding of midlatitude continental convective cloud system lifecycles, microphysics, and GPM precipitation retrieval algorithms. To achieve these objectives a multi-scale surface- and aircraft-based in situ and remote sensing observing strategy was employed. A variety of cloud and precipitation events were sampled during the MC3E, of which results from three deep convective events are highlighted. Vertical structure, air motions, precipitation drop-size distributions and ice properties were retrieved from multi-wavelength radar, profiler, and aircraft observations for an MCS on 11 May. Aircraft observations for another MCS observed on 20 May were used to test agreement between observed radar reflectivities and those calculated with forward-modeled reflectivity and microwave brightness temperatures using in situ particle size distributions and ice water content. Multi-platform observations of a supercell that occurred on 23 May allowed for an integrated analysis of kinematic and microphysical interactions. A core updraft of 25 ms-1 supported growth of hail and large rain drops. Data collected during the MC3E campaign is being used in a number of current and ongoing research projects and is available through the DOE ARM and NASA data archives.

Jensen, Mark P.↗