Engineering topics
Publications and source records attributed to Collis, Scott.
Focal Area(s): The paper addresses focal area 1: data acquisition/assimilation enabled by AI and advanced methods including model-driven experiments, 5G, and hardware-related efforts involving edge computing.
The fundamental objective of the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) radar facility is to provide high-quality radar observations to the scientific user community with an overarching goal of improving the treatment of clouds and precipitation in climate models. ARM has a complement of 32 scanning and zenith pointing radars, not including the radar wind profilers. This large number of radars and deployment locales require an enormous commitment of effort to keep operational. It has been the experience in ARM that all these radars cannot be always operated at a high level, and priorities need to be developed to properly manage operations every fiscal year. This document provides an overview of the FY20 activities and documents the priorities for FY21.
Mountains are the natural water towers of the world, effectively turning water vapor into readily available fresh water through precipitation, snowpack, and runoff. Unfortunately, Earth system models (ESMs) have persistently been unable to predict the timing and availability of water resources from mountains because the source(s) of model error are difficult to isolate in complex terrain with limited atmospheric or land-surface observations. Further complications arise from the gross scale mismatch between ESM grid box sizes and the relevant scales of mountainous hydrological processes. The mountain hydrometeorology community has repeatedly called for integrated atmospheric and land observations of water and energy budgets in complex terrain that span these scales to establish benchmarks against which scale-dependent models can be further developed.
Abstract. The U.S. Department of Energy Atmospheric Radiation Measurement program Tropical Western Pacific site hosted a C-band polarization (CPOL) radar in Darwin, Australia. It provides 2 decades of tropical rainfall characteristics useful for validating global circulation models. Rainfall retrievals from radar assume characteristics about the droplet size distribution (DSD) that vary significantly. To minimize the uncertainty associated with DSD variability, new radar rainfall techniques use dual polarization and specific attenuation estimates. This study challenges the applicability of several specific attenuation and dual-polarization-based rainfall estimators in tropical settings using a 4-year archive of Darwin disdrometer datasets in conjunction with CPOL observations. This assessment is based on three metrics: statistical uncertainty estimates, principal component analysis (PCA), and comparisons of various retrievals from CPOL data. The PCA shows that the variability in R can be consistently attributed to reflectivity, but dependence on dual-polarization quantities was wavelength dependent for 1 10mmh-1. Rainfall estimates during these conditions primarily originate from deep convective clouds with median drop diameters greater than 1.5 mm. An uncertainty analysis and intercomparison with CPOL show that a Colorado State University blended technique for tropical oceans, with modified estimators developed from video disdrometer observations, is most appropriate for use in all cases, such as when 1 10mmh-1 (deeper convective rain).
Raw data from ARM precipitation radars must be corrected for atmospheric phenomena and instrument characteristics (e.g., attenuation, clutter) to retrieve precipitation properties. The Corrected Moments in Antenna Coordinates Version 2 (CMAC2) value-added product (VAP) is a set of algorithms and code that does such corrections, and it also retrieves precipitation quantities from the radar measurements. Similar to the X-SAPR radars at the ARM main site, C-SAPR2 radar data also needs corrections and improvements. CMAC 2.0 has been updated to work with the ARM C-SAPR 2 radar. Data and fields that have been processed to: correct for velocity aliasing, unfold and generate a cross-polarimetric phase difference that is monotonically increasing, removing impulses caused by non-uniform beam filling and phase shift on backscatter, recalculate specific differential phase using a 20-point Sobel filter on the aforementioned phase, correct for liquid path attenuation using the polarimetric signals, estimate rainfall rates at the gate using the specific attenuation. In addition, CMAC writes the data out into a community-standard format netCDF File using the CF/Radial conventions. The data are therefore compatible with new and existing National Center for Atmospheric Research (NCAR) tools such as RadXConvert for converting to a variety of popular file formats.
This software assimilates data from an arbitrary number of weather radars together with other spatial wind fields (eg numerical weather forecasting model data) in order to retrieve high resolution three dimensional wind fields. PyDDA uses NumPy and SciPy’s optimization techniques combined with the Python Atmospheric Radiation Measurement (ARM) Radar Toolkit (Py-ART) in order to create wind fields using the 3D variational technique (3DVAR). PyDDA is hosted and distributed on GitHub at https://github.com/openradar/PyDDA. PyDDA has the potential to be used by the atmospheric science community to develop high resolution wind retrievals from radar networks. These retrievals can be used for the evaluation of numerical weather forecasting models and plume modelling. This paper shows how wind fields from 2 NEXt generation RADar (NEXRAD) WSR-88D radars and the High Resolution Rapid Refresh can be assimilated together using PyDDA to create a high resolution wind field inside Hurricane Florence.
C-SAPR2 data processed and corrected from the CACTI campaign, using CMAC2.0.
Objective analysis (OA) is a method of mapping unstructured data to a structured grid. In the context of scanning radar data, OA is used to interpolate data in antenna coordinates (range, azimuth, and elevation) onto a regularly spaced Cartesian grid (Trapp and Doswell 2000). The U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility’s Mapped Moments to a Cartesian Grid (MMCG) Value-Added Product (VAP) uses the Python ARM Radar Toolkit (Py-ART), a data model-driven interactive architecture for working with weather radar data, to map the data to a Cartesian grid (Helmus and Collis 2016). MMCG, with Py-ART built in, has the ability to take radar data in antenna coordinates and map the gates to a Cartesian grid using inverse distance weight functions such as Cressman (square) and Barnes (exponential), but also can filter the data during the interpolation. MMCG also allows arbitrary formulations for the radius of influence, which are matched to particular radar scanning strategies. This creates a complex parameter space for optimizing the retention of storm structure detail while minimizing artifacts. MMCG takes data processed with ARM’s Corrected Precipitation Radar Moments in Antenna Coordinates (CMAC) VAP and maps it to a Cartesian grid as the output product. A variety of fields that have been mapped to the Cartesian grid are then saved to plots to complement each grid file.
This dataset contains Velocity Azimuth Displays from the XSAPR radars in SGP. Velocity Azimith Displays provide estimates of the 2D wind direction with height from radar scans. For more information on how these Velocity Azimuth Displays are derived, see Michelson and Seaman (2000). Michelson, S.A. and N.L. Seaman, 2000: Assimilation of NEXRAD-VAD Winds in Summertime Meteorological Simulations over the Northeastern United States. J. Appl. Meteor., 39, 367–383, https://doi.org/10.1175/1520-0450(2000)039<0367:AONVWI>2.0.CO;2
Rapidly expanding, new telecommunications infrastructure based on 5G technologies will disrupt and transform how we design, build, operate, and optimize scientific infrastructure and the experiments and services enabled by that infrastructure, from continental-scale sensor networks to centralized scientific user facilities, from intelligent Internet of Things devices to supercomputers. Concurrently, 5G will introduce, or exacerbate, challenges related to protecting infrastructure and associated scientific data as well as to fully leveraging opportunities related to expanded infrastructure scale and complexity. The U.S. Department of Energy (DOE) Office of Science operates scientific infrastructure, supporting some of the nation’s most advanced intellectual discoveries, spanning the country and including 30 world-class user facilities from supercomputers to accelerators. Along with field experiments and remote observatories, every aspect of DOE’s scientific enterprise will be affected by 5G, which amounts to a complete renovation of the underpinnings of the nation’s information infrastructure. In this report we explore the scientific opportunities and new research challenges associated with 5G, ranging from scalability to heterogeneity to cybersecurity. The rapid commercial deployment of 5G opens the opportunity to rethink and reinvent DOE’s scientific infrastructure and experimentation, from intelligent sensor networks at unprecedented scales to a digital continuum of cyberinfrastructure spanning low-power sensors, high-performance computing embedded within and at the edge of the network, and DOE’s large-scale user instrument and computing facilities. New programming paradigms, workflow and data frameworks, and AI-based system design, operation, and autonomous adaptation and optimization will be necessary in order to exploit these new opportunities. Field deployments and centralized scientific instruments can also be revolutionized, moving (without traditional performance penalties) from wired to wireless connectivity for data and control systems, improving flexibility, and opening new sensing modalities, including the use of the 5G electromagnetic spectrum itself as an environmental probe. For DOE science, in contrast to commercial 5G applications and settings, devices will be deployed in extreme environments such as cryogenically cooled instrument control systems and in remote settings with harsh conditions, requiring the design of new materials for RF communication and edge processing to operate in these regimes. Concurrently, 5G infrastructure comprises both hardware and sophisticated software systems - currently closed and proprietary. The cybersecurity challenges to 5G-empowered reinvention mirror the complexity and variety of new 5G features, from virtualization to private network slices to ubiquitous access. Research is also needed in order to accelerate the development of secure and open 5G software infrastructure, reducing reliance on hardware and software produced outside the United States and providing the transparency and rigorous evaluation and testing afforded through open software. Twelve broad research thrusts are laid out in four chapters, with a companion fifth chapter (and three additional research thrusts) underscoring the needs and opportunities for an aggressive testbed program co-designed by networking experts and scientists involved in the 15 research thrusts. The urgency of undertaking this research is fueled by a global, accelerating deployment of new telecommunications infrastructure that is designed for entertainment and commercial applications - barely scratching the surface of what 5G can do to extend U.S. leadership in scientific discovery.
The Stockpile Stewardship Program (SSP) is an integrated technical program for maintaining the safety, surety, and reliability of the U.S. nuclear stockpile. The SSP incorporates nuclear test data, computational modeling and simulation, and experimental facilities to advance understanding of nuclear weapons. The suite of data analyzed comes from activities including stockpile surveillance, experimental research, and development and engineering programs. This integrated national program requires the continued use of experimental facilities and the computational capabilities to support the SSP missions. These component parts, in addition to an appropriately scaled production capability, enable the National Nuclear Security Administration (NNSA) to support stockpile requirements. The ultimate goal of the Stewardship Program, and thus the Advanced Simulation and Computing (ASC) Program, is to ensure that the U.S. maintains a safe, secure, and effective strategic deterrent.
In order to address mission-relevant questions, DOE’s Energy Exascale Earth System Model (E3SM) is pushing the limits of horizontal resolution among global Earth System Models (ESMs). The atmospheric resolution in the recently released E3SMv1 is 110 km in the standard configuration, and 25 km in the high-resolution configuration. Furthermore, E3SM is fairly unique among global ESMs with a capability for multiresolution modeling using refined grids in most of its components (atmosphere, land, ocean, sea-ice). This Regionally Refined Model (RRM) capability is critical for future E3SM simulation campaigns that have a strong regional focus to meet DOE's needs in support of energy-sector decisions. RRM grids are also particularly well suited for model evaluation and process-level studies taking advantage of the vast amount of ASR site data collected over the past several decades. The overarching goal of this project was to understand and improve the behavior of clouds and convection in E3SM across these scales using RRM. Key deliverables that resulted from the project are summarized below, with more details provided in the subsequent sections of the report.
PyDDA (Pythonic Direct Data Assimilation) is a new community framework aimed at wind retrievals that depends only upon utilities in the SciPy ecosystem such as scipy, numpy, and dask. It can support retrievals of winds using information from weather radar networks constrained by high resolution forecast models over grids that cover thousands of kilometers at kilometer-scale resolution. Unlike past wind retrieval packages, this package can be installed using anaconda for easy installation and, with a focus on ease of use can retrieve winds from gridded radar and model data with just a few lines of code. The package is currently available for download at https://github.com/openradar/PyDDA.
To probe the potential value of a radar-driven field campaign to constrain simulation of isolated convection subject to a strong aerosol perturbation, convective cells observed by the operational KHGX weather radar in the vicinity of Houston, Texas, are examined individually and statistically. Cells observed in a single case study of onshore flow conditions during July 2013 are first examined and compared with cells in a regional model simulation. Observed and simulated cells are objectively identified and tracked from observed or calculated positive specific differential phase (K(sub DP)) above the melting level, which is related to the presence of supercooled liquid water. Several observed and simulated cells are subjectively selected for further examination. Below the melting level, we compare sequential cross sections of retrieved and simulated raindrop size distribution parameters. Above the melting level, we examine time series of KDP and radar differential reflectivity (Z(sub DR)) statistics from observations and calculated from simulated supercooled rain properties, alongside simulated vertical wind and supercooled rain mixing ratio statistics. Results indicate that the operational weather radar measurements offer multiple constraints on the properties of simulated convective cells, with substantial value added from derived K(sub DP) and retrieved rain properties. The value of collocated three-dimensional lightning mapping array measurements, which are relatively rare in the continental US, supports the choice of Houston as a suitable location for future field studies to improve the simulation and understanding of convective updraft physics. However, rapid evolution of cells between routine volume scans motivates consideration of adaptive scan strategies or radar imaging technologies to amend operational weather radar capabilities. A 3-year climatology of isolated cell tracks, prepared using a more efficient algorithm, yields additional relevant information. Isolated cells are found within the KHGX domain on roughly 40 % of days year-round, with greatest concentration in the northwest quadrant, but roughly 5-fold more cells occur during June through September. During this enhanced occurrence period, the cells initiate following a strong diurnal cycle that peaks in the early afternoon, typically follow a south-to-north flow, and dissipate within 1 h, consistent with the case study examples. Statistics indicate that ∼ 150 isolated cells initiate and dissipate within 70 km of the KHGX radar during the enhanced occurrence period annually, and roughly 10 times as many within 200 km, suitable for multi-instrument Lagrangian observation strategies. In addition to ancillary meteorological and aerosol measurements, robust vertical wind speed retrievals would add substantial value to a radar-driven field campaign.
PyDDA is an expandable framework that integrates data from weather radars and forecasting models using SciPy’s optimization package to create meteorological fields.
Climate Model Development and Validation: (1) DOE's E3SM model being developed with goal of an increased resolution of 13 km (right arrow) assumptions made in convective parameterizations may not apply. (2) Need long term dataset with quantifiable large scale forcings to evaluate performance of convective parameterizations. (3) Vertical velocities are critical for calculating mass fluxes but are poorly represented in GCMs. (4) Dual Doppler techniques can retrieve vertical velocities, but uncertainties can be high due to sampling, mass continuity assumptions, fall speed assumptions, boundary conditions. (5) Can use high-resolution model simulated radar variables to assess impacts of such uncertainties
No abstract available
No abstract available