A Flash Clustering Algorithm for North Alabama Lightning Mapping Array Data
No abstract available
flash↗
SEARCH · Search NASA
Results for “Flash clustering algorithm”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Lightning Strike Distance Distribution Beyond a Preexisting Lightning Area
The 45th Weather Squadron (45 WS) asked the Applied Meteorology Unit (AMU) to review the 30-year-old, lightning stand-off distances of 5 nautical miles (nmi) for applicability to today's operations. This was based on the realization that previous lightning strike distance studies did not match how 45 WS issues lightning warnings (Roeder, 2008). The previous lightning distance studies were from the point of origin of the lightning or the average starting location that would tend to be in the core of the thunderstorm. However, the 45 WS issues lightning warnings based on the edge of a preexisting lightning area. Before beginning the AMU project, it took several years to develop a method to calculate a distance distribution beyond a preexisting area (Roeder, 2015). The AMU pulled Lightning Detection and Ranging (LDAR) sensor data from 1/1/2013 to 12/31/2013. This dataset consisted of 37 million individual source data points from the LDAR sensors. Only sources within 50 km north, south, east or west of the LDAR grid center were included in the dataset. This limited the use of LDAR data to that with the greatest accuracy of source detection and increased data processing. Points were grouped into flashes based on spatial and temporal criteria. Based on the sensitivity analysis the AMU performed on the flash clustering algorithm, a time value of 0.3 seconds was found to model flashes adequately. Distance parameters were tested from 1,500 to 7,500 meters (m) in 500 m increments. Distance parameters of both 3,000 m and 4,000 m produced results in the plotting tool that were most representative of the physical behavior of lightning. Thus statistics were gathered for the most representative of these spatial and temporal criteria on the flash size and the polygon expansion distance in order to find the correct data distributions. The best fit curves for the LDAR polygon expansion frequency vs. distances for both the 3 kilometer (km) and 4 km distance threshold values were exponential decay functions and had R2 values of > 0.998, indicating good model fits. The equations of the best fit curves were then used to calculate a desired safety radius of 4 nmi for either 3 km or 4 km distance threshold criteria. The AMU analysis concludes the safe reduction of the 5 nmi lightning warning circles to 4 nmi should improve the operational impact by 36% if based on distance from the center of the property area being protected. If based on the edge of the property being protected, then the reduction is 4.5 nmi to 4 nmi and the operational impact is decreased by 21%. For the 6 nmi lightning warning circles, the recommended 4 nmi stand-off distance will result in a safe reduction of operational impact of 31% if based on the center of the area being protected, or 16% if based on the edge of the property.
Flash clustering algorithm↗
A New CIERRA Gridded Product Climatology for LIS/OTD
The GLM-CIERRA processing generates two products: A level-2 cluster feature dataset that merges flashes that should have been a single flash following the clustering algorithm flash definition (for GLM, this is due to the LCFA thresholds); A set of level-3 gridded products including Flash Extent Density, Convective Probability, etc. The CIERRA Level-3 products are generated on a 0.1 degree grid. This is suboptimal given the spatial variations in GLM pixel size AND the size of gridpoints in the output grid. As a stepping-stone towards developing improved GLM-CIERRA grids, I am testing pixel matching techniques on the LIS/OTD data. The goal is to maintain accuracy while vastly improving computational efficiency. The CIERRA reclustering codes are also being applied to LIS / OTD to merge flashes split by the “first fit” clustering technique. The end result will be standardized grids at the nominal resolution of each instrument (5 km for LIS, 10 km for OTD) that take into account the pixel geometry for each event that comprises a given reclustered flash.
54 ENVIRONMENTAL SCIENCES↗
Performance Assessment of the Optical Transient Detector and Lightning Imaging Sensor: Clustering Algorithm - Part 2
We describe the clustering algorithm used by the Lightning Imaging Sensor (LIS) and the Optical Transient Detector (OTD) for combining the lightning pulse data into events, groups, flashes, and areas. Events are single pixels that exceed the LIS/OTD background level during a single frame (2 ms). Groups are clusters of events that occur within the same frame and in adjacent pixels. Flashes are clusters of groups that occur within 330 ms and either 5.5 km (for LIS) or 16.5 km (for OTD) of each other. Areas are clusters of flashes that occur within 16.5 km of each other. Many investigators are utilizing the LIS/OTD flash data; therefore, we test how variations in the algorithms for the event group and group-flash clustering affect the flash count for a subset of the LIS data. We divided the subset into areas with low (1-3), medium (4-15), high (16-63), and very high (64+) flashes to see how changes in the clustering parameters affect the flash rates in these different sizes of areas. We found that as long as the cluster parameters are within about a factor of two of the current values, the flash counts do not change by more than about 20%. Therefore, the flash clustering algorithm used by the LIS and OTD sensors create flash rates that are relatively insensitive to reasonable variations in the clustering algorithms.
Mach, Douglas M.↗
GOES-R AWG GLM Val Tool Development
We are developing tools needed to enable the validation of the Geostationary Lightning Mapper (GLM). In order to develop and test these tools, we have need of a robust, high-fidelity set of GLM proxy data. Many steps have been taken to ensure that the proxy data are high quality. LIS is the closest analog that exists for GLM, so it has been used extensively in developing the GLM proxy. We have verified the proxy data both statistically and algorithmically. The proxy data are pixel (event) data, called Level 1B. These data were then clustered into flashes by the Lightning Cluster-Filter Algorithm (LCFA), generating proxy Level 2 data. These were then compared with the data used to generate the proxy, and both the proxy data and the LCFA were validated. We have developed tools to allow us to visualize and compare the GLM proxy data with several other sources of lightning and other meteorological data (the so-called shallow-dive tool). The shallow-dive tool shows storm-level data and can ingest many different ground-based lightning detection networks, including: NLDN, LMA, WWLLN, and ENTLN. These are presented in a way such that it can be seen if the GLM is properly detecting the lightning in location and time comparable to the ground-based networks. Currently in development is the deep-dive tool, which will allow us to dive into the GLM data, down to flash, group and event level. This will allow us to assess performance in comparison with other data sources, and tell us if there are detection, timing, or geolocation problems. These tools will be compatible with the GLM Level-2 data format, so they can be used beginning on Day 0.
Bateman, Monte↗
Comparing Synoptic Pattern Evolution for Flash‐Flood‐Producing and Non‐Flash‐Flood‐Producing Mesoscale Convective Systems in the United States
Understanding how the short-term evolution of synoptic weather patterns influence Mesoscale Convective Systems (MCSs) is essential, as these systems are responsible for over half of central U.S. flash floods, leading to substantial socioeconomic and water resource management impacts. This study analyzes long-term MCS data, flash flood reports, and atmospheric reanalyses from 2007 to 2017 using a machine learning clustering algorithm to examine how the synoptic weather patterns evolve prior to MCS initiation. While the clusters reflect seasonal and regional differences in MCS occurrence, they do not consistently distinguish between MCSs that do and do not produce flash floods. Systems in the southern Great Plains are more flood-prone when a synoptic-scale forcing, located near the system, drives strong water vapor transport from the nearby moisture source. More generally under different synoptic weather patterns, a broader precipitating area is the most dominant factor governing MCS flash flood potential.
atmospheric dynamics↗
Automated Storm Tracking and the Lightning Jump Algorithm Using GOES-R Geostationary Lightning Mapper (GLM) Proxy Data
This study develops a fully automated lightning jump system encompassing objective storm tracking, Geostationary Lightning Mapper proxy data, and the lightning jump algorithm (LJA), which are important elements in the transition of the LJA concept from a research to an operational based algorithm. Storm cluster tracking is based on a product created from the combination of a radar parameter (vertically integrated liquid, VIL), and lightning information (flash rate density). Evaluations showed that the spatial scale of tracked features or storm clusters had a large impact on the lightning jump system performance, where increasing spatial scale size resulted in decreased dynamic range of the system's performance. This framework will also serve as a means to refine the LJA itself to enhance its operational applicability. Parameters within the system are isolated and the system's performance is evaluated with adjustments to parameter sensitivity. The system's performance is evaluated using the probability of detection (POD) and false alarm ratio (FAR) statistics. Of the algorithm parameters tested, sigma-level (metric of lightning jump strength) and flash rate threshold influenced the system's performance the most. Finally, verification methodologies are investigated. It is discovered that minor changes in verification methodology can dramatically impact the evaluation of the lightning jump system.
lightning jump↗
A VXS [VITA41] Trigger Processor for the 12GEV Experimental Programs at Jefferson Lab
The VXS_Trigger_Processor [VTP] was developed and commissioned for CLAS12 in the fall of 2016. This board is a VITA41 switch card and it collects data from a variety of front-end TDC and Flash ADC modules. The VTP has since been used in several experiments at Jefferson Lab serving as the L1 trigger module for a variety of detector types, such as stacked calorimeters, strip calorimeters, time-of-flight, Cerenkov, hodoscopes, drift chambers, and silicon strips. Trigger algorithms implemented include cluster finding (1D, 2D), drift chamber segment and road finding, geometry matching between various detectors, particle counting, and general global trigger bit processing. The VTP is also capable of reading out each front-end crate with up to 40Gbps Ethernet which is an enormous increase compared to the currently used 200MB/s VME bus. Recent progress has been made to show that a firmware and software upgrade can enable existing Jefferson Lab front-end crates to operate in a streaming DAQ mode. In February 2020, tests will be performed on a full calorimeter and matched hodoscope which are components of the CLAS12 Forward Tagger detector system with beam in Hall B. This paper details the hardware performance, triggered, and streaming applications that have been implemented using the VTP for several experiments at Jefferson Lab.
ABBOTT, David↗
The determination of the core mass at the helium flash in globular cluster stars
Evolutionary sequences for the red giant branch (RGB) phase of a representative globular cluster star have been computed in order to investigate the reliability of the current canonical values of the core mass M(sub c) at the helium flash. These computations were motivated by recent suggestions that the canonical values of M(sub c) may be systematically too small due to the numerical algorithms for shifting the hydrogen shell and advancing the chemical composition during the RGB phase. Our results show that these algorithms do not, in fact, introduce a significant error in the values of M(sub c). Moreover, we demonstrate that a procedure for advancing the chemical composition which is implicit only in the hydrogen abundance will systematically underestimate the amount of hydrogen fuel consumption between RGB models and therefore should not be used in RGB computations. Overall we estimate the uncertainty in the core masses of Sweigart & Gross (1978) due to numerical effects to be approximately equals 0.003 solar mass. From a consideration of the available canonical models we conclude that a change in the canonical values of M(sub c) by a few 10(exp -2) solar mass would require either a substantial change in the canonical input physics or some noncanonical effect such as rotation. Finally our models show that the use of short time steps can significantly increase the extent of the inner tail of the hydrogen shell. This effect may enhance the likelihood of hydrogen mixing following a helium shell flash in an asymptotic giant branch star.
Sweigart, Allen V.↗
Severe Storms Branch research report (April 1984 April 1985)
The Mesoscale Atmospheric Processes Research Program is a program of integrated studies which are to achieve an improved understanding of the basic behavior of the atmosphere through the use of remotely sensed data and space technology. The program consist of four elements: (1) special observations and analysis of mesoscale systems; (20 the development of quanitative algorithms to use remotely sensed observations; (3) the development of new observing systems; and (4) numerical modeling. The Severe Storms Branch objectives are the improvement of the understanding, diagnosis, and prediction of a wide range of atmospheric storms, which includes severe thunderstorms, tornadoes, flash floods, tropical cyclones, and winter snowstorms. The research often shed light upon various aspects of local weather, such as fog, sea breezes, air pollution, showers, and other products of nonsevere cumulus cloud clusters. The part of the program devoted to boundary layer processes, gust front interactions, and soil moisture detection from satellites gives insights into storm growth and behavior.
Dubach, L.↗
Unsupervised learning-enabled pulsed infrared thermographic microscopy of subsurface defects in stainless steel
Metallic structures produced with laser powder bed fusion (LPBF) additive manufacturing method (AM) frequently contain microscopic porosity defects, with typical approximate size distribution from one to 100 microns. Presence of such defects could lead to premature failure of the structure. In principle, structural integrity assessment of LPBF metals can be accomplished with nondestructive evaluation (NDE). Pulsed infrared thermography (PIT) is a non-contact, one-sided NDE method that allows for imaging of internal defects in arbitrary size and shape metallic structures using heat transfer. PIT imaging is performed using compact instrumentation consisting of a flash lamp for deposition of a heat pulse, and a fast frame infrared (IR) camera for measuring surface temperature transients. However, limitations of imaging resolution with PIT include blurring due to heat diffusion, sensitivity limit of the IR camera. We demonstrate enhancement of PIT imaging capability with unsupervised learning (UL), which enables PIT microscopy of subsurface defects in high strength corrosion resistant stainless steel 316 alloy. PIT images were processed with UL spatial–temporal separation-based clustering segmentation (STSCS) algorithm, refined by morphology image processing methods to enhance visibility of defects. The STSCS algorithm starts with wavelet decomposition to spatially de-noise thermograms, followed by UL principal component analysis (PCA), fine-tuning optimization, and neural learning-based independent component analysis (ICA) algorithms to temporally compress de-noised thermograms. The compressed thermograms were further processed with UL-based graph thresholding K-means clustering algorithm for defects segmentation. The STSCS algorithm also includes online learning feature for efficient re-training of the model with new data. For this study, metallic specimens with calibrated microscopic flat bottom hole defects, with diameters in the range from 203 to 76 µm, were produced using electro discharge machining (EDM) drilling. While the raw thermograms do not show any material defects, using STSCS algorithm to process PIT images reveals defects as small as 101 µm in diameter. To the best of our knowledge, this is the smallest reported size of a sub-surface defect in a metal imaged with PIT, which demonstrates the PIT capability of detecting defects in the size range relevant to quality control requirements of LPBF-printed high-strength metals.
36 MATERIALS SCIENCE↗
Smart Pixels: In-pixel AI for on-sensor data filtering
We present a smart pixel prototype readout integrated circuit (ROIC) designed in CMOS 28 nm bulk process, with in-pixel implementation of an artificial intelligence (AI) / machine learning (ML) based data filtering algorithm designed as proof-of-principle for a Phase III upgrade at the Large Hadron Collider (LHC) pixel detector. The first version of the ROIC consists of two matrices of 256 smart pixels, each 25$\times$25 $\mu$m$^2$ in size. Each pixel consists of a charge-sensitive preamplifier with leakage current compensation and three auto-zero comparators for a 2-bit flash-type ADC. The frontend is capable of synchronously digitizing the sensor charge within 25 ns. Measurement results show an equivalent noise charge (ENC) of $\sim$30e$^-$ and a total dispersion of $\sim$100e$^-$ The second version of the ROIC uses a fully connected two-layer neural network (NN) to process information from a cluster of 256 pixels to determine if the pattern corresponds to highly desirable high-momentum particle tracks for selection and readout. The digital NN is embedded in-between analog signal processing regions of the 256 pixels without increasing the pixel size and is implemented as fully combinatorial digital logic to minimize power consumption and eliminate clock distribution, and is active only in the presence of an input signal. The total power consumption of the neural network is $\sim$ 300 $\mu$W. The NN performs momentum classification based on the generated cluster patterns and even with a modest momentum threshold, it is capable of 54.4% - 75.4% total data rejection, opening the possibility of using the pixel information at 40MHz for the trigger. The total power consumption of analog and digital functions per pixel is $\sim$ 6 $\mu$W per pixel, which corresponds to $\sim$ 1 W/cm$^2$ staying within the experimental constraints.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Smart Pixels: In-pixel AI for on-sensor data filtering
We present a smart pixel prototype readout integrated circuit (ROIC) designed in CMOS 28 nm bulk process, with in-pixel implementation of an artificial intelligence (AI) / machine learning (ML) based data filtering algorithm designed as proof-of-principle for a Phase III upgrade at the Large Hadron Collider (LHC) pixel detector. The first version of the ROIC consists of two matrices of 256 smart pixels, each 25$\times$25 µm\textsuperscript{2} in size. Each pixel consists of a charge-sensitive preamplifier with leakage current compensation and three auto-zero comparators for a 2-bit flash-type ADC. The frontend is capable of synchronously digitizing the sensor charge within 25 ns. Measurement results show an equivalent noise charge (ENC) of $\sim$30e\textsuperscript{-} and a total dispersion of $\sim$100e\textsuperscript{-} The second version of the ROIC uses a fully connected two-layer neural network (NN) to process information from a cluster of 256 pixels to determine if the pattern corresponds to highly desirable high-momentum particle tracks for selection and readout. The digital NN is embedded in-between analog signal processing regions of the 256 pixels without increasing the pixel size and is implemented as fully combinatorial digital logic to minimize power consumption and eliminate clock distribution, and is active only in the presence of an input signal. The total power consumption of the neural network is $\sim$ 300 $\mu$W. The NN performs momentum classification based on the generated cluster patterns and even with a modest momentum threshold, it is capable of 54.4\% – 75.4\% total data rejection, opening the possibility of using the pixel information at 40MHz for the trigger. The total power consumption of analog and digital functions per pixel is $\sim$ 6 $\mu$W per pixel, which corresponds to $\sim$ 1 W/cm\textsuperscript{2} staying within the experimental constraints.
Parpillon, Benjamin↗
Climatology of Linear Mesoscale Convective System Morphology in the United States based on Random Forests Method
This study uses machine learning methods, specifically the random forest (RF), on a radar-based mesoscale convective system (MCS) tracking dataset to classify the five types of linear MCS morphology in the contiguous United States during the period 2004-2016. The algorithm is trained using radar- and satellite-derived spatial and morphological parameters, and reanalysis environmental information from 5-26yr manually identified nonlinear and five linear MCS modes. The algorithm is then used to automate the classification of linear MCSs over 8 years with high accuracy, providing a systematic, long-term climatology of linear MCSs. Results reveal that nearly 40% of MCSs are classified as linear MCSs, in which half of the linear events belong to the type of system having a leading convective line. The occurrence of linear MCSs shows large annual and seasonal variations. On average, 113 linear MCSs occur annually during the warm season (through March to October), with most of these events clustered from May through August in the central eastern Great Plains. MCS characteristics, including duration, propagation speed, orientation, and system cloud size, have large variability among the different linear modes. The systems having a trailing convective line and the systems having a back-building area of convection typically move more slowly and have higher precipitation rate, and thus have higher potential in producing extreme rainfall and flash flooding. Analysis of the environmental conditions associated with linear MCSs show that the storm-relative flow is of most importance in determining the organization mode of linear MCSs.
Cui, Wenjun↗