Joint statistical clustering of spatio-temporal Weather data to predict wildfires
Explore the source record for details and available documents.
Sharma, Arjun [Sandia National Laboratories (SNL-N↗
SEARCH · Search NASA
Results for “temporal clustering”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
How time weathers galaxies: the temporal impact of the cluster environment on galaxy formation and evolution
Explore the source record for details and available documents.
Stephanie O’Neil↗
A search for clustering among the meteoroid impacts detected by the Apollo lunar seismic network
A difference is noted between meteoroids whose masses lie above and below 1 kg in temporal meteoroid-impact clusterings detected by the Apollo lunar seismic network; while the former exhibit little temporal clustering, small ones show strong clusterings identifiable with terrestrial meteor showers. This finding suggests a different type and origin for the two meteoroid sizes; the small may be primarily cometary, while the large may derive from near-earth asteroids and short-period comets. It is speculated that the June, 1975, and January, 1977 swarms of large meteoroids may contain high-density meteoritic objects interpretable as meteorite streams.
Oberst, Juergen↗
Scalable, In-situ Data Clustering Data Analysis for Extreme Scale Scientific Computing (Final Report)
The objective of this project is to address challenges in the design and development of scalable in-situ data clustering and analytics algorithms and software. Our goal is to develop parallel software consisting of a set of spatio-temporal data clustering and anomaly detection functions, both of which are very important for large-scale analysis and have wide applicability for in-situ runs as well as post-processing analysis. Our design principles for in-situ analysis consider the following: (1) identify parts of the computation can be done close to the data within the nodes, while it is still in memory; (2) extract analysis components can (and should) be performed in remote staging and analysis nodes; (3) develop error-bound approximation methods for applications tolerable for small errors; (4) identify the type of derived distributions and statistics, for spatio-temporal data, that can be kept locally in order to both accelerate computations and meet energy constraints in subsequent iterations and phases; (5) use a self-describing data format so that data can be consistent and understood among local storage (memory and SSDs) and at staging and analysis nodes, thereby providing portability and flexibility; (6) develop service-oriented functions that can schedule in-situ and post-hoc analysis tasks based on the dynamic requirements of applications. Our development focus is to produce the parallel data analysis software/library that will be scalable, reusable, extensible, and generic for applications in different disciplines. The software will be able to run in-situ with the simulations as well as post-hoc analysis. This approach will satisfy many synergistic requirements for data intensive applications executed on data coming from instruments and experiments. In particular, the proposed multilevel approach is directly applicable to perform design tradeoffs for running part of the algorithms near the instruments and the rest on remote (analysis) systems.
97 MATHEMATICS AND COMPUTING↗
Solar Wind/Magnetosphere/Ionosphere Coupling and the Temporal and Spatial Evolution of Boundary Layers using Cluster, Polar and other ISTP Satellites
Our analysis concerns macro and meso-scale aspects of coupling between the IMF and the magnetosphere-ionosphere system, as opposed to the microphysics of determining how electron gyrotropy is broken and merging actually occurs. We correlate observed behaviors at Cluster and at Polar with temporal variations in other regions, such as in the ionosphere as measured by SuperDARN. Addressing problems with simultaneous observations from diverse locations properly constrains our interpretations.
Maynard, Nelson C.↗
Mapping crops within the growing season across the United States
Not Available
54 ENVIRONMENTAL SCIENCES↗
Atmospheric River Frequency-Category Characteristics Shape U.S. West Coast Runoff
Abstractrunoff response to atmospheric rivers (ARs) over the U.S. West Coast. We focused on runoff time series variations impacted by AR characteristics (e.g., category and frequency) and land preconditions during Northern Hemisphere cool seasons in the period of 1940–2023. Results show that high-category ARs significantly increase local runoff with higher hourly precipitation rates leading to a greater incremental rate and peak runoff. Extreme runoff increases greatly with the AR category with an increase rate up to 12.5 times stronger than non-extreme runoff. Besides the AR category, land preconditions such as soil moisture and snowpack also play crucial roles in modulating runoff response. We found that runoff induced by weak-category ARs is more sensitive to land preconditions than high-category ARs, with high peak runoff occurring when soil is nearly saturated. Additionally, more than 50% of high-peak-runoff events in snow-covered grid cells are associated with rain-on-snow events particularly for the events associated with weaker ARs. Regression analysis reveals that AR precipitation and land preconditions jointly influence runoff, emphasizing the importance of including soil moisture and snowpack levels in AR impact assessments. The study also highlights the intensified runoff response to back-to-back ARs with short intervals, which may become more frequent with climate warming, posing increased flood risks via facilitating wet soil conditions. Our findings have significant implications for AR risk predictions and the development of prediction models for AR-induced runoff.
54 ENVIRONMENTAL SCIENCES↗
Evaluation of Clustering Algorithms for Spatio-Temporal Multivariate Weather Data
Explore the source record for details and available documents.
Environmental sciences↗
Geospatial analysis of preterm and small-for-gestational age births in Washington D.C.
Background: This study is based on the recognition that adverse pregnancy outcomes significantly affect maternal and infant health, leading to increased morbidity and mortality. These outcomes are shaped by a complex interplay of individual-level factors—like maternal age and education—and community-level influences, including socio-economic status and access to healthcare. Understanding these determinants is crucial for developing effective public health strategies, especially for marginalized populations, by identifying high-risk areas and informing targeted interventions that address both individual and structural barriers. Methods: We utilized geospatial analysis to explore the association between individual- and community-level factors and adverse pregnancy outcomes, specifically preterm birth (PTB) and small-for-gestational-age (SGA) birthweight in Washington, D.C. We used Empirical Bayes smoothing methods to calculate rates of adverse birth outcomes from 2010 to 2018 at the U.S. Census tract–level. Spatial scan statistics were used to investigate if adverse birth outcomes clustered in specific areas. ANOVA tests were conducted for individual- and community-level factors within identified clusters. Results: Spatial analysis identified significant high-risk clusters for PTB and SGA infants primarily in southeastern Washington, D.C., particularly in Wards 7 and 8. Individuals residing within these clusters experienced a 47% increased risk of PTB (RR = 1.467) and a 56% increased risk of SGA (RR = 1.560) compared to those outside clusters. Space–time analysis revealed temporal variation, with PTB clusters persisting from 2011 to 2014 and SGA clusters extending through 2017. Compared to low-risk clusters, high-risk clusters had younger birthing individuals (mean age ~26.5 vs. ~33 years), lower maternal college degree attainment (~20% vs. ~80%), higher rates of late or no prenatal care (~16% vs. 11%), and increased prevalence of smoking and hypertension (all P < 0.001). Community-level indicators showed lower median household incomes ($\$40,000$ vs. ~$\$105,000$), greater poverty (~16% vs. ~7% below $\$10,000$/year), higher public assistance use (~32% vs. ~5%), and reduced healthcare access (greater distances to emergency and specialty care) in high-risk areas (all P < 0.001). Neighborhood deprivation indices were significantly elevated, commutes were longer, and population density was lower in these clusters. These findings highlight that adverse birth outcomes cluster in neighborhoods with pronounced socioeconomic and health disparities. Conclusion: High-risk birth clusters highlight intertwined factors: individual, socio-economic, and geographic. Addressing these requires comprehensive interventions focusing on social and structural determinants of health.
Birth outcomes↗
Big Data and AI at DoE's Legacy Sites - 20546
More than 30 years have passed since DOE started the decommissioning of nuclear weapon complexes and the clean-up of soil and groundwater. All the sites have been collecting and archiving soil and groundwater monitoring datasets; particularly contaminant concentration time-series. These datasets provide unparalleled opportunities to understand the system behavior (including more fundamental hydrological and geochemical processes, the response to various perturbations, the long-term trend and environmental decay rate towards the regulatory limit). This understanding is critical for providing multiple lines of evidences that can support site closure. In this study, we explore the machine learning (ML) and artificial intelligence (AI) applications to the long-term soil and groundwater management at DoE's legacy sites. ML can improve our understanding of the subsurface systems, which is critical for long-term monitoring and management of the sites, while AI can automate or support some of decision-making processes (e.g., anomaly detection, monitoring well placements). The particular focuses are to develop general algorithms to: (1) to identify distinct spatiotemporal patterns and to identify several groups that have similar temporal behaviors, using unsupervised clustering methods, (2) identify the different temporal scales of hydrological responses to climate perturbations by time-series analysis, and (3) reduce the number of monitoring wells by identifying the minimum sufficient number of wells to capture the heterogeneity of the groundwater contaminant plume and concentration distribution, using the Gaussian Process model. We demonstrate our methodology at the Savannah River Site F-Area. (authors)
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
Evaluation of Traditional and Deep Clustering Algorithms for Multivariate Spatio-Temporal Data
Explore the source record for details and available documents.
Mathematics and Computing↗
FIRED (Fire Events Delineation): An Open, Flexible Algorithm and Database of US Fire Events Derived from the MODIS Burned Area Product (2001–2019)
Harnessing the fire data revolution, i.e., the abundance of information from satellites, government records, social media, and human health sources, now requires complex and challenging data integration approaches. Defining fire events is key to that effort. In order to understand the spatial and temporal characteristics of fire, or the classic fire regime concept, we need to critically define fire events from remote sensing data. Events, fundamentally a geographic concept with delineated spatial and temporal boundaries around a specific phenomenon that is homogenous in some property, are key to understanding fire regimes and more importantly how they are changing. Here, we describe Fire Events Delineation (FIRED), an event-delineation algorithm, that has been used to derive fire events (N = 51,871) from the MODIS MCD64 burned area product for the coterminous US (CONUS) from January 2001 to May 2019. The optimized spatial and temporal parameters to cluster burned area pixels into events were an 11-day window and a 5-pixel (2315 m) distance, when optimized against 13,741 wildfire perimeters in the CONUS from the Monitoring Trends in Burn Severity record. The linear relationship between the size of individual FIRED and Monitoring Trends in Burn Severity (MTBS) events for the CONUS was strong (R2 = 0.92 for all events). Importantly, this algorithm is open-source and flexible, allowing the end user to modify the spatio-temporal threshold or even the underlying algorithm approach as they see fit. We expect the optimized criteria to vary across regions, based on regional distributions of fire event size and rate of spread. We describe the derived metrics provided in a new national database and how they can be used to better understand US fire regimes. The open, flexible FIRED algorithm could be utilized to derive events in any satellite product. We hope that this open science effort will help catalyze a community-driven, data-integration effort (termed OneFire) to build a more complete picture of fire.
54 ENVIRONMENTAL SCIENCES↗
Spatio-Temporal Surrogates for Interaction of a Jet with High Explosives: Part II - Clustering Extremely High-Dimensional Grid-Based Data
Building an accurate surrogate model for the spatio-temporal outputs of a computer simulation is a challenging task. A simple approach to improve the accuracy of the surrogate is to cluster the outputs based on similarity and build a separate surrogate model for each cluster. This clustering is relatively straightforward when the output at each time step is of moderate size. However, when the spatial domain is represented by a large number of grid points, numbering in the millions, the clustering of the data becomes more challenging. In this report, we consider output data from simulations of a jet interacting with high explosives. These data are available on spatial domains of different sizes, at grid points that vary in their spatial coordinates, and in a format that distributes the output across multiple files at each time step of the simulation. We first describe how we bring these data into a consistent format prior to clustering. Borrowing the idea of random projections from data mining, we reduce the dimension of our data by a factor of thousand, making it possible to use the iterative k-means method for clustering. We show how we can use the randomness of both the random projections, and the choice of initial centroids in k-means clustering, to determine the number of clusters in our data set. Our approach makes clustering of extremely high dimensional data tractable, generating meaningful cluster assignments for our problem, despite the approximation introduced in the random projections.
97 MATHEMATICS AND COMPUTING↗
Unsupervised learning from three-component accelerometer data to monitor the spatiotemporal evolution of meso-scale hydraulic fractures
Enhanced geothermal systems can provide a substantial share of the global energy demand. There exist several hurdles in the engineering implementations of such geothermal systems. One such hurdle is the accurate monitoring of the fracture networks created in subsurface through hydraulic stimulation of these systems. Micro seismicity associated with the stimulation is the primary means to locate the event hypocenters for estimating the stimulated rock volume. Existing methods for location the hypocenters are restricted to only the highest amplitude impulsive signals that are simultaneously detected on several sensors. Consequently, a large portion (usually ~99%) of the measurements are left unused. In this paper, an unsupervised manifold-approximation followed by clustering of 3-component accelerometer data is used to analyze the seismicity recorded on a monitoring well. With this method, a larger portion of the measured signal is used for the monitoring of the hydraulic fracture network. We analyze the EGS Collab experiment 1 microseismic data, recorded at the Sanford Underground Research Facility, South Dakota. Using the data from a single three-component accelerometer, the polarization features viz. Azimuth, incidence, rectilinearity, and planarity are used as inputs for the unsupervised manifold approximation followed by clustering. Our study shows that density-based clusters in the projected 3D space correspond to distinct types of hydraulically fractured zones around the injection point. Finally, we show that the temporal evolution of these clusters can be used to track fracture creation and propagation.
58 GEOSCIENCES↗
Using Clustering to Establish Climate Regimes from PCM Output
A multivariate statistical clustering technique--based on the k-means algorithm of Hartigan has been used to extract patterns of climatological significance from 200 years of general circulation model (GCM) output. Originally developed and implemented on a Beowulf-style parallel computer constructed by Hoffman and Hargrove from surplus commodity desktop PCs, the high performance parallel clustering algorithm was previously applied to the derivation of ecoregions from map stacks of 9 and 25 geophysical conditions or variables for the conterminous U.S. at a resolution of 1 sq km. Now applied both across space and through time, the clustering technique yields temporally-varying climate regimes predicted by transient runs of the Parallel Climate Model (PCM). Using a business-as-usual (BAU) scenario and clustering four fields of significance to the global water cycle (surface temperature, precipitation, soil moisture, and snow depth) from 1871 through 2098, the authors' analysis shows an increase in spatial area occupied by the cluster or climate regime which typifies desert regions (i.e., an increase in desertification) and a decrease in the spatial area occupied by the climate regime typifying winter-time high latitude perma-frost regions. The patterns of cluster changes have been analyzed to understand the predicted variability in the water cycle on global and continental scales. In addition, representative climate regimes were determined by taking three 10-year averages of the fields 100 years apart for northern hemisphere winter (December, January, and February) and summer (June, July, and August). The result is global maps of typical seasonal climate regimes for 100 years in the past, for the present, and for 100 years into the future. Using three-dimensional data or phase space representations of these climate regimes (i.e., the cluster centroids), the authors demonstrate the portion of this phase space occupied by the land surface at all points in space and time. Any single spot on the globe will exist in one of these climate regimes at any single point in time. By incrementing time, that same spot will trace out a trajectory or orbit between and among these climate regimes (or atmospheric states) in phase (or state) space. When a geographic region enters a state it never previously visited, a climatic change is said to have occurred. Tracing out the entire trajectory of a single spot on the globe yields a 'manifold' in state space representing the shape of its predicted climate occupancy. This sort of analysis enables a researcher to more easily grasp the multivariate behavior of the climate system.
Oglesby, Robert↗
An interregional optimization approach for time series aggregation in continent-scale electricity system models
Modeling electric power systems with high shares of weather-dependent resources requires tradeoffs between temporal, spatial, and operational resolution. Many studies perform time series aggregation using clustering algorithms to reduce the temporal dimension, but when modeling continent-scale electricity systems that are large enough to contain multiple independent weather systems, this approach requires large numbers of representative periods to minimize errors in regional wind and solar capacity factors. Here, a new optimization-based approach for representative period selection and weighting is introduced that minimizes regional errors in average renewable capacity factors and electricity demand. The method delivers higher regional fidelity with fewer representative periods than alternative clustering methods when applied to wind, solar, and demand profiles for the contiguous United States. When representative periods are selected from multiple weather years, the optimized method reproduces regional averages with lower error than a complete 365-day time series from any single weather year. The method identifies only representative (as opposed to outlying) periods but can be combined with an iterative "stress period" identification approach to guide efficient decision-making considering both average and high-risk weather conditions.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Unsupervised learning-enabled pulsed infrared thermographic microscopy of subsurface defects in stainless steel
Metallic structures produced with laser powder bed fusion (LPBF) additive manufacturing method (AM) frequently contain microscopic porosity defects, with typical approximate size distribution from one to 100 microns. Presence of such defects could lead to premature failure of the structure. In principle, structural integrity assessment of LPBF metals can be accomplished with nondestructive evaluation (NDE). Pulsed infrared thermography (PIT) is a non-contact, one-sided NDE method that allows for imaging of internal defects in arbitrary size and shape metallic structures using heat transfer. PIT imaging is performed using compact instrumentation consisting of a flash lamp for deposition of a heat pulse, and a fast frame infrared (IR) camera for measuring surface temperature transients. However, limitations of imaging resolution with PIT include blurring due to heat diffusion, sensitivity limit of the IR camera. We demonstrate enhancement of PIT imaging capability with unsupervised learning (UL), which enables PIT microscopy of subsurface defects in high strength corrosion resistant stainless steel 316 alloy. PIT images were processed with UL spatial–temporal separation-based clustering segmentation (STSCS) algorithm, refined by morphology image processing methods to enhance visibility of defects. The STSCS algorithm starts with wavelet decomposition to spatially de-noise thermograms, followed by UL principal component analysis (PCA), fine-tuning optimization, and neural learning-based independent component analysis (ICA) algorithms to temporally compress de-noised thermograms. The compressed thermograms were further processed with UL-based graph thresholding K-means clustering algorithm for defects segmentation. The STSCS algorithm also includes online learning feature for efficient re-training of the model with new data. For this study, metallic specimens with calibrated microscopic flat bottom hole defects, with diameters in the range from 203 to 76 µm, were produced using electro discharge machining (EDM) drilling. While the raw thermograms do not show any material defects, using STSCS algorithm to process PIT images reveals defects as small as 101 µm in diameter. To the best of our knowledge, this is the smallest reported size of a sub-surface defect in a metal imaged with PIT, which demonstrates the PIT capability of detecting defects in the size range relevant to quality control requirements of LPBF-printed high-strength metals.
36 MATERIALS SCIENCE↗
Simulating water dynamics related to pedogenesis across space and time: Implications for four-dimensional digital soil mapping
Digital soil mapping (DSM) relies on machine-learning and geostatistics to represent soil property observations across space. DSM techniques are powerful but often empirical, being limited to the quality and density of point samples. Water dynamics are closely related to soil variability, and the physics that govern water movement are well known. Hydrological properties can hence be simulated by physical models through space and time, unveiling key characteristics about soils. We propose the use of hydrologic models to map soils across the surface (2D), depth (1D), and time (1D)–which provides a 4D approach to digital soil mapping (4DSM). The Distributed Hydrology Soil Vegetation Model (DHSVM) was applied to a watershed currently under pasture. Moisture sensors and wells were installed at different depths in the watershed on summit, sideslope and toeslope positions to validate the model. DHSVM simulations of soil moisture distribution and depth to saturation were performed during the hydrological year (October 2008-September 2009). Clusters of similar pixels based on soil moisture values were determined using Dynamic Time Warping (DTW) to align temporal data and K-means. Clustering was performed both seasonally and for the entire year. Temporal patterns simulated by DHSVM matched measurements given by moisture sensors and wells. Seasonal clusters differed from the annual cluster. Distinct clusters were observed for each season and with depth, showing that spatiotemporal soil variability is lost when statically assessing soils. Spatiotemporal clusters corroborated field observations of fragipan occurrence not explicitly spatially mapped by Soil Survey Geographic Database (SSURGO). If a connection can be made between water and soils, static and dynamic soil variability can be predicted using physically based hydrologic models. Hydrologic models can benefit soil mapping by enabling reliable 4D simulation of water dynamics, which are fundamental to soil variability and soil classification and directly relate to biological, physical and chemical soil processes not captured by typical soil sampling protocols.
54 ENVIRONMENTAL SCIENCES↗