SEARCH · Search NASA
Results for “Cluster Analysis”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Machine-learning identification of the variability of mean velocity and turbulence intensity for wakes generated by onshore wind turbines: Cluster analysis of wind LiDAR measurements
Light detection and ranging (LiDAR) measurements of isolated wakes generated by wind turbines installed at an onshore wind farm are leveraged to characterize the variability of the wake mean velocity and turbulence intensity during typical operations, which encompass a breadth of atmospheric stability regimes and rotor thrust coefficients. The LiDAR measurements are clustered through the k-means algorithm, which enables identifying the most representative realizations of wind turbine wakes while avoiding the imposition of thresholds for the various wind and turbine parameters. Considering the large number of LiDAR samples collected to probe the wake velocity field, the dimensionality of the experimental dataset is reduced by projecting the LiDAR data on an intelligently truncated basis obtained with the proper orthogonal decomposition (POD). The coefficients of only five physics-informed POD modes are then injected in the k-means algorithm for clustering the LiDAR dataset. The analysis of the clustered LiDAR data and the associated supervisory control and data acquisition and meteorological data enables the study of the variability of the wake velocity deficit, wake extent, and wake-added turbulence intensity for different thrust coefficients of the turbine rotor and regimes of atmospheric stability. Furthermore, the cluster analysis of the LiDAR data allows for the identification of systematic off-design operations with a certain yaw misalignment of the turbine rotor with the mean wind direction.
Cluster analysis of occupancy schedules in residential buildings in the United States
The energy performance of residential buildings significantly depends on the building occupants’ behavior, which can be highly variable. When the heating, ventilation and air conditioning (HVAC) system is controlled based on the presence or absence of occupants in a building, occupant behavior is of even further importance to its energy performance. In current practice, building energy simulation tools generally use a single occupancy profile to represent the building’s occupancy schedule, the schedule of which is considered to be the same, regardless of the type of household being modeled. Thus, there is significant potential for improvement to allow for more flexibility and accuracy in calculation of occupancy. The objective of this study is to assess the variations in the typical types of occupancy schedules followed by the U.S. population using cluster analysis. American Time Use Survey data, which statically represents the overall U.S. population’s activities, across 12 years (2006–2017), is used. The ATUS data is segregated into smaller groups based on age and weekday/weekend, then divided into activities that are considered “at home” and “away from home”, which are mapped to the presence or non-presence of occupants in the home. Cluster analysis is then used to identify common types of occupancy schedule patterns for each age group. Three main types of patterns are obtained from cluster analysis for each age group, which together represent approximately 88% of people in the United States. The output of the cluster analysis is further analyzed to evaluate the variation in characteristics, including the number of times leaving home, time of day when leaving the home, and the timespan of absence from the home. The results of this study provide detailed insights on how typical occupants in the United States spend their time in residential spaces which can be used to create occupancy profiles for residential buildings. Finally, these occupancy profiles could be utilized inform an assessment of the energy use impact of occupancy-based controls of energy consuming systems and technologies.
Clustering Analysis of Commercial Vehicles Using Automatically Extracted Features from Time Series Data
Standard of practice approaches to time series cluster analysis involve careful feature engineering, often utilizing expert input to tune and select features by hand. In many cases, expert input may not be readily available, or there may not yet exist a community consensus on the ideal features for a given application. This paper compares the results of several cluster analysis methods, using both hand selected features and those extracted automatically, when applied to large geospatial time series telematics data from commercial trucking fleets. The impacts of feature selection, dimensionality reduction, and choice of clustering algorithm on the quality of clustering results are explored. Results from this analysis confirm prior results that domain agnostic features are competitive with the hand engineered features with respect to clustering quality metrics. These results also provide new insight into the most successful strategies for identifying structure in large unstructured vehicle telematics data, and suggest that time series clustering using automatic feature extraction can be an effective approach to extract structure from large scale geospatial time series data in cases when hand selected features are not available.
Characterization of Surface Heterogeneity-Induced Convection Using Cluster Analysis
Land-atmosphere interactions and boundary layer processes often control the formation of shallow clouds and subsequently deep convective precipitation over the Southern Great Plains. In this study, we examine the impacts of large-scale advection on the cloud populations and land-atmospheric coupling observed during the Holistic Interactions of Shallow Clouds, Aerosols, and Land-Ecosystems (HI-SCALE) field campaign in 2016. In this work, we performed two Large Eddy Simulations (LES) using the Weather Research and Forecasting Model for a day in which the transition from clear-sky to shallow precipitating clouds and isolated deep convection was observed. The control simulation reproduced the overall distribution of cloud populations by using realistic soil conditions with an interactive land model. In the sensitivity simulation where large-scale advection is removed, a strong relationship between the land and boundary layer is found. To study the timing, location, and intensity of convective initiation and the relationship of clouds with land surface properties, a cluster analysis of equivalent potential temperature is performed for the simulation without large-scale advection. That analysis shows that convective clouds first form over regions with higher surface sensible heat flux. Precipitation from those convective clouds likely triggers new updrafts nearby about two hours later through the lifting associated with cold pools. Furthermore, the cluster analysis also shows that in addition to the spatial pattern of soil moisture, land use and soil texture in western Oklahoma also influence the location of convective initiation.
Code Artifact for: Clustering Analysis of Commercial Vehicles Using Automatically Extracted Features from Time Series Data [SWR-21-96]
This repository contains data ingestion, feature extraction, and analysis code used in NREL Technical report "Clustering Analysis of Commercial Vehicles Using Automatically Extracted Features from Time Series Data." The code is written in Python. The ETL and feature extraction code must be run in a Spark context. The analysis code can be run without Spark, provided you have pre-computed features in a CSV file. Analysis code related to the NREL Technical Report NREL/TP-2C00-74212. Includes PySpark functions to perform trip segmentation and feature extraction over big time series data in Apache Spark. Includes "domain specific" features such as Aerodynamic Speed (ft/s), Characteristic Acceleration (ft/s2), Percent Below 55 (%), Percent Zero (%), Stops Per Mile, Average Speed (mph), Maximum Speed (mph), and Speed Standard Deviation (mph). Includes Pyspark UDF to compute "domain agnostic" features using the TSFresh library. This software record also includes the analysis notebooks and code to generate the results in the previously mentioned technical report.
Resilient Node Cluster Analysis Tool (ReNCAT) v. 2.1
SAND2021-6533 O The Resilient Node Cluster Analysis Tool (ReNCAT) identifies portions of a power distribution network that can be isolated into microgrids in order to supply services to a community during disruptive events. Candidate microgrid configurations are evaluated using a burden metric, a measure of the relative effort community members must expend to acquire services required for their well-being. Results present multiple candidate configurations, each with a different balance of cost and burden. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.
Subseasonal Representation and Predictability of North American Weather Regimes Using Cluster Analysis
Abstract This study focuses on assessing the representation and predictability of North American weather regimes, which are persistent large-scale atmospheric patterns, in a set of initialized subseasonal reforecasts created using the Community Earth System Model, version 2 (CESM2). The k -means clustering was used to extract four key North American (10°–70°N, 150°–40°W) weather regimes within ERA5 reanalysis, which were used to interpret CESM2 subseasonal forecast performance. Results show that CESM2 can recreate the climatology of the four main North American weather regimes with skill but exhibits biases during later lead times with overoccurrence of the West Coast high regime and underoccurrence of the Greenland high and Alaskan ridge regimes. Overall, the West Coast high and Pacific trough regimes exhibited higher predictability within CESM2, partly related to El Niño. Despite biases, several reforecasts were skillful and exhibited high predictability during later lead times, which could be partly attributed to skillful representation of the atmosphere from the tropics to extratropics upstream of North America. The high predictability at the subseasonal time scale of these case-study examples was manifested as an “ensemble realignment,” in which most ensemble members agreed on a prediction despite ensemble trajectory dispersion during earlier lead times. Weather regimes were also shown to project distinct temperature and precipitation anomalies across North America that largely agree with observational products. This study further demonstrates that unsupervised learning methods can be used to uncover sources and limits of subseasonal predictability, along with systematic biases present in numerical prediction systems. Significance Statement North American weather regimes are large-scale atmospheric patterns that can persist for several days. Their skillful subseasonal (2 weeks or greater) prediction can provide valuable lead time to prepare for temperature and precipitation anomalies that can stress energy and water resources. The purpose of this study was to assess the climatological representation and subseasonal predictability of four key North American weather regimes using a research subseasonal prediction system and clustering analysis. We found that the Pacific trough and West Coast high regimes exhibited higher predictability than other regimes and that skillful representation of conditions across the tropics and extratropics can increase predictability during later lead times. Future work will quantify causal pathways associated with high predictability.
Clustering analysis of medium-band selected high-redshift galaxies
Next-generation large-scale structure spectroscopic surveys will probe cosmology at high redshifts (2.3 < z < 3.5), relying on abundant galaxy tracers such as Lyα emitters (LAEs) and Lyman break galaxies (LBGs). Medium-band photometry has emerged as a potential technique for efficiently selecting these high-redshift galaxies. In this work, we present clustering analysis of medium-band selected galaxies at high redshift, utilizing photometric data from the Intermediate Band Imaging Survey (IBIS) and spectroscopic data from the Dark Energy Spectroscopic Instrument (DESI). We interpret the clustering of such samples using both Halo Occupation Distribution (HOD) modeling and a perturbation theory description of large-scale structure. Our modeling indicates that the current target sample is composed from an overlapping mixture of LAEs and LBGs with emission lines. Despite differences in target selection, we find that the clustering properties are consistent with previous studies, with correlation lengths r 0 ≃ 3-4 h -1 Mpc and a linear bias of b ∼ 1.8-2.5. Finally, we discuss the simulation requirements implied by these measurements and demonstrate that the properties of the samples would make them excellent targets to enhance our understanding of the high-z universe.
Cluster Analysis of Combined EDS and EBSD Data to Solve Ambiguous Phase Identifications
A common problem in analytical scanning electron microscopy (SEM) using electron backscatter diffraction (EBSD) is the differentiation of phases with distinct chemistry but the same or very similar crystal structure. X-ray energy dispersive spectroscopy (EDS) is useful to help differentiate these phases of similar crystal structures but different elemental makeups. However, open, automated, and unbiased methods of differentiating phases of similar EBSD responses based on their EDS response are lacking. This paper describes a simple data analytics-based method, using a combination of singular value decomposition and cluster analysis, to merge simultaneously acquired EDS + EBSD information and automatically determine phases from both their crystal and elemental data. I use hexagonal TiB 2 ceramic contaminated with multiple crystallographically ambiguous but chemically distinct cubic phases to illustrate the method. Code, in the form of a Python 3 Jupyter Notebook, and the necessary data to replicate the analysis are provided as Supplementary material.
Cluster analysis of day-to-day traffic data in networks
Not Available
Three dimensional cluster analysis for atom probe tomography using Ripley’s K-function and machine learning
The size and structure of spatial molecular and atomic clustering can significantly impact material properties and is therefore important to accurately quantify. Ripley’s K-function (K(r)), a measure of spatial correlation, can be used to perform such quantification when the material system of interest can be represented as a marked point pattern. This work demonstrates how machine learning models based on K (r)-derived metrics can accurately estimate cluster size and intra-cluster density in simulated three dimensional (3D) point patterns containing spherical clusters of varying size; over 90% of model estimates for cluster size and intra-cluster density fall within 11% and 18% error of the true values, respectively. These K (r)-based size and density estimates are then applied to an experimental APT reconstruction to characterize MgZn clusters in a 7000 series aluminum alloy. Here we find that the estimates are more accurate, consistent, and robust to user interaction than estimates from the popular maximum separation algorithm. Using K (r) and machine learning to measure clustering is an accurate and repeatable way to quantify this important material attribute.
Future projections of wind patterns in California with the variable-resolution CESM: a clustering analysis approach
Wind energy production is expected to be affected by shifts in wind patterns that will accompany climate change. However, many questions remain on the magnitude and character of this impact, especially on regional scales. In this study, clustering is used to group and analyze large-scale wind patterns in California using model simulations from the variable-resolution Community Earth System Model (VR-CESM). Specifically, simulations have been produced that cover historical (1980–2000), mid-century (2030–2050), and end-of-century (2080–2100) time periods. Once clustered, observed changes to wind patterns can be analyzed in terms of both the change in frequency of those clusters and changes to winds within-clusters. Statistically significant capacity factors changes have been found at all five wind plant sites. Further, decomposition of the capacity factor changes into frequency changes and within-cluster changes enables a better understanding of their drivers. A further examination of the synoptic-scale fields associated with each cluster then provides a better understanding of how changes to large-scale meteorological fields are important for driving changes in localized wind speeds.
Coarse-grain cluster analysis of tensors with application to climate biome identification
A tensor provides a concise way to codify the interdependence of complex data. Treating a tensor as a d-way array, each entry records the interaction between the different indices. Clustering provides a way to parse the complexity of the data into more readily understandable information. Clustering methods are heavily dependent on the algorithm of choice, as well as the chosen hyperparameters of the algorithm. However, their sensitivity to data scales is largely unknown. In this work, we apply the discrete wavelet transform to analyze the effects of coarse-graining on clustering tensor data. We are particularly interested in understanding how scale affects clustering of the Earth's climate system. The discrete wavelet transform allows classification of the Earth's climate across a multitude of spatial-temporal scales. The discrete wavelet transform is used to produce an ensemble of classification estimates, as opposed to a single classification. Each element of the ensemble is a clustering at a different spatial-temporal scale. Information theoretic approaches are used to identify important scale lengths in clustering the L15 Climate Datset. We also discover a sub-collection of the ensemble that spans the majority of the variance observed, allowing for efficient consensus clustering techniques that can be used to identify climate biomes.
Upscaling Soil Organic Carbon Measurements at the Continental Scale Using Multivariate Clustering Analysis and Machine Learning
Abstract Estimates of soil organic carbon (SOC) stocks are essential for many environmental applications. However, significant inconsistencies exist in SOC stock estimates for the U.S. across current SOC maps. We propose a framework that combines unsupervised multivariate geographic clustering (MGC) and supervised Random Forests regression, improving SOC maps by capturing heterogeneous relationships with SOC drivers. We first used MGC to divide the U.S. into 20 SOC regions based on the similarity of covariates (soil biogeochemical, bioclimatic, biological, and physiographic variables). Subsequently, separate Random Forests models were trained for each SOC region, utilizing environmental covariates and SOC observations. Our estimated SOC stocks for the U.S. (52.6 ± 3.2 Pg for 0–30 cm and 108.3 ± 8.2 Pg for 0–100 cm depth) were within the range estimated by existing products like Harmonized World Soil Database, HWSD (46.7 Pg for 0–30 cm and 90.7 Pg for 0–100 cm depth) and SoilGrids 2.0 (45.7 Pg for 0–30 cm and 133.0 Pg for 0–100 cm depth). However, independent validation with soil profile data from the National Ecological Observatory Network showed that our approach ( R 2 = 0.51) outperformed the estimates obtained from Harmonized World Soil Database ( R 2 = 0.23) and SoilGrids 2.0 ( R 2 = 0.39) for the topsoil (0–30 cm). Uncertainty analysis (e.g., low representativeness and high coefficients of variation) identified regions requiring more measurements, such as Alaska and the deserts of the U.S. Southwest. Our approach effectively captures the heterogeneous relationships between widely available predictors and the current SOC baseline across regions, offering reliable SOC estimates at 1 km resolution for benchmarking Earth system models.
X-ray nano-imaging of defects in thin film catalysts via cluster analysis
Functional properties of transition-metal oxides strongly depend on crystallographic defects; crystallographic lattice deviations can affect ionic diffusion and adsorbate binding energies. Scanning x-ray nanodiffraction enables imaging of local structural distortions across an extended spatial region of thin samples. Yet, localized lattice distortions remain challenging to detect and localize using nanodiffraction, due to their weak diffuse scattering. Here, in this study, we apply an unsupervised machine learning clustering algorithm to isolate the low-intensity diffuse scattering in as-grown and alkaline-treated thin epitaxially strained SrIrO 3 films. We pinpoint the defect locations, find additional strain variation in the morphology of electrochemically cycled SrIrO 3 , and interpret the defect type by analyzing the diffraction profile through clustering. Our findings demonstrate the use of a machine learning clustering algorithm for identifying and characterizing hard-to-find crystallographic defects in thin films of electrocatalysts and highlight the potential to study electrochemical reactions at defect sites in operando experiments.
Exploring HOD-dependent systematics for the DESI 2024 Full-Shape galaxy clustering analysis
We analyze the robustness of the DESI 2024 cosmological inference from the full shape of the galaxy power spectrum to uncertainties in the Halo Occupation Distribution (HOD) model of the galaxy-halo connection and the choice of priors on nuisance parameters. We assess variations in the recovered cosmological parameters across a range of mocks populated with different HOD models and find that shifts are often greater than 20% of the expected statistical uncertainties from the DESI data. We encapsulate the effect of such shifts in terms of a systematic covariance term, C HOD , and an additional diagonal contribution quantifying the impact of our choice of nuisance parameter priors on the ability of the effective field theory (EFT) model to correctly recover the cosmological parameters of the simulations. These two covariance contributions are designed to be added to the usual covariance term, C stat , describing the statistical uncertainty in the power spectrum measurement, in order to fairly represent these sources of systematic uncertainty. This novel approach should be more general and robust to the choice of model or additional external datasets used in cosmological fits than the alternative approach of adding systematic uncertainties to the recovered marginalised parameter posteriors. We compare the approaches within the context of a fixed ΛCDM model and demonstrate that our method gives conservative estimates of the systematic uncertainty that nevertheless have little impact on the final posteriors obtained from DESI data.