Search NASA⌕ Search

SEARCH · Search NASA

Results for “Cluster Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Scalable, In-situ Data Clustering Data Analysis for Extreme Scale Scientific Computing (Final Report)

The objective of this project is to address challenges in the design and development of scalable in-situ data clustering and analytics algorithms and software. Our goal is to develop parallel software consisting of a set of spatio-temporal data clustering and anomaly detection functions, both of which are very important for large-scale analysis and have wide applicability for in-situ runs as well as post-processing analysis. Our design principles for in-situ analysis consider the following: (1) identify parts of the computation can be done close to the data within the nodes, while it is still in memory; (2) extract analysis components can (and should) be performed in remote staging and analysis nodes; (3) develop error-bound approximation methods for applications tolerable for small errors; (4) identify the type of derived distributions and statistics, for spatio-temporal data, that can be kept locally in order to both accelerate computations and meet energy constraints in subsequent iterations and phases; (5) use a self-describing data format so that data can be consistent and understood among local storage (memory and SSDs) and at staging and analysis nodes, thereby providing portability and flexibility; (6) develop service-oriented functions that can schedule in-situ and post-hoc analysis tasks based on the dynamic requirements of applications. Our development focus is to produce the parallel data analysis software/library that will be scalable, reusable, extensible, and generic for applications in different disciplines. The software will be able to run in-situ with the simulations as well as post-hoc analysis. This approach will satisfy many synergistic requirements for data intensive applications executed on data coming from instruments and experiments. In particular, the proposed multilevel approach is directly applicable to perform design tradeoffs for running part of the algorithms near the instruments and the rest on remote (analysis) systems.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2022

Lumping and splitting: Toward a classification of mineral natural kinds

How does one best subdivide nature into kinds? All classification systems require rules for lumping similar objects into the same category, while splitting differing objects into separate categories. Mineralogical classification systems are no exception. Our work in placing mineral species within their evolutionary contexts necessitates this lumping and splitting because we classify “mineral natural kinds” based on unique combinations of formational environments and continuous temperature-pressure-composition phase space. Consequently, we lump two minerals into a single natural kind only if they: (1) are part of a continuous solid solution; (2) are isostructural or members of a homologous series; and (3) form by the same process. A systematic survey based on these criteria suggests that 2310 (~41%) of 5659 IMA-approved mineral species can be lumped with one or more other mineral species, corresponding to 667 “root mineral kinds,” of which 353 lump pairs of mineral species, while 129 lump three species. Eight mineral groups, including cancrinite, eudialyte, hornblende, jahnsite, labuntsovite, satorite, tetradymite, and tourmaline, are represented by 20 or more lumped IMA-approved mineral species. A list of 5659 IMA-approved mineral species corresponds to 4016 root mineral kinds according to these lumping criteria. The evolutionary system of mineral classification assigns an IMA-approved mineral species to two or more mineral natural kinds under either of two splitting criteria: (1) if it forms in two or more distinct paragenetic environments, or (2) if cluster analysis of the attributes of numerous specimens reveals more than one discrete combination of chemical and physical attributes. A total of 2310 IMA-approved species are known to form by two or more paragenetic processes and thus correspond to multiple mineral natural kinds; however, adequate data resources are not yet in hand to perform cluster analysis on more than a handful of mineral species. We find that 1623 IMA-approved species (~29%) correspond exactly to mineral natural kinds; i.e., they are known from only one paragenetic environment and are not lumped with another species in our evolutionary classification. Greater complexity is associated with 587 IMA-approved species that are both lumped with one or more other species and occur in two or more paragenetic environments. In these instances, identification of mineral natural kinds may involve both lumping and splitting of the corresponding IMA-approved species on the basis of multiple criteria. Based on the numbers of root mineral kinds, their known varied modes of formation, and predictions of minerals that occur on Earth but are as yet undiscovered and described, we estimate that Earth holds more than 10000 mineral natural kinds.

Philosophy of mineralogy↗
NASA NTRS2005

Identification of a typical flight patterns

Method and system for analyzing aircraft data, including multiple selected flight parameters for a selected phase of a selected flight, and for determining when the selected phase of the selected flight is atypical, when compared with corresponding data for the same phase for other similar flights. A flight signature is computed using continuous- valued and discrete-valued flight parameters for the selected flight parameters and is optionally compared with a statistical distribution of other observed flight signatures, yielding a typicality scores for the same phase for other similar flights. A cluster analysis is optionally applied to the flight signatures to define an optimal collection of clusters. A level of atypicality for a selected flight is estimated, based upon an index associated with the cluster analysis.

Irving C Statler↗
NASA NTRS2005

Identification of atypical flight patterns

Method and system for analyzing aircraft data, including multiple selected flight parameters for a selected phase of a selected flight, and for determining when the selected phase of the selected flight is atypical, when compared with corresponding data for the same phase for other similar flights. A flight signature is computed using continuous-valued and discrete-valued flight parameters for the selected flight parameters and is optionally compared with a statistical distribution of other observed flight signatures, yielding atypicality scores for the same phase for other similar flights. A cluster analysis is optionally applied to the flight signatures to define an optimal collection of clusters. A level of atypicality for a selected flight is estimated, based upon an index associated with the cluster analysis.

Statler, Irving C.↗
DOE OSTI2020

Cluster Expansion Analysis of Atomic Order in Li-Ion Battery Cathode Material LiCo y Ni 1-y O 2

A modified cluster-expansion treatment was developed recently to analyze atomic order in LiCo y Ni 1-y O 2 , a model cathode material for Li-ion batteries. In this treatment, referred to as a “spin-atom” cluster expansion, the occupant of a lattice site is identified by its spin state as well as its atomic species. Further, Effective Cluster Interaction (ECI) coefficients are derived from a large training data set (i.e., the set of atomic arrangements for which DFT calculations are performed) which is filtered by an anomaly detection algorithm to eliminate poorly converged DFT calculations. The cluster expansion incorporates Li-Ni (LN) exchange as well as intralayer Co-Ni (CN) exchange. Monte Carlo simulations based on the cluster expansion were applied to the Ni-rich part of the phase diagram. The simulations predict a miscibility gap between y = 0.05 and y = 0.65.

25 ENERGY STORAGE↗
NASA NTRS1974

An analysis of pilot error-related aircraft accidents

A multidisciplinary team approach to pilot error-related U.S. air carrier jet aircraft accident investigation records successfully reclaimed hidden human error information not shown in statistical studies. New analytic techniques were developed and applied to the data to discover and identify multiple elements of commonality and shared characteristics within this group of accidents. Three techniques of analysis were used: Critical element analysis, which demonstrated the importance of a subjective qualitative approach to raw accident data and surfaced information heretofore unavailable. Cluster analysis, which was an exploratory research tool that will lead to increased understanding and improved organization of facts, the discovery of new meaning in large data sets, and the generation of explanatory hypotheses. Pattern recognition, by which accidents can be categorized by pattern conformity after critical element identification by cluster analysis.

Kowalsky, N. B.↗
DOE OSTI2022

Poisson hurdle model-based method for clustering microbiome features

Abstract Motivation High-throughput sequencing technologies have greatly facilitated microbiome research and have generated a large volume of microbiome data with the potential to answer key questions regarding microbiome assembly, structure and function. Cluster analysis aims to group features that behave similarly across treatments, and such grouping helps to highlight the functional relationships among features and may provide biological insights into microbiome networks. However, clustering microbiome data are challenging due to the sparsity and high dimensionality. Results We propose a model-based clustering method based on Poisson hurdle models for sparse microbiome count data. We describe an expectation–maximization algorithm and a modified version using simulated annealing to conduct the cluster analysis. Moreover, we provide algorithms for initialization and choosing the number of clusters. Simulation results demonstrate that our proposed methods provide better clustering results than alternative methods under a variety of settings. We also apply the proposed method to a sorghum rhizosphere microbiome dataset that results in interesting biological findings. Availability and implementation R package is freely available for download at https://cran.r-project.org/package=PHclust. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

BiG-SLiCE 2 v1.0.0

BiG-SLiCE was originally an open source Python-based command line bioinformatics software that offers a highly scalable clustering analysis on biosynthetic gene clusters (BGC) data. It allows a simultaneous analysis of millions of BGCs, exceeding the capability of other existing tools (around one hundred thousands). As a tradeoff, the clustering accuracy is relatively lower and sometimes fall short in corner cases and specific BGC classes such as the RiPPs (Ribosomally-translated, Post-translationally modified Peptides). In BiG-SLiCE V2 (developed in LBNL), the clustering algorithm has been significantly improved to deliver a much accurate result even for RiPPs and other previous corner case classes. Moreover, the speed of the overall pipeline has been improved by 50-100%. Finally, additional features were implemented to support downstream analyses of BiG-SLiCE results, such as customized tabular (TSV/CSV) and columnar (Parquet) outputs.

Kautsar, Satria↗
NASA NTRS2011

Determining the Optimal Number of Clusters with the Clustergram

Cluster analysis aids research in many different fields, from business to biology to aerospace. It consists of using statistical techniques to group objects in large sets of data into meaningful classes. However, this process of ordering data points presents much uncertainty because it involves several steps, many of which are subject to researcher judgment as well as inconsistencies depending on the specific data type and research goals. These steps include the method used to cluster the data, the variables on which the cluster analysis will be operating, the number of resulting clusters, and parts of the interpretation process. In most cases, the number of clusters must be guessed or estimated before employing the clustering method. Many remedies have been proposed, but none is unassailable and certainly not for all data types. Thus, the aim of current research for better techniques of determining the number of clusters is generally confined to demonstrating that the new technique excels other methods in performance for several disparate data types. Our research makes use of a new cluster-number-determination technique based on the clustergram: a graph that shows how the number of objects in the cluster and the cluster mean (the ordinate) change with the number of clusters (the abscissa). We use the features of the clustergram to make the best determination of the cluster-number.

Fluegemann, Joseph K.↗
NASA NTRS2021

Satellite Remote Sensing to Assess Cyanobacterial Bloom Frequency Across the United States at Multiple Spatial Scales

Cyanobacterial blooms can have negative effects on human health and local ecosystems. Field monitoring of cyanobacterial blooms can be costly, but satellite remote sensing has shown utility for more efficient spatial and temporal monitoring across the United States. Here, satellite imagery was used to assess the annual frequency of surface cyanobacterial blooms, defined for each satellite pixel as the percentage of images for that pixel throughout the year exhibiting detectable cyanobacteria. Cyanobacterial frequency was assessed across 2,196 large lakes in 46 states across the continental United States (CONUS) using imagery from the European Space Agency’s Ocean and Land Colour Imager for the years 2017 through 2019. In 2019, across all satellite pixels considered, annual bloom frequency had a median value of 4% and a maximum value of 100%, the latter indicating that for those satellite pixels, a cyanobacterial bloom was detected by the satellite sensor for every satellite image considered. In addition to annual pixel-scale cyanobacterial frequency, results were summarized at the lake- and state-scales by averaging annual pixel-scale results across each lake and state. For 2019, average annual lake-scale frequencies also had a maximum value of 100%, and Oregon and Ohio had the highest average annual state-scale frequencies at 65% and 52%. Pixel-scale frequency results can assist in identifying portions of a lake that are more prone to cyanobacterial blooms, while lake- and state-scale frequency results can assist in the prioritization of sampling resources and mitigation efforts. Satellite imagery is limited by the presence of snow and ice, as imagery collected in these conditions are quality flagged and discarded. Thus, annual bloom frequencies within nine climate regions were investigated to determine whether missing data biased results in climate regions more prone to snow and ice, given that their annual summaries would be weighted toward the summer months when cyanobacterial blooms tend to occur. Results were unbiased by the time period selected in most climate regions, but a large bias was observed for the Northwest Rockies and Plains climate region. Moderate biases were observed for the Ohio Valley and the Southeast climate regions. Finally, a clustering analysis was used to identify areas of high and low cyanobacterial frequency across CONUS based on average annual lake-scale cyanobacterial frequencies for 2019. Several clusters were identified that transcended state, watershed, and eco-regional boundaries. Combined with additional data, results from the clustering analysis may offer insight regarding large-scale drivers of cyanobacterial blooms.

remote sensing↗
NASA NTRS1985

Technical support for creating an artificial intelligence system for feature extraction and experimental design

Techniques for classifying objects into groups or clases go under many different names including, most commonly, cluster analysis. Mathematically, the general problem is to find a best mapping of objects into an index set consisting of class identifiers. When an a priori grouping of objects exists, the process of deriving the classification rules from samples of classified objects is known as discrimination. When such rules are applied to objects of unknown class, the process is denoted classification. The specific problem addressed involves the group classification of a set of objects that are each associated with a series of measurements (ratio, interval, ordinal, or nominal levels of measurement). Each measurement produces one variable in a multidimensional variable space. Cluster analysis techniques are reviewed and methods for incuding geographic location, distance measures, and spatial pattern (distribution) as parameters in clustering are examined. For the case of patterning, measures of spatial autocorrelation are discussed in terms of the kind of data (nominal, ordinal, or interval scaled) to which they may be applied.

Glick, B. J.↗
DOE OSTI2025

Spatio-temporal multivariate cluster evolution analysis for detecting and tracking climate impacts

Recent years have seen a growing concern about climate change and its impacts. While Earth System Models (ESMs) can be invaluable tools for studying the impacts of climate change, the complex coupling processes encoded in ESMs and the large amounts of data produced by these models, together with the high internal variability of the Earth system, can obscure important source-to-impact relationships. Here, this paper presents a novel and efficient unsupervised data-driven approach for detecting statistically-significant impacts and tracing spatio-temporal source-impact pathways in the climate through a unique combination of ideas from anomaly detection, clustering and Natural Language Processing (NLP). Using as an exemplar the 1991 eruption of Mount Pinatubo in the Philippines, we demonstrate that the proposed approach is capable of detecting known post-eruption impacts/events. We additionally describe a methodology for extracting meaningful sequences of post-eruption impacts/events by using NLP to efficiently mine frequent multivariate cluster evolutions, which can be used to confirm or discover the chain of physical processes between a climate source and its impact(s).

Anomaly detection↗
NASA NTRS1984

Preliminary Comparisons of the Information Content and Utility of TM Versus MSS Data

Comparisons were made between subscenes from the first TM scene acquired of the Washington, D.C. area and a MSS scene acquired approximately one year earlier. Three types of analyses were conducted to compare TM and MSS data: a water body analysis, a principal components analysis and a spectral clustering analysis. The water body analysis compared the capability of the TM to the MSS for detecting small uniform targets. Of the 59 ponds located on aerial photographs 34 (58%) were detected by the TM with six commission errors (15%) and 13 (22%) were detected by the MSS with three commission errors (19%). The smallest water body detected by the TM was 16 meters; the smallest detected by the MSS was 40 meters. For the principal components analysis, means and covariance matrices were calculated for each subscene, and principal components images generated and characterized. In the spectral clustering comparison each scene was independently clustered and the clusters were assigned to informational classes. The preliminary comparison indicated that TM data provides enhancements over MSS in terms of (1) small target detection and (2) data dimensionality (even with 4-band data). The extra dimension, partially resultant from TM band 1, appears useful for built-up/non-built-up area separation.

Markham, B. L.↗
NASA NTRS2004

Functional Groups Based on Leaf Physiology: Are they Spatially and Temporally Robust?

The functional grouping hypothesis, which suggests that complexity in ecosystem function can be simplified by grouping species with similar responses, was tested in the Florida scrub habitat. Functional groups were identified based on how species in fire maintained Florida scrub regulate exchange of carbon and water with the atmosphere as indicated by both instantaneous gas exchange measurements and integrated measures of function (%N, delta C-13, delta N-15, C-N ratio). Using cluster analysis, five distinct physiologically-based functional groups were identified in the fire maintained scrub. These functional groups were tested to determine if they were robust spatially, temporally, and with management regime. Analysis of Similarities (ANOSIM), a non-parametric multivariate analysis, indicated that these five physiologically-based groupings were not altered by plot differences (R = -0.115, p = 0.893) or by the three different management regimes; prescribed burn, mechanically treated and burn, and fire-suppressed (R = 0.018, p = 0.349). The physiological groupings also remained robust between the two climatically different years 1999 and 2000 (R = -0.027, p = 0.725). Easy-to-measure morphological characteristics indicating functional groups would be more practical for scaling and modeling ecosystem processes than detailed gas-exchange measurements, therefore we tested a variety of morphological characteristics as functional indicators. A combination of non-parametric multivariate techniques (Hierarchical cluster analysis, non-metric Multi-Dimensional Scaling, and ANOSIM) were used to compare the ability of life form, leaf thickness, and specific leaf area classifications to identify the physiologically-based functional groups. Life form classifications (ANOSIM; R = 0.629, p 0.001) were able to depict the physiological groupings more adequately than either specific leaf area (ANOSIM; R = 0.426, p = 0.001) or leaf thickness (ANOSIM; R 0.344, p 0.001). The ability of life forms to depict the physiological groupings was improved by separating the parasitic Ximenia americana from the shrub category (ANOSIM; R = 0.794, p = 0.001). Therefore, a life form classification including parasites was determined to be a good indicator of the physiological processes of scrub species, and would be a useful method of grouping for scaling physiological processes to the ecosystem level.

Foster, Tammy E.↗
DOE OSTI2021

Using action space clustering to constrain the recent accretion history of Milky Way-like galaxies

ABSTRACT In the currently favoured cosmological paradigm galaxies form hierarchically through the accretion of satellites. Since a satellite is less massive than the host, its stars occupy a smaller volume in action space. Actions are conserved when the potential of the host halo changes adiabatically, so stars from an accreted satellite would remain clustered in action space as the host evolves. In this paper, we identify recently disrupted accreted satellites in three Milky Way-like disc galaxies from the cosmological baryonic FIRE-2 simulations by tracking satellites through simulation snapshots. We try to recover these satellites by applying the cluster analysis algorithm Enlink to the orbital actions of accreted star particles in the z = 0 snapshot. Even with completely error-free mock data we find that only 35 per cent (14/39) satellites are well recovered while the rest (25/39) are poorly recovered (i.e. either contaminated or split up). Most (10/14 ∼70 per cent) of the well-recovered satellites have infall times <7.1 Gyr ago and total mass >4 × 108M⊙ (stellar mass more than 1.2 × 106 M⊙, although our upper mass limit is likely to be resolution dependent). Since cosmological simulations predict that stellar haloes include a population of in situ stars, we test our ability to recover satellites when the data include 10–50 per cent in situ contamination. We find that most previously well-recovered satellites stay well recovered even with 50 per cent contamination. With the wealth of 6D phase space data becoming available we expect that cluster analysis in action space will be useful in identifying the majority of recently accreted and moderately massive satellites in the Milky Way.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Extension of Large Fire Emissions From Summer to Autumn and Its Drivers in the Western US

Abstract Burned areas in the western US have increased ten‐fold since 1980s, which are attributable to multiple factors, including increasing heat, changing precipitation patterns, and extended drought. To better understand how these factors contribute to large fire emissions (gridded monthly fire emissions >95th percentile of all the fire emissions in the western US; 0.009 Gg/month), we build a machine learning model to predict fire emissions (PM 2.5 ) over the western US at 0.25° resolution, interpreted using explainable artificial intelligence (XAI). From the predictor contributions derived from XAI, we conduct k‐means clustering analysis to identify four clusters of predictor variables representing different drivers of large fire emissions. The four clusters feature the contributions of fuel load (Cluster 1) and different levels of dryness (Cluster 2–4), controlled by fuel moisture, drought condition, and fire‐favorable large‐scale meteorological patterns featuring high temperature, high pressure, and low relative humidity. In the past two decades, large fire emissions peak in summer. However, large fire emissions increased significantly in September and October in 2010–2020 relative to 2000–2009, extending the peak large fire emissions from summer to autumn. The larger enhancements of large fire emissions during autumn compared to summer are contributed by decreased fuel moisture, along with more frequent concurrent fire‐favorable large‐scale meteorological patterns and drought. These results highlight fuel drying as a common driver supported by multiple drivers, such as warmer temperature and more frequent synoptic patterns favorable for fires, in increasing the autumn risk of large fire emissions across the western US.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Detecting Living-off-the-land Attacks Using K-means And Graph Convolutional Networks

The code ingests Zeek logs derived from network packet captures and goes through data preprocessing before it gets passed into a K-Means model that labels each device as either a client or server. Graph Convolutional Network (GCN) model is used to obtain the embeddings to represent the features in lower dimension. Last, K-means cluster analysis is used to cluster the embeddings for each class.

Quach, Anna [Idaho National Laboratory (INL), Idah↗
DOE OSTI2023

Synoptic Typing of Multiduration, Heavy Precipitation Records in the Northeastern United States: 1895–2017

Much of the previous research on total and heavy precipitation trends across the northeastern United States (herein, the Northeast) used daily precipitation totals over relatively short periods of record, which do not capture the full range of climate variability and change. Less well understood are the characteristics of long-term changes and synoptic patterns in longer-duration heavy precipitation events across the Northeast. A multiduration (1, 2, 3, 7, 14, and 30 days), multi-return-interval (2, 5, 10, and 50 years) precipitation dataset was used to diagnose changes in various types of precipitation events across the Northeast from 1895 to 2017. Increasing trends were found in all duration and return-interval event combinations with the rarest, longest duration events increasing at faster rates than more-frequent, shorter-duration ones. Daily 850-hPa geopotential height patterns associated with precipitation events were extracted from rotated principal component analysis and k -means clustering analysis, which allowed for the main synoptic types present as well as their structure and evolution to be analyzed. The daily synoptic patterns thus identified were found to be similar across all durations and return intervals and included coastal low (Northeasters, tropical cyclones, and predecessor rain events), deep trough, East Coast trough, zonal, and high pressure patterns.

54 ENVIRONMENTAL SCIENCES↗