Search NASA⌕ Search

SEARCH · Search NASA

Results for “data clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Applications of TOMS Ozone Data and Clustering Techniques in Satellite Soundings

Two primary areas are being pursued: (1) Applications of TOMS total ozone data to estimate the tropopause height and consequently to improve the temperature retrievals from satellite measurements; and (2) Applications of cluster analysis to satellite soundings. The strategy is to either use regression retrievals stratified by tropopause or to use the tropopause information as a constraint on the solution to the physical retrieval method.

Munteanu, M. J.↗

Main-sequence Scatter is Real: The Joint Dependence of Galaxy Clustering on Star Formation and Stellar Mass

We present new measurements of the clustering of stellar-mass-complete samples of ∼40,000 SDSS galaxies at z ∼ 0.03 as a joint function of stellar mass and specific star formation rate (sSFR). Our results confirm what Coil et al. find at z ∼ 0.7: galaxy clustering is a stronger function of sSFR at fixed stellar mass than of stellar mass at fixed sSFR. We also find that galaxies above the star-forming main sequence (SFMS) with higher sSFR are less clustered than galaxies below the SFMS with lower sSFR, at a given stellar mass. A similar trend is present for quiescent galaxies. This confirms that main-sequence scatter, and scatter within the quiescent sequence, is physically connected to the large-scale cosmic density field. We compare the resulting galaxy bias versus sSFR, and relative bias versus sSFR ratio, for different galaxy samples across 0 < z < 1.2 to mock galaxy catalogs based on the empirical galaxy evolution model of Behroozi et al. This model fits PRIMUS and DEEP2 clustering data well at intermediate redshift, but agreement with SDSS is not as strong. We show that increasing the correlation between galaxy SFR and halo accretion rate at z ∼ 0 in the model substantially improves agreement with SDSS data. Mock catalogs suggest that central galaxies contribute substantially to the dependence of clustering on sSFR at a given stellar mass and that the signal is not simply an effect of satellite galaxy fraction differences with sSFR. Our results are highly constraining for galaxy evolution models and show that the stellar-to-halo mass relation depends on sSFR.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Predicting the Seawater Chemistry of an Ocean World Using Machine Learning on Isotopic Measurements of Volatile CO2

Introduction: Given the long time intervals required for data transmission to and from ocean worlds targets, low bandwidth for data transmission, time required for data processing and analysis, and potentially extreme radiation environments (e.g., Europa), it is clear that ocean worlds missions will need more autonomous flight instruments and software in order to achieve established science goals. Protracted time intervals for data analysis (e.g., Europa Lander) strongly motivates the development of rapid, consistent and streamlined methods for interpreting data from flight mass spectrometers to e.g., determine how mass spectra from a plume or surface liquid/ice relates to the surface/subsurface. Since mass spectrometry also has the potential to correctly identify biosignatures[1], it is imperative that such methods for interpreting data are consistent and accurate. We used 848 isotope ratio mass spectra from laboratory analyses of CO2 that interacted with ocean worlds-relevant seawaters as a ‘training’ dataset for ‘unsupervised’ machine learning. In unsupervised learning, characteristics of the data are not labeled or linked, and any similarities found only result from the neural network. CO2 isotopologues analyzed for this dataset mimic the remote measurements of CO2 by a flight mass spectrometer, and are detailed in Theiling [2]. From this dataset, we used measured features of the spectra, such as retention time, intensity, and (isotopologue) mass ratios as inputs for our autoencoder neural network. Our neural network was trained to find similarities in these and other spectral features for seawaters of a particular composition and amount of initial CO2. Successful training then created an output of these similarities for various seawaters, which included MgSO4, Na2SO4, NaCl, MgCl2, KCl, and NaHCO3, and combinations of these salts. We then applied dimensionality reduction techniques such as Principal Component Analysis (PCA), T-Distributed Stochastic Neighbor Embedding (TSNE), and Uniform Manifold Approximation and Projection (UMAP) to demonstrate latent data features as a two-dimensional projection in a unitless, high-dimensional space. In this projection, a data point represents the combined effect of spectral features such as intensity, retention time, and isotope ratio. Our initial UMAP demonstrates data clustering (organization of the data by the neural network) based on the amount of CO2 that had initially interacted with each seawater. Further training using more ‘supervised’ learning techniques demonstrate strong clustering of preliminary data based on initial CO2 concentration, seawater chemical composition, and ionic strength (salinity). Our preliminary work therefore suggests that machine learning has the potential to identify compositional variants of an ocean world seawater based on mass spectra from volatile CO2 measurements. Acknowledgments: This work was funded through a Strategic Task Group at NASA Goddard Space Flight Center. The training dataset was collected through funding from the Oklahoma Space Grant Consortium. References: [1] Pappalardo, R. et al. (2013) Astrobiology, 13, 740–773. [2] Theiling (2020) Icarus, 114216.

Europa↗

Analysis of RXTE data on Clusters of Galaxies

This grant provided support for the reduction, analysis and interpretation of of hard X-ray (HXR, for short) observations of the cluster of galaxies RXJO658--5557 scheduled for the week of August 23, 2002 under the RXTE Cycle 7 program (PI Vahe Petrosian, Obs. ID 70165). The goal of the observation was to search for and characterize the shape of the HXR component beyond the well established thermal soft X-ray (SXR) component. Such hard components have been detected in several nearby clusters. distant cluster would provide information on the characteristics of this radiation at a different epoch in the evolution of the imiverse and shed light on its origin. We (Petrosian, 2001) have argued that thermal bremsstrahlung, as proposed earlier, cannot be the mechanism for the production of the HXRs and that the most likely mechanism is Compton upscattering of the cosmic microwave radiation by relativistic electrons which are known to be present in the clusters and be responsible for the observed radio emission. Based on this picture we estimated that this cluster, in spite of its relatively large distance, will have HXR signal comparable to the other nearby ones. The planned observation of a relatively The proposed RXTE observations were carried out and the data have been analyzed. We detect a hard X-ray tail in the spectrum of this cluster with a flux very nearly equal to our predicted value. This has strengthen the case for the Compton scattering model. We intend the data obtained via this observation to be a part of a larger data set. We have identified other clusters of galaxies (in archival RXTE and other instrument data sets) with sufficiently high quality data where we can search for and measure (or at least put meaningful limits) on the strength of the hard component. With these studies we expect to clarify the mechanism for acceleration of particles in the intercluster medium and provide guidance for future observations of this intriguing phenomenon by instrument on GLAST. The details of the nonthermal particle population has important implications for the theories of cluster formation, mergers and evolution. The results of this work were first presented at the High Energy Division meeting of the American astronomical Society at Mt. Tremblene, Canada (Petrosian et al. 2003). and in an invited review talk at the General Assembly of the International Astronomical Union at Sydney, Australia (Petrosian, 2003). A paper describe the observations, the data analysis and its implication is being prepared for publication in the Astrophysical Journal.

Petrosian, Vahe↗

ASA-FTL: An adaptive separation aware flash translation layer for solid state drives

Here, the flash-memory based Solid State Drive (SSD) presents a promising storage solution for increasingly critical data-intensive applications due to its low latency (high throughput), high bandwidth, and low power consumption. Within an SSD, its Flash Translation Layer (FTL) is responsible for exposing the SSD’s flash memory storage to the computer system as a simple block device. The FTL design is one of the dominant factors determining an SSD’s lifespan and performance. To reduce the garbage collection overhead and deliver better performance, we propose a new, low-cost, adaptive separation-aware flash translation layer (ASA-FTL) that combines sampling, data clustering and selective caching of recency information to accurately identify and separate hot/cold data while incurring minimal overhead. We use sampling for light-weight identification of separation criteria, and our dedicated selective caching mechanism is designed to save the limited RAM resource in contemporary SSDs. Using simulations of ASA-FTL with both real-world and synthetic workloads, we have shown that our proposed approach reduces the garbage collection overhead by up to 28% and the overall response time by 15% compared to one of the most advanced existing FTLs. We find that the data clustering using a small sample size provides significant performance benefit while only incurring a very small computation and memory cost. In addition, our evaluation shows that ASA-FTL is able to adapt to the changes in the access pattern of workloads, which is a major advantage comparing to existing fixed data separation methods.

97 MATHEMATICS AND COMPUTING↗

Multi-scale visual analysis of time-varying electrocorticography data via clustering of brain regions

There exists a need for effective and easy-to-use software tools supporting the analysis of complex Electrocorticography (ECoG) data. Understanding how epileptic seizures develop or identifying diagnostic indicators for neurological diseases require the in-depth analysis of neural activity data from ECoG. Such data is multi-scale and is of high spatio-temporal resolution. Comprehensive analysis of this data should be supported by interactive visual analysis methods that allow a scientist to understand functional patterns at varying levels of granularity and comprehend its time-varying behavior. We introduce a novel multi-scale visual analysis system, ECoG ClusterFlow, for the detailed exploration of ECoG data. Our system detects and visualizes dynamic high-level structures, such as communities, derived from the time-varying connectivity network. The system supports two major views: 1) an overview summarizing the evolution of clusters over time and 2) an electrode view using hierarchical glyph-based design to visualize the propagation of clusters in their spatial, anatomical context. We present case studies that were performed in collaboration with neuroscientists and neurosurgeons using simulated and recorded epileptic seizure data to demonstrate our system's effectiveness. ECoG ClusterFlow supports the comparison of spatio-temporal patterns for specific time intervals and allows a user to utilize various clustering algorithms. Neuroscientists can identify the site of seizure genesis and its spatial progression during various the stages of a seizure. Our system serves as a fast and powerful means for the generation of preliminary hypotheses that can be used as a basis for subsequent application of rigorous statistical methods, with the ultimate goal being the clinical treatment of epileptogenic zones.

60 APPLIED LIFE SCIENCES↗

Substructure in the stellar halo near the Sun: I. Data-driven clustering in integrals-of-motion space

Context. Merger debris is expected to populate the stellar haloes of galaxies. In the case of the Milky Way, this debris should be apparent as clumps in a space defined by the orbital integrals of motion of the stars. Aims. Our aim is to develop a data-driven and statistics-based method for finding these clumps in integrals-of-motion space for nearby halo stars and to evaluate their significance robustly. Methods. We used data from Gaia EDR3, extended with radial velocities from ground-based spectroscopic surveys, to construct a sample of halo stars within 2.5 kpc from the Sun. We applied a hierarchical clustering method that makes exhaustive use of the single linkage algorithm in three-dimensional space defined by the commonly used integrals of motion energy E, together with two components of the angular momentum, L z and L ⊥ . To evaluate the statistical significance of the clusters, we compared the density within an ellipsoidal region centred on the cluster to that of random sets with similar global dynamical properties. By selecting the signal at the location of their maximum statistical significance in the hierarchical tree, we extracted a set of significant unique clusters. By describing these clusters with ellipsoids, we estimated the proximity of a star to the cluster centre using the Mahalanobis distance. Additionally, we applied the HDBSCAN clustering algorithm in velocity space to each cluster to extract subgroups representing debris with different orbital phases. Results. Our procedure identifies 67 highly significant clusters (> 3σ), containing 12% of the sources in our halo set, and 232 subgroups or individual streams in velocity space. In total, 13.8% of the stars in our data set can be confidently associated with a significant cluster based on their Mahalanobis distance. Inspection of the hierarchical tree describing our data set reveals a complex web of relations between the significant clusters, suggesting that they can be tentatively grouped into at least six main large structures, many of which can be associated with previously identified halo substructures, and a number of independent substructures. This preliminary conclusion is further explored in a companion paper, in which we also characterise the substructures in terms of their stellar populations. Conclusions. Our method allows us to systematically detect kinematic substructures in the Galactic stellar halo with a data-driven and interpretable algorithm. The list of the clusters and the associated star catalogue are provided in two tables available at the CDS.

79 ASTRONOMY AND ASTROPHYSICS↗

Determining the Number of Clusters in a Data Set Without Graphical Interpretation

Cluster analysis is a data mining technique that is meant ot simplify the process of classifying data points. The basic clustering process requires an input of data points and the number of clusters wanted. The clustering algorithm will then pick starting C points for the clusters, which can be either random spatial points or random data points. It then assigns each data point to the nearest C point where "nearest usually means Euclidean distance, but some algorithms use another criterion. The next step is determining whether the clustering arrangement this found is within a certain tolerance. If it falls within this tolerance, the process ends. Otherwise the C points are adjusted based on how many data points are in each cluster, and the steps repeat until the algorithm converges,

Aguirre, Nathan S.↗

How good are polarizable and flexible models for water: Insights from a many-body perspective

We present a systematic analysis of state-of-the-art polarizable and flexible water models from a many-body perspective, with a specific focus on their ability to represent the Born–Oppenheimer potential energy surface of water from the gas to the liquid phase. Using coupled cluster data in the completed basis set limit as a reference, we examine the accuracy of the polarizable models in reproducing individual many-body contributions to interaction energies and harmonic frequencies of water clusters and compare their performance with that of MB-pol, an explicit many-body model that has been shown to correctly predict the properties of water across the entire phase diagram. Based on these comparisons, we use MB-pol as a reference to analyze the ability of the polarizable models to reproduce the energy landscape of liquid water under ambient conditions. We find that, while correctly reproducing the energetics of minimum-energy structures, the polarizable models examined in this study suffer from inadequate representations of many-body effects for distorted configurations. To investigate the role played by geometry-dependent representations of 1-body charge distributions in reproducing coupled cluster data for both interaction and many-body energies, we introduce a simplified version of MB-pol that adopts fixed atomic charges and demonstrate that the new model retains the same accuracy as the original MB-pol model. Based on the analyses presented in this study, we believe that future developments of both polarizable and explicit many-body models should continue in parallel and would benefit from synergistic efforts aimed at integrating the best aspects of the two theoretical/computational frameworks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Optical galaxy cluster mock catalogs with realistic projection effects: Validations with the SDSS clusters

Galaxy clusters identified in optical imaging surveys suffer from projection effects: Physically unassociated galaxies along a cluster’s line of sight can be counted as its members and boost the observed richness (the number of cluster members). To model the impact of projection on cluster cosmology analyses, we apply a halo occupation distribution model to 𝑁-body simulations to simulate the red galaxies contributing to cluster members, and we use the number of galaxies in a cylinder along the line of sight (counts in cylinders) to model the impact of projection on cluster richness. We compare three projection models: uniform, quadratic, and Gaussian, and we convert between them by matching their effective cylinder volumes. We validate our mock catalogs using SDSS redMaPPer clusters’ data vectors, including counts vs richness, stacked lensing signal, spectroscopic redshift distribution of member galaxies, and richness remeasured on a redshift grid. We find the former two are insensitive to the projection model, while the latter two favor a quadratic projection model with a width of ≈180 ℎ −1 Mpc (equivalent to the volume of a uniform model with a width of 100 ℎ −1 Mpc and a Gaussian model with a width of 110 ℎ −1 Mpc, or a Gaussian redshift error of 0.04). Furthermore, our framework provides an efficient and flexible way to model optical cluster data vectors, paving the way for a simulation-based joint analysis for clusters, galaxies, and shear.

79 ASTRONOMY AND ASTROPHYSICS↗

Results and forecasts on cosmic inflation from weak lensing

Here, we highlight the role of weak lensing measurements from current and upcoming stage-IV imaging surveys in the search for cosmic inflation, specifically in measuring the scalar spectral index n s . To do so, we combine the Dark Energy Survey 3 years of observation weak lensing and clustering data with Bicep/Keck, Planck, and Sloan Digital Sky Survey data in r⁢Λ⁢ CDM (cold dark matter) where r is the tensor-to-scalar ratio. While there is no significant improvement in constraining power, we obtain a 1⁢σ shift on n s . Additionally, we forecast a weak lensing and clustering data vector from the 10-year Legacy Survey of Space and Time by the Vera C. Rubin Observatory and show its combination with current data would improve their n s constraints by 25% in r⁢Λ⁢ CDM.

79 ASTRONOMY AND ASTROPHYSICS↗

Development of an Advanced Force Field for Water Using Variational Energy Decomposition Analysis

Given the piecewise approach to modeling intermolecular interactions for force fields, they can be difficult to parametrize since they are fit to data like total energies that only indirectly connect to their separable functional forms. Furthermore, by neglecting certain types of molecular interactions such as charge penetration and charge transfer, most classical force fields must rely on, but do not always demonstrate, how cancellation of errors occurs among the remaining molecular interactions accounted for such as exchange repulsion, electrostatics, and polarization. In this work we present the first generation of the (many-body) MB-UCB force field that explicitly accounts for the decomposed molecular interactions commensurate with a variational energy decomposition analysis, including charge transfer, with force field design choices that reduce the computational expense of the MB-UCB potential while remaining accurate. We optimize parameters using only a single water molecule and water cluster data up through pentamers, with no fitting to condensed phase data, and we demonstrate that high accuracy is maintained when the force field is subsequently validated against conformational energies of larger water cluster data sets, radial distribution functions of the liquid phase, and the temperature dependence of thermodynamic and transport water properties. Finally, we conclude that MB-UCB is comparable in performance to MB-Pol but is less expensive and more transferable by eliminating the need to represent short-ranged interactions through large parameter fits to high order polynomials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Deterministic Annealing Approach to Clustering AIRS Data

We will examine the validity of means and standard deviations as a basis for climate data products. We will explore the conditions under which these two simple statistics are inadequate summaries of the underlying empirical probability distributions by contrasting them with a nonparametric, method called Deterministic Annealing technique

clustering↗

Liver metastases of neuroendocrine tumors: is it possible to diagnose different histologic subtypes depending on multiphasic CT features?

Purpose: To assess and compare the multiphasic computed tomography (CT) features of neuroendocrine tumor (NET) liver metastases and to investigate the possibility to predict the histologic subtype of the primary tumor. Materials and methods: Between January 2013 and December 2017 patients with biopsy proven NET with at least one liver metastasis who underwent multiphasic CT were enrolled in this study. All cases were acquired using a standardized multiphasic liver CT protocol, arterial, portal, and hepatic venous phases were obtained. Images were retrospectively analyzed in consensus by two abdominal radiologists blinded to clinical data and histologic subtype. The size, number, and location of lesions were noted. Enhancement patterns of each lesion on arterial, portal, and hepatic venous phases were assessed. For quantitative analysis, CT attenuation of tumors, liver parenchyma, and aorta were measured using a circular region of interest (ROI) on arterial, portal, and hepatic venous phases for reflecting the blood supply of the tumor. Tumor-to-aorta and tumor-to-liver ratio were calculated in all three phases. Differences between subtypes of NET liver metastases were studied using ROC analysis of clustered data. Results: A total of 255 neuroendocrine tumor liver metastases divided into 101 (39.6%) pancreatic, 60 (23.5%) gastroenteric and 94 (36.8%) lung NET liver metastases were analyzed. Contrast enhancement of lesions was homogeneous in 78% of patients (n = 199), which was significantly more frequent in patients with pancreatic group than in those with gastroenteric origin (n = 90, 89.1% vs. n = 28, 46.7%; p < 0.001). Gastroenteric NET metastases frequently showed heterogeneous enhancement, which was significantly higher than in the other two groups (50% vs. 3% and 2%). With respect to the location of the primary tumor, the difference in enhancement patterns of the liver lesions was statistically significant (p < 0.001). Pancreatic NET metastases were mostly hyperdense on arterial images and isodense on portal and hepatic venous phase images (79.2%, n = 80). Gastroenteric NET metastases were mostly hyperdense on arterial phase images and hypodense on portal and hepatic venous phase images (n = 28, 46.7%). The most frequent pattern for lung NET metastases was hypoattenuation on all three phase images (n = 44, 46.8%). ROC analysis of clustered data revealed statistically significant differences between pancreatic NET liver metastases, gastroenteric NET liver metastases, and lung NET liver metastases in terms of tumor-to-aorta (T–A) ratio and tumor-to-liver (T–L) ratio (p < 0.001). Conclusion: We observed statistically significant differences in multiphasic CT features (enhancement pattern, T–A ratio, and T–L ratio) between histologic subtypes of NET liver metastases. As the difference in histological subtypes of NET liver metastases results in a different prognosis and different management strategy, these CT features might help to identify the primary tumor when it is not known to ensure accurate tumor staging and to provide optimal treatment.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

The applicability and effectiveness of cluster analysis

An insight into the characteristics which determine the performance of a clustering algorithm is presented. In order for the techniques which are examined to accurately cluster data, two conditions must be simultaneously satisfied. First the data must have a particular structure, and second the parameters chosen for the clustering algorithm must be correct. By examining the structure of the data from the Cl flight line, it is clear that no single set of parameters can be used to accurately cluster all the different crops. The effectiveness of either a noniterative or iterative clustering algorithm to accurately cluster data representative of the Cl flight line is questionable. Thus extensive a prior knowledge is required in order to use cluster analysis in its present form for applications like assisting in the definition of field boundaries and evaluating the homogeneity of a field. New or modified techniques are necessary for clustering to be a reliable tool.

Ingram, D. S.↗

Self-Taught Anomaly Detection With Hybrid Unsupervised/Supervised Machine Learning in Optical Networks

Here this paper proposes a self-taught anomaly detection framework for optical networks. The proposed framework makes use of a hybrid unsupervised and supervised machine learning scheme. First, it employs an unsupervised data clustering module (DCM) to analyze the patterns of monitoring data. The DCM enables a self-learning capability that eliminates the requirement of prior knowledge of abnormal network behaviors and therefore can potentially detect unforeseen anomalies. Second, we introduce a self-taught mechanism that transfers the patterns learned by the DCM to a supervised data regression and classification module (DRCM). The DRCM, whose complexity is mainly related to the scale of the applied supervised learning model, can potentially facilitate more scalable and time-efficient online anomaly detection by avoiding excessively traversing the original dataset. We designed the DCM and DRCM based on the density-based clustering algorithm and the deep neural network structure, respectively. Evaluations with experimental data from two use cases (i.e., single-point detection and end-to-end detection) demonstrate that up to 99% anomaly detection accuracy can be achieved with a false positive rate below 1%.

42 ENGINEERING↗

Clustering algorithm evaluation and the development of a replacement for procedure 1

An efficient procedure which clusters data using a completely unsupervised clustering algorithm and then uses labeled pixels to label the resulting clusters or perform a stratified estimate using the clusters as strata is developed. Three clustering algorithms, CLASSY, AMOEBA, and ISOCLS, are compared for efficiency. Three stratified estimation schemes and three labeling schemes are also considered and compared.

Lennington, R. K.↗