Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Discovery of Extended Tidal Tails around the Globular Cluster Palomar 13

We use photometry from the DECam Legacy Survey to detect candidate tidal tails extending ~5° on either side of the Palomar 13 globular cluster. The tails are aligned with the proper motion of Palomar 13 and are consistent with its old, metal-poor stellar population. We identify three RR Lyrae stars (RRLs) that are plausibly associated with the tails, in addition to four previously known in the cluster. From these RRLs, we find that the mean distance to the cluster and tails is 23.6 ± 0.2 kpc and estimate the total (initial) luminosity of the cluster to be , consistent with previous claims that its initial luminosity was higher than its current luminosity. Combined with previously determined proper motion and radial velocity measurements of the cluster, we find that Palomar 13 is on a highly eccentric orbit (e ~ 0.8) with a pericenter of ~9 kpc and an apocenter of ~69 kpc, and a recent pericentric passage of the cluster ~75 Myr ago. We note a prominent linear structure in the interstellar dust map that runs parallel to the candidate tidal features, but conclude that reddening due to dust is unlikely to account for the structure that we observe. If confirmed, the Palomar 13 stellar stream would be one of very few streams with a known progenitor system, making it uniquely powerful for studying the disruption of globular clusters, the formation of the stellar halo, and the distribution of matter within our Galaxy.

79 ASTRONOMY AND ASTROPHYSICS↗

HiSpatialCluster: A novel high‐performance software tool for clustering massive spatial points

Abstract In the era of big data, spatial clustering is a very important means for geo‐data analysis. When clustering big geo‐data such as social media check‐in data, geotagged photos, and taxi trajectory points, traditional spatial clustering algorithms are facing more challenges. On the one hand, existing spatial clustering tools cannot support the clustering of massive point sets; on the other hand, there is no perfect solution for self‐adaptive spatial clustering. In order to achieve clustering of millions or even billions of points adaptively, a new spatial clustering tool—HiSpatialCluster—was proposed, in which the CFSFDP (clustering by fast search and finding density peaks) idea to find cluster centers and the DBSCAN (density‐based spatial clustering of applications with noise) idea of density‐connect filtering for classification are introduced. The tool’s source codes and other resources have been released on Github, and experimental evaluation was performed through clustering massive taxi trajectory points and Flickr geotagged photos in Beijing, China. The spatial clustering results were compared with those through K‐means and DBSCAN as well. As a spatial clustering tool, HiSpatialCluster is expected to play a fundamental role in big geo‐data research. First, this tool enables clustering adaptively on massive point datasets with uneven spatial density distribution. Second, the density‐connect filter method is applied to generate homogeneous analysis units from geotagged data. Third, the tool is accelerated by both parallel CPU and GPU computing so that millions or even billions of points can be clustered efficiently.

Chen, Yiran↗

Chandra Follow-up of the SDSS DR8 Redmapper Catalog Using the MATCha Pipeline

In order to place constraints on cosmology through optical surveys of galaxy clusters, one must first understand the properties of those clusters. To this end, we introduce the Mass Analysis Tool for Chandra (MATCha), a pipeline that uses a parallellized algorithm to analyze archival Chandra data. MATCha simultaneously calculates X-ray temperatures and luminosities and performs centering measurements for hundreds of potential galaxy clusters using archival X-ray exposures. We run MATCha on the redMaPPer SDSS DR8 cluster catalog and use MATCha's output X-ray temperatures and luminosities to analyze the galaxy cluster temperature–richness, luminosity–richness, luminosity–temperature, and temperature–luminosity scaling relations. We detect 447 clusters and determine 246 r 2500 temperatures across all redshifts. Within 0.1 < z < 0.35, we find that r 2500 T X scales with optical richness (λ) as $\mathrm{ln}\left(\tfrac{{k}_{B}{T}_{{\rm{X}}}}{1.0\,\mathrm{keV}}\right)=(0.52\pm 0.05)\mathrm{ln}\left(\tfrac{\lambda }{70}\right)+(1.85\pm 0.03)$ with an intrinsic scatter of $0.27\pm 0.02$ ($1\sigma $). We investigate the distribution of offsets between the X-ray center and redMaPPer center within 0.1 < z < 0.35, finding that 68%.3 ± 6.5% of clusters are well-centered. However, we find a broad tail of large offsets in this distribution, and we explore some of the causes of redMaPPer miscentering.

79 ASTRONOMY AND ASTROPHYSICS↗

Scaling Irregular Applications through Data Aggregation and Software Multithreading

Bioinformatics, data analytics, semantic databases, knowledge discovery are emerging high performance application areas that exploit dynamic, linked data structures such as graphs, unbalanced trees or unstructured grids. These data structures usually are very large, requiring significantly more memory than available on single shared memory systems. Additionally, these data structures are difficult to partition on distributed memory systems. They also present poor spatial and temporal locality, thus generating unpredictable memory and network accesses. The Partitioned Global Address Space (PGAS) programming model seems suitable for these applications, because it allows using a shared memory abstraction across distributed-memory clusters. However, current PGAS languages and libraries are built to target regular remote data accesses and block transfers. Furthermore, they usually rely on the Single Program Multiple Data (SPMD) parallel control model, which is not well suited to the fine grained, dynamic and unbalanced parallelism of irregular applications. In this paper we present {\bf GMT} (Global Memory and Threading library), a custom runtime library that enables efficient execution of irregular applications on commodity clusters. GMT integrates a PGAS data substrate with simple fork/join parallelism and provides automatic load balancing on a per node basis. It implements multi-level aggregation and lightweight multithreading to maximize memory and network bandwidth with fine-grained data accesses and tolerate long data access latencies. A key innovation in the GMT runtime is its thread specialization (workers, helpers and communication threads) that realize the overall functionality. We compare our approach with other PGAS models, such as UPC running using GASNet, and hand-optimized MPI code on a set of typical large-scale irregular applications, demonstrating speedups of an order of magnitude.

Aggregation↗

Measuring the homogeneity of the universe using polarization drift

We propose a method to probe the homogeneity of a general universe, without assuming symmetry. We show that isotropy can be tested at remote locations on the past lightcone by comparing the line-of-sight and transverse expansion rates, using the time dependence of the polarization of Cosmic Microwave Background photons that have been inverse-Compton scattered by the hot gas in massive clusters of galaxies. This probes a combination of remote transverse and parallel components of the expansion rate of the metric, and we may use radial baryon acoustic oscillations or cosmic clocks to measure the parallel expansion rate. Thus we can test remote isotropy, which is a key requirement of a homogeneous universe. We provide explicit formulas that connect observables and properties of the metric.

79 ASTRONOMY AND ASTROPHYSICS↗

Scalable Distributed Memory Community Detection Using Vite

Graph clustering, popularly known as community detection, is a fundamental graph operation used in several applications of relevance for network analysis and cybersecurity. Community detection is an operation of partitioning a network into an arbitrary number of “communities” such that each com munity represents a tightly-knit group of nodes with relatively sparser connections to the rest of the nodes in the network. The need to compute clustering on large-scale networks necessitates the development of efficient parallel algorithms capable of exploiting modern parallel architectures. However, due to their irregular and inherently sequential nature, many of the current algorithms for community detection are challenging to parallelize. In response to the 2018 Graph Challenge, we present Vite which is a distributed-memory parallel community detection method. Vite is a parallelization of the algorithmic template provided by a widely-used serial method (Louvain). Alongside, it uses several heuristics aimed at significantly improving parallelism and performance, while preserving output quality. Using the inputs from the 2018 Graph Challenge (static and streaming), we demonstrate superior performance and high quality solutions based on four parallelization heuristics.

community detection↗

DAAP (Data Analytics Application Profiling)

The Data Analytics Application Profiling (DAAP) utility is a software program comprised of a library and scripts to provide basic instrumenting of parallel scientific software applications while they are running on an HPC cluster, and transmit this data over a secure connection as individual records to aggregators. When used in conjunction with additional open source software components (Telegraf, FluentD, and RabbitMQ), the data collected by DAAP can be transmitted off-cluster to an analytics system of the user’s choice for further analysis.

Shereda, Charles↗

Exploring Flexible Communications for Streamlining DNN Ensemble Training Pipelines

Parallel training of a Deep Neural Network (DNN) ensemble on a cluster of nodes is a common practice to train multiple models in order to construct a model with a higher prediction accuracy. Existing ensemble training pipelines can perform a great deal of redundant operations, resulting in unnecessary CPU usage, or even poor pipeline performance. In order to remove these redundancies, we need pipelines with more communication flexibility than existing DNN frameworks can provide. This project investigates a series of designs to improve pipeline flexibility and adaptivity, while also increasing performance. We implement our designs using Tensorflow with Horovod, and test it using several large DNNs in a large scale GPU cluster, Titan supercomputer at Oak Ridge National Lab. Our results show that the CPU time spent during training is reduced by 2-11X. Furthermore, our implementation can achieve up to 10X speedups when CPU core limits are imposed. Our best pipeline also reduces the average power draw of the ensemble training process by 5-16% when compared to the baseline.

97 MATHEMATICS AND COMPUTING↗

Exploring flexible communications for streamlining DNN ensemble training pipelines

Parallel training of a Deep Neural Network (DNN) ensemble on a cluster of nodes is a common practice to train multiple models in order to construct a model with a higher prediction accuracy, or to quickly tune the parameters of a training model. Existing ensemble training pipelines perform a great deal of redundant operations, resulting in unnecessary CPU usage, or even poor pipeline performance. In order to remove these redundancies, we need pipelines with more communication flexibility than existing DNN frameworks can provide. This project investigates a series of designs to improve pipeline flexibility and adaptivity, while also increasing performance. We implement our designs using Tensorflow with Horovod, and test it using several large DNNs in a large scale GPU cluster, the Titan supercomputer at Oak Ridge National Lab. Our results show that with the new flexible communication schemes, the CPU time spent during training is reduced by 2-11X. Furthermore, our implementation can achieve up to 10X speedups when CPU core limits are imposed. Our best pipeline also reduces the average power draw of the ensemble training process by 5--16% when compared to the baseline.

Pittman, Randall↗

FLEET: Flexible Efficient Ensemble Training for Heterogeneous Deep Neural Networks

Parallel training of an ensemble of Deep Neural Networks (DNN) on a cluster of nodes is an effective approach to shorten the process of neural network architecture search and hyper-parameter tuning for a given learning task. Prior efforts have shown that data sharing, where the common preprocessing operation is shared across the DNN training pipelines, saves computational resources and improves pipeline efficiency. Data sharing strategy, however, performs poorly for a heterogeneous set of DNNs where each DNN has varying computational needs and thus different training rate and convergence speed. This paper proposes FLEET, a flexible ensemble DNN training framework for efficiently training a heterogeneous set of DNNs. We build FLEET via several technical innovations. We theoretically prove that an optimal resource allocation is NP-hard and propose a greedy algorithm to efficiently allocate resources for training each DNN with data sharing. We integrate data-parallel DNN training into ensemble training to mitigate the differences in training rates and introduce checkpointing into this context to address the issue of different convergence speeds. Experiments show that FLEET significantly improves the training efficiency of DNN ensembles without compromising the quality of the result.

Guan, Hui↗

Massively parallel quantum chemical density matrix renormalization group method

Here, we present, to the best of our knowledge, the first attempt to exploit the supercomputer platform for quantum chemical density matrix renormalization group (QCDMRG) calculations. We have developed the parallel scheme based on the in-house MPI global memory library, which combines operator and symmetry sector parallelisms, and tested its performance on three different molecules, all typical candidates for QC-DMRG calculations. In case of the largest calculation, which is the nitrogenase FeMo cofactor cluster with the active space comprising 113 electrons in 76 orbitals and bond dimension equal to 6000, our parallel approach scales up to approximately 2000 CPU cores.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

An Approach to Selecting Thread + Process Mixes for Hybrid MPI + OpenMP Applications, In: 2015 IEEE International Conference on Cluster Computing

Hybrid MPI + OpenMP is a popular means of programming modern machines that feature substantial parallelism both off-node and on-node. Determining the right mix of the two programming models to use, however, is not as straightforward as simply using exclusively OpenMP on-node and limiting MPI to only inter-node communication. We present a step-by-step methodology to help make the decision of which mix of the two programming models to use. It starts with an estimate of the performance of a generic hybrid application on a given machine and incorporates additional available information about the specific application and the machine to provide guidance for selecting effective mixes of MPI processes and OpenMP threads to use when running that application on the machine in question. We validate our approach on four different applications on an IBM Blue Gene/Q, a Cray XK7, and a Cray XC30.

Computer Science↗

Scaling Semantic Graph Databases in Size and Performance

In this paper we present SGEM, a full software system for accelerating large-scale semantic graph databases on commodity clusters. Unlike current approaches, SGEM addresses semantic graph databases by only employing graph methods at all the levels of the stack. On one hand, this allows exploiting the space efficiency of graph data structures and the inherent parallelism of graph algorithms. These features adapt well to the increasing system memory and core counts of modern commodity clusters. On the other hand, however, these systems are optimized for regular computation and batched data transfers, while graph methods usually are irregular and generate fine-grained data accesses with poor spatial and temporal locality. Our framework comprises a SPARQL to data parallel C compiler, a library of parallel graph methods and a custom, multithreaded runtime system. We introduce our stack, motivate its advantages with respect to other solutions and show how we solved the challenges posed by irregular behaviors. We present the result of our software stack on the Berlin SPARQL benchmarks with datasets up to 10 billion triples (a triple corresponds to a graph edge), demonstrating scaling in dataset size and in performance as more nodes are added to the cluster.

Aggregation↗

Layout-aware I/O Scheduling for terabits data movement

Many science facilities, such as the Department of Energy's Leadership Computing Facilities and experimental facilities including the Spallation Neutron Source, Stanford Linear Accelerator Center, and Advanced Photon Source, produce massive amounts of experimental and simulation data. These data are often shared among the facilities and with collaborating institutions. Moving large datasets over the wide-area network (WAN) is a major problem inhibiting collaboration. Next-generation, terabit-networks will help alleviate the problem, however, the parallel storage systems on the endsystem hosts at these institutions can become a bottleneck for terabit data movement. The parallel storage system (PFS) is shared by simulation systems, experimental systems, analysis and visualization clusters, in addition to wide-area data movers. These competing uses often induce temporary, but significant, I/O load imbalances on the storage system, which impact the performance of all the users. The problem is a serious concern because some resources are more expensive (e.g. super computers) or have time-critical deadlines (e.g. experimental data from a light source), but parallel file systems handle all requests fairly even if some storage servers are under heavy load. This paper investigates the problem of competing workloads accessing the parallel file system and how the performance of wide-area data movement can be improved in these environments. First, we study the I/O load imbalance problems using actual I/O performance data collected from the Spider storage system at the Oak Ridge Leadership Computing Facility. Second, we present I/O optimization solutions with layout-awareness on end-system hosts for bulk data movement. With our evaluation, we show that our I/O optimization techniques can avoid the I/O congested disk groups, improving storage I/O times on parallel storage systems for terabit data movement.

Kim, Youngjae↗

Layout-Aware I/O Scheduling for Terabits Data Movement

Many science facilities, such as the Department of Energy s Leadership Computing Facilities and experimental facilities including the Spallation Neutron Source, Stanford Linear Accelerator Center, and Advanced Photon Source, produce massive amounts of experimental and simulation data. These data are often shared among the facilities and with collaborating institutions. Moving large datasets over the wide- area network (WAN) is a major problem inhibiting collaboration. Next- generation, terabit-networks will help alleviate the problem, however, the parallel storage systems on the end-system hosts at these institutions can become a bottleneck for terabit data movement. The parallel storage system (PFS) is shared by simulation systems, experimental systems, analysis and visualization clusters, in addition to wide-area data movers. These competing uses often induce temporary, but significant, I/O load imbalances on the storage system, which impact the performance of all the users. The problem is a serious concern because some resources are more expensive (e.g. super computers) or have time-critical deadlines (e.g. experimental data from a light source), but parallel file systems handle all requests fairly even if some storage servers are under heavy load. This paper investigates the problem of competing workloads accessing the parallel file system and how the performance of wide-area data movement can be improved in these environments. First, we study the I/O load imbalance problems using actual I/O performance data collected from the Spider storage system at the Oak Ridge Leadership Computing Facility. Second, we present I/O optimization solutions with layout-awareness on end-system hosts for bulk data movement. With our evaluation, we show that our I/O optimization techniques can avoid the I/O congested disk groups, improving storage I/O times on parallel storage systems for terabit data movement.

I/O and Storage↗

Exceptionally Complex Electronic Structures of Lanthanide Oxides and Small Molecules

Lanthanide (Ln) oxide clusters and molecular systems provide a bottom-up look at the electronic structures of the bulk materials because of close parallels in the patterns of Ln 4fN subshell occupancy between the molecular and bulk Ln 2 O 3 size limits. At the same time, these clusters and molecules offer a challenge to the theory community to find appropriate and robust treatments for the 4f N patterns across the Ln series. Anion photoelectron (PE) spectroscopy provides a powerful experimental tool for studying these systems, mapping the energies of the ground and low-lying excited states of the neutral relative to the initial anion state, providing spectroscopic patterns that reflect the Ln 4f N occupancy. In this Account, we review our anion PE spectroscopic and computational studies on a range of small lanthanide molecules and cluster species. The PE spectra of LnO – (Ln = Ce, Pr, Sm, Eu) diatomic molecules show spectroscopic signatures associated with detachment of an electron from what can be described as a diffuse Ln 6s-like orbital. While the spectra of all four diatomics share this common transition, the fine structure in the transition becomes more complex with increasing 4f occupancy. This effect reflects increased coupling between the electrons occupying the corelike 4f and diffuse 6s orbitals with increasing N. Understanding the PE spectra of these diatomics sets the stage for interpreting the spectra of polyatomic molecular and cluster species. In general, the results confirm that the partial 4f N subshell occupancy is largely preserved between molecular and bulk oxides and borides. However, they also suggest that surfaces and edges of bulk materials may support a low-energy, diffuse Ln 6s band, in contrast to bulk interiors, in which the 6s band is destabilized relative to the 5d band. We also identify cases in which the molecular Ln centers have 4f N+1 occupancy rather than bulklike 4f N , which results in weaker Ln–O bonding. Specifically, Sm centers in mixed Ce–Sm oxides or in Sm x O y – (y ≤ x) clusters have this higher 4f N+1 occupancy. The PE spectra of these particular species exhibit a striking increase in the relative intensities of excited-state transitions with decreasing photon energy (resulting in lower photoelectron kinetic energy). Furthermore, this is opposite of what is expected on the basis of the threshold laws that govern photodetachment. We relate this phenomenon to strong electron–neutral interactions unique to these complex electronic structures. The time scale of the interaction, which shakes up the electronic configuration of the neutral, increases with decreasing electron momentum. From a computational standpoint, we point out that special care must be taken when considering Ln cluster and molecular systems toward the center of the Ln series (e.g., Sm, Eu), where treatment of electrons explicitly or using an effective core potential can yield conflicting results on competing subshell occupancies. However, despite the complex electronic structures associated with partially filled 4f N subshells, we demonstrate that inexpensive and tractable calculations yield useful qualitative insight into the general electronic structural features.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Revisiting benzene cluster cations for the chemical ionization of dimethyl sulfide and select volatile organic compounds

Benzene cluster cations were revisited as a sensitive and selective reagent ion for the chemical ionization of dimethyl sulfide (DMS) and a select group of volatile organic compounds (VOCs). Laboratory characterization was performed using both a new set of compounds (i.e., DMS, β -caryophyllene) as well as previously studied VOCs (i.e., isoprene, α -pinene). Using a field deployable chemical-ionization time-of-flight mass spectrometer (CI-ToFMS), benzene cluster cations demonstrated high sensitivity (> 1 ncps ppt &minus;1 ) to DMS, isoprene, and α -pinene standards. Parallel measurements conducted using a chemical-ionization quadrupole mass spectrometer, with a much weaker electric field, demonstrated that ion–molecule reactions likely proceed through a combination of ligand-switching and direct charge transfer mechanisms. Laboratory tests suggest that benzene cluster cations may be suitable for the selective ionization of sesquiterpenes, where minimal fragmentation (< 25 %) was observed for the detection of β -caryophyllene, a bicyclic sesquiterpene. The in-field stability of benzene cluster cations using CI-ToFMS was examined in the marine boundary layer during the High Wind Gas Exchange Study (HiWinGS). The use of benzene cluster cation chemistry for the selective detection of DMS was validated against an atmospheric pressure ionization mass spectrometer, where measurements from the two instruments were highly correlated ( R 2 > 0.95, 10 s averages) over a wide range of sampling conditions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Properties of polycyclic aromatic hydrocarbons in the northwest photon dominated region of NGC 7023. II. Traditional PAH analysis using k-means as a visualization tool

Polycyclic aromatic hydrocarbon (PAH) emission in the Spitzer-IRS spectral map of the northwest photon dominated region (PDR) in NGC 7023 is analyzed using the 'traditional' approach in which the PAH bands and plateaus between 5.2-19.5 μm are isolated by subtracting the underlying continuum and removing H{sub 2} emission lines. The spectra are organized into seven spectroscopic bins by using k-means clustering. Each cluster corresponds to, and reveals, a morphological zone within NGC 7023. The zones self-organize parallel to the well-defined PDR front that coincides with an increase in intensity of the H{sub 2} emission lines. PAH band profiles and integrated strengths are measured, classified, and mapped. The morphological zones revealed by the k-means clustering provides deeper insight into the conditions that drive variations in band strength ratios and evolution of the PAH population that otherwise would be lost. For example, certain band-band relations are bifurcated, revealing two limiting cases; one associated with the PDR, the other with the diffuse medium. Traditionally, PAH band strength ratios are used to gain insight into the properties of the emitting PAH population, i.e., charge, size, structure, and composition. Insights inferred from this work are compared and contrasted to those from Boersma et al. (first paper in this series), where the PAH emission in NGC 7023 is decomposed exclusively using the PAH spectra and tools made available through the NASA Ames PAH IR Spectroscopic Database.

79 ASTRONOMY AND ASTROPHYSICS↗