Search NASA⌕ Search

SEARCH · Search NASA

Results for “data clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

Multi-Parent Clustering Algorithms from Stochastic Grammar Data Models

We introduce a statistical data model and an associated optimization-based clustering algorithm which allows data vectors to belong to zero, one or several "parent" clusters. For each data vector the algorithm makes a discrete decision among these alternatives. Thus, a recursive version of this algorithm would place data clusters in a Directed Acyclic Graph rather than a tree. We test the algorithm with synthetic data generated according to the statistical data model. We also illustrate the algorithm using real data from large-scale gene expression assays.

Mjoisness, Eric↗

ANCHORING THE DISTANCE SCALE VIA X-RAY/INFRARED DATA FOR CEPHEID CLUSTERS: SU Cas

New X-ray (XMM-Newton) and JHK{sub s} (Observatoire du Mont-Megantic) observations for members of the star cluster Alessi 95, which Turner et al. discovered hosts the classical Cepheid SU Cas, were used in tandem with UCAC3 (proper motion) and Two Micron All Sky Survey observations to determine precise cluster parameters: E(J - H) = 0.08 {+-} 0.02 and d = 405 {+-} 15 pc. The ensuing consensus among cluster, pulsation, and trigonometric distances (d=414{+-}5({sigma}{sub x}-bar){+-}10 ({sigma}) pc) places SU Cas in a select group of nearby fundamental Cepheid calibrators ({delta} Cep, {zeta} Gem). High-resolution X-ray observations may be employed to expand that sample as the data proved pertinent for identifying numerous stars associated with SU Cas. Acquiring X-ray observations of additional fields may foster efforts to refine Cepheid calibrations used to constrain H{sub 0}.

79 ASTRONOMY AND ASTROPHYSICS↗

Dimension Reduction of Hyperspectral Data on Beowulf Clusters

Traditional remote sensing instruments are multispectral, where observations are collected at a few different spectral bands. Recently, many hyperspectral instruments, that can collect observations at hundreds of bands, have been operation. Furthermore, there have been ongoing research efforts on ultraspectral instruments that can produce observations at thousands of spectral bands. While these remote sensing technology developments hold a great promise for new findings in the area of Earth and space science, they present many challenges. These include the need for faster processing of such increased data volumes, and methods for data reduction. Dimension Reduction is a spectral transformation, which is used widely in remote sensing, is the Principal Components Analysis (PCA). In light of the growing number of spectral channels of modern instruments, the paper reports on the development of a parallel PCA and its implementation on two Beowulf cluster configurations, on with fast Ethernet switch and the other is with a Myrinet interconnection.

El-Ghazawi, Tarek↗

Demonstrating the feasibility of line intensity mapping using mock data of galaxy clustering from simulations

Visbal and Loeb (2010) have shown that it is possible to measure the clustering of galaxies by cross correlating the cumulative emission from two different spectral lines which originate at the same redshift. Through this cross correlation, one can study galaxies which are too faint to be individually resolved. This technique, known as intensity mapping, is a promising probe of the global properties of high redshift galaxies. Here, we test the feasibility of such measurements with synthetic data generated from cosmological dark matter simulations. We use a simple prescription for associating galaxies with dark matter halos and create a realization of emitted radiation as a function of angular position and wavelength over a patch of the sky. This is then used to create synthetic data for three different hypothetical instruments, one aboard the Space Infrared Telescope for Cosmology and Astrophysics (SPICA), another consisting of a pair of ground based radio telescopes designed to measure the CO(1-0) and CO(2-1) emission lines, and one on the Cornell Caltech Atacama Telescope (CCAT). We find that the line cross power spectrum can be measured accurately from the synthetic data with errors consistent with the analytical prediction of Visbal and Loeb (2010). Removal of astronomical backgrounds and masking bright line emission from foreground contaminating galaxies do not prevent accurate cross power spectrum measurements.

79 ASTRONOMY AND ASTROPHYSICS↗

Mass and galaxy distributions of four massive galaxy clusters from Dark Energy Survey Science Verification data

We measure the weak lensing masses and galaxy distributions of four massive galaxy clusters observed during the Science Verification phase of the Dark Energy Survey (DES). This pathfinder study is meant to (1) validate the Dark Energy Camera (DECam) imager for the task of measuring weak lensing shapes, and (2) utilize DECam's large field of view to map out the clusters and their environments over 90 arcmin. We conduct a series of rigorous tests on astrometry, photometry, image quality, point spread function (PSF) modelling, and shear measurement accuracy to single out flaws in the data and also to identify the optimal data processing steps and parameters. We find Science Verification data from DECam to be suitable for the lensing analysis described in this paper. The PSF is generally well behaved, but the modelling is rendered difficult by a flux-dependent PSF width and ellipticity. We employ photometric redshifts to distinguish between foreground and background galaxies, and a red-sequence cluster finder to provide cluster richness estimates and cluster-galaxy distributions. By fitting Navarro-Frenk-White profiles to the clusters in this study, we determine weak lensing masses that are in agreement with previous work. For Abell 3261, we provide the first estimates of redshift, weak lensing mass, and richness. In addition, the cluster-galaxy distributions indicate the presence of filamentary structures attached to 1E 0657-56 and RXC J2248.7-4431, stretching out as far as 1. (approximately 20 Mpc), showcasing the potential of DECam and DES for detailed studies of degree-scale features on the sky.

79 ASTRONOMY AND ASTROPHYSICS↗

Production of alternate realizations of DESI fiber assignment for unbiased clustering measurement in data and simulations

A critical requirement of spectroscopic large scale structure analyses is correcting for selection of which galaxies to observe from an isotropic target list. This selection is often limited by the hardware used to perform the survey which will impose angular constraints of simultaneously observable targets, requiring multiple passes to observe all of them. In SDSS this manifested solely as the collision of physical fibers and plugs placed in plates. In DESI, there is the additional constraint of the robotic positioner which controls each fiber being limited to a finite patrol radius. A number of approximate methods have previously been proposed to correct the galaxy clustering statistics for these effects, but these generally fail on small scales. To accurately correct the clustering we need to upweight pairs of galaxies based on the inverse probability that those pairs would be observed (Bianchi & Percival 2017). This paper details an implementation of that method to correct the Dark Energy Spectroscopic Instrument (DESI) survey for incompleteness. To calculate the required probabilities, we need a set of alternate realizations of DESI where we vary the relative priority of otherwise identical targets. These realizations take the form of alternate Merged Target Ledgers (AMTL), the files that link DESI observations and targets. We present the method used to generate these alternate realizations and how they are tracked forward in time using the real observational record and hardware status, propagating the survey as though the alternate orderings had been adopted. We detail the first applications of this method to the DESI One-Percent Survey (SV3) and the DESI year 1 data. We include evaluations of the pipeline outputs, estimation of survey completeness from this and other methods, and validation of the method using mock galaxy catalogs.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: Double-probe measurements from BOSS galaxy clustering and Planck data – towards an analysis without informative priors

Here, we develop a new methodology called double-probe analysis with the aim of minimizing informative priors in the estimation of cosmological parameters. We extract the dark-energy-model-independent cosmological constraints from the joint data sets of Baryon Oscillation Spectroscopic Survey (BOSS) galaxy sample and Planck cosmic microwave background (CMB) measurement. We measure the mean values and covariance matrix of {R, l a , Ω b h 2 , n s , log(A s ), Ω k , H(z), D A (z), f(z)σ 8 (z)}, which give an efficient summary of Planck data and 2-point statistics from BOSS galaxy sample, where R = √Ω m H 2 0 , and l a = πr(z * )/r s (z * ), z * is the redshift at the last scattering surface, and r(z * ) and r s (z * ) denote our comoving distance to z * and sound horizon at z * respectively. The advantage of this method is that we do not need to put informative priors on the cosmological parameters that galaxy clustering is not able to constrain well, i.e. Ω b h 2 and n s . Using our double-probe results, we obtain Ω m = 0.304 ± 0.009, H 0 = 68.2 ± 0.7, and σ 8 = 0.806 ± 0.014 assuming ΛCDM; and Ω k = 0.002 ± 0.003 and w = –1.00 ± 0.07 assuming owCDM. The results show no tension with the flat ΛCDM cosmological paradigm. By comparing with the full-likelihood analyses with fixed dark energy models, we demonstrate that the double-probe method provides robust cosmological parameter constraints which can be conveniently used to study dark energy models. We extend our study to measure the sum of neutrino mass and obtain Σm ν < 0.10/0.22 (68%/95%) assuming ΛCDM and Σm ν < 0.26/0.52 (68%/95%) assuming wCDM. This paper is part of a set that analyses the final galaxy clustering dataset from BOSS.

79 ASTRONOMY AND ASTROPHYSICS↗

Active Learning A Neural Network Model For Gold Clusters & Bulk From Sparse First Principles Training Data

Small metal clusters are of fundamental scientific interest and of tremendous significance in catalysis. These nanoscale clusters display diverse geometries and structural motifs depending on the cluster size; a knowledge of this size-dependent structural motifs and their dynamical evolution has been of longstanding interest. Given the high computational cost of first-principles calculations, molecular modeling and atomistic simulations such as molecular dynamics (MD) has proven to be an important complementary tool to aid this understanding. Classical MD typically employ predefined functional forms which limits their ability to capture such complex size-dependent structural and dynamical transformation. Neural Network (NN) based potentials represent flexible alternatives and in principle, well-trained NN potentials can provide high level of flexibility, transferability and accuracy on-par with the reference model used for training. A major challenge, however, is that NN models are interpolative and requires large quantities (similar to 10 4 or greater) of training data to ensure that the model adequately samples the energy landscape both near and far-from-equilibrium. A highly desirable goal is minimize the number of training data, especially if the underlying reference model is first-principles based and hence expensive. In this work, we introduce an active learning (AL) scheme that trains a NN model on-the-fly with minimal amount of first-principles based training data. Our AL workflow is initiated with a sparse training dataset (similar to 1 to 5 data points) and is updated on-the-fly via a Nested Ensemble Monte Carlo scheme that iteratively queries the energy landscape in regions of failure and updates the training pool to improve the network performance. Using a representative system of gold clusters, we demonstrate that our AL workflow can train a NN with similar to 500 total reference calculations. Using an extensive DFT test set of similar to 1100 configurations, we show that our AL-NN is able to accurately predict both the DFT energies and the forces for clusters of a myriad of different sizes. Our NN predictions are within 30 meV/atom and 40 meV/angstrom of the reference DFT calculations. Moreover, our AL-NN model also adequately captures the various size-dependent structural and dynamical properties of gold clusters in excellent agreement with DFT calculations and available experiments. We finally show that our AL-NN model also captures bulk properties reasonably well, even though they were not included in the training data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

XRootD popularity on hadoop clusters

Performance data and metadata of the computing operations at the CMS experiment are collected through a distributed monitoring infrastructure, currently relying on a traditional Oracle database system. This paper shows how to harness Big Data architectures in order to improve the throughput and the efficiency of such monitoring. A large set of operational data - user activities, job submissions, resources, file transfers, site efficiencies, software releases, network traffic, machine logs - is being injected into a readily available Hadoop cluster, via several data streamers. The collected metadata is further organized running fast arbitrary queries; this offers the ability to test several Map&Reduce-based frameworks and measure the system speed-up when compared to the original database infrastructure. By leveraging a quality Hadoop data store and enabling an analytics framework on top, it is possible to design a mining platform to predict dataset popularity and discover patterns and correlations.

97 MATHEMATICS AND COMPUTING↗

Harnessing Data Movement in Virtual Clusters for In-Situ Execution

As a result of increasing data volume and velocity, Big Data science at exascale has shifted towards the in-situ paradigm, where large scale simulations run concurrently alongside data analytics. With in-situ, data generated from simulations can be processed while still in memory, thereby avoiding the slow storage bottleneck. However, running simulations and analytics together on shared resources will likely result in substantial contention if left unmanaged, as demonstrated in this work, leading to much reduced efficiency of simulations and analytics. Recently, virtualization technologies such as Linux containers have been widely applied to data centers and physical clusters to provide highly efficient and elastic resource provisioning for consolidated workloads including scientific simulations and data analytics. In this paper, we investigate to facilitate network traffic manipulation and reduce mutual interference on the network for in-situ applications in virtual clusters. In order to dynamically allocate the network bandwidth when it is needed, we adopt SARIMA-based techniques to analyze and predict MPI traffic issued from simulations. Although this can be an effective technique, the naïve usage of network virtualization can lead to performance degradation for bursty asynchronous transmissions within an MPI job. Here, we analyze and resolve this performance degradation in virtual clusters.

97 MATHEMATICS AND COMPUTING↗

Nontargeted virus sequence discovery pipeline and virus clustering for metagenomic data

The analysis of large microbiome data sets holds great promise for the delineation of the biological and metabolic functioning of living organisms and their role in the environment. In the midst of this genomic puzzle, viruses, especially those that infect microbial communities, represent a major reservoir of genetic diversity with great impact on biogeochemical cycles and organismal health. Overcoming the limitations associated with virus detection directly from microbiomes can provide key insights into how ecosystem dynamics are modulated. Here, we present a computational protocol for accurate detection and grouping of viral sequences from microbiome samples. Our approach relies on an expanded and curated set of viral protein families used as bait to identify viral sequences directly from metagenomic assemblies. This protocol describes how to use the viral protein families catalog (~7 h) and recommended filters for the detection of viral contigs in metagenomic samples (~6 h), and it describes the specific parameters for a nucleotide-sequence-identity-based method of organizing the viral sequences into quasi-species taxonomic-level groups (~10 min).

Paez-Espino, David↗

Deprojection of X-ray data in galaxy clusters: confronting simulations with observations

ABSTRACT Numerical simulations with varying realism indicate an emergent principle−multiphase condensation and large cavity power occur when the ratio of the cooling time to the free-fall time (tcool/tff) falls below a threshold value close to 10. Observations indeed show cool-core signatures when this ratio falls below 20–30, but the prevalence of cores with tcool/tff ratio below 10 is rare as compared to simulations. In X-ray observations, we obtain projected spectra from which we have to infer radial gas density and temperature profiles. Using idealized models of X-ray cavities and multiphase gas in the core and 3D hydro jet-ICM simulations, we quantify the biases introduced by deprojection based on the assumption of spherical symmetry in determining tcool/tff. We show that while the used methods are able to recover the tcool/tff ratio for relaxed clusters, they have an uncertainty of a factor of 2−3 in systems containing large cavities (≳ 20 kpc). We also show that the mass estimates from these methods, in the absence of X-ray spectra close to the virial radius, suffer from a degeneracy between the virial mass (M200) and the concentration parameter (c) in the form of M200c2 ≈ constant. Additionally, the lack of soft-X-ray (≲ 0.5 keV) coverage and poor spatial resolution makes us overestimate min(tcool/tff) by a factor of few in clusters with min(tcool/tff) ≲ 5. This bias can largely explain the lack of cool-core clusters with min(tcool/tff) ≲ 5.

Sarkar, Kartick C. (ORCID:0000000277678472)↗

Quantum cluster algorithm for data classification

Abstract We present a quantum algorithm for data classification based on the nearest-neighbor learning algorithm. The classification algorithm is divided into two steps: Firstly, data in the same class is divided into smaller groups with sublabels assisting building boundaries between data with different labels. Secondly we construct a quantum circuit for classification that contains multi control gates. The algorithm is easy to implement and efficient in predicting the labels of test data. To illustrate the power and efficiency of this approach, we construct the phase transition diagram for the metal-insulator transition of VO 2 , using limited trained experimental data, where VO 2 is a typical strongly correlated electron materials, and the metallic-insulating phase transition has drawn much attention in condensed matter physics. Moreover, we demonstrate our algorithm on the classification of randomly generated data and the classification of entanglement for various Werner states, where the training sets can not be divided by a single curve, instead, more than one curves are required to separate them apart perfectly. Our preliminary result shows considerable potential for various classification problems, particularly for constructing different phases in materials.

97 MATHEMATICS AND COMPUTING↗