Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

CO adsorption on (111) and (100) surfaces of the Pt sub 3 Ti alloy. Evidence for parallel binding and strong activation of CO

The CO adsorption on a 40 atom cluster model of the (111) surface and a 36 atom cluster model of the (100) surface of the Pt3Ti alloy was studied. Parallel binding to high coordinate sites associated with Ti and low CO bond scission barriers are predicted for both surfaces. The binding of CO to Pt sites occurs in an upright orientation. These orientations are a consequence of the nature of the CO pi donation interactions with the surface. On the Ti sites the orbitals donate to the nearly empty Ti 3d band and the antibonding counterpart orbitals are empty. On the Pt sites, however, they are in the filled Pt 5d region of the alloy band, which causes CO to bond in a vertical orientation by 5 delta donation from the carbon end.

Mehandru, S. P.↗

A multi-platform evaluation of the randomized CX low-rank matrix factorization in Spark

We investigate the performance and scalability of the randomized CX low-rank matrix factorization and demonstrate its applicability through the analysis of a 1TB mass spectrometry imaging (MSI) dataset, using Apache Spark on an Amazon EC2 cluster, a Cray XC40 system, and an experimental Cray cluster. We implemented this factorization both as a parallelized C implementation with hand-tuned optimizations and in Scala using the Apache Spark high-level cluster computing framework. We obtained consistent performance across the three platforms: using Spark we were able to process the 1TB size dataset in under 30 minutes with 960 cores on all systems, with the fastest times obtained on the experimental Cray cluster. In comparison, the C implementation was 21X faster on the Amazon EC2 system, due to careful cache optimizations, bandwidth-friendly access of matrices and vector computation using SIMD units. We report these results and their implications on the hardware and software issues arising in supporting data-centric workloads in parallel and distributed environments.

Gittens, Alex↗

miniVite: A Graph Analytics Benchmarking Tool for Massively Parallel Systems

Benchmarking of high performance computing sys-tems can help provide critical insights for ef?cient design of computing systems and software applications. Although a large number of tools for benchmarking exist, there is a lack of rep-resentative benchmarks for the class of irregular computations as exempli?ed by graph analytics. In this paper, we propose miniVite as a representative graph analytics benchmark tool to test a variety of distributed-memory systems. Graph clustering, popularly known as community detection, is a prototypical graph operation used in numerous scienti?c computing and analytics applications. The goal of clustering is to partition a graph into clusters (or communities) such that each cluster consists of vertices that are densely connected within the cluster and sparsely connected to the rest of the graph. Modu-larity optimization is a popular technique for identifying clusters in a graph. Ef?cient parallelization of modularity optimization-based algorithms is challenging. One successful approach was conceived in Vite, a distributed-memory implementation of the Louvain algorithm that incorporates several heuristics. We introduce miniVite as a representative but simpli?ed variant of Vite, to serve as a prototypical graph analytics benchmarking tool. Unlike other graph-based methods such as breadth-?rst search and betweenness centrality, miniVite represents highly complex computational patterns stressing a variety of system features, which can provide crucial insight for co-design of future computing systems.

community detection, parallel, MPI↗

Scalable, In-situ Data Clustering Data Analysis for Extreme Scale Scientific Computing (Final Report)

The objective of this project is to address challenges in the design and development of scalable in-situ data clustering and analytics algorithms and software. Our goal is to develop parallel software consisting of a set of spatio-temporal data clustering and anomaly detection functions, both of which are very important for large-scale analysis and have wide applicability for in-situ runs as well as post-processing analysis. Our design principles for in-situ analysis consider the following: (1) identify parts of the computation can be done close to the data within the nodes, while it is still in memory; (2) extract analysis components can (and should) be performed in remote staging and analysis nodes; (3) develop error-bound approximation methods for applications tolerable for small errors; (4) identify the type of derived distributions and statistics, for spatio-temporal data, that can be kept locally in order to both accelerate computations and meet energy constraints in subsequent iterations and phases; (5) use a self-describing data format so that data can be consistent and understood among local storage (memory and SSDs) and at staging and analysis nodes, thereby providing portability and flexibility; (6) develop service-oriented functions that can schedule in-situ and post-hoc analysis tasks based on the dynamic requirements of applications. Our development focus is to produce the parallel data analysis software/library that will be scalable, reusable, extensible, and generic for applications in different disciplines. The software will be able to run in-situ with the simulations as well as post-hoc analysis. This approach will satisfy many synergistic requirements for data intensive applications executed on data coming from instruments and experiments. In particular, the proposed multilevel approach is directly applicable to perform design tradeoffs for running part of the algorithms near the instruments and the rest on remote (analysis) systems.

97 MATHEMATICS AND COMPUTING↗

Parallel k-means++

A parallelization of the k-means++ seed selection algorithm on three distinct hardware platforms: GPU, multicore CPU, and multithreaded architecture. K-means++ was developed by David Arthur and Sergei Vassilvitskii in 2007 as an extension of the k-means data clustering technique. These algorithms allow people to cluster multidimensional data, by attempting to minimize the mean distance of data points within a cluster. K-means++ improved upon traditional k-means by using a more intelligent approach to selecting the initial seeds for the clustering process. While k-means++ has become a popular alternative to traditional k-means clustering, little work has been done to parallelize this technique. We have developed original C++ code for parallelizing the algorithm on three unique hardware architectures: GPU using NVidia's CUDA/Thrust framework, multicore CPU using OpenMP, and the Cray XMT multithreaded architecture. By parallelizing the process for these platforms, we are able to perform k-means++ clustering much more quickly than it could be done before.

Source record↗

rustpix

rustpix is a high-performance, open-source Rust library with first-class Python bindings (via PyO3) for processing pixel-detector data in neutron imaging. It targets time-stamping detectors such as Timepix3 (TPX3) at ORNL's Spallation Neutron Source (VENUS beamline), where each detected neutron deposits charge across a cluster of pixels within a very high-rate event stream (96M+ hits/sec). rustpix parses TPX3 event data in parallel using memory-mapped I/O, offers four interchangeable clustering algorithms (ABS adjacency-based search, DBSCAN, graph/union-find connected components, and a parallel grid method), and extracts weighted, super-resolved centroids to produce neutron-event lists. A streaming architecture lets it process files larger than available memory. rustpix is distributed as a pip-installable Python package (with NumPy integration), Rust crates, a command-line tool, and an interactive GUI; it writes HDF5, Apache Arrow, and CSV; and it is designed to extend to TPX4 and other detector types. Released as open-source under the MIT License.

Zhang, Chen [Oak Ridge National Laboratory (ORNL),↗

Simulation of Coupled Processes of Flow, Transport, and Storage of CO 2 in Saline Aquifers

This report is the final scientific one for the award DE- FE0000988 entitled “Simulation of Coupled Processes of Flow, Transport, and Storage of CO 2 in Saline Aquifers.” The work has been divided into six tasks. In task, “Development of a Three-Phase Non-Isothermal CO 2 Flow Module,” we developed a fluid property module for brine-CO 2 mixtures designed to handle all possible phase combinations of aqueous phase, sub-critical liquid and gaseous CO 2 , supercritical CO 2 , and solid salt. The thermodynamic and thermophysical properties of brine-CO 2 mixtures (density, viscosity, and specific enthalpy of fluid phases; partitioning of mass components among the different phases) use the same correlations as an earlier fluid property module that does not distinguish between gaseous and liquid CO 2 -rich phases. We verified the fluid property module using two leakage scenarios, one that involves CO 2 migration up a blind fault and subsequent accumulation in a secondary “parasitic” reservoir at shallower depth, and another investigating leakage of CO 2 from a deep storage reservoir along a vertical fault zone. In task, “Development of a Rock Mechanical Module,” we developed a massively parallel reservoir simulator for modeling THM processes in porous media brine aquifers. We derived, from the fundamental equations describing deformation of porous elastic media, a momentum conservation equation relating mean stress, pressure, and temperature, and incorporated it alongside the mass and energy conservation equations from the TOUGH2 formulation, the starting point for the simulator. In addition, rock properties, namely permeability and porosity, are functions of effective stress and other variables that are obtained from the literature. We verified the simulator formulation and numerical implementation using analytical solutions and example problems from the literature. For the former, we matched a one-dimensional consolidation problem and a two-dimensional simulation of the Mandel-Cryer effect. For the latter, we obtained a good match of temperature and gas saturation profiles, and surface uplift, after injection of hot fluid into a model of a caldera structure. In task, “Incorporation of Geochemical Reactions of Selected Important Species,” we developed a novel mathematical model of THMC processes in porous and fractured saline aquifers, simulating geo-chemical reactions associated with CO 2 sequestration in saline aquifers. Two computational frameworks, sequentially coupled and fully coupled, were used to simulate the reactions and transport. We verified capabilities of the THMC model to treat complex THMC processes during CO 2 sequestration by analytical solutions and we constructed reactive transport models to analyze the THMC process quantitatively. Three of these are 1D reactive transport under chemical equilibrium, a batch reaction model with equilibrium chemical reactions, and a THMC model with CO 2 dissolution. In task “Study of Instability in CO 2 Dissolution-Diffusion-Convection Processes,” We reviewed literature related to the study of density driven convective flows and on the instability of CO 2 dissolution-diffusion-convection processes. We ran simulations that model the density-driven flow instability that would occur during CO 2 sequestration. CO 2 diffused through the top of the system and dissolved in the aqueous phase there, increasing its density. Density fingers formed along the top boundary, and coalesced into a few prominent ones, causing convective flow that forced the fluid to the system bottom. These simulations were in two and three dimensions. We ran additional simulations of convective mixing with density contrast caused by variable dissolved CO 2 concentration in saline water, modeled after laboratory experiments in which supercritical CO2 was circulated in the headspace above a brine saturated packed sand in a pressure vessel. As CO 2 dissolved into the upper part of the saturated sand, liquid phase density increases causing instability and setting off convective mixing. We obtained good agreement with the laboratory experiments, which were characterized by finger development and associated mixing of dissolved CO 2 into the system. We then varied a wide range of parameters and conceptual models in order to analyze the possibility of convective mixing under different conditions, such as various boundary conditions, and chemical reaction conditions. The CO 2 fingers from different simulations showed great differences as time progressed, caused by permeability heterogeneity. The early time diffusive phenomenon was captured by fine grid resolution, and the permeability heterogeneity affected the pattern of the CO 2 fingers. In addition, the fingers from three-dimensional simulations tended to be larger and flatter than the two-dimensional ones. In task “Implementation of Efficient Parallel Computing Technologies,” we made enhancements and modifications to our code in order to substantially increase the grid size that could be run. We installed and ran it on various platforms, including a multi-core PC and a cluster, and verified the numerical implementation and parallel code using an example problem from the literature. This problem, with a grid size of sixty million, utilized the cluster’s entire memory and all of its processors. In task “Implementation of General Fracture Conceptual Models,” we used the MINC approach, a generalization of the double-porosity concept, to model flow through porous and fractured media. In this approach, flow within the matrix is described by subdividing the matrix into nested volumes, with flow occurring between adjacent nested matrix volumes as well as between the fractures and the outer matrix volume. We generalized Hooke’s law to a thermo-multi- poroelastic medium, and derived from the fundamental equations describing deformation of porous and fractured elastic media a momentum conservation equation for thermo-multi- poroelastic media. This equation is a generalization to multi-poroelastic media of the one derived in Task 3.0 for single porosity media. We describe two simulations to provide model verification and application examples. The first, one-dimensional consolidation of a double-porosity medium, is compared to an analytical solution. The second is a match of published results from the literature, a simulation of CO 2 injection into hypothetical aquifer-caprock systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Parallel multiple-chain DRAM MCMC for large-scale geosteering inversion and uncertainty quantification

Geosteering is the proactive control of a wellbore placement based on the downhole measurements, aiming at maximizing economic production from the well. We report the recent development of azimuthal resistivity logging-whiledrilling (LWD) tools have a much larger depth of detection, thus sets higher demands for geosteering inversion as more unknown parameters need to be taken into account in the earth model to be inverted. For complicated nonlinear problems, traditional deterministic inversion methods are more likely to be trapped near local optima. In general, Markov chain Monte Carlo (MCMC) inversion methods are more capable of finding the global optimal solution and providing additional uncertainty analyses by sampling from the target distribution. However, MCMC methods usually have the problem of slow convergence. Even though optimization methods like delayed rejection (DR) and adaptive Metropolis (AM) were proposed to speed up the convergence, using MCMC methods to solve geosteering inversion problems may still incur unacceptable time cost. To reduce the sampling cost of MCMC methods, we use parallel multiple-chain DRAM MCMC methods to solve geosteering inverse problems and estimate the corresponding uncertainty. A clustering method, density-based spatial clustering of applications with noise (DBSCAN), is applied to select the best solution among many results. Lastly, the simulation results show that running many relatively short MCMC chains for one problem can obtain a similar result as running a long single chain. Besides, avoiding communications between multiple Markov chains during the sampling process yields almost linear scalability.

97 MATHEMATICS AND COMPUTING↗

Parallel Signal Processing and System Simulation using aCe

Recently, networked and cluster computation have become very popular for both signal processing and system simulation. A new language is ideally suited for parallel signal processing applications and system simulation since it allows the programmer to explicitly express the computations that can be performed concurrently. In addition, the new C based parallel language (ace C) for architecture-adaptive programming allows programmers to implement algorithms and system simulation applications on parallel architectures by providing them with the assurance that future parallel architectures will be able to run their applications with a minimum of modification. In this paper, we will focus on some fundamental features of ace C and present a signal processing application (FFT).

Dorband, John E.↗

Expressing Parallelism with ROOT

The need for processing the ever-increasing amount of data generated by the LHC experiments in a more efficient way has motivated ROOT to further develop its support for parallelism. Such support is being tackled both for shared-memory and distributed-memory environments. The incarnations of the aforementioned parallelism are multi-threading, multi-processing and cluster-wide executions. In the area of multi-threading, we discuss the new implicit parallelism and related interfaces, as well as the new building blocks to safely operate with ROOT objects in a multi-threaded environment. Regarding multi-processing, we review the new MultiProc framework, comparing it with similar tools (e.g. multiprocessing module in Python). Finally, as an alternative to PROOF for cluster-wide executions, we introduce the efforts on integrating ROOT with state-of-the-art distributed data processing technologies like Spark, both in terms of programming model and runtime design (with EOS as one of the main components). For all the levels of parallelism, we discuss, based on real-life examples and measurements, how our proposals can increase the productivity of scientists.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

A non-destructive method for crack quantification in photovoltaic backsheets under accelerated and real-world exposures

The long-term durability of photovoltaic modules is paramount for the continued growth of the industry. Polymer backsheets are of particular concern since they provide electrical insulation and an environmental barrier. In this paper, 23 freestanding, multilayer backsheets with nine unique material combinations underwent four different weathering exposures under accelerated and real-world conditions. Besides changes in color and gloss, the induced degradation included parallel or mudflat cracks on 11 backsheets, sometimes in combination with delamination or blistering. Similar degradation has been observed in previous studies and is concerning since cracks compromise the mechanical integrity and electrical safety of backsheets. Quantitative parameters are desirable to reliably classify categories of cracks and supply unbiased features for statistical analysis in predictive lifetime models. We developed an analysis technique that utilizes surface profilometry data to quantify the depth, width, area, spacing, and number of cracks. Parameters are automatically extracted from the raw data by an algorithm running on a high performance distributed computing cluster. Our algorithm excelled at characterizing parallel cracks with minimal de-adhesion, and only an estimated 4% of crack detections were false positives. The addition of humidity and temperature variation formed up to three times as many cracks on a photodose basis compared to dry, constant temperature exposures. Cracks in real-world and accelerated exposures propagated to similar depths with equivalent photodoses; however, the number of cracks formed in accelerated exposures was far greater on a photodose basis. Of samples that cracked, the best performing backsheet configuration was polyvinyl fluoride/poly (ethylene-terephthalate)/polyethylene (PVF/PET/PE) while the least durable was PET/PET/ethylene-vinyl acetate. Finally, none of the six PVF/PET/PVF backsheets cracked in any of the exposures.

14 SOLAR ENERGY↗

Deconvolute individual genomes from metagenome sequences through short read clustering

Metagenome assembly from short next-generation sequencing data is a challenging process due to its large scale and computational complexity. Clustering short reads by species before assembly offers a unique opportunity for parallel downstream assembly of genomes with individualized optimization. However, current read clustering methods suffer either false negative (under-clustering) or false positive (over-clustering) problems. Here we extended our previous read clustering software, SpaRC, by exploiting statistics derived from multiple samples in a dataset to reduce the under-clustering problem. Using synthetic and real-world datasets we demonstrated that this method has the potential to cluster almost all of the short reads from genomes with sufficient sequencing coverage. The improved read clustering in turn leads to improved downstream genome assembly quality.

59 BASIC BIOLOGICAL SCIENCES↗

THE MISSING WEAK LENSING MASS IN A781

Measuring cluster masses accurately is important for testing the cosmological paradigm. Weak lensing is one of the most promising methods for detecting, measuring, and calibrating cluster mass estimates made using other mass proxies (e.g., X-ray, Sunyaev-Zel'dovich effect, spectroscopy). However, it is still essential to characterize and understand the causes of systematic error and bias in weak lensing measurements. A781D is a cluster of galaxies with a mass and redshift that places it well within the theoretical detection limits of weak lensing analyses from the ground yet has evaded detection in previous weak lensing studies. Previous weak lensing measurements in the region surrounding this cluster from the Deep Lens Survey were unable to detect it and placed a 1{sigma} limit on the mass of <5 Multiplication-Sign 10{sup 13} M{sub Sun }. Given independent estimates of the cluster mass by X-ray and spectroscopic measurements and its spectroscopically confirmed redshift of 0.43, it is difficult to explain its absence from the weak lensing mass reconstructions. We re-analyzed this cluster using imaging from the Orthogonal Parallel Transfer Imaging Camera and archival Suprime-Cam data. We successfully detect A781A in both analyses, but A781D remains undetected. We use these two new independent analyses to rule out systematic effects from the telescope, instrument, and point-spread function correction as the cause of the null detection. We also demonstrate the first use of an orthogonal transfer camera for weak lensing analysis and demonstrate its suitability for weak lensing studies.

79 ASTRONOMY AND ASTROPHYSICS↗

Parallel Rendering of Large Time-Varying Volume Data

Interactive visualization of large time-varying 3D volume datasets has been and still is a great challenge to the modem computational world. It stretches the limits of the memory capacity, the disk space, the network bandwidth and the CPU speed of a conventional computer. In this SURF project, we propose to develop a parallel volume rendering program on SGI's Prism, a cluster computer equipped with state-of-the-art graphic hardware. The proposed program combines both parallel computing and hardware rendering in order to achieve an interactive rendering rate. We use 3D texture mapping and a hardware shader to implement 3D volume rendering on each workstation. We use SGI's VisServer to enable remote rendering using Prism's graphic hardware. And last, we will integrate this new program with ParVox, a parallel distributed visualization system developed at JPL. At the end of the project, we Will demonstrate remote interactive visualization using this new hardware volume renderer on JPL's Prism System using a time-varying dataset from selected JPL applications.

Garbutt, Alexander E.↗

Young galaxy candidates in the Hubble Frontier Fields. III. MACS J0717.5+3745

In this paper we present the results of our search for and study of z≳6 galaxy candidates behind the third Frontier Fields (FFs) cluster, MACS J0717.5+3745, and its parallel field, combining data from Hubble and Spitzer. We select 39 candidates using the Lyman break technique, for which the clear non-detection in optical make the extreme mid-z interlopers hypothesis unlikely. We also take benefit from z≳6 samples selected using the previous FF data sets of Abell 2744 and MACS 0416 to improve the constraints on the properties of very high redshift objects. We compute the redshift and the physical properties such emission lines properties, star formation rate, reddening, and stellar mass for all FF objects from their spectral energy distribution using templates including nebular emission lines. We study the relationship between several physical properties and confirm the trend already observed in previous surveys for evolution of star formation rate with galaxy mass and between the size and the UV luminosity of our candidates. The analysis of the evolution of the UV luminosity function with redshift seems more compatible with an evolution of density. Moreover, no robust z⩾8.5 object is selected behind the cluster field and few z∼9 candidates have been selected in the two previous data sets from this legacy survey, suggesting a strong evolution in the number density of galaxies between z∼8 and 9. Thanks to the use of the lensing cluster, we study the evolution of the star formation rate density produced by galaxies with L > 0.03 L{sup ⋆}, and confirm the strong decrease observed between z∼8 and 9.

79 ASTRONOMY AND ASTROPHYSICS↗

Massively parallel phase-field simulations targeting exascale

The interface thickness in the phase-field (PF) method limits its simulation scales. Consequently, large-scale PF simulations become prohibitively expensive for resolving the extremely fine microstructures that typically form during rapid solidification processing. This challenge is significant in predicting microstructure evolution in metal additive manufacturing and has been identified by the United States Department of Energy’s Exascale Computing Project. Here, to address this, we develop a multi-GPU and MPI-based massively parallel simulation code, utilizing state-of-the-art algorithms, software, and libraries, for large-scale three-dimensional (3D) PF simulations. We report the first GPU-parallel PF simulations on Frontier (currently the second TOP500 exascale cluster) and Summit machines, taking dendritic growth as an example problem. We evaluate the parallel performance of our implementation using scaling studies with more than 24 000 GPUs (among the largest known computations to date) and the acceleration performance using large-scale simulations of dendritic growth in 3D. Finally, massively parallel GPUs in these supercomputers enabled the first coupled multiscale simulations of laser melting and subsequent dendritic solidification on the scale of a full melt-pool, demonstrating the feasibility of performing PF simulations with a point total over 2 billion grid points within an acceptable time.

Exascale↗

A Parallel Solver for Graph Laplacians

Problems from graph drawing, spectral clustering, network flow and graph partitioning can all be expressed in terms of graph Laplacian matrices. There are a variety of practical approaches to solving these problems in serial. However, as problem sizes increase and single core speeds stagnate, parallelism is essential to solve such problems quickly. We present an unsmoothed aggregation multigrid method for solving graph Laplacians in a distributed memory setting. We introduce new parallel aggregation and low degree elimination algorithms targeted specifically at irregular degree graphs. These algorithms are expressed in terms of sparse matrix-vector products using generalized sum and product operations. This formulation is amenable to linear algebra using arbitrary distributions and allows us to operate on a 2D sparse matrix distribution, which is necessary for parallel scalability. Our solver outperforms the natural parallel extension of the current state of the art in an algorithmic comparison. We demonstrate scalability to 576 processes and graphs with up to 1.7 billion edges.

Konolige, Tristan↗

Accelerating semantic graph databases on commodity clusters

We are developing a full software system for accelerating semantic graph databases on commodity cluster that scales to hundreds of nodes while maintaining constant query throughput. Our framework comprises a SPARQL to C++ compiler, a library of parallel graph methods and a custom multithreaded runtime layer, which provides a Partitioned Global Address Space (PGAS) programming model with fork/join parallelism and automatic load balancing over a commodity clusters. We present preliminary results for the compiler and for the runtime.

commodity clusters↗