SEARCH · Search NASA

Results for “parallel clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

All sources NASA NTRS DOE OSTI

DOE OSTI2022

The Portals 4.3 Network Programming Interface

This report presents a specification for the Portals 4 network programming interface. Portals 4 is intended to allow scalable, high-performance network communication between nodes of a parallel computing system. Portals 4 is well suited to massively parallel processing and embedded systems. Portals 4 represents an adaption of the data movement layer developed for massively parallel processing platforms, such as the 4500-node Intel TeraFLOPS machine. Sandia's Cplant cluster project motivated the development of Version 3.0, which was later extended to Version 3.3 as part of the Cray Red Storm machine and XT line. Version 4 is targeted to the next generation of machines employing advanced network interface architectures that support enhanced offload capabilities.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Matrix Factorizations at Scale: a Comparison of Scientific Data Analytics in Spark and C+MPI Using Three Case Studies

We explore the trade-offs of performing linear algebra using Apache Spark, compared to traditional C and MPI implementations on HPC platforms. Spark is designed for data analytics on cluster computing platforms with access to local disks and is optimized for data-parallel tasks. We examine three widely-used and important matrix factorizations: NMF (for physical plausibility), PCA (for its ubiquity) and CX (for data interpretability). We apply these methods to 1.6TB particle physics, 2.2TB and 16TB climate modeling and 1.1TB bioimaging data. The data matrices are tall-and-skinny which enable the algorithms to map conveniently into Spark’s data parallel model. We perform scaling experiments on up to 1600 Cray XC40 nodes, describe the sources of slowdowns, and provide tuning guidance to obtain high performance.

Gittens, Alex↗

DOE OSTI2015

A spectral scheme for Kohn–Sham density functional theory of clusters

Starting from the observation that one of the most successful methods for solving the Kohn–Sham equations for periodic systems – the plane-wave method – is a spectral method based on eigenfunction expansion, we formulate a spectral method designed towards solving the Kohn–Sham equations for clusters. This allows for efficient calculation of the electronic structure of clusters (and molecules) with high accuracy and systematic convergence properties without the need for any artificial periodicity. The basis functions in this method form a complete orthonormal set and are expressible in terms of spherical harmonics and spherical Bessel functions. Computation of the occupied eigenstates of the discretized Kohn–Sham Hamiltonian is carried out using a combination of preconditioned block eigensolvers and Chebyshev polynomial filter accelerated subspace iterations. Several algorithmic and computational aspects of the method, including computation of the electrostatics terms and parallelization are discussed. We have implemented these methods and algorithms into an efficient and reliable package called ClusterES (Cluster Electronic Structure). A variety of benchmark calculations employing local and non-local pseudopotentials are carried out using our package and the results are compared to the literature. Convergence properties of the basis set are discussed through numerical examples. Computations involving large systems that contain thousands of electrons are demonstrated to highlight the efficacy of our methodology. The use of our method to study clusters with arbitrary point group symmetries is briefly discussed.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2019

Visualization of clustered protocadherin neuronal self-recognition complexes

Neurite self-recognition and avoidance are fundamental properties of all nervous systems. These processes facilitate dendritic arborization, prevent formation of autapses and allow free interaction among non-self neurons. Avoidance among self neurites is mediated by stochastic cell-surface expression of combinations of about 60 isoforms of α-, β- and γ-clustered protocadherin that provide mammalian neurons with single-cell identities. Avoidance is observed between neurons that express identical protocadherin repertoires, and single-isoform differences are sufficient to prevent self-recognition. Protocadherins form isoform-promiscuous cis dimers and isoform-specific homophilic trans dimers. Although these interactions have previously been characterized in isolation, structures of full-length protocadherin ectodomains have not been determined, and how these two interfaces engage in self-recognition between neuronal surfaces remains unknown. Here we determine the molecular arrangement of full-length clustered protocadherin ectodomains in single-isoform self-recognition complexes, using X-ray crystallography and cryo-electron tomography. We determine the crystal structure of the clustered protocadherin γB4 ectodomain, which reveals a zipper-like lattice that is formed by alternating cis and trans interactions. Using cryo-electron tomography, we show that clustered protocadherin γB6 ectodomains tethered to liposomes spontaneously assemble into linear arrays at membrane contact sites, in a configuration that is consistent with the assembly observed in the crystal structure. These linear assemblies pack against each other as parallel arrays to form larger two-dimensional structures between membranes. Furthermore, our results suggest that the formation of ordered linear assemblies by clustered protocadherins represents the initial self-recognition step in neuronal avoidance, and thus provide support for the isoform-mismatch chain-termination model of protocadherin-mediated self-recognition, which depends on these linear chains.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2018

HyperSpace: Distributed Bayesian Hyperparameter Optimization

As machine learning models continue to increase in complexity, so does the potential number of free model parameters commonly known as hyperparameters. While there has been considerable progress toward finding optimal configurations of these hyperparameters, many optimization procedures are treated as black boxes. We believe optimization methods should not only return a set of optimized hyperparameters, but also give insight into the effects of model hyperparameter settings. To this end, we present HyperSpace, a parallel implementation of Bayesian sequential model-based optimization. HyperSpace leverages high performance computing (HPC) resources to better understand unknown, potentially non-convex hyperparameter search spaces. We show that it is possible to learn the dependencies between model hyperparameters through the optimization process. By partitioning large search spaces and running many optimization procedures in parallel, we also show that it is possible to discover families of good hyperparameter settings over a variety of models including unsupervised clustering, regression, and classification tasks.

Computer Science↗

DOE OSTI2021

A Step Towards the Final Frontier: Lessons Learned from Acceptance Testing of the First HPE/Cray EX 3000 System at ORNL

In this paper, we summarize the deployment of the Air Force Weather (AFW) HPC11 system at Oak Ridge National Laboratory (ORNL) including the process followed to successfully complete acceptance testing of the system. HPC11 is the first HPE/Cray EX 3000 system that has been successfully released to its user community in a federal facility. HPC11 consists of two identical 800-node supercomputers, Fawbush and Miller, with access to two independent and identical Lustre parallel file systems. HPC11 is equipped with Slingshot 10 interconnect technology and relies on the HPE Performance Cluster Manager (HPCM) software for system configuration. ORNL has a clearly defined acceptance testing process used to ensure that every new system deployed can provide the necessary capabilities to support user workloads. We worked closely with HPE and AFW to develop a set of tests that used the United Kingdom’s Meteorological Office’s Unified Model (UM) and 4DVAR. We also included benchmarks and applications from the Oak Ridge Leadership Computing Facility (OLCF) portfolio to fully exercise the HPE/Cray programming environment and evaluate the functionality and performance of the system. Acceptance testing of HPC11 required parallel execution of each element on Fawbush and Miller. In addition, careful coordination was needed to ensure successful acceptance of the newly deployed Lustre file systems alongside the compute resources. In this work, we present test results from specific system components and provide an overview of the issues identified, challenges encountered, and the lessons learned along the way.

Melesse Vergara, Veronica↗

DOE OSTI2015

Ion-Stockmayer clusters: Minima, classical thermodynamics, and variational ground state estimates of Li{sup +}(CH{sub 3}NO{sub 2}){sub n} (n = 1–20)

Structural optimizations, classical NVT ensemble, and variational Monte Carlo simulations of ion Stockmayer clusters parameterized to approximate the Li{sup +}(CH{sub 3}NO{sub 2}){sub n} (n = 1–20) systems are performed. The Metropolis algorithm enhanced by the parallel tempering strategy is used to measure internal energies and heat capacities, and a parallel version of the genetic algorithm is employed to obtain the most important minima. The first solvation sheath is octahedral and this feature remains the dominant theme in the structure of clusters with n ≥ 6. The first “magic number” is identified using the adiabatic solvent dissociation energy, and it marks the completion of the second solvation layer for the lithium ion-nitromethane clusters. It corresponds to the n = 18 system, a solvated ion with the first sheath having octahedral symmetry, weakly bound to an eight-membered and a four-membered ring crowning a vertex of the octahedron. Variational Monte Carlo estimates of the adiabatic solvent dissociation energy reveal that quantum effects further enhance the stability of the n = 18 system relative to its neighbors.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

A step towards the final frontier: Lessons learned from acceptance testing of the first HPE/Cray EX 3000 system at ORNL

Summary In this article, we summarize the deployment of the Air Force Weather (AFW) HPC11 system at Oak Ridge National Laboratory (ORNL) including the process followed to successfully complete acceptance testing of the system. HPC11 is the first HPE/Cray EX 3000 system that has been successfully released to its user community in a federal facility. HPC11 consists of two identical 800‐node supercomputers, Fawbush and Miller, with access to two independent and identical lustre parallel file systems. HPC11 is equipped with Slingshot 10 interconnect technology and relies on the HPE Performance Cluster Manager software for system configuration. ORNL has a clearly defined acceptance testing process used to ensure that every new system deployed can provide the necessary capabilities to support user workloads. We worked closely with HPE and AFW to develop a set of tests that used the United Kingdom's Meteorological Office's Unified Model and 4‐dimensional variational data assimilation. We also included benchmarks and applications from the Oak Ridge Leadership Computing Facility portfolio to fully exercise the HPE/Cray programming environment and evaluate the functionality and performance of the system. Acceptance testing of HPC11 required parallel execution of each element on Fawbush and Miller. In addition, careful coordination was needed to ensure successful acceptance of the newly deployed lustre file systems alongside the compute resources. In this work, we present test results from specific system components and provide an overview of the issues identified, challenges encountered, and the lessons learned along the way.

Melesse Vergara, Verónica G.↗

DOE OSTI2012

Data-Parallel Halo Finder Operator in PISTON

PISTON is a portable framework which supports the development of visualization and analysis operators using a platform-independent, data-parallel programming model. Operators such as isosurface, cut-surface and threshold have been implemented in this framework, with the exact same operator code achieving good parallel performance on different architectures. An important analysis operator in cosmology is the halo finder. A halo is a cluster of particles and is considered a common feature of interest found in cosmology data. As the number of cosmological simulations carried out in the recent past has increased, the resultant data of these simulations and the required analysis tasks have increased as well. As a consequence, there is a need to develop scalable and efficient tools to carry out the needed analysis. Therefore, we are currently implementing a halo finder operator using PISTON. Researchers have developed a wide variety of techniques to identify halos in raw particle data. The most basic algorithm is the friend-of-friends (FOF) halo finder, where the particles are clustered based on two parameters: linking length and halo size. In a FOF halo finder, all particles which lie within the linking length are considered as one halo and the halos are filtered based on the halo size parameter. A naive implementation of a FOF halo finder compares each and every particle pair, requiring O(n{sup 2}) operations. Our data-parallel halo finder operator uses a balanced k-d tree to reduce this number of operations in the average case, and implements the algorithm using only the data-parallel primitives in order to achieve portability and performance.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2018

Investigation of the Inductive-Resonant Interaction of Water Clusters in Fluorapatite Minerals Using Polarization IR and Raman Spectroscopy

Polarization studies of the vibrational spectra of clusters (H{sub 2}O){sub n} in samples of single-crystal fluorapatite of different deposits were carried out by infrared (IR) spectroscopy and Raman spectroscopy. The IR and Raman spectra were measured for two orthogonal directions of the external excitation field: parallel and perpendicular to the c-channels in fluorapatite single crystals. Interpretation of the interaction of the external electromagnetic field with clusters of water situated in microvoids and microcracks of the crystal with the structural groups of F…HO of apatite is given in the framework of the dipole–dipole approximation. This interaction has an induction–resonance character (IR spectroscopy) caused by the transfer of vibrational excitation from the F…HO groups to (H{sub 2}O){sub n} clusters. In the case of Raman spectroscopy, the induction character of the excitation of (H{sub 2}O){sub n} clusters takes place due to the transfer of electron excitation from the F…HO groups.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2017

Building Halo Merger Trees from the Q Continuum Simulation

Cosmological N-body simulations rank among the most computationally intensive efforts today. A key challenge is the analysis of structure, substructure, and the merger history for many billions of compact particle clusters, called halos. Effectively representing the merging history of halos is essential for many galaxy formation models used to generate synthetic sky catalogs, an important application of modern cosmological simulations. Generating realistic mock catalogs requires computing the halo formation history from simulations with large volumes and billions of halos over many time steps, taking hundreds of terabytes of analysis data. We present fast parallel algorithms for producing halo merger trees and tracking halo substructure from a single-level, density-based clustering algorithm. Merger trees are created from analyzing the halo-particle membership function in adjacent snapshots, and substructure is identified by tracking the "cores" of merging halos - sets of particles near the halo center. Core tracking is performed after creating merger trees and uses the relationships found during tree construction to associate substructures with hosts. The algorithms are implemented with MPI and evaluated on a Cray XK7 supercomputer using up to 16,384 processes on data from HACC, a modern cosmological simulation framework. We present results for creating merger trees from 101 analysis snapshots taken from the Q Continuum, a large volume, high mass resolution, cosmological simulation evolving half a trillion particles.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2015

Three complexes of Cu(I) cluster with flexible and rigid ligands: Synthesis, characterization and photoluminescent properties

Three new Cu(I) cluster complexes, viz. [(Cu{sub 4}I{sub 4})(Cu{sub 2}I{sub 2})(dimb){sub 3}]{sub n} (1; dimb=1,4-diimidazol-1-ylbutane), [(Cu{sub 3}I{sub 2})(dimb)(dmtz)]{sub n} (2; dmtz=3,5-dimethyl-1,2,4-triazole), and [Cu{sub 6}(mbt){sub 6}] (3; mbt=2-mercaptobenzothiazole), have been solvothermally synthesized and structurally characterized. In 1, a Cu{sub 4}I{sub 4} cubane core as a 4-connecting node, connects the neighboring nodes either through single dimb or μ{sub 2}-[(Cu{sub 2}I{sub 2})(dimb){sub 2}] linkers, affording an undulated 2D (4,4) net. Parallel interpenetration occurs between the adjacent nets and thus the overall 2D→3D network is formed. Complex 2 is constructed by 2D (4,4) topological plane grid layers of AB stacking. The core, a distorted triangular bipyramidal Cu{sub 3}I{sub 2} cluster, is acted as a 4-connecting node and connected with dimb and μ{sub 3}-dmtz to form the layer. Complex 3 contains a (Cu{sub 6}S{sub 6}) core in discrete paddle-wheel molecule, which serves as a 4-connecting node to link equivalent ones via π···π interaction, forming 2D (4,4) layers. Solid-state luminescence properties and thermogravimetric analyses of 1, 2 and 3 were investigated. - Graphical abstract: Complexes based on Cu(I) clusters possess two-fold 2D→3D parallel interpenetrating (1), triple emissions (2) and near-infrared emission (3). - Highlights: • Complex 1 represents two-fold 2D→3D parallel interpenetrating framework. • Complex 2 shows triple emissions. • Complex 3 displays a quite intense near-infrared (NIR) emission. • These complexes have good thermal stability.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2017

The Portals 4.1 Network Programming Interface

This report presents a specification for the Portals 4 networ k programming interface. Portals 4 is intended to allow scalable, high-performance network communication betwee n nodes of a parallel computing system. Portals 4 is well suited to massively parallel processing and embedded syste ms. Portals 4 represents an adaption of the data movement layer developed for massively parallel processing platfor ms, such as the 4500-node Intel TeraFLOPS machine. Sandia's Cplant cluster project motivated the development of Version 3.0, which was later extended to Version 3.3 as part of the Cray Red Storm machine and XT line. Version 4 is tar geted to the next generation of machines employing advanced network interface architectures that support enh anced offload capabilities.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Electronic Structure Theory Calculations Using Modern Architectures: KNL vs Haswell

The time to solution and parallel efficiency of several commonly used electronic structure methods (Hartree–Fock, density functional theory, second order perturbation theory, resolution of the identity second order perturbation theory, coupled cluster) are evaluated on both the Intel Xeon Haswell and the Intel Xeon Phi Knights Landing (KNL) architectures. The Haswell completes the benchmark calculations with a faster time to solution than the KNL for all molecules and methods tested. While the Haswell exhibits an average speedup of at least 3.5 relative to the KNL for all nonthreaded computations, the KNL has a better parallel efficiency than the Haswell with increasing core counts. The architectures are further tested using a more computationally costly coupled cluster method on a transition state reaction. The Haswell appears to be the best choice to minimize the time to solution, though for very large systems and high levels of theory that require memory intensive processes the superior memory hierarchy and larger on node memory of the KNL can make it a better choice. These results are used to showcase aspects of novel architectures that will increase efficiency for quantum chemistry applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2015

Legacy extragalactic UV survey (LEGUS) with the Hubble space telescope. I. Survey description

The Legacy ExtraGalactic UV Survey (LEGUS) is a Cycle 21 Treasury program on the Hubble Space Telescope aimed at the investigation of star formation and its relation with galactic environment in nearby galaxies, from the scales of individual stars to those of ∼kiloparsec-size clustered structures. Five-band imaging from the near-ultraviolet to the I band with the Wide-Field Camera 3 (WFC3), plus parallel optical imaging with the Advanced Camera for Surveys (ACS), is being collected for selected pointings of 50 galaxies within the local 12 Mpc. The filters used for the observations with the WFC3 are F275W(λ2704 Å), F336W(λ3355 Å), F438W(λ4325 Å), F555W(λ5308 Å), and F814W(λ8024 Å); the parallel observations with the ACS use the filters F435W(λ4328 Å), F606W(λ5921 Å), and F814W(λ8057 Å). The multiband images are yielding accurate recent (≲50 Myr) star formation histories from resolved massive stars and the extinction-corrected ages and masses of star clusters and associations. The extensive inventories of massive stars and clustered systems will be used to investigate the spatial and temporal evolution of star formation within galaxies. This will, in turn, inform theories of galaxy evolution and improve the understanding of the physical underpinning of the gas–star formation relation and the nature of star formation at high redshift. This paper describes the survey, its goals and observational strategy, and the initial scientific results. Because LEGUS will provide a reference survey and a foundation for future observations with the James Webb Space Telescope and with ALMA, a large number of data products are planned for delivery to the community.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2015

Active Storage with Analytics Capabilities and I/O Runtime System for Petascale Systems

Computational scientists must understand results from experimental, observational and computational simulation generated data to gain insights and perform knowledge discovery. As systems approach the petascale range, problems that were unimaginable a few years ago are within reach. With the increasing volume and complexity of data produced by ultra-scale simulations and high-throughput experiments, understanding the science is largely hampered by the lack of comprehensive I/O, storage, acceleration of data manipulation, analysis, and mining tools. Scientists require techniques, tools and infrastructure to facilitate better understanding of their data, in particular the ability to effectively perform complex data analysis, statistical analysis and knowledge discovery. The goal of this work is to enable more effective analysis of scientific datasets through the integration of enhancements in the I/O stack, from active storage support at the file system layer to MPI-IO and high-level I/O library layers. We propose to provide software components to accelerate data analytics, mining, I/O, and knowledge discovery for large-scale scientific applications, thereby increasing productivity of both scientists and the systems. Our approaches include 1) design the interfaces in high-level I/O libraries, such as parallel netCDF, for applications to activate data mining operations at the lower I/O layers; 2) Enhance MPI-IO runtime systems to incorporate the functionality developed as a part of the runtime system design; 3) Develop parallel data mining programs as part of runtime library for server-side file system in PVFS file system; and 4) Prototype an active storage cluster, which will utilize multicore CPUs, GPUs, and FPGAs to carry out the data mining workload.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Fast shared-memory streaming multilevel graph partitioning

In this report we show that a fast parallel graph partitioner can benefit many applications by reducing data transfers. The online methods for partitioning graphs have to be fast and they often rely on simple one-pass streaming algorithms, while the offline methods for partitioning graphs contain more involved algorithms and the most successful methods in this category belong to the multilevel approaches. In this work, we assess the feasibility of using streaming graph partitioning algorithms within the multilevel framework. Our end goal is to come up with a fast parallel offline multilevel partitioner that can produce competitive cutsize quality. We rely on a simple but fast and flexible streaming algorithm throughout the entire multilevel framework. This streaming algorithm serves multiple purposes in the partitioning process: a clustering algorithm in the coarsening, an effective algorithm for the initial partitioning, and a fast refinement algorithm in the uncoarsening. Its simple nature also lends itself easily for parallelization. The experiments on various graphs show that our approach is on the average up to 5.1x faster than the multi-threaded MeTiS, which comes at the expense of only 2x worse cutsize.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Structures of 38-atom gold-platinum nanoalloy clusters

Bimetallic nanoclusters, such as gold-platinum nanoclusters, are nanomaterials promising wide range of applications. We perform a numerical study of 38-atom gold-platinum nanoalloy clusters, Au{sub n}Pt{sub 38−n} (0 ≤ n ≤ 38), to elucidate the geometrical structures of these clusters. The lowest-energy structures of these bimetallic nanoclusters at the semi-empirical level are obtained via a global-minimum search algorithm known as parallel tempering multi-canonical basin hopping plus genetic algorithm (PTMBHGA), in which empirical Gupta many-body potential is used to describe the inter-atomic interactions among the constituent atoms. The structures of gold-platinum nanoalloy clusters are predicted to be core-shell segregated nanoclusters. Gold atoms are observed to preferentially occupy the surface of the clusters, while platinum atoms tend to occupy the core due to the slightly smaller atomic radius of platinum as compared to gold’s. The evolution of the geometrical structure of 38-atom Au-Pt clusters displays striking similarity with that of 38-atom Au-Cu nanoalloy clusters as reported in the literature.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗