SEARCH · Search NASA

Results for “parallel clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

All sources NASA NTRS DOE OSTI

DOE OSTI2019

Visualization of clustered protocadherin neuronal self-recognition complexes

Neurite self-recognition and avoidance are fundamental properties of all nervous systems. These processes facilitate dendritic arborization, prevent formation of autapses and allow free interaction among non-self neurons. Avoidance among self neurites is mediated by stochastic cell-surface expression of combinations of about 60 isoforms of α-, β- and γ-clustered protocadherin that provide mammalian neurons with single-cell identities. Avoidance is observed between neurons that express identical protocadherin repertoires, and single-isoform differences are sufficient to prevent self-recognition. Protocadherins form isoform-promiscuous cis dimers and isoform-specific homophilic trans dimers. Although these interactions have previously been characterized in isolation, structures of full-length protocadherin ectodomains have not been determined, and how these two interfaces engage in self-recognition between neuronal surfaces remains unknown. Here we determine the molecular arrangement of full-length clustered protocadherin ectodomains in single-isoform self-recognition complexes, using X-ray crystallography and cryo-electron tomography. We determine the crystal structure of the clustered protocadherin γB4 ectodomain, which reveals a zipper-like lattice that is formed by alternating cis and trans interactions. Using cryo-electron tomography, we show that clustered protocadherin γB6 ectodomains tethered to liposomes spontaneously assemble into linear arrays at membrane contact sites, in a configuration that is consistent with the assembly observed in the crystal structure. These linear assemblies pack against each other as parallel arrays to form larger two-dimensional structures between membranes. Furthermore, our results suggest that the formation of ordered linear assemblies by clustered protocadherins represents the initial self-recognition step in neuronal avoidance, and thus provide support for the isoform-mismatch chain-termination model of protocadherin-mediated self-recognition, which depends on these linear chains.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2018

HyperSpace: Distributed Bayesian Hyperparameter Optimization

As machine learning models continue to increase in complexity, so does the potential number of free model parameters commonly known as hyperparameters. While there has been considerable progress toward finding optimal configurations of these hyperparameters, many optimization procedures are treated as black boxes. We believe optimization methods should not only return a set of optimized hyperparameters, but also give insight into the effects of model hyperparameter settings. To this end, we present HyperSpace, a parallel implementation of Bayesian sequential model-based optimization. HyperSpace leverages high performance computing (HPC) resources to better understand unknown, potentially non-convex hyperparameter search spaces. We show that it is possible to learn the dependencies between model hyperparameters through the optimization process. By partitioning large search spaces and running many optimization procedures in parallel, we also show that it is possible to discover families of good hyperparameter settings over a variety of models including unsupervised clustering, regression, and classification tasks.

Computer Science↗

DOE OSTI2021

A Step Towards the Final Frontier: Lessons Learned from Acceptance Testing of the First HPE/Cray EX 3000 System at ORNL

In this paper, we summarize the deployment of the Air Force Weather (AFW) HPC11 system at Oak Ridge National Laboratory (ORNL) including the process followed to successfully complete acceptance testing of the system. HPC11 is the first HPE/Cray EX 3000 system that has been successfully released to its user community in a federal facility. HPC11 consists of two identical 800-node supercomputers, Fawbush and Miller, with access to two independent and identical Lustre parallel file systems. HPC11 is equipped with Slingshot 10 interconnect technology and relies on the HPE Performance Cluster Manager (HPCM) software for system configuration. ORNL has a clearly defined acceptance testing process used to ensure that every new system deployed can provide the necessary capabilities to support user workloads. We worked closely with HPE and AFW to develop a set of tests that used the United Kingdom’s Meteorological Office’s Unified Model (UM) and 4DVAR. We also included benchmarks and applications from the Oak Ridge Leadership Computing Facility (OLCF) portfolio to fully exercise the HPE/Cray programming environment and evaluate the functionality and performance of the system. Acceptance testing of HPC11 required parallel execution of each element on Fawbush and Miller. In addition, careful coordination was needed to ensure successful acceptance of the newly deployed Lustre file systems alongside the compute resources. In this work, we present test results from specific system components and provide an overview of the issues identified, challenges encountered, and the lessons learned along the way.

Melesse Vergara, Veronica↗

DOE OSTI2015

Ion-Stockmayer clusters: Minima, classical thermodynamics, and variational ground state estimates of Li{sup +}(CH{sub 3}NO{sub 2}){sub n} (n = 1–20)

Structural optimizations, classical NVT ensemble, and variational Monte Carlo simulations of ion Stockmayer clusters parameterized to approximate the Li{sup +}(CH{sub 3}NO{sub 2}){sub n} (n = 1–20) systems are performed. The Metropolis algorithm enhanced by the parallel tempering strategy is used to measure internal energies and heat capacities, and a parallel version of the genetic algorithm is employed to obtain the most important minima. The first solvation sheath is octahedral and this feature remains the dominant theme in the structure of clusters with n ≥ 6. The first “magic number” is identified using the adiabatic solvent dissociation energy, and it marks the completion of the second solvation layer for the lithium ion-nitromethane clusters. It corresponds to the n = 18 system, a solvated ion with the first sheath having octahedral symmetry, weakly bound to an eight-membered and a four-membered ring crowning a vertex of the octahedron. Variational Monte Carlo estimates of the adiabatic solvent dissociation energy reveal that quantum effects further enhance the stability of the n = 18 system relative to its neighbors.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

A step towards the final frontier: Lessons learned from acceptance testing of the first HPE/Cray EX 3000 system at ORNL

Summary In this article, we summarize the deployment of the Air Force Weather (AFW) HPC11 system at Oak Ridge National Laboratory (ORNL) including the process followed to successfully complete acceptance testing of the system. HPC11 is the first HPE/Cray EX 3000 system that has been successfully released to its user community in a federal facility. HPC11 consists of two identical 800‐node supercomputers, Fawbush and Miller, with access to two independent and identical lustre parallel file systems. HPC11 is equipped with Slingshot 10 interconnect technology and relies on the HPE Performance Cluster Manager software for system configuration. ORNL has a clearly defined acceptance testing process used to ensure that every new system deployed can provide the necessary capabilities to support user workloads. We worked closely with HPE and AFW to develop a set of tests that used the United Kingdom's Meteorological Office's Unified Model and 4‐dimensional variational data assimilation. We also included benchmarks and applications from the Oak Ridge Leadership Computing Facility portfolio to fully exercise the HPE/Cray programming environment and evaluate the functionality and performance of the system. Acceptance testing of HPC11 required parallel execution of each element on Fawbush and Miller. In addition, careful coordination was needed to ensure successful acceptance of the newly deployed lustre file systems alongside the compute resources. In this work, we present test results from specific system components and provide an overview of the issues identified, challenges encountered, and the lessons learned along the way.

Melesse Vergara, Verónica G.↗

DOE OSTI2012

Data-Parallel Halo Finder Operator in PISTON

PISTON is a portable framework which supports the development of visualization and analysis operators using a platform-independent, data-parallel programming model. Operators such as isosurface, cut-surface and threshold have been implemented in this framework, with the exact same operator code achieving good parallel performance on different architectures. An important analysis operator in cosmology is the halo finder. A halo is a cluster of particles and is considered a common feature of interest found in cosmology data. As the number of cosmological simulations carried out in the recent past has increased, the resultant data of these simulations and the required analysis tasks have increased as well. As a consequence, there is a need to develop scalable and efficient tools to carry out the needed analysis. Therefore, we are currently implementing a halo finder operator using PISTON. Researchers have developed a wide variety of techniques to identify halos in raw particle data. The most basic algorithm is the friend-of-friends (FOF) halo finder, where the particles are clustered based on two parameters: linking length and halo size. In a FOF halo finder, all particles which lie within the linking length are considered as one halo and the halos are filtered based on the halo size parameter. A naive implementation of a FOF halo finder compares each and every particle pair, requiring O(n{sup 2}) operations. Our data-parallel halo finder operator uses a balanced k-d tree to reduce this number of operations in the average case, and implements the algorithm using only the data-parallel primitives in order to achieve portability and performance.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2018

Investigation of the Inductive-Resonant Interaction of Water Clusters in Fluorapatite Minerals Using Polarization IR and Raman Spectroscopy

Polarization studies of the vibrational spectra of clusters (H{sub 2}O){sub n} in samples of single-crystal fluorapatite of different deposits were carried out by infrared (IR) spectroscopy and Raman spectroscopy. The IR and Raman spectra were measured for two orthogonal directions of the external excitation field: parallel and perpendicular to the c-channels in fluorapatite single crystals. Interpretation of the interaction of the external electromagnetic field with clusters of water situated in microvoids and microcracks of the crystal with the structural groups of F…HO of apatite is given in the framework of the dipole–dipole approximation. This interaction has an induction–resonance character (IR spectroscopy) caused by the transfer of vibrational excitation from the F…HO groups to (H{sub 2}O){sub n} clusters. In the case of Raman spectroscopy, the induction character of the excitation of (H{sub 2}O){sub n} clusters takes place due to the transfer of electron excitation from the F…HO groups.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2017

Building Halo Merger Trees from the Q Continuum Simulation

Cosmological N-body simulations rank among the most computationally intensive efforts today. A key challenge is the analysis of structure, substructure, and the merger history for many billions of compact particle clusters, called halos. Effectively representing the merging history of halos is essential for many galaxy formation models used to generate synthetic sky catalogs, an important application of modern cosmological simulations. Generating realistic mock catalogs requires computing the halo formation history from simulations with large volumes and billions of halos over many time steps, taking hundreds of terabytes of analysis data. We present fast parallel algorithms for producing halo merger trees and tracking halo substructure from a single-level, density-based clustering algorithm. Merger trees are created from analyzing the halo-particle membership function in adjacent snapshots, and substructure is identified by tracking the "cores" of merging halos - sets of particles near the halo center. Core tracking is performed after creating merger trees and uses the relationships found during tree construction to associate substructures with hosts. The algorithms are implemented with MPI and evaluated on a Cray XK7 supercomputer using up to 16,384 processes on data from HACC, a modern cosmological simulation framework. We present results for creating merger trees from 101 analysis snapshots taken from the Q Continuum, a large volume, high mass resolution, cosmological simulation evolving half a trillion particles.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS2008

Parallel Implementation of the Recursive Approximation of an Unsupervised Hierarchical Segmentation Algorithm

The hierarchical image segmentation algorithm (referred to as HSEG) is a hybrid of hierarchical step-wise optimization (HSWO) and constrained spectral clustering that produces a hierarchical set of image segmentations. HSWO is an iterative approach to region grooving segmentation in which the optimal image segmentation is found at N(sub R) regions, given a segmentation at N(sub R+1) regions. HSEG's addition of constrained spectral clustering makes it a computationally intensive algorithm, for all but, the smallest of images. To counteract this, a computationally efficient recursive approximation of HSEG (called RHSEG) has been devised. Further improvements in processing speed are obtained through a parallel implementation of RHSEG. This chapter describes this parallel implementation and demonstrates its computational efficiency on a Landsat Thematic Mapper test scene.

Tilton, James C.↗

NASA NTRS2000

Performance Assessment of OVERFLOW on Distributed Computing Environment

The aerodynamic computer code, OVERFLOW, with a multi-zone overset grid feature, has been parallelized to enhance its performance on distributed and shared memory paradigms. Practical application benchmarks have been set to assess the efficiency of code's parallelism on high-performance architectures. The code's performance has also been experimented with in the context of the distributed computing paradigm on distant computer resources using the Information Power Grid (IPG) toolkit, Globus. Two parallel versions of the code, namely OVERFLOW-MPI and -MLP, have developed around the natural coarse grained parallelism inherent in a multi-zonal domain decomposition paradigm. The algorithm invokes a strategy that forms a number of groups, each consisting of a zone, a cluster of zones and/or a partition of a large zone. Each group can be thought of as a process with one or multithreads assigned to it and that all groups run in parallel. The -MPI version of the code uses explicit message-passing based on the standard MPI library for sending and receiving interzonal boundary data across processors. The -MLP version employs no message-passing paradigm; the boundary data is transferred through the shared memory. The -MPI code is suited for both distributed and shared memory architectures, while the -MLP code can only be used on shared memory platforms. The IPG applications are implemented by the -MPI code using the Globus toolkit. While a computational task is distributed across multiple computer resources, the parallelism can be explored on each resource alone. Performance studies are achieved with some practical aerodynamic problems with complex geometries, consisting of 2.5 up to 33 million grid points and a large number of zonal blocks. The computations were executed primarily on SGI Origin 2000 multiprocessors and on the Cray T3E. OVERFLOW's IPG applications are carried out on NASA homogeneous metacomputing machines located at three sites, Ames, Langley and Glenn. Plans for the future will exploit the distributed parallel computing capability on various homogeneous and heterogeneous resources and large scale benchmarks. Alternative IPG toolkits will be used along with sophisticated zonal grouping strategies to minimize the communication time across the computer resources.

Djomehri, M. Jahed↗

DOE OSTI2015

Three complexes of Cu(I) cluster with flexible and rigid ligands: Synthesis, characterization and photoluminescent properties

Three new Cu(I) cluster complexes, viz. [(Cu{sub 4}I{sub 4})(Cu{sub 2}I{sub 2})(dimb){sub 3}]{sub n} (1; dimb=1,4-diimidazol-1-ylbutane), [(Cu{sub 3}I{sub 2})(dimb)(dmtz)]{sub n} (2; dmtz=3,5-dimethyl-1,2,4-triazole), and [Cu{sub 6}(mbt){sub 6}] (3; mbt=2-mercaptobenzothiazole), have been solvothermally synthesized and structurally characterized. In 1, a Cu{sub 4}I{sub 4} cubane core as a 4-connecting node, connects the neighboring nodes either through single dimb or μ{sub 2}-[(Cu{sub 2}I{sub 2})(dimb){sub 2}] linkers, affording an undulated 2D (4,4) net. Parallel interpenetration occurs between the adjacent nets and thus the overall 2D→3D network is formed. Complex 2 is constructed by 2D (4,4) topological plane grid layers of AB stacking. The core, a distorted triangular bipyramidal Cu{sub 3}I{sub 2} cluster, is acted as a 4-connecting node and connected with dimb and μ{sub 3}-dmtz to form the layer. Complex 3 contains a (Cu{sub 6}S{sub 6}) core in discrete paddle-wheel molecule, which serves as a 4-connecting node to link equivalent ones via π···π interaction, forming 2D (4,4) layers. Solid-state luminescence properties and thermogravimetric analyses of 1, 2 and 3 were investigated. - Graphical abstract: Complexes based on Cu(I) clusters possess two-fold 2D→3D parallel interpenetrating (1), triple emissions (2) and near-infrared emission (3). - Highlights: • Complex 1 represents two-fold 2D→3D parallel interpenetrating framework. • Complex 2 shows triple emissions. • Complex 3 displays a quite intense near-infrared (NIR) emission. • These complexes have good thermal stability.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2017

The Portals 4.1 Network Programming Interface

This report presents a specification for the Portals 4 networ k programming interface. Portals 4 is intended to allow scalable, high-performance network communication betwee n nodes of a parallel computing system. Portals 4 is well suited to massively parallel processing and embedded syste ms. Portals 4 represents an adaption of the data movement layer developed for massively parallel processing platfor ms, such as the 4500-node Intel TeraFLOPS machine. Sandia's Cplant cluster project motivated the development of Version 3.0, which was later extended to Version 3.3 as part of the Cray Red Storm machine and XT line. Version 4 is tar geted to the next generation of machines employing advanced network interface architectures that support enh anced offload capabilities.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2013

Profiling and Improving I/O Performance of a Large-Scale Climate Scientific Application

Exascale computing systems are soon to emerge, which will pose great challenges on the huge gap between computing and I/O performance. Many large-scale scientific applications play an important role in our daily life. The huge amounts of data generated by such applications require highly parallel and efficient I/O management policies. In this paper, we adopt a mission-critical scientific application, GEOS-5, as a case to profile and analyze the communication and I/O issues that are preventing applications from fully utilizing the underlying parallel storage systems. Through in-detail architectural and experimental characterization, we observe that current legacy I/O schemes incur significant network communication overheads and are unable to fully parallelize the data access, thus degrading applications' I/O performance and scalability. To address these inefficiencies, we redesign its I/O framework along with a set of parallel I/O techniques to achieve high scalability and performance. Evaluation results on the NASA discover cluster show that our optimization of GEOS-5 with ADIOS has led to significant performance improvements compared to the original GEOS-5 implementation.

Exascale↗

NASA NTRS1993

Fast parallel algorithms that compute transitive closure of a fuzzy relation

The notion of a transitive closure of a fuzzy relation is very useful for clustering in pattern recognition, for fuzzy databases, etc. The original algorithm proposed by L. Zadeh (1971) requires the computation time O(n(sup 4)), where n is the number of elements in the relation. In 1974, J. C. Dunn proposed a O(n(sup 2)) algorithm. Since we must compute n(n-1)/2 different values s(a, b) (a not equal to b) that represent the fuzzy relation, and we need at least one computational step to compute each of these values, we cannot compute all of them in less than O(n(sup 2)) steps. So, Dunn's algorithm is in this sense optimal. For small n, it is ok. However, for big n (e.g., for big databases), it is still a lot, so it would be desirable to decrease the computation time (this problem was formulated by J. Bezdek). Since this decrease cannot be done on a sequential computer, the only way to do it is to use a computer with several processors working in parallel. We show that on a parallel computer, transitive closure can be computed in time O((log(sub 2)(n))2).

Kreinovich, Vladik YA.↗

DOE OSTI2021

Electronic Structure Theory Calculations Using Modern Architectures: KNL vs Haswell

The time to solution and parallel efficiency of several commonly used electronic structure methods (Hartree–Fock, density functional theory, second order perturbation theory, resolution of the identity second order perturbation theory, coupled cluster) are evaluated on both the Intel Xeon Haswell and the Intel Xeon Phi Knights Landing (KNL) architectures. The Haswell completes the benchmark calculations with a faster time to solution than the KNL for all molecules and methods tested. While the Haswell exhibits an average speedup of at least 3.5 relative to the KNL for all nonthreaded computations, the KNL has a better parallel efficiency than the Haswell with increasing core counts. The architectures are further tested using a more computationally costly coupled cluster method on a transition state reaction. The Haswell appears to be the best choice to minimize the time to solution, though for very large systems and high levels of theory that require memory intensive processes the superior memory hierarchy and larger on node memory of the KNL can make it a better choice. These results are used to showcase aspects of novel architectures that will increase efficiency for quantum chemistry applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2015

Legacy extragalactic UV survey (LEGUS) with the Hubble space telescope. I. Survey description

The Legacy ExtraGalactic UV Survey (LEGUS) is a Cycle 21 Treasury program on the Hubble Space Telescope aimed at the investigation of star formation and its relation with galactic environment in nearby galaxies, from the scales of individual stars to those of ∼kiloparsec-size clustered structures. Five-band imaging from the near-ultraviolet to the I band with the Wide-Field Camera 3 (WFC3), plus parallel optical imaging with the Advanced Camera for Surveys (ACS), is being collected for selected pointings of 50 galaxies within the local 12 Mpc. The filters used for the observations with the WFC3 are F275W(λ2704 Å), F336W(λ3355 Å), F438W(λ4325 Å), F555W(λ5308 Å), and F814W(λ8024 Å); the parallel observations with the ACS use the filters F435W(λ4328 Å), F606W(λ5921 Å), and F814W(λ8057 Å). The multiband images are yielding accurate recent (≲50 Myr) star formation histories from resolved massive stars and the extinction-corrected ages and masses of star clusters and associations. The extensive inventories of massive stars and clustered systems will be used to investigate the spatial and temporal evolution of star formation within galaxies. This will, in turn, inform theories of galaxy evolution and improve the understanding of the physical underpinning of the gas–star formation relation and the nature of star formation at high redshift. This paper describes the survey, its goals and observational strategy, and the initial scientific results. Because LEGUS will provide a reference survey and a foundation for future observations with the James Webb Space Telescope and with ALMA, a large number of data products are planned for delivery to the community.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2015

Active Storage with Analytics Capabilities and I/O Runtime System for Petascale Systems

Computational scientists must understand results from experimental, observational and computational simulation generated data to gain insights and perform knowledge discovery. As systems approach the petascale range, problems that were unimaginable a few years ago are within reach. With the increasing volume and complexity of data produced by ultra-scale simulations and high-throughput experiments, understanding the science is largely hampered by the lack of comprehensive I/O, storage, acceleration of data manipulation, analysis, and mining tools. Scientists require techniques, tools and infrastructure to facilitate better understanding of their data, in particular the ability to effectively perform complex data analysis, statistical analysis and knowledge discovery. The goal of this work is to enable more effective analysis of scientific datasets through the integration of enhancements in the I/O stack, from active storage support at the file system layer to MPI-IO and high-level I/O library layers. We propose to provide software components to accelerate data analytics, mining, I/O, and knowledge discovery for large-scale scientific applications, thereby increasing productivity of both scientists and the systems. Our approaches include 1) design the interfaces in high-level I/O libraries, such as parallel netCDF, for applications to activate data mining operations at the lower I/O layers; 2) Enhance MPI-IO runtime systems to incorporate the functionality developed as a part of the runtime system design; 3) Develop parallel data mining programs as part of runtime library for server-side file system in PVFS file system; and 4) Prototype an active storage cluster, which will utilize multicore CPUs, GPUs, and FPGAs to carry out the data mining workload.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Fast shared-memory streaming multilevel graph partitioning

In this report we show that a fast parallel graph partitioner can benefit many applications by reducing data transfers. The online methods for partitioning graphs have to be fast and they often rely on simple one-pass streaming algorithms, while the offline methods for partitioning graphs contain more involved algorithms and the most successful methods in this category belong to the multilevel approaches. In this work, we assess the feasibility of using streaming graph partitioning algorithms within the multilevel framework. Our end goal is to come up with a fast parallel offline multilevel partitioner that can produce competitive cutsize quality. We rely on a simple but fast and flexible streaming algorithm throughout the entire multilevel framework. This streaming algorithm serves multiple purposes in the partitioning process: a clustering algorithm in the coarsening, an effective algorithm for the initial partitioning, and a fast refinement algorithm in the uncoarsening. Its simple nature also lends itself easily for parallelization. The experiments on various graphs show that our approach is on the average up to 5.1x faster than the multi-threaded MeTiS, which comes at the expense of only 2x worse cutsize.

97 MATHEMATICS AND COMPUTING↗