Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Energy-efficient mapping of large-scale workflows under deadline constraints in big data computing systems

Large-scale workflows for big data analytics have become a main consumer of energy in data centers where moldable parallel computing models such as MapReduce are widely applied to meet high computational demands with time-varying computing resources. The granularity of task partitioning in each moldable job of such big data workflows has a significant impact on energy efficiency, which remains largely unexplored. In this paper, we analyze the properties of moldable jobs and formulate a workflow mapping problem to minimize the dynamic energy consumption of a given workflow request under a deadline constraint in big data systems. Since this problem is strongly NP-hard, we design a fully polynomial-time approximation scheme (FPTAS) for a special case with a pipeline-structured workflow on a homogeneous cluster and a heuristic for the generalized problem with an arbitrary workflow on a heterogeneous cluster. The performance superiority of the proposed solution in terms of dynamic energy saving and deadline missing rate is illustrated by extensive simulation results in comparison with existing algorithms, and further validated by real-life workflow implementation and experimental results in Hadoop/YARN systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Energy-Screened Many-Body Expansion: A Practical Yet Accurate Fragmentation Method for Quantum Chemistry

We introduce an implementation of the truncated many-body expansion, MBE(n), in which the n-body corrections are screened using the effective fragment potential force field, and only those that exceed a specified energy threshold are computed at a quantum-mechanical level of theory. This energy-screened MBE(n) approach is tested at the n = 3 level for a sequence of water clusters, (H2O) N=6–34 . A threshold of 0.25 kJ/mol eliminates more than 80% of the subsystem electronic structure calculations and is even more efficacious in that respect than is distance-based screening. Even so, the energy-screened MBE(3) method is faithful to a full-system quantum chemistry calculation to within 1–2 kJ/mol/monomer, even in good quality basis sets such as aug-cc-pVTZ. These errors can be reduced by means of a two-layer approach that involves a Hartree–Fock calculation for the entire cluster. Such a correction proves to be necessary in order to obtain accurate relative energies for conformational isomers of (H2O)20, but the cost of a full-system Hartree–Fock calculation remains smaller than the cost of three-body subsystem calculations at correlated levels of theory. At the level of second-order Møller–Plesset perturbation theory (MP2), a screened MBE(3) calculation plus a full-system Hartree–Fock calculation is less expensive than a full-system MP2 calculation starting at N = 12 water molecules. This is true even if all MBE(3) subsystem calculations are performed on a single 40-core compute node, i.e., without significant parallelization. Energy-screened MBE(n) thus provides a fragment-based method that is accurate, stable in large basis sets, and low in cost, even when the latter is measured in aggregate computer time.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2023

Membrane-mediated dimerization of spherocylindrical nanoparticles

Here we present a numerical investigation of the modes of adhesion and endocytosis of two spherocylindrical nanoparticles (SCNPs) on planar and tensionless lipid membranes, using systematic molecular dynamics simulations of an implicit-solvent model, with varying values of the SCNPs' adhesion strength and dimensions. We found that at weak values of the adhesion energy per unit of area, ξ, the SCNPs are monomeric and adhere to the membrane in the parallel mode. As ξ is slightly increased, the SCNPs dimerize into wedged dimers, with an obtuse angle between their major axes that decreases with increasing ξ. However, as ξ is further increased, we found that the final adhesion state of the two SCNPs is strongly affected by the initial distance, d 0 , between their centers of mass, upon their adhesion. Namely, the SCNPs dimerize into wedged dimers, with an acute angle between their major axes, if d 0 is relatively small. However, for relatively high d 0 , they adhere individually to the membrane in the monomeric normal mode. For even higher values of ξ and small values of d 0 , the SCNPs cluster into tubular dimers. However, they remain monomeric if d 0 is high. Finally, the SCNPs endocytose either as a tubular dimer, if d 0 is low or as monomers for large d 0 , with the onset value of ξ of dimeric endocytosis being lower than that of monomeric endocytosis. Dimeric endocytosis requires that the SCNPs adhere simultaneously at nearby locations.

36 MATERIALS SCIENCE↗
DOE OSTI2011

Implementing High Performance Remote Method Invocation in CCA

We report our effort in engineering a high performance remote method invocation (RMI) mechanism for the Common Component Architecture (CCA). This mechanism provides a highly efficient and easy-to-use mechanism for distributed computing in CCA, enabling CCA applications to effectively leverage parallel systems to accelerate computations. This work is built on the previous work of Babel RMI. Babel is a high performance language interoperability tool that is used in CCA for scientific application writers to share, reuse, and compose applications from software components written in different programming languages. Babel provides a transparent and flexible RMI framework for distributed computing. However, the existing Babel RMI implementation is built on top of TCP and does not provide the level of performance required to distribute fine-grained tasks. We observed that the main reason the TCP based RMI does not perform well is because it does not utilize the high performance interconnect hardware on a cluster efficiently. We have implemented a high performance RMI protocol, HPCRMI. HPCRMI achieves low latency by building on top of a low-level portable communication library, Aggregated Remote Message Copy Interface (ARMCI), and minimizing communication for each RMI call. Our design allows a RMI operation to be completed by only two RDMA operations. We also aggressively optimize our system to reduce copying. In this paper, we discuss the design and our experimental evaluation of this protocol. Our experimental results show that our protocol can improve RMI performance by an order of magnitude.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2018

Distributed Louvain Algorithm for Graph Community Detection

In most real-world networks, the nodes/vertices tend to be organized into tightly-knit modules known as communities or clusters, such that nodes within a community are more likely to be “related” to one another than they are to the rest of the network. The goodness of partitioning into communities is typically measured using a well known measure called modularity. However, modularity optimization is an NP-complete problem. In 2008, Blondel, et al. introduced a multi-phase, iterative heuristic for modularity optimization, called the Louvain method. Owing to its speed and ability to yield high quality communities, the Louvain method continues to be one of the most widely used tools for serial community detection. In this paper, we present the design of a distributed memory implementation of the Louvain algorithm for parallel community detection. Our approach begins with an arbitrarily partitioned distributed graph input, and employs several heuristics to speedup the computation of the different steps of the Louvain algorithm. We evaluate our implementation and its different variants using real-world networks from various application domains (including internet, biology, social networks). Our MPI+OpenMP implementation yields about 7x speedup (on 4K processes) for soc-friendster network (1.8B edges) over a state-of-the-art shared-memory multicore implementation (on 64 threads), without compromising output quality. Furthermore, our distributed implementation was able to process a larger graph (uk-2007; 3.3B edges) in 32 seconds on 1K cores (64 nodes) of NERSC Cori, when the state-of-the-art shared-memory implementation failed to run due to insufficient memory on a single Cori node containing 128 GB.

High performance computing, graph algorithms, Dist↗
DOE OSTI2021

The inheritance of anthracnose (Colletotrichum sublineola) resistance in sorghum differential lines QL3 and IS18760

Anthracnose caused by the fungal pathogen C. sublineola is an economically important constraint on worldwide sorghum production. The most effective strategy to safeguard yield is through the introgression of resistance alleles. This requires elucidation of the genetic basis of the different resistance sources that have been identified. In this study, 223 recombinant inbred lines (RILs) derived from crossing anthracnose-differentials QL3 (96 RILs) and IS18760 (127 RILs) with the common susceptible parent PI609251 were evaluated at four field locations in the United States (Florida, Georgia, Texas, and Puerto Rico) for their anthracnose resistance response. Both RIL populations were highly susceptible to anthracnose in Florida and Georgia, while in Puerto Rico and Texas they were segregating for anthracnose resistance response. A genome scan using a composite linkage map of 982 single nucleotide polymorphisms (SNPs) detected two genomic regions of 4.31 and 0.85 Mb on chromosomes 4 and 8, respectively, that explained 10–27% of the phenotypic variation in Texas and Puerto Rico. In parallel, a subset of 43 RILs that contained 67% of the recombination events were evaluated against anthracnose pathotypes from Arkansas (2), Puerto Rico (2) and Texas (4) in the greenhouse. A genome scan showed that the 7.57 Mb region at the distal end of the short arm of chromosome 5 is associated with the resistance response against the pathotype AMP-048 from Arkansas. Comparative analysis identified the genomic region on chromosome 4 overlaps with an anthracnose resistance locus identified in another anthracnose-differential line, SC414-12E, indicating this genomic region is of interest for introgression in susceptible sorghum germplasm. Candidate gene analysis for the resistance locus on chromosome 5 identified an R -gene cluster that has high similarity to another R -gene cluster associated with anthracnose resistance on chromosome 9.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Wide-angle effects for peculiar velocities

ABSTRACT The line-of-sight peculiar velocities of galaxies contribute to their observed redshifts, breaking the translational invariance of galaxy clustering down to a rotational invariance around the observer. This becomes important when the line-of-sight direction varies significantly across a survey, leading to what are known as ‘wide-angle’ effects in redshift-space distortions. Wide-angle effects will also be present in measurements of the momentum field, i.e. the galaxy density-weighted velocity field, in upcoming peculiar velocity surveys. In this work, we study how wide-angle effects modify the predicted correlation function and power spectrum for momentum statistics, both in autocorrelation and in cross-correlation with the density field. Using both linear theory and the Zel'dovich approximation, we find that deviations from the plane-parallel limit are large and could become important in data analysis for low-redshift surveys. We point out that even multipoles in the cross-correlation between density and momentum are non-zero regardless of the choice of line of sight, and therefore contain new cosmological information that could be exploited. We discuss configuration space, Fourier space, and spherical analyses; providing exact expressions in each case rather than relying on an expansion in small angles. We hope these expressions will be of use in the analysis of upcoming surveys for redshift-space distortions and peculiar velocities.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

Two-photon absorption cross sections within equation-of-motion coupled-cluster formalism using resolution-of-the-identity and Cholesky decomposition representations: Theory, implementation, and benchmarks

The equation-of-motion coupled-cluster (EOM-CC) methods provide a robust description of electronically excited states and their properties. Here, we present a formalism for two-photon absorption (2PA) cross sections for the equation-of-motion for excitation energies CC with single and double substitutions (EOM-CC for electronically excited states with single and double substitutions) wave functions. Rather than the response theory formulation, we employ the expectation-value approach which is commonly used within EOM-CC, configuration interaction, and algebraic diagrammatic construction frameworks. In addition to canonical implementation, we also exploit resolution-of-the-identity (RI) and Cholesky decomposition (CD) for the electron-repulsion integrals to reduce memory requirements and to increase parallel efficiency. The new methods are benchmarked against the CCSD and CC3 response theories for several small molecules. We found that the expectation-value 2PA cross sections are within 5% from the quadratic response CCSD values. The RI and CD approximations lead to small errors relative to the canonical implementation (less than 4%) while affording computational savings. RI/CD successfully address the well-known issue of large basis set requirements for 2PA cross sections calculations. The capabilities of the new code are illustrated by calculations of the 2PA cross sections for model chromophores of the photoactive yellow and green fluorescent proteins.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2012

ABWR start-up test analysis using BWR core simulator with three-dimensional direct response matrix method

The ABWR start-up test analysis has been done with the BWR core simulator using the three--dimensional direct response matrix (3D-DRM) method. The Monte Carlo code VMONT made the sub-response matrices for the 3D-DRM method. Each boundary surface was subdivided by 4 x 4 for transverse segments, by 4 for angular segments and by 4 for axial zones in a node. For the calculation speedup, the 3D-DRM code used the divided sub-response matrices data set. The code used the MPI and OpenMP for the parallelized method. The median value is set as the average critical eigenvalues. The changes from the maximum value to the minimum value are 0.34 %{Delta}k with the spectral history method and 0.40 %{Delta}k without it, and the respective standard deviations were 0.12 % and 0.14 %. Using the spectral history method decreased the variation by 0.06 %{Delta}k. The root mean square differences of the axial power distribution were about 6 % between the analysis results and the plant data. Using the currents which converged in the previous exposure step reduced the number of iterations when the CR pattern changed only slightly. The averaged calculation time for each exposure step was about 5 hours on 12 PC Linux cluster servers with Core 2 Quad 3 GHz. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2026

FitCache: A Transparent Drop-In Framework for Multi-Tier Caching to Accelerate Distributed Deep Learning Workloads

Training in Deep learning (DL) remains highly compute- and data-intensive, with I/O becoming a critical bottleneck as models and datasets scale. Recent studies report that data loading can dominate training time, especially on large-scale HPC systems with shared parallel file systems (PFS). Existing caching approaches either rely on single-tier designs or require intrusive modifications to training pipelines, limiting their portability and effectiveness. In this work, we present FitCache, a transparent drop-in framework for multi-tier caching to accelerate distributed DL training by coordinating fast local memory (e.g., DRAM, Persistent Memory (PMem)) and NVMe as hierarchical caches atop PFS. Our design adapts to hardware diversity, i.e., if NVMe is missing, memory transparently acts as a caching tier, ensuring stable performance. FitCache transparently intercepts I/O requests and issues concurrent fetches across all tiers, returning data from the fastest responder without centralized metadata or static redirection paths. FitCache adapts to dynamic workloads and heterogeneous clusters while maintaining POSIX compatibility. Experiments on Frontier (2048 GPUs) and smaller research clusters show that FitCache reduces training time by up to 40% and per-batch I/O latency by up to 71.6% compared to Lustre Orion PFS, offering a drop-in solution for scalable DL training.

Hu, Guangxing [ORNL] (ORCID:0009000283203614)↗
DOE OSTI2020

Self-sustaining sound in collisionless, high- β plasma

Using analytical theory and hybrid-kinetic numerical simulations, we demonstrate that, in a collisionless plasma, long-wavelength ion-acoustic waves (IAWs) with amplitudes $δn/n_0≳2/β$ (where $β\gg1$ is the ratio of thermal to magnetic pressure) generate sufficient pressure anisotropy to destabilize the plasma to firehose and mirror instabilities. These kinetic instabilities grow rapidly to reduce the pressure anisotropy by pitch-angle scattering and trapping particles, respectively, thereby impeding the maintenance of Landau resonances that enable such waves’ otherwise potent collisionless damping. The result is wave dynamics that evince a weakly collisional plasma: the ion distribution function is near-Maxwellian, the field-parallel flow of heat resembles its Braginskii form (except in regions where large-amplitude magnetic mirrors strongly suppress particle transport), and the relations between various thermodynamic quantities are more ‘fluid-like’ than kinetic. A nonlinear fluctuation–dissipation relation for self-sustaining IAWs is obtained by solving a plasma-kinetic Langevin problem, which demonstrates suppressed damping, enhanced fluctuation levels and weakly collisional thermodynamics when IAWs with $δn/n_0≳2/β$ are stochastically driven. We investigate how our results depend upon the scale separation between the wavelength of the IAW and the Larmor radius of the ions, and discuss briefly their implications for our understanding of turbulence and transport in the solar wind and the intracluster medium of galaxy clusters.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2014

Parallel Heuristics for Scalable Community Detection

Community detection has become a fundamental operation in numerous graph-theoretic applications. It is used to reveal natural divisions that exist within real world networks without imposing prior size or cardinality constraints on the set of communities. Despite its potential for application, there is only limited support for community detection on large-scale parallel computers, largely owing to the irregular and inherently sequential nature of the underlying heuristics. In this paper, we present parallelization heuristics for fast community detection using the Louvain method as the serial template. The Louvain method is an iterative heuristic for modularity optimization. Originally developed by Blondel et al. in 2008, the method has become increasingly popular owing to its ability to detect high modularity community partitions in a fast and memory-efficient manner. However, the method is also inherently sequential, thereby limiting its scalability to problems that can be solved on desktops. Here, we observe certain key properties of this method that present challenges for its parallelization, and consequently propose multiple heuristics that are designed to break the sequential barrier. Our heuristics are agnostic to the underlying parallel architecture. For evaluation purposes, we implemented our heuristics on shared memory (OpenMP) and distributed memory (MapReduce-MPI) machines, and tested them over real world graphs derived from multiple application domains (internet, biological, natural language processing). Experimental results demonstrate the ability of our heuristics to converge to high modularity solutions comparable to those output by the serial algorithm in nearly the same number of iterations, while also drastically reducing time to solution.

clustering, graph algorithms↗
DOE OSTI2013

ON THE FORMATION OF THE PECULIAR LOW-MASS X-RAY BINARY IGR J17480-2446 IN TERZAN 5

IGR J17480-2446 is an accreting X-ray pulsar in a low-mass X-ray binary harbored in the Galactic globular cluster Terzan 5. Compared with other accreting millisecond pulsars, IGR J17480-2446 is peculiar for its low spin frequency (11 Hz), which suggests that it might be a mildly recycled neutron star at the very early phase of mass transfer. However, this model seems to be in contrast with the low field strength deduced from the kilo-Hertz quasi-periodic oscillations observed in IGR J17480-2446. Here, we suggest an alternative interpretation, assuming that the current binary system was formed during an exchange encounter either between a binary (which contains a recycled neutron star) and the current donor, or between a binary and an isolated, recycled neutron star. In the resulting binary, the spin axis of the neutron star could be parallel or anti-parallel with the orbital axis. In the latter case, the abnormally low frequency of IGR J17480-2446 may result from the spin-down to spin-up evolution of the neutron star. We also briefly discuss the possible observational implications of the pulsar in this scenario.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2016

2016 High Performance Computing Annual Report

Sandia National Laboratories’ California site is celebrating its 60th anniversary (1956 to 2016), and high performance computing has been a key enabler for its scientists and engineers throughout much of its history. Since its founding, Sandia California has helped pioneer the use of HPC platforms including hosting Sandia’s first Cray-1 supercomputer in the 1970s and supporting development of scalable cluster computing platforms to create a new paradigm for cost-effective supercomputing in the 1990s. Recent decades of investment in creation of scalable application frameworks for scientific computing have also enabled new generations of modeling and simulation codes. These resources have facilitated computational analysis of complex phenomena in diverse applications spanning national defense, energy, and homeland security. Today, Sandia California researchers work with partners in academia, industry, and national labs to evolve the state-of-the-art in HPC, modeling, and data analysis (including foundational capabilities for exascale computing platforms) and apply them in transformational ways. Research efforts include mitigating the effects of silent hardware failures that can jeopardize the results of large-scale computations, developing exascale-capable asynchronous task-parallel programming models and runtime systems, formulating new techniques to better explore and analyze extreme-scale data sets, and increasing our understanding of materials and chemical sciences which has applications spanning nuclear weapons stockpile stewardship to more efficient automobile engines. The following section highlights some of these research and applications projects and further illustrates the breadth of our HPC capabilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

A scalable approach to solving dense linear algebra problems on hybrid CPU-GPU systems

Aiming to fully exploit the computing power of all CPUs and all graphics processing units (GPUs) on hybrid CPU-GPU systems to solve dense linear algebra problems, in this paper we design a class of heterogeneous tile algorithms to maximize the degree of parallelism, to minimize the communication volume, and to accommodate the heterogeneity between CPUs and GPUs. The new heterogeneous tile algorithms are executed upon our decentralized dynamic scheduling runtime system, which schedules a task graph dynamically and transfers data between compute nodes automatically. The runtime system uses a new distributed task assignment protocol to solve data dependencies between tasks without any coordination between processing units. By overlapping computation and communication through dynamic scheduling, we are able to attain scalable performance for the double-precision Cholesky factorization and QR factorization. Finally, our approach demonstrates a performance comparable to Intel MKL on shared-memory multicore systems and better performance than both vendor (e.g., Intel MKL) and open source libraries (e.g., StarPU) in the following three environments: heterogeneous clusters with GPUs, conventional clusters without GPUs, and shared-memory systems with multiple GPUs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

HipBone: A performance-portable graphics processing unit-accelerated C++ version of the NekBone benchmark

We present hipBone, an open-source performance-portable proxy application for the Nek5000 (and NekRS) computational fluid dynamics applications. HipBone is a fully GPU-accelerated C++ implementation of the original NekBone CPU proxy application with several novel algorithmic and implementation improvements which optimize its performance on modern fine-grain parallel GPU accelerators. Our optimizations include a conversion to store the degrees of freedom of the problem in assembled form in order to reduce the amount of data moved during the main iteration and a portable implementation of the main Poisson operator kernel. We demonstrate near-roofline performance of the operator kernel on three different modern GPU accelerators from two different vendors. We present a novel algorithm for splitting the application of the Poisson operator on GPUs which aggressively hides MPI communication required for both halo exchange and assembly. Our implementation of nearest-neighbor MPI communication then leverages several different routing algorithms and GPU-Direct RDMA capabilities, when available, which improves scalability of the benchmark. We demonstrate the performance of hipBone on three different clusters housed at Oak Ridge National Laboratory, namely, the Summit supercomputer and the Frontier early-access clusters, Spock and Crusher. Our tests demonstrate both portability across different clusters and very good scaling efficiency, especially on large problems.

Computer Science↗
DOE OSTI2015

Trinity Benchmarks on the Intel Xeon Phi (Knights Corner)

This report documents the early experiences with porting and performance analysis of the Tri-Lab Trinity benchmark applications on Intel Xeon Phi (Knights Corner) (KNC) processor. KNC, the second generation of the Intel Many Integrated Core (MIC) architectures, uses a large number of small P54C-x86 cores with wide vector units and is deployed as PCI bus attached process accelerators. Sandia has experimental test beds of small InifiniBand clusters and workstations to investigate the performance of the MIC architecture. On these experimental test beds the programming models that may be investigated are "offload", "symmetric" and "native". Among these program usage models our primary interest is in the so called "native" mode, because the planned Trinity system to be deployed in 2016 using the next generation MIC processor architecture called Knights Landing would be self-hosted. Trinity / NERSC-8 benchmark programs cover a variety of scientific disciplines and they were used to guide the procurement of these systems. Architectures such as the Intel MIC are well suited to study evolving processor architectures and a usage model commonly referred to as MPI + X that facilitates migration of our applications to use both coarse grain and fine grain parallelism. Our focus with the applications selected is on the efficacy of algorithms in these applications to take advantage of features like: large number of cores, wide vector units, higher-bandwidth and deeper memory sub-system. This is a first step towards understanding applications, algorithms and programming environments for Trinity and future exascale computing systems.

97 MATHEMATICS AND COMPUTING↗