Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

Distributed Computing (MPI)

This powerpoint is part of a Parallel Summer Computing Research Internship lecture on distributed computing (MPI). It goes over how MPI works and then shows examples.

97 MATHEMATICS AND COMPUTING↗

TuckerMPI: A Parallel C++/MPI Software Package for Large-scale Data Compression via the Tucker Tensor Decomposition

With this study, our goal is compression of massive-scale grid-structured data, such as the multi-terabyte output of a high-fidelity computational simulation. For such data sets, we have developed a new software package called TuckerMPI, a parallel C++/MPI software package for compressing distributed data. The approach is based on treating the data as a tensor, i.e., a multidimensional array, and computing its truncated Tucker decomposition, a higher-order analogue to the truncated singular value decomposition of a matrix. The result is a low-rank approximation of the original tensor-structured data. Compression efficiency is achieved by detecting latent global structure within the data, which we contrast to most compression methods that are focused on local structure. In this work, we describe TuckerMPI, our implementation of the truncated Tucker decomposition, including details of the data distribution and in-memory layouts, the parallel and serial implementations of the key kernels, and analysis of the storage, communication, and computational costs. We test the software on 4.5 and 6.7 terabyte data sets distributed across 100 s of nodes (1,000 s of MPI processes), achieving compression ratios between 100 and 200,000×, which equates to 99--99.999% compression (depending on the desired accuracy) in substantially less time than it would take to even read the same dataset from a parallel file system. Moreover, we show that our method also allows for reconstruction of partial or down-sampled data on a single node, without a parallel computer so long as the reconstructed portion is small enough to fit on a single machine, e.g., in the instance of reconstructing/visualizing a single down-sampled time step or computing summary statistics. The code is available at https://gitlab.com/tensors/TuckerMPI.

97 MATHEMATICS AND COMPUTING↗

Petascale direct numerical simulation of blood flow on 200K cores and heterogeneous architectures

We present a fast, petaflop-scalable algorithm for Stokesian particulate flows. Our goal is the direct simulation of blood, which we model as a mixture of a Stokesian fluid (plasma) and red blood cells (RBCs). Directly simulating blood is a challenging multiscale, multiphysics problem. We report simulations with up to 260 million deformable RBCs. The largest simulation amounts to 90 billion unknowns in space. In terms of the number of cells, we improve the state-of-the art by several orders of magnitude: the previous largest simulation, at the same physical fidelity as ours, resolved the flow of O(1,000-10,000) RBCs. Our approach has three distinct characteristics: (1) we faithfully represent the physics of RBCs by using nonlinear solid mechanics to capture the deformations of each cell; (2) we accurately resolve the long-range, N-body, hydrodynamic interactions between RBCs (which are caused by the surrounding plasma); and (3) we allow for highly non-uniform spatial distributions of RBCs. The new method has been implemented in the software library MOBO (for 'Moving Boundaries'). We designed MOBO to support parallelism at all levels, including inter-node distributed memory parallelism, intra-node shared memory parallelism, data parallelism (vectorization), and fine-grained multithreading for GPUs. We have implemented and optimized the majority of the computation kernels on both Intel/AMD x86 and NVidia's Tesla/Fermi platforms for single and double floating point precision. Overall, the code has scaled on 256 CPU-GPUs on the Teragrid's Lincoln cluster and on 200,000 AMD cores of the Oak Ridge National Laboratory's Jaguar PF system. In our largest simulation, we have achieved 0.7 Petaflops/s of sustained performance on Jaguar.

59 BASIC BIOLOGICAL SCIENCES↗

A matrix-algebraic formulation of distributed-memory maximal cardinality matching algorithms in bipartite graphs

We describe parallel algorithms for computing maximal cardinality matching in a bipartite graph on distributed-memory systems. Unlike traditional algorithms that match one vertex at a time, our algorithms process many unmatched vertices simultaneously using a matrix-algebraic formulation of maximal matching. This generic matrix-algebraic framework is used to develop three efficient maximal matching algorithms with minimal changes. The newly developed algorithms have two benefits over existing graph-based algorithms. First, unlike existing parallel algorithms, cardinality of matching obtained by the new algorithms stays constant with increasing processor counts, which is important for predictable and reproducible performance. Second, relying on bulk-synchronous matrix operations, these algorithms expose a higher degree of parallelism on distributed-memory platforms than existing graph-based algorithms. We report high-performance implementations of three maximal matching algorithms using hybrid OpenMP-MPI and evaluate the performance of these algorithm using more than 35 real and randomly generated graphs. On real instances, our algorithms achieve up to 200 × speedup on 2048 cores of a Cray XC30 supercomputer. Even higher speedups are obtained on larger synthetically generated graphs where our algorithms show good scaling on up to 16,384 cores.

97 MATHEMATICS AND COMPUTING↗

Methods and apparatus for capture and storage of semantic information with sub-files in a parallel computing system

Techniques are provided for storing files in a parallel computing system using sub-files with semantically meaningful boundaries. A method is provided for storing at least one file generated by a distributed application in a parallel computing system. The file comprises one or more of a complete file and a plurality of sub-files. The method comprises the steps of obtaining a user specification of semantic information related to the file; providing the semantic information as a data structure description to a data formatting library write function; and storing the semantic information related to the file with one or more of the sub-files in one or more storage nodes of the parallel computing system. The semantic information provides a description of data in the file. The sub-files can be replicated based on semantically meaningful boundaries.

97 MATHEMATICS AND COMPUTING↗

Ordinary mode instability associated with thermal ring distribution

The purely growing ordinary (O) mode instability driven by excessive parallel temperature anisotropy has recently received renewed attention owing to its potential applicability to the solar wind plasma. Previous studies of O mode instability have assumed either bi-Maxwellian or counter-streaming velocity distributions. For solar wind plasma trapped in magnetic mirror-like geometry such as magnetic clouds or in the vicinity of the Earth's collisionless bow shock environment, however, the velocity distribution function may possess a loss-cone feature. The O-mode instability in such a case may be excited for cyclotron harmonics as well as the purely growing branch. The present paper investigates the O-mode instability for plasmas characterized by the parallel Maxwellian distribution and perpendicular thermal ring velocity distribution in order to understand the general stability characteristics.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Iterative Importance Sampling Algorithms for Parameter Estimation

In parameter estimation problems one computes a posterior distribution over uncertain parameters defined jointly by a prior distribution, a model, and noisy data. Markov chain Monte Carlo (MCMC) is often used for the numerical solution of such problems. An alternative to MCMC is importance sampling, which can exhibit near perfect scaling with the number of cores on high performance computing systems because samples are drawn independently. However, finding a suitable proposal distribution is a challenging task. Several sampling algorithms have been proposed over the past years that take an iterative approach to constructing a proposal distribution. We investigate the applicability of such algorithms by applying them to two realistic and challenging test problems, one in subsurface flow, and one in combustion modeling. More specifically, we implement importance sampling algorithms that iterate over the mean and covariance matrix of Gaussian or multivariate t-proposal distributions. Our implementation leverages massively parallel computers, and we present strategies to initialize the iterations using “coarse” MCMC runs or Gaussian mixture models.

97 MATHEMATICS AND COMPUTING↗

Performance Evaluation of Multi-Vendor Grid-Forming Inverters for Grid-Connected Operation Through Hardware Experimentation: Preprint

Existing real-world projects of GFM inverters that operate in parallel to power grids typically are sized between dozens and a few hundreds Megawatt (MW) scale according to a recent NERC GFM inverter white paper. These large systems are often difficult to evaluate prior to deployments because of their large size. The performance of smaller GFM inverters (dozens to a few hundreds MW) that operate parallel with power grids (distribution systems) is even less understood. There is an opportunity to better understand these systems through hardware testing under controlled laboratory conditions. Therefore, this paper presents the functional performance evaluation tests of multiple (three) commercial GFM inverters when they operate in parallel with the grid through hardware experiments. The goal of these tests is to explore the GFM inverters' functionalities and dynamic response when in parallel with power grids to eventually develop universal specifications for GFM inverters. Both steady state (changing the inverter's frequency and voltage droop) and transient (adding step change in grid's frequency/voltage) tests are performed for each GFM inverter with the same testing circuit and testing protocol. The experimental results indicate the bench-marked performance that: 1) the GFM inverters can be dispatched through frequency and voltage droop intercepts to output the target power when paralleled to the grid; 2) the GFM inverters automatically respond to system frequency and voltage events to output the needed power, however, the GFM inverters all show stability issues when absorbing reactive power from the grid.

grid-forming inverters↗

Reconnection and electron temperature anisotropy in sub-proton scale plasma turbulence

Knowledge of turbulent behavior at sub-proton scales in magnetized plasmas is important for a full understanding of the energetics of astrophysical flows such as the solar wind. We study the formation of electron temperature anisotropy due to reconnection in the turbulent decay of sub-proton scale fluctuations using two-dimensional, particle-in-cell plasma simulations with a realistic electron-proton mass ratio and a guide field perpendicular to the simulation plane. A power spectrum fluctuation with approximately power-law form is created down to scales of the order of the electron gyroradius. We identify the signatures of collisionless reconnection at sites of X-point field geometry in the dynamic magnetic field topology, which gradually relaxes in complexity. The reconnection sites are generally associated with regions of strong parallel electron temperature anisotropy. The evolving topology of magnetic field lines connected to a reconnection site allows for the spatial mixing of electrons accelerated at multiple, spatially separated reconnection regions. This leads to the formation of multi-peaked velocity distribution functions with strong parallel temperature anisotropy. In a three-dimensional system that can support the appropriate wave vectors, the multi-peaked distribution functions would be expected to be unstable to kinetic instabilities, contributing to dissipation. The proposed mechanism of anisotropy formation is also relevant to space and astrophysical systems where the evolution of the plasma is constrained by linear temperature anisotropy instability thresholds. The presence of reconnection sites leads to electron energy gain, nonlocal velocity space mixing, and the formation of strong temperature anisotropy; this is evidence of an important role for reconnection in the dissipation of turbulent fluctuations.

79 ASTRONOMY AND ASTROPHYSICS↗

Diagnosis of Performance Faults in LargeScale MPI Applications via Probabilistic Progress-Dependence Inference

Debugging large-scale parallel applications is challenging. Most existing techniques provide little information about failure root causes. Further, most debuggers significantly slow down program execution, and run sluggishly with massively parallel applications. This paper presents a novel technique that scalably infers the tasks in a parallel program on which a failure occurred, as well as the code in which it originated. Our technique combines scalable runtime analysis with static analysis to determine the least-progressed task(s) and to identify the code lines at which the failure arose. We present a novel algorithm that infers probabilistically progress dependence among MPI tasks using a globally constructed Markov model that represents tasks' control-flow behavior. In comparison to previous work, our algorithm infers more precisely the least-progressed task. Further, we combine this technique with static backward slicing analysis, further isolating the code responsible for the current state. A blind study demonstrates that our technique isolates the root cause of a concurrency bug in a molecular dynamics simulation, which only manifests itself at 7,996 tasks or more. We extensively evaluate fault coverage of our technique via fault injections in 10 HPC benchmarks and show that our analysis takes less than a few seconds on thousands of parallel tasks.

97 MATHEMATICS AND COMPUTING↗

On the Structure of the AGN Torus through the Fraction of Optically Selected Type 1 AGNs

The ratio in number between unobscured (type 1) and obscured (type 2) active galactic nuclei (AGNs) is often used to explore the structure of the torus in the unified scheme for AGNs. Oh et al. (2015) investigated the type 1 AGN fraction on two-dimensional space in terms of black hole mass (M{sub BH}) and bolometric luminosity (L{sub bol}) and found that the fraction changes depending on both M{sub BH} and L{sub bol}, forming a ridge-shaped distribution. In this study, based on the up-to-date type 1 AGN catalog of Oh et al. (2015), we examine how the trend of the type 1 AGN fraction in the M{sub BH}–L{sub bol} plane is affected by the different methods used to derive M{sub BH} and L{sub bol}, and suggest an analytic model to explain the observations. We use galaxies from the Sloan Digital Sky Survey Data Release 7 in the redshift range 0.01⩽z⩽0.2. In estimating L{sub bol}, we employ two different methods using [O III] and/or [O I] emission lines, and find that the L{sub bol} values obtained from the two methods agree well. We consider the M{sub BH}--σ{sub ∗} relation, the M{sub BH}–L {sub bulge} relation, and the single-epoch Hα-based M{sub BH} estimate in calculating M{sub BH}. We find that the trends of the type 1 AGN fraction with respect to M{sub BH} and L{sub bol} are similar for the different methods of deriving L{sub bol} but different when using different methods to derive M{sub BH}. We present a model based on the clumpy-torus scheme that reproduces the ridge-shaped distribution of the fraction parallel to the iso-Eddington ratio lines.

79 ASTRONOMY AND ASTROPHYSICS↗

diBELLA: Distributed Long Read to Long Read Alignment

© 2019 ACM. We present a parallel algorithm and scalable implementation for genome analysis, specifically the problem of finding overlaps and alignments for data from "third generation" long read sequencers [29]. While long sequences of DNA offer enormous advantages for biological analysis and insight, current long read sequencing instruments have high error rates and therefore require different approaches to analysis than their short read counterparts. Our work focuses on an efficient distributed-memory parallelization of an accurate single-node algorithm for overlapping and aligning long reads. We achieve scalability of this irregular algorithm by addressing the competing issues of increasing parallelism, minimizing communication, constraining the memory footprint, and ensuring good load balance. The resulting application, diBELLA, is the first distributed memory overlapper and aligner specifically designed for long reads and parallel scalability. We describe and present analyses for high level design trade-offs and conduct an extensive empirical analysis that compares performance characteristics across state-of-the-art HPC systems as well as a commercial cloud architectures, highlighting the advantages of state-of-the-art network technologies.

Ellis, Marquita↗

On the existence of Weibel instability in a magnetized plasma. I. Parallel wave propagation

In a uniform static magnetic field (B{sub 0}) the nonresonant mechanism of Weibel instability is certainly affected, because the free energy stored in the temperature anisotropy (e.g., T{sub perpendicular}>T{sub parallel}), where perpendicular and parallel denote directions relative to the background magnetic field) can drive the parallel whistler instability due to the cyclotron resonance with plasma particles. However, it has already been shown that the nonresonant mechanism should exist and drive the whistler instability, at least, for anisotropic distribution functions with no parallel thermal velocity spread. Here the investigation is extended to a magnetized plasma with a bi-Maxwellian temperature anisotropy. New frequency limits are introduced and the necessary physical conditions are provided: The nonresonant whistler instabilities have in general small frequencies, smaller than the growth rate, {omega}{sub r}<{omega}{sub i}, and for a large temperature anisotropy, the nonresonant instabilities are limited to a {beta}{sub perpendicular}=8{pi}n{sub 0}T{sub perpendicular}/B{sub 0}{sup 2}>1 regime. These criteria restricts in general the magnetic field to strengths less than a threshold value proportional to the square root of the temperature anisotropy. In the presence of an ambient magnetic field, the growth rate of the nonresonant instability is reduced and the large wavelengths are stabilized.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Distributed Training for High Resolution Images: A Domain and Spatial Decomposition Approach

In this work we developed two Pytorch libraries using the PyTorch RPC interface for distributed deep learning approaches on high resolution images. The spatial decomposition library allows for distributedtraining on very large images, which otherwise won’t be possible on a single GPU. The domain parallelism library allows for distributed training across multiple domain unlabeled data, by leveraging the domain separation architecture. Both of those libraries were tested on the Summit supercomputer at a moderate scale, and we are releasing the code for both of them.

97 MATHEMATICS AND COMPUTING↗

Scaling neural simulations in STACS

Abstract As modern neuroscience tools acquire more details about the brain, the need to move towards biological-scale neural simulations continues to grow. However, effective simulations at scale remain a challenge. Beyond just the tooling required to enable parallel execution, there is also the unique structure of the synaptic interconnectivity, which is globally sparse but has relatively high connection density and non-local interactions per neuron. There are also various practicalities to consider in high performance computing applications, such as the need for serializing neural networks to support potentially long-running simulations that require checkpoint-restart. Although acceleration on neuromorphic hardware is also a possibility, development in this space can be difficult as hardware support tends to vary between platforms and software support for larger scale models also tends to be limited. In this paper, we focus our attention on Simulation Tool for Asynchronous Cortical Streams (STACS), a spiking neural network simulator that leverages the Charm++ parallel programming framework, with the goal of supporting biological-scale simulations as well as interoperability between platforms. Central to these goals is the implementation of scalable data structures suitable for efficiently distributing a network across parallel partitions. Here, we discuss a straightforward extension of a parallel data format with a history of use in graph partitioners, which also serves as a portable intermediate representation for different neuromorphic backends. We perform scaling studies on the Summit supercomputer, examining the capabilities of STACS in terms of network build and storage, partitioning, and execution. We highlight how a suitably partitioned, spatially dependent synaptic structure introduces a communication workload well-suited to the multicast communication supported by Charm++. We evaluate the strong and weak scaling behavior for networks on the order of millions of neurons and billions of synapses, and show that STACS achieves competitive levels of parallel efficiency.

59 BASIC BIOLOGICAL SCIENCES↗

Characteristics of elliptical sources in BEAMnrc Monte Carlo system: Implementation and application

Recently, several papers noticed that the electron focal spot of a linear accelerator (linac) could be elliptical which would cause dosimetric discrepancies between measurements and Monte Carlo simulations. To resolve the mismatch, two elliptical source models were developed in BEAMnrc code. The first was a parallel beam elliptical source with uniform distribution where the shape of the source was primarily considered. The other was a parallel beam elliptical source with Gaussian distribution whose source distribution follows the normal distribution. To validate the elliptical source models, uniform and Gaussian electron beams were impinged on a thin air target. Both models successfully reproduced the elliptical shapes and source distributions. Then, this study investigated the characteristics of the elliptical Gaussian source for a 6 MV photon beam in a Varian 21EX linac. The linac head model was implemented in the BEAMnrc/EGSnrc system and commissioned by comparing the lateral and depth dose profiles to the ion chamber measurements acquired from the annual quality assurance (QA). It was found that the circular Gaussian beam with 6 MeV/0.2 cm full width half maximum (FWHM) produces the best matches to the QA data. To explore the characteristics of the elliptical Gaussian source, this study employed an elliptical Gaussian electron source with 0.1 cm FWHM in the x axis and 0.2 cm FWHM in the y axis which was incident on the target of the linac head. Two circular Gaussian beams with 0.1 and 0.2 cm FWHM were employed to compare the differences between circular and elliptical sources. For all the sources, planar and energy fluences were acquired and analyzed. This study also compared the lateral and depth dose profiles in a water phantom by using a DOSXYZnrc user code. In results, a constricted shoulder effect was observed in both planar and energy fluence plots when the FWHM value was increased and the field size is larger than 30x30 cm{sup 2}. The same effect was also noticed in the lateral dose profiles, while the depth dose profile did not vary much.

43 PARTICLE ACCELERATORS↗

Self-action of Bessel wave packets in a system of coupled light guides and formation of light bullets

The self-action of two-dimensional and three-dimensional Bessel wave packets in a system of coupled light guides is considered using the discrete nonlinear Schrödinger equation. The features of the self-action of such wave fields are related to their initial strong spatial inhomogeneity. The numerical simulation shows that for the field amplitude exceeding a critical value, the development of an instability typical of a medium with the cubic nonlinearity is observed. Various regimes are studied: the self-channeling of a wave beam in one light guide at powers not strongly exceeding a critical value, the formation of the “kaleidoscopic” picture of a wave packet during the propagation of higher-power radiation along a stratified medium, the formation of light bullets during competition between self-focusing and modulation instabilities in the case of three-dimensional wave packets, etc. In the problem of laser pulse shortening, the situation is considered when the wave-field stratification in the transverse direction dominates. This process is accompanied by the self-compression of laser pulses in well enough separated light guides. The efficiency of conversion of the initial Bessel field distribution to two flying parallel light bullets is about 50%.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Low-Rank Correction Methods for Algebraic Domain Decomposition Preconditioners

This study presents a parallel preconditioning method for distributed sparse linear systems, based on an approximate inverse of the original matrix, that adopts a general framework of distributed sparse matrices and exploits domain decomposition (DD) and low-rank corrections. The DD approach decouples the matrix and, once inverted, a low-rank approximation is applied by exploiting the Sherman--Morrison--Woodbury formula, which yields two variants of the preconditioning methods. The low-rank expansion is computed by the Lanczos procedure with reorthogonalizations. Numerical experiments indicate that, when combined with Krylov subspace accelerators, this preconditioner can be efficient and robust for solving symmetric sparse linear systems. Comparisons with pARMS, a DD-based parallel incomplete LU (ILU) preconditioning method, are presented for solving Poisson's equation and linear elasticity problems.

97 MATHEMATICS AND COMPUTING↗