Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

Compile-time estimation of communication costs in multicomputers

An important problem facing numerous research projects on parallelizing compilers for distributed memory machines is that of automatically determining a suitable data partitioning scheme for a program. Any strategy for automatic data partitioning needs a mechanism for estimating the performance of a program under a given partitioning scheme, the most crucial part of which involves determining the communication costs incurred by the program. A methodology is described for estimating the communication costs at compile-time as functions of the numbers of processors over which various arrays are distributed. A strategy is described along with its theoretical basis, for making program transformations that expose opportunities for combining of messages, leading to considerable savings in the communication costs. For certain loops with regular dependences, the compiler can detect the possibility of pipelining, and thus estimate communication costs more accurately than it could otherwise. These results are of great significance to any parallelization system supporting numeric applications on multicomputers. In particular, they lay down a framework for effective synthesis of communication on multicomputers from sequential program references.

Gupta, Manish↗

GronOR: Massively Parallel and GPU-Accelerated Non-Orthogonal Configuration Interaction for Large Molecular Systems

GronOR is a program package for non-orthogonal configuration interaction calculations for an electronic wave function built in terms of anti-symmetrized products of multi-configuration molecular fragment wave functions. The two-electron integrals that have to be processed may be expressed in terms of atomic orbitals or in terms of an orbital basis determined from the molecular orbitals of the fragments. The code has been specifically designed for execution on distributed memory massively parallel and Graphics Processing Unit (GPU)-accelerated computer architectures, using an MPI+OpenACC/OpenMP programming approach. The task-based execution model used in the implementation allows for linear scaling with the number of nodes on the largest pre-exascale architectures available, provides hardware fault resiliency, and enables effective execution on systems with distinct central processing unit-only and GPU-accelerated partitions. The code interfaces with existing multi-configuration electronic structure codes that provide optimized molecular fragment orbitals, configuration interaction coefficients, and the required integrals. Algorithm and implementation details, parallel and accelerated performance benchmarks, and an analysis of the sensitivity of the accuracy of results and computational performance to thresholds used in the calculations are presented.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Optimization and Parallelization of the Thermal-Hydraulic Sub-channel Code CTF for High-Fidelity Multi-physics Applications

This paper describes major improvements to the computational infrastructure of the CTF sub-channel code so that full-core sub-channel-resolved simulations can now be performed in much shorter run-times, either in stand-alone mode or as part of coupled-code multi-physics calculations. These improvements support the goals of the Department Of Energy (DOE) Consortium for Advanced Simulations of Light Water (CASL) Energy Innovation Hub to develop high fidelity multi-physics simulation tools for nuclear energy design and analysis. A set of serial code optimizations--including fixing computational inefficiencies, optimizing the numerical approach, and making smarter data storage choices--are first described and shown to reduce both execution time and memory usage by about a factor of ten. Next, a Single Program Multiple Data (SPMD) parallelization strategy targeting distributed memory Multiple Instruction Multiple Data (MIMD) platforms and utilizing domain-decomposition is presented. In this approach, data communication between processors is accomplished by inserting standard MPI calls at strategic points in the code. The domain decomposition approach implemented assigns one MPI process to each fuel assembly, with each domain being represented by its own CTF input file. The creation of CTF input files, both for serial and parallel runs, is also fully automated through use of a pre-processor utility that takes a greatly reduced set of user input over the traditional CTF input file. To run CTF in parallel, two additional libraries are currently needed; MPI, for inter-processor message passing, and the Parallel Extensible Toolkit for Scientific Computation (PETSc), which is leveraged to solve the global pressure matrix in parallel. Results presented include a set of testing and verification calculations and performance tests assessing parallel scaling characteristics up to a full core, sub-channel-resolved model of Watts Bar Unit 1 under hot full-power conditions (193 17x17 assemblies, ~56,000 pins, ~59,000 sub-channels, ~2.8 million thermal-hydraulic (TH) control volumes). Results demonstrate that CTF can now perform full core analysis (not previously possible due to excessively long runtimes and memory requirements) on the order of 20 minutes. This new capability is not only useful to standalone CTF users, but is also being leveraged in support of coupled code multi-physics calculations being done in the CASL program.

COBRA-TF↗

Global Arrays Parallel Programming Toolkit

The two predominant classes of programming models for parallel computing are distributed memory and shared memory. Both shared memory and distributed memory models have advantages and shortcomings. Shared memory model is much easier to use but it ignores data locality/placement. Given the hierarchical nature of the memory subsystems in modern computers this characteristic can have a negative impact on performance and scalability. Careful code restructuring to increase data reuse and replacing fine grain load/stores with block access to shared data can address the problem and yield performance for shared memory that is competitive with message-passing. However, this performance comes at the cost of compromising the ease of use that the shared memory model advertises. Distributed memory models, such as message-passing or one-sided communication, offer performance and scalability but they are difficult to program. The Global Arrays toolkit attempts to offer the best features of both models. It implements a shared-memory programming model in which data locality is managed by the programmer. This management is achieved by calls to functions that transfer data between a global address space (a distributed array) and local storage. In this respect, the GA model has similarities to the distributed shared-memory models that provide an explicit acquire/release protocol. However, the GA model acknowledges that remote data is slower to access than local data and allows data locality to be specified by the programmer and hence managed. GA is related to the global address space languages such as UPC, Titanium, and, to a lesser extent, Co-Array Fortran. In addition, by providing a set of data-parallel operations, GA is also related to data-parallel languages such as HPF, ZPL, and Data Parallel C. However, the Global Array programming model is implemented as a library that works with most languages used for technical computing and does not rely on compiler technology for achieving parallel efficiency. It also supports a combination of task- and data-parallelism and is available as an extension of the message passing (MPI) model. The GA model exposes to the programmer the hierarchical memory of modern high-performance computer systems, and by recognizing the communication overhead for remote data transfer, it promotes data reuse and locality of reference. Virtually all the scalable architectures possess non-uniform memory access characteristics that reflect their multi-level memory hierarchies. These hierarchies typically comprise processor registers, multiple levels of cache, local memory, and remote memory. Over time, both the number of levels and the cost (in processor cycles) of accessing deeper levels has been increasing. It is important for any scalable programming model to address memory hierarchy since it is critical to the efficient execution of scalable applications.

ARMCI↗

Electron acceleration by magnetic islands in a dynamically evolved coronal current sheet

This work simulated the electron acceleration by magnetic islands in a drastically evolved solar coronal current sheet via the combined 2.5-dimensional (2.5D) resistive Magnetohydrodynamics (MHD) and guiding-center approximation test-particle methods. With high magnetic Reynolds number of 105, the long–thin current sheet is evolved into a chain of magnetic islands, growing in size and coalescing with each other, due to tearing instability. The acceleration of electrons is studied in one typical phase when several large magnetic islands are formed. The results show that the electrons with an initial Maxwell distribution evolve into a heavy-tailed distribution and more than 20% of the electrons can be accelerated higher than 200 keV within 0.1 second and some of them can even be energized up to MeV ranges. The most energetic electrons have a tendency to be around the outer regions of the magnetic islands or to be located in the small secondary magnetic islands. We find that the acceleration and spatial distributions of the energetic electrons is caused by the trapping effect of the magnetic islands and the distributions of the parallel electric field E{sub p}.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A portable MPI-based parallel vector template library

This paper discusses the design and implementation of a polymorphic collection library for distributed address-space parallel computers. The library provides a data-parallel programming model for C++ by providing three main components: a single generic collection class, generic algorithms over collections, and generic algebraic combining functions. Collection elements are the fourth component of a program written using the library and may be either of the built-in types of C or of user-defined types. Many ideas are borrowed from the Standard Template Library (STL) of C++, although a restricted programming model is proposed because of the distributed address-space memory model assumed. Whereas the STL provides standard collections and implementations of algorithms for uniprocessors, this paper advocates standardizing interfaces that may be customized for different parallel computers. Just as the STL attempts to increase programmer productivity through code reuse, a similar standard for parallel computers could provide programmers with a standard set of algorithms portable across many different architectures. The efficacy of this approach is verified by examining performance data collected from an initial implementation of the library running on an IBM SP-2 and an Intel Paragon.

Sheffler, Thomas J.↗

A Portable MPI-Based Parallel Vector Template Library

This paper discusses the design and implementation of a polymorphic collection library for distributed address-space parallel computers. The library provides a data-parallel programming model for C + + by providing three main components: a single generic collection class, generic algorithms over collections, and generic algebraic combining functions. Collection elements are the fourth component of a program written using the library and may be either of the built-in types of c or of user-defined types. Many ideas are borrowed from the Standard Template Library (STL) of C++, although a restricted programming model is proposed because of the distributed address-space memory model assumed. Whereas the STL provides standard collections and implementations of algorithms for uniprocessors, this paper advocates standardizing interfaces that may be customized for different parallel computers. Just as the STL attempts to increase programmer productivity through code reuse, a similar standard for parallel computers could provide programmers with a standard set of algorithms portable across many different architectures. The efficacy of this approach is verified by examining performance data collected from an initial implementation of the library running on an IBM SP-2 and an Intel Paragon.

Sheffler, Thomas J.↗

Tomographic analysis of tangential viewing cameras (invited)

Many tokamaks now use visible light cameras to observe plasma-wall interactions and integrated line emission. The DIII-D coherence imaging spectroscopy diagnostic cameras image interferograms that encode line integrated velocity. By modeling the 2D camera image pixels as line of sight integrals through an axisymmetric discrete grid, it is possible to do tomographic analysis to determine the local plasma line emissivity and parallel velocity. Approaches to solve the inverse problem posed by these tangential viewing cameras are presented. The inversion begins with calculation of the sparse response matrix that encompasses all the geometry and diagnostic information and reduces the process of image formation to a sparse matrix-vector multiply. Our report includes techniques for determining the detailed geometry of the camera views and methods for handling physical quantities that vary spatially. Additionally, the size of the response matrix has driven the development of capability to distribute the coarse parallel calculation across a heterogeneous cluster of computers on the Energy Sciences Network. Iterative techniques are then used to solve the sparse matrix-vector linear system

47 OTHER INSTRUMENTATION↗

Ion-cyclotron instability in plasmas described by product-bi-kappa distributions

The dispersion relation for parallel propagating waves in the ion-cyclotron branch is investigated numerically by considering that the velocity distribution of the ion population is a function of type product-bi-kappa. We investigate the effects of the non-thermal features and of the anisotropy associated with this type of distribution on the ion-cyclotron instability, as well as the influence of different forms of the electron distribution, by considering Maxwellian distributions, bi-kappa distributions, and product-bi-kappa distributions. The cases of ions described by either Maxwellian or bi-kappa distributions are also considered, for comparison. The results of the numerical analysis show that the increase in the non-thermal character associated with the anisotropic kappa distributions for ions contributes to enhance the instability as compared to that obtained in the Maxwellian case, in magnitude and in wave number range, with more significant enhancement for the case of ion product-bi-kappa distributions than for the case of ion bi-kappa distributions. It is also shown that the ion-cyclotron instability is decreased if the electrons are described by product-bi-kappa distributions, while electrons described by bi-kappa distributions lead to growth rates which are very similar to those obtained considering a Maxwellian distribution for the electron population.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A new sawtooth control mechanism relying on toroidally propagating ion cyclotron resonance frequency waves: Theory and Joint European Torus tokamak experimental evidence

The sawtooth control mechanism in plasmas employing toroidally propagating ion cyclotron resonance waves is extended. The asymmetrically distributed energetic passing ions are shown to modify the ideal internal kink mode when the position of the minority ion cyclotron resonance resides within a narrow region close to the q=1 surface. An analytical treatment of the internal kink mode in the presence of model distribution function with parallel velocity asymmetry is developed. The fast ion mechanism explains the strong sensitivity of sawteeth to resonance position, and moreover is consistent with dedicated Joint European Torus [F. Romanelli, Nucl. Fusion 49, 104006 (2009)] experiments which controlled sawteeth despite negligible current drive.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Evidence for specularly reflected ions upstream from the quasi-parallel bow shock

Ion velocity distributions in the form of bunches of gyrating particles traveling along helical paths have been observed moving sunward immediately upstream from quasi-parallel parts of the earth's bow shock using Los Alamos/Garching instruments on ISEE-1 and -2. These distributions have characteristics which indicate that they are produced by the nearly specular reflection at the shock of a portion of the incident solar wind ions. In particular, the guiding center motion and the gyrospeeds of the gyrating ions are quantitatively consistent with simple geometrical considerations for specular reflection. These considerations reveal that specularly reflected ions can escape upstream when the angle between the upstream magnetic field and the local shock normal is less than 45 deg but not when the angle is greater than 45 deg. These upstream gyrating ions are an important signature of one of the processes by which solar wind streaming energy is dissipated into other forms of energy at the shock.

Gosling, J. T.↗

A Game-Theoretic Data-Driven Approach for Pseudo-Measurement Generation in Distribution System State Estimation

In this paper, we present an efficient computational framework with the purpose of generating weighted pseudo-measurements to improve the quality of distribution system state estimation (DSSE) and provide observability with advanced metering infrastructure (AMI) against unobservable customers and missing data. The proposed technique is based on a game-theoretic expansion of relevance vector machines (RVMs). This platform is able to estimate the nodal power consumption and quantify its uncertainty while reducing the prohibitive computational burden of model training for large AMI datasets. To achieve this objective, the large training set is decomposed and distributed among multiple parallel learning entities. The resulting estimations from the parallel RVMs are then combined using a game-theoretic model based on the idea of repeated games with vector payoff. It is observed that through this approach and by exploiting the seasonal changes in customers’ behavior the accuracy of pseudo-measurements can be considerably improved, while introducing robustness against bad training data samples. The proposed pseudo-measurement generation model is integrated into a DSSE using a closed-loop information system, which takes advantage of a branch current state estimator (BCSE) to further improve the performance of the designed machine learning framework. This method has been tested on a practical distribution feeder model with smart meter data for verification.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Tomosaic : efficient acquisition and reconstruction of teravoxel tomography data using limited-size synchrotron X-ray beams

X-rays offer high penetration with the potential for tomography of centimetre-sized specimens, but synchrotron beamlines often provide illumination that is only millimetres wide. Here an approach is demonstrated termed Tomosaic for tomographic imaging of large samples that extend beyond the illumination field of view of an X-ray imaging system. This includes software modules for image stitching and calibration, while making use of existing modules available in other packages for alignment and reconstruction. The approach is compatible with conventional beamline hardware, while providing a dose-efficient method of data acquisition. By using parallelization on a distributed computing system, it provides a solution for handling teravoxel-sized or larger datasets that cannot be processed on a single workstation in a reasonable time. Using experimental data, the package is shown to provide good quality three-dimensional reconstruction for centimetre-sized samples with sub-micrometre pixel size.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Dense LU Factorization on Multicore Supercomputer Nodes

Dense LU factorization is a prominent benchmark used to rank the performance of supercomputers. Many implementations, including the reference code HPL, use block-cyclic distributions of matrix blocks onto a two-dimensional process grid. The process grid dimensions drive a trade-off between communication and computation and are architecture- and implementation-sensitive. We show how the critical panel factorization steps can be made less communication-bound by overlapping asynchronous collectives for pivot identification and exchange with the computation of rank-k updates. By shifting this trade-off, a modified block-cyclic distribution can beneficially exploit more available parallelism on the critical path, and reduce panel factorization's memory hierarchy contention on now-ubiquitous multi-core architectures. The missed parallelism in traditional block-cyclic distributions arises because active panel factorization, triangular solves, and subsequent broadcasts are spread over single process columns or rows (respectively) of the process grid. Increasing one dimension of the process grid decreases the number of distinct processes in the other dimension. To increase parallelism in both dimensions, periodic 'rotation' is applied to the process grid to recover the row-parallelism lost by a tall process grid. During active panel factorization, rank-1 updates stream through memory with minimal reuse. In a column-major process grid, the performance of this access pattern degrades as too many streaming processors contend for access to memory. A block-cyclic mapping in the more popular row-major order does not encounter this problem, but consequently sacrifices node and network locality in the critical pivoting steps. We introduce 'striding' to vary between the two extremes of row- and column-major process grids. As a test-bed for further mapping experiments, we describe a dense LU implementation that allows a block distribution to be defined as a general function of block to processor. Other mappings can be tested with only small, local changes to the code.

99 GENERAL AND MISCELLANEOUS↗

Block-Parallel Data Analysis with DIY2

DIY2 is a programming model and runtime for block-parallel analytics on distributed-memory machines. Its main abstraction is block-structured data parallelism: data are decomposed into blocks; blocks are assigned to processing elements (processes or threads); computation is described as iterations over these blocks, and communication between blocks is defined by reusable patterns. By expressing computation in this general form, the DIY2 runtime is free to optimize the movement of blocks between slow and fast memories (disk and flash vs. DRAM) and to concurrently execute blocks residing in memory with multiple threads. This enables the same program to execute in-core, out-of-core, serial, parallel, single-threaded, multithreaded, or combinations thereof. This paper describes the implementation of the main features of the DIY2 programming model and optimizations to improve performance. DIY2 is evaluated on benchmark test cases to establish baseline performance for several common patterns and on larger complete analysis codes running on large-scale HPC machines.

97 MATHEMATICS AND COMPUTING↗

Exploring Anomalous Photoelectron Angular Distributions in the Photoelectron Spectra of Gd 3 O 3 – : Study of Gd 3 O 2 – and Gd 3 O 3 – Using Photoelectron Spectroscopy and Density Functional Theory Calculations

Anion photoelectron (PE) spectra of lanthanide oxide clusters obtained previously have exhibited anomalous photoelectron angular distributions which were attributed to strong PE–valence electron (PEVE) interactions. Here, to further explore this effect, we have obtained the PE spectra of Gd 3 O 2 – and Gd 3 O 3 – , two clusters that have similarly complex electronic structures but contrasting symmetries. The spectra exhibit manifolds of detachment transitions at similar binding energies in a 0.5 eV window of energy. The electron affinity of Gd 3 O 2 is measured to be 1.29 ± 0.05 eV, and that of Gd 3 O 3 is 1.31 ± 0.05 eV. As seen in previous studies on lanthanide oxide cluster anions in lower than conventional oxidation states, transitions in spectra obtained lower photon energies are more congested than those obtained with higher photon energy, a signature of strong PEVE interactions. While the detachment transitions have predominantly parallel photoelectron angular distributions (PAD), the PAD varies across the manifold of transitions in the PE spectrum of Gd 3 O 3 – in a way that suggests four different subgroups of transitions. Results of calculations on Gd 3 O 2 – suggest kite or V-shape structures with antiferromagnetic coupling between one of the 4f 7 subshells with the two others. Calculations on Gd 3 O 3 – more definitively point to ring structures with a nearly isoenergetic ferromagnetically coupled high spin (24-tet) state and a dectet state in which one of the 4f 7 subshells is antiferromagnetically coupled with the other two. Taking these results as qualitative, we propose that strong mixing between the unperturbed states predicted computationally leads to overlapping transitions with different PADs.

anions↗

Implementation of Parallel Dynamic Simulation on Shared-Memory vs. Distributed-Memory Environments

Power system dynamic simulation computes the system response to a sequence of large disturbance, such as sudden changes in generation or load, or a network short circuit followed by protective branch switching operation. It consists of a large set of differential and algebraic equations, which is computational intensive and challenging to solve using single-processor based dynamic simulation solution. High-performance computing (HPC) based parallel computing is a very promising technology to speed up the computation and facilitate the simulation process. This paper presents two different parallel implementations of power grid dynamic simulation using Open Multi-processing (OpenMP) on shared-memory platform, and Message Passing Interface (MPI) on distributed-memory clusters, respectively. The difference of the parallel simulation algorithms and architectures of the two HPC technologies are illustrated, and their performances for running parallel dynamic simulation are compared and demonstrated.

dynamic simulation↗

Storing files in a parallel computing system based on user-specified parser function

Techniques are provided for storing files in a parallel computing system based on a user-specified parser function. A plurality of files generated by a distributed application in a parallel computing system are stored by obtaining a parser from the distributed application for processing the plurality of files prior to storage; and storing one or more of the plurality of files in one or more storage nodes of the parallel computing system based on the processing by the parser. The plurality of files comprise one or more of a plurality of complete files and a plurality of sub-files. The parser can optionally store only those files that satisfy one or more semantic requirements of the parser. The parser can also extract metadata from one or more of the files and the extracted metadata can be stored with one or more of the plurality of files and used for searching for files.

97 MATHEMATICS AND COMPUTING↗