Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Hydromagnetic waves and instabilities in kappa distribution plasma

Stability properties of hydromagnetic waves (shear and compressional Alfven waves) in spatially homogeneous plasma are investigated when the equilibrium particle velocity distributions in both parallel and perpendicular directions (in reference to the ambient magnetic field) are modeled by kappa distributions. Analysis is presented for the limiting cases |{xi}{sub {alpha}}|<<1 and |{xi}{sub {alpha}}|>>1 for which solutions of the dispersion relations are analytically tractable. Here {xi}{sub {alpha}}({alpha}=e,i) is the ratio of the wave phase speed and the electron (ion) thermal speed. Both low and high {beta} (=plasma pressure/magnetic pressure) plasmas are considered. The distinguishing features of the hydromagnetic waves in kappa distribution plasma are (1) both Landau damping and transit-time damping rates are larger than those in Maxwellian plasma because of the enhanced high-energy tail of the kappa distribution and (2) density and temperature perturbations in response to the electromagnetic perturbations are different from those in Maxwellian plasma when |{xi}{sub {alpha}}|<<1. Moreover, frequency of the oscillatory stable modes (e.g., kinetic shear Alfven wave) and excitation condition of the nonoscillatory (zero frequency) unstable modes (e.g., mirror instability) in kappa distribution plasma are also different from those in Maxwellian plasma. Quantitative estimates of the differences depend on the specific choice of the kappa distribution. For simplicity of notations, same spectral indices {kappa}{sub ||} and {kappa}{sub perpendicular} have been assumed for both electron and ion population. However, the analysis can be easily generalized to allow for different values of the spectral indices for the two charged populations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A 2D simulation study of Langmuir, whistler, and cyclotron maser instabilities induced by an electron ring-beam distribution

We carried out a series of 2D simulations to study the beam instability and cyclotron maser instability (CMI) with the initial condition that a population of tenuous energetic electrons with a ring-beam distribution is present in a magnetized background plasma. In this paper, weakly relativistic cases are discussed with the ring-beam kinetic energy ranging from 25 to 100 keV. The beam component leads to the two-stream or beam instability at an earlier stage, and the beam mode is coupled with Langmuir or whistler mode, leading to excitation of beam-Langmuir or beam-whistler waves. When the beam velocity is large with a strong beam instability, the initial ring-beam distribution is diffused in the parallel direction rapidly. The diffused distribution may still support CMI to amplify the X1 mode (the fundamental X mode). On the contrary, when the beam velocity is small and the beam instability is weak, CMI can amplify the Z1 (the fundamental Z mode) effectively while the O1 (the fundamental O mode) and X2 (the second harmonic X mode) modes are very weak and the X1 mode is not excited. In this report, different cases with various parameters are presented and discussed for a comprehensive understanding of ring-beam instabilities.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Measurement of PVFS2 performance on InfiniBand

InfiniBand is becoming increasingly popular as a fast interconnect technology between servers and storage. It has far better price/performance ratio compared to both Gigabit Ethernet and 10 Gigabit Ethernet, and hence is being increasingly used for highperformance computing applications. PVFS2, the second generation Parallel Virtual File System (PVFS), is a distributed file system for parallel data access that is being increasingly used in clustered applications. As previous studies have shown, in general, PVFS2 over InfiniBand offers enhanced I/O rates compared to PVFS2 over TCP and Gigabit Ethernet. Apart from the hardware technology, the application programming interface into the file system also makes a difference. To get better parallel performance, the choice of a file system interface is important. Our study is to benchmark and compare the performance of PVFS2 running over InfiniBand using different file system interfaces. IOR is a popular I/O benchmarking tool that supports the POSIX and MPI I/O file system interfaces. In addition to testing these already supported interfaces, we have written a PVFS2 module extension for IOR to support native PVFS2 interfaces into the PVFS2 file system. As we shall see in this study, using native PVFS2 interface offers significant performance benefit compared to other file system interfaces on the PVFS2 file system. Our benchmarking effort also involves studying the effect of a multi-client environment on the I/O performance of different file system interfaces. Based on the benchmarking results we obtain, we determine the most efficient application programming interface for parallel I/O on PVFS2 in a typical multi-client parallel application scenario.

97 MATHEMATICS AND COMPUTING↗

Porting AMG2013 to Heterogeneous CPU+GPU Nodes

LLNL's future advanced technology system SIERRA will feature heterogeneous compute nodes that consist of IBM PowerV9 CPUs and NVIDIA Volta GPUs. Conceptually, the motivation for such an architecture is quite straightforward: While GPUs are optimized for throughput on massively parallel workloads, CPUs strive to minimize latency for rather sequential operations. Yet, making optimal use of heterogeneous architectures raises new challenges for the development of scalable parallel software, e.g., with respect to work distribution. Porting LLNL's parallel numerical libraries to upcoming heterogeneous CPU+GPU architectures is therefore a critical factor for ensuring LLNL's future success in ful lling its national mission. One of these libraries, called HYPRE, provides parallel solvers and precondi- tioners for large, sparse linear systems of equations. In the context of this intern- ship project, I consider AMG2013 which is a proxy application for major parts of HYPRE that implements a benchmark for setting up and solving di erent systems of linear equations. In the following, I describe in detail how I ported multiple parts of AMG2013 to the GPU (Section 2) and present results for di erent experiments that demonstrate a successful parallel implementation on the heterogeneous ma- chines surface and ray (Section 3). In Section 4, I give guidelines on how my code should be used. Finally, I conclude and give an outlook for future work (Section 5).

97 MATHEMATICS AND COMPUTING↗

CG-Kit: Code Generation Toolkit for performant and maintainable variants of source code applied to Flash-X hydrodynamics simulations

CG-Kit is a new Code Generation tool-Kit that we have developed as a part of the solution for portability and maintainability for multiphysics computing applications. The development of CG-Kit is rooted in the urgent need created by the shifting landscape of high-performance computing platforms and the algorithmic complexities of a particular large-scale multiphysics application: Flash-X. To efficiently use computing resources on a heterogeneous node, an application must have a map of computation to resources and a mechanism to move the data and computation to the resources according to the map. Most existing performance portability solutions are focussed on abstracting the expression of computations so that a unified source code can be specialized to run on different resources. However, such an approach is insufficient for a code like Flash-X, which has a multitude of code components that can be assembled in various permutations and combinations to form different instances of applications. Similar challenges apply to any code that has composability, where a single specified way of apportioning work among devices may not be optimal. Additionally, use cases arise where the optimal control flow of computation may differ for different devices while the underlying numerics remain identical. This combination leads to unique challenges including handling an existing large code base in Fortran and/or C/C++, subdivision of code into a great variety of units supporting a wide range of physics and numerical methods, different parallelization techniques for distributed and shared memory systems and accelerator devices, and heterogeneity of computing platforms requiring coexisting variants of parallel algorithms. All of these challenges demand that scientific software developers apply existing knowledge about domain applications, algorithms, and computing platforms to determine custom abstractions and granularity for code generation. There is a critical lack of tools to tackle those problems. CG-Kit is designed to fill this gap by providing a user with the ability to express their desired control flow and computation-to-resource map in the form a pseudocode-like recipe. It consists of standalone tools that can be combined into highly specific and, we argue, highly effective portability and maintainability toolchains. Here we present the design of our new tools: parametrized source trees, control flow graphs, and recipes. The tools are implemented in Python. They are agnostic to the programming language of the source code targeted for code generation. In conclusion, we demonstrate the capabilities of the toolkit with two examples, first, multithreaded variants of the basic AXPY operation, and second, variants of parallel algorithms within a hydrodynamics solver, called Spark, from Flash-X that operates on block-structured adaptive meshes.

Algorithmic portability↗

Advances in Domain Mapping of Massively Parallel Scientific Computations

One of the most important concerns in parallel computing is the proper distribution of workload across processors. For most scientific applications on massively parallel machines, the best approach to this distribution is to employ data parallelism; that is, to break the datastructures supporting a computation into pieces and then to assign those pieces to different processors. Collectively, these partitioning and assignment tasks comprise the domain mapping problem.

97 MATHEMATICS AND COMPUTING↗

Efficient distributed continual learning for steering experiments in real-time

Deep learning has emerged as a powerful method for extracting valuable information from large volumes of data. However, when new training data arrives continuously (i.e., is not fully available from the beginning), incremental training suffers from catastrophic forgetting (i.e., new patterns are reinforced at the expense of previously acquired knowledge). Training from scratch each time new training data becomes available would result in extremely long training times and massive data accumulation. Rehearsal-based continual learning has shown promise for addressing the catastrophic forgetting challenge, but research to date has not addressed performance and scalability. To fill this gap, we propose an approach based on a distributed rehearsal buffer that efficiently complements data-parallel training on multiple GPUs to achieve high accuracy, short runtime, and scalability. It leverages a set of buffers (local to each GPU) and uses several asynchronous techniques for updating these local buffers in an embarrassingly parallel fashion, all while handling the communication overheads necessary to augment input minibatches using unbiased, global sampling. We further propose a generalization of rehearsal buffers to support both classification and generative learning tasks, as well as more advanced rehearsal strategies (notably Dark Experience Replay, leveraging knowledge distillation). We illustrate this approach with a real-life HPC streaming application from the domain of ptychographic image reconstruction. Furthermore, we run extensive experiments on up to 128 GPUs of the ThetaGPU supercomputer to compare our approach with baselines representative of training-from-scratch (the upper bound in terms of accuracy) and incremental training (the lower bound). Results show that rehearsal-based continual learning achieves a top-5 validation accuracy close to the upper bound, while simultaneously exhibiting a runtime close to the lower bound.

Asynchronous data management↗

Photodissociation dynamics of the methyl perthiyl radical at 248 and 193 nm using fast-beam photofragment translational spectroscopy

The photodissociation dynamics of the methyl perthiyl radical (CH{sub 3}SS) have been investigated using fast-beam coincidence translational spectroscopy. Methyl perthiyl radicals were produced by photodetachment of the CH{sub 3}SS{sup −} anion followed by photodissociation at 248 nm (5.0 eV) and 193 nm (6.4 eV). Photofragment mass distributions and translational energy distributions were measured at each dissociation wavelength. Experimental results show S atom loss as the dominant (96%) dissociation channel at 248 nm with a near parallel, anisotropic angular distribution and translational energy peaking near the maximal energy available to ground state CH{sub 3}S and S fragments, indicating that the dissociation occurs along a repulsive excited state. At 193 nm, S atom loss remains the major fragmentation channel, although S{sub 2} loss becomes more competitive and constitutes 32% of the fragmentation. The translational energy distributions for both channels are very broad at this wavelength, suggesting the formation of the S{sub 2} and S atom products in several excited electronic states.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Processes setting the structure of the electron distribution function within the exhausts of anti-parallel reconnection

In situ spacecraft observations within the exhausts of magnetic reconnection document a large variation in the velocity space structure of the electron distribution function. Multiple mechanisms help govern the underlying electron dynamics, yielding a range of signatures for collisionless reconnection. These signatures include passing beams of electrons separated by well-defined boundaries from betatron heated/cooled trapped electrons. The present study emphasizes how localized regions of non-adiabatic electron dynamics can mix electrons across the trapped/passing boundaries and impact the form of the electron distributions in the full width of the exhaust. Here, while our study is based on 2D simulations, the described principles shaping the velocity space distributions also apply to 3D geometries making our findings relevant to spacecraft observation of reconnection in the Earth's magnetosphere.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

FPDeep: Scalable Acceleration of CNN Training on Deeply-Pipelined FPGA Clusters

In this paper, we propose a framework called FPDeep, which uses a hybrid of model and layer paral- lelism to configure distributed reconfigurable clusters to train DNNs. This approach has numerous benefits. First, the design does not suffer from batch size growth. Second, novel workload and weight partitioning leads to balanced loads of both among nodes. And third, the entire system is fine-grained pipeline. This leads to high parallelism and utilization and also minimizes the time features need to be cached while waiting for back-propagation.

Wang, Tianqi↗

A Massively Parallel Implementation of the CCSD(T) Method Using the Resolution-of-the-Identity Approximation and a Hybrid Distributed/Shared Memory Parallelization Model

In this work, a parallel algorithm is described for the coupled-cluster singles and doubles method augmented with a perturbative correction for triple excitations [CCSD(T)] using the resolution-of-the-identity (RI) approximation for two-electron repulsion integrals (ERIs). The algorithm bypasses the storage of four-center ERIs by adopting an integral-direct strategy. The CCSD amplitude equations are given in a compact quasi-linear form by factorizing them in terms of amplitude-dressed three-center intermediates. A hybrid MPI/OpenMP parallelization scheme is employed, which uses the OpenMP-based shared memory model for intranode parallelization and the MPI-based distributed memory model for internode parallelization. Parallel efficiency has been optimized for all terms in the CCSD amplitude equations. Two different algorithms have been implemented for the rate-limiting terms in the CCSD amplitude equations that entail and -scaling computational costs, where N O and N V denote the number of correlated occupied and virtual orbitals, respectively. One of the algorithms assembles the four-center ERIs requiring N V 4 and N O 2 N V 2 -scaling memory costs in a distributed manner on a number of MPI ranks, while the other algorithm completely bypasses the assembling of quartic memory-scaling ERIs and thus largely reduces the memory demand. It is demonstrated that the former memory-expensive algorithm is faster on a few hundred cores, while the latter memory-economic algorithm shows a better strong scaling in the limit of a few thousand cores. The program is shown to exhibit a near-linear scaling, in particular for the compute-intensive triples correction step, on up to 8000 cores. The performance of the program is demonstrated via calculations involving molecules with 24–51 atoms and up to 1624 atomic basis functions. As the first application, the complete basis set (CBS) limit for the interaction energy of the π-stacked uracil dimer from the S66 data set has been investigated. This work reports the first calculation of the interaction energy at the CCSD(T)/aug-cc-pVQZ level without local orbital approximation. The CBS limit for the CCSD correlation contribution to the interaction energy was found to be -8.01 kcal/mol, which agrees very well with the value -7.99 kcal/mol reported by Schmitz, Hättig, and Tew [ Phys. Chem. Chem. Phys. 2014 , 16 , 22167-22178]. The CBS limit for the total interaction energy was estimated to be -9.64 kcal/mol.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

High-Performance Design Patterns for Modern Fortran

This paper presents ideas for using coordinate-free numerics in modern Fortran to achieve code flexibility in the partial differential equation (PDE) domain. We also show how Fortran, over the last few decades, has changed to become a language well-suited for state-of-the-art software development. Fortran’s new coarray distributed data structure, the language’s class mechanism, and its side-effect-free, pure procedure capability provide the scaffolding on which we implement HPC software. These features empower compilers to organize parallel computations with efficient communication. We present some programming patterns that support asynchronous evaluation of expressions comprised of parallel operations on distributed data. We implemented these patterns using coarrays and the message passing interface (MPI). We compared the codes’ complexity and performance. The MPI code is much more complex and depends on external libraries. The MPI code on Cray hardware using the Cray compiler is 1.5–2 times faster than the coarray code on the same hardware. The Intel compiler implements coarrays atop Intel’s MPI library with the result apparently being 2–2.5 times slower than manually coded MPI despite exhibiting nearly linear scaling efficiency. As compilers mature and further improvements to coarrays comes in Fortran 2015, we expect this performance gap to narrow.

Haveraaen, Magne↗

FOX: A Fault Oblivious Extreme-Scale Execution Environment

The FOX project, funded under the ASCR X-stack I program, developed systems software and runtime libraries for a new approach to the data and work distribution for massively parallel, fault oblivious application execution. Our work was motivated by the premise that exascale computing systems will provide a thousand-fold increase in parallelism and a proportional increase in failure rate relative to today’s machines. To deliver the capability of exascale hardware, the systems software must provide the infrastructure to support existing applications while simultaneously enabling efficient execution of new programming models that naturally express dynamic, adaptive, irregular computation; coupled simulations; and massive data analysis in a highly unreliable hardware environment with billions of threads of execution. Our OS research has prototyped new methods to provide efficient resource sharing, synchronization, and protection in a many-core compute node. We have experimented with alternative task/dataflow programming models and shown scalability in some cases to hundreds of thousands of cores. Much of our software is in active development through open source projects. Concepts from FOX are being pursued in next generation exascale operating systems. Our OS work focused on adaptive, application tailored OS services optimized for multi → many core processors. We developed a new operating system NIX that supports role-based allocation of cores to processes which was released to open source. We contributed to the IBM FusedOS project, which promoted the concept of latency-optimized and throughput-optimized cores. We built a task queue library based on distributed, fault tolerant key-value store and identified scaling issues. A second fault tolerant task parallel library was developed, based on the Linda tuple space model, that used low level interconnect primitives for optimized communication. We designed fault tolerance mechanisms for task parallel computations employing work stealing for load balancing that scaled to the largest existing supercomputers. Finally, we implemented the Elastic Building Blocks runtime, a library to manage object-oriented distributed software components. To support the research, we won two INCITE awards for time on Intrepid (BG/P) and Mira (BG/Q). Much of our work has had impact in the OS and runtime community through the ASCR Exascale OS/R workshop and report, leading to the research agenda of the Exascale OS/R program. Our project was, however, also affected by attrition of multiple PIs. While the PIs continued to participate and offer guidance as time permitted, losing these key individuals was unfortunate both for the project and for the DOE HPC community.

97 MATHEMATICS AND COMPUTING↗

NEUTRAL ATOM PROPERTIES IN THE DIRECTION OF THE IBEX RIBBON

In this paper, we present results from our three-dimensional (3D) simulations of the interaction between the solar wind and local interstellar medium with an emphasis on the phase-space properties of energetic neutral atoms (ENAs) along a sight line that intersects the ribbon of enhanced ENA flux seen by NASA’s Interstellar Boundary EXplorer spacecraft. The majority of these ENAs have velocities directed away from the heliosphere, but it is believed that interactions between heliospheric ENAs and ions outside the heliosphere may result in a population of secondary ENAs that return to the heliosphere and generate the ribbon. While we do not consider the ion dynamics that result in secondary ENAs, our analysis is of key importance to the process since the heliospheric ENAs we consider form the source population for those ions. We present the moments of the hydrogen distribution, along with moments parallel and perpendicular to the local magnetic field for the pick-up ions (PUIs) that these neutrals generate. Finally, we present gyro-averaged velocity distributions relative to the local magnetic field for the PUIs created from our simulated H-atoms, along with analytic fits to these distributions in the secondary ENA source region just beyond the heliopause.

79 ASTRONOMY AND ASTROPHYSICS↗

Lateral distribution of pulse height in a parallel-plate avalanche counter

The nuclear γ-ray spectroscopy enters a new era when the first engineering run of GRETINA [1], a new generation of γ-ray tracking arrays, was successfully carried out at LBNL in early April, 2011. A parallel effort has been setup to develop the auxiliary charged-particle detector arrays with a matching position resolution to fully exploit the potential of GRETINA. Improving the position resolution of the existing charged-particle detector array, CHICO [2], is a part of this coordinated effort. The proposal to upgrade CHICO was approved by DOE/SC at the end of FY10. The goal is to redesign the cathode board and amplifier to improve the resolution for both and coordinates to better than one degree to match that of GRETINA. This project is scheduled to complete by the summer of 2012.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗