SEARCH · Search NASA
Results for “distributed parallelization”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
TorchBraid: High-Performance Layer-Parallel Training of Deep Neural Networks with MPI and GPU Acceleration
TorchBraid is a high-performance implementation of layer-parallel training for deep neural networks (DNNs) supporting MPI-based parallelism and GPU acceleration. Layer-parallel training has been developed to overcome the serialization inherent in forward and backward propagation of DNNs that limits utilization of computational resources in the strong scaling limit. To achieve this, TorchBraid integrates the PyTorch neural network framework with the state-of-the-art XBraid time-parallel library. Furthermore, this article presents the use and performance of TorchBraid, in addition to solutions for overcoming the algorithmic challenges inherent in combining automatic differentiation with layer-parallel. Results are presented with and without GPU acceleration for the Tiny ImageNet and MNIST image classification data sets, as well as recurrent neural networks. Overall, TorchBraid enables fast training of DNNs, both in a strong and weak scaling context. In addition to the TorchBraid software, several new advances in applying layer-parallel algorithms are detailed. Integration of layer-parallel with data-parallel algorithms is presented for the first time, showing the computational advantages of the combination. Standard deep learning techniques, like batch-normalization, are developed for layer-parallel training. Finally, a new approach combining layer-parallel with spatial coarsening in order to accelerate training for 3D image classification shows roughly a 10× speedup over serial execution.
Pebble-bed pebble motion: Simulation and Applications
Pebble bed reactors (PBR) have moving graphite fuel pebbles. This unique feature provides advantages, but also means that simulation of the reactor requires understanding the typical motion and location of the granular flow of pebbles. This report presents a method for simulation of motion of the pebbles in a PBR. A new mechanical motion simulator, PEBBLES, efficiently simulates the key elements of motion of the pebbles in a PBR. This model simulates gravitational force and contact forces including kinetic and true static friction. It's used for a variety of tasks including simulation of the effect of earthquakes on a PBR, calculation of packing fractions, Dancoff factors, pebble wear and the pebble force on the walls. The simulator includes a new differential static friction model for the varied geometries of PBRs. A new static friction benchmark was devised via analytically solving the mechanics equations to determine the minimum pebble-to-pebble friction and pebble-to-surface friction for a five pebble pyramid. This pyramid check as well as a comparison to the Janssen formula was used to test the new static friction equations. Because larger pebble bed simulations involve hundreds of thousands of pebbles and long periods of time, the PEBBLES code has been parallelized. PEBBLES runs on shared memory architectures and distributed memory architectures. For the shared memory architecture, the code uses a new O(n) lock-less parallel collision detection algorithm to determine which pebbles are likely to be in contact. The new collision detection algorithm improves on the traditional non-parallel O(n log(n)) collision detection algorithm. These features combine to form a fast parallel pebble motion simulation. The PEBBLES code provides new capabilities for understanding and optimizing PBRs. The PEBBLES code has provided the pebble motion data required to calculate the motion of pebbles during a simulated earthquake. The PEBBLES code provides the ability to determine the contact forces and the lengths of motion in contact. This information combined with the proper wear coefficients can be used to determine the dust production from mechanical wear. These new capabilities enhance the understanding of PBRs, and the capabilities of the code will allow future improvements in understanding.
Hybrid Simulation of Proton Cyclotron Waves Upstream of Mars Generated by Pickup Ion Beam Distribution
Linear instability analysis as well as a corresponding two‐dimensional hybrid simulation are performed to examine the excitation of the proton cyclotron waves observed upstream of Mars. The waves are believed to be excited by the pickup ions produced from the ionization of the Martian hydrogen exosphere. And previous statistical analysis of wave observations suggested that the waves are mostly related to pickup ion beam velocity distributions. While earlier linear instability analysis of pickup ion beam distributions has mainly been focused on the parallel unstable modes, our analysis reveals that the maximum growth rate occurs at very oblique propagation. The corresponding hybrid simulation confirms the linear analysis results and further demonstrates that the pickup ions are scattered toward an isotropic shell velocity distribution by the waves excited. Interestingly, the waves at oblique propagation gradually damp out and the system is eventually dominated by waves of quasi‐parallel propagation.
Impact of Module Configuration on Lithium-Ion Battery Performance and Degradation: Part I. Energy Throughput, Voltage Spread, and Current Distribution
Batteries are commonly connected in series and parallel to create modules that fulfill the power and energy requirements of specific applications. However, conclusions about battery performance and degradation under different conditions, as well as predictive models, are often derived from single cell cycling results. In this study, we evaluate the performance of six different series-parallel configurations of commercial lithium nickel manganese cobalt cells over hundreds of cycles. Each cell within the modules was individually instrumented for voltage, current, and temperature monitoring. We quantified the impact of module configuration on overall energy throughput, the voltage spread among series-connected cells, and the current heterogeneity in parallel-connected cells. This module cycling study, one of the broadest reported to date, supports systematic evaluation of the performance trade-offs, pack penalty, and safety implications of different module configurations.
SMALL-SCALE PRESSURE-BALANCED STRUCTURES DRIVEN BY OBLIQUE SLOW MODE WAVES MEASURED IN THE SOLAR WIND
Recently, small-scale pressure-balanced structures (PBSs) were identified in the solar wind, but their formation mechanism remains unclear. This work aims to reveal the dependence of the properties of small-scale PBSs on the background magnetic field (B{sub 0}) direction and thus to corroborate the in situ mechanism that forms them. We analyze the plasma and magnetic field data obtained by WIND in the quiet solar wind at 1 AU. First, we use a developed moving-average method to obtain B{sub 0}(s, t) for every temporal scale (s) at each time moment (t). By wavelet cross-coherence analysis, we obtain the correlation coefficients between the thermal pressure P{sub th} and the magnetic pressure P{sub B}, distributing against the temporal scale and the angle {theta}{sub xB} between B{sub 0}(s, t) and Geocentric Solar Ecliptic coordinates (GSE)-x. We note that the angle coverage of a PBS decreases with shorter temporal scale, but the occurrence of the PBSs is independent of {theta}{sub xB}. Suspecting that the isolated small PBSs are formed by compressive waves in situ, we continue this study by testing the wave modes forming a small-scale PBS with B{sub 0}(s, t) quasi-parallel to GSE-x. As a result, we identify that the cross-helicity and the compressibility attain values for a slow mode from theoretical calculations. The wave vector is derived from minimum variance analysis. Besides, the proton temperatures obey T < T{sub Parallel-To} derived from the velocity distribution functions, excluding a mirror mode, which is the other candidate for the formation of PBSs in situ. Thus, a small-scale PBS is shown to be driven by oblique, slow-mode waves in the solar wind.
A communication-avoiding, hybrid-parallel, rank-revealing orthogonalization method.
Orthogonalization consumes much of the run time of many iterative methods for solving sparse linear systems and eigenvalue problems. Commonly used algorithms, such as variants of Gram-Schmidt or Householder QR, have performance dominated by communication. Here, 'communication' includes both data movement between the CPU and memory, and messages between processors in parallel. Our Tall Skinny QR (TSQR) family of algorithms requires asymptotically fewer messages between processors and data movement between CPU and memory than typical orthogonalization methods, yet achieves the same accuracy as Householder QR factorization. Furthermore, in block orthogonalizations, TSQR is faster and more accurate than existing approaches for orthogonalizing the vectors within each block ('normalization'). TSQR's rank-revealing capability also makes it useful for detecting deflation in block iterative methods, for which existing approaches sacrifice performance, accuracy, or both. We have implemented a version of TSQR that exploits both distributed-memory and shared-memory parallelism, and supports real and complex arithmetic. Our implementation is optimized for the case of orthogonalizing a small number (5-20) of very long vectors. The shared-memory parallel component uses Intel's Threading Building Blocks, though its modular design supports other shared-memory programming models as well, including computation on the GPU. Our implementation achieves speedups of 2 times or more over competing orthogonalizations. It is available now in the development branch of the Trilinos software package, and will be included in the 10.8 release.
MPI-hybrid Parallelism for Volume Rendering on Large, Multi-core Systems
This work studies the performance and scalability characteristics of"hybrid'" parallel programming and execution as applied to raycasting volume rendering -- a staple visualization algorithm -- on a large, multi-core platform. Historically, the Message Passing Interface (MPI) has become the de-facto standard for parallel programming and execution on modern parallel systems. As the computing industry trends towards multi-core processors, with four- and six-core chips common today and 128-core chips coming soon, we wish to better understand how algorithmic and parallel programming choices impact performance and scalability on large, distributed-memory multi-core systems. Our findings indicate that the hybrid-parallel implementation, at levels of concurrency ranging from 1,728 to 216,000, performs better, uses a smaller absolute memory footprint, and consumes less communication bandwidth than the traditional, MPI-only implementation.
Hybrid Parallelism for Volume Rendering on Large, Multi-core Systems
This work studies the performance and scalability characteristics of"hybrid'"parallel programming and execution as applied to raycasting volume rendering -- a staple visualization algorithm -- on a large, multi-core platform. Historically, the Message Passing Interface (MPI) has become the de-facto standard for parallel programming and execution on modern parallel systems. As the computing industry trends towards multi-core processors, with four- and six-core chips common today and 128-core chips coming soon, we wish to better understand how algorithmic and parallel programming choices impact performance and scalability on large, distributed-memory multi-core systems. Our findings indicate that the hybrid-parallel implementation, at levels of concurrency ranging from 1,728 to 216,000, performs better, uses a smaller absolute memory footprint, and consumes less communication bandwidth than the traditional, MPI-only implementation.
Hybrid Parallelism for Volume Rendering on Large, Multi-core Systems
This work studies the performance and scalability characteristics of"hybrid" parallel programming and execution as applied to raycasting volume rendering -- a staple visualization algorithm -- on a large, multi-core platform. Historically, the Message Passing Interface (MPI) has become the de-facto standard for parallel programming and execution on modern parallel systems. As the computing industry trends towards multi-core processors, with four- and six-core chips common today and 128-core chips coming soon, we wish to better understand how algorithmic and parallel programming choices impact performance and scalability on large, distributed-memory multi-core systems. Our findings indicate that the hybrid-parallel implementation, at levels of concurrency ranging from 1,728 to 216,000, performs better, uses a smaller absolute memory footprint, and consumes less communication bandwidth than the traditional, MPI-only implementation.
Extension of the Bissell-Johnson plasma-sheath model for application to fusion-relevant and general plasmas
This article presents an approach to solving a special Fredholm-type integral equation of the first kind with a particular kernel containing a modified Bessel function for applications in plasma physics. From the physical point of view, the problem was defined by Bissell and Johnson (B and J) [Phys. Fluids 30, 779 (1987)] as a task to find the potential profile and the ion velocity distribution function in a plane-parallel discharge with a Maxwellian ion source. The B and J model is a generalization of the well-known Tonks-Langmuir (T and L) [Phys. Rev. 34, 876 (1929)] discharge model characterized by a ''cold'' ion source. Unlike the T and L model, which can be readily solved analytically, attempts to solve the B and J model with a ''warm'' ion source have been done only numerically. However, the validity of numerical solutions up to date remains constrained to a rather limited range of a crucial independent parameter of the B and J integral equation, which mathematically is the width of a Gaussian distribution and physically represents the ion temperature. It was solved only for moderately warm ion sources. This paper presents the exact numerical solution of the B and J model, which is valid without any restriction regarding the above-mentioned parameter. It is shown that the ion temperature is very different from the temperature of the ion source. The new results with high-temperature ion sources are not only of particular importance for understanding and describing the plasma-sheath boundary in fusion plasmas, but are of considerable interest for discharge problems in general. The eigenvalue of the problem, found analytically by Harrison and Thompson [Proc. Phys. Soc. 74, 145 (1959)] for the particular case of a cold ion source, is here extended to arbitrary ion-source temperatures.
PaKman: Scalable Assembly of Large Genomes on Distributed Memory Machines
De novo genome assembly is a fundamental problem in the field of bioinformatics, that aims to assemble the DNA sequence of an unknown genome from numerous short DNA fragments (aka reads) obtained from it. With the advent of high-throughput sequencing technologies, billions of reads can be generated in a matter of hours, necessitating efficient parallelization of the assembly process. While multiple parallel solutions have been proposed in the past, conducting a large- scale assembly at scale remains a challenging problem because of the inherent complexities associated with data movement, and irregular access footprints of memory and I/O operations. In this paper, we present a novel algorithm, called PaKman, to address the problem of performing large-scale genome assemblies on a distributed memory parallel computer. Our approach focuses on improving performance through a combination of novel data structures and algorithmic strategies for reducing the communication and I/O footprint during the assembly process. A key aspect of our algorithm is its graph data structure, which comprises fat nodes (or what we call "macro-nodes") that reduce the communication burden during contig generation. We present an extensive performance and qualitative evaluation of our algorithm, including comparisons to other state-of-the-art parallel assemblers. Our results demonstrate the ability to achieve near-linear speedups on up to 8K cores (tested); outperform state-of-the-art distributed memory and shared memory tools in performance while delivering comparable (if not better) quality; and reduce time to solution significantly. For instance, PaKman is able to complete an assembly of the full human genome in just over a minute on 8K cores.
Massively parallel modeling and inversion of electrical resistivity tomography data using PFLOTRAN
Abstract. Electrical resistivity tomography (ERT) is a broadly accepted geophysical method for subsurface investigations. Interpretation of field ERT data usually requires the application of computationally intensive forward modeling and inversion algorithms. For large-scale ERT data, the efficiency of these algorithms depends on the robustness, accuracy, and scalability on high-performance computing resources. In this regard, we present a robust and highly scalable implementation of forward modeling and inversion algorithms for ERT data. The implementation is publicly available and developed within the framework of PFLOTRAN, an open-source, state-of-the-art massively parallel subsurface flow and transport simulation code. The forward modeling is based on a finite-volume discretization of the governing differential equations, and the inversion uses a Gauss–Newton optimization scheme. To evaluate the accuracy of the forward modeling, two examples are first presented by considering layered (1D) and 3D earth conductivity models. The computed numerical results show good agreement with the analytical solutions for the layered earth model and results from a well-established code for the 3D model. Inversion of ERT data, simulated for a 3D model, is then performed to demonstrate the inversion capability by recovering the conductivity of the model. To demonstrate the parallel performance of PFLOTRAN's ERT process model and inversion capabilities, large-scale scalability tests are performed by using up to 131 072 processes on a leadership class supercomputer. These tests are performed for the two most computationally intensive steps of the ERT inversion: forward modeling and Jacobian computation. For the forward modeling, we consider models with up to 122 ×106 degrees of freedom (DOFs) in the resulting system of linear equations and demonstrate that the code exhibits almost linear scalability on up to 10 000 DOFs per process. On the other hand, the code shows superlinear scalability for the Jacobian computation, mainly because all computations are fairly evenly distributed over each process with no parallel communication.
A thermal porosimetry method to estimate pore size distribution in highly porous insulating materials
Standard pore size determination methods such as mercury porosimetry, nitrogen sorption, microscopy, or x-ray tomography are not always applicable to highly porous, low density, and thus very fragile materials. For this kind of materials, a method based on thermal characterization is proposed. Indeed, the thermal conductivity of a highly porous and insulating medium is significantly dependent on the thermal conductivity of the interstitial gas that depends on both gas pressure and size of the considered pore (Knudsen effect). It is also possible to link the pore size with the thermal conductivity of the medium. Thermal conductivity measurements are realized on specimens placed in an enclosure where the air pressure is successively set to different values varying from 10{sup -1} to 10{sup 5} Pa. Knowing the global porosity ratio, an effective thermal conductivity model for a two-phase air-solid material based on a combined serial-parallel model is established. Pore size distribution can be identified by minimizing the sum of the quadratic differences between measured values and modeled ones. The results of the estimation process are the volume fractions of the chosen ranges of pore size. In order to validate the method, measurements done on insulating materials are presented. The results are discussed and show that pore size distribution estimated by the proposed method is coherent.
Steady-state benchmarks of DK4D: A time-dependent, axisymmetric drift-kinetic equation solver
The DK4D code has been written to solve a set of time-dependent, axisymmetric, finite-Larmor-radius drift-kinetic equations (DKEs) for the non-Maxwellian part of the electron and ion distribution functions using the full, linearized Fokker–Planck–Landau collision operator. The plasma is assumed to be in the low- to finite-collisionality regime, as is found in the cores of modern and future magnetic confinement fusion experiments. Each DKE is formulated such that the perturbed distribution function carries no net density, parallel momentum, or kinetic energy. Rather, these quantities are contained within the background Maxwellians and would be evolved by an appropriate set of extended magnetohydrodynamic (MHD) equations. This formulation allows for straight-forward coupling of DK4D to existing extended MHD time evolution codes. DK4D uses a mix of implicit and explicit temporal representations and finite element and spectral spatial representations. These, along with other computational methods used, are discussed extensively. Steady-state benchmarks are then presented comparing the results of DK4D to expected analytic results at low collisionality, qualitatively, and to the Sauter analytic fits for the neoclassical conductivity and bootstrap current, quantitatively. These benchmarks confirm that DK4D is capable of solving for the correct, gyroaveraged distribution function in stationary magnetic equilibria. Furthermore, the results presented demonstrate how the exact drift-kinetic solution varies with collisionality as a function of the magnetic moment and the poloidal angle.
Heating and acceleration of ions with Kappa distribution functions by low‐frequency Alfvén wave
Abstract Heating and acceleration of ions with Kappa distribution functions (with parameter ) in low‐beta plasmas, by a low‐frequency Alfvén wave, is investigated using test‐particle simulations, yielding interesting new results. As long as the Alfvén wave amplitude is sufficiently large, the computed net heating energy of ions becomes independent of the wave frequency and amplitude, always approaching the same value of . The eventual energy of ions is dictated only by the initial ion energy and the ratio of the magnetic field energy density to the plasma density. The heating effect of the Kappa ions increases with . During the heating process, the ions are picked up by the Alfvén wave and pitch angle scattered, forming a quasi‐isotropic spherical shell velocity distribution. The Kappa ions are accelerated in the parallel direction, reaching a bulk flow speed roughly equal to the local Alfvén speed. Higher ‐value in the initial Kappa distribution leads to faster saturation. The above results may explain certain features of the ion heating and acceleration in the solar wind and corona.
Klystron Cluster Scheme for ILC High Power RF Distribution
We present a concept for powering the main linacs of the International Linear Collider (ILC) by delivering high power RF from the surface via overmoded, low-loss waveguides at widely spaced intervals. The baseline design employs a two-tunnel layout, with klystrons and modulators evenly distributed along a service tunnel running parallel to the accelerator tunnel. This new idea eliminates the need for the service tunnel. It also brings most of the warm heat load to the surface, dramatically reducing the tunnel water cooling and HVAC requirements. In the envisioned configuration, groups of 70 klystrons and modulators are clustered in surface buildings every 2.5 km. Their outputs are combined into two half-meter diameter circular TE{sub 01} mode evacuated waveguides. These are directed via special bends through a deep shaft and along the tunnel, one upstream and one downstream. Each feeds approximately 1.25 km of linac with power tapped off in 10 MW portions at 38 m intervals. The power is extracted through a novel coaxial tap-off (CTO), after which the local distribution is as it would be from a klystron. The tap-off design is also employed in reverse for the initial combining.
Improving Block-level Efficiency with scsi-mq
Current generation solid-state storage devices are exposing a new bottlenecks in the SCSI and block layers of the Linux kernel, where IO throughput is limited by lock contention, inefficient interrupt handling, and poor memory locality. To address these limitations, the Linux kernel block layer underwent a major rewrite with the blk-mq project to move from a single request queue to a multi-queue model. The Linux SCSI subsystem rework to make use of this new model, known as scsi-mq, has been merged into the Linux kernel and work is underway for dm-multipath support in the upcoming Linux 4.0 kernel. These pieces were necessary to make use of the multi-queue block layer in a Lustre parallel filesystem with high availability requirements. We undertook adding support of the 3.18 kernel to Lustre with scsi-mq and dm-multipath patches to evaluate the potential of these efficiency improvements. In this paper we evaluate the block-level performance of scsi-mq with backing storage hardware representative of a HPC-targerted Lustre filesystem. Our findings show that SCSI write request latency is reduced by as much as 13.6%. Additionally, when profiling the CPU usage of our prototype Lustre filesystem, we found that CPU idle time increased by a factor of 7 with Linux 3.18 and blk-mq as compared to a standard 2.6.32 Linux kernel. Our findings demonstrate increased efficiency of the multi-queue block layer even with disk-based caching storage arrays used in existing parallel filesystems.