MDTM: Optimizing Data Transfer using Multicore-Aware I/O Scheduling [Slides]
Outline - Background & Motivation; MDTM Description & Design; Evaluation of MDTM; Conclusion.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Outline - Background & Motivation; MDTM Description & Design; Evaluation of MDTM; Conclusion.
Streamline computation in a very large vector field data set represents a significant challenge due to the nonlocal and data-dependent nature of streamline integration. In this paper, we conduct a study of the performance characteristics of hybrid parallel programming and execution as applied to streamline integration on a large, multicore platform. With multicore processors now prevalent in clusters and supercomputers, there is a need to understand the impact of these hybrid systems in order to make the best implementation choice. We use two MPI-based distribution approaches based on established parallelization paradigms, parallelize over seeds and parallelize over blocks, and present a novel MPI-hybrid algorithm for each approach to compute streamlines. Our findings indicate that the work sharing between cores in the proposed MPI-hybrid parallel implementation results in much improved performance and consumes less communication and I/O bandwidth than a traditional, nonhybrid distributed implementation.
Based on a genetic algorithm, we have solved in this paper the problem of finding the parameters of optical Gaussian pulses which make their efficient nonlinear combining possible in one of the peripheral cores of a 7-core hexagonal fibre. Two approaches based on individual selection of peak powers and field phases of the pulses launched into the fibre are considered. Finally, the found regimes of Gaussian pulse combining open up new possibilities for the development of devices for controlling optical radiation.
Not provided.
The successful exploitation of multicore processor architectures is a key element of the LHC distributed computing system in the coming era of the LHC Run 2. High-pileup complex-collision events represent a challenge for the traditional sequential programming in terms of memory and processing time budget. The CMS data production and processing framework is introducing the parallel execution of the reconstruction and simulation algorithms to overcome these limitations. CMS plans to execute multicore jobs while still supporting singlecore processing for other tasks difficult to parallelize, such as user analysis. The CMS strategy for job management thus aims at integrating single and multicore job scheduling across the Grid. This is accomplished by employing multicore pilots with internal dynamic partitioning of the allocated resources, capable of running payloads of various core counts simultaneously. An extensive test programme has been conducted to enable multicore scheduling with the various local batch systems available at CMS sites, with the focus on the Tier-0 and Tier-1s, responsible during 2015 of the prompt data reconstruction. Scale tests have been run to analyse the performance of this scheduling strategy and ensure an efficient use of the distributed resources. This paper presents the evolution of the CMS job management and resource provisioning systems in order to support this hybrid scheduling model, as well as its deployment and performance tests, which will enable CMS to transition to a multicore production model for the second LHC run.
In this report we present PPT-Multicore, an analytical model embedded in the Performance Prediction Toolkit (PPT) to predict parallel applications’ performance running on a multicore processor. PPT-Multicore builds upon our previous work towards a multicore cache model. We extract LLVM basic block labeled memory trace using an architecture-independent LLVM-based instrumentation tool only once in an application’s lifetime. The model uses the memory trace and other parameters from an instrumented sequentially executed binary. We use probabilistic and computationally efficient reuse profiles to predict the cache hit rates and runtimes of OpenMP programs’ parallel sections. We model Intel’s Broadwell, Haswell, and AMD’s Zen2 architectures and validate our framework using different applications from PolyBench and PARSEC benchmark suites. The results show that PPT-Multicore can predict cache hit rates with an overall average error rate of 1.23% while predicting the runtime with an error rate of 9.08%.
The algorithmic and implementation principles are explored in gainfully exploiting GPU accelerators in conjunction with multicore processors on high-end systems with large numbers of compute nodes, and evaluated in an implementation of a scalable block tridiagonal solver. The accelerator of each compute node is exploited in combination with multicore processors of that node in performing block-level linear algebra operations in the overall, distributed solver algorithm. Optimizations incorporated include: (1) an efficient memory mapping and synchronization interface to minimize data movement, (2) multi-process sharing of the accelerator within a node to obtain balanced load with multicore processors, and (3) an automatic memory management system to efficiently utilize accelerator memory when sub-matrices spill over the limits of device memory. Results are reported from our novel implementation that uses MAGMA and CUBLAS accelerator software systems simultaneously with ACML for multithreaded execution on processors. Overall, using 940 nVidia Tesla X2090 accelerators and 15,040 cores, the best heterogeneous execution delivers a 10.9-fold reduction in run time relative to an already efficient parallel multicore-only baseline implementation that is highly optimized with intra-node and inter-node concurrency and computation-communication overlap. Detailed quantitative results are presented to explain all critical runtime components contributing to hybrid performance.
Modern hardware contains parallel execution resources that are well-suited for data-parallelism-vector units-and task parallelism-multicores. However, most work on parallel scheduling focuses on one type of hardware or the other. In this work, we present a scheduling framework that allows for a unified treatment of task- and data-parallelism. Our key insight is an abstraction, task blocks, that uniformly handles data-parallel iterations and task-parallel tasks, allowing them to be scheduled on vector units or executed independently as multicores. Our framework allows us to define schedulers that can dynamically select between executing task- blocks on vector units or multicores. We show that these schedulers are asymptotically optimal, and deliver the maximum amount of parallelism available in computation trees. To evaluate our schedulers, we develop program transformations that can convert mixed data- and task-parallel pro- grams into task block-based programs. Using a prototype instantiation of our scheduling framework, we show that, on an 8-core system, we can simultaneously exploit vector and multicore parallelism to achieve 14×-108× speedup over sequential baselines.
Algebraic multigrid (AMG) solvers have proven to be extremely efficient on distributed-memory architectures. However, when executed on modern multicore cluster architectures, we face new challenges that can significantly harm AMG's performance. We discuss our experiences on such an architecture and present a set of techniques that help users to overcome the associated problems, including thread and process pinning and correct memory associations. We have implemented most of the techniques in a MultiCore SUPport library (MCSup), which helps to map OpenMP applications to multicore machines. We present results using both an MPI-only and a hybrid MPI/OpenMP model.
In the next years, processor architectures based on much larger numbers of cores will be most likely the model to continue 'Moore's Law' style throughput gains. This not only results in many more jobs in parallel running the LHC Run 1 era monolithic applications, but also the memory requirements of these processes push the workernode architectures to the limit. One solution is parallelizing the application itself, through forking and memory sharing or through threaded frameworks. CMS is following all of these approaches and has a comprehensive strategy to schedule multicore jobs on the GRID based on the glideinWMS submission infrastructure. The main component of the scheduling strategy, a pilot-based model with dynamic partitioning of resources that allows the transition to multicore or whole-node scheduling without disallowing the use of single-core jobs, is described. This contribution also presents the experiences made with the proposed multicore scheduling schema and gives an outlook of further developments working towards the restart of the LHC in 2015.
Algebraic multigrid (AMG) solvers have proven to be extremely efficient on distributed-memory architectures. However, when executed on modern multicore cluster architectures, we face new challenges that can significantly harm AMG's performance. We discuss our experiences on such an architecture and present a set of techniques that help users to overcome the associated problems, including thread and process pinning and correct memory associations. We have implemented most of the techniques in a MultiCore SUPport library (MCSup), which helps to map OpenMP applications to multicore machines. We present results using both an MPI-only and a hybrid MPI/OpenMP model.
Existing data movement tools are still bound by major inefficiencies when running on multicore systems. To address these inefficiencies and limitations, DOE’s Advanced Scientific Computing Research (ASCR) office has funded Fermilab and Brookhaven National Laboratory to collaboratively work on the Multicore-Aware Data Transfer Middleware (MDTM) project. MDTM aims to accelerate data movement toolkits on multicore systems. A prototype version of MDTM is currently undergoing evaluation and enhancement.
The present invention relates to optical coupling systems that employ a multicore optical fiber. The system can be employed with two multicore optical fibers or with a single multicore optical fiber for use with an optical device. In particular, the system includes connectors having a lens assembly configured to relay optical signals and a ferrule to position the lens assembly. Additional details for making and using such systems are described herein.
Block iterative methods are extremely important as smoothers for multigrid methods, as preconditioners for Krylov methods, and as solvers for diagonally dominant linear systems. Developing robust and efficient smoother algorithms suitable for current and evolving GPU and multicore CPU systems is a significant challenge. We address this issue in the case of constant-coefficient stencils arising in the solution of elliptic partial differential equations on structured 3D uniform and adaptively refined block structured grids. Robust, highly parallel implementations of block Jacobi and chaotic block Gauss-Seidel algorithms with exact inversion of the blocks are developed using different parallelization techniques. Experimental results for NVIDIA Fermi/Kepler GPUs and AMD multicore systems are presented.
The first switchable fibre-optic delay line based on a 1300-m-long multicore optical fibre has been fabricated and investigated. We have obtained signal delay times of up to 45 μs at 6.43-μs intervals. Sequential signal propagation through the cores of the multicore optical fibre makes it possible to reduce the fibre length necessary for obtaining a predetermined delay time, which is important for reducing the weight and dimensions of devices based on the use of fibre-optic delay lines. (paper)
We have proposed an optimal design of five different core structures (St1 – St5) for a seven-core trench-assisted multicore fibre, whose core radius and relative core/cladding refractive index difference Δ{sub 1} are optimal for single-mode propagation. The crosstalk performance and its reduction (in dB) with respect to variations in fibre design parameters, such as cladding/trench (Δ{sub 2}) depth and trench width W{sub t} are investigated. It is shown that some fibre operational parameters, like, fibre bending radius, operating wavelength and fibre length, affect the crosstalk performance. It is found that the crosstalk and its reduction amount are extremely dependent on the values of Δ{sub 1} and Δ{sub 2}. The dispersion characteristics of the cores are investigated for all the proposed core structures and the observed dispersion is found to be about – 6 ps nm{sup −1} km{sup −1} at an operating wavelength of 1550 nm. An ultralow crosstalk (less than –100 dB per 100 km) can be achieved mainly with two proposed core structures (St4 and St5) having a suitably higher values of Δ{sub 1} along with a small core radius for single-mode propagation. The multicore fibre design with many cores (up to 55) is demonstrated to ensure a high transmission capacity. (optical fibres)
Scientific applications use collective communication operations in Message Passing Interface (MPI) for global synchronization and data exchanges. Alltoall and AlltoallV are two important collective operations. They are used by MPI jobs to exchange messages among all of MPI processes. AlltoallV is a generalization of Alltoall, supporting messages of varying sizes. However, the existing MPI AlltoallV implementation has linear complexity, i.e., each process has to send messages to all other processes in the job. Such linear complexity can result in sub optimal scalability of MPI applications when they are deployed on millions of cores. To address above challenge, in this paper, we introduce a new Scalable LOgarithmic AlltoallV algorithm, named SLOAV, for MPI AlltoallV collective operation. SLOAV aims to achieve global exchange of small messages of different sizes in a logarithmic number of rounds. Furthermore, given the prevalence of multicore systems with shared memory, we design a hierarchical AlltoallV algorithm based on SLOAV by leveraging the advantages of shared memory, which is referred to as SLOAVx. Compared to SLOAV, SLOAVx significantly reduces the inter-node communication, thus improving the entire system performance and mitigating the impact of message latency. We have implemented and embedded both algorithms in Open MPI. Our evaluation on large-scale computer systems shows that for the 8-byte and 1024-process MPI Alltoallv operation, the SLOAV can reduce the latency by as much as 86.4%, when compared to the state-of-the-art, and SLOAVx can further optimize the SLOAV by up to 83.1% in terms of message latency on multicore systems. In addition, experiments with NAS Parallel Benchmark (NPB) demonstrate that our algorithms are very effective for real-world applications.
The improvement of neutron imaging towards and beyond the microscale is a well-documented need for the iterative characterization and modeling of numerous microstructured X-ray opaque materials. This work presents the recent progress in evaluating a SCIntillating FIber (SCIFI) proof-of-concept towards micron-level thermal neutron radiography. These SCIFIs are composed of 6 Li-enriched silicate glass cores doped with a Ce activator. The cores possess ~8.5 μm diameters and ~10 μm pitch following fiber drawing with a cladding glass into an all-solid multicore fiber. A polished 5 mm × 5 mm array of 100 microstructured multicore SCIFI pixels was fabricated into a 1 mm thick faceplate. The neutron efficiency and light yield of the faceplate are characterized as functions of the 7.38 weight percent of Li 2 O, thickness, and the 70% active volume. It was determined that approximately 39% of a thermal neutron (2 Å) beam can be absorbed by the faceplate. The 6 L i ( n , α<!-- α --> ) t reaction is estimated to produce 7,700 ± 1,000 scintillation photons per event, referencing light collection from 241 Am irradiation of the faceplate. Simulations suggest that on average 17.5 ± 1.4% of these photons will be transported to an end of the fiber array for a thermal beam, with at least 7.2% of that total scintillation light being confined into the fiber cores in which it originated. The SCIFI faceplate was integrated into the Neutron Microscope (NM) at the Pulse OverLap DIffractometer (POLDI) beamline located at the Paul Scherrer Institut to image a Siemens star test object. Processed neutron radiographs acquired with the proof-of-concept faceplate resolved features at a state-of-the-art resolution of 16.1 ± 0.5 μm. The potential for even high resolution designs having smaller pitch or different cladding material is discussed.