Efficient Sparse Matrix-Matrix Multiplication on Heterogeneous High Performance Systems

Abstract

The efficient implementation of sparse matrix-matrix multiplications on high performance parallel machines poses several challenges: large size of input matrices, compressed representation, density of the output matrices, partitioning and load balancing of matrices that present parts with large differences in density and, thus, in computation times. In this paper we show how, starting from the requirements of such application, we developed a framework that allows its efficient implementation on heterogeneous clusters. We introduce a task based programming model and a runtime supported execution model which provides dynamic load balancing on clusters composed by CPUs and GPUs, allowing better utilization of the system while easing the handling of sparse matrices. Our results show that our solution, which co-designs the application together with the programming model and the runtime system, is able to obtain significant speed-ups due to a virtually perfect load balancing with respect to other programming approaches.