DOE OSTI · 1651320
3D Coded SUMMA: Communication-Efficient and Robust Parallel Matrix Multiplication
Abstract
In this paper, we propose a novel fault-tolerant parallel matrix multiplication algorithm called 3D Coded SUMMA that achieves higher failure-tolerance than replication-based schemes for the same amount of redundancy. This work bridges the gap between recent developments in coded computing and fault-tolerance in high-performance computing (HPC). The core idea of coded computing is the same as algorithm-based fault-tolerance (ABFT), which is weaving redundancy in the computation using error-correcting codes. In particular, we show that MatDot codes, an innovative code construction for parallel matrix multiplications, can be integrated into three-dimensional SUMMA (Scalable Universal Matrix Multiplication Algorithm [30]) in a communication-avoiding manner. To tolerate any two node failures, the proposed 3D Coded SUMMA requires ~50% less redundancy than replication, while the overhead in execution time is only about 5–10%.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Jeong, Haewon, Yang, Yaoqing, Gupta, Vipul, Engelmann, Christian, Meng Low, Tze, Cadambe, Viveck, Ramchandran, Kannan, Grover, Pulkit. 2020-08-01. 3D Coded SUMMA: Communication-Efficient and Robust Parallel Matrix Multiplication. https://doi.org/10.1007/978-3-030-57675-2_25. https://searchnasa.com/record/osti%3A1651320/3d-coded-summa-communication-efficient-and-robust-parallel-matrix-multiplication
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Search NASA by Indiciae: BibTeX · RIS.