DOE OSTI · 3374061
A Survey on Error-Bounded Lossy Compression for Scientific Datasets
Abstract
Error-bounded lossy compression has been effective in significantly reducing the data storage/transfer burden while preserving the reconstructed data fidelity very well. Many error-bounded lossy compressors have been developed for a wide range of parallel and distributed use cases for years. They are designed with distinct compression models and principles, such that each of them features particular pros and cons. In this article, we provide a comprehensive survey of emerging error-bounded lossy compression techniques. The key contribution is fourfold. (1) We summarize a novel taxonomy of lossy compression into six classic models. (2) We provide a comprehensive survey of 10 commonly used compression components/modules. (3) We summarized pros and cons of 47 state-of-the-art lossy compressors and present how state-of-the-art compressors are designed based on different compression techniques. (4) We discuss how customized compressors are designed for specific scientific applications and use-cases. We believe this survey is useful to multiple communities including scientific applications, high-performance computing, lossy compression, and big data.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Di, Sheng [Argonne National Laboratory (ANL), Argonne, IL (United States)] (ORCID:0000000299355674), Liu, Jinyang [Univ. of California, Riverside, CA (United States)] (ORCID:000000030177502X), Zhao, Kai [Florida State Univ., Tallahassee, FL (United States)] (ORCID:0000000153283962), Liang, Xin [Univ. of Kentucky, Lexington, KY (United States)] (ORCID:0000000206301600), Underwood, Robert [Argonne National Laboratory (ANL), Argonne, IL (United States)] (ORCID:000000021464729X), Zhang, Zhaorui [Hong Kong Polytechnic Univ. (Hong Kong)] (ORCID:0000000302841113), Shah, Milan [North Carolina State University, Raleigh, NC (United States)] (ORCID:0000000235727408), Huang, Yafan [Univ. of Iowa, Iowa City, IA (United States)] (ORCID:0000000173706766), Huang, Jiajun [Univ. of California, Riverside, CA (United States); Univ. of South Florida, Tampa, FL (United States)] (ORCID:0000000150923987), Yu, Xiaodong [Stevens Inst. of Technology, Hoboken, NJ (United States)] (ORCID:0000000162441264), Ren, Congrong [The Ohio State Univ., Columbus, OH (United States)] (ORCID:0009000662857271), Guo, Hanqi [The Ohio State Univ., Columbus, OH (United States)] (ORCID:0000000177761834), Wilkins, Grant [Univ. of Cambridge (United Kingdom)] (ORCID:0000000191260673), Tao, Dingwen [Indiana Univ., Bloomington, IN (United States)] (ORCID:0000000154224497), Tian, Jiannan [Indiana Univ., Bloomington, IN (United States); Univ. of Kentucky, Lexington, KY (United States)] (ORCID:0000000311019148), Jin, Sian [Temple Univ., Philadelphia, PA (United States)] (ORCID:0009000992500611), Jian, Zizhe [Univ. of California, Riverside, CA (United States)] (ORCID:0009000520798130), Wang, Daoce [Indiana Univ., Bloomington, IN (United States)] (ORCID:0000000244443634), Rahman, Md Hasanur [Univ. of Iowa, Iowa City, IA (United States)] (ORCID:0009000255408751), Zhang, Boyuan [Indiana Univ., Bloomington, IN (United States)] (ORCID:0009000389374067), Song, Shihui [Univ. of Iowa, Iowa City, IA (United States)] (ORCID:0009000794268591), Calhoun, Jon [Clemson Univ., SC (United States)] (ORCID:0000000171914422), Li, Guanpeng [Univ. of Iowa, Iowa City, IA (United States)] (ORCID:0000000177737826), Yoshii, Kazutomo [Argonne National Laboratory (ANL), Argonne, IL (United States)] (ORCID:0000000319045383), Alharthi, Khalid [Univ. of Bihac (Bosnia and Herzegovina)] (ORCID:0000000246867414), Cappello, Franck [Argonne National Laboratory (ANL), Argonne, IL (United States)] (ORCID:0000000278903934). 2025-06-11. A Survey on Error-Bounded Lossy Compression for Scientific Datasets. https://doi.org/10.1145/3733104
Cite the original work for its findings. Save a collection to share your selection of sources.