Lossy Scientific Data Compression With SPERR

Shaomeng Li; Peter Lindstrom; John Clyne

doi:10.1109/IPDPS54959.2023.00104

Lossy Scientific Data Compression With SPERR

Shaomeng Li
, Peter Lindstrom
, John Clyne

Visualization and Analysis Software Team

National Center for Atmospheric Research
Lawrence Livermore Natl. Laboratory

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

50 Scopus citations

Abstract

As the need for data reduction in high-performance computing (HPC) continues to grow, we introduce a new and highly effective tool to help achieve this goal - SPERR. SPERR is a versatile lossy compressor for structured scientific data; it is built on top of an advanced wavelet compression algorithm, SPECK, and provides additional capabilities valued in HPC environments. These capabilities include parallel execution for large volumes and a compression mode that satisfies a maximum point-wise error tolerance. Evaluation shows that in most settings SPERR achieves the best rate-distortion trade-off among current popular lossy scientific data compressors.

Original language	English
Title of host publication	Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1007-1017
Number of pages	11
ISBN (Electronic)	9798350337662
DOIs	https://doi.org/10.1109/IPDPS54959.2023.00104
State	Published - 2023
Event	37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023 - St. Petersburg, United States Duration: May 15 2023 → May 19 2023

Publication series

Name	Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023

Conference

Conference	37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023
Country/Territory	United States
City	St. Petersburg
Period	05/15/23 → 05/19/23

Access to Document

10.1109/IPDPS54959.2023.00104

Cite this

Li, S., Lindstrom, P., & Clyne, J. (2023). Lossy Scientific Data Compression With SPERR. In Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023 (pp. 1007-1017). (Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IPDPS54959.2023.00104

@inproceedings{41d3ba22a8114945b1ccf2224c468420,

title = "Lossy Scientific Data Compression With SPERR",

abstract = "As the need for data reduction in high-performance computing (HPC) continues to grow, we introduce a new and highly effective tool to help achieve this goal - SPERR. SPERR is a versatile lossy compressor for structured scientific data; it is built on top of an advanced wavelet compression algorithm, SPECK, and provides additional capabilities valued in HPC environments. These capabilities include parallel execution for large volumes and a compression mode that satisfies a maximum point-wise error tolerance. Evaluation shows that in most settings SPERR achieves the best rate-distortion trade-off among current popular lossy scientific data compressors.",

author = "Shaomeng Li and Peter Lindstrom and John Clyne",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023 ; Conference date: 15-05-2023 Through 19-05-2023",

year = "2023",

doi = "10.1109/IPDPS54959.2023.00104",

language = "English",

series = "Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1007--1017",

booktitle = "Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023",

address = "United States",

}

Li, S, Lindstrom, P & Clyne, J 2023, Lossy Scientific Data Compression With SPERR. in Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023. Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023, Institute of Electrical and Electronics Engineers Inc., pp. 1007-1017, 37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023, St. Petersburg, United States, 05/15/23. https://doi.org/10.1109/IPDPS54959.2023.00104

Lossy Scientific Data Compression With SPERR. / Li, Shaomeng; Lindstrom, Peter; Clyne, John.
Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023. Institute of Electrical and Electronics Engineers Inc., 2023. p. 1007-1017 (Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Lossy Scientific Data Compression With SPERR

AU - Li, Shaomeng

AU - Lindstrom, Peter

AU - Clyne, John

PY - 2023

Y1 - 2023

N2 - As the need for data reduction in high-performance computing (HPC) continues to grow, we introduce a new and highly effective tool to help achieve this goal - SPERR. SPERR is a versatile lossy compressor for structured scientific data; it is built on top of an advanced wavelet compression algorithm, SPECK, and provides additional capabilities valued in HPC environments. These capabilities include parallel execution for large volumes and a compression mode that satisfies a maximum point-wise error tolerance. Evaluation shows that in most settings SPERR achieves the best rate-distortion trade-off among current popular lossy scientific data compressors.

AB - As the need for data reduction in high-performance computing (HPC) continues to grow, we introduce a new and highly effective tool to help achieve this goal - SPERR. SPERR is a versatile lossy compressor for structured scientific data; it is built on top of an advanced wavelet compression algorithm, SPECK, and provides additional capabilities valued in HPC environments. These capabilities include parallel execution for large volumes and a compression mode that satisfies a maximum point-wise error tolerance. Evaluation shows that in most settings SPERR achieves the best rate-distortion trade-off among current popular lossy scientific data compressors.

UR - https://www.scopus.com/pages/publications/85166633679

U2 - 10.1109/IPDPS54959.2023.00104

DO - 10.1109/IPDPS54959.2023.00104

M3 - Conference contribution

AN - SCOPUS:85166633679

T3 - Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023

SP - 1007

EP - 1017

BT - Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023

Y2 - 15 May 2023 through 19 May 2023

ER -

Li S, Lindstrom P, Clyne J. Lossy Scientific Data Compression With SPERR. In Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 1007-1017. (Proceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023). doi: 10.1109/IPDPS54959.2023.00104

Lossy Scientific Data Compression With SPERR

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this