TY - GEN
T1 - Using Neural Networks for Two Dimensional Scientific Data Compression
AU - Hayne, Lucas
AU - Clyne, John
AU - Li, Shaomeng
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - Continual advances in high-performance computing have enabled the development of higher resolution and more realistic simulations of a wide variety of scientific phenomena. As a result, many computational science communities are increasingly constrained by the massive volumes of data produced, for example, strict storage constraints often force reductions in the number of output variables, data output frequency, or simulation length. Accordingly, modelers across many scientific domains are beginning to adopt purpose-built scientific data compression techniques as an effective mitigation for these challenges. The origins of scientific data compression tools every so often lie in image and video compression. Recently, compression researchers have achieved state-of-the-art performance using neural networks for natural image compression, but this achievement has yet to be adapted to scientific data. This paper assesses the performance of an existing autoencoder neural network compression algorithm on two sets of two-dimensional floating-point scientific data. Compared to state-of-the-art scientific data compression algorithms SZ and ZFP, this out-of-the-box neural network achieves higher peak signal-to-noise ratios at low bit rates, and remains competitive in controlling maximum point-wise error. This preliminary assessment paves the way for future research into neural network compression on floating-point scientific data.
AB - Continual advances in high-performance computing have enabled the development of higher resolution and more realistic simulations of a wide variety of scientific phenomena. As a result, many computational science communities are increasingly constrained by the massive volumes of data produced, for example, strict storage constraints often force reductions in the number of output variables, data output frequency, or simulation length. Accordingly, modelers across many scientific domains are beginning to adopt purpose-built scientific data compression techniques as an effective mitigation for these challenges. The origins of scientific data compression tools every so often lie in image and video compression. Recently, compression researchers have achieved state-of-the-art performance using neural networks for natural image compression, but this achievement has yet to be adapted to scientific data. This paper assesses the performance of an existing autoencoder neural network compression algorithm on two sets of two-dimensional floating-point scientific data. Compared to state-of-the-art scientific data compression algorithms SZ and ZFP, this out-of-the-box neural network achieves higher peak signal-to-noise ratios at low bit rates, and remains competitive in controlling maximum point-wise error. This preliminary assessment paves the way for future research into neural network compression on floating-point scientific data.
UR - https://www.scopus.com/pages/publications/85125337157
U2 - 10.1109/BigData52589.2021.9671627
DO - 10.1109/BigData52589.2021.9671627
M3 - Conference contribution
AN - SCOPUS:85125337157
T3 - Proceedings - 2021 IEEE International Conference on Big Data, Big Data 2021
SP - 2956
EP - 2965
BT - Proceedings - 2021 IEEE International Conference on Big Data, Big Data 2021
A2 - Chen, Yixin
A2 - Ludwig, Heiko
A2 - Tu, Yicheng
A2 - Fayyad, Usama
A2 - Zhu, Xingquan
A2 - Hu, Xiaohua Tony
A2 - Byna, Suren
A2 - Liu, Xiong
A2 - Zhang, Jianping
A2 - Pan, Shirui
A2 - Papalexakis, Vagelis
A2 - Wang, Jianwu
A2 - Cuzzocrea, Alfredo
A2 - Ordonez, Carlos
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2021 IEEE International Conference on Big Data, Big Data 2021
Y2 - 15 December 2021 through 18 December 2021
ER -