Toward a multi-method approach: Lossy data compression for climate simulation data

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

40 Scopus citations

Abstract

Earth System Model (ESM) simulations are increasingly constrained by the amount of data that they generate rather than by computational resources. The use of lossy data compression on model output can reduce storage costs and data transmission overheads, but care must be taken to ensure that science results are not impacted. Choosing appropriate compression algorithms and parameters is not trivial given the diversity of data produced by ESMs and requires an understanding of both the attributes of the data and the properties of the chosen compression methods. Here we discuss the properties of two distinct approaches for lossy compression in the context of a well-known ESM, demonstrating the different strengths of each, to motivate the development of an automated multi-method approach for compression of climate model output.

Original languageEnglish
Title of host publicationHigh Performance Computing - ISC High Performance 2017 International Workshops, DRBSD, ExaComm, HCPM, HPC-IODC, IWOPH, IXPUG, P^3MA, VHPC, Visualization at Scale, WOPSSS, Revised Selected Papers
EditorsRio Yokota, Julian M. Kunkel, Michela Taufer, John Shalf
PublisherSpringer Verlag
Pages30-42
Number of pages13
ISBN (Print)9783319676296
DOIs
StatePublished - 2017
Event32nd International Conference on High Performance Computing, ISC High Performance 2017 - Frankfurt, Germany
Duration: Jun 18 2017Jun 22 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10524 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference32nd International Conference on High Performance Computing, ISC High Performance 2017
Country/TerritoryGermany
CityFrankfurt
Period06/18/1706/22/17

Fingerprint

Dive into the research topics of 'Toward a multi-method approach: Lossy data compression for climate simulation data'. Together they form a unique fingerprint.

Cite this