Setting the Standard: Recommended Practices for Data Preprocessing in Data-Driven Climate Prediction

Artificial intelligence (AI)—and specifically machine learning (ML)—applications for climate prediction across time scales are proliferating quickly. The emergence of these methods prompts a revisit to the impact of data preprocessing, a topic familiar to the climate community, as more traditional statistical models work with relatively small sample sizes. Indeed, the skill and confidence in the forecasts produced by data-driven models are directly influenced by the quality of the datasets and how they are treated during model development, yielding the colloquialism, “garbage in, garbage out.” As such, this article establishes protocols for the proper preprocessing of input data for AI/ML models designed for climate prediction (i.e., subseasonal-to-decadal and longer time scales). The three aims are to 1) educate researchers, developers, and end users on the effects that data preprocessing has on climate prediction; 2) provide recommended practices for data preprocessing for such applications; and 3) empower end users to decipher whether the models they are using are properly designed for their objectives. Specific topics covered include creating (standardized) anomalies, dealing with nonstationarity and the spatiotemporally correlated nature of climate data, and handling of extreme values and variables with potentially complex distributions. Case studies will illustrate how using different preprocessing techniques can produce different predictions from the same model, which can create confusion and decrease confidence in the overall process. Ultimately, implementing the recommended practices set forth in this article will enhance the robustness and transparency of AI/ML in climate prediction studies. SIGNIFICANCE STATEMENT: With the rapid expansion of artificial intelligence (AI) in atmospheric science, the need for high-quality, properly prepared data for input into AI/ML models is important. In this article, we offer several recommended steps to properly preprocess input data for AI models used for climate predictions (i.e., time scale ranging from few weeks to many years). Among other topics, we discuss appropriate ways to calculate departures (or anomalies) from data that vary in time and space, how to handle large trends, and what to do with extreme values. We then conduct two case studies to illustrate how using different techniques for preprocessing can produce different predictions from the same model. Ultimately, following these recommendations will help make such studies more transparent, reproducible, and trustworthy.