Skip to main content

Introduction into chunking for large gridded datasets


About This Course

This course shows the importance of data chunking when working with spatio-temporal gridded data sets. Data sets for the Earth system sciences are getting larger and larger, and they can no longer be completely loaded into working memory. The programming language used for the course is Julia. In brief, we will learn how to load files with different chunks and explore their properties. We will compute global per-pixel statistics and time steps and compare data processing performance for different chunks. Finally, we will demonstrate the relevance of chunking when processing data for different data axes (Access this course on GitLab).




Programming experience, experience in working with gridded data

Subject Area

Earth System Sciences

What You Will Learn

  • Understanding data chunking
  • Data exploration
  • Statistical computation


Understanding data chunking by Lina M. Estupinan-Suarez, Felix Cremer, Fabian Gans


Farzaneh Sadeghi