MGARD (MultiGrid Adaptive Reduction of Data) is a technique for multilevel lossy compression and refactoring of scientific data based on the theory of multigrid methods. We encourage you to make a GitHub issue if you run into any problems using MGARD, have any questions or suggestions, etc.

MGARD framework consists of the following modules. Please see the detailed instructions for each module to build and install MGARD.

MGARD-CPU: MGARD compression implementation for CPUs

MGARD-CPU is designed for running compression on CPUs. See the detailed user guide here. In addition, MGARD-CPU can be configured to preserve region-of-interest (RoI user guide) and linear quantity-of-interest (QoI user guide) during data compression.

MGARD-X: Accelerated and portable compression

MGARD-X is designed for portable compression on NVIDIA GPUs, AMD GPUs, and CPUs. See the detailed user guide here. In addition, MGARD-X can be configured to preserve region-of-interest (RoI user guide) and linear quantity-of-interest (QoI user guide) during data compression.

MGARD$\text{-}\lambda$: Preserving Non-Linear Quantity-of-Interest

MGARD$\text{-}\lambda$ is specifically designed for preserving non-linear quantity-of-interest during data compression. This is an experimental part of MGARD. Currently, it only supports certain QoIs derived from XGC 5D data. See theory in here and example in here.

MDR/MDR-X: Fine-grain progressive data reconstruction

MDR and MDR-X are designed to enable fine-grain data refactoring and progressive data reconstruction. See the detailed user guide here.

Self-describing format for compressed and refactored data

Data produced by MGARD, MGARD-X, and MDR-X are designed to follow a unified self-describing format. See format details in here.

Publications

Fundamental Theory

• Xin Liang et al. MGARD+: Optimizing Multilevel Methods for Error-bounded Scientific Data Reduction. IEEE Transactions on Computers, 2021
• Mark Ainsworth et al. Multilevel Techniques for Compression and Reduction of Scientific Data—The Unstructured Case. SIAM Journal on Scientific Computing, 42 (2), A1402–A1427, 2020.
• Mark Ainsworth et al. Multilevel Techniques for Compression and Reduction of Scientific Data—Quantitative Control of Accuracy in Derived Quantities. SIAM Journal on Scientific Computing 41 (4), A2146–A2171, 2019.
• Mark Ainsworth et al. Multilevel Techniques for Compression and Reduction of Scientific Data—The Multivariate Case. SIAM Journal on Scientific Computing 41 (2), A1278–A1303, 2019.
• Mark Ainsworth et al. Multilevel Techniques for Compression and Reduction of Scientific Data—The Univariate Case. Computing and Visualization in Science 19, 65–76, 2018.
• Ben Whitney. Multilevel Techniques for Compression and Reduction of Scientific Data. PhD thesis, Brown University, 2018.

Preserving Quantities of Interest (QoIs)

• Xuan Wu et al. Error-controlled Progressive Retrieval of Scientific Data under Derivable Quantities of Interest. the International Conference for High Performance Computing, Networking, Storage and Analysis 2024, Nov, 2024
• Tania Banerjee et al. Scalable Hybrid Learning Techniques for Scientific Data Compression., Arxiv, 2022
• Qian Gong et al. Region-adaptive, Error-controlled Scientific Data Compression using Multilevel Decomposition. the 34th International Conference on Scientific and Statistical Database Management, Jul. 2022
• Tania Benerjee et al. An algorithmic and software pipeline for very large-scale scientific data compression with error guarantees. International Conference on High Performance Computing, Data, and Analytics, 2022
• Jaemoon Lee et al. Error-bounded learned scientific data compression with preservation of derived quantities. Applied Sciences, 2022
• Qian Gong et al. Maintaining trust in reduction: Preserving the accuracy of quantities of interest for lossy compression. 21st Smoky Mountains Computational Sciences and Engineering Conference, Oct. 2021

Progressive Retrieval

• Jinzheng Wang et al. Improving Progressive Retrieval for HPC Scientific Data using Deep Neural Network. IEEE International Conference on Data Engineering (ICDE), 2023
• Xin Liang et al. Error-controlled, progressive, and adaptable retrieval of scientific data with multilevel decomposition. the International Conference for High Performance Computing, Networking, Storage and Analysis 2021, Nov, 2021

Parallelization and GPU Acceleration

• Jieyang Chen et al. HPDR: High-Performance Portable Scientific Data Reduction Framework. 39th IEEE International Parallel and Distributed Processing Symposium, June, 2025
• Jieyang Chen et al. Scalable Multigrid-based Hierarchical Scientific Data Refactoring on GPUs. Arxiv
• Jieyang Chen et al. Accelerating Multigrid-based Hierarchical Scientific Data Refactoring on GPUs. 35th IEEE International Parallel & Distributed Processing Symposium, May, 2021.

System Optimizations

• Lipeng Wan et al. RAPIDS: Reconciling Availability, Accuracy, and Performance in Managing Geo-Distributed Scientific Data. * The International ACM Symposium on High-Performance Parallel and Distributed Computing*, Jun. 2023
• Xinying Wang et al. Unbalanced Parallel I/O: An Often-Neglected Side Effect of Lossy Scientific Data Compression. 7th International Workshop on Data Analysis and Reduction for Big Scientific Data, Nov. 2021