A Generic and Efficient Framework for Estimating Lossy Compressibility of Scientific Data

• Published in 38th International Conference on Massive Storage Systems and Technology (MSST), 2024.
• Authors: Md Hasanur Rahman, Sheng Di, Guanpeng Li and Franck Cappello.
• This project was done with the collaboration of Argonne National Laboratory.

Goal

This paper proposes a compressor-agnostic lossy compression framework XTIMATE that is capable of accurately and efficiently estimating scientific data compressibility. Please check the paper for more details.

Citations

If you want to include our paper in your work, please cite our paper. You can find the bibtex citation here.

Introduction to different folders

• Baselines: this folder contains the code and evaluations of our two baselines.
• Compressor-executables: this folder has instructions of how to execute each of the compressors.
• Datasets: we provide the description of dataset source and usage here. We also provide one sample dataset application to test XTIMATE.
• Our-framework: this folder provides code and step-by-step guidelines of how to install and run our framework XTIMATE.

License

This work is licensed under a Creative Commons Attribution 4.0 International License.