基于HDF5格式的FAST望远镜数据重分布策略

目前，世界上最大的单口径射电望远镜FAST (Five-hundred-meter Aperture Spherical radio Telescope)处于调试期，采集的数据面临着加载效率上的瓶颈，给后续的数据处理效率带来了挑战。本文结合科学数据存储格式HDF5（Hierarchical Data Format Release 5）和数据降维，提出一种适合大多数FAST望远镜数据处理场景的存储优化方法。通过优化，硬盘中的数据从二进制表格模型转换为按类型分布的多个数据集。实验结果表明本文提出的方法能够显著提高FAST望远镜的数据加载效率。

Data Redistribution Strategy of FAST Telescope Based on HDF5 Format

At present, FAST(Five-hundred-meter Aperture Spherical radio Telescope), the world’s largest single-caliber radio telescope, is in the period of pre-run. The collected data faces bottlenecks in loading efficiency, which brings challenges to subsequent efficiencies of data processing. Combined with scientific data storage format of HDF5（Hierarchical Data Format Release 5） and data reduction, this paper proposes a storage optimization method suitable for most data processing scenarios of FAST telescope. Through optimization, data in the hard disk is converted from a binary table model to multiple data sets distributed by type. The experimental results show that the proposed method can significantly improve the data loading efficiency of FAST telescope.