基于HDF5的结构网格计算流体动力学程序并行I/O技术

大规模计算流体动力学(CFD)计算对数据I/O能力提出了很高需求。层次式文件格式(HDF5)可有效管理大规模科学数据,并对并行I/O具有良好的支持。针对结构网格CFD并行程序,设计了其数据文件的HDF5存储模式,并基于HDF5并行I/O编程接口实现了其数据文件的并行I/O,在并行计算机系统上进行了性能测试与分析。结果表明,在使用4~32个进程时,基于HDF5并行I/O方式的写文件性能比每进程独立写普通文件的方式高6.9~16.1倍;基于HDF5并行I/O方式的读文件性能不及后者,为后者的20%~70%,但是读文件的时间开销远小于写文件的时间开销,因此对总体性能的影响较小。

HDF5 based parallel I/O technology for structure grid CFD applications

Large scale parallel Computational Fluid Dynamics (CFD) simulation has posed high demand on the data I/O capabilities. Hierarchical Data Format version 5 (HDF5) can effectively manage large scale scientific data and has a good support to parallel I/O. The HDF5 data storage model for a parallel CFD application was designed. The parallel I/O method for the application's data was implemented based on the HDF5 parallel I/O API. Performance experiments were performed on a parallel computer system. The results show that the data writing speed of this HDF5 based parallel I/O method outperforms the parallel I/O methods (i.e. each process writes an ordinary data file independently) by 6.9 to 16.1 times, when 4 to 32 processes are used. The data reading speed of this HDF5 based parallel I/O method is slower than the parallel I/O methods, with a speed of 20% to 70% times of the latter. However, the cost of data reading time is much smaller than the cost of data writing time, hence has minor effect on the total performance.