几个常见分布式文件系统特征分析和性能对比

近年来随着云计算市场规模不断增长,作为云计算平台基础设施的云存储系统也随之显得越来越重要。数以万计的互联网应用已经运行于云计算环境,同时大量不同的应用也即将从传统运行环境转移到云计算平台。不同的互联网应用的存储需求可能不一样。例如:应用中涉及的单个文件大小,文件数量,IO访问模式,读写比率等,都对底层存储系统提出了不同的要求。这说明在云计算环境中,单个文件系统可能无法满足全部应用的存储需求,本文尝试通过在单一云计算平台中部署多个不同分布式文件系统来优化存储系统的总体性能。为了优化混合式文件系统的性能,首先需要分析不同文件系统的性能特征。本文通过量化方法分析了云计算环境下几个常用的分布式文件系统,这些文件系统分别是ceph,moosefs,glusterfs和hdfs。实验结果显示:即使针对同一文件的相同读写操作,不同分布式文件系统之间的性能也差异显著,当单个文件的大小小于256MB时,moosefs的平均写性能比其它几个文件系统高22.3%;当单个文件大小大于256KB时,glusterfs的平均读性能比其它几个文件系统高21.0%。这些结果为设计和实现一个基于以上几个分布式文件系统的混合式文件系统提供了基础。

A Characterization and Analysis of Distributed File Systems

Recently, there has been an explosive growth in cloud computing, greatly increasing the importance of storage in such systems. A wide range of applications have been running in cloud and more and more variant applications are rushing into this platform. Different applications may have different requirements for storages such as file size, the number of files, and I/O performance. This indicates only a unified file system in cloud would keep the overall system performance suboptimal or even cannot satisfy the need of all applications in a cloud. However, it is unclear that whether it is beneficial to optimize the overall I/O performance by employing variant file systems in a single cloud computing platform. In this paper, we address the above problem by characterizing several popular distributed files systems used in cloud computing. These file systems are ceph, moosefs, glusterfs and hdfs. Through the characterization, we find that the performance of the same operation such as read or write may be dramatically different for different file systems. When the file size is less than 256 MB, moosefs has the best writing performance. On average, its writing performance outperforms others by 22.3%. As for reading performance, glusterfs is the best when the file size is larger than 256KB. Its reading performance is 21.0% higher than other file systems. These findings lead us to design a hybrid file system for cloud computing platform, attempting significantly improve the overall performance.