基于Hadoop的海量MP3文件存储架构研究

MP3作为数字音乐时代的事实标准，数量已相当巨大，用户的访问需求也与日俱增，有效存储和管理海量的MP3文件，为用户提供良好的体验，越来越引起人们的关注。Hadoop的出现，提供了一种新的思路。但由于Hadoop自身并不适合海量小文件处理，提出一种基于Hadoop的海量MP3文件存储架构，充分利用MP3文件自身包含的丰富描述信息，通过预处理模块的归类算法，将小文件归并到Sequence File中，同时引入高效的索引机制，很好地解决了小文件过多时NameNode内存瓶颈问题。实验证明，该架构能达到良好的效果。

Hadoop-based storage architecture for mass MP3 files

MP3 as the de facto standard for digital music,the number of files is quite large and user access requirements rapidly grow up.How to effectively store and manage vast amounts of MP3 files to provide good user experience has become a matter of bigger concern.The emergence of Hadoop provides us with new ideas.However,because Hadoop itself is not suitable for handling massive small files,this paper presented a Hadoop-based storage architecture for massive MP3 files,fullly using the MP3 file’s rich metadata.The classification algorithm by pre-processing module would merged small files into sequence file,and the introduction of efficient indexing mechanism served as a good solution to the problem of small files.The experimental results show that the approach can achieve a better performance.