| 基于P2P的分布式文件系统下载效率优化 |
| |
| 引用本文： | 廖彬,于炯,张陶,杨兴耀. 基于P2P的分布式文件系统下载效率优化[J]. 计算机应用, 2011, 31(9): 2317-2320. DOI: 10.3724/SP.J.1087.2011.02317 |
| |
| 作者姓名： | 廖彬 于炯 张陶 杨兴耀 |
| |
| 作者单位： | 新疆大学 信息科学与工程学院，乌鲁木齐 830046 |
| |
| 基金项目： | 国家自然科学基金资助项目(60863003;61063042);新疆大学博士科研启动基金资助项目(BS090153) |
| |
| 摘 要： | 对分布式文件系统(HDFS)集群内部数据块存储机制与下载流程进行分析研究，结合P2P多点与多线程下载思想，从数据块、文件、集群三个方面提出了数据下载效率优化算法。考虑到集群内部可能因多线程下载出现的负载均衡问题，提出下载点选择算法以优化下载点的选择。实验结果表明，三种优化算法都能提高下载效率，下载点选择算法能够很好地实现集群内部DataNode负载均衡。
|
| 关 键 词： | 云计算 分布式文件系统 对等网 并行下载 负载均衡 |
| 收稿时间： | 2011-03-07 |
| 修稿时间： | 2011-05-13 |
Download performance optimization in Hadoop distributed file system based on P2P |
| |
| LIAO Bin,YU Jiong,ZHANG Tao,YANG Xing-yao. Download performance optimization in Hadoop distributed file system based on P2P[J]. Journal of Computer Applications, 2011, 31(9): 2317-2320. DOI: 10.3724/SP.J.1087.2011.02317 |
| |
| Authors: | LIAO Bin YU Jiong ZHANG Tao YANG Xing-yao |
| |
| Affiliation: | College of Information Science and Technology, Xinjiang University, Urumqi Xingjiang 830046, China |
| |
| Abstract: | The data block storage mechanism and downloading process in Hadoop Distributed File System (HDFS) cluster were analyzed. In combination with multi-point and multi-threaded Peer-to-Peer (P2P) download idea, an efficiency optimization algorithm was proposed from the aspects of data-block, file and cluster. Concerning the possible imbalanced load problem caused by multi-thread download in HDFS cluster, a download-point selection algorithm was put forward to optimize the download-point selection. The mathematical analysis and experiments prove that the three methods can improve the download efficiency and download-point selection algorithm can achieve loading balance among DataNodes in HDFS cluster. |
| |
| Keywords: | cloud computing Hadoop Distributed File System (HDFS) Peer-to-Peer (P2P) parallel download load balance |
| 本文献已被 CNKI 等数据库收录！ |
| 点击此处可从《计算机应用》浏览原始摘要信息 |
|
点击此处可从《计算机应用》下载全文 |
