HDFS数据副本均衡放置策略的改进

HDFS默认的数据副本放置策略仅仅只根据磁盘空间使用单个指标进行负载衡量,无法实现各节点真正的负载均衡。提出了一种基于性能的副本负载均衡放置改进策略,从磁盘空间负载能力、CPU处理能力、内存处理能力、磁盘读写处理能力、带宽等5个方面考究节点实际工作负载,并定义了一个负载能力模型。实验结果表明,该改进策略比默认策略能更好地实现副本的均衡放置。

Improvement of HDFS Balanced Placement Strategy

The default data replica placement policy for HDFS is only measured using a single metric based on disk space,the true load balancing of each node cannot be realized. This paper proposed an improvement strategy of the load balancing based on performance through five aspects,such as disk space,CPU processing power,memory processing power,disk read/write,bandwidth,and a load capacity model was defined.Experimental results indicate that the improvement strategy is better than the default policy.