支持文件重删的HDFS分布式备份系统研究

随着传统行业电子化的速度日渐加快,电脑上的数据量与日俱增,数据备份领域面临的挑战也越来越大.传统备份因为需要大量的磁盘阵列作为存储介质,所以在成本控制上一直是个难题.针对该情况,结合云存储平台低廉的成本、高效的资源伸缩与利用率,研究在HDFS系统中加入新的重复数据删除技术,并对原有备份策略进行优化,设计出一种基于云存储中的重复数据删除技术的备份系统.最后通过实验,对改进后的方案系统与传统备份方案备份文件所占空间、时间等参数进行了对比.

A deduplication of HDFS distributed backup system

With the speed of electronic of traditional industry gradually accelerating,the amount of data on the computer is growing,challenges in the field of data backup are also growing.Because traditional backup requires a lot of disk arrays as the storage medium,so the cost control has been a problem.For this situation,combined with the low cost of cloud storage platform,scalable and efficient resource utilization,research in HDFS system by adding new data de-duphcation technology,and optimize the existing backup strategy,design a cloud-based storage the deduplication backup system.Finally,through the improved scheme system and the traditional backup program backup file with useing less space and time parameters were compared.