基于位串内容感知的数据分块算法

针对基于内容的可变长度的分块CDC算法中数字签名计算需要耗费大量CPU开销的问题,提出了一种基于位串内容感知的数据块分块算法。算法利用每一次失败匹配尝试所带来的位特征信息,最大限度地排除不能匹配的位置,从而获得最大的跳跃长度,减少中间计算和比较的开销。实验结果表明,本算法减小了数据分块过程中数字签名计算的开销,降低了确定块边界时的CPU资源消耗,从而优化了数据分块的时间性能。

A bit string content aware data chunking algorithm

Aiming at the problem of a large amount of overhead introduced by the content defined chunking algorithm (CDC) in calculating the digital signature, we present a novel data chunking algorithm based on bit string content awareness.The proposed algorithm eliminates unmatched positions to the utmost by taking advantage of the bit feature information acquired through each failure matching.Since the maximum jump length is obtained, intermediate calculation and comparison cost are reduced.Experimental results show that the algorithm can reduce the overhead of digital signature calculation in the process of data chunking, cut down CPU resource consumption for chunk boundary determination, and optimize the time performance of data chunking.