基于Bigtable与MapReduce的Apriori算法改进

为提高Apriori算法挖掘频繁项目集的效率,引进了Bigtable技术与MapReduce模型来对Apriori算法进行优化,设计出大数据环境下挖掘频繁项目集的新算法BM-Apriori算法。与单纯基于MapReduce模型的Apriori改进算法相比,新算法利用Bigtable的时间戳属性代替了键/值对的产生,只需扫描数据库一次即可,节约了模式匹配的时间。同时,BM-Apriori算法在项集列表中新增事务标号列,自动获取事务标号以计算支持度。将BM-Apriori算法在Hadoop平台上进行了实验,结果表明Bigtable技术的融入使得BM-Apriori算法具有更高的效率与可拓展性。

Improved Apriori Algorithm Based on Bigtable and MapReduce

BM-Apriori algorithm was designed for big data to address the poor efficiency problem of Apriori in mining frequent item sets.BM-Apriori takes advantages of Bigtable and MapReduce together to optimize Apriori algorithm.Compared with the improved Apriori algorithm simply based on MapReduce model,timestamp of Bigtable is utilized in this algorithm to avoid generating a large number of key/value pairs.It saves the pattern matching time and scans the database only once.Also,to obtain transaction marks automatically,transaction mark column is added to set list for computing support numbers.BM-Apriori was executed on Hadoop platform.The experimental results show that BM-Apriori has higher efficiency and scalability.