基于Spark的矩阵分解推荐算法

针对传统矩阵分解算法在处理海量数据信息时所面临的处理速度和计算资源的瓶颈问题,利用Spark在内存计算和迭代计算上的优势,提出了Spark框架下的矩阵分解并行化算法。首先,依据历史数据矩阵初始化用户因子矩阵和项目因子矩阵;其次,迭代更新因子矩阵,将迭代结果置于内存中作为下次迭代的输入;最后,迭代结束时得到矩阵推荐模型。通过在GroupLens网站上提供的MovieLens数据集上的实验结果表明,加速比(Speedup)值达到了线性的结果,该算法可以提高协同过滤推荐算法在大数据规模下的执行效率。

Matrix factorization recommendation algorithm based on Spark

In order to solve the bottleneck problems of processing speed and resource allocation, a Spark based matrix factorization recommendation algorithm was proposed. Firstly, user factor matrix and item factor matrix were initialized according to historical data. Secondly, factor matrix was iteratively updated and the result was stored in memory as the input of next iteration. Finally, recommendation model was generated when iteration ended. The experiment on MovieLens shows that the speedup is linear and the proposed Spark based algorithm can save time and significantly improve the execution efficiency of collaborative filtering recommendation algorithm.