基于Spark的混合推荐算法研究

随着电子商务的发展，基于协同过滤的推荐算法越来越受欢迎，与此同时，该算法的缺陷也越来越明显，如数据稀疏性、系统可扩展性等。另外传统的单机计算模型也难以满足海量数据的实时推荐需求。为此，提出一种利用Spark计算模型实现分布式推荐的方法。该推荐方法采用基于谱聚类和朴素贝叶斯的混合推荐算法，同时使用增量式更新，在不全部重新训练模型的基础上，对模型进行局部修改。实验结果表明，较传统的单机模式推荐算法，基于Spark计算模型的分布式推荐算法，在一定程度上克服了数据稀疏性，提高了系统的可扩展性，降低了系统的响应时间。

Research on hybrid recommendation algorithm based on Spark Technology

Collaborative filtering-based recommender systems have become extremely popular in recent years, due to the development of e-commerce. By the way, it has some limitations such as sparsity, scalability. Moreover traditional stand-alone model is difficult to meet the needs of massive data in real-time recommendation. Therefore, a distributed recommendation method based on Spark computing model is proposed, which the theory is based on spectral clustering and Naive Bayes. In addition, the hybrid method used the increment update schemes to refresh the ratings and improve the precision of the system, without all the re-training model. The experimental results demonstrate that to be compared with traditional stand-alone mode recommendation algorithm, the distributed recommendation algorithm proposed overcomes sparsity and scalability problem to a certain extent and has higher scalability and reduces the response time of the system.