Collaborative filtering and deep learning based recommendation system for cold start items

Recommender system is a specific type of intelligent systems, which exploits historical user ratings on items and/or auxiliary information to make recommendations on items to the users. It plays a critical role in a wide range of online shopping, e-commercial services and social networking applications. Collaborative filtering (CF) is the most popular approaches used for recommender systems, but it suffers from complete cold start (CCS) problem where no rating record are available and incomplete cold start (ICS) problem where only a small number of rating records are available for some new items or users in the system. In this paper, we propose two recommendation models to solve the CCS and ICS problems for new items, which are based on a framework of tightly coupled CF approach and deep learning neural network. A specific deep neural network SADE is used to extract the content features of the items. The state of the art CF model, timeSVD++, which models and utilizes temporal dynamics of user preferences and item features, is modified to take the content features into prediction of ratings for cold start items. Extensive experiments on a large Netflix rating dataset of movies are performed, which show that our proposed recommendation models largely outperform the baseline models for rating prediction of cold start items. The two proposed recommendation models are also evaluated and compared on ICS items, and a flexible scheme of model retraining and switching is proposed to deal with the transition of items from cold start to non-cold start status. The experiment results on Netflix movie recommendation show the tight coupling of CF approach and deep learning neural network is feasible and very effective for cold start item recommendation. The design is general and can be applied to many other recommender systems for online shopping and social networking applications. The solution of cold start item problem can largely improve user experience and trust of recommender systems, and effectively promote cold start items.     

基于码本聚类和因子分解机的多指标推荐算法

传统的协同过滤推荐算法普遍存在数据稀疏问题,且仅利用单一综合评分来计算用户相似度,无法找到在多个指标上偏好相似的用户,因而影响推荐的准确度。多指标评分推荐算法力图寻找在多个指标上偏好相似的用户,但是其评价成本高,导致数据稀疏性问题更加严重。为了找到与目标用户在多个指标上偏好相似的用户,提出基于码本聚类的思想来获取用户在各指标上的评分风格信息,然后基于评分风格信息将用户和项目在各指标上进行双向聚类,最后利用因子分解机模型(Factorization Machines,FMs)基于同一簇内的用户、项目、多指标评分信息、评分风格信息进行推荐。实验结果表明,与传统的协同过滤算法和其他多指标推荐方法相比,基于多指标评分信息的因子分解机推荐算法能够在一定程度上缓解数据稀疏问题,提高推荐的准确度。     

Recommendation algorithm based on item quality and user rating preferences

Recommender systems are one of the most important technologies in e-commerce to help users filter out the overload of information. However, current mainstream recommendation algorithms, such as the collaborative filtering CF family, have problems such as scalability and sparseness. These problems hinder further developments of recommender systems. We propose a new recommendation algorithm based on item quality and user rating preferences, which can significantly decrease the computing complexity. Besides, it is interpretable and works better when the data is sparse. Through extensive experiments on three benchmark data sets, we show that our algorithm achieves higher accuracy in rating prediction compared with the traditional approaches. Furthermore, the results also demonstrate that the problem of rating prediction depends strongly on item quality and user rating preferences, thus opens new paths for further study.     

Collaborative filtering with ordinal scale-based implicit ratings for mobile music recommendations

Collaborative filtering (CF)-based recommender systems represent a promising solution for the rapidly growing mobile music market. However, in the mobile Web environment, a traditional CF system that uses explicit ratings to collect user preferences has a limitation: mobile customers find it difficult to rate their tastes directly because of poor interfaces and high telecommunication costs. Implicit ratings are more desirable for the mobile Web, but commonly used cardinal (interval, ratio) scales for representing preferences are also unsatisfactory because they may increase estimation errors. In this paper, we propose a CF-based recommendation methodology based on both implicit ratings and less ambitious ordinal scales. A mobile Web usage mining (mWUM) technique is suggested as an implicit rating approach, and a specific consensus model typically used in multi-criteria decision-making (MCDM) is employed to generate an ordinal scale-based customer profile. An experiment with the participation of real mobile Web customers shows that the proposed methodology provides better performance than existing CF algorithms in the mobile Web environment.     

基于声誉与基于网络结构的用户聚类比较分析

用户聚类问题是在线用户行为分析的一个重要研究方向。基于在线评分系统,用户声誉反映的是用户对产品打分的准确程度,用户——产品二部分网络结构反映的是用户对产品的品味偏好。结合声誉度量算法,分别采用DBSCAN方法和基于模块度的贪婪算法从用户打分准确程度和品味偏好角度进行用户聚类,提出一种一致性度量指标来衡量根据用户声誉与根据网络结构得到的两种聚类结果之间的联系。两个实证数据集上的实验结果表明根据用户声誉与根据网络结构得到的两种聚类结果是不一致的,说明打分准确程度相似的用户的品味偏好并不相似。     

Opinion-enhanced collaborative filtering for recommender systems through sentiment analysis

The motivation of collaborative filtering (CF) comes from the idea that people often get the best recommendations from someone with similar tastes. With the growing popularity of opinion-rich resources such as online reviews, new opportunities arise as we can identify the preferences from user opinions. The main idea of our approach is to elicit user opinions from online reviews, and map such opinions into preferences that can be understood by CF-based recommender systems. We divide recommender systems into two types depending on the number of product category recommended: the multiple-category recommendation and the single-category recommendation. For the former, sentiment polarity in coarse-grained manner is identified while for the latter fine-grained sentiment analysis is conducted for each product aspect. If the evaluation frequency for an aspect by a user is greater than the average frequency by all users, it indicates that the user is more concerned with that aspect. If a user's rating for an aspect is lower than the average rating by all users, he or she is much pickier than others on that aspect. Through sentiment analysis, we then build an opinion-enhanced user preference model, where the higher the similarity between user opinions the more consistent preferences between users are. Experiment results show that the proposed CF algorithm outperforms baseline methods for product recommendation in terms of accuracy and recall.     

Improving memory-based collaborative filtering via similarity updating and prediction modulation

Memory-based collaborative filtering (CF) makes recommendations based on a collection of user preferences for items. The idea underlying this approach is that the interests of an active user will more likely coincide with those of users who share similar preferences to the active user. Hence, the choice and computation of a similarity measure between users is critical to rating items. This work proposes a similarity update method that uses an iterative message passing procedure. Additionally, this work deals with a drawback of using the popular mean absolute error (MAE) for performance evaluation, namely that ignores ratings distribution. A novel modulation method and an accuracy metric are presented in order to minimize the predictive accuracy error and to evenly distribute predicted ratings over true rating scales. Preliminary results show that the proposed similarity update and prediction modulation techniques significantly improve the predicted rankings.     

基于差值矩阵分解的推荐算法

矩阵分解已经成为预测用户对物品评分的一种常用方法。传统的矩阵分解技术没有考虑到用户评分之间的差异性,针对上述问题在矩阵分解的基础上,提出差值矩阵分解模型。算法将每个用户对物品的评分减去与其社会属性相似用户对该物品评分的平均分,得到一个差值矩阵,然后对差值矩阵进行分解。在Movielens 1M数据集的实验结果表明,该算法的预测精度较贝叶斯概率矩阵分解、矩阵分解、融合用户属性的隐语义模型都有较为明显的提升。     

Collaborative Filtering Using a Regression-Based Approach

The task of collaborative filtering is to predict the preferences of an active user for unseen items given preferences of other users. These preferences are typically expressed as numerical ratings. In this paper, we propose a novel regression-based approach that first learns a number of experts describing relationships in ratings between pairs of items. Based on ratings provided by an active user for some of the items, the experts are combined by using statistical methods to predict the users preferences for the remaining items. The approach was designed to efficiently address the problem of data sparsity and prediction latency that characterise collaborative filtering. Extensive experiments on Eachmovie and Jester benchmark collaborative filtering data show that the proposed regression-based approach achieves improved accuracy and is orders of magnitude faster than the popular neighbour-based alternative. The difference in accuracy was more evident when the number of ratings provided by an active user was small, as is common for real-life recommendation systems. Additional benefits were observed in predicting items with large rating variability. To provide a more detailed characterisation of the proposed algorithm, additional experiments were performed on synthetic data with second-order statistics similar to that of the Eachmovie data. Strong experimental evidence was obtained that the proposed approach can be applied to data over a large range of sparsity scenarios and is superior to non-personalised predictors even when ratings data are very sparse.     

Improvement of collaborative filtering using rating normalization

With the advent of the Internet, the types and amount of information one can access have increased dramatically. In today’s overwhelming information environment, recommendation systems that quickly analyze large amounts of available information and help users find items of interest are increasingly needed. This paper proposes an improvement of an existing preference prediction algorithm to increase the accuracy of recommendation systems. In a recommendation system, prediction of items preferred by users is based on their ratings. However, individual users with the same degree of satisfaction to an item may give different ratings to the item. We intend to make more precise preference prediction by perceiving differences in users’ rating dispositions. The proposed method consists of two processes of perceiving users’ rating dispositions with clustering and of performing rating normalization according to such rating dispositions. The experimental results show that our method yields higher performance than ordinary collaborative filtering approach.     

一种结合共同邻居和用户评分信息的相似度算法

随着互联网的发展,推荐系统逐步得到广泛应用,协同过滤(CF)是其中运用得最早.最成功的技术之一.CF首先根据用户间的相似度,找出每个用户的近邻;然后根据目标用户近邻的评分预测目标用户的评分;最后把预测评分较高的项目推荐给目标用户.因此相似度计算方法直接关系到预测结果的准确性,对推荐起着至关重要的作用.目前,学者们已从不同的角度提出了各种各样的相似度计算方法,其中共同邻居算法(common-neighbors)是一种简单有效的方法.但此法仅考虑了两用户间的共同邻居数,忽略了用户的具体评分信息.针对这个问题对共同邻居算法进行了改进,同时考虑了共同邻居数和用户的评分信息.实验结果表明,改进的共同邻居算法在一定程度上可提高评分预测的准确性.     

A probabilistic rating inference framework for mining user preferences from reviews

We propose a novel Probabilistic Rating infErence Framework, known as Pref, for mining user preferences from reviews and then mapping such preferences onto numerical rating scales. Pref applies existing linguistic processing techniques to extract opinion words and product features from reviews. It then estimates the sentimental orientations (SO) and strength of the opinion words using our proposed relative-frequency-based method. This method allows semantically similar words to have different SO, thereby addresses a major limitation of existing methods. Pref takes the intuitive relationships between class labels, which are scalar ratings, into consideration when assigning ratings to reviews. Empirical results validated the effectiveness of Pref against several related algorithms, and suggest that Pref can produce reasonably good results using a small training corpus. We also describe a useful application of Pref as a rating inference framework. Rating inference transforms user preferences described as natural language texts into numerical rating scales. This allows Collaborative Filtering (CF) algorithms, which operate mostly on databases of scalar ratings, to utilize textual reviews as an additional source of user preferences. We integrated Pref with a classical CF algorithm, and empirically demonstrated the advantages of using rating inference to augment ratings for CF.     

Experimental evaluation of context-dependent collaborative filtering using item splitting

Collaborative Filtering (CF) computes recommendations by leveraging a historical data set of users’ ratings for items. CF assumes that the users’ recorded ratings can help in predicting their future ratings. This has been validated extensively, but in some domains the user’s ratings can be influenced by contextual conditions, such as the time, or the goal of the item consumption. This type of contextual information is not exploited by standard CF models. This paper introduces and analyzes a novel technique for context-aware CF called Item Splitting. In this approach items experienced in two alternative contextual conditions are “split” into two items. This means that the ratings of a split item, e.g., a place to visit, are assigned (split) to two new fictitious items representing for instance the place in summer and the same place in winter. This split is performed only if there is statistical evidence that under these two contextual conditions the items ratings are different; for instance, a place may be rated higher in summer than in winter. These two new fictitious items are then used, together with the unaffected items, in the rating prediction algorithm. When the system must predict the rating for that “split” item in a particular contextual condition (e.g., in summer), it will consider the new fictitious item representing the original one in that particular contextual condition, and will predict its rating. We evaluated this approach on real world, and semi-synthetic data sets using matrix factorization, and nearest neighbor CF algorithms. We show that Item Splitting can be beneficial and its performance depends on the method used to determine which items to split. We also show that the benefit of the method is determined by the relevance of the contextual factors that are used to split.     

An algorithm for efficient privacy-preserving item-based collaborative filtering

Collaborative filtering (CF) methods are widely adopted by existing recommender systems, which can analyze and predict user “ratings” or “preferences” of newly generated items based on user historical behaviors. However, privacy issue arises in this process as sensitive user private data are collected by the recommender server. Recently proposed privacy-preserving collaborative filtering (PPCF) methods, using computation-intensive cryptography techniques or data perturbation techniques are not appropriate in real online services. In this paper, an efficient privacy-preserving item-based collaborative filtering algorithm is proposed, which can protect user privacy during online recommendation process without compromising recommendation accuracy and efficiency. The proposed method is evaluated using the Netflix Prize dataset. Experimental results demonstrate that the proposed method outperforms a randomized perturbation based PPCF solution and a homomorphic encryption based PPCF solution by over 14X and 386X, respectively, in recommendation efficiency while achieving similar or even better recommendation accuracy.     

一种融合项目特征和移动用户信任关系的推荐算法

协同过滤推荐系统中普遍存在评分数据稀疏问题.传统的协同过滤推荐系统中的余弦、Pearson 等方法都是基于共同评分项目来计算用户间的相似度;而在稀疏的评分数据中,用户间共同评分的项目所占比重较小,不能准确地找到偏好相似的用户,从而影响协同过滤推荐的准确度.为了改变基于共同评分项目的用户相似度计算,使用推土机距离（earth mover's distance,简称EMD）实现跨项目的移动用户相似度计算,提出了一种融合项目特征和移动用户信任关系的协同过滤推荐算法.实验结果表明：与余弦、Pearson 方法相比,融合项目特征的用户相似度计算方法能够缓解评分数据稀疏对协同过滤算法的影响.所提出的推荐算法能够提高移动推荐的准确度.     

基于受约束偏置的概率矩阵分解算法

在概率矩阵分解(PMF)模型拟合之后,评分较少用户的特征趋近于先验分布的平均值,导致对其评分预测接近物品的平均评分.受约束概率矩阵分解(CPMF)未考虑到不同评分系统的整体差异以及数据集内部用户与物品存在的固有属性.针对以上问题,提出将传统矩阵分解中的用户和物品偏置项以及全局平均分结合受约束概率矩阵分解来建立新的矩阵分解算法.算法利用整体平均分衡量不同评分系统,在采用偏置来表示用户以及物品之间相互独立的属性的同时,引入约束使行为相近用户拥有相近的用户偏置,从而提高预测精度.在两个真实数据集上的实验结果表明,该算法相对于PMF和CPMF算法预测精度得到了提高.     

Personalized recommendation based on review topics

The traditional collaborative filtering algorithm is a successful recommendation technology. The core idea of this algorithm is to calculate user or item similarity based on user ratings and then to predict ratings and recommend items based on similar users’ or similar items’ ratings. However, real applications face a problem of data sparsity because most users provide only a few ratings, such that the traditional collaborative filtering algorithm cannot produce satisfactory results. This paper proposes a new topic model-based similarity and two recommendation algorithms: user-based collaborative filtering with topic model algorithm (UCFTM, in this paper) and item-based collaborative filtering with topic model algorithm (ICFTM, in this paper). Each review is processed using the topic model to generate review topic allocations representing a user’s preference for a product’s different features. The UCFTM algorithm aggregates all topic allocations of reviews by the same user and calculates the user most valued features representing product features that the user most values. User similarity is calculated based on user most valued features, whereas ratings are predicted from similar users’ ratings. The ICFTM algorithm aggregates all topic allocations of reviews for the same product, and item most valued features representing the most valued features of the product are calculated. Item similarity is calculated based on item most valued features, whereas ratings are predicted from similar items’ ratings. Experiments on six data sets from Amazon indicate that when most users give only one review and one rating, our algorithms exhibit better prediction accuracy than other traditional collaborative filtering and state-of-the-art topic model-based recommendation algorithms.     

基于位置的非对称相似性度量的协同过滤推荐算法

为提升推荐系统的准确率,针对传统协同过滤(CF)推荐算法没有有效使用位置信息的问题,提出了一种基于位置的非对称相似性度量的协同过滤推荐算法(LBASCF)。首先,分别利用用户-商品评分矩阵和用户历史消费位置,计算出用户间的余弦相似性和基于位置的非对称相似性;其次,将余弦相似性与基于位置的相似性融合,得到一个新的非对称用户相似性,融合后的相似性能够同时反映用户在位置上和兴趣上的偏好;最后,根据用户的最近邻居对商品的评分向用户推荐新的商品。用某点评数据集和Foursquare数据集对算法的有效性进行了评估。在某点评数据集实验结果证明,与CF相比,LBASCF的召回率和精确率分别提高了1.64%和0.37%;与位置感知协同过滤推荐系统(LARS)方法比较,LBASCF的召回率和精确率分别提高了1.53%和0.35%。实验结果表明,LBASCF相对于CF和LARS在基于位置服务的应用中能够有效提高系统的推荐质量。     

基于评分填充和时间的加权Slope One算法

针对数据稀疏性,常用的评分矩阵填充方法主要是通过平均数、中位数等进行填充,该文提出一种新的评分矩阵填充方法。利用项目-属性矩阵计算用户对项目属性偏好,由于每个项目都有各自属性,从而可以获得用户对项目的偏好值,以用户平均评分为基准,实现对评分矩阵填充。基于填充后的评分矩阵,又考虑到用户兴趣爱好随时间会发生改变,因此引入时间因子,提出一种基于评分矩阵填充和时间因子的加权slope one算法(FTWSOA)。通过时间函数修正评分矩阵,优化的评分数据可以更好地体现用户兴趣爱好随时间变化的情况。在时间加权的评分矩阵下,计算出属性兴趣偏好,在共同评分很少甚至没有时,利用属性兴趣偏好可以较为准确地计算用户相似度。由于在共同很少或者没有时,原始评分矩阵中用户没有交集,而在属性兴趣矩阵下用户会存在交集,因此,使用参数λ将填充矩阵下的用户相似度和属性兴趣偏好矩阵下的用户相似度相结合得到最终的用户相似度,可以缓解在稀疏数据下相似度计算性能差的问题,最后使用加权slope one预测评分时,将时间衰减函数加入到预测公式中来优化预测评分公式。通过在MovieLens100k数据集上的实验表明,相比于其他算法,FTWSOA算法准确度有所提高。     

Personalizing recommendation diversity based on user personality

In recent years, diversity has attracted increasing attention in the field of recommender systems because of its ability of catching users’ various interests by providing a set of dissimilar items. There are few endeavors to personalize the recommendation diversity being tailored to individual users’ diversity needs. However, they mainly depend on users’ behavior history such as ratings to customize diversity, which has two limitations: (1) They neglect taking into account a user’s needs that are inherently caused by some personal factors such as personality; (2) they fail to work well for new users who have little behavior history. In order to address these issues, this paper proposes a generalized, dynamic personality-based greedy re-ranking approach to generating the recommendation list. On one hand, personality is used to estimate each user’s diversity preference. On the other hand, personality is leveraged to alleviate the cold-start problem of collaborative filtering recommendations. The experimental results demonstrate that our approach significantly outperforms related methods (including both non-diversity-oriented and diversity-oriented methods) in terms of metrics measuring recommendation accuracy and personalized diversity degree, especially in the cold-start setting.