Parallel matrix factorization for recommender systems

Matrix factorization, when the matrix has missing values, has become one of the leading techniques for recommender systems. To handle web-scale datasets with millions of users and billions of ratings, scalability becomes an important issue. Alternating least squares (ALS) and stochastic gradient descent (SGD) are two popular approaches to compute matrix factorization, and there has been a recent flurry of activity to parallelize these algorithms. However, due to the cubic time complexity in the target rank, ALS is not scalable to large-scale datasets. On the other hand, SGD conducts efficient updates but usually suffers from slow convergence that is sensitive to the parameters. Coordinate descent, a classical optimization approach, has been used for many other large-scale problems, but its application to matrix factorization for recommender systems has not been thoroughly explored. In this paper, we show that coordinate descent-based methods have a more efficient update rule compared to ALS and have faster and more stable convergence than SGD. We study different update sequences and propose the CCD++ algorithm, which updates rank-one factors one by one. In addition, CCD++ can be easily parallelized on both multi-core and distributed systems. We empirically show that CCD++ is much faster than ALS and SGD in both settings. As an example, with a synthetic dataset containing 14.6 billion ratings, on a distributed memory cluster with 64 processors, to deliver the desired test RMSE, CCD++ is 49 times faster than SGD and 20 times faster than ALS. When the number of processors is increased to 256, CCD++ takes only 16 s and is still 40 times faster than SGD and 20 times faster than ALS.     

Two-level matrix factorization for recommender systems

Many existing recommendation methods such as matrix factorization (MF) mainly rely on user–item rating matrix, which sometimes is not informative enough, often suffering from the cold-start problem. To solve this challenge, complementary textual relations between items are incorporated into recommender systems (RS) in this paper. Specifically, we first apply a novel weighted textual matrix factorization (WTMF) approach to compute the semantic similarities between items, then integrate the inferred item semantic relations into MF and propose a two-level matrix factorization (TLMF) model for RS. Experimental results on two open data sets not only demonstrate the superiority of TLMF model over benchmark methods, but also show the effectiveness of TLMF for solving the cold-start problem.     

User-controlled federated matrix factorization for recommender systems

Journal of Intelligent Information Systems - Recommendation services have been extensively adopted in various user-centered applications to help users navigate a vast space of possible choices. In...     

Joint user knowledge and matrix factorization for recommender systems

Currently, most of the existing recommendation methods treat social network users equally, which assume that the effect of recommendation on a user is decided by the user’s own preferences and social influence. However, a user’s own knowledge in a field has not been considered. In other words, to what extent does a user accept recommendations in social networks need to consider the user’s own knowledge or expertise in the field. In this paper, we propose a novel matrix factorization recommendation algorithm based on integrating social network information such as trust relationships, rating information of users and users’ own knowledge. Specifically, since we cannot directly measure a user’s knowledge in the field, we first use a user’s status in a social network to indicate a user’s knowledge in a field, and users’ status is inferred from the distributions of users’ ratings and followers across fields or the structure of domain-specific social network. Then, we model the final rating of decision-making as a linear combination of the user’s own preferences, social influence and user’s own knowledge. Experimental results on real world data sets show that our proposed approach generally outperforms the state-of-the-art recommendation algorithms that do not consider the knowledge level difference between the users.     

Forgetting techniques for stream-based matrix factorization in recommender systems

Forgetting is often considered a malfunction of intelligent agents; however, in a changing world forgetting has an essential advantage. It provides means of adaptation to changes by removing effects of obsolete (not necessarily old) information from models. This also applies to intelligent systems, such as recommender systems, which learn users’ preferences and predict future items of interest. In this work, we present unsupervised forgetting techniques that make recommender systems adapt to changes of users’ preferences over time. We propose eleven techniques that select obsolete information and three algorithms that enforce the forgetting in different ways. In our evaluation on real-world datasets, we show that forgetting obsolete information significantly improves predictive power of recommender systems.     

引入评分偏置的二项矩阵分解推荐算法

针对推荐系统中的评分预测问题,在矩阵分解的基础上实现了一种修正的二项矩阵分解算法。假设用户对物品的评分基于二项分布,由于用户的评分习惯存在差异,物品的受欢迎程度也存在差异,导致用户—物品评分矩阵存在偏置量。通过引入偏置量对矩阵分解和评分预测进行修正,采用最大后验估计建模,并通过随机梯度下降算法优化模型。实验结果表明,在MovieLens 100K数据集上,引入评分偏置的二项矩阵分解算法在推荐精度、离线计算时间等方面均优于传统的二项矩阵分解算法。     

Analyzing and improving stability of matrix factorization for recommender systems

Journal of Intelligent Information Systems - Thanks to their flexibility and scalability, collaborative embedding-based models are widely employed for the top-N recommendation task. Their goal is...     

Bounded matrix factorization for recommender system

Matrix factorization has been widely utilized as a latent factor model for solving the recommender system problem using collaborative filtering. For a recommender system, all the ratings in the rating matrix are bounded within a pre-determined range. In this paper, we propose a new improved matrix factorization approach for such a rating matrix, called Bounded Matrix Factorization (BMF), which imposes a lower and an upper bound on every estimated missing element of the rating matrix. We present an efficient algorithm to solve BMF based on the block coordinate descent method. We show that our algorithm is scalable for large matrices with missing elements on multicore systems with low memory. We present substantial experimental results illustrating that the proposed method outperforms the state of the art algorithms for recommender system such as stochastic gradient descent, alternating least squares with regularization, SVD++ and Bias-SVD on real-world datasets such as Jester, Movielens, Book crossing, Online dating and Netflix.     

NEG-MF:一种针对推荐系统的矩阵分解图嵌入模型

传统的矩阵分解图嵌入模型由于不对大量未知关系建模,其性能面临着很大的挑战性.为了提升矩阵分解模型的性能,提出了一种基于负采样技术的矩阵分解模型NEG-MF.该模型能够从跳数大于6的邻居节点中进行负采样,以降低模型生成图嵌入时对于负样本的偏差.在DBLP数据集上做的大量实验结果表明,相比其他的基线方法,基于NEG-MF的推荐算法在学术合作关系推荐问题上的性能有明显地提升.     

A framework for tag-aware recommender systems

In social tagging system, a user annotates a tag to an item. The tagging information is utilized in recommendation process. In this paper, we propose a hybrid item recommendation method to mitigate limitations of existing approaches and propose a recommendation framework for social tagging systems. The proposed framework consists of tag and item recommendations. Tag recommendation helps users annotate tags and enriches the dataset of a social tagging system. Item recommendation utilizes tags to recommend relevant items to users. We investigate association rule, bigram, tag expansion, and implicit trust relationship for providing tag and item recommendations on the framework. The experimental results show that the proposed hybrid item recommendation method generates more appropriate items than existing research studies on a real-world social tagging dataset.     

A framework for collaborative filtering recommender systems

As the use of recommender systems becomes more consolidated on the Net, an increasing need arises to develop some kind of evaluation framework for collaborative filtering measures and methods which is capable of not only testing the prediction and recommendation results, but also of other purposes which until now were considered secondary, such as novelty in the recommendations and the users’ trust in these. This paper provides: (a) measures to evaluate the novelty of the users’ recommendations and trust in their neighborhoods, (b) equations that formalize and unify the collaborative filtering process and its evaluation, (c) a framework based on the above-mentioned elements that enables the evaluation of the quality results of any collaborative filtering applied to the desired recommender systems, using four graphs: quality of the predictions, the recommendations, the novelty and the trust.     

A parallel matrix factorization based recommender by alternating stochastic gradient decent

Collaborative Filtering (CF) can be achieved by Matrix Factorization (MF) with high prediction accuracy and scalability. Most of the current MF based recommenders, however, are serial, which prevent them sharing the efficiency brought by the rapid progress in parallel programming techniques. Aiming at parallelizing the CF recommender based on Regularized Matrix Factorization (RMF), we first carry out the theoretical analysis on the parameter updating process of RMF, whereby we can figure out that the main obstacle preventing the model from parallelism is the inter-dependence between item and user features. To remove the inter-dependence among parameters, we apply the Alternating Stochastic Gradient Solver (ASGD) solver to deal with the parameter training process. On this basis, we subsequently propose the parallel RMF (P-RMF) model, of which the training process can be parallelized through simultaneously training different user/item features. Experiments on two large, real datasets illustrate that our P-RMF model can provide a faster solution to CF problem when compared to the original RMF and another parallel MF based recommender.     

Hybreed: A software framework for developing context-aware hybrid recommender systems

This article introduces Hybreed, a software framework for building complex context-aware applications, together with a set of components that are specifically targeted at developing hybrid, context-aware recommender systems. Hybreed is based on a concept for processing context that we call dynamic contextualization. The underlying notion of context is very generic, enabling application developers to exploit sensor-based physical factors as well as factors derived from user models or user interaction. This approach is well aligned with context definitions that emphasize the dynamic and activity-oriented nature of context. As an extension of the generic framework, we describe Hybreed RecViews, a set of components facilitating the development of context-aware and hybrid recommender systems. With Hybreed and RecViews, developers can rapidly develop context-aware applications that generate recommendations for both individual users and groups. The framework provides a range of recommendation algorithms and strategies for producing group recommendations as well as templates for combining different methods into hybrid recommenders. Hybreed also provides means for integrating existing user or product data from external sources such as social networks. It combines aspects known from context processing frameworks with features of state-of-the-art recommender system frameworks, aspects that have been addressed only separately in previous research. To our knowledge, Hybreed is the first framework to cover all these aspects in an integrated manner. To evaluate the framework and its conceptual foundation, we verified its capabilities in three different use cases. The evaluation also comprises a comparative assessment of Hybreed’s functional features, a comparison to existing frameworks, and a user study assessing its usability for developers. The results of this study indicate that Hybreed is intuitive to use and extend by developers.     

A cross-layer framework for privacy enhancement in RFID systems

In this paper, we introduce a cross-layer framework for enhancing privacy in RFID systems. The framework relies on mechanisms in the physical (PHY) layer, as well as the medium access control (MAC) layer, to provide flexible protection over the unique identifiers of low-cost RFID tags. Such a framework prevents adversaries and malicious parties from tracking RFID tags through the monitoring of their unique identifiers. More specifically, our framework relies on masking of the identifier at the PHY layer, whereby bit-collisions are induced between the backscattered tag identifier and a protective mask, such that a legitimate reader can be allowed to recover the tag identifier but an illegitimate party would not be able to do so. To strengthen the level of protection provided by the bit-collision masking method, we present the randomized bit encoding scheme that is used in our framework. In addition, we also incorporate mechanisms in the MAC layer, and make use of cross-layer interactions between the MAC and the PHY layers to provide flexible privacy protection. This allows tags that do not require privacy protection to be read conveniently while allowing tags that need to be protected to stay protected.     

融合标签相似度的差分隐私矩阵分解推荐算法

推荐系统需要利用到大量的用户行为数据,这些数据极有可能暴露用户的喜好,给人们关心的隐私问题带来巨大的挑战。为保证推荐精度与用户隐私,提出一种结合差分隐私与标签信息的矩阵分解推荐模型。该模型首先将标签信息加入到项目相似度的计算过程;随后融入到矩阵分解推荐模型中提高推荐精度;最后运用随机梯度下降法求解模型最优值。为解决用户隐私问题,将拉普拉斯噪声划分成两部分,分别加入项目相似度与梯度求解过程中,使得整个推荐过程满足ε-差分隐私,并在一个真实的数据集上分析验证算法的有效性。实验表明,提出的方法能在保证用户隐私的情况下,仍具有较高的推荐精度。     

A flexible framework for evaluating user and item fairness in recommender systems

User Modeling and User-Adapted Interaction - One common characteristic of research works focused on fairness evaluation (in machine learning) is that they call for some form of parity (equality)...     

A nonnegative matrix factorization framework for semi-supervised document clustering with dual constraints

In this paper, we propose a new semi-supervised co-clustering algorithm Orthogonal Semi-Supervised Nonnegative Matrix Factorization (OSS-NMF) for document clustering. In this new approach, the clustering process is carried out by incorporating both prior domain knowledge of data points (documents) in the form of pair-wise constraints and category knowledge of features (words) into the NMF co-clustering framework. Under this framework, the clustering problem is formulated as the problem of finding the local minimizer of objective function, taking into account the dual prior knowledge. The update rules are derived, and an iterative algorithm is designed for the co-clustering process. Theoretically, we prove the correctness and convergence of our algorithm and demonstrate its mathematical rigorous. Our experimental evaluations show that the proposed document clustering model presents remarkable performance improvements with those constraints.     

A novel framework to alleviate the sparsity problem in context-aware recommender systems

Recommender systems have become indispensable for services in the era of big data. To improve accuracy and satisfaction, context-aware recommender systems (CARSs) attempt to incorporate contextual information into recommendations. Typically, valid and influential contexts are determined in advance by domain experts or feature selection approaches. Most studies have focused on utilizing the unitary context due to the differences between various contexts. Meanwhile, multi-dimensional contexts will aggravate the sparsity problem, which means that the user preference matrix would become extremely sparse. Consequently, there are not enough or even no preferences in most multi-dimensional conditions. In this paper, we propose a novel framework to alleviate the sparsity issue for CARSs, especially when multi-dimensional contextual variables are adopted. Motivated by the intuition that the overall preferences tend to show similarities among specific groups of users and conditions, we first explore to construct one contextual profile for each contextual condition. In order to further identify those user and context subgroups automatically and simultaneously, we apply a co-clustering algorithm. Furthermore, we expand user preferences in a given contextual condition with the identified user and context clusters. Finally, we perform recommendations based on expanded preferences. Extensive experiments demonstrate the effectiveness of the proposed framework.