Explaining the user experience of recommender systems

Research on recommender systems typically focuses on the accuracy of prediction algorithms. Because accuracy only partially constitutes the user experience of a recommender system, this paper proposes a framework that takes a user-centric approach to recommender system evaluation. The framework links objective system aspects to objective user behavior through a series of perceptual and evaluative constructs (called subjective system aspects and experience, respectively). Furthermore, it incorporates the influence of personal and situational characteristics on the user experience. This paper reviews how current literature maps to the framework and identifies several gaps in existing work. Consequently, the framework is validated with four field trials and two controlled experiments and analyzed using Structural Equation Modeling. The results of these studies show that subjective system aspects and experience variables are invaluable in explaining why and how the user experience of recommender systems comes about. In all studies we observe that perceptions of recommendation quality and/or variety are important mediators in predicting the effects of objective system aspects on the three components of user experience: process (e.g. perceived effort, difficulty), system (e.g. perceived system effectiveness) and outcome (e.g. choice satisfaction). Furthermore, we find that these subjective aspects have strong and sometimes interesting behavioral correlates (e.g. reduced browsing indicates higher system effectiveness). They also show several tradeoffs between system aspects and personal and situational characteristics (e.g. the amount of preference feedback users provide is a tradeoff between perceived system usefulness and privacy concerns). These results, as well as the validated framework itself, provide a platform for future research on the user-centric evaluation of recommender systems.     

Effects of time affordance and operation mode on a smart microwave oven touch-sensitive user Interface design

Smart homes have revolutionized our daily lives. With today’s fast-paced lifestyle, pursuing a high quality of life has become many people’s goal and motivation. The purpose of this study is to investigate the user interface design of smart microwave ovens pertinent to time affordance and operation mode. A 2?×?3 mixed factorial design was planned to help explore whether different time affordances (i.e., high and low) and operation modes (i.e., traditional, touch, and smart) may affect users’ task performance and subjective evaluation. Using the convenience sampling method, 24 adults were recruited to participate in the experiment. The experimental data were collected pertinent to task performance, the system usability scale, and through questionnaires created using a 7-point Likert scale, and semi-structured interviews. The generated results revealed that: (1) there was a significant difference in the time affordance. Multiple time information cues can help reduce uncertainty, providing high time affordance to the participants. (2) There were significant differences among different operation modes. A simple and intuitive “smart” type is in line with user expectations. (3) The overall analysis of task performance and satisfaction consistently showed that high time affordance is better than low time affordance in all aspects, and the “smart” type had the best task performance. (4) The user interface design should be followed by users’ experience and the features of the touch product. Partially smart and custom function adjustments may effectively improve the user’s control of smart products.

     

Behavioral data mining to produce novel and serendipitous friend recommendations in a social bookmarking system

In the last few years, social media systems have experienced a fast growth. The amount of content shared in these systems increases fast, leading users to face the well known “interaction overload” problem, i.e., they are overwhelmed by content, so it becomes difficult to come across interesting items. To overcome this problem, social recommender systems have been recently designed and developed in order to filter content and recommend to users only interesting items. This type of filtering is usually affected by the “over-specialization” problem, which is related to recommendations that are too similar to the items already considered by the users. This paper proposes a friend recommender system that operates in the social bookmarking application domain and is based on behavioral data mining, i.e., on the exploitation of the users activity in a social bookmarking system. Experimental results show how this type of mining is able to produce accurate friend recommendations, allowing users to get to know bookmarked resources that are both novel and serendipitous. Using this approach, the impact of the “interaction overload” and the “over-specialization” problems is strongly reduced.     

Incremental critiquing

Conversational recommender systems guide users through a product space, alternatively making concrete product suggestions and eliciting the user's feedback. Critiquing is a common form of user feedback, where users provide limited feedback at the feature-level by constraining a feature's value-space. For example, a user may request a cheaper product, thus critiquing the price feature. Usually, when critiquing is used in conversational recommender systems, there is little or no attempt to monitor successive critiques within a given recommendation session. In our experience this can lead to inefficiencies on the part of the recommender system, and confusion on the part of the user. In this paper we describe an approach to critiquing that attempts to consider a user's critiquing history, as well as their current critique, when making new recommendations. We provide experimental evidence to show that this has the potential to significantly improve recommendation efficiency.     

Evaluating product search and recommender systems for E-commerce environments

Online systems that help users select the most preferential item from a large electronic catalog are known as product search and recommender systems. Evaluation of various proposed technologies is essential for further development in this area. This paper describes the design and implementation of two user studies in which a particular product search tool, known as example critiquing, was evaluated against a chosen baseline model. The results confirm that example critiquing significantly reduces users’ task time and error rate while increasing decision accuracy. Additionally, the results of the second user study show that a particular implementation of example critiquing also made users more confident about their choices. The main contribution is that through these two user studies, an evaluation framework of three criteria was successfully identified, which can be used for evaluating general product search and recommender systems in E-commerce environments. These two experiments and the actual procedures also shed light on some of the most important issues which need to be considered for evaluating such tools, such as the preparation of materials for evaluation, user task design, the context of evaluation, the criteria, the measures and the methodology of result analyses.     

The effects of recommendations’ presentation on persuasion and satisfaction in a movie recommender system

The explosive growth of Internet applications and content, during the last decade, has revealed an increasing need for information filtering and recommendation. Most research in the area of recommendation systems has focused on designing and implementing efficient algorithms that provide accurate recommendations. However, the selection of appropriate recommendation content and the presentation of information are equally important in creating successful recommender applications. This paper addresses issues related to the presentation of recommendations in the movies domain. The current work reviews previous research approaches and popular recommender systems, and focuses on user persuasion and satisfaction. In our experiments, we compare different presentation methods in terms of recommendations’ organization in a list (i.e. top N-items list and structured overview) and recommendation modality (i.e. simple text, combination of text and image, and combination of text and video). The most efficient presentation methods, regarding user persuasion and satisfaction, proved to be the “structured overview” and the “text and video” interfaces, while a strong positive correlation was also found between user satisfaction and persuasion in all experimental conditions.     

基于用户隐性反馈行为的下一个购物篮推荐

下一个购物篮推荐是当前电子商务领域中极其重要的一项任务,传统的下一个购物篮推荐方法主要分为时序推荐模型和总体推荐模型。这些方法对点击、收藏、加入购物车等用户的隐性反馈行为利用得不够,并且没有考虑用户行为偏好的时间敏感性。该文提出了一种基于用户隐性反馈行为的下一个购物篮推荐方法,将用户行为按照一定的时间窗口进行划分,对于每个窗口从多个维度抽取用户对商品的时序偏好特征,运用深度学习领域的卷积神经网络模型进行分类器训练。在真实数据集中的实验结果表明,与传统的线性模型和树模型等分类器相比,该文提出的卷积神经网络框架具有较强的特征萃取能力和泛化能力,提高了推荐系统的用户满意度。     

Body in the interactive game: How interface embodiment affects physical activity and health behavior change

Does the delivery platform for a health behavior game contribute to its effectiveness? With the growing popularity of interactive video games that combine physical exercise with gameplay, known as “exergames,” there has been a burgeoning interest in their impact on users’ exercise attitudes and behavioral outcomes. This study examines how the level of user interface embodiment, the degree to which the user’s body interacts with the game, affects the user’s experience, game behavior, and intention for behavior change. We conducted a between-participants experiment in which participants (N = 119) played an exergame under one of the three levels of user interface embodiment (low, medium, and high). Our results revealed a significant positive main effect of user interface embodiment on user experience (i.e., the sense of being in the game, “presence,” and enjoyment); level of energy expenditure (change in heart rate); and intention to further engage in exergame-play exercise but not necessarily to increase exercise in the physical world. A further analysis revealed the mediating roles of user experience in the association between user interface embodiment and intention to repeat exergaming and a potential link between heart rate change and level of presence in the game. We conclude that type of interface is a key variable in this health communication environment, affecting user experience, behavior, and some intention for behavior change.     

Evaluating user experience of adaptive digital educational games with Activity Theory

Adaptive digital educational games (DEGs) providing players with relevant interventions can enhance gameplay experience. This advance in game design, however, renders the user experience (UX) evaluation of DEGs even more challenging. To tackle this challenge, we developed a four-dimension evaluation framework (i.e., gaming experience, learning experience, adaptivity, and usability) and applied it to an empirical study with a DEG on teaching geography. Mixed-method approaches were adopted to collect data with 16 boys aged 10–11. Specifically, a so-called Dyadic User Experience Tests (DUxT) was employed; participants were paired up to assume different roles during gameplay. Learning efficacy was evaluated with a pre-post intervention measurement using a domain-specific questionnaire. Learning experience, gaming experiences and usability were evaluated with intensive in situ observations and interviews guided by a multidimensional scheme; content analysis of these transcribed audio data was supplemented by video analysis. Effectiveness of adaptivity algorithms was planned to be evaluated with automatic logfiles, which, unfortunately, could not be realised due to some technical problem. Nonetheless, the user-based data could offer some insights into this issue. Furthermore, we attempted to bridge the existing gap in UX research – the lack of theoretical frameworks in understanding user experience – by adopting Engeström's (1987) extended framework of Activity Theory (AT) that provides contextual information essential for understanding contradictions and breakdowns observed in the interactions between the game players. The dyadic gameplay setting allows us to explore the issue of group UX. Implications for further applications of the AT framework in the UX research, especially the interplay between evaluation and redesign (i.e., downstream utility of UX evaluation methods), are discussed.     

Ecosystem on the Web: non-linear mining and forecasting of co-evolving online activities

Given a large collection of co-evolving online activities, such as searches for the keywords “Xbox”, “PlayStation” and “Wii”, how can we find patterns and rules? Are these keywords related? If so, are they competing against each other? Can we forecast the volume of user activity for the coming month? We conjecture that online activities compete for user attention in the same way that species in an ecosystem compete for food. We present EcoWeb, (i.e., Ecosystem on the Web), which is an intuitive model designed as a non-linear dynamical system for mining large-scale co-evolving online activities. Our second contribution is a novel, parameter-free, and scalable fitting algorithm, EcoWeb-Fit, that estimates the parameters of EcoWeb. Extensive experiments on real data show that EcoWeb is effective, in that it can capture long-range dynamics and meaningful patterns such as seasonalities, and practical, in that it can provide accurate long-range forecasts. EcoWeb consistently outperforms existing methods in terms of both accuracy and execution speed.     

User-controllable personalization: A case study with SetFusion

In this research we investigated the role of user controllability on personalized systems by implementing and studying a novel interactive recommender interface, SetFusion. We examined whether allowing the user to control the process of fusing or integrating different algorithms (i.e., different sources of relevance) resulted in increased engagement and a better user experience. The essential contribution of this research stems from the results of a user study (N=40) of controllability in a scenario where users could fuse different recommendation approaches, with the possibility of inspecting and filtering the items recommended. First, we introduce an interactive Venn diagram visualization, which combined with sliders, can provide an efficient visual paradigm for information filtering. Second, we provide a three-fold evaluation of the user experience: objective metrics, subjective user perception, and behavioral measures. Through the analysis of these metrics, we confirmed results from recent studies, such as the effect of trusting propensity on accepting the recommendations and also unveiled the importance of features such as being a native speaker. Our results present several implications for the design and implementation of user-controllable personalized systems.     

Sequential event prediction

In sequential event prediction, we are given a “sequence database” of past event sequences to learn from, and we aim to predict the next event within a current event sequence. We focus on applications where the set of the past events has predictive power and not the specific order of those past events. Such applications arise in recommender systems, equipment maintenance, medical informatics, and in other domains. Our formalization of sequential event prediction draws on ideas from supervised ranking. We show how specific choices within this approach lead to different sequential event prediction problems and algorithms. In recommender system applications, the observed sequence of events depends on user choices, which may be influenced by the recommendations, which are themselves tailored to the user’s choices. This leads to sequential event prediction algorithms involving a non-convex optimization problem. We apply our approach to an online grocery store recommender system, email recipient recommendation, and a novel application in the health event prediction domain.     

An algorithm for efficient privacy-preserving item-based collaborative filtering

Collaborative filtering (CF) methods are widely adopted by existing recommender systems, which can analyze and predict user “ratings” or “preferences” of newly generated items based on user historical behaviors. However, privacy issue arises in this process as sensitive user private data are collected by the recommender server. Recently proposed privacy-preserving collaborative filtering (PPCF) methods, using computation-intensive cryptography techniques or data perturbation techniques are not appropriate in real online services. In this paper, an efficient privacy-preserving item-based collaborative filtering algorithm is proposed, which can protect user privacy during online recommendation process without compromising recommendation accuracy and efficiency. The proposed method is evaluated using the Netflix Prize dataset. Experimental results demonstrate that the proposed method outperforms a randomized perturbation based PPCF solution and a homomorphic encryption based PPCF solution by over 14X and 386X, respectively, in recommendation efficiency while achieving similar or even better recommendation accuracy.     

User demographics prediction based on mobile data

Demographics prediction is an important component of user profile modeling. The accurate prediction of users’ demographics can help promote many applications, ranging from web search, personalization to behavior targeting. In this paper, we focus on how to predict users’ demographics, including “gender”, “job type”, “marital status”, “age” and “number of family members”, based on mobile data, such as users’ usage logs, physical activities and environmental contexts. The core idea is to build a supervised learning framework, where each user is represented as a feature vector and users’ demographics are considered as prediction targets. The most important component is to construct features from raw data and then supervised learning models can be applied. We propose a feature construction framework, CFC (contextual feature construction), where each feature is defined as the conditional probability of one user activity under the given contexts. Consequently, besides employing standard supervised learning models, we propose a regularized multi-task learning framework to model different kinds of demographics predictions collectively. We also propose a cost-sensitive classification framework for regression tasks, in order to benefit from the existing dimension reduction methods. Finally, due to the limited training instances, we employ ensemble to avoid overfitting. The experimental results show that the framework achieves classification accuracies on “gender”, “job” and “marital status” as high as 96%, 83% and 86%, respectively, and achieves Root Mean Square Error (RMSE) on “age” and “number of family members” as low as 0.69 and 0.66 respectively, under the leave-one-out evaluation.     

Product recommendation with temporal dynamics

In many E-commerce recommender systems, a special class of recommendation involves recommending items to users in a life cycle. For example, customers who have babies will shop on Diapers.com within a relatively long period, and purchase different products for babies within different growth stages. Traditional recommendation algorithms produce recommendation lists similar to items that the target user has accessed before (content filtering), or compute recommendation by analyzing the items purchased by the users who are similar to the target user (collaborative filtering). Such recommendation paradigms cannot effectively resolve the situation with a life cycle, i.e., the need of customers within different stages might vary significantly. In this paper, we model users’ behavior with life cycles by employing hand-crafted item taxonomies, of which the background knowledge can be tailored for the computation of personalized recommendation. In particular, our method first formalizes a user’s long-term behavior using the item taxonomy, and then identifies the exact stage of the user. By incorporating collaborative filtering into recommendation, we can easily provide a personalized item list to the user through other similar users within the same stage. An empirical evaluation conducted on a purchasing data collection obtained from Diapers.com demonstrates the efficacy of our proposed method.     

Noisy but non-malicious user detection in social recommender systems

Social recommender systems largely rely on user-contributed data to infer users’ preference. While this feature has enabled many interesting applications in social networking services, it also introduces unreliability to recommenders as users are allowed to insert data freely. Although detecting malicious attacks from social spammers has been studied for years, little work was done for detecting Noisy but Non-Malicious Users (NNMUs), which refers to those genuine users who may provide some untruthful data due to their imperfect behaviors. Unlike colluded malicious attacks that can be detected by finding similarly-behaved user profiles, NNMUs are more difficult to identify since their profiles are neither similar nor correlated from one another. In this article, we study how to detect NNMUs in social recommender systems. Based on the assumption that the ratings provided by a same user on closely correlated items should have similar scores, we propose an effective method for NNMU detection by capturing and accumulating user’s “self-contradictions”, i.e., the cases that a user provides very different rating scores on closely correlated items. We show that self-contradiction capturing can be formulated as a constrained quadratic optimization problem w.r.t. a set of slack variables, which can be further used to quantify the underlying noise in each test user profile. We adopt three real-world data sets to empirically test the proposed method. The experimental results show that our method (i) is effective in real-world NNMU detection scenarios, (ii) can significantly outperform other noisy-user detection methods, and (iii) can improve recommendation performance for other users after removing detected NNMUs from the recommender system.     

Leveraging prior ratings for recommender systems in e-commerce

User ratings are the essence of recommender systems in e-commerce. Lack of motivation to provide ratings and eligibility to rate generally only after purchase restrain the effectiveness of such systems and contribute to the well-known data sparsity and cold start problems. This article proposes a new information source for recommender systems, called prior ratings. Prior ratings are based on users’ experiences of virtual products in a mediated environment, and they can be submitted prior to purchase. A conceptual model of prior ratings is proposed, integrating the environmental factor presence whose effects on product evaluation have not been studied previously. A user study conducted in website and virtual store modalities demonstrates the validity of the conceptual model, in that users are more willing and confident to provide prior ratings in virtual environments. A method is proposed to show how to leverage prior ratings in collaborative filtering. Experimental results indicate the effectiveness of prior ratings in improving predictive performance.     

“Turn that frown upside-down”: A contextual account of emoticon usage on different virtual platforms

Evidence highlights the prevalent usage of emoticons within digital forms of textual communication and the impact on the recipient. However, little evidence demonstrates the interpersonal functions for the user and whether this varies as a product of virtual platform. This formed the basis for the current study in which participants (N = 92) provided open-ended accounts of their reasons for using emoticons across three virtual platforms (email, text message, and social networking site), and their general emoticon usage across these. Responses revealed a number of themes on reasons for emoticon usage. The first was; “aiding personal expression”, with sub-themes of; “establishing emotional tone”; and “to lighten the mood”. Other themes were “reducing ambiguity of discourse” and “appropriateness of context”. Overall, there was consistency across platforms, on both the personal and interpersonal functions which emoticons served. However, some disparity was identified as email platforms were deemed inappropriate for emoticon use, regardless of the fact that emoticons were recognised as important emotional aids for communication. Taken together these findings highlight the importance of emoticon usage for the user, through a contextual lens to recognise the influential factors upon these behaviours and the implications this has for digital text-based communication. In this regard, this contributes further conceptualisation of one aspect of hyperpersonal communication within virtual interactions, and how different platforms may permit these self-presentational efforts to a greater or lesser extent.