Recommender Systems: An Introduction,by Dietmar Jannach,Markus Zanker,Alexander Felfernig,and Gerhard Friedrich

In addressing user experience issues, users’ perceptions and emotions need to be considered important. This study examines the relationships between perceived usability/aesthetics and emotional valence/arousal/engagement through an experiment using 15 existing websites from various domains and questionnaire items developed to measure users’ responses. According to the experimental results, both perceived usability and perceived aesthetics were positively correlated with emotional valence and negatively correlated with emotional engagement. No specific relationship was found between perceived usability/aesthetics and emotional arousal. Perceived aesthetics potentially had a greater impact on valence than perceived usability. Unlike valence, engagement could be more influenced by perceived usability than by perceived aesthetics. These findings can be utilized as bases for applying users’ emotional responses in each dimension to the product-use situations in the chain of perceptions, emotions, and behaviors.     

Effects of Perceived Usability and Aesthetics on Emotional Responses in Different Contexts of Use

In understanding user experience issues, it is necessary to consider contextual features in user interaction with an application. The present study aims to examine the differences among applications with difference contexts of use (perceived type: interaction styles based on use purposes) in terms of the relationships between user perception (perceived usability and perceived aesthetics) and emotion (valence and arousal). A study was performed using 15 existing websites from various domains. From the responses of participants, the relationships between perceived usability and emotional valence were found to differ according to perceived type, but those between perceived aesthetics and emotional valence were not. On the other hand, sufficient reliability for the arousal items was not secured. Through the meaningful categorization of arousal into two factors (negative and positive), significant differences in the relationships between perceived usability/aesthetics and emotional arousal could be found. Implications of the findings are discussed in the connection of contextual/design features, perceptual characteristics, and emotional responses in the development of interactive applications.     

The influence of web aesthetics on customers’ PAD

One key focus of an online retail website is to enhance the consumers’ online shopping behavior. Based on the Stimulus-Organism-Response (S-O-R) framework and pleasure, arousal, and dominance (PAD) emotional model, we investigated the relationship between consumers’ emotional model and purchase behavior from the perspective of web aesthetics, and how web aesthetics affect their purchase behaviors through the emotional model. Using 441 questionnaire responses and structural equation modeling, we verified that both aesthetic formality and aesthetic appeal influence purchase behaviors through the emotional model. In the emotional model, web aesthetics have significant and positive influences on control, which is composed of behavior control, cognitive control, and decisional control. Control indirectly influences pleasure through the mediations of energetic arousal and tense arousal. Additionally, pleasure and searching on other websites positively influences purchasing behavior. The aim of this study was to provide practical recommendations in the establishment of a pattern of web aesthetics that influence consumers’ emotions.     

Affective video content analysis based on multimodal data fusion in heterogeneous networks

In heterogeneous networks, different modalities are coexisting. For example, video sources with certain lengths usually have abundant time-varying audiovisual data. From the users’ perspective, different video segments will trigger different kinds of emotions. In order to better interact with users in heterogeneous networks and improve their user experiences, affective video content analysis to predict users’ emotions is essential. Academically, users’ emotions can be evaluated by arousal and valence values, and fear degree, which provides an approach to quantize the prediction accuracy of the reaction of the audience and users towards videos. In this paper, we propose the multimodal data fusion method for integrating the visual and audio data in order to perform the affective video content analysis. Specifically, to align the visual and audio data, the temporal attention filters are proposed to obtain the time-span features of the entire video segments. Then, by using the two-branch network structure, matched visual and audio features are integrated in the common space. At last, the fused audiovisual feature is employed for the regression and classification subtasks in order to measure the emotional responses of users. Simulation results show that the proposed method can accurately predict the subjective feelings of users towards the video contents, which provides a way to predict users’ preferences and recommend videos according to their own demand.     

Touch gestures in communicating emotional intention via vibrotactile stimulation

Remote communication between people typically relies on audio and vision although current mobile devices are increasingly based on detecting different touch gestures such as swiping. These gestures could be adapted to interpersonal communication by using tactile technology capable of producing touch stimulation to a user's hand. It has been suggested that such mediated social touch would allow for new forms of emotional communication. The aim was to study whether vibrotactile stimulation that imitates human touch can convey intended emotions from one person to another. For this purpose, devices were used that converted touch gestures of squeeze and finger touch to vibrotactile stimulation. When one user squeezed his device or touched it with finger(s), another user felt corresponding vibrotactile stimulation on her device via four vibrating actuators. In an experiment, participant dyads comprising a sender and receiver were to communicate variations in the affective dimensions of valence and arousal using the devices. The sender's task was to create stimulation that would convey unpleasant, pleasant, relaxed, or aroused emotional intention to the receiver. Both the sender and receiver rated the stimulation using scales for valence and arousal so that the match between sender's intended emotions and receiver's interpretations could be measured. The results showed that squeeze was better at communicating unpleasant and aroused emotional intention, while finger touch was better at communicating pleasant and relaxed emotional intention. The results can be used in developing technology that enables people to communicate via touch by choosing touch gesture that matches the desired emotion.     

A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies

The popularity of computer games has exploded in recent years, yet methods of evaluating user emotional state during play experiences lag far behind. There are few methods of assessing emotional state, and even fewer methods of quantifying emotion during play. This paper presents a novel method for continuously modeling emotion using physiological data. A fuzzy logic model transformed four physiological signals into arousal and valence. A second fuzzy logic model transformed arousal and valence into five emotional states relevant to computer game play: boredom, challenge, excitement, frustration, and fun. Modeled emotions compared favorably with a manual approach, and the means were also evaluated with subjective self-reports, exhibiting the same trends as reported emotions for fun, boredom, and excitement. This approach provides a method for quantifying emotional states continuously during a play experience.     

A quantitative aesthetic measurement method for product appearance design

Product appearance is one of the crucial factors that influence consumers’ purchase decisions. The attractiveness of product appearance is mainly determined by the inherent aesthetics of the design composition related to the arrangement of visual design elements. Hence, it is critical to study and improve the arrangement of visual design elements for product appearance design. Strategies that apply aesthetic design principles to assist designers in effectively arranging visual design elements are widely acknowledged in both academia and industry. However, applying aesthetic design principles relies heavily on the designer’s perception and experience, while it is rather challenging for novice designers. Meanwhile, it is hard to measure and quantify design aesthetics in designing artefacts when designers refer to existing successful designs. In this regard, this study aims to introduce a method that assists designers in applying aesthetic design principles to improve the attractiveness of product appearance. Furthermore, formulas for aesthetic measurement based on aesthetic design principles are also developed, and it makes an early attempt to provide quantified aesthetic measurements of design artefacts. A case study on camera design was conducted to demonstrate the merits of the proposed method where the improved strategies for the camera appearance design offer insights for concept generation in product appearance design based on aesthetic design principles.     

Aesthetic images modulate emotional responses to reading news messages on a small screen: A psychophysiological investigation

Abstract pictures, such as artistic drawings, may evoke subtle emotions in their observers via aesthetic experiences. We examined among 44 participants the emotional responses as measured by facial electromyography (EMG) to aesthetic background images that varied on the emotional valence (unpleasant to pleasant) and arousal (calming to exciting) dimensions and were presented both separately and as background images for news messages. Effects of image arousal on free recall of news messages were also examined. A priori pleasant compared to unpleasant images were associated with higher orbicularis oculi EMG responses, both when presented alone and when presented as news message backgrounds. Analyses based on the participants' subjective pleasantness ratings also showed greater corrugator supercilii EMG activity for unpleasant compared to pleasant images. High-arousal as compared to low-arousal images improved recall for the superimposed news messages. In contrast, recall was not affected by a priori image valence or subjective pleasantness ratings. The results demonstrate that abstract images can be used to evoke emotional responses in the viewers that persist even when unrelated messages are superimposed on them. Similarly, high-arousal images can be used to enhance memory for superimposed textual messages.     

Photo aesthetic quality estimation using visual complexity features

The amount of visual data available on the Web is growing explosively and it is becoming increasingly important to explore methods for automatically estimating the quality of this content in a manner that is consistent with the aesthetic perceptions of humans. The key to this challenging problem is to design an appropriate set of features to extract the aesthetic properties from content. Most previous studies designed a set of aesthetic features based on photographic criteria, which were unavoidably limited to specific examples and they lacked an interpretation based on the mechanism of human aesthetic perception. According to psychological theory, visual complexity is an important property of the stimuli, because it directly influences the viewer’s arousal level, which is believed to be closely related to aesthetic perception. In this study, we propose an alternative set of features for aesthetic estimation based on a visual complexity principle. We extracted the visual complexity properties from an input image in terms of their composition, shape, and distribution. In addition, we demonstrated that the proposed features are consistent with human perception on the complexity in our visual complexity dataset. Next, we employed these features for photo-aesthetic quality estimation using a large-scale dataset. Various experiments were conducted under different conditions and comparisons with state-of-the-art methods shows that the proposed visual complexity feature outperforms photography rule-based features and even better than deep features.     

Preparing computers for affective communication: a psychophysiological concept and preliminary results

Despite rapid advances in technology, computers remain incapable of responding to human emotions. An exploratory study was conducted to find out what physiological parameters might be useful to differentiate among 4 emotional states, based on 2 dimensions: pleasantness versus unpleasantness and arousal versus relaxation. The 4 emotions were induced by exposing 26 undergraduate students to different combinations of olfactory and auditory stimuli, selected in a pretest from 12 stimuli by subjective ratings of arousal and valence. Changes in electroencephalographic (EEG), heart rate variability, and electrodermal measures were used to differentiate the 4 emotions. EEG activity separates pleasantness from unpleasantness only in the aroused but not in the relaxed domain, where electrodermal parameters are the differentiating ones. All three classes of parameters contribute to a separation between arousal and relaxation in the positive valence domain, whereas the latency of the electrodermal response is the only differentiating parameter in the negative domain. We discuss how such a psychophysiological approach may be incorporated into a systemic model of a computer responsive to affective communication from the user.     

An investigation into the effects of product design on incremental and radical innovations from the perspective of consumer perceptions: Evidence from China

By integrating research on product design with ideas from marketing and consumer perceptions, this paper formulates a set of coherent hypotheses about the effects of tripartite design dimensions on consumer perceptions and responses. We conduct two surveys on incremental and radical technology innovations, respectively, and accordingly get two different samples to test the hypotheses. Empirical results reveal that the effects of aesthetic and functional dimensions on consumer purchase intention and word of mouth vary across two types of innovations. Specifically, for incrementally innovative products, aesthetic and functional dimensions influence consumer purchase intention and word of mouth, both directly and indirectly, through the mediation of consumer perceived quality. By contrast, for radically innovative products, the effects of aesthetic and functional dimensions on purchase intention and word of mouth are totally mediated by perceived quality. Additionally, the effects of symbolic dimension on consumer perceptions and responses achieve consistency across radical and incremental technology innovations. That is, the higher the level of symbolic dimension, the stronger purchase intention and the better word of mouth there will be, whereas the direct influence of symbolic dimension on perceived quality is not significant in either innovation category.     

Modeling of operators' emotion and task performance in a virtual driving environment

Emotional human–computer interactions are attracting increasing interest with the improvement in the available technology. Through presenting affective stimuli and empathic communication, computer agents are able to adjust to users' emotional states. As a result, users may produce better task performance. Existing studies have mainly focused on the effect of only a few basic emotions, such as happiness and frustration, on human performance. Furthermore, most research explored this issue from the psychological perspective. This paper presents an emotion and performance relation model in the context of vehicle driving. This general emotion–performance model is constructed on an arousal–valence plane and is not limited to basic emotions. Fifteen paid participants took part in two driving simulation experiments that induced 115 pairs of emotion–performance sample. These samples revealed the following: (1) driving performance has a downward U-shaped relationship with both intensities of arousal and valence. It deteriorates at extreme arousal and valence. (2) Optimal driving performance, corresponding to the appropriate emotional state, matches the “sweet spot” phenomenon of the engagement psychology. (3) Arousal and valence are not perfectly independent across the entire 2-D emotion plane. Extreme valence is likely to stimulate a high level of arousal, which, in turn, deteriorates task performance. The emotion–performance relation model proposed in the paper is useful in designing emotion-aware intelligent systems to predict and prevent task performance degradation at an early stage and throughout the human–computer interactions.     

Service robot feature design effects on user perceptions and emotional responses

Service robots have been developed to assist nurses in routine patient services. Prior research has recognized that patient emotional experiences with robots may be as important as robot task performance in terms of user acceptance and assessments of effectiveness. The objective of this study was to understand the effect of different service robot interface features on elderly perceptions and emotional responses in a simulated medicine delivery task. Twenty-four participants sat in a simulated patient room and a service robot delivered a bag of “medicine” to them. Repeated trials were used to present variations on three robot features, including facial configuration, voice messaging and interactivity. Participant heart rate (HR) and galvanic skin response (GSR) were collected. Participant ratings of robot humanness [perceived anthropomorphism (PA)] were collected post-trial along with subjective ratings of arousal (bored–excited) and valence (unhappy–happy) using the self-assessment manikin (SAM) questionnaire. Results indicated the presence of all three types of robot features promoted higher PA, arousal and valence, compared to a control condition (a robot without any of the features). Participant physiological responses varied with events in their interaction with the robot. The three types of features also had different utility for stimulating participant arousal and valence, as well as physiological responses. In general, results indicated that adding anthropomorphic and interactive features to service robots promoted positive emotional responses [increased excitement (GSR) and happiness (HR)] in elderly users. It is expected that results from this study could be used as a basis for developing affective robot interface design guidelines to promote user emotional experiences.     

Emotional clouds: Showing arousal and valence through the movement and darkness of digital cartoonish clouds

While nowadays the most usual way to show emotions in digital contexts is via virtual characters, its use may raise false expectations (the user attributes human abilities to the virtual character). This paper proposes and explores an approach to express emotions which intends to minimize the user's expectations by using a non-anthropomorphic model. Emotions are represented in terms of arousal and valence dimensions. They are visualized in a simple way through the behaviour and appearance of a series of cartoonish clouds. In particular, the arousal value is expressed through the movement of these clouds (controlled by a flocking algorithm), while the valence value is expressed through their degree of darkness. Furthermore, the paper describes a user experiment which investigated whether the arousal and valence expressed by our model are appropriately interpreted by the users or not. The results suggest that movement and darkness are interpreted as arousal and valence respectively and that they are independent of each other.     

中国国画情感-美感数据库

目的 有关艺术作品审美感受的量化或计算,在心理学上已有许多研究。近年来,人工智能的研究成为热点,而对人类感知的定量分析也随之受到极大关注,例如基于图片或者音乐内容的情感计算等。美感作为一种艺术化的审美情感,与之相关的定量研究有较大潜力。为便于进行中国文化背景下的审美研究,同时为丰富图像情感与审美计算相关研究的数据基础,需建立一个国画美感和情感分析所用的图像数据库。方法 从多种渠道收集筛选511幅国画素材及350个国画美感形容词,通过词汇筛选和因子分析获得国画美感主要因子;采用离散词汇和PAD（pleasure-arousal-dominance）情感连续维度空间这两种描述方式对国画的审美感受进行标注;对数据库进行情感和美感的模式分类,从而验证其实用性。结果 获得5个国画美感主要语义标签：气势、清幽、生机、雅致和萧瑟;标注数据结果满足有效性验证;不同美感的PAD情感分布呈现一定极化;经测试,情感分类精度平均可达0.68,美感分类精度最高可达0.77。结论 本文得到的5个国画美感评价范畴,可基本概括国画的审美感受;所建立的数据库,能为视觉美感及情感的定量研究或者计算机视觉、实验美学等领域的研究提供有效数据基础;PAD分布对美感有较好区分性。下一步将进一步扩充数据库,以解决数据分布不均问题,同时进一步挖掘PAD分布与美感分布之间的关联。     

Scoliosis brace design: influence of visual aesthetics on user acceptance and compliance

Abstract

Adolescent idiopathic scoliosis is a common condition found in adolescents. A rigid brace is often prescribed as the treatment for this spinal deformity, which negatively affects user compliance due to the discomfort caused by the brace, and the psychological distress resulting from its appearance. However, the latter, which is the impact of visual aesthetics, has not been thoroughly studied for scoliosis braces. Therefore, a qualitative study with in-depth interviews has been carried out with 10 participants who have a Cobb angle of 20°–30° to determine the impact of visual aesthetics on user acceptance and compliance towards the brace. It is found that co-designing with patients on the aesthetic aspects of the surface design of the brace increases the level of user compliance and induces positive user perception. Therefore, aesthetic preferences need to be taken into consideration in the design process of braces.

Practitioner Summary: The impact of visual aesthetics on user acceptance and compliance towards a rigid brace for scoliosis is investigated. The findings indicate that an aesthetically pleasing brace and the involvement of patients in the design process of the brace are important for increasing user compliance and addressing psychological issues during treatment.     

Showing emotions through movement and symmetry

This paper proposes and explores a minimalist abstract approach to express emotions through movement and symmetry, which intends to minimize the user’s expectations. Emotions are represented in terms of arousal and valence dimensions and they are visually expressed in a simple way through the behaviour and appearance of a series of dynamic horizontal and vertical lines. In particular, the arousal value is expressed through the movement of these lines, while the valence value is expressed through their symmetry. The line movement is controlled by a flocking algorithm which is influenced by the arousal value, while the line symmetry/asymmetry is expressed through their rotation and length. Furthermore, the paper describes a user experiment which investigated whether the arousal and valence expressed by our model are appropriately perceived by the users or not. The results suggest that combinations of movement and symmetry are perceived correctly as particular emotions, that movement and symmetry are perceived as arousal and valence respectively and that they are independent of each other.     

人机交互中的智能体情感计算研究

机器的情感是通过融入具有情感能力的智能体实现的,虽然目前在人机交互领域已经有大量研究成果,但有关智能体情感计算方面的研究尚处起步阶段,深入开展这项研究对推动人机交互领域的发展具有重要的科学和应用价值。本文通过检索Scopus数据库选择有代表性的文献,重点关注情感在智能体和用户之间的双向流动,分别从智能体对用户的情绪感知和对用户情绪调节的角度开展分析总结。首先梳理了用户情绪的识别方法,即通过用户的表情、语音、姿态、生理信号和文本信息等多通道信息分析用户的情绪状态,归纳了情绪识别中的一些机器学习方法。其次从用户体验角度分析具有情绪表现力的智能体对用户的影响,总结了智能体的情绪生成和表现技术,指出智能体除了通过表情之外,还可以通过注视、姿态、头部运动和手势等非言语动作来表现情绪。并且梳理了典型的智能体情绪架构,举例说明了强化学习在智能体情绪设计中的作用。同时为了验证模型的准确性,比较了已有的情感评估手段和评价指标。最后指出智能体情感计算急需解决的问题。通过对现有研究的总结,智能体情感计算研究是一个很有前景的研究方向,希望本文能够为深入开展相关研究提供借鉴。     

Visual complexity of websites: Effects on users’ experience, physiology, performance, and memory

Visual complexity is an apparent feature in website design yet its effects on cognitive and emotional processing are not well understood. The current study examined website complexity within the framework of aesthetic theory and psychophysiological research on cognition and emotion. We hypothesized that increasing the complexity of websites would have a detrimental cognitive and emotional impact on users. In a passive viewing task (PVT) 36 website screenshots differing in their degree of complexity (operationalized by JPEG file size; correlation with complexity ratings in a preliminary study r=.80) were presented to 48 participants in randomized order. Additionally, a standardized visual search task (VST) assessing reaction times, and a one-week-delayed recognition task on these websites were conducted and participants rated all websites for arousal and valence. Psychophysiological responses were assessed during the PVT and VST. Visual complexity was related to increased experienced arousal, more negative valence appraisal, decreased heart rate, and increased facial muscle tension (musculus corrugator). Visual complexity resulted in increased reaction times in the VST and decreased recognition rates. Reaction times in the VST were related to increases in heart rate and electrodermal activity. These findings demonstrate that visual complexity of websites has multiple effects on human cognition and emotion, including experienced pleasure and arousal, facial expression, autonomic nervous system activation, task performance, and memory. It should thus be considered an important factor in website design.     

Emotional design of multimedia learning using background images with motivational cues

Emotional design for multimedia learning can be implemented primarily using aesthetic design elements (e.g., color, shape and layout); however, we still have limited understanding on emotional design using content features visually depicting motivational cues. This study examines how valence and arousal of emotional learning content, generated by background images, in multimedia learning influenced learners' retention of factual knowledge and cognitive load, using a 2 (valence: positive, negative) × 2 (arousal: moderately low, moderately high) between-subjects experiment. The results showed significant interaction effects, implying the positive effects of using moderately low arousing negative images on both recognition and cued-recall test scores. Both scores were significantly lower when moderately high arousing negative images were used; however, they did not differ with increasing arousal of positive images. The emotional content did not influence germane and extraneous loads. Our findings emphasize the need for considering optimal levels of valence and arousal in emotional design.