Human emotion recognition and analysis in response to audio music using brain signals

Human emotion recognition using brain signals is an active research topic in the field of affective computing. Music is considered as a powerful tool for arousing emotions in human beings. This study recognized happy, sad, love and anger emotions in response to audio music tracks from electronic, rap, metal, rock and hiphop genres. Participants were asked to listen to audio music tracks of 1 min for each genre in a noise free environment. The main objectives of this study were to determine the effect of different genres of music on human emotions and indicating age group that is more responsive to music. Thirty men and women of three different age groups (15–25 years, 26–35 years and 36–50 years) underwent through the experiment that also included self reported emotional state after listening to each type of music. Features from three different domains i.e., time, frequency and wavelet were extracted from recorded EEG signals, which were further used by the classifier to recognize human emotions. It has been evident from results that MLP gives best accuracy to recognize human emotion in response to audio music tracks using hybrid features of brain signals. It is also observed that rock and rap genres generated happy and sad emotions respectively in subjects under study. The brain signals of age group (26–35 years) gave best emotion recognition accuracy in accordance to the self reported emotions.     

Dissecting emotion and user influence in social media communities: An interaction modeling approach

Human emotion expressed in social media plays an increasingly important role in shaping policies and decisions. However, the process by which emotion produces influence in online social media networks is relatively unknown. Previous works focus largely on sentiment classification and polarity identification but do not adequately consider the way emotion affects user influence. This research developed a novel framework, a theory-based model, and a proof-of-concept system for dissecting emotion and user influence in social media networks. The system models emotion-triggered influence and facilitates analysis of emotion-influence causality in the context of U.S. border security (using 5,327,813 tweets posted by 1,303,477 users). Motivated by a theory of emotion spread, the model was integrated in an influence-computation method, called the interaction modeling (IM) approach, which was compared with a benchmark using a user centrality (UC) approach based on social positions. IM was found to have identified influential users who are more broadly related to U.S. cultural issues. Influential users tended to express intense emotions of fear, anger, disgust, and sadness. The emotion trust distinguishes influential users from others, whereas anger and fear contributed significantly to causing user influence. The research contributes to incorporating human emotion into the data-information-knowledge-wisdom model of knowledge management and to providing new information systems artifacts and new causality findings for emotion-influence analysis.     

Classification of physiological signals for wheel loader operators using Multi-scale Entropy analysis and case-based reasoning

Sensor signal fusion is becoming increasingly important in many areas including medical diagnosis and classification. Today, clinicians/experts often do the diagnosis of stress, sleepiness and tiredness on the basis of information collected from several physiological sensor signals. Since there are large individual variations when analyzing the sensor measurements and systems with single sensor, they could easily be vulnerable to uncertain noises/interferences in such domain; multiple sensors could provide more robust and reliable decision. Therefore, this paper presents a classification approach i.e. Multivariate Multiscale Entropy Analysis–Case-Based Reasoning (MMSE–CBR) that classifies physiological parameters of wheel loader operators by combining CBR approach with a data level fusion method named Multivariate Multiscale Entropy (MMSE). The MMSE algorithm supports complexity analysis of multivariate biological recordings by aggregating several sensor measurements e.g., Inter-beat-Interval (IBI) and Heart Rate (HR) from Electrocardiogram (ECG), Finger Temperature (FT), Skin Conductance (SC) and Respiration Rate (RR). Here, MMSE has been applied to extract features to formulate a case by fusing a number of physiological signals and the CBR approach is applied to classify the cases by retrieving most similar cases from the case library. Finally, the proposed approach i.e. MMSE–CBR has been evaluated with the data from professional drivers at Volvo Construction Equipment, Sweden. The results demonstrate that the proposed system that fuses information at data level could classify ‘stressed’ and ‘healthy’ subjects 83.33% correctly compare to an expert’s classification. Furthermore, with another data set the achieved accuracy (83.3%) indicates that it could also classify two different conditions ‘adapt’ (training) and ‘sharp’ (real-life driving) for the wheel loader operators. Thus, the new approach of MMSE–CBR could support in classification of operators and may be of interest to researchers developing systems based on information collected from different sensor sources.     

A generalized adaptive ensemble generation and aggregation approach for multiple classifier systems

In this paper, a generalized adaptive ensemble generation and aggregation (GAEGA) method for the design of multiple classifier systems (MCSs) is proposed. GAEGA adopts an “over-generation and selection” strategy to achieve a good bias-variance tradeoff. In the training phase, different ensembles of classifiers are adaptively generated by fitting the validation data globally with different degrees. The test data are then classified by each of the generated ensembles. The final decision is made by taking into consideration both the ability of each ensemble to fit the validation data locally and reducing the risk of overfitting. In this paper, the performance of GAEGA is assessed experimentally in comparison with other multiple classifier aggregation methods on 16 data sets. The experimental results demonstrate that GAEGA significantly outperforms the other methods in terms of average accuracy, ranging from 2.6% to 17.6%.     

Facial expression and body gesture emotion recognition: A systematic review on the use of visual data in affective computing

Emotion is an important driver of human decision-making and communication. With the recent rise of human–computer interaction, affective computing has become a trending research topic, aiming to develop computational systems that can understand human emotions and respond to them. A systematic review has been conducted to fill these gaps since previous reviews regarding machine-enabled automated visual emotion recognition neglect important methodological aspects, including emotion models and hardware usage. 467 relevant papers were initially found and examined. After the screening process with specific inclusion and exclusion criteria, 30 papers were selected. Methodological aspects including emotion models, devices, architectures, and classification techniques employed by the selected studies were analyzed, and the most popular techniques and current trends in visual emotion recognition were identified. This review not only offers a comprehensive and up-to-date overview of the topic but also provides researchers with insights regarding methodological aspects like emotion models employed, devices used, and classification techniques for automated visual emotion recognition. By identifying current trends, like the increased use of deep learning algorithms and the need for further study on body gestures, this review advocates the advantages of implementing emotion recognition with the use of visual data and builds a solid foundation for applying relevant techniques in different fields.     

基于核的多元区别分析算法的特征抽取及其在人脸识别中的应用

提出了一种基于核技术的求多元区别分析最佳解的K1PMDA算法，并把这一算法应用于人脸识别中。对线性人脸识别中存在两个突出问题：1、在光照、表情、姿态变化较大时，人脸图像分类是复杂的、非线性的；2、小样本问题，即当训练样本数量小于样本特征空间维数时，导致类内散布矩阵奇异。对于前一个问题，可以采用核技术提取人脸图像样本的非线性特征，对于后一个问题，采用加入一个扰动参数的扰动算法。通过对ORL，Yale Group B以及UMIST三个人脸库的实验表明，该算法是可行的、高效的。     

Smoothed additive estimators for non-error rates in multiple discriminant analysis

A generalization of smoothed additive estimators for non-error rates to the case of more than two groups is discussed. Several properties the smoothing should have are shown to be satisfied. The problem of choosing a smoothing parameter is considered and a parameter choice depending on the sample is proposed. In simulation experiments with normal, uniform and discrete distributions the smoothed additive estimators with fixed and variable smoothing parameter are compared to the leaving-one out method and the resubstitution method with respect to bias and variance.     

Discriminability and reliability indexes: Two new measures to enhance multi-image face recognition

In order to handle complex face image variations in face recognition, multi-image face recognition has been proposed, instead of using a single still-image-based approach. In many practical scenarios, multiple images can be easily obtained in enrollment and query stages, for example, using video. By assessing these images, a good “quality” image(s) will be selected for recognition using conventional still-image-based recognition algorithms so that the recognition performance can be improved. However, existing methods do not fully utilize all images information. In this paper, two new measurements, namely discriminability index (DI) and reliability index (RI), are proposed to evaluate the enrolled and query images, respectively. By considering the distribution of enrolled images from individuals, the discriminability index of each image is calculated and a weight is assigned. For testing images, a reliability index is calculated based on matching quality between the testing images and enrolled images. If the reliability index of a testing image is small, the testing image will be discarded as it may degrade the recognition performance. To evaluate and demonstrate the use of DI and RI, we adopt the combining classifier method with eigenface representations in input and kernel feature spaces. CMU-PIE, YaleB and FRGC V2.0 databases are used for experiments. Experimental results show that the recognition performance, with three popular combination rules, can be increased by more than 10% on average using DI and RI.     

Discrete contours in multiple views: approximation and recognition

Recognition of discrete planar contours under similarity transformations has received a lot of attention but little work has been reported on recognizing them under more general transformations. Planar object boundaries undergo projective or affine transformations across multiple views. We present two methods to recognize discrete curves in this paper. The first method computes a piecewise parametric approximation of the discrete curve that is projectively invariant. A polygon approximation scheme and a piecewise conic approximation scheme are presented here. The second method computes an invariant sequence directly from the sequence of discrete points on the curve in a Fourier transform space. The sequence is shown to be identical up to a scale factor in all affine related views of the curve. We present the theory and demonstrate its applications to several problems including numeral recognition, aircraft recognition, and homography computation.     

Social networking and digital gaming media convergence: Classification and its consequences for appropriation

Within the field of Information Systems, a good proportion of research is concerned with the work organisation and this has, to some extent, restricted the kind of application areas given consideration. Yet, it is clear that information and communication technology deployments beyond the work organisation are acquiring increased importance in our lives. With this in mind, we offer a field study of the appropriation of an online play space known as Habbo Hotel. Habbo Hotel, as a site of media convergence, incorporates social networking and digital gaming functionality. Our research highlights the ethical problems such a dual classification of technology may bring. We focus upon a particular set of activities undertaken within and facilitated by the space—scamming. Scammers dupe members with respect to their ‘Furni’, virtual objects that have online and offline economic value. Through our analysis we show that sometimes, online activities are bracketed off from those defined as offline and that this can be related to how the technology is classified by members—as a social networking site and/or a digital game. In turn, this may affect members’ beliefs about rights and wrongs. We conclude that given increasing media convergence, the way forward is to continue the project of educating people regarding the difficulties of determining rights and wrongs, and how rights and wrongs may be acted out with respect to new technologies of play online and offline.

Introducing the Wunderkammer as a tool for emotion research: Unconstrained gaze and movement patterns in three emotionally evocative virtual worlds

Here we introduce the “Wunderkammer”, a suite of immersive virtual worlds with different types of emotionally-charged content. We use these worlds to examine the effects of affective context on unconstrained gaze and movement. In the Affect Gallery, participants freely explored a virtual art museum filled with objects that varied in valence and arousal. Participants approached and gazed more at positively valenced objects. This preference was amplified by more arousing objects and was strongest among individuals with resilient emotion regulation tendencies. This bias of avoiding negative valence did not emerge in The Crowded Room, an environment in which participants encountered virtual humans expressing different emotions. Instead, participants gazed more at negative than neutral emotional displays although they physically avoided angry (but not sad or neutral) agents. When placed inside Room 101, an unpredictable environment filled with a series of disturbing events, frightened participants became relatively immobile in terms of both gaze and movement. This freezing-type response was particularly strong among dispositionally resilient individuals. Together these results demonstrate that distinct affective contexts elicit unique patterns in unconstrained gaze and movement. They further illustrate the benefits of using virtual reality to study affect as it naturally emerges.     

SAMAR: Subjectivity and sentiment analysis for Arabic social media

SAMAR is a system for subjectivity and sentiment analysis (SSA) for Arabic social media genres. Arabic is a morphologically rich language, which presents significant complexities for standard approaches to building SSA systems designed for the English language. Apart from the difficulties presented by the social media genres processing, the Arabic language inherently has a high number of variable word forms leading to data sparsity. In this context, we address the following 4 pertinent issues: how to best represent lexical information; whether standard features used for English are useful for Arabic; how to handle Arabic dialects; and, whether genre specific features have a measurable impact on performance. Our results show that using either lemma or lexeme information is helpful, as well as using the two part of speech tagsets (RTS and ERTS). However, the results show that we need individualized solutions for each genre and task, but that lemmatization and the ERTS POS tagset are present in a majority of the settings.     

Distance and similarity measures between intuitionistic fuzzy sets: A comparative analysis from a pattern recognition point of view

A detailed analysis of the distance and similarity measures for intuitionistic fuzzy sets proposed in the past is presented in this paper. This study aims to highlight the main theoretical and computational properties of the measures under study, while the relationships between them are also investigated. Along with the literature review, a comparison of the analyzed distance and similarity measures from a pattern recognition point of view in three different classification cases is also presented. Initially, some artificial counter-intuitive recognition cases are considered, while in a second phase real data from medical and well known pattern recognition benchmark problems are used to examine the discrimination abilities of the studied measures. Moreover, all the measures are applied in a face recognition problem for the first time and useful conclusions are drawn regarding the accuracy and confidence of the recognition results. Finally, the measures’ suitability and their drawbacks that make the development of more robust and efficient measures’ a still open issue are discussed.     

Improved discriminate analysis for high-dimensional data and its application to face recognition

Many pattern recognition applications involve the treatment of high-dimensional data and the small sample size problem. Principal component analysis (PCA) is a common used dimension reduction technique. Linear discriminate analysis (LDA) is often employed for classification. PCA plus LDA is a famous framework for discriminant analysis in high-dimensional space and singular cases. In this paper, we examine the theory of this framework and find out that even if there is no small sample size problem the PCA dimension reduction cannot guarantee the subsequent successful application of LDA. We thus develop an improved discriminate analysis method by introducing an inverse Fisher criterion and adding a constrain in PCA procedure so that the singularity phenomenon will not occur. Experiment results on face recognition suggest that this new approach works well and can be applied even when the number of training samples is one per class.     

字符多特征提取方法及其在车牌识别中的应用

针对车牌字符识别中大部分单一特征提取方法在字符识别上的局限性,提出了一种车牌字符多特征提取方法。在经过预处理后的车牌细化字符基础上提取字符4个侧面的笔画特征、拐点特征、轮廓累积特征及字符内部像素特征,构建出一个维度较低的特征向量集,然后分别采用支持向量机、K近邻算法、BP神经网络、径向基神经网络对陆丰高速公路实地拍摄的车牌图片进行测试并分别与模板匹配方法、网格法、基于小波矩方法比较,实验结果表明提出的车牌字符多特征提取方法识别率高,鲁棒性好。     

Access by content to handwritten archive documents: generic document recognition method and platform for annotations

This paper presents annotations needed for handwritten archive document retrieval by content. We propose two complementary ways of producing these annotations: automatically by using document image analysis and collectively by using the Internet and manual input by users. A platform for managing these annotations is presented as well as examples of automatic annotations on civil status registers, military forms (tested on 165,000 pages) and naturalization decrees, using a generic method for structured document recognition and handwriting recognition on names. Examples of collective annotations built on automatic annotations are also given. This platform is already open to the public in the reading room of the new building of the Archives départementales des Yvelines and on the Internet. About 1,450,000 images of civil status registers are available for collective annotation as well as 105,000 pages of military forms with automatic annotation of handwritten names.     

Deep sentiment analysis: Mining the causality between personality-value-attitude for analyzing business ads in social media

IT vendors routinely use social media such as YouTube not only to disseminate their IT product information, but also to acquire customer input efficiently as part of their market research strategies. Customer responses that appear in social media, however, are typically unstructured; thus, a fairly large data set is needed for meaningful analysis. Although identifying customers’ value structures and attitudes may be useful for developing targeted or niche markets, the unstructured and volume-heavy nature of customer data prohibits efficient and economical extraction of such information. Automatic extraction of customer information would be valuable in determining value structure and strength. This paper proposes an intelligent method of estimating causality between user profiles, value structures, and attitudes based on the replies and published content managed by open social network systems such as YouTube. To show the feasibility of the idea proposed in this paper, information richness and agility are used as underlying concepts to create performance measures based on media/information richness theory. The resulting deep sentiment analysis proves to be superior to legacy sentiment analysis tools for estimation of causality among the focal parameters.