Modelling human perception of static facial expressions

A recent internet based survey of over 35,000 samples has shown that when different human observers are asked to assign labels to static human facial expressions, different individuals categorize differently the same image. This fact results in a lack of an unique ground-truth, an assumption held by the large majority of existing models for classification. This is especially true for highly ambiguous expressions, especially in the lack of a dynamic context. In this paper we propose to address this shortcoming by the use of discrete choice models (DCM) to describe the choice a human observer is faced to when assigning labels to static facial expressions. Different models of increasing complexity are specified to capture the causal effect between features of an image and its associated expression, using several combinations of different measurements. The sets of measurements we used are largely inspired by FACS but also introduce some new ideas, specific to a static framework. These models are calibrated using maximum likelihood techniques and they are compared with each other using a likelihood ratio test, in order to test for significance in the improvement resulting from adding supplemental features. Through a cross-validation procedure we assess the validity of our approach against overfitting and we provide a comparison with an alternative model based on Neural Networks for benchmark purposes.     

心理学视角下的自动表情识别

自动表情识别是心理学与计算机科学等深度交叉的前沿领域。情绪心理学、模式识别、情感计算等领域的研究者发展表情识别相关的理论、数据库和算法,极大地推动了自动表情识别技术的进步。文章基于心理学视角,结合我们前期开展的相关工作,首先梳理自动表情识别的心理学基础、情绪的面部表达方式、表情数据的演化、表情样本的标注等方面的理论观点与实践进展,然后分析指出自动表情识别面临的主要问题,最后基于预测加工理论的建构观点,提出注重交互过程中的表情“理解”,有望进一步提高自动表情识别的有效性,并预期这可能是自动表情识别研究的未来发展方向。     

Suboptimal facial expression primes in textual media messages: Evidence for the affective congruency effect

The effectiveness of suboptimal affective primes in real media applications has remained debatable. We investigated the effects of suboptimally (at 10-ms exposure) presented facial expression primes on emotional responses to, and cognitive evaluations of, textual (business news) messages ranging from slightly unpleasant to slightly pleasant among 33 participants. Facial expression primes were presented repeatedly on a simulated small screen of a mobile device during a news reading task. Facial electromyography (EMG) recordings were used as physiological indices of positive and negative emotions. Our results showed affective congruency effects between news messages and facial expression primes: joyful facial expressions, compared to angry primes or no priming, elicited higher positive affect evaluations and more positive facial EMG responses, as well as higher interest, when embedded in affectively more positive news. On the other hand, the mere presence of suboptimal primes was detrimental to the perceived trustworthiness of news. These results suggest that embedding suboptimal facial expression primes into textual media messages may exert an influence on affectively congruent messages; at the same time, our results highlight the potential hazards and difficulties of utilizing such primes.     

Information extraction from image sequences of real-world facial expressions

Information extraction of facial expressions deals with facial-feature detection, feature tracking, and capture of the spatiotemporal relationships among features. It is a fundamental task in facial expression analysis and will ultimately determine the performance of expression recognition. For a real-world facial expression sequence, there are three challenges: (1) detection failure of some or all facial features due to changes in illumination and rapid head movement; (2) nonrigid object tracking resulting from facial expression change; and (3) feature occlusion due to out-of-plane head rotation. In this paper, a new approach is proposed to tackle these challenges. First, we use an active infrared (IR) illumination to reliably detect pupils under variable lighting conditions and head orientations. The pupil positions are then used to guide the entire information-extraction process. The simultaneous use of a global head motion constraint and Kalman filtering can robustly track individual facial features even in condition of rapid head motion and significant expression change. To handle feature occlusion, we propose a warping-based reliability propagation method. The reliable neighbor features and the spatial semantics among these features are used to detect and infer occluded features through an interframe warping transformation. Experimental results show that accurate information extraction can be achieved for video sequences with real-world facial expressions.Received: 16 August 2003, Accepted: 20 September 2004, Published online: 20 December 2004 Correspondence to: Qiang Ji     

Invariant representation of facial expressions for blended expression recognition on unknown subjects

Facial expressions analysis plays an important part in emotion detection. However, having an automatic and non-intrusive system to detect blended facial expression is still a challenging problem, especially when the subject is unknown to the system. Here, we propose a method that adapts to the morphology of the subject and that is based on a new invariant representation of facial expressions. In our system, one expression is defined by its relative position to 8 other expressions. As the mode of representation is relative, we show that the resulting expression space is person-independent. The 8 expressions are synthesized for each unknown subject from plausible distortions. Recognition tasks are performed in this space with a basic algorithm. The experiments have been performed on 22 different blended expressions and on either known or unknown subjects. The recognition results on known subjects demonstrate that the representation is robust to the type of data (shape and/or texture information) and to the dimensionality of the expression space. The recognition results on 22 expressions of unknown subjects show that a dimensionality of the expression space of 4 is enough to outperform traditional methods based on active appearance models and accurately describe an expression.     

Automatic coding of facial expressions displayed during posed and genuine pain

We present initial results from the application of an automated facial expression recognition system to spontaneous facial expressions of pain. In this study, 26 participants were videotaped under three experimental conditions: baseline, posed pain, and real pain. The real pain condition consisted of cold pressor pain induced by submerging the arm in ice water. Our goal was to (1) assess whether the automated measurements were consistent with expression measurements obtained by human experts, and (2) develop a classifier to automatically differentiate real from faked pain in a subject-independent manner from the automated measurements. We employed a machine learning approach in a two-stage system. In the first stage, a set of 20 detectors for facial actions from the Facial Action Coding System operated on the continuous video stream. These data were then passed to a second machine learning stage, in which a classifier was trained to detect the difference between expressions of real pain and fake pain. Naïve human subjects tested on the same videos were at chance for differentiating faked from real pain, obtaining only 49% accuracy. The automated system was successfully able to differentiate faked from real pain. In an analysis of 26 subjects with faked pain before real pain, the system obtained 88% correct for subject independent discrimination of real versus fake pain on a 2-alternative forced choice. Moreover, the most discriminative facial actions in the automated system were consistent with findings using human expert FACS codes.     

Automatic questionnaire survey by using the collective message over the Internet

As to implement questionnaire survey, a researcher usually has to spend much time and efforts on questionnaire delivery, questionnaire collection and questionnaire analysis. In order to enhance efficiency and effectiveness in questionnaire survey, this research develops an automatic questionnaire survey model. In the proposed model, the questions in the questionnaires can be converted into structured components and the component characteristics can be obtained from the related online message. By using the Artificial Neural Network method, the relationship model between the online message and questionnaire questions can be derived and the derived model can be used to acquire the answers to the questions from the message highly related to the questions. Moreover, distributions of the answers can be visually depicted. By utilizing the proposed methodology, researchers can efficiently realize the results about their questionnaire derived from the online population.On the basis of the proposed model, this research develops a platform for automatic questionnaire survey and analysis. Moreover, real-world questions and message, Yahoo! Answers, were used to evaluate feasibility and performance of the proposed model and platform. According to the experimental result question answers can be effectively acquired from the message highly related to the questions. The proposed method and platform can assist people quickly and effectively accomplish questionnaire survey.     

The impact of external facial features on the construction of facial composites

Witnesses may construct a composite face of a perpetrator using a computerised interface. Police practitioners guide witnesses through this unusual process, the goal being to produce an identifiable image. However, any changes a perpetrator makes to their external facial-features may interfere with this process. In Experiment 1, participants constructed a composite using a holistic interface one day after target encoding. Target faces were unaltered, or had altered external-features: (i) changed hair, (ii) external-features removed or (iii) naturally-concealed external-features (hair, ears, face-shape occluded by a hooded top). These manipulations produced composites with more error-prone internal-features: participants’ familiar with a target’s unaltered appearance less often provided a correct name. Experiment 2 applied external-feature alterations to composites of unaltered targets; although whole-face composites contained less error-prone internal-features, identification was impaired. Experiment 3 replicated negative effects of changing target hair on construction and tested a practical solution: selectively concealing hair and eyes improved identification.

Practitioner Summary: The research indicates that when a target identity disguises or changes hair, this can lead to a witness (or victim) constructing a composite that is less readily identified. We assess a practical method to overcome this forensic issue.

Abbreviation: GEE: Generalized Estimating Equations     

On the revision of preferences and rational inference processes

Orderings and inference relations can be successfully used to model the behavior of a rational agent. This behavior is indeed represented either by a set of ordered pairs that reflect the agent's preferences, or by a rational inference relation that describes the agent's internal logics. In the finite case where we work, both structures admit a simple representation by means of logical chains. The problem of revising such inference processes arises when it appears necessary to modify the original model in order to take into account new facts about the agent's behavior. How is it then possible to perform the desired modification? We study here the possibilities offered by the technique of ‘chain revision’ which appears to be the easiest way to treat this kind of problem: the revision is performed through a simple modification of the logical chain attached to the agent's behavior, and the revision problem boils down to adding, retracting or modifying some of the links of the original chain. This perspective permits an effective treatment of the problems of both simple and multiple revision. The technique developed can also be used in some limiting cases, when the agent's inference process is only partially known, encoded by an incomplete set of preferences or a conditional knowledge base.     

英中可比语料库中多词表达自动提取与对齐

多词表达（MWE）不仅用来提高当前机器翻译系统质量,而且也用于跨语言检索和数据挖掘等其他自然语言处理领域。为此,提出了基于语义模板与基于统计工具相结合的方法从三元组可比语料库中自动提取本族英语MWE。采用基于词表和分布方法计算词语间的相似度,扩大MWE覆盖范围。利用GIZA++对齐算法提取对译的中文MWE,依据统计方法计算互译概率信息,根据概率大小,选择最佳英汉MWE互译对。实验结果表明上述方法可以有效提高MWE提取和对齐的准确率。     

The cluster assessment of facial attractiveness using fuzzy neural network classifier based on 3D Moiré features

Facial attractiveness has long been argued upon varied emphases by philosophers, artists, psychologists and biologists. A number of studies empirically investigated how facial attractiveness was influenced by 2D facial characteristics, such as symmetry, averageness and golden ratio. However, few implementations of facial beauty assessment were based on 3D facial features. The purpose of this paper is to propose a novel cluster assessment system for facial attractiveness that is characterized by the incorporation of 3D geometric Moiré features with an adjusted fuzzy neural network (FNN). We first extract 3D facial features from images acquired by a 3dMD scanner. Seven Moiré features are employed to represent a 3D facial image. The FNN classifier, taking the Moiré features as the parameters, is then trained and validated against independently conducted attractiveness ratings. A number of diverse referees were invited and offered their attractiveness ratings over a five-item Likert scale for 100 female facial images. The proposed assessment presents a high accuracy rate of 90%, and the area under curve (AUC) computed from the receiver operating characteristic (ROC) curve is 0.95. The results show that the perceptions of facial attractiveness are essentially consensus among raters, and can be mathematically modeled through supervised learning techniques. The high accuracy achieved proves that the proposed FNN classifier can serve as a general, automated and human-like judgment tool for objective classification of female facial attractiveness, and thus has potential applications to the entertainment industry, cosmetic industry, virtual media, and plastic surgery.     

Non-manual grammatical marker recognition based on multi-scale,spatio-temporal analysis of head pose and facial expressions

Changes in eyebrow configuration, in conjunction with other facial expressions and head gestures, are used to signal essential grammatical information in signed languages. This paper proposes an automatic recognition system for non-manual grammatical markers in American Sign Language (ASL) based on a multi-scale, spatio-temporal analysis of head pose and facial expressions. The analysis takes account of gestural components of these markers, such as raised or lowered eyebrows and different types of periodic head movements. To advance the state of the art in non-manual grammatical marker recognition, we propose a novel multi-scale learning approach that exploits spatio-temporally low-level and high-level facial features. Low-level features are based on information about facial geometry and appearance, as well as head pose, and are obtained through accurate 3D deformable model-based face tracking. High-level features are based on the identification of gestural events, of varying duration, that constitute the components of linguistic non-manual markers. Specifically, we recognize events such as raised and lowered eyebrows, head nods, and head shakes. We also partition these events into temporal phases. We separate the anticipatory transitional movement (the onset) from the linguistically significant portion of the event, and we further separate the core of the event from the transitional movement that occurs as the articulators return to the neutral position towards the end of the event (the offset). This partitioning is essential for the temporally accurate localization of the grammatical markers, which could not be achieved at this level of precision with previous computer vision methods. In addition, we analyze and use the motion patterns of these non-manual events. Those patterns, together with the information about the type of event and its temporal phases, are defined as the high-level features. Using this multi-scale, spatio-temporal combination of low- and high-level features, we employ learning methods for accurate recognition of non-manual grammatical markers in ASL sentences.     

基于照片的人体尺寸信息自动提取技术

针对3D扫描仪成本高,传统手工软尺测量效率低的状况,在保证精度的前提下,尝试应用图像处理理论从满足一定条件的照片提取人体尺寸信息,为建立人体尺寸信息库提供一种信息采集的自动化方法.通过对实验结果的分析,本文验证了此方法的可行性,并提出未来的研究方向.     

用于网上舆论观点抽取的几种方法

互联网上的海量信息中包含和反映了人们的观点和舆论倾向。从网上相关信息中抽取出人们的主观意向如消费者的购买倾向、政治观点等已成为了Web研究的热点。提出了几种适用于实时抽取网上舆论观点的算法,主要描述了各算法分析观点的原理及分类过程。     

Extension of Brzozowski’s derivation calculus of rational expressions to series over the free partially commutative monoids

We introduce an extension of the derivatives of rational expressions to expressions denoting formal power series over partially commuting variables. The expressions are purely noncommutative, however they denote partially commuting power series. The derivations (which are so-called -derivations) are shown to satisfy the commutation relations.Our main result states that for every so-called rigid rational expression, there exists a stable finitely generated submodule containing it. Moreover, this submodule is generated by what we call Words, that is by products of letters and of pure stars.Consequently this submodule is free and it follows that every rigid rational expression represents a recognizable series in . This generalizes the previously known property where the star was restricted to mono-alphabetic and connected series.     

Uncertainties of facial emotion recognition technologies and the automation of emotional labour

ABSTRACT

Automated detection and recognition of faces have been implemented in a broad range of media environments. Following that development, what concerns us in this article is the analysis of emotions from facial expressions using computer-based systems, in relation to which we critically investigate the use of theories of basic emotions. We explore in depth the company Affectiva’s attempts to translate, represent and schematize human emotions, as they raise a variety of problems and issues of uncertainty. We analyse the uncertainties concerning the processing of the human face ‘as image’ due to issues concerning temporality and static images as well as polyphony and modulations of the spectrum of expressions. One of our key arguments concerns the temporal character of human emotions, and we address how algorithmically regulated protocols of discretization may be said to prompt specific patterns of emotional responses and expressions based on an ideal of eliminating uncertainty. Through discussions via art pieces by Lauren McCarthy and Kyle McDonald, we question what happens when the protocols of computer systems start to perform aspects of emotional labour for us, making judgments by predicting adequate emotional responses based on the use of the strict metrics criticized in the article.     

Retrieval of aerosol type and optical thickness over the Mediterranean from SeaWiFS images using an automatic neural classification method

We present an automatic classification method based on topological neural network algorithms to retrieve aerosol optical properties from multi-spectral ocean-color satellite imagery. The first step of the method consisted in an unsupervised classification of a large set of clear-sky top of the atmosphere reflectance spectra measured by the sensor. We used the so-called Kohonen map which aggregates similar spectra into a reduced set of pertinent groups. The second step consisted in labeling these groups by clustering them with synthetic TOA reflectance spectra whose optical properties (i.e., aerosol type or optical thickness) are known. These synthetic spectra have been computed using a radiative transfer model. In the present study, we dealt with five aerosol types (maritime, coastal, tropospheric, oceanic and mineral) and several aerosol optical thickness values ranging from 0.05 to 0.8. These simulated spectra were then projected onto the Kohonen map to label each group of the map. The last step consisted in applying this method to the SeaWiFS imagery of the Mediterranean region for the years 1999 and 2000. The Kohonen map was “educated” from pixels randomly extracted during the year 1999 in this region. We accounted for the viewing geometry of the sensor by clustering the simulated spectra into ten groups of similar geometries, as defined by both scattering and sun zenith angles. The analysis of SeaWiFS images was performed pixel-by-pixel by selecting the suitable labeling (in terms of viewing geometry), then by identifying the closest spectrum in the Kohonen map, which finally gives the aerosol optical properties. This method led to accurate and coherent results, as shown by the comparison with in situ aerosol measurements provided by the AERONET station at Lampedusa and by the study of two aerosol events over the Mediterranean. One of the major advantages of this method is that it enables us to automatically identify the aerosol type and to retrieve the aerosol optical properties with a better accuracy than classical methods such as those used by SeaWifs. It gives accurate results for optical thickness values larger than 0.35 and is able to retrieve dust aerosols such as African dust aerosol (absorbing aerosol). These should ensure a more precise inversion of ocean-color imagery where the knowledge of atmospheric optical parameters is essential. Moreover the method is able to give probabilities for the estimate values of aerosol properties.     

基于混合核函数的SVM在文本自动分类的应用

核函数是SVM的关键技术,核函数的选择将影响着学习机器的学习能力和泛化能力。不同的核函数确定了不同的非线性变换和特征空间,选取不同核函数训练SVM就会得到不同的分类效果。本文提出了一种混合的核函数[1]Kmix=λKpoly+(1-λ)Krbf,从而兼并二项式核函数及径向基核函数的优势。实验证明选用混合核函数的支持向量机,与普通核函数构造的支持向量机的评估效果进行比较,混合核函数支持向量机具有较高的分类精度。     

Collaborative virtual prototyping of product assemblies over the Internet

Product assembly design is a complex activity involving collaboration between several designers. Currently, many manufacturing firms out source the design and development of different portions of a product to different suppliers. Due to the absence of an integrated tool for collaborative product assembly design, designers currently follow an indirect approach. This makes collaboration an unorganized and inefficient activity. This paper discusses the architectural novelties and design considerations utilized to develop an integrated collaborative assembly design tool. Further, the scenarios of using such a tool are discussed in detail. This discussion highlights how the tool contributes towards making collaboration an organized and efficient activity and enables designers to rapidly design and prototype the product assembly.