Facial expression recognition of intercepted video sequences based on feature point movement trend and feature block texture variation

Facial Expression Recognition (FER) is an important subject of human–computer interaction and has long been a research area of great interest. Accurate Facial Expression Sequence Interception (FESI) and discriminative expression feature extraction are two enormous challenges for the video-based FER. This paper proposes a framework of FER for the intercepted video sequences by using feature point movement trend and feature block texture variation. Firstly, the feature points are marked by Active Appearance Model (AAM) and the most representative 24 of them are selected. Secondly, facial expression sequence is intercepted from the face video by determining two key frames whose emotional intensities are minimum and maximum, respectively. Thirdly, the trend curve which represents the Euclidean distance variations between any two selected feature points is fitted, and the slopes of specific points on the trend curve are calculated. Finally, combining Slope Set which is composed by the calculated slopes with the proposed Feature Block Texture Difference (FBTD) which refers to the texture variation of facial patch, the final expressional feature are formed and inputted to One-dimensional Convolution Neural Network (1DCNN) for FER. Five experiments are conducted in this research, and three average FER rates 95.2%, 96.5%, and 97% for Beihang University (BHU) facial expression database, MMI facial expression database, and the combination of two databases, respectively, have shown the significant advantages of the proposed method over the existing ones.     

Level of match between facial dimensions of Chilean workers and respirator fit test panels proposed by LANL and NIOSH

The facial fit of respirators is crucial for determining how effectively respirators may protect users from exposure to airborne contaminants, when their use is required in the workplace. In the Chilean market, all the respirators available have been designed and manufactured using foreign regulations. The aim of this research was to determine the facial dimensions in a sample of Chilean workers (users or potential users of respiratory protective equipment) and the possible mismatch between their anthropometric characteristics and the respirator fit test panels proposed by Los Alamos National Laboratory (LANL) and National Institute for Occupational Safety and Health (NIOSH). An anthropometric survey that included 11 measurements was conducted, based on ISO/TS 16976–2 and ISO 15535 to ensure the highest standards possible, and a total of 474 workers (female: 229, male: 245), aged 18–66 years old participated in the survey. The anthropometric measurements were then contrasted with the fit test panels used in LANL (for half and full facepieces) and NIOSH (Bivariate and Principal component analysis (PCA)), to verify the level of mismatch. The results showed that LANL panels presented a level of mismatch of 11.8% and 21% for the half-facepiece and the full-facepiece, respectively. Considering the NIOSH bivariate and PCA panels, 4.6% and 4.4% of the sample remains without an assigned cell, respectively. It can be concluded that the LANL panels for half and full facepieces do not match the facial dimensions of the Chilean working population. The panels developed by NIOSH and considered by the ISO/TS 16976–2 (bivariate and PCA), are applicable to the Chilean working population.Relevance for the IndustryThis research provides anthropometric measurements of Chilean workers, to determine the dimensions for half- and full-facepiece respirators, which are currently not available. The NIOSH or ISO fit test panels, as opposed to LANL panels, should be used when manufacturing respirators for Chilean workers.     

Validity of facial features’ geometric measurements for real-time assessment of mental fatigue in construction equipment operators

Operating construction equipment for extended periods of time may lead to mental fatigue and, as a result, an increased risk of human error-related accidents and jeopardized health problems for the operators. Therefore, to limit the risk of accidents and protect operators' wellbeing, their mental fatigue must be monitored reliably and in real time. Recently, many invasive technologies have been employed to alleviate this problem, but they entail the wearing of physical sensors, which may instigate irritation and discomfort. This study proposes a non-invasive mental fatigue monitoring method using geometric measurements of their facial features that does not require the operators to wear sensors on their body. The study further validates the proposed method by comparing it with wearable electroencephalography (EEG) technology to establish its ecological validity for construction equipment operators. To serve the purpose, a one-hour excavator operation by sixteen construction equipment operators was conducted on a construction site. Ground truth, brain activity using wearable EEG, and geometric measurements of facial features were extracted and analyzed at the baseline and every 20 min for one hour. A considerable temporal variation was found in the reported metrics (eye aspect ratio, eye distance, mouth aspect ratio, face area, and head motion) and were significantly correlated with ground truth and EEG metric. Furthermore, the brain visualization pattern obtained from EEG was also associated with the variations in the facial features. The findings of the study reveal that construction equipment operators’ mental fatigue can be monitored non-invasively using geometrical measurements of facial features.     

江南园林在北方居住区景观设计中的应用——以宝鸡阳光百翠园为例

中国古典园林是中国传统文化艺术的结晶,是师法自然的写意山水园林,其中江南园林作为中国古典园林的典型代表,更是体现了古典园林"天人合一"的哲学思想.本文通过对江南园林艺术的研究与理解,以宝鸡阳光百翠园居住区为例,分析江南园林艺术要素及手法在北方现代居住区景观设计当中的应用,探索研究江南园林艺术传承与发展创新的一些思考,提出适于时代发展的现代居住区景观设计思路.     

Types of humor that robots can play

Although humor is a well-known social lubricant defusing a complicated conflict between two parties, the efficacy of humor in human–robot interaction has barely been tested yet. This study compared the characteristics of humor performed by a robot and human to identify the possible type of jokes that a robot may play.In the experiment, a human actor performed disparaging – racist and sexist jokes, and non-disparaging (human condition and sexual) jokes, and a robot counterpart mimicked the same performance. Fifty-eight university students, 30 male and 28 female with mean age 23.10 (SD = 2.00), watched the randomly assigned jokes performed either by the robot or the human actor. The participants rated perceived humorousness, offensiveness, and willingness to share the joke with others, the perceived social presence and social attractions of the actor. The result showed that participants perceived non-disparaging jokes to be more humorous when performed by the human actor. On the other hand, the participants exhibited less disgust toward disparaging jokes when they were performed by the robot actor. This shows that humor can be used as an effective way to enrich the interaction between human and robot; but the acceptable types of humor should be carefully selected.     

Cross grouping strategy based 2DPCA method for face recognition

Grouping strategy exactly specifies the form of covariance matrix, therefore it is very essential. Most 2DPCA methods use the original 2D image matrices to form the covariance matrix which actually means that the strategy is to group the random variables by row or column of the input image. Because of their grouping strategies these methods have two main drawbacks. Firstly, 2DPCA and some of its variants such as A2DPCA, DiaPCA and MatPCA preserve only the covariance information between the elements of these groups. This directly implies that 2DPCA and these variants eliminate some covariance information while PCA preserves such information that can be useful for recognition. Secondly, all the existing methods suffer from the relatively high intra-group correlation, since the random variables in a row, column, or a block are closely located and highly correlated. To overcome such drawbacks we propose a novel grouping strategy named cross grouping strategy. The algorithm focuses on reducing the redundancy among the row and the column vectors of the image matrix. While doing this the algorithm completely preserves the covariance information of PCA between local geometric structures in the image matrix which is partially maintained in 2DPCA and its variants. And also in the proposed study intra-group correlation is weak according to the 2DPCA and its variants because the random variables spread over the whole face image. These make the proposed algorithm superior to 2DPCA and its variants. In order to achieve this, image cross-covariance matrix is calculated from the summation of the outer products of the column and the row vectors of all images. The singular value decomposition (SVD) is then applied to the image cross-covariance matrix. The right and the left singular vectors of SVD of the image cross-covariance matrix are used as the optimal projective vectors. Further in order to reduce the dimension LDA is applied on the feature space of the proposed method that is proposed method + LDA. The exhaustive experimental results demonstrate that proposed grouping strategy for 2DPCA is superior to 2DPCA, its specified variants and PCA, and proposed method outperforms bi-directional PCA + LDA.     

Representation of facial expression categories in continuous arousal–valence space: Feature and correlation

Representation of facial expressions using continuous dimensions has shown to be inherently more expressive and psychologically meaningful than using categorized emotions, and thus has gained increasing attention over recent years. Many sub-problems have arisen in this new field that remain only partially understood. A comparison of the regression performance of different texture and geometric features and the investigation of the correlations between continuous dimensional axes and basic categorized emotions are two of these. This paper presents empirical studies addressing these problems, and it reports results from an evaluation of different methods for detecting spontaneous facial expressions within the arousal–valence (AV) dimensional space. The evaluation compares the performance of texture features (SIFT, Gabor, LBP) against geometric features (FAP-based distances), and the fusion of the two. It also compares the prediction of arousal and valence, obtained using the best fusion method, to the corresponding ground truths. Spatial distribution, shift, similarity, and correlation are considered for the six basic categorized emotions (i.e. anger, disgust, fear, happiness, sadness, surprise). Using the NVIE database, results show that the fusion of LBP and FAP features performs the best. The results from the NVIE and FEEDTUM databases reveal novel findings about the correlations of arousal and valence dimensions to each of six basic emotion categories.     

Wireless Face Interface: Using voluntary gaze direction and facial muscle activations for human-computer interaction

The present aim was to investigate the functionality of a new wireless prototype called Face Interface. The prototype combines the use of voluntary gaze direction and facial muscle activations, for pointing and selecting objects on a computer screen, respectively. The subjective and objective functionality of the prototype was evaluated with a series of pointing tasks using either frowning (i.e., frowning technique) or raising the eyebrows (i.e., raising technique) as the selection technique. Pointing task times and accuracies were measured using three target diameters (i.e., 25, 30, 40 mm), seven pointing distances (i.e., 60, 120, 180, 240, 260, 450, and 520 mm), and eight pointing angles (0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°). The results showed that the raising technique was faster selection technique than the frowning technique for the objects that were presented in the pointing distances from 60 mm to 260 mm. For those pointing distances the overall pointing task times were 2.4 s for the frowning technique, and 1.6 s for the raising technique. Fitts’ law computations showed that the correlations for the Fitts’ law model were r = 0.77 for the frowning technique and r = 0.51 for the raising technique. Further, the index of performance (IP) value was 1.9 bits/s for the frowning technique and 5.4 bits/s for raising the eyebrows technique. Based on the results, the prototype functioned well and was adjustable so that two different facial activations can be used in combination with gaze direction for pointing and selecting objects on a computer screen.     

Output-associative RVM regression for dimensional and continuous emotion prediction

Many problems in machine learning and computer vision consist of predicting multi-dimensional output vectors given a specific set of input features. In many of these problems, there exist inherent temporal and spatial dependencies between the output vectors, as well as repeating output patterns and input–output associations, that can provide more robust and accurate predictors when modeled properly. With this intrinsic motivation, we propose a novel Output-Associative Relevance Vector Machine (OA-RVM) regression framework that augments the traditional RVM regression by being able to learn non-linear input and output dependencies. Instead of depending solely on the input patterns, OA-RVM models output covariances within a predefined temporal window, thus capturing past, current and future context. As a result, output patterns manifested in the training data are captured within a formal probabilistic framework, and subsequently used during inference. As a proof of concept, we target the highly challenging problem of dimensional and continuous prediction of emotions, and evaluate the proposed framework by focusing on the case of multiple nonverbal cues, namely facial expressions, shoulder movements and audio cues. We demonstrate the advantages of the proposed OA-RVM regression by performing subject-independent evaluation using the SAL database that constitutes naturalistic conversational interactions. The experimental results show that OA-RVM regression outperforms the traditional RVM and SVM regression approaches in terms of accuracy of the prediction (evaluated using the Root Mean Squared Error) and structure of the prediction (evaluated using the correlation coefficient), generating more accurate and robust prediction models.