Emotion recognition using eigenvalues and Levenberg–Marquardt algorithm-based classifier

In this paper, a simple and computationally efficient approach is proposed for person independent facial emotion recognition. The proposed approach is based on the significant features of an image, i.e., the collection of few largest eigenvalues (LE). Further, a Levenberg–Marquardt algorithm-based neural network (LMNN) is applied for multiclass emotions classification. This leads to a new facial emotion recognition approach (LE-LMNN) which is systematically examined on JAFFE and Cohn–Kanade databases. Experimental results illustrate that the LE-LMNN approach is effective and computationally efficient for facial emotion recognition. The robustness of the proposed approach is also tested on low-resolution facial emotion images. The performance of the proposed approach is found to be superior as compared to the various existing methods.     

Facial expression recognition by using differential geometric features

ABSTRACT

In recent years, a growing interest has been created for improvement of human interaction with computers. Hence, automatic recognition of facial expressions has become one of the active research topics. The purpose of this paper is to identify facial expressions, by using differential geometric features. In the proposed method, only the first and last images are used and differential features are extracted from these two images. Differential geometric features are extracted from changes in the important points of the face in the two images. In this method, the distance between the important points of the face and the reference point was calculated in both directions x and y, for two images, and with the difference between the distance, the differential geometric features between the two images were obtained. Based on the results, with this method, recognition accuracy of six facial expressions in the database was 96.44%, CK +.     

Multimodal Emotion Recognition Based on Facial Expression and ECG Signal

As a key link in human-computer interaction, emotion recognition can enable robots to correctly perceive user emotions and provide dynamic and adjustable services according to the emotional needs of different users, which is the key to improve the cognitive level of robot service. Emotion recognition based on facial expression and electrocardiogram has numerous industrial applications. First, three-dimensional convolutional neural network deep learning architecture is utilized to extract the spa...     

Mental Illness Disorder Diagnosis Using Emotion Variation Detection from Continuous English Speech

Automatic recognition of human emotions in a continuous dialog model remains challenging where a speaker’s utterance includes several sentences that may not always carry a single emotion. Limited work with standalone speech emotion recognition (SER) systems proposed for continuous speech only has been reported. In the recent decade, various effective SER systems have been proposed for discrete speech, i.e., short speech phrases. It would be more helpful if these systems could also recognize emotions from continuous speech. However, if these systems are applied directly to test emotions from continuous speech, emotion recognition performance would not be similar to that achieved for discrete speech due to the mismatch between training data (from training speech) and testing data (from continuous speech). The problem may possibly be resolved if an existing SER system for discrete speech is enhanced. Thus, in this work the author’s existing effective SER system for multilingual and mixed-lingual discrete speech is enhanced by enriching the cepstral speech feature set with bi-spectral speech features and a unique functional set of Mel frequency cepstral coefficient features derived from a sine filter bank. Data augmentation is applied to combat skewness of the SER system toward certain emotions. Classification using random forest is performed. This enhanced SER system is used to predict emotions from continuous speech with a uniform segmentation method. Due to data scarcity, several audio samples of discrete speech from the SAVEE database that has recordings in a universal language, i.e., English, are concatenated resulting in multi-emotional speech samples. Anger, fear, sad, and neutral emotions, which are vital during the initial investigation of mentally disordered individuals, are selected to build six categories of multi-emotional samples. Experimental results demonstrate the suitability of the proposed method for recognizing emotions from continuous speech as well as from discrete speech.     

融合人脸表情的手语到汉藏双语情感语音转换

针对聋哑人与正常人之间存在的交流障碍问题,提出了一种融合人脸表情的手语到汉藏双语情感语音转换的方法。首先使用深度置信网络模型得到手势图像的特征信息,并通过深度神经网络模型得到人脸信息的表情特征。其次采用支持向量机对手势特征和人脸表情特征分别进行相应模型的训练及分类,根据识别出的手势信息和人脸表情信息分别获得手势文本及相应的情感标签。同时,利用普通话情感训练语料,采用说话人自适应训练方法,实现了一个基于隐Markov模型的情感语音合成系统。最后,利用识别获得的手势文本和情感标签,将手势及人脸表情转换为普通话或藏语的情感语音。客观评测表明,静态手势的识别率为92.8%,在扩充的Cohn-Kanade数据库和日本女性面部表情(Japanese Female Facial Expression,JAFFE)数据库上的人脸表情识别率为94.6%及80.3%。主观评测表明,转换获得的情感语音平均情感主观评定得分4.0分,利用三维情绪模型(Pleasure-Arousal-Dominance,PAD)分别评测人脸表情和合成的情感语音的PAD值,两者具有很高的相似度,表明合成的情感语音能够表达人脸表情的情感。     

Emotion recognition based on facial components

Machine analysis of facial emotion recognition is a challenging and an innovative research topic in human–computer interaction. Though a face displays different facial expressions, which can be immediately recognized by human eyes, it is very hard for a computer to extract and use the information content from these expressions. This paper proposes an approach for emotion recognition based on facial components. The local features are extracted in each frame using Gabor wavelets with selected scales and orientations. These features are passed on to an ensemble classifier for detecting the location of face region. From the signature of each pixel on the face, the eye and the mouth regions are detected using the ensemble classifier. The eye and the mouth features are extracted using normalized semi-local binary patterns. The multiclass Adaboost algorithm is used to select and classify these discriminative features for recognizing the emotion of the face. The developed methods are deployed on the RML, CK and CMU-MIT databases, and they exhibit significant performance improvement owing to their novel features when compared with the existing techniques.     

Facial Expression Recognition Model Depending on Optimized Support Vector Machine

In computer vision, emotion recognition using facial expression images is considered an important research issue. Deep learning advances in recent years have aided in attaining improved results in this issue. According to recent studies, multiple facial expressions may be included in facial photographs representing a particular type of emotion. It is feasible and useful to convert face photos into collections of visual words and carry out global expression recognition. The main contribution of this paper is to propose a facial expression recognition model (FERM) depending on an optimized Support Vector Machine (SVM). To test the performance of the proposed model (FERM), AffectNet is used. AffectNet uses 1250 emotion-related keywords in six different languages to search three major search engines and get over 1,000,000 facial photos online. The FERM is composed of three main phases: (i) the Data preparation phase, (ii) Applying grid search for optimization, and (iii) the categorization phase. Linear discriminant analysis (LDA) is used to categorize the data into eight labels (neutral, happy, sad, surprised, fear, disgust, angry, and contempt). Due to using LDA, the performance of categorization via SVM has been obviously enhanced. Grid search is used to find the optimal values for hyperparameters of SVM (C and gamma). The proposed optimized SVM algorithm has achieved an accuracy of 99% and a 98% F1 score.     

Processing of 3D sheet metal components in STEP AP-203 format. Part I: feature recognition system

To automate planning activities in a computer integrated manufacturing environment, an integrated system of feature recognition and reasoning is essential. An attempt is made in the present work to develop such a system for 3D sheet metal components. Though certain part-modellers use feature-based methodology, they lack the information required for manufacturing and entire feature information is lost when converted to a neutral format such as STEP AP-203. The proposed feature recognition identifies manufacturing features in a generic manner, while feature reasoning gives the information required for manufacturing. Taking 3D model data in STEP AP-203 format as input to the feature recognition system, the central plane of the component is first generated. Further processing of faces is carried out and various features with similar manufacturing attributes are identified using a set of rules based on the topology, geometry and Boolean logic. Different types of manufacturing features such as cut, stretched, drawn and bent features as well as composite features are effectively identified irrespective of their shape. The system proposed here was tested with components taken from industry and examples available in the published literature. The proposed feature recognition system serves as input to the feature reasoning system dealt with in Part II of this work (Kannan, T.R. and Shunmugam, M.S., Processing of 3D Sheet metal components in STEP AP-203 format. Part II: feature reasoning system. Int. J. Prod. Res., 2009 (in press)).     

A comprehensive study on face recognition: methods and challenges

ABSTRACT

Face Recognition is the process of identifying and verifying the faces. Face recognition has vast importance in the field of Security, Healthcare, Banking, Criminal Identification, Payment, and Advertising. In this paper, we have reviewed various techniques and challenges for the face recognition. Illumination, pose variation, facial expressions, occlusions, aging, etc. are the key challenges to the success of face recognition. Pre-processing, Face Detection, Feature Extraction, Optimal Feature Selection, and Classification are primary steps in any face recognition system. This paper provides a detailed review of each. Feature extraction techniques can be classified as appearance-based methods or geometry-based methods, such method may be local or global. Feature extraction is the most crucial stage for the success of the face recognition system. However, deep learning methods have freed the user from handcrafting the features. In this article, we have surveyed state-of-the-art methods of last few decades and the comparative study of various feature extraction methods is provided. Article also describes the current challenges in the area.     

非均衡训练集过采样的印刷套准识别方法

目的 针对印刷标志图像训练数据集非均衡性导致印刷标志图像中少类数据套准状态识别准确率低的问题,提出改进的SMOTE训练集过采样方法,以提高少类数据的识别准确率。方法 提取印刷标志图像灰度行程矩阵的纹理特征,组成多维的模型输入特征数据。基于少类样本的邻域信息,得到少类样本的过采样参数。对少类样本采取不同的过采样策略,实现训练集样本的均衡。使用均衡的训练集建立支持向量机模型,实现对印刷套准状态的识别。结果 实验结果表明,文中方法在不同非均衡印刷数据集上,获得的平均分类准确率几何平均数Gmean为0.8507,召回率Re为0.7192,ROC曲线下面积A为0.8549。结论 文中方法在不同非均衡印刷套准数据集上的分类性能要优于实验中的SMOTE,IS和SVM等方法。     

Stacked denoising autoencoder‐based feature learning for out‐of‐control source recognition in multivariate manufacturing process

In multivariate statistical process control (MSPC), regular multivariate control charts (eg, T²) are shown to be effective in detecting out‐of‐control signals based upon an overall statistic. But these charts do not relieve the need for multivariate process pattern recognition (MPPR). MPPR would be very useful for quality operators to locate the assignable causes that give rise to out‐of‐control situation in multivariate manufacturing process. Deep learning has been widely applied and obtained many successes in image and visual analysis. This paper presents an effective and reliable deep learning method known as stacked denoising autoencoder (SDAE) for MPPR in manufacturing processes. This study will concentrate on developing a SDAE model to learn effective discriminative features from the process signals through deep network architectures. Feature visualization is performed to explicitly present feature representations of the proposed SDAE model. The experimental results illustrate that the proposed SDAE model is capable of implementing detection and recognition of various process patterns in complicated multivariate processes. Analysis from this study provides the guideline in developing deep learning‐based MSPC systems.     

基于注意力机制的LSTM语音情感主要特征选择

传统的语音情感识别方式采用的语音特征具有数据量大且无关特征多的特点,因此选择出与情感相关的语音特征具有重要意义。通过提出将注意力机制结合长短时记忆网络(Long Short Term Memory, LSTM),根据注意力权重进行特征选择,在两个数据集上进行了实验。结果发现:(1)基于注意力机制的LSTM相比于单独的LSTM模型,识别率提高了5.4%,可见此算法有效提高了模型的识别效果;(2)注意力机制是一种有效的特征选择方法。采用注意力机制选择出了具有实际物理意义的声学特征子集,此特征集相比于原有公用特征集在降低了维数的情况下,提高了识别准确率;(3)根据选择结果对声学特征进行分析,发现有声片段长度特征、无声片段长度特征、梅尔倒谱系数(Mel-Frequency Cepstral Coefficient, MFCC)、F0基频等特征与情感识别具有较大相关性。     

Cotton Leaf Diseases Recognition Using Deep Learning and Genetic Algorithm

Globally, Pakistan ranks 4 in cotton production, 6 as an importer of raw cotton, and 3 in cotton consumption. Nearly 10% of GDP and 55% of the country's foreign exchange earnings depend on cotton products. Approximately 1.5 million people in Pakistan are engaged in the cotton value chain. However, several diseases such as Mildew, Leaf Spot, and Soreshine affect cotton production. Manual diagnosis is not a good solution due to several factors such as high cost and unavailability of an expert. Therefore, it is essential to develop an automated technique that can accurately detect and recognize these diseases at their early stages. In this study, a new technique is proposed using deep learning architecture with serially fused features and the best feature selection. The proposed architecture consists of the following steps: (a) a self-collected dataset of cotton diseases is prepared and labeled by an expert; (b) data augmentation is performed on the collected dataset to increase the number of images for better training at the earlier step; (c) a pre-trained deep learning model named ResNet101 is employed and trained through a transfer learning approach; (d) features are computed from the third and fourth last layers and serially combined into one matrix; (e) a genetic algorithm is applied to the combined matrix to select the best points for further recognition. For final recognition, a Cubic SVM approach was utilized and validated on a prepared dataset. On the newly prepared dataset, the highest achieved accuracy was 98.8% using Cubic SVM, which shows the perfection of the proposed framework..     

基于栈式自编码神经网络的脑电信号情绪识别

针对维度情感模型生理信号情绪识别准确率较低的问题,本文基于DEAP维度情绪生理数据集,利用AR模型功率谱估计方法,提取脑电θ,α,β,γ节律的功率谱密度;采用小波包分解提取脑电小波包系数和能量占比时频特征;通过非线性分析提取脑电样本熵和小波包熵特征.然后,设计栈式自编码神经网络算法对脑电组合特征在效价和唤醒度两个情感维...     

Occlusion invariant face recognition using mean based weight matrix and support vector machine

In this paper, a novel occlusion invariant face recognition algorithm based on Mean based weight matrix (MBWM) technique is proposed. The proposed algorithm is composed of two phases—the occlusion detection phase and the MBWM based face recognition phase. A feature based approach is used to effectively detect partial occlusions for a given input face image. The input face image is first divided into a finite number of disjointed local patches, and features are extracted for each patch, and the occlusion present is detected. Features obtained from the corresponding occlusion-free patches of training images are used for face image recognition. The SVM classifier is used for occlusion detection for each patch. In the recognition phase, the MBWM bases of occlusion-free image patches are used for face recognition. Euclidean nearest neighbour rule is applied for the matching. GTAV face database that includes many occluded face images by sunglasses and hand are used for the experiment. The experimental results demonstrate that the proposed local patch-based occlusion detection technique works well and the MBWM based method shows superior performance to other conventional approaches.     

A Combined Ordered Macro‐Mesoporous Architecture Design and Surface Engineering Strategy for High‐Performance Sulfur Immobilizer in Lithium–Sulfur Batteries

The practical application of lithium–sulfur (Li–S) batteries is hindered by the “shuttle” of lithium polysulfides (LiPS) and sluggish Li–S kinetics issues. Herein, a synergistic strategy combining mesoporous architecture design and defect engineering is proposed to synthesize multifunctional defective 3D ordered mesoporous cobalt sulfide (3DOM N‐Co₉S_8?x) to address the shuttling and sluggish reaction kinetics of polysulfide in Li–S batteries. The unique 3DOM design provides abundant voids for sulfur storage and enlarged active interfaces that reduce electron/ion diffusion pathways. Meanwhile, X‐ray absorption spectroscopy shows that the surface defect engineering tunes the CoS₄ tetrahedra to CoS₆ octahedra on Co₉S₈, endowing abundance of S vacancies on the Co₉S₈ octahedral sites. The ever‐increasing S vacancies over the course of electrochemical process further promotes the chemical trapping of LiPS and its conversion kinetics, rendering fast and durable Li–S chemistry. Benefiting from these features, the as‐developed 3DOM N‐Co₉S_8?x/S cathode delivers high areal capacity, superb rate capability, and excellent cyclic stability with ultralow capacity fading rate under raised sulfur loading and low electrolyte content. This design strategy promotes the development of practically viable Li–S batteries and sheds lights on the material engineering in related energy storage application.     

A conceptual development framework for intuitive human pattern recognition

Abstract

In human visual perception, there is an intuitive tolerance (perceptual constancy) for perceiving raw stimulus patterns which may be rotated, scaled, deformed, or noisy to some extent for a learned pattern. Such an intuitive property of perception is a major feature of human pattern recognition, where a recognition mechanism (consider the character recognition for example) may not necessarily follow in a stroke‐based (feature‐oriented) scheme; rather, it may follow in a whole‐image‐based (spatial‐oriented) scheme. In this paper, we present a conceptual development framework for intuitive human pattern recognition. The implementation model is called I‐net and basically consists of three parts: an attention mechanism, a generalization mechanism, and a recognition mechanism. The paper begins with describing the intuitive properties involved in recognizing 2‐D binary patterns, following an introduction to intuitive human pattern recognition. Then the details of the I‐net are described and experimental results are presented. The paper concludes that the proposed framework provides another direction for approaching human pattern recognition.     

不均衡训练集特征选择的印刷套准状态识别

目的 针对不均训练集导致印刷套准识别模型无法较好识别印刷套不准图像的问题,提出基于最大相关、最小冗余的印刷标志图像数据特征选择方法.方法 提取印刷标志图像的多维特征数据,计算特征与印刷套准和印刷套不准2类之间的相关性和特征之间的冗余度.确定特征选择的目标函数,通过增量搜索方法寻找最优特征,加入特征子集,实现不均衡印刷标志图像的特征选择.结果 文中的特征选择方法获得了3项不均衡数据分类性能评价指标,A为0.9900,R为0.9400,Gmean为0.9466.结论 在不均衡印刷标志图像套准识别中,文中提出的方法性能优于实验中的未处理方法、PCA方法、Relief方法和NCA方法.     

Performance degradation assessment of rolling bearing based on convolutional neural network and deep long-short term memory network

Many traditional approaches for performance degradation assessment of rolling bearings, using sensor data, make assumptions about how they degrade or fault evolve. However, the sequential sensor data cannot be directly taken as input in the traditional models since the data always contain noise and change in length. To solve these problems, a convolutional neural network and deep long-short term memory (CNN-DLSTM) based architecture is proposed to obtain an unsupervised H-statistic for performance degradation assessment of rolling bearing using sensor time-series data. Firstly, a CNN is applied to extract local abstract features from raw sensor data. Secondly, a deep LSTM is explored to extract temporal features. CNN-DLSTM is trained to reconstruct the time-series sensor signal reflecting the health condition of rolling bearing. The D- and Q-statistic are used to compute H-statistic which is then used for performance degradation assessment. The proposed approach is evaluated on an experiment with rolling bearings and the results are presented on a public dataset of rolling bearing, verifying that the proposed approach outperforms several state-of-the-art methods.