Benchmarking lightweight face architectures on specific face recognition scenarios

Artificial Intelligence Review - This paper studies the impact of lightweight face models on real applications. Lightweight architectures proposed for face recognition are analyzed and evaluated on...     

Bidirectional representation for face recognition across pose

Conventional representation methods try to express the test sample as a weighting sum of training samples and exploit the deviation between the test sample and the weighting sum of the training samples from each class (also referred to as deviation between the test sample and each class) to classify the test sample. In particular, the methods assign the test sample to the class that has the smallest deviation among all the classes. This paper analyzes the relationship between face images under different poses and, for the first time, devises a bidirectional representation method-based pattern classification (BRBPC) method for face recognition across pose. BRBPC includes the following three steps: the first step uses the procedure of conventional representation methods to express the test sample and calculates the deviation between the test sample and each class. The second step first expresses the training sample of a class as a weighting sum of the test sample and the training samples from all the other classes and then obtains the corresponding deviation (referred to as complementary deviation). The third step uses the score-level fusion to integrate the scores, that is, deviations generated from the first and second steps for final classification. The experimental results show that BRBPC classifies more accurately than conventional representation methods.     

Orthogonal discriminant vector for face recognition across pose

Recognizing face images across pose is one of the challenging tasks for reliable face recognition. This paper presents a new method to tackle this challenge based on orthogonal discriminant vector (ODV). The result of our theoretical analysis shows that an individual’s probe image captured with a new pose can be represented by a linear combination of his/her gallery images. Based on this observation, in contrast to the conventional methods which model face images of different individuals on a single manifold, we propose to model face images of different individuals on different linear manifolds. The contribution of our approach includes: (1) to prove that the orthogonality to ODVs is a pose-invariant feature.; (2) to categorize each person with a set of ODVs, where his/her face images posses zero projections while other persons’ images are characterized by maximum projections; (3) to define a metric to measure the distance between a face image and an ODV, and classify the face images based on this metric. Our experimental results validate the feasibility of modeling the face images of different individuals on different linear manifolds. The proposed method achieves higher accuracy on face recognition and verification than the existing techniques.     

A novel face recognition algorithm for distinguishing faces with various angles

In order to distinguish faces of various angles during face recognition, an algorithm of the combination of approximate dynamic programming （ADP） called action dependent heuristic dynamic programming （ADHDP） and particle swarm optimization （PSO） is presented. ADP is used for dynamically changing the values of the PSO parameters. During the process of face recognition, the discrete cosine transformation （DCT） is first introduced to reduce negative effects. Then, Karhunen-Loeve （K-L） transformation can be used to compress images and decrease data dimensions. According to principal component analysis （PCA）, the main parts of vectors are extracted for data representation. Finally, radial basis function （RBF） neural network is trained to recognize various faces. The training of RBF neural network is exploited by ADP-PSO. In terms of ORL Face Database, the experimental result gives a clear view of its accurate efficiency.     

Illumination robust single sample face recognition using multi-directional orthogonal gradient phase faces

Due to the limitation of the storage space in the real-world face recognition application systems, only one sample image per person is often stored in the system, which is the so-called single sample problem. Moreover, real-world illumination has impact on recognition performance. This paper presents an illumination robust single sample face recognition approach, which utilizes multi-directional orthogonal gradient phase faces to solve the above limitations. In the proposed approach, an illumination insensitive orthogonal gradient phase face is obtained by using two vertical directional gradient values of the original image. Multi-directional orthogonal gradient phase faces can be used to extend samples for single sample face recognition. Simulated experiments and comparisons on a subset of Yale B database, Yale database, a subset of PIE database and VALID face database show that the proposed approach is not only an outstanding method for single sample face recognition under illumination but also more effective when addressing illumination, expression, decoration, etc.     

Subject-specific and pose-oriented facial features for face recognition across poses

Most face recognition scenarios assume that frontal faces or mug shots are available for enrollment to the database, faces of other poses are collected in the probe set. Given a face from the probe set, one needs to determine whether a match in the database exists. This is under the assumption that in forensic applications, most suspects have their mug shots available in the database, and face recognition aims at recognizing the suspects when their faces of various poses are captured by a surveillance camera. This paper considers a different scenario: given a face with multiple poses available, which may or may not include a mug shot, develop a method to recognize the face with poses different from those captured. That is, given two disjoint sets of poses of a face, one for enrollment and the other for recognition, this paper reports a method best for handling such cases. The proposed method includes feature extraction and classification. For feature extraction, we first cluster the poses of each subject's face in the enrollment set into a few pose classes and then decompose the appearance of the face in each pose class using Embedded Hidden Markov Model, which allows us to define a set of subject-specific and pose-priented (SSPO) facial components for each subject. For classification, an Adaboost weighting scheme is used to fuse the component classifiers with SSPO component features. The proposed method is proven to outperform other approaches, including a component-based classifier with local facial features cropped manually, in an extensive performance evaluation study.     

Comparison of human and computer performance across face recognition experiments

Since 2005, human and computer performance has been systematically compared as part of face recognition competitions, with results being reported for both still and video imagery. The key results from these competitions are reviewed. To analyze performance across studies, the cross-modal performance analysis (CMPA) framework is introduced. The CMPA framework is applied to experiments that were part of face a recognition competition. The analysis shows that for matching frontal faces in still images, algorithms are consistently superior to humans. For video and difficult still face pairs, humans are superior. Finally, based on the CMPA framework and a face performance index, we outline a challenge problem for developing algorithms that are superior to humans for the general face recognition problem.     

Tied factor analysis for face recognition across large pose differences

Face recognition algorithms perform very unreliably when the pose of the probe face is different from the gallery face: typical feature vectors vary more with pose than with identity. We propose a generative model that creates a one-to-many mapping from an idealized "identity" space to the observed data space. In identity space, the representation for each individual does not vary with pose. We model the measured feature vector as being generated by a pose-contingent linear transformation of the identity variable in the presence of Gaussian noise. We term this model "tied" factor analysis. The choice of linear transformation (factors) depends on the pose, but the loadings are constant (tied) for a given individual. We use the EM algorithm to estimate the linear transformations and the noise parameters from training data. We propose a probabilistic distance metric which allows a full posterior over possible matches to be established. We introduce a novel feature extraction process and investigate recognition performance using the FERET, XM2VTS and PIE databases. Recognition performance compares favourably to contemporary approaches.     

Degradation adaptive texture classification for real-world application scenarios

Images captured under non-laboratory conditions potentially suffer from various degradations. Especially noise, blur and scale-variations are often prevalent in real world images and are known to potentially affect the classification process of textured images. We show that these degradations not necessarily strongly affect the discriminative powers of computer based classifiers in a scenario with similar degradations in training and evaluation set. We propose a degradation-adaptive classification approach, which exploits this knowledge by dividing one large data set into several smaller ones, each containing images with some kind of degradation-similarity. In a large experimental study, it can be shown that our method continuously enhances the classification accuracies in case of simulated as well as real world image degradations. Surprisingly, by means of a pre-classification, the framework turns out to be beneficial even in case of idealistic images which are free from strong degradations.     

View-based recognition of real-world textures

A new method for recognizing 3D textured surfaces is proposed. Textures are modeled with multiple histograms of micro-textons, instead of more macroscopic textons used in earlier studies. The micro-textons are extracted with the recently proposed multiresolution local binary pattern operator. Our approach has many advantages compared to the earlier approaches and provides the leading performance in the classification of Columbia-Utrecht database textures imaged under different viewpoints and illumination directions. It also provides very promising results in the classification of outdoor scene images. An approach for learning appearance models for view-based texture recognition using self-organization of feature distributions is also proposed. The method performs well in experiments. It can be used for quickly selecting model histograms and rejecting outliers, thus providing an efficient tool for vision system training even when the feature data has a large variability.     

End-to-end multimodal affect recognition in real-world environments

Automatic affect recognition in real-world environments is an important task towards a natural interaction between humans and machines. The recent years, several advancements have been accomplished in determining the emotional states with the use of Deep Neural Networks (DNNs). In this paper, we propose an emotion recognition system that utilizes the raw text, audio and visual information in an end-to-end manner. To capture the emotional states of a person, robust features need to be extracted from the various modalities. To this end, we utilize Convolutional Neural Networks (CNNs) and propose a novel transformer-based architecture for the text modality that can robustly capture the semantics of sentences. We develop an audio model to process the audio channel, and adopt a variation of a high resolution network (HRNet) to process the visual modality. To fuse the modality-specific features, we propose novel attention-based methods. To capture the temporal dynamics in the signal, we utilize Long Short-Term Memory (LSTM) networks. Our model is trained on the SEWA dataset of the AVEC 2017 research sub-challenge on emotion recognition, and produces state-of-the-art results in the text, visual and multimodal domains, and comparable performance in the audio case when compared with the winning papers of the challenge that use several hand-crafted and DNN features. Code is available at: https://github.com/glam-imperial/multimodal-affect-recognition.     

Animating expressive faces across languages

This paper describes a morphing-based audio driven facial animation system. Based on an incoming audio stream, a face image is animated with full lip synchronization and synthesized expressions. A novel scheme to implement a language independent system for audio-driven facial animation given a speech recognition system for just one language, in our case, English, is presented. The method presented here can also be used for text to audio-visual speech synthesis. Visemes in new expressions are synthesized to be able to generate animations with different facial expressions. An animation sequence using optical flow between visemes is constructed, given an incoming audio stream and still pictures of a face representing different visemes. The presented techniques give improved lip synchronization and naturalness to the animated video.     

人脸识别活体检测研究方法综述

人脸识别具有广泛的应用,但容易受到伪造的欺骗人脸攻击而影响安全性,设计检测准确率高、泛化能力强、满足实时性需求的活体检测方法是目前的研究重点。将现有的人脸活体检测研究方法分为基于手工设计特征表达、基于深度学习和基于融合策略的方法,介绍每类方法所包含的典型算法的基本思想、实现步骤及优缺点。最后对已公开的人脸活体检测数据库进行整理说明,对人脸活体检测的发展趋势以及还需要进一步解决的问题进行综述,为今后人脸活体检测的研究提供参考和借鉴。     

The analysis and recognition of real-world textures in threedimensions

The observed image texture for a rough surface has a complex dependence on the illumination and viewing angles due to effects such as foreshortening, local shading, interreflections, and the shadowing and occlusion of surface elements. We introduce the dimensionality surface as a representation for the visual complexity of a material sample. The dimensionality surface defines the number of basis textures that are required to represent the observed textures for a sample as a function of ranges of illumination and viewing angles. Basis textures are represented using multiband correlation functions that consider both within and between color band correlations. We examine properties of the dimensionality surface for real materials using the Columbia Utrecht Reflectance and Texture (CUReT) database. The analysis shows that the dependence of the dimensionality surface on ranges of illumination and viewing angles is approximately linear with a slope that depends on the complexity of the sample. We extend the analysis to consider the problem of recognizing rough surfaces in color images obtained under unknown illumination and viewing geometry. We show, using a set of 12,505 images from 61 material samples, that the information captured by the multiband correlation model allows surfaces to be recognized over a wide range of conditions. We also show that the use of color information provides significant advantages for three-dimensional texture recognition     

动态人脸识别系统的构建

提出了一种实际应用环境下的动态人脸识别系统。首先讨论了动态人脸识别系统硬件环境的搭建,然后详细介绍了动态人脸识别系统的软件流程、主要的功能模块、人脸库的构建及管理等,最后对系统进行了测试。测试结果表明,所设计的软硬件系统能够满足实际应用需求。     

Face recognition using common faces method

In this paper, we propose a face recognition method called the commonface by using the common vector approach. A face image is regarded as a summation of a common vector which represents the invariant properties of the corresponding face class, and a difference vector which presents the specific properties of the corresponding face image such as face appearance, pose and expression. Thus, by deriving the common vector of each face class, the common feature of each person is obtained which removes the differences of face images belonging to the same person. For test face image, the remaining vector with each face class is derived with the similar procedure to the common vector, which is then compared with the common vector of each face class to predict the class label of query face by finding the minimum distance between the remaining vector and the common vector. Furthermore, we extend the common vector approach (CVP) to kernel CVP to improve the performance of CVP. The experimental results suggest that the proposed commonface approach provides a better representation of individual common feature and achieves lower error rates in face recognition.     

Multibiometrics for face recognition

Fusion is a popular practice to combine multiple sources of biometric information to achieve systems with greater performance and flexibility. In this paper various approaches to fusion within a multibiometrics context are considered and an application to the fusion of 2D and 3D face information is discussed. An optimal method for fusing the accept/reject decisions of individual biometric sources by means of simple logical rules is presented. Experimental results on the FRGC 2D and 3D face data show that the proposed technique performs effectively without the need for score normalization.