Incremental learning of bidirectional principal components for face recognition

Recently, bidirectional principal component analysis (BDPCA) has been proven to be an efficient tool for pattern recognition and image analysis. Encouraging experimental results have been reported and discussed in the literature. However, BDPCA has to be performed in batch mode, it means that all the training data has to be ready before we calculate the projection matrices. If there are additional samples need to be incorporated into an existing system, it has to be retrained with the whole updated training set. Moreover, the scatter matrices of BDPCA are formulated as the sum of K (samples size) image covariance matrices, this leads to the incremental learning directly on the scatters impossible, thus it presents new challenge for on-line training.In fact, there are two major reasons for building incremental algorithms. The first reason is that in some cases, when the number of training images is very large, the batch algorithm cannot process the entire training set due to large computational or space requirements of the batch approach. The second reason is when the learning algorithm is supposed to operate in a dynamical settings, that all the training data is not given in advance, and new training samples may arrive at any time, and they have to be processed in an on-line manner. Through matricizations of third-order tensor, we successfully transfer the eigenvalue decomposition problem of scatters to the singular value decomposition (SVD) of corresponding unfolded matrices, followed by complexity and memory analysis on the novel algorithm. A theoretical clue for selecting suitable dimensionality parameters without losing classification information is also presented in this paper. Experimental results on FERET and CMU PIE (pose, illumination, and expression) databases show that the IBDPCA algorithm gives a close approximation to the BDPCA method, but using less time.     

Multiscale facial structure representation for face recognition under varying illumination

Facial structure of face image under lighting lies in multiscale space. In order to detect and eliminate illumination effect, a wavelet-based face recognition method is proposed in this paper. In this work, the effect of illuminations is effectively reduced by wavelet-based denoising techniques, and meanwhile the multiscale facial structure is generated. Among others, the proposed method has the following advantages: (1) it can be directly applied to single face image, without any prior information of 3D shape or light sources, nor many training samples; (2) due to the multiscale nature of wavelet transform, it has better edge-preserving ability in low frequency illumination fields; and (3) the parameter selection process is computationally feasible and fast. Experiments are carried out upon the Yale B and CMU PIE face databases, and the results demonstrate that the proposed method achieves satisfactory recognition rates under varying illumination conditions.     

解纠缠表示学习在跨年龄人脸识别中的应用

跨年龄人脸识别因其在现实生活中的广泛应用而成为人脸识别领域的热门话题.针对跨年龄人脸识别精度较低的问题,引入解纠缠表示学习,提出了一个基于生成对抗网络的解纠缠表示学习(IPDRL)网络来实现人脸图像的识别.该网络由编码器、生成器和鉴别器构成.编码器在对特征中的年龄变化进行解纠缠的同时,对人脸图像的身份信息进行编码,提取只利于身份鉴别的特征,实现身份特征和年龄特征的解纠缠;生成器根据输入的年龄特征生成对应的身份保持的年龄图像;鉴别器通过对抗学习和多任务学习实现年龄和身份的类分布预测.通过将解纠缠表示学习、对抗学习和多任务学习相结合的方法,很好地保留了人脸图像的身份信息,并使跨年龄人脸图像识别的精度得到了提高.     

基于自步学习的加权稀疏表示人脸识别方法

近年来基于稀疏表示的分类方法（SRC）成为了一个新的热点问题,在人脸识别领域取得了很大的成功。但基于稀疏表示的方法在重建待测样本时,有可能会利用与待测样本相差较大的训练样本,并且没有考虑到表示系数的局部信息,从而导致分类结果不稳定。提出一种基于自步学习的加权稀疏表示算法SPL-WSRC,在字典中有效剔除与待测样本相差较大的训练样本,并利用加权手段考虑样本间的局部信息,以提高分类精度和稳定性。通过3个典型的人脸数据集中的实验,实验结果表明,所提算法优于原稀疏表示算法SRC,特别是当训练样本足够多时,效果更明显。     

Deep learning face representation by fixed erasing in facial landmarks

Multimedia Tools and Applications - Face verification (FV) is a challenging problem, because occlusion, posture, illumination, aging will affect the accuracy of FV. Deep convolutional neural...     

2D representation of facial surfaces for multi-pose 3D face recognition

The increasing availability of 3D facial data offers the potential to overcome the intrinsic difficulties faced by conventional face recognition using 2D images. Instead of extending 2D recognition algorithms for 3D purpose, this letter proposes a novel strategy for 3D face recognition from the perspective of representing each 3D facial surface with a 2D attribute image and taking the advantage of the advances in 2D face recognition. In our approach, each 3D facial surface is mapped homeomorphically onto a 2D lattice, where the value at each site is an attribute that represents the local 3D geometrical or textural properties on the surface, therefore invariant to pose changes. This lattice is then interpolated to generate a 2D attribute image. 3D face recognition can be achieved by applying the traditional 2D face recognition techniques to obtained attribute images. In this study, we chose the pose invariant local mean curvature calculated at each vertex on the 3D facial surface to construct the 2D attribute image and adopted the eigenface algorithm for attribute image recognition. We compared our approach to state-of-the-art 3D face recognition algorithms in the FRGC (Version 2.0), GavabDB and NPU3D database. Our results show that the proposed approach has improved the robustness to head pose variation and can produce more accurate 3D multi-pose face recognition.     

Partially-supervised learning from facial trajectories for face recognition in video surveillance

Face recognition (FR) is employed in several video surveillance applications to determine if facial regions captured over a network of cameras correspond to a target individuals. To enroll target individuals, it is often costly or unfeasible to capture enough high quality reference facial samples a priori to design representative facial models. Furthermore, changes in capture conditions and physiology contribute to a growing divergence between these models and faces captured during operations. Adaptive biometrics seek to maintain a high level of performance by updating facial models over time using operational data. Adaptive multiple classifier systems (MCSs) have been successfully applied to video-to-video FR, where the face of each target individual is modeled using an ensemble of 2-class classifiers (trained using target vs. non-target samples). In this paper, a new adaptive MCS is proposed for partially-supervised learning of facial models over time based on facial trajectories. During operations, information from a face tracker and individual-specific ensembles is integrated for robust spatio-temporal recognition and for self-update of facial models. The tracker defines a facial trajectory for each individual that appears in a video, which leads to the recognition of a target individual if the positive predictions accumulated along a trajectory surpass a detection threshold for an ensemble. When the number of positive ensemble predictions surpasses a higher update threshold, then all target face samples from the trajectory are combined with non-target samples (selected from the cohort and universal models) to update the corresponding facial model. A learn-and-combine strategy is employed to avoid knowledge corruption during self-update of ensembles. In addition, a memory management strategy based on Kullback–Leibler divergence is proposed to rank and select the most relevant target and non-target reference samples to be stored in memory as the ensembles evolves. For proof-of-concept, a particular realization of the proposed system was validated with videos from Face in Action dataset. Initially, trajectories captured from enrollment videos are used for supervised learning of ensembles, and then videos from various operational sessions are presented to the system for FR and self-update with high-confidence trajectories. At a transaction level, the proposed approach outperforms baseline systems that do not adapt to new trajectories, and provides comparable performance to ideal systems that adapt to all relevant target trajectories, through supervised learning. Subject-level analysis reveals the existence of individuals for which self-updating ensembles with unlabeled facial trajectories provides a considerable benefit. Trajectory-level analysis indicates that the proposed system allows for robust spatio-temporal video-to-video FR, and may therefore enhance security and situation analysis in video surveillance.     

Hybrid-boost learning for multi-pose face detection and facial expression recognition

This paper proposes a hybrid-boost learning algorithm for multi-pose face detection and facial expression recognition. To speed-up the detection process, the system searches the entire frame for the potential face regions by using skin color detection and segmentation. Then it scans the skin color segments of the image and applies the weak classifiers along with the strong classifier for face detection and expression classification. This system detects human face in different scales, various poses, different expressions, partial-occlusion, and defocus. Our major contribution is proposing the weak hybrid classifiers selection based on the Harr-like (local) features and Gabor (global) features. The multi-pose face detection algorithm can also be modified for facial expression recognition. The experimental results show that our face detection system and facial expression recognition system have better performance than the other classifiers.     

Incremental complete LDA for face recognition

The complete linear discriminant analysis (CLDA) algorithm has been proven to be an effective tool for face recognition. The CLDA method can make full use of the discriminant information of the training samples. However, the original implementation of CLDA may not suitable for incremental learning problem. In this paper, we first propose a new implementation of CLDA, which is theoretically equivalent to the original implementation of CLDA but is more efficient than the original one. Then, based on our proposed novel implementation of CLDA, we propose the incremental CLDA method which can accurately update the discriminant vectors of CLDA when new samples are inserted into the training set. Experiments on ORL, AR and PIE face databases show the efficiency of our proposed CLDA algorithms over the original implementation of CLDA.     

基于分块非负矩阵分解人脸识别增量学习*

非负矩阵分解(NMF)算法可以提取图像的局部特征,然而NMF算法有两个主要缺点：a）当矩阵维数较大时,NMF算法非常耗时;b）当增加新的训练样本或类别时,NMF算法必须进行重复学习。为克服NMF算法这些缺点,提出了一种新的分块NMF算法(BNMF)。特别地,该方法还可用于增量学习。通过在FERET和CMU PIE人脸数据库上进行实验,结果表明该算法均优于NMF和PCA算法。     

Bimode model for face recognition and face representation

Tensorface based approaches decompose an image into its constituent factors (i.e., person, lighting, viewpoint, etc.), and then utilize these factor spaces for recognition. However, tensorface is not a preferable choice, because of the complexity of its multimode. In addition, a single mode space, except the person-space, could not be used for recognition directly. From the viewpoint of practical application, we propose a bimode model for face recognition and face representation. This new model can be treated as a simplified model representation of tensorface. However, their respective algorithms for training are completely different, due to their different definitions of subspaces. Thanks to its simpler model form, the proposed model requires less iteration times in the process of training and testing. Moreover bimode model can be further applied to an image reconstruction and image synthesis via an example image. Comprehensive experiments on three face image databases (PEAL, YaleB frontal and Weizmann) validate the effectiveness of the proposed new model.     

Discriminative sparse representation for face recognition

Recently Sparse Representation (or coding) based Classification (SRC) has gained great success in face recognition. In SRC, the testing image is expected to be best represented as a sparse linear combination of training images from the same class, and the representation fidelity is measured by the ?₂-norm or ?₁-norm of the coding residual. However, SRC emphasizes the sparsity too much and overlooks the spatial information during local feature encoding process which has been demonstrated to be critical in real-world face recognition problems. Besides, some work considers the spatial information but overlooks the different discriminative ability in different face regions. In this paper, we propose to weight spatial locations based on their discriminative abilities in sparse coding for robust face recognition. Specifically, we learn the weights at face locations according to the information entropy in each face region, so as to highlight locations in face images that are important for classification. Furthermore, in order to construct a robust weights to fully exploit structure information of each face region, we employed external data to learn the weights, which can cover all possible face image variants of different persons, so the robustness of obtained weights can be guaranteed. Finally, we consider the group structure of training images (i.e. those from the same subject) and added an ?_2,1-norm (group Lasso) constraint upon the formulation, which enforcing the sparsity at the group level. Extensive experiments on three benchmark face datasets demonstrate that our proposed method is much more robust and effective than baseline methods in dealing with face occlusion, corruption, lighting and expression changes, etc.     

Deep representation learning for face hallucination

Multimedia Tools and Applications - Recently, deep learning, as a novel emerging algorithm, offers an end-to-end effective paradigm for super-resolution. Various successful practices with the deep...     

一种基于优化“词袋”模型的物体识别方法*

针对传统基于“词袋”模型物体识别现有方法的不足,对现特征表达、视觉词典和图像表示方法进行优化,以提高物体识别正确率。采用HUE直方图与SIFT特征描述符分别描述兴趣点周围的颜色和形状特征,实现“词袋”模型下两种特征的特征级和图像级融合,引入K-means++聚类算法生成视觉词典,并利用软权重思想将特征向量映射到视觉单词形成图像直方图。实验结果表明,所述方法会产生较高的物体识别正确率,且识别结果不受两种特征融合权重的影响。     

Incremental template updating for face recognition in home environments

The extreme variability of faces in smart environment applications, due to continuous changes in terms of pose, illumination and subject appearance (hairstyle, make-up, etc.), requires the relevant mode of variations of the subject's faces to be encoded in the templates and to be continuously updated based on new inputs. This work proposes a new video-based template updating approach suitable for home environments where the image acquisition process is totally unconstrained but a large amount of face data is available for continuous learning. A small set of labeled images is initially used to create the templates and the updating is then totally unsupervised. Although the method is here presented in conjunction with a subspace-based face recognition approach, it can be easily adapted to deal with different kinds of face representations. A thorough performance evaluation is carried out to show the efficacy and reliability of the proposed technique.     

Incremental face recognition for large-scale social network services

Due to the rapid growth of social network services such as Facebook and Twitter, incorporation of face recognition in these large-scale web services is attracting much attention in both academia and industry. The major problem in such applications is to deal efficiently with the growing number of samples as well as local appearance variations caused by diverse environments for the millions of users over time. In this paper, we focus on developing an incremental face recognition method for Twitter application. Particularly, a data-independent feature extraction method is proposed via binarization of a Gabor filter. Subsequently, the dimension of our Gabor representation is reduced considering various orientations at different grid positions. Finally, an incremental neural network is applied to learn the reduced Gabor features. We apply our method to a novel application which notifies new photograph uploading to related users without having their ID being identified. Our extensive experiments show that the proposed algorithm significantly outperforms several incremental face recognition methods with a dramatic reduction in computational speed. This shows the suitability of the proposed method for a large-scale web service with millions of users.     

Incremental linear discriminant analysis for face recognition.

Dimensionality reduction methods have been successfully employed for face recognition. Among the various dimensionality reduction algorithms, linear (Fisher) discriminant analysis (LDA) is one of the popular supervised dimensionality reduction methods, and many LDA-based face recognition algorithms/systems have been reported in the last decade. However, the LDA-based face recognition systems suffer from the scalability problem. To overcome this limitation, an incremental approach is a natural solution. The main difficulty in developing the incremental LDA (ILDA) is to handle the inverse of the within-class scatter matrix. In this paper, based on the generalized singular value decomposition LDA (LDA/GSVD), we develop a new ILDA algorithm called GSVD-ILDA. Different from the existing techniques in which the new projection matrix is found in a restricted subspace, the proposed GSVD-ILDA determines the projection matrix in full space. Extensive experiments are performed to compare the proposed GSVD-ILDA with the LDA/GSVD as well as the existing ILDA methods using the face recognition technology face database and the Carneggie Mellon University Pose, Illumination, and Expression face database. Experimental results show that the proposed GSVD-ILDA algorithm gives the same performance as the LDA/GSVD with much smaller computational complexity. The experimental results also show that the proposed GSVD-ILDA gives better classification performance than the other recently proposed ILDA algorithms.     

Bidirectional representation for face recognition across pose

Conventional representation methods try to express the test sample as a weighting sum of training samples and exploit the deviation between the test sample and the weighting sum of the training samples from each class (also referred to as deviation between the test sample and each class) to classify the test sample. In particular, the methods assign the test sample to the class that has the smallest deviation among all the classes. This paper analyzes the relationship between face images under different poses and, for the first time, devises a bidirectional representation method-based pattern classification (BRBPC) method for face recognition across pose. BRBPC includes the following three steps: the first step uses the procedure of conventional representation methods to express the test sample and calculates the deviation between the test sample and each class. The second step first expresses the training sample of a class as a weighting sum of the test sample and the training samples from all the other classes and then obtains the corresponding deviation (referred to as complementary deviation). The third step uses the score-level fusion to integrate the scores, that is, deviations generated from the first and second steps for final classification. The experimental results show that BRBPC classifies more accurately than conventional representation methods.     

Multi-resolution dictionary collaborative representation for face recognition

Pattern Analysis and Applications - In this paper, a multi-resolution dictionary collaborative representation(MRDCR) method for face recognition is proposed. Unlike most of the traditional sparse...