A globally convergent learning algorithm for PCA neural networks

Principal component analysis (PCA) by neural networks is one of the most frequently used feature extracting methods. To process huge data sets, many learning algorithms based on neural networks for PCA have been proposed. However, traditional algorithms are not globally convergent. In this paper, a new PCA learning algorithm based on cascade recursive least square (CRLS) neural network is proposed. This algorithm can guarantee the network weight vector converges to an eigenvector associated with the largest eigenvalue of the input covariance matrix globally. A rigorous mathematical proof is given. Simulation results show the effectiveness of the algorithm.     

Image retrieval based on incremental subspace learning

Many problems in information processing involve some form of dimensionality reduction, such as face recognition, image/text retrieval, data visualization, etc. The typical linear dimensionality reduction algorithms include principal component analysis (PCA), random projection, locality-preserving projection (LPP), etc. These techniques are generally unsupervised which allows them to model data in the absence of labels or categories. In this paper, we propose a semi-supervised subspace learning algorithm for image retrieval. In relevance feedback-driven image retrieval system, the user-provided information can be used to better describe the intrinsic semantic relationships between images. Our algorithm is fundamentally based on LPP which can incorporate user's relevance feedbacks. As the user's feedbacks are accumulated, we can ultimately obtain a semantic subspace in which different semantic classes can be best separated and the retrieval performance can be enhanced. We compared our proposed algorithm to PCA and the standard LPP. Experimental results on a large collection of images have shown the effectiveness and efficiency of our proposed algorithm.     

核方法在人脸识别中的应用

1 引言人脸识别技术广泛应用于身份验证、门检系统以及人员监视等方面,在过去的几年里,人脸识别技术有了很大的发展。人脸识别技术与普通的模式识别不同,主要是因为在一般的模式识别中,有几个分类,每个分类中有很多样本,这样可以安排大量样本进行训练;相反,人脸识别中通常会有很多不同的人脸,每个人脸代表一个分类,而每个分类中的样本数都比较少,在很多情况下,甚至每个人只有一张图片(如身份证照片),在文[4]中提出了处理只有一个样本情况下的人脸识别。     

A general theory of a class of linear neural nets for principal and minor component analysis

This paper presents a unified theory of a class of learning neural nets for principal component analysis (PCA) and minor component analysis (MCA). First, some fundamental properties are addressed which all neural nets in the class have in common. Second, a subclass called the generalized asymmetric learning algorithm is investigated, and the kind of asymmetric structure which is required in general to obtain the individual eigenvectors of the correlation matrix of a data sequence is clarified. Third, focusing on a single-neuron model, a systematic way of deriving both PCA and MCA learning algorithms is shown, through which a relation between the normalization in PCA algorithms and that in MCA algorithms is revealed. This work was presented, in part, at the Third International Symposium on Artificial Life and Robotics, Oita, Japan, January 19–21, 1998     

Online eigenvector transformation reflecting concept drift for improving network intrusion detection

Currently, large data streams are constantly being generated in diverse environments, and continuous storage of the data and periodic batch-type principal component analysis (PCA) are becoming increasingly difficult. Various online PCA algorithms have been proposed to solve this problem. In this study, we propose an online PCA methodology based on online eigenvector transformation with the moving average of the data stream that can reflect concept drift. We compared the network intrusion detection performance based on online transformation of eigenvectors with that of offline methods by applying three machine learning algorithms. Both online and offline methods demonstrated excellent performance in terms of precision. However, in terms of the recall ratio, the performance of the proposed methodology with integrated online eigenvector transformation was better; thus, the F1-measure also indicated better performance. The visualization of the principal component score shows the effectiveness of our method.     

小波变换和特征加权融合的人脸识别

在人脸识别领域,提取人脸特征和降低维数是人脸识别的关键。传统的基于小波变换的人脸识别算法仅在小波分解的低频分量上提取用于分类的图像特征,造成了高频分量中部分对识别有利信息的丢失。为了更有效地提取人脸图像特征,提出一种基于小波变换和特征加权融合的人脸识别算法。首先通过小波变换对人脸图像进行降维处理,然后对4个小波子图分别运用主成分分析法(PCA)提取特征,并把这4部分特征加权融合,最后利用支持向量机(SVM)进行分类识别。在ORL人脸库上进行实验验证,识别准确率可达到97.5%,实验结果表明该算法能够有效提高人脸识别能力,与传统识别算法相比具有较高的识别准确率和识别速度。     

一种全局收敛的PCA神经网络学习算法

主元分析(PCA)也称为K-L变换是进行特征提取的一种重要方法。近年来，为了处理海量数据，许多基于Hebbian学习算法的PCA神经网络被提出来。传统的算法，通常不能保证其收敛性或者收敛速度较慢。基于CRLS神经网络，本文提出了一种新的确保权向量收敛的学习算法，本算法无须在计算中规格化权向量。同时也证明了该学习算法使得权向量收敛到最大特征值所对应的特征向量。实验表明，与传统的CRLS神经网络比较，本文算法准确性得到极大提高。     

A novel hybrid approach based on sub-pattern technique and whitened PCA for face recognition

Recently, in a task of face recognition, some researchers presented that independent component analysis (ICA) Architecture I involves a vertically centered principal component analysis (PCA) process (PCA I) and ICA Architecture II involves a whitened horizontally centered PCA process (PCA II). They also concluded that the performance of ICA strongly depends on its involved PCA process. This means that the computationally expensive ICA projection is unnecessary for further process and involved PCA process of ICA, whether PCA I or II, can be used directly for face recognition. But these approaches only consider the global information of face images. Some local information may be ignored. Therefore, in this paper, the sub-pattern technique was combined with PCA I and PCA II, respectively, for face recognition. In other words, two new different sub-pattern based whitened PCA approaches (which are called Sp-PCA I and Sp-PCA II, respectively) were performed and compared with PCA I, PCA II, PCA, and sub-pattern based PCA (SpPCA). Then, we find that sub-pattern technique is useful to PCA I but not to PCA II and PCA. Simultaneously, we also discussed what causes this result in this paper. At last, by simultaneously considering global and local information of face images, we developed a novel hybrid approach which combines PCA II and Sp-PCA I for face recognition. The experimental results reveal that the proposed novel hybrid approach has better recognition performance than that obtained using other traditional methods.     

综述人脸识别中的子空间方法

如何描述每个个体人脸的特征,使之区别于其他个体,是人脸识别研究中的关键问题 之一.近年来提出了大量的方法,其中随着主元分析在人脸识别中的成功应用之后,子空间分析 因其具有描述性强、计算代价小、易实现及可分性好的特点,受到了广泛的关注.文中结合近年 来已发表的文献,按照线性和非线性的划分,对子空间分析在人脸识别中的应用作一回顾、比较 和总结,以供其他人参考.     

基于最大相关熵准则的2维主成分分析

目的 本文针对基于最小均方差准则的主成分分析算法(如2DPCA-L₂(two-dimensional PCA with L₂-norm)算法和2DPCA-L₁(two-dimensional PCA with L₁-norm)算法)对外点敏感、识别率低的问题,结合信息论中的最大相关熵准则,提出了一种基于最大相关熵准则的2DPCA(2DPCA-MCC)。方法 2DPCA-MCC算法采用最大相关熵表示目标函数,通过半二次优化技术解决相关熵问题,降低了外点在目标函数评价中的贡献,从而提高了算法的鲁棒性和识别精度。结果 通过对比2DPCA-MCC算法和2DPCA-L₂、2DPCA-L₁在ORL人脸数据库上的识别效果,表明了2DPCA-MCC算法的识别率比2维主成分分析算法的识别率最低提高了近10%,最高提高了近30%。结论 提出了一种基于最大相关熵的2DPCA算法,通过半二次优化技术解决非线性优化问题,实验结果表明,本算法能够较好地解决外点问题,显著提高识别精度,适用于解决人脸识别中的外点问题。     

Fusion of appearance-based face recognition algorithms

Although many algorithms have been proposed, face recognition and verification systems can guarantee a good level of performances only for controlled environments. In order to improve the performance and robustness of face recognition and verification systems, multi-modal and mono-modal systems based on the fusion of multiple recognisers using different or similar biometrics have been proposed, especially for verification purposes. In this paper, a recognition and verification system based on the combination of two well-known appearance-based representations of the face, namely, principal component analysis (PCA) and linear discriminant analysis (LDA), is proposed. Both PCA and LDA are used as feature extractors from frontal view images. The benefits of such a fusion are shown for different environmental conditions, namely, ideal conditions, characterised by a very limited variability of environmental parameters, and real conditions with a large variability of lighting, scale and facial expression.     

最坏分离的联合分辨率判别分析

现实中,常需辨识低分辨率(low-resolution,简称LR)图像(如监控系统所捕捉的人脸),但相比通常的高(high-resolution,简称HR)或超(super-resolution,简称SR)分辨率图像而言,其含有相对较少的判别信息,致使通常的子空间学习算法,如结合主成分分析(principal components analysis,简称PCA)的线性判别分析(linear discriminant analysis,简称LDA)难以获得理想的识别效果.为了缓和该问题,最近所提出的联合判别分析(如SDA)借助与低分辨率相配对的高分辨率图像辅助设计LR图像分类器.在SDA的实现中,其采用了类似LDA的平均散度定义,使SDA遗传了LDA在投影时难以使相对靠近的类充分分离的问题.为了克服该不足,提出了针对LR图像识别的最坏分离的联合分辨率判别分析(worst-separated couple-resolution discriminant analysis,简称WSCR),从而使:(1) LR和HR投影到同一低维子空间;(2) 投影后的最小类间隔最大化.实验结果表明:与SDA相比,WSCR更适用于低分辨率的图像识别.     

An EDBoost algorithm towards robust face recognition in JPEG compressed domain

In this paper, we describe a novel multiclass boosting algorithm, EDBoost, to achieve robust face recognition directly in JPEG compressed domain. In comparison with existing boosting algorithms, the proposed EDBoost exploits Euclidean distance (ED) to eliminate non-effective weak classifiers in each iteration of the boosted learning, and hence improves both feature selection and classifier learning by using fewer weak classifiers and producing lower error rates. When applied to face recognition, the EDBoost algorithm is capable of selecting the most discriminative DCT features directly in JPEG compressed domain to achieve high recognition performances. In addition, a new DC replacement scheme is also proposed to reduce the effect of illumination changes. In comparison with the existing techniques, the proposed scheme achieves robust face recognition without losing the important information carried by all DC coefficients. Extensive experiments support the conclusion that the proposed algorithm outperforms all representative existing techniques in terms of boosted learning, multiclass classification, lighting effect reduction and face recognition rates.     

Weighted and robust learning of subspace representations

A reliable system for visual learning and recognition should enable a selective treatment of individual parts of input data and should successfully deal with noise and occlusions. These requirements are not satisfactorily met when visual learning is approached by appearance-based modeling of objects and scenes using the traditional PCA approach. In this paper we extend standard PCA approach to overcome these shortcomings. We first present a weighted version of PCA, which, unlike the standard approach, considers individual pixels and images selectively, depending on the corresponding weights. Then we propose a robust PCA method for obtaining a consistent subspace representation in the presence of outlying pixels in the training images. The method is based on the EM algorithm for estimation of principal subspaces in the presence of missing data. We demonstrate the efficiency of the proposed methods in a number of experiments.     

Adaptive multiple minor directions extraction in parallel using a PCA neural network

A principal component analysis (PCA) neural network is developed for online extraction of the multiple minor directions of an input signal. The neural network can extract the multiple minor directions in parallel by computing the principal directions of the transformed input signal so that the stability-speed problem of directly computing the minor directions can be avoided to a certain extent. On the other hand, the learning algorithms for updating the net weights use constant learning rates. This overcomes the shortcoming of the learning rates approaching zero. In addition, the proposed algorithms are globally convergent so that it is very simple to choose the initial values of the learning parameters. This paper presents the convergence analysis of the proposed algorithms by studying the corresponding deterministic discrete time (DDT) equations. Rigorous mathematical proof is given to prove the global convergence. The theoretical results are further confirmed via simulations.     

A class of learning algorithms for principal component analysis andminor component analysis

Principal component analysis (PCA) and minor component analysis (MCA) are a powerful methodology for a wide variety of applications such as pattern recognition and signal processing. In this paper, we first propose a differential equation for the generalized eigenvalue problem. We prove that the stable points of this differential equation are the eigenvectors corresponding to the largest eigenvalue. Based on this generalized differential equation, a class of PCA and MCA learning algorithms can be obtained. We demonstrate that many existing PCA and MCA learning algorithms are special cases of this class, and this class includes some new and simpler MCA learning algorithms. Our results show that all the learning algorithms of this class have the same order of convergence speed, and they are robust to implementation error.     

基于PCA-LDA-SVM的多普勒雷达车型识别算法

车辆检测和车型识别是智能交通系统(Intelligent transportation system,ITS)中的一个重要方面,而目标识别是低分辨率雷达领域的一个难点.该文提出一种用多普勒雷达进行车型识别的方法,把车辆建模成包含多个散射中心的目标体,散射中心与雷达的距离与频谱能量有关,因此同一目标的频谱变化反映了该目标长高等轮廓特征.然后将有效的频谱特征结合主成分分析(Principal component and analysis,PCA)和线性判别分析(Linear discriminant analysis,LDA)进行降维,再利用支持向量机(Support vector machine,SVM)等分类器实现分型.文章对不同识别算法交叉验证的实验结果进行比较,表明基于PCA-LDA-SVM的车型识别算法效果理想,有广泛的应用前景.     

一种基于人脸垂直对称性的变形2DPCA算法

本文分析了人脸的对称性和主成分分析法(PCA)、二维主成分分析法(2DPCA)的特性,证明了2DPCA协方差矩阵就是PCA协方差矩阵的主角线的平均值,同时表明2DPCA减少了对人脸识别有用的协方差信息。提出了一种基于人脸垂直对称性的变形2DPCA算法(S2DPCA),该算法最大程度地利用了协方差鉴别信息,用更少的系数表示一张人脸图像。通过在ORL的实验比较表明,该算法与PCA算法相比降低了计算复杂性,与2DPCA方法和PCA方法相比提高了人脸识别率,在识别率方面优于传统算法(PCA(Eigenfaces)、ICA、Kernel Eigenfaces),同时也压缩了人脸的存储空间。     

一种融合小波变化和张量PCA的人耳识别的算法*

小波变换具有良好的时频分析特性,而且具有较快的算法特点,同时还能起到降维的作用。张量主成分分析方法用于人耳识别能获得比PCA方法更高的识别率。综合利用这两个算法的优点,提出了一种新的人耳识别方法,对人耳图像先采用小波变换做预处理得到4个子带图像,然后对每个子带图像用张量PCA进行特征提取,最后利用最近邻的方法实现人耳图像识别。实验结果表明,利用此方法与只用主成分分析识别相比,提高了识别率。     

局部切空间对齐算法的核主成分分析解释

基于核方法的降维技术和流形学习是两类有效而广泛应用的非线性降维技术,它们有着各自不同的出发点和理论基础,在以往的研究中很少有研究关注两者的联系。LTSA算法利用数据的局部结构构造一种特殊的核矩阵,然后利用该核矩阵进行核主成分分析。本文针对局部切空间对齐这种流形学习算法,重点研究了LTSA算法与核PCA的内在联系。研究表明,LTSA在本质上是一种基于核方法的主成分分析技术。