Sparse data-dependent kernel principal component analysis based on least squares support vector machine for feature extraction and recognition

Kernel learning is widely used in many areas, and many methods are developed. As a famous kernel learning method, kernel principal component analysis (KPCA) endures two problems in the practical applications. One is that all training samples need to be stored for the computing the kernel matrix during kernel learning. Second is that the kernel and its parameter have the heavy influence on the performance of kernel learning. In order to solve the above problem, we present a novel kernel learning namely sparse data-dependent kernel principal component analysis through reducing the training samples with sparse learning-based least squares support vector machine and adaptive self-optimizing kernel structure according to the input training samples. Experimental results on UCI datasets, ORL and YALE face databases, and Wisconsin Breast Cancer database show that it is feasible to improve KPCA on saving consuming space and optimizing kernel structure.     

A novel neural-network approach of analog fault diagnosis based on kernel discriminant analysis and particle swarm optimization

Kernel principal component analysis (KPCA) and kernel linear discriminant analysis (KLDA) are two commonly used and effective methods for dimensionality reduction and feature extraction. In this paper, we propose a KLDA method based on maximal class separability for extracting the optimal features of analog fault data sets, where the proposed KLDA method is compared with principal component analysis (PCA), linear discriminant analysis (LDA) and KPCA methods. Meanwhile, a novel particle swarm optimization (PSO) based algorithm is developed to tune parameters and structures of neural networks jointly. Our study shows that KLDA is overall superior to PCA, LDA and KPCA in feature extraction performance and the proposed PSO-based algorithm has the properties of convenience of implementation and better training performance than Back-propagation algorithm. The simulation results demonstrate the effectiveness of these methods.     

Identity management based on PCA and SVM

A new approach for face recognition, based on kernel principal component analysis (KPCA) and support vector machines (SVMs), is presented to improve the recognition performance of the method based on principal component analysis (PCA). This method can simultaneously be applied to solve both the over-fitting problem and the small sample problem. The KPCA method is performed on every facial image of the training set to get the core facial features of the training samples. To ensure that the loss of the image information will be as less as possible, the facial data of high-dimensional feature space is projected into low-dimensional space, and then the SVM face recognition model is established to identify the low-dimensional space facial data. Our experimental results demonstrate that the approach proposed in this paper is efficient, and the recognition accuracy of the proposed method reaches 95.4 %.     

基于小波特征的快速核主分量分析技术

论文提出了基于小波特征的核主分量分析技术,即在进行非线性映射之前,首先利用小波变换对原始输入训练样本进行预处理,获取低频平滑、水平细节和垂直细节等三个子图的小波特征,然后在频域上,对它们分别进行核主分量分析(KPCA),对最终获得的3组特征向量设计了一种特征融合的方法。在ORL标准人脸库上的试验结果表明所提方法不仅在识别性能上优于现有的核主分量分析方法,而且,特征抽取速度提高了11倍。     

Principal manifolds and probabilistic subspaces for visualrecognition

Investigates the use of linear and nonlinear principal manifolds for learning low-dimensional representations for visual recognition. Several leading techniques - principal component analysis (PCA), independent component analysis (ICA) and nonlinear kernel PCA (KPCA) - are examined and tested in a visual recognition experiment using 1,800+ facial images from the "FERET" (FacE REcognition Technology) database. We compare the recognition performance of nearest-neighbor matching with each principal manifold representation to that of a maximum a-posteriori (MAP) matching rule using a Bayesian similarity measure derived from dual probabilistic subspaces. The experimental results demonstrate the simplicity, computational economy and performance superiority of the Bayesian subspace method over principal manifold techniques for visual matching     

基于传感器模式噪声特性的图像篡改检测方法

针对传统基于传感器模式噪声特性的图像篡改检测算法由于需要知道参考图像数据库因而应用局限性大的问题,提出了一种基于噪声子空间投影的图像篡改检测框架,分别采用主成分分析（ PCA）、二维主成分分析（2DPCA）和核主成分分析（KPCA）实现了基于图像噪声特性的篡改检测,并通过实验验证了此方法的有效性。     

改进的基于Gabor-LDA的人脸识别方法

鉴于Gabor特征对光照、表情等变化比较鲁棒,并已在人脸识别领域取得成功应用,提出了一种改进的Gabor-LDA算法.首先对人脸图像进行多方向、多尺度Gabor小渡滤波,然后对得到的特征向量使用改进的主成分分析方法(PCA)变换降维,采用自适应加权原理重建类内散布矩阵和类间散布矩阵,从而改进了最佳鉴别分析(LDA)判别函数,有效地解决了训练样本类均值与类中心的偏离问题.对Yale人脸库的数值试验表明,该算法比传统算法有更好的性能.     

Kernel ICA: An alternative formulation and its application to face recognition

This paper formulates independent component analysis (ICA) in the kernel-inducing feature space and develops a two-phase kernel ICA algorithm: whitened kernel principal component analysis (KPCA) plus ICA. KPCA spheres data and makes the data structure become as linearly separable as possible by virtue of an implicit nonlinear mapping determined by kernel. ICA seeks the projection directions in the KPCA whitened space, making the distribution of the projected data as non-gaussian as possible. The experiment using a subset of FERET database indicates that the proposed kernel ICA method significantly outperform ICA, PCA and KPCA in terms of the total recognition rate.     

融合小波变换与KPCA的分块人脸特征抽取与识别算法

鉴于小波多尺度变换对高维图像特征具有良好的压缩和表达能力,提出了一种融合小波变换与KPCA(核主成分分析)方法的分块人脸特征抽取与识别算法。该算法首先对人脸图像进行分块小波变换,再根据图像块的位置分布选取不同的频率分量;然后对此分量进行KPCA特征抽取,并通过对抽取到的特征进行融合来得到最终人脸鉴别特征;最后利用支持向量机分类器进行特征分类与识别。通过对ORL和Yale标准人脸图像库的实验仿真结果表明,该算法不仅在识别性能和分类速度上明显高于传统的PCA方法及融合小波特征的KPCA方法,而且对于人脸光照、姿态和表情变化均具有良好的鲁棒性。     

基于测地距离的KPCA人脸识别

针对人脸检测数据集中的信息均为高维特征向量且人脸识别易受表情变化影响等问题,本文提出一种基于测地距离的KPCA人脸识别方法,该方法利用非线性方法提取主成分。先采用KPCA方法把人脸数据映射到高维空间,进而在高维空间中提取人脸的主成分,其中核函数为多项式核函数;然后引入测地距离替换原来的欧氏距离进行相似度量,其能更准确地测量出两像素点间的实际距离,使得人脸识别率受表情变化影响小。该方法不但可以实现降维,而且还能达到有效提取特征的目的。在ORL人脸库上的实验结果表明,该方法的识别率明显优于PCA、KPCA等方法的识别率。     

基于分块核主元分析的人脸识别方法研究

传统的基于数据二阶统计矩的主元分析法（PCA）是一种有效的数据特征提取方法,是基于原始特征的一种线性变换。但是,当原始数据中存在非线性属性时,用主元分析法后留下的显著成分就可能不再反映这种非线性属性。而核主元分析则是基于原始数据的高阶统计量,是一种非线性变换,在图像识别中它可以描述多个像素之间的相关性。而KPCA方法只考虑了人脸图像的整体信息,没有考虑到局部特征信息。文章提出了分块核主元分析（MKPCA）的方法进行人脸识别,取得了很好的效果。     

A Fast Feature-based Dimension Reduction Algorithm for Kernel Classifiers

This paper presents a novel dimension reduction algorithm for kernel based classification. In the feature space, the proposed algorithm maximizes the ratio of the squared between-class distance and the sum of the within-class variances of the training samples for a given reduced dimension. This algorithm has lower complexity than the recently reported kernel dimension reduction (KDR) for supervised learning. We conducted several simulations with large training datasets, which demonstrate that the proposed algorithm has similar performance or is marginally better compared with KDR whilst having the advantage of computational efficiency. Further, we applied the proposed dimension reduction algorithm to face recognition in which the number of training samples is very small. This proposed face recognition approach based on the new algorithm outperforms the eigenface approach based on the principal component analysis (PCA), when the training data is complete, that is, representative of the whole dataset.     

Learning Linear and Nonlinear PCA with Linear Programming

An SVM-like framework provides a novel way to learn linear principal component analysis (PCA). Actually it is a weighted PCA and leads to a semi-definite optimization problem (SDP). In this paper, we learn linear and nonlinear PCA with linear programming problems, which are easy to be solved and can obtain the unique global solution. Moreover, two algorithms for learning linear and nonlinear PCA are constructed, and all principal components can be obtained. To verify the performance of the proposed method, a series of experiments on artificial datasets and UCI benchmark datasets are accomplished. Simulation results demonstrate that the proposed method can compete with or outperform the standard PCA and kernel PCA (KPCA) in generalization ability but with much less memory and time consuming.     

基于PCA与主动学习AdaBoost的频谱感知算法

针对认知网络中各低信噪比环境下主用户信号检测率偏低的问题,提出一种基于主成分分析和主动学习AdaBoost的主用户信号频谱感知算法.该算法首先采用主成分分析算法对信号特征参数进行提取,获得信号的主成分,之后利用主动学习算法通过多次迭代抽样,获取有利于提高分类性能的样本,并对AdaBoost分类器进行训练,最后利用训练完成的AdaBoost分类器对待测信号进行分类检测.仿真实验表明,在各低信噪比情况下与ANN和MME算法相比较,所提算法具有较高的分类感知性能,有效地实现了对主用户信号的频谱感知.     

Color face recognition based on quaternion matrix representation

There are several methods to recognize and reconstruct a human face image. The principal component analysis (PCA) is a successful approach because of its effective extraction of the global feature and excellent reconstruction of face image. However, the crucial shortcomings of PCA are its low recognition rate and overfitting of feature extraction which leads to the dependence of training data on training samples. In this paper, a modified two-dimension principal component analysis (2DPCA) and bidirectional principal component analysis (BDPCA) methods based on quaternion matrix are proposed to recognize and reconstruct a color face image. In these methods, the spatial distribution information of color images is used to represent a color face, and the 2DPCA or BDPCA feature of color face image is extracted by reducing the dimensionality in both column and row directions. A method obtaining orthogonal eigenvector set of quaternion matrix is proposed. Numerous experiments show that the present approach based on quaternion matrix can effectively smooth the overfitting issue and substantially enhance the recognition rate.     

基于Laplacian正则化最小二乘的半监督SAR目标识别

提出了一种基于核主成分分析(kernel principal component analysis,简称KPCA)和拉普拉斯正则化最小二乘(Laplacian regularized least squares,简称LapRLS)的合成孔径雷达(synthetic aperture radar,简称SAR)目标识别方法.KPCA特征提取方法不仅能够提取目标主要特征,而且有效地降低了特征维数.Laplacian正则化最小二乘分类是一种半监督学习方法,将训练集样本作为有标识样本,测试集样本作为无标识样本,在学习过程中将测试集样本包含进来以获得更高的识别率.在MSTAR实测SAR地面目标数据上进行实验,结果表明,该方法具有较高的识别率,并对目标角度间隔具有鲁棒性.与模板匹配法、支撑矢量机以及正则化最小二乘监督学习方法相比,具有更高的SAR目标识别正确率.此外,还通过实验分析了不同情况下有标识样本数目对目标识别性能的影响.     

全局与局部判别信息融合的转子故障数据集降维方法研究

针对传统的数据降维方法无法兼顾保持全局特征信息与局部判别信息的问题,提出一种核主元分析（Kernel principal component analysis,KPCA）和正交化局部敏感判别分析（Orthogonal locality sensitive discriminant analysis,OLSDA）相结合的转子故障数据集降维方法.该方法首先利用KPCA算法有效降低数据集的相关性、消除冗余属性,由此实现了最大程度地保留原始数据全局非线性信息的作用;然后利用OLSDA算法充分挖掘出数据的局部流形结构信息,达到了提取出具有高判别力低维本质特征的目的.上述方法的特点是通过同时进行的正交化处理可避免局部子空间结构发生失真,采用三维图直观显示出低维结果,以低维特征子集输入最近邻分类器（K-nearest neighbor,KNN）的识别率和聚类分析之类间距Sb、类内距Sw作为衡量降维效果的指标.实验表明该方法能够全面地提取出全局与局部判别信息,使故障分类更清晰,相应地识别准确率得到了明显提升.该研究可为解决高维和非线性机械故障数据集的可视化与分类问题,提供理论参考依据.     

Weighted principal component extraction with genetic algorithms

Pattern recognition techniques have been widely used in a variety of scientific disciplines including computer vision, artificial intelligence, biology, and so forth. Although many methods present satisfactory performances, they still have several weak points, thus leaving a lot of space for further improvements. In this paper, we propose two performance-driven subspace learning methods by extending the principal component analysis (PCA) and the kernel PCA (KPCA). Both methods adopt a common structure where genetic algorithms are employed to pursue optimal subspaces. Because the proposed feature extractors aim at achieving high classification accuracy, enhanced generalization ability can be expected. Extensive experiments are designed to evaluate the effectiveness of the proposed algorithms in real-world problems including object recognition and a number of machine learning tasks. Comparative studies with other state-of-the-art techniques show that the methods in this paper are capable of enhancing generalization ability for pattern recognition systems.     

基于半监督LDA特征子空间优化的人脸识别算法

人脸特征提取是人脸识别流程最重要的步骤,特征的好坏直接影响了识别效果。为了得到更好的人脸识别效果,需要充分利用样本的信息。为了充分利用训练样本和测试样本包含的信息,提出了利用样本散度矩阵将主成分分析PCA算法和线性判别分析LDA算法加权组合的半监督LDA（SLDA）特征提取算法。同时,受组合优化问题的启发,利用二进制遗传算法对半监督特征提取算法得到的特征空间进行优化。在ORL人脸数据库上的实验结果表明：与人脸识别经典算法和部分改进算法相比,SLDA算法获得了更高的识别率。