A novel multiset integrated canonical correlation analysis framework and its application in feature fusion

Multiset canonical correlation analysis (MCCA) is difficult to effectively express the integrated correlation among multiple feature vectors in feature fusion. Thus, this paper firstly presents a novel multiset integrated canonical correlation analysis (MICCA) framework. The MICCA establishes a discriminant correlation criterion function of multi-group variables based on generalized correlation coefficient. The criterion function can clearly depict the integrated correlation among multiple feature vectors. Then the paper presents a multiple feature fusion theory and algorithm using the MICCA method. The detailed process of the algorithm is as follows: firstly, extract multiple feature vectors from the same patterns by using different feature extraction methods; then extract multiset integrated canonical correlation features using MICCA; finally form effective discriminant feature vectors through two given feature fusion strategies for pattern classification. The multi-group feature fusion method based on MICCA not only achieves the aim of feature fusion, but also removes the redundancy between features. The experiment results on CENPARMI handwritten Arabic numerals and UCI multiple features database show that the MICCA method has better recognition rates and robustness than the fusion methods based on canonical correlation analysis (CCA) and MCCA.     

MRCCA: A novel CCA based method and its application in feature extraction and fusion for matrix data

Multiset features extracted from the same pattern usually represent different characteristics of data, meanwhile, matrices or 2-order tensors are common forms of data in real applications. Hence, how to extract multiset features from matrix data is an important research topic for pattern recognition. In this paper, by analyzing the relationship between CCA and 2D-CCA, a novel feature extraction method called multiple rank canonical correlation analysis (MRCCA) is proposed, which is an extension of 2D-CCA. Different from CCA and 2D-CCA, in MRCCA k pairs left transforms and k pairs right transforms are sought to maximize correlation. Besides, the multiset version of MRCCA termed as multiple rank multiset canonical correlation analysis (MRMCCA) is also developed. Experimental results on five real-world data sets demonstrate the viability of the formulation, they also show that the recognition rate of our method is higher than other methods and the computing time is competitive.     

Fractional-order embedding canonical correlation analysis and its applications to multi-view dimensionality reduction and recognition

Due to the noise disturbance and limited number of training samples, within-set and between-set sample covariance matrices in canonical correlation analysis (CCA) usually deviate from the true ones. In this paper, we re-estimate within-set and between-set covariance matrices to reduce the negative effect of this deviation. Specifically, we use the idea of fractional order to respectively correct the eigenvalues and singular values in the corresponding sample covariance matrices, and then construct fractional-order within-set and between-set scatter matrices which can obviously alleviate the problem of the deviation. On this basis, a new approach is proposed to reduce the dimensionality of multi-view data for classification tasks, called fractional-order embedding canonical correlation analysis (FECCA). The proposed method is evaluated on various handwritten numeral, face and object recognition problems. Extensive experimental results on the CENPARMI, UCI, AT&T, AR, and COIL-20 databases show that FECCA is very effective and obviously outperforms the existing joint dimensionality reduction or feature extraction methods in terms of classification accuracy. Moreover, its improvements for recognition rates are statistically significant on most cases below the significance level 0.05.     

基于集成学习的多重集典型相关分析方法

特征提取是模式识别中的关键问题之一,对提高系统分类性能具有重要意义。常用的特征提取方法包括主成分分析、线性鉴别分析、典型相关分析等等,多重集典型相关分析是基于传统的典型相关分析基础上发展而来,利用多组（大于2）特征数据集进行特征提取。基于集成学习的多重集典型相关分析的方法（EMCCA）,是通过将样本化分成若干小的样本,形成若干个特征数据集,利用多重集典型相关分析对这组数据集做特征提取,并结合集成学习对样本进行分类。在UCI上的多特征手写体数据集上的实验结果表明：相比于传统的PCA,CCA特征提取方法,多重集典型相关分析具有更优的特征提取效果,结合集成学习后具有更好的分类效果。     

Fast regularized canonical correlation analysis

Canonical correlation analysis is a popular statistical method for the study of the correlations between two sets of variables. Finding the canonical correlations between these datasets requires the inversion of their corresponding sample correlation matrices. When the number of variables is large compared to the number of experimental units it is impossible to calculate the inverse of these matrices directly and therefore it is necessary to add a multiple of the identity matrix to them. This procedure is known as regularization. In this paper we present an alternative method to the existing regularization algorithm. The proposed method is based on the estimates of the correlation matrices which minimize the mean squared error risk function. The solution of this optimization problem can be found analytically and consists of a small set of computationally inexpensive equations. We also present material which shows that the proposed method is more stable and provides more accurate results than the standard regularized canonical correlation method. Finally, the application of our original method to NCI-60 microRNA cancer data proves that it can deliver useful insights in study cases which involve hundreds of variables.     

图嵌入投影非负矩阵分解图像特征提取方法

针对投影非负矩阵分解(PNMF)不能揭示数据流形几何结构和判别信息的问题,提出了一种图嵌入投影非负矩阵分解(GEPNMF)特征提取方法。首先分别构建了描述数据流形几何结构和类间分离度的近邻图,然后采用它们的拉普拉斯矩阵设计了一个图嵌入正则项,并将其与PNMF的目标函数融合,以构造GEPNMF的目标函数。在GEPNMF目标函数中引入了图嵌入正则项,使求得的子空间能够在保持数据流形几何结构的同时,类间间距也最大。另外,还在目标函数中引入了一个正交正则项,以确保GEPNMF子空间基向量具有数据局部表示能力。对求解GEPNMF目标函数的累乘更新规则(MUR)进行了详细的推导。在Yale和CMU PIE人脸数据库上的实验结果表明,提出的图嵌入投影非负矩阵分解特征提取方法比PNMF更适用于解决分类问题。     

Concurrent quality and process monitoring with canonical correlation analysis

Canonical correlation analysis (CCA) is a well-known data analysis technique that extracts multidimensional correlation structure between two sets of variables. CCA focuses on maximizing the correlation between quality and process data, which leads to the efficient use of latent dimensions. However, CCA does not focus on exploiting the variance or the magnitude of variations in the data, making it rarely used for quality and process monitoring. In addition, it suffers from collinearity problems that often exist in the process data. To overcome this shortcoming of CCA, a modified CCA method with regularization is developed to extract correlation between process variables and quality variables. Next, to handle the issue that CCA focuses only on correlation but ignores variance information, a new concurrent CCA (CCCA) modeling method with regularization is proposed to exploit the variance and covariance in the process-specific and quality-specific spaces. The CCCA method retains the CCA's efficiency in predicting the quality while exploiting the variance structure for quality and process monitoring using subsequent principal component decompositions. The corresponding monitoring statistics and control limits are then developed in the decomposed subspaces. Numerical simulation examples and the Tennessee Eastman process are used to demonstrate the effectiveness of the CCCA-based monitoring method.     

基于改进KCCA的快速特征提取方法

KCCA特征提取技术具有处理非线性数据的良好性能,但是存在计算量大、特征提取缓慢的局限性.针对KCCA的这一缺点,在研究KCCA特征提取技术和SVDD分类理论的基础上,提出了一种基于改进KCCA的快速特征提取方法,并将改进后的KCCA与SVDD的优势相结合应用于人脸识别中.通过在ORL人脸库上的实验仿真和对比结果验证了所提出方法的有效性.     

Dimensionality reduction of image features using the canonical contextual correlation projection

A linear, discriminative, supervised technique for reducing feature vectors extracted from image data to a lower-dimensional representation is proposed. It is derived from classical linear discriminant analysis (LDA), extending this technique to cases where there is dependency between the output variables, i.e., the class labels, and not only between the input variables. (The latter can readily be dealt with in standard LDA.) The novel method is useful, for example, in supervised segmentation tasks in which high-dimensional feature vectors describe the local structure of the image.

The principal idea is that where standard LDA merely takes into account a single class label for every feature vector, the new technique incorporates class labels of its neighborhood in the analysis as well. In this way, the spatial class label configuration in the vicinity of every feature vector is accounted for, resulting in a technique suitable for, e.g. image data.

This extended LDA, that takes spatial label context into account, is derived from a formulation of standard LDA in terms of canonical correlation analysis. The novel technique is called the canonical contextual correlation projection (CCCP).

An additional drawback of LDA is that it cannot extract more features than the number of classes minus one. In the two-class case this means that only a reduction to one dimension is possible. Our contextual LDA approach can avoid such extreme deterioration of the classification space and retain more than one dimension.

The technique is exemplified on a pixel-based medical image segmentation problem in which it is shown that it may give significant improvement in segmentation accuracy.     

Median Fisher Discriminator: a robust feature extraction method with applications to biometrics

In existing Linear Discriminant Analysis (LDA) models, the class population mean is always estimated by the class sample average. In small sample size problems, such as face and palm recognition, however, the class sample average does not suffice to provide an accurate estimate of the class population mean based on a few of the given samples, particularly when there are outliers in the training set. To overcome this weakness, the class median vector is used to estimate the class population mean in LDA modeling. The class median vector has two advantages over the class sample average: (1) the class median (image) vector preserves useful details in the sample images, and (2) the class median vector is robust to outliers that exist in the training sample set. In addition, a weighting mechanism is adopted to refine the characterization of the within-class scatter so as to further improve the robustness of the proposed model. The proposed Median Fisher Discriminator (MFD) method was evaluated using the Yale and the AR face image databases and the PolyU (Polytechnic University) palmprint database. The experimental results demonstrated the robustness and effectiveness of the proposed method.     

Breast cancer diagnosis using genetic programming generated feature

This paper proposes a novel method for breast cancer diagnosis using the feature generated by genetic programming (GP). We developed a new feature extraction measure (modified Fisher linear discriminant analysis (MFLDA)) to overcome the limitation of Fisher criterion. GP as an evolutionary mechanism provides a training structure to generate features. A modified Fisher criterion is developed to help GP optimize features that allow pattern vectors belonging to different categories to distribute compactly and disjoint regions. First, the MFLDA is experimentally compared with some classical feature extraction methods (principal component analysis, Fisher linear discriminant analysis, alternative Fisher linear discriminant analysis). Second, the feature generated by GP based on the modified Fisher criterion is compared with the features generated by GP using Fisher criterion and an alternative Fisher criterion in terms of the classification performance. The classification is carried out by a simple classifier (minimum distance classifier). Finally, the same feature generated by GP is compared with a original feature set as the inputs to multi-layer perceptrons and support vector machine. Results demonstrate the capability of this method to transform information from high-dimensional feature space into one-dimensional space and automatically discover the relationship among data, to improve classification accuracy.     

A theorem on the generalized canonical projective vectors

This paper proposes a kind of generalized canonical projective vectors (GCPV), based on the framework of canonical correlation analysis (CCA) applying image recognition. Apart from canonical projective vectors (CPV), the process of obtaining GCPV contains the class information of samples, such that the combined features extracted according to the basis of GCPV can give a better classification performance. The experimental result based on the Concordia University CENPARMI handwritten Arabian numeral database has proved that our method is superior to the method based on CPV.     

Combined pattern search optimization of feature extraction and classification parameters in facial recognition

Constantly, the assumption is made that there is an independent contribution of the individual feature extraction and classifier parameters to the recognition performance. In our approach, the problems of feature extraction and classifier design are viewed together as a single matter of estimating the optimal parameters from limited data. We propose, for the problem of facial recognition, a combination between an Interest Operator based feature extraction technique and a k-NN statistical classifier having the parameters determined using a pattern search based optimization technique. This approach enables us to achieve both higher classification accuracy and faster processing time.     

一种新的有监督的局部保持典型相关分析算法

从模式识别的角度出发,在局部保持典型相关分析的基础上,提出一种有监督的局部保持典型相关分析算法(SALPCCA)。该方法在构造样本近邻图时将样本的类别信息考虑在内,由样本间的距离度量确定权重,建立样本间的多重权重相关,通过使同类内的成对样本及其近邻间的权重相关性最大,从而能够在利用样本的类别信息的同时,也能保持数据的局部结构信息。此外,为了能够更好地提取样本的非线性信息,将特征集映射到核特征空间,又提出一种核化的SALPCCA(KSALPCCA)算法。在ORL、Yale、AR等人脸数据库上的实验结果表明,该方法较其他的传统典型相关分析方法有着更好的识别效果。     

Semi-supervised kernel canonical correlation analysis with application to human fMRI

Kernel canonical correlation analysis (KCCA) is a general technique for subspace learning that incorporates principal components analysis (PCA) and Fisher linear discriminant analysis (LDA) as special cases. By finding directions that maximize correlation, KCCA learns representations that are more closely tied to the underlying process that generates the data and can ignore high-variance noise directions. However, for data where acquisition in one or more modalities is expensive or otherwise limited, KCCA may suffer from small sample effects. We propose to use semi-supervised Laplacian regularization to utilize data that are present in only one modality. This approach is able to find highly correlated directions that also lie along the data manifold, resulting in a more robust estimate of correlated subspaces.Functional magnetic resonance imaging (fMRI) acquired data are naturally amenable to subspace techniques as data are well aligned. fMRI data of the human brain are a particularly interesting candidate. In this study we implemented various supervised and semi-supervised versions of KCCA on human fMRI data, with regression to single and multi-variate labels (corresponding to video content subjects viewed during the image acquisition). In each variate condition, the semi-supervised variants of KCCA performed better than the supervised variants, including a supervised variant with Laplacian regularization. We additionally analyze the weights learned by the regression in order to infer brain regions that are important to different types of visual processing.     

Automatic facial feature extraction and 3D face modeling using two orthogonal views with application to 3D face recognition

We present a fully automated algorithm for facial feature extraction and 3D face modeling from a pair of orthogonal frontal and profile view images of a person's face taken by calibrated cameras. The algorithm starts by automatically extracting corresponding 2D landmark facial features from both view images, then compute their 3D coordinates. Further, we estimate the coordinates of the features that are hidden in the profile view based on the visible features extracted in the two orthogonal face images. The 3D coordinates of the selected feature points obtained from the images are used first to align, then to locally deform the corresponding facial vertices of the generic 3D model. Preliminary experiments to assess the applicability of the resulted models for face recognition show encouraging results.     

Complete discriminant evaluation and feature extraction in kernel space for face recognition

This work proposes a method to decompose the kernel within-class eigenspace into two subspaces: a reliable subspace spanned mainly by the facial variation and an unreliable subspace due to limited number of training samples. A weighting function is proposed to circumvent undue scaling of eigenvectors corresponding to the unreliable small and zero eigenvalues. Eigenfeatures are then extracted by the discriminant evaluation in the whole kernel space. These efforts facilitate a discriminative and stable low-dimensional feature representation of the face image. Experimental results on FERET, ORL and GT databases show that our approach consistently outperforms other kernel based face recognition methods.

基于典型相关分析方法的尺度不变特征变换误匹配剔除

针对尺度不变特征变换(SIFT)描述子仅利用特征点的局部邻域灰度信息而对图像内具有相似灰度分布的特征点易产生误匹配的问题,提出一种基于典型相关分析(CCA)的SIFT误匹配剔除方法.该方法首先利用SIFT算法进行匹配,得到初始匹配对; 然后根据典型相关成分的线性关系拟合直线,利用点到直线的距离剔除大部分误匹配点对; 对剩余的匹配点对,逐一分析其对典型相关成分的共线性的影响,剔除影响程度大的特征点对.实验结果表明,该方法能够在剔除误匹配的同时保留更多的正确匹配,提高了图像配准的精度.