An MRF-based kernel method for nonlinear feature extraction

Nonlinear kernel-based feature extraction algorithms have recently been proposed to alleviate the loss of class discrimination after feature extraction. When considering image classification, a kernel function may not be sufficiently effective if it depends only on an information resource from the Euclidean distance in the original feature space. This study presents an extended radial basis kernel function that integrates multiple discriminative information resources, including the Euclidean distance, spatial context, and class membership. The concepts related to Markov random fields (MRFs) are exploited to model the spatial context information existing in the image. Mutual closeness in class membership is defined as a similarity measure with respect to classification. Any dissimilarity from the additional information resources will improve the discrimination between two samples that are only a short Euclidean distance apart in the feature space. The proposed kernel function is used for feature extraction through linear discriminant analysis (LDA) and principal component analysis (PCA). Experiments with synthetic and natural images show the effectiveness of the proposed kernel function with application to image classification.     

A fast method of feature extraction for kernel MSE

In this paper, a fast method of selecting features for kernel minimum squared error (KMSE) is proposed to mitigate the computational burden in the case where the size of the training patterns is large. Compared with other existent algorithms of selecting features for KMSE, this iterative KMSE, viz. IKMSE, shows better property of enhancing the computational efficiency without sacrificing the generalization performance. Experimental reports on the benchmark data sets, nonlinear autoregressive model and real problem address the efficacy and feasibility of the proposed IKMSE. In addition, IKMSE can be easily extended to classification fields.     

Complete discriminant evaluation and feature extraction in kernel space for face recognition

This work proposes a method to decompose the kernel within-class eigenspace into two subspaces: a reliable subspace spanned mainly by the facial variation and an unreliable subspace due to limited number of training samples. A weighting function is proposed to circumvent undue scaling of eigenvectors corresponding to the unreliable small and zero eigenvalues. Eigenfeatures are then extracted by the discriminant evaluation in the whole kernel space. These efforts facilitate a discriminative and stable low-dimensional feature representation of the face image. Experimental results on FERET, ORL and GT databases show that our approach consistently outperforms other kernel based face recognition methods.

The use of kernel principal component analysis to model data distributions

We describe the use of kernel principal component analysis (KPCA) to model data distributions in high-dimensional spaces. We show that a previous approach to representing non-linear data constraints using KPCA is not generally valid, and introduce a new ‘proximity to data’ measure that behaves correctly. We investigate the relation between this measure and the actual density for various low-dimensional data distributions. We demonstrate the effectiveness of the method by applying it to the higher-dimensional case of modelling an ensemble of images of handwritten digits, showing how it can be used to extract the digit information from noisy input images.     

Nonlinear feature extraction based on centroids and kernel functions

A nonlinear feature extraction method is presented which can reduce the data dimension down to the number of classes, providing dramatic savings in computational costs. The dimension reducing nonlinear transformation is obtained by implicitly mapping the input data into a feature space using a kernel function, and then finding a linear mapping based on an orthonormal basis of centroids in the feature space that maximally separates the between-class relationship. The experimental results demonstrate that our method is capable of extracting nonlinear features effectively so that competitive performance of classification can be obtained with linear classifiers in the dimension reduced space.     

Kernel Self-optimized Locality Preserving Discriminant Analysis for feature extraction and recognition

We propose Kernel Self-optimized Locality Preserving Discriminant Analysis (KSLPDA) for feature extraction and recognition. The procedure of KSLPDA is divided into two stages, i.e., one is to solve the optimal expansion of the data-dependent kernel with the proposed kernel self-optimization method, and the second is to seek the optimal projection matrix for dimensionality reduction. Since the optimal parameters of data-dependent kernel are achieved automatically through solving the constraint optimization equation, based on maximum margin criterion and Fisher criterion in the empirical feature space, KSLPDA works well on feature extraction for classification. The comparative experiments show that KSLPDA outperforms PCA, LDA, LPP, supervised LPP and kernel supervised LPP.     

Improving angular error via systematically designed near-circular Gaussian-based feature extraction operators

In image filtering, the ‘circularity’ of an operator is an important factor affecting its accuracy. For example, circular differential edge operators are effective in minimising the angular error in the estimation of image gradient direction. We present a general approach to the computation of scalable circular low-level image processing operators that is based on the finite element method. We show that the use of Gaussian basis functions within the finite element method provides a framework for a systematic and efficient design procedure for operators that are scalable to near-circular neighbourhoods through the use of an explicit scale parameter. The general design technique may be applied to a range of operators. Here we evaluate the approach for the design of the image gradient operator. We illustrate that this design procedure significantly reduces angular error in comparison to other well-known gradient approximation methods.     

基于低秩稀疏评分的非监督特征选择

在处理高维数据过程中,特征选择是一个非常重要的数据降维步骤。低秩表示模型具有揭示数据全局结构信息的能力和一定的鉴别能力。稀疏表示模型能够利用较少的连接关系揭示数据的本质结构信息。在低秩表示模型的基础上引入稀疏约束项,构建一种低秩稀疏表示模型学习数据间的低秩稀疏相似度矩阵;基于该矩阵提出一种低秩稀疏评分机制用于非监督特征选择。在不同数据库上将选择后的特征进行聚类和分类实验,同传统特征选择算法进行比较。实验结果表明了低秩特征选择算法的有效性。     

Kernel nonnegative matrix factorization for spectral EEG feature extraction

Nonnegative matrix factorization (NMF) seeks a decomposition of a nonnegative matrix X0 into a product of two nonnegative factor matrices U0 and V0, such that a discrepancy between X and UV is minimized. Assuming U=XW in the decomposition (for W0), kernel NMF (KNMF) is easily derived in the framework of least squares optimization. In this paper we make use of KNMF to extract discriminative spectral features from the time–frequency representation of electroencephalogram (EEG) data, which is an important task in EEG classification. Especially when KNMF with linear kernel is used, spectral features are easily computed by a matrix multiplication, while in the standard NMF multiplicative update should be performed repeatedly with the other factor matrix fixed, or the pseudo-inverse of a matrix is required. Moreover in KNMF with linear kernel, one can easily perform feature selection or data selection, because of its sparsity nature. Experiments on two EEG datasets in brain computer interface (BCI) competition indicate the useful behavior of our proposed methods.     

Kernel-view based discriminant approach for embedded feature extraction in high-dimensional space

Derived from the traditional manifold learning algorithms, local discriminant analysis methods identify the underlying submanifold structures while employing discriminative information for dimensionality reduction. Mathematically, they can all be unified into a graph embedding framework with different construction criteria. However, such learning algorithms are limited by the curse-of-dimensionality if the original data lie on the high-dimensional manifold. Different from the existing algorithms, we consider the discriminant embedding as a kernel analysis approach in the sample space, and a kernel-view based discriminant method is proposed for the embedded feature extraction, where both PCA pre-processing and the pruning of data can be avoided. Extensive experiments on the high-dimensional data sets show the robustness and outstanding performance of our proposed method.     

Hierarchical palmprint identification via multiple feature extraction

Biometric computing offers an effective approach to identify personal identity by using individual's unique, reliable and stable physical or behavioral characteristics. This paper describes a new method to authenticate individuals based on palmprint identification and verification. Firstly, a comparative study of palmprint feature extraction is presented. The concepts of texture feature and interesting points are introduced to define palmprint features. A texture-based dynamic selection scheme is proposed to facilitate the fast search for the best matching of the sample in the database in a hierarchical fashion. The global texture energy, which is characterized with high convergence of inner-palm similarities and good dispersion of inter-palm discrimination, is used to guide the dynamic selection of a small set of similar candidates from the database at coarse level for further processing. An interesting point based image matching is performed on the selected similar patterns at fine level for the final confirmation. The experimental results demonstrate the effectiveness and accuracy of the proposed method.     

Bilinear Lanczos components for fast dimensionality reduction and feature extraction

Matrix-based methods such as generalized low rank approximations of matrices (GLRAM) have gained wide attention from researchers in pattern recognition and machine learning communities. In this paper, a novel concept of bilinear Lanczos components (BLC) is introduced to approximate the projection vectors obtained from eigen-based methods without explicit computing eigenvectors of the matrix. This new method sequentially reduces the reconstruction error for a Frobenius-norm based optimization criterion, and the resulting approximation performance is thus improved during successive iterations. In addition, a theoretical clue for selecting suitable dimensionality parameters without losing classification information is presented in this paper. The BLC approach realizes dimensionality reduction and feature extraction by using a small number of Lanczos components. Extensive experiments on face recognition and image classification are conducted to evaluate the efficiency and effectiveness of the proposed algorithm. Results show that the new approach is competitive with the state-of-the-art methods, while it has a much lower training cost.     

A feature extraction method for use with bimodal biometrics

Bimodal biometrics has been found to outperform single biometrics and are usually implemented using the matching score level or decision level fusion, though this fusion will enable less information of bimodal biometric traits to be exploited for personal authentication than fusion at the feature level. This paper proposes matrix-based complex PCA (MCPCA), a feature level fusion method for bimodal biometrics that uses a complex matrix to denote two biometric traits from one subject. The method respectively takes the two images from two biometric traits of a subject as the real part and imaginary part of a complex matrix. MCPCA applies a novel and mathematically tractable algorithm for extracting features directly from complex matrices. We also show that MCPCA has a sound theoretical foundation and the previous matrix-based PCA technique, two-dimensional PCA (2DPCA), is only one special form of the proposed method. On the other hand, the features extracted by the developed method may have a large number of data items (each real number in the obtained features is called one data item). In order to obtain features with a small number of data items, we have devised a two-step feature extraction scheme. Our experiments show that the proposed two-step feature extraction scheme can achieve a higher classification accuracy than the 2DPCA and PCA techniques.     

Robust visual tracking via augmented kernel SVM

Most current tracking approaches utilize only one type of feature to represent the target and learn the appearance model of the target just by using the current frame or a few recent ones. The limited representation of one single type of feature might not represent the target well. What's more, the appearance model learning from the current frame or a few recent ones is intolerant of abrupt appearance changes in short time intervals. These two factors might cause the track's failure. To overcome these two limitations, in this paper, we apply the Augmented Kernel Matrix (AKM) classification to combine two complementary features, pixel intensity and LBP (Local Binary Pattern) features, to enrich the target's representation. Meanwhile, we employ the AKM clustering to group the tracking results into a few aspects. And then, the representative patches are selected and added into the training set to learn the appearance model. This makes the appearance model cover more aspects of the target appearance and more robust to abrupt appearance changes. Experiments compared with several state-of-the-art methods on challenging sequences demonstrate the effectiveness and robustness of the proposed algorithm.     

Generalized K-L transform based combined feature extraction

In this paper, we combine two kinds of features together by virtue of complex vectors and then use the developed generalized K-L transform (or expansion) for feature extraction. The experiments on NUST603 handwritten Chinese character database and CENPARMI handwritten digit database indicate that the proposed method can improve the recognition rate significantly.     

联合低秩稀疏的多核子空间聚类算法

针对多核子空间谱聚类算法没有考虑噪声和关系图结构的问题，提出了一种新的联合低秩稀疏的多核子空间聚类算法(JLSMKC)。首先，通过联合低秩与稀疏表示进行子空间学习，使关系图具有低秩和稀疏结构属性；其次，建立鲁棒的多核低秩稀疏约束模型，用于减少噪声对关系图的影响和处理数据的非线性结构；最后，通过多核方法充分利用共识核矩阵来增强关系图质量。7个数据集上的实验结果表明，所提算法JLSMKC在聚类精度(ACC)、标准互信息(NMI)和纯度(Purity)上优于5种流行的多核聚类算法，同时减少了聚类时间，提高了关系图块对角质量。该算法在聚类性能上有较大优势。     

一种可最优化计算特征规模的互信息特征提取

利用矩阵特征向量分解，提出一种可最优化计算特征规模的互信息特征提取方法．首先，论述了高斯分布假设下的该互信息判据的类可分特性，并证明了现有典型算法都是本算法的特例；然后，在给出该互信息判据严格的数学意义基础上，提出了基于矩阵特征向量分解计算最优化特征规模算法；最后，通过实际数据验证了该方法的有效性     

Theoretical analysis on feature extraction capability of class-augmented PCA

In this paper, we present a theoretical analysis on a novel supervised feature extraction method called class-augmented principal component analysis (CA-PCA), which is composed of processes for encoding the class information, augmenting the encoded information to data, and extracting features from class-augmented data by applying PCA. Through a combination of these processes, CA-PCA can extract features appropriate for classification. Our theoretical analysis aims to clarify the role of these processes and to provide an explanation on how CA-PCA can extract good features. Experimental results for various datasets are provided in order to show the validity of the proposed method for real problems. The effect of parameters on the quality of extracted features is also investigated and the rules of thumb for determining the appropriate parameters are provided.     

Local structure based supervised feature extraction

In the past few years, the computer vision and pattern recognition community has witnessed the rapid growth of a new kind of feature extraction method, the manifold learning methods, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among them, locality preserving projection (LPP) is one of the most promising feature extraction techniques. However, when LPP is applied to the classification tasks, it shows some limitations, such as the ignorance of the label information. In this paper, we propose a novel feature extraction method, called locally discriminating projection (LDP). LDP utilizes class information to guide the procedure of feature extraction. In LDP, the local structure of the original data is constructed according to a certain kind of similarity between data points, which takes special consideration of both the local information and the class information. The similarity has several good properties which help to discover the true intrinsic structure of the data, and make LDP a robust technique for the classification tasks. We compare the proposed LDP approach with LPP, as well as other feature extraction methods, such as PCA and LDA, on the public available data sets, FERET and AR. Experimental results suggest that LDP provides a better representation of the class information and achieves much higher recognition accuracies.     

基于随机子空间的多标签类属特征提取算法

目前多标签学习已广泛应用到很多场景中,在此类学习问题中,一个样本往往可以同时拥有多个类别标签。由于类别标签可能带有的特有属性（即类属属性）将更有助于标签分类,所以已经出现了一些基于类属属性的多标签学习算法。针对类属属性构造会导致属性空间存在冗余的问题,本文提出了一种多标签类属特征提取算法LIFT_RSM。该方法基于类属属性空间通过综合利用随机子空间模型及成对约束降维思想提取有效的特征信息,以达到提升分类性能的目的。在多个数据集上的实验结果表明：与若干经典的多标签算法相比,提出的LIFT_RSM算法能得到更好的分类效果。