Sparse kernel minimum squared error using Householder transformation and givens rotation

Two obvious limitations exist for baseline kernel minimum squared error (KMSE): lack of sparseness of the solution and the ill-posed problem. Previous sparse methods for KMSE have overcome the second limitation using a regularization strategy, which introduces an increase in the computational cost to determine the regularization parameter. Hence, in this paper, a constructive sparse algorithm for KMSE (CS-KMSE) and its improved version (ICS-KMSE) are proposed which will simultaneously address the two limitations described above. CS-KMSE chooses the training samples that incur the largest reductions on the objective function as the significant nodes on the basis of the Householder transformation. In contrast with CS-KMSE, there is an additional replacement mechanism using Givens rotation in ICS-KMSE, which results in ICS-KMSE giving better performance than CS-KMSE in terms of sparseness. CS-KMSE and ICS-KMSE do not require the regularization parameter at all before they begin to choose significant nodes, which is beneficial since it saves on the model selection time. More importantly, CS-KMSE and ICS-KMSE terminate their procedures with an early stopping strategy that acts as an implicit regularization term, which avoids overfitting and curbs the sparse level on the solution of the baseline KMSE. Finally, in comparison with other algorithms, both ICS-KMSE and CS-KMSE have superior sparseness, and extensive comparisons confirm their effectiveness and feasibility.     

A fast method of feature extraction for kernel MSE

In this paper, a fast method of selecting features for kernel minimum squared error (KMSE) is proposed to mitigate the computational burden in the case where the size of the training patterns is large. Compared with other existent algorithms of selecting features for KMSE, this iterative KMSE, viz. IKMSE, shows better property of enhancing the computational efficiency without sacrificing the generalization performance. Experimental reports on the benchmark data sets, nonlinear autoregressive model and real problem address the efficacy and feasibility of the proposed IKMSE. In addition, IKMSE can be easily extended to classification fields.     

An MRF-based kernel method for nonlinear feature extraction

Nonlinear kernel-based feature extraction algorithms have recently been proposed to alleviate the loss of class discrimination after feature extraction. When considering image classification, a kernel function may not be sufficiently effective if it depends only on an information resource from the Euclidean distance in the original feature space. This study presents an extended radial basis kernel function that integrates multiple discriminative information resources, including the Euclidean distance, spatial context, and class membership. The concepts related to Markov random fields (MRFs) are exploited to model the spatial context information existing in the image. Mutual closeness in class membership is defined as a similarity measure with respect to classification. Any dissimilarity from the additional information resources will improve the discrimination between two samples that are only a short Euclidean distance apart in the feature space. The proposed kernel function is used for feature extraction through linear discriminant analysis (LDA) and principal component analysis (PCA). Experiments with synthetic and natural images show the effectiveness of the proposed kernel function with application to image classification.     

A fast computational method for minimum square error transform

In this paper, we describe a basic minimum square error transform for point pattern matching and propose a fast computational method for minimum square error transform. The computational analysis revealed that the proposed method is faster than that of Groen et al. (1985).     

一种新的核广义鉴别特征抽取方法

在基于核的广义鉴别特征模型的基础上,提出了一种新的核广义鉴别特征抽取方法。利用空间变换的有关理论,使得变换后的核总体散布矩阵满足非奇异性;同时通过核共轭特征抽取方法,抽取满足核共轭正交条件的特征向量,使抽取的特征满足统计不相关性。在ORL人脸库上的实验表明了所提方法的有效性,达到了比核鉴别分析等方法更好的识别效果。     

Unsupervised feature extraction via kernel subspace techniques

This paper provides a new insight into unsupervised feature extraction techniques based on kernel subspace models. The data projected onto kernel subspace models are new data representations which might be better suited for classification. The kernel subspace models are always described exploiting the dual form for the basis vectors which requires that the training data must be available even during the test phase. By exploiting an incomplete Cholesky decomposition of the kernel matrix, a computationally less demanding implementation is proposed. Online benchmark data sets allow the evaluation of these feature extraction methods comparing the performance of two classifiers which both have as input either the raw data or the new representations.     

Bilinear analysis for kernel selection and nonlinear feature extraction.

This paper presents a unified criterion, Fisher + kernel criterion (FKC), for feature extraction and recognition. This new criterion is intended to extract the most discriminant features in different nonlinear spaces, and then, fuse these features under a unified measurement. Thus, FKC can simultaneously achieve nonlinear discriminant analysis and kernel selection. In addition, we present an efficient algorithm Fisher + kernel analysis (FKA), which utilizes the bilinear analysis, to optimize the new criterion. This FKA algorithm can alleviate the ill-posed problem existed in traditional kernel discriminant analysis (KDA), and usually, has no singularity problem. The effectiveness of our proposed algorithm is validated by a series of face-recognition experiments on several different databases.     

极化SAR相干斑抑制的非局部加权最小均方误差滤波算法

目的 相干斑的存在严重影响了极化合成孔径雷达(PolSAR)的影像质量.对相干斑的抑制是使用SAR数据的必不可少的预处理程序.提出一种基于非局部加权的线性最小均方误差(LMMSE)滤波器的极化SAR滤波的方法.方法 该方法的主要过程是利用非局部均值的理论来获取LMMSE估计器中像素样本的权重.同时,在样本像素的选取过程中,利用待处理像素的极化散射特性和邻域块的异质性来排除不相似像素以加速算法,同时达到保持点目标和自适应调节块窗口大小的目的.结果 模拟影像和真实影像上进行的实验结果表明,采用这种方法滤波后影像的质量得到明显改善.和传统的LMMSE算法相比,无论是单视的影像还是多视的影像,本文方法去噪结果的等效视数都高出8视以上;峰值信噪比也提升了5.8 dB.同时,去噪后影像分类的总体精度也达到了83％以上,该方法的运行效率也比非局部均值算法有了较大提升.结论 本文方法不仅能够有效抑制相干斑噪声,还能较好地保持边缘和细节信息以及极化散射特性.这将会为后续高效利用SAR数据提供保障.     

Nonlinear feature extraction and classification of multivariate data in kernel feature space

Batch processes have played an essential role in the production of high value-added product of chemical, pharmaceutical, food, bio-chemical, and semi-conductor industries. For productivity and quality improvement, several multivariate statistical techniques such as principal component analysis (PCA) and Fisher discriminant analysis (FDA) have been developed to solve a fault diagnosis problem of batch processes. Fisher discriminant analysis, as a traditional statistical technique for feature extraction and classification, has been shown to be a good linear technique for fault diagnosis and outperform PCA based diagnosis methods. This paper proposes a more efficient nonlinear diagnosis method for batch processes using a kernel version of Fisher discriminant analysis (KFDA). A case study on two batch processes has been conducted. In addition, the diagnosis performance of the proposed method was compared with that of an existing diagnosis method based on linear FDA. The diagnosis results showed that the proposed KFDA based diagnosis method outperforms the linear FDA based method.     

Nonlinear feature extraction based on centroids and kernel functions

A nonlinear feature extraction method is presented which can reduce the data dimension down to the number of classes, providing dramatic savings in computational costs. The dimension reducing nonlinear transformation is obtained by implicitly mapping the input data into a feature space using a kernel function, and then finding a linear mapping based on an orthonormal basis of centroids in the feature space that maximally separates the between-class relationship. The experimental results demonstrate that our method is capable of extracting nonlinear features effectively so that competitive performance of classification can be obtained with linear classifiers in the dimension reduced space.     

Bilinear Lanczos components for fast dimensionality reduction and feature extraction

Matrix-based methods such as generalized low rank approximations of matrices (GLRAM) have gained wide attention from researchers in pattern recognition and machine learning communities. In this paper, a novel concept of bilinear Lanczos components (BLC) is introduced to approximate the projection vectors obtained from eigen-based methods without explicit computing eigenvectors of the matrix. This new method sequentially reduces the reconstruction error for a Frobenius-norm based optimization criterion, and the resulting approximation performance is thus improved during successive iterations. In addition, a theoretical clue for selecting suitable dimensionality parameters without losing classification information is presented in this paper. The BLC approach realizes dimensionality reduction and feature extraction by using a small number of Lanczos components. Extensive experiments on face recognition and image classification are conducted to evaluate the efficiency and effectiveness of the proposed algorithm. Results show that the new approach is competitive with the state-of-the-art methods, while it has a much lower training cost.     

Improved wavelet feature extraction using kernel analysis for text independent speaker recognition

A text independent speaker recognition system based on improved wavelet transform is proposed. Learning of the correlation between the wavelet transform and the expression vector is performed by kernel canonical correlation analysis. Kernel canonical correlation analysis is a nonlinear extension of canonical correlation analysis. Moreover, we also propose an improved kernel canonical correlation algorithm to tackle the singularity problem of the wavelet matrix. The identification model underlying the Gaussian mixture model is presented; in particular, an expectation-maximization algorithm is also proposed for adjusting the parameters. The experimental results on the TALUNG database and KING database illustrate the effectiveness of the proposed method.     

基于贪心核特征提取方法的中期峰值负荷预测

针对中期电力负荷预测, 提出基于贪心核主元回归(GKPCR)、贪心核岭回归(GKRR) 的特征提取建模方法. 通过对核矩阵的稀疏逼近, GKPCR和GKRR两种贪心核特征提取方法旨在寻找特征空间中数据的低维表示, 计算需求低, 适用于大数据集的在线学习. 将所提出的方法应用于不同地区的电力负荷中期峰值预测, 并与现有预测方法进行了比较. 实验结果表明, 在同等条件下, 所提出的方法能有效地改进预测精度, 而且性能更好, 显示了其有效性和应用潜力.

     

Complete discriminant evaluation and feature extraction in kernel space for face recognition

This work proposes a method to decompose the kernel within-class eigenspace into two subspaces: a reliable subspace spanned mainly by the facial variation and an unreliable subspace due to limited number of training samples. A weighting function is proposed to circumvent undue scaling of eigenvectors corresponding to the unreliable small and zero eigenvalues. Eigenfeatures are then extracted by the discriminant evaluation in the whole kernel space. These efforts facilitate a discriminative and stable low-dimensional feature representation of the face image. Experimental results on FERET, ORL and GT databases show that our approach consistently outperforms other kernel based face recognition methods.

Analysing the Linux kernel feature model changes using FMDiff

Evolving a large scale, highly variable system is a challenging task. For such a system, evolution operations often require to update consistently both their implementation and its feature model. In this context, the evolution of the feature model closely follows the evolution of the system. The purpose of this work is to show that fine-grained feature changes can be used to guide the evolution of the highly variable system. In this paper, we present an approach to obtain fine-grained feature model changes with its supporting tool “FMDiff”. Our approach is tailored for Kconfig-based variability models and proposes a feature change classification detailing changes in features, their attributes and attribute values. We apply our approach to the Linux kernel feature model, extracting feature changes occurring in sixteen official releases. In contrast to previous studies, we found that feature modifications are responsible for most of the changes. Then, by taking advantage of the multi-platform aspect of the Linux kernel, we observe the effects of a feature change across the different architecture-specific feature models of the kernel. We found that between 10 and 50 % of feature changes impact all the architecture-specific feature models, offering a new perspective on studies of the evolution of the Linux feature model and development practices of its developers.     

Chained aggregation model with minimum truncation error

A reduced–order model for a multi–input single–output or a single–input multi–output continuous system is proposed. The model is especially suitable for the case where accuracy in the high-frequency region is important. It also satisfies basic conditions for reduced-order models, such as uniqueness, stability, and a simple calculation requirement. Chained aggregation with the minimization of truncation errors introduces an internal system representation characterized by a system matrix of Hessenberg form and a diagonalized covariance matrix.     

A fast algorithm for feature extraction of hyperspectral images using the first order statistics

Multimedia Tools and Applications - A new supervised feature extraction method appropriate for small sample size situations is proposed in this work. The proposed method is based on the first-order...     

基于特征增强与最小误差分割的变化检测方法

以河北省石家庄市2003年和2004年的专题制图仪(TM)遥感影像为例,针对各波段光谱特征,提出了一种基于地物特征增强的变化检测方法.在两期影像上对各类地物采样并计算样本在不同波段的均值、标准差等特征量,以确定波段组合运算的加权系数,计算特征增强图像,实现两期影像中所指定地物类型的特征增强;计算两期特征增强影像的差异影像;使用最小误差分割法获取变化检测结果.通过对比实验可知:方法提取变化区域总体精度达到90%,相对于传统的基于主成分分析(PCA)的变化检测方法,具有较高的检测精度,较好的可行性与适应性.     

Learning the kernel parameters in kernel minimum distance classifier

Choosing appropriate values for kernel parameters is one of the key problems in many kernel-based methods because the values of these parameters have significant impact on the performances of these methods. In this paper, a novel approach is proposed to learn the kernel parameters in kernel minimum distance (KMD) classifier, where the values of the kernel parameters are computed through optimizing an objective function designed for measuring the classification reliability of KMD. Experiments on both artificial and real-world datasets show that the proposed approach works well on learning kernel parameters of KMD.