Orthogonal sparse linear discriminant analysis

Linear discriminant analysis (LDA) is a linear feature extraction approach, and it has received much attention. On the basis of LDA, researchers have done a lot of research work on it, and many variant versions of LDA were proposed. However, the inherent problem of LDA cannot be solved very well by the variant methods. The major disadvantages of the classical LDA are as follows. First, it is sensitive to outliers and noises. Second, only the global discriminant structure is preserved, while the local discriminant information is ignored. In this paper, we present a new orthogonal sparse linear discriminant analysis (OSLDA) algorithm. The k nearest neighbour graph is first constructed to preserve the locality discriminant information of sample points. Then, L_2,1-norm constraint on the projection matrix is used to act as loss function, which can make the proposed method robust to outliers in data points. Extensive experiments have been performed on several standard public image databases, and the experiment results demonstrate the performance of the proposed OSLDA algorithm.     

Dynamic graph learning for spectral feature selection

Previous spectral feature selection methods generate the similarity graph via ignoring the negative effect of noise and redundancy of the original feature space, and ignoring the association between graph matrix learning and feature selection, so that easily producing suboptimal results. To address these issues, this paper joints graph learning and feature selection in a framework to obtain optimal selected performance. More specifically, we use the least square loss function and an ? _2,1-norm regularization to remove the effect of noisy and redundancy features, and use the resulting local correlations among the features to dynamically learn a graph matrix from a low-dimensional space of original data. Experimental results on real data sets show that our method outperforms the state-of-the-art feature selection methods for classification tasks.

     

Multi-label feature selection based on manifold regularization and imbalance ratio

The dimension of data in the domain of multi-label learning is usually high, which makes the calculation cost very high. As an important data dimension reduction technology, feature selection has attracted the attention of many researchers. And the imbalance of data labels is also one of the factors that perplex multi-label learning. To tackle these problems, we propose a new multi-label feature selection algorithm named IMRFS, which combines manifold learning and label imbalance. Firstly, in order to keep the manifold structure between samples, the Laplacian graph is used to construct the manifold regularization. In addition, the local manifold structure of each label is considered to find the correlation between labels. And the imbalance distribution of labels is also considered, which is embedded into the manifold structure of labels. Furthermore, in order to ensure the robustness and sparsity of the IMRFS method, the L_2,1-norm is applied to loss function and sparse regularization term simultaneously. Then, we adopt an iterative strategy to optimize the objective function of IMRFS. Finally, comparison results on multiple datasets show the effectiveness of IMRFS method.

     

Unsupervised feature selection based on kernel fisher discriminant analysis and regression learning

In this paper, we propose a new feature selection method called kernel fisher discriminant analysis and regression learning based algorithm for unsupervised feature selection. The existing feature selection methods are based on either manifold learning or discriminative techniques, each of which has some shortcomings. Although some studies show the advantages of two-steps method benefiting from both manifold learning and discriminative techniques, a joint formulation has been shown to be more efficient. To do so, we construct a global discriminant objective term of a clustering framework based on the kernel method. We add another term of regression learning into the objective function, which can impose the optimization to select a low-dimensional representation of the original dataset. We use L_2,1-norm of the features to impose a sparse structure upon features, which can result in more discriminative features. We propose an algorithm to solve the optimization problem introduced in this paper. We further discuss convergence, parameter sensitivity, computational complexity, as well as the clustering and classification accuracy of the proposed algorithm. In order to demonstrate the effectiveness of the proposed algorithm, we perform a set of experiments with different available datasets. The results obtained by the proposed algorithm are compared against the state-of-the-art algorithms. These results show that our method outperforms the existing state-of-the-art methods in many cases on different datasets, but the improved performance comes with the cost of increased time complexity.

     

Unsupervised feature selection based on decision graph

In applications of algorithms, feature selection has got much attention of researchers, due to its ability to overcome the curse of dimensionality, reduce computational costs, increase the performance of the subsequent classification algorithm and output the results with better interpretability. To remove the redundant and noisy features from original feature set, we define local density and discriminant distance for each feature vector, wherein local density is used for measuring the representative ability of each feature vector, and discriminant distance is used for measuring the redundancy and similarity between features. Based on the above two quantities, the decision graph score is proposed as the evaluation criterion of unsupervised feature selection. The method is intuitive and simple, and its performances are evaluated in the data classification experiments. From statistical tests on the averaged classification accuracies over 16 real-life dataset, it is observed that the proposed method obtains better or comparable ability of discriminant feature selection in 98% of the cases, compared with the state-of-the-art methods.

     

Frobenius norm-regularized robust graph learning for multi-view subspace clustering

Graph learning methods have been widely used for multi-view clustering. However, such methods have the following challenges: (1) they usually perform simple fusion of fixed similarity graph matrices, ignoring its essential structure. (2) they are sensitive to noise and outliers because they usually learn the similarity matrix from the raw features. To solve these problems, we propose a novel multi-view subspace clustering method named Frobenius norm-regularized robust graph learning (RGL), which inherits desirable advantages (noise robustness and local information preservation) from the subspace clustering and manifold learning. Specifically, RGL uses Frobenius norm constraint and adjacency similarity learning to simultaneously explore the global information and local similarity of views. Furthermore, the l_2,1 norm is imposed on the error matrix to remove the disturbance of noise and outliers. An effectively iterative algorithm is designed to solve the RGL model by the alternation direction method of multipliers. Extensive experiments on nine benchmark databases show the clear advantage of the proposed method over fifteen state-of-the-art clustering methods.

     

基于全局不相关的多流形学习

为提升人脸识别算法的鲁棒性,减少判别信息的冗余度,提出基于全局不相关的多流形判别学习算法(UFDML)。使用特征空间到特征空间的距离,学习样本局部判别信息,提出全局不相关约束,使提取的判别特征是统计不相关的。在Yale,AR,ORL人脸库上的实验结果表明,与LPP(局部保持投影)、LDA(线性判别分析)、UDP(非监督判别投影)等人脸识别算法相比,所提算法的平均识别率高于其它算法,验证了其有效性。     

基于主元和判别集成分析的模拟电路故障诊断

提出了主元和线性判别的集成分析算法以实施模拟故障数据的特征提取过程和方法。该集成分析方法首先对模拟故障数据进行主元分析,然后在主元变换空间实行线性判别分析,最后将所获得的最优判别特征模式应用于模式分类器进行故障诊断。仿真结果表明,所提出的方法能够充分利用线性方法的计算简便优势,增强单一主元分析或线性判别分析的特征提取性能,获取故障数据集的本质特征,简化模式分类器的结构,降低系统运行的计算成本。     

Feature extraction using constrained maximum variance mapping

In this paper, an efficient feature extraction method named as constrained maximum variance mapping (CMVM) is developed. The proposed algorithm can be viewed as a linear approximation of multi-manifolds learning based approach, which takes the local geometry and manifold labels into account. The CMVM and the original manifold learning based approaches have a point in common that the locality is preserved. Moreover, the CMVM is globally maximizing the distances between different manifolds. After the local scatters have been characterized, the proposed method focuses on developing a linear transformation that can maximize the dissimilarities between all the manifolds under the constraint of locality preserving. Compared to most of the up-to-date manifold learning based methods, this trick makes contribution to pattern classification from two aspects. On the one hand, the local structure in each manifold is still kept; on the other hand, the discriminant information between manifolds can be explored. Finally, FERET face database, CMU PIE face database and USPS handwriting data are all taken to examine the effectiveness and efficiency of the proposed method. Experimental results validate that the proposed approach is superior to other feature extraction methods, such as linear discriminant analysis (LDA), locality preserving projection (LPP), unsupervised discriminant projection (UDP) and maximum variance projection (MVP).     

Kernel-view based discriminant approach for embedded feature extraction in high-dimensional space

Derived from the traditional manifold learning algorithms, local discriminant analysis methods identify the underlying submanifold structures while employing discriminative information for dimensionality reduction. Mathematically, they can all be unified into a graph embedding framework with different construction criteria. However, such learning algorithms are limited by the curse-of-dimensionality if the original data lie on the high-dimensional manifold. Different from the existing algorithms, we consider the discriminant embedding as a kernel analysis approach in the sample space, and a kernel-view based discriminant method is proposed for the embedded feature extraction, where both PCA pre-processing and the pruning of data can be avoided. Extensive experiments on the high-dimensional data sets show the robustness and outstanding performance of our proposed method.     

Unsupervised double weight graphs based discriminant analysis for dimensionality reduction

ABSTRACT

Dimensionality reduction plays an important role in pattern recognition tasks. Locality preserving projection and neighbourhood preserving embedding are popular unsupervised feature extraction methods, which try to preserve a certain local structure in the low-dimensional subspace. However, only considering the local neighbour information will limit the methods to achieve higher recognition accuracy. In this paper, an unsupervised double weight graphs based discriminant analysis method (uDWG-DA) is proposed. First, uDWG-DA considers both similar and dissimilar relationships among samples by using double weight graphs. In order to explore the dissimilar information, a new partitioning strategy is proposed to divide the data set into different clusters, where samples of different clusters are dissimilar. Then, based on L_2,1 norm, uDWG-DA finds the optimal projection to not only preserve the similar local structure but also increase the separability among different clusters of the data set. Experiments on four hyperspectral images validate the advantage and feasibility of the proposed method compared with other dimensionality reduction methods.     

基于自步学习的鲁棒多样性多视角聚类

目的 大数据环境下的多视角聚类是一个非常有价值且极具挑战性的问题。现有的适合大规模多视角数据聚类的方法虽然在一定程度上能够克服由于目标函数非凸性导致的局部最小值,但是缺乏对异常点鲁棒性的考虑,且在样本选择过程中忽略了视角多样性。针对以上问题,提出一种基于自步学习的鲁棒多样性多视角聚类模型（RD-MSPL）。方法 1）通过在目标函数中引入结构稀疏范数L_2,1来建模异常点;2）通过在自步正则项中对样本权值矩阵施加反结构稀疏约束来增加在多个视角下所选择样本的多样性。结果 在Extended Yale B、Notting-Hill、COIL-20和Scene15公开数据集上的实验结果表明：1）在4个数据集上,所提出的RD-MSPL均优于现有的2个最相关多视角聚类方法。与鲁棒多视角聚类方法（RMKMC）相比,聚类准确率分别提升4.9%,4.8%,3.3%和1.3%;与MSPL相比,准确率分别提升7.9%,4.2%,7.1%和6.5%。2）通过自对比实验,证实了所提模型考虑鲁棒性和样本多样性的有效性;3）与单视角以及多个视角简单拼接的实验对比表明,RD-MSPL能够更有效地探索视角之间关联关系。结论 本文提出一种基于自步学习的鲁棒多样性多视角聚类模型,并针对该模型设计了一种高效求解算法。所提方法能够有效克服异常点对聚类性能的影响,在聚类过程中逐步加入不同视角下的多样性样本,在避免局部最小值的同时,能更好地获取不同视角的互补信息。实验结果表明,本文方法优于现有的相关方法。     

FitConnect: Connecting Noisy 2D Samples by Fitted Neighbourhoods

We propose a parameter‐free method to recover manifold connectivity in unstructured 2D point clouds with high noise in terms of the local feature size. This enables us to capture the features which emerge out of the noise. To achieve this, we extend the reconstruction algorithm HNN‐Crust , which connects samples to two (noise‐free) neighbours and has been proven to output a manifold for a relaxed sampling condition. Applying this condition to noisy samples by projecting their k‐nearest neighbourhoods onto local circular fits leads to multiple candidate neighbour pairs and thus makes connecting them consistently an NP‐hard problem. To solve this efficiently, we design an algorithm that searches that solution space iteratively on different scales of k. It achieves linear time complexity in terms of point count plus quadratic time in the size of noise clusters. Our algorithm FitConnect extends HNN‐Crust seamlessly to connect both samples with and without noise, performs as local as the recovered features and can output multiple open or closed piecewise curves. Incidentally, our method simplifies the output geometry by eliminating all but a representative point from noisy clusters. Since local neighbourhood fits overlap consistently, the resulting connectivity represents an ordering of the samples along a manifold. This permits us to simply blend the local fits for denoising with the locally estimated noise extent. Aside from applications like reconstructing silhouettes of noisy sensed data, this lays important groundwork to improve surface reconstruction in 3D. Our open‐source algorithm is available online.     

基于稀疏流形聚类嵌入模型和L1范数正则化的标签错误检测

综合利用含错标签中的有用信息和数据结构中蕴含的鉴别信息,提出一种基于稀疏流形聚类嵌入模型和L1范数正则化的标签错误检测修正方法.首先,用稀疏流形聚类嵌入模型将数据投影到易分类的空间,利用标注正确的极少量样本和最近邻分类器获得新标签;然后,构造标签错误检测模型,获得仅含0、1元素的检测向量,正确、错误的标签分别对应着1、0的位置;最后,给出了相应的优化算法及收敛证明,并在相关实验上验证了算法的有效性.     

基于局部判别约束的半监督特征选择方法*

特征选择旨在选择待处理数据中最具代表性的特征,降低特征空间的维度.文中提出基于局部判别约束的半监督特征选择方法,充分利用已标记样本和未标记样本训练特征选择模型,并借助相邻数据间的局部判别信息提高模型的准确度,引入l_2,1约束提高特征之间可区分度,避免噪声干扰.最后通过实验验证文中方法的有效性.     

动态加权非参数判别分析

线性判别分析(LDA)是最经典的子空间学习和有监督判别特征提取方法之一.受到流形学习的启发,近年来众多基于LDA的改进方法被提出.尽管出发点不同,但这些算法本质上都是基于欧氏距离来度量样本的空间散布度.欧氏距离的非线性特性带来了如下两个问题:1)算法对噪声和异常样本点敏感;2)算法对流形或者是多模态数据集中局部散布度较大的样本点过度强调,导致特征提取过程中数据的本质结构特征被破坏.为了解决这些问题,提出一种新的基于非参数判别分析(NDA)的维数约减方法,称作动态加权非参数判别分析(DWNDA).DWNDA采用动态加权距离来计算类间散布度和类内散布度,不仅能够保留多模态数据集的本质结构特征,还能有效地利用边界样本点对之间的判别信息.因此,DWNDA在噪声实验中展现出对噪声和异常样本的强鲁棒性.此外,在人脸和手写体数据库上进行实验,DWNDA方法均取得了优异的实验结果.     

全局判别与局部稀疏保持HSI半监督特征提取

针对高光谱图像存在“维数灾难”的问题，提出一种全局判别与局部稀疏保持的高光谱图像半监督特征提取算法（GLSSFE）。该算法通过LDA算法的散度矩阵保存有类标样本的全局类内判别信息和全局类间判别信息，结合利用半监督PCA算法对有类标和无类标样本进行主成分分析，保存样本的全局结构；利用稀疏表示优化模型自适应揭示样本数据间的非线性结构，将局部类间判别权值和局部类内判别权值嵌入半监督LPP算法保留样本数据的局部结构，从而最大化同类样本的相似性和异类样本的差异性。通过1-NN和SVM两个分类器分别对Indian Pines和Pavia University两个公共高光谱图像数据集进行分类，验证所提特征提取方法的有效性。实验结果表明，该GLSSFE算法最高总体分类精度分别达到89.10%和92.09%，优于现有的特征提取算法，能有效地挖掘高光谱图像的全局特征和局部特征，极大地提升高光谱图像的地物分类效果。     

Speed up kernel discriminant analysis

Linear discriminant analysis (LDA) has been a popular method for dimensionality reduction, which preserves class separability. The projection vectors are commonly obtained by maximizing the between-class covariance and simultaneously minimizing the within-class covariance. LDA can be performed either in the original input space or in the reproducing kernel Hilbert space (RKHS) into which data points are mapped, which leads to kernel discriminant analysis (KDA). When the data are highly nonlinear distributed, KDA can achieve better performance than LDA. However, computing the projective functions in KDA involves eigen-decomposition of kernel matrix, which is very expensive when a large number of training samples exist. In this paper, we present a new algorithm for kernel discriminant analysis, called Spectral Regression Kernel Discriminant Analysis (SRKDA). By using spectral graph analysis, SRKDA casts discriminant analysis into a regression framework, which facilitates both efficient computation and the use of regularization techniques. Specifically, SRKDA only needs to solve a set of regularized regression problems, and there is no eigenvector computation involved, which is a huge save of computational cost. The new formulation makes it very easy to develop incremental version of the algorithm, which can fully utilize the computational results of the existing training samples. Moreover, it is easy to produce sparse projections (Sparse KDA) with a L ₁-norm regularizer. Extensive experiments on spoken letter, handwritten digit image and face image data demonstrate the effectiveness and efficiency of the proposed algorithm.     

Face recognition by sparse discriminant analysis via joint L2,1-norm minimization

Recently, joint feature selection and subspace learning, which can perform feature selection and subspace learning simultaneously, is proposed and has encouraging ability on face recognition. In the literature, a framework of utilizing L_2,1-norm penalty term has also been presented, but some important algorithms cannot be covered, such as Fisher Linear Discriminant Analysis and Sparse Discriminant Analysis. Therefore, in this paper, we add L_2,1-norm penalty term on FLDA and propose a feasible solution by transforming its nonlinear model into linear regression type. In addition, we modify the optimization model of SDA by replacing elastic net with L_2,1-norm penalty term and present its optimization method. Experiments on three standard face databases illustrate FLDA and SDA via L_2,1-norm penalty term can significantly improve their recognition performance, and obtain inspiring results with low computation cost and for low-dimension feature.     

基于流形鉴别信息的特征选择及其结构化稀疏表示

针对启发式特征选择策略忽略了特征间相关信息导致子最优的问题, 提出一种基于流形鉴别信息的特征选择(MDFS) 算法. 该算法根据近邻信息和标签信息刻画高维数据类内和类间流形结构, 以最小化流形散度差为准则构建目标函数, 并增加结构化稀疏正则项降低特征间冗余. 通过统一框架下的特征权重迭代优化获得最优特征子集. 在ORL 库、COIL20 库、Isolet1 库上的聚类实验表明, MDFS算法选取的特征子集相比传统算法具有更高的识别准确率和归一化互信息, 验证了所提出算法的有效性.