子空间半监督Fisher判别分析

Fisher判别分析寻找一个使样本数据类间散度与样本数据类内散度比值最大的子空间, 是一种很流行的监督式特征降维方法. 标注样本数据所属的类别通常需要大量的人工, 消耗大量的时间, 付出昂贵的成本. 为了解决同时利用有类别信息的样本数据和没有类别信息的样本数据用于寻找降维子空间的问题, 我们提出了一种子空间半监督Fisher判别分析方法. 子空间半监督Fisher判别分析寻找这样一个子空间, 这个子空间即保留了从有类别信息的样本数据中学习的类别判别结构, 也保留了从有类别信息的样本数据和没有类别信息的样本数据中学习的样本结构信息. 我们还推导了基于核的子空间半监督Fisher判别分析方法. 通过人脸识别实验验证了本文算法的有效性.     

半监督的局部保留投影降维方法

针对现有数据降维算法不能同时利用标记样本和无标记样本的问题,提出一种半监督局部保留投影降维方法。定义类间相似度和类内相似度,同时最大化类间分离度、最小化类内分离度,保持样本总体结构和局部结构,从而提高数据降维的效果。在人工数据集、UCI数据库和Olivetti人脸库中的测试结果表明,该方法的识别率较高。     

局部敏感的半监督鉴别分析方法

为解决图像处理中的高维特征在模式分类中带来的问题,提出一种基于半监督学习理论的数据降维方法,称为局部敏感的半监督鉴别分析算法.为能够发现局部的流形结构,算法寻找一个能够最小化类内距离的同时最大化类间距离的投影,并且在最优化过程中充分利用无标签数据,控制局部邻域的散度.在人脸识别数据库和行为数据库中的测试结果表明了该算法是有效的.     

基于核的半监督的局部保留投影降维方法

为能有效捕捉数据的非线性特征,特提出一种新的非线性数据降维算法——核半监督局部保留投影(KSSLPP)。该方法利用标记样本的标记信息及所有训练样本的结构重新定义了类间相似度和类内相似度,然后将原始数据映射到高维核空间,在核空间中最大化类间分离度,最小化类内分离度。该方法在核空间保持了数据的局部结构和全局结构,以及数据的标签信息。在Olivetti人脸库和UCI数据库中的对比实验验证了该算法的有效性。     

基于矩阵指数变换的边界Fisher分析

边界Fisher分析是一种经典的有监督线性降维方法,被广泛用于高维数据的模式分类.由于边界Fisher分析算法中涉及到矩阵求逆的运算,在数值计算中会产生矩阵的奇异性问题,尤其当样本的个数小于样本的维数时,导致所谓的"小样本问题".采用主成分分析方法对样本数据进行预处理可以克服奇异性问题,然而可能会损失样本的某些判别信息.针对此不足之处,根据矩阵指数的非奇异性,对边界Fisher分析中的散度矩阵进行矩阵指数变换,从而克服了矩阵求逆中的奇异性问题.理论分析表明,该方法等价于零空间上的边界Fisher分析,有效利用了类内散度矩阵的零空间上的信息,因此其判别能力得到了增强.数据可视化和人脸识别实验表明,该方法可以有效挖掘样本中潜在的判别特性,提高分类性能.     

MFASSC:基于间隔Fisher分析的半监督聚类方法

针对高维数据的聚类问题,提出一种基于间隔Fisher分析(MFA)的半监督聚类算法。该算法首先使用已标记样本进行MFA映射,得到投影矩阵W后,再利用求得的投影方法对未标记样本进行降维;然后在低维空间引入基于约束的球形K-means(PCSKM)算法对降维后的数据进行半监督聚类,根据第一次的聚类结果,交替进行降维与聚类操作,直到算法收敛为止。该算法利用监督信息有效地集成了数据降维和半监督聚类。实验结果表明,该方法能够有效处理高维数据,同时能提高聚类性能。     

基于张量的半监督判别分析算法

边界Fisher判别分析算法因采用一维向量表示而无法很好保持图像的空间几何结构,且无法利用大量未标记样本信息.为此,提出一种基于张量的半监督判别分析算法.采用二维张量表示人脸空间中的样本图像,揭示流形的内在几何结构,利用有判别信息的标记样本和大量未标记样本,使数据在投影空间的类间分离度最大,同时保证高维空间中不相邻的点在低维空间中也不相邻.在PIE和FERET人脸库上的实验结果表明,该算法能够获得较高的识别率.     

基于Fisher判别分析的增量式非负矩阵分解算法

增量式非负矩阵分解算法是基于子空间降维技术的无监督增量学习方法.文中将Fisher判别分析思想引入增量式非负矩阵分解中,提出基于Fisher判别分析的增量式非负矩阵分解算法.首先,利用初始样本训练的先验信息,通过索引矩阵对新增系数矩阵进行初始化赋值.然后,将增量式非负矩阵分解算法的目标函数改进为批量式的增量学习算法,在此基础上施加类间散度最大和类内散度最小的约束.最后,采用乘性迭代的方法计算分解后的因子矩阵.在ORL、Yale B和PIE等3个不同规模人脸数据库上的实验验证文中算法的有效性.     

相异度导引的有监督鉴别分析方法

提出了相异度导引的有监督鉴别分析方法（D-SDA）。结合模式局部信息和全局信息,定义了类内散度权重矩阵[RW]和类间散度权重矩阵[RB],分别表示类内样本的相异度、类间样本的相异度。由[RW]、[RB]导出类内散度矩阵[SW]和类间散度矩阵[SB],根据Fisher鉴别准则函数确定最优变换矩阵。在YALE和AR人脸图像库上的实验验证了这一算法的有效性。     

半监督的稀疏保持二维边界Fisher分析降维算法

针对样本集中类别标签样本不足的问题,提出一种半监督的稀疏保持二维边界fisher分析降维算法.首先利用图像像素间的空间结构信息,基于图像矩阵进行降维;然后设计类内散度矩阵和类间散度矩阵,以保持样本间的类内紧凑性和类间分离性;最后通过稀疏保持对特征间的稀疏重构性加以约束,所获得的稀疏重构权重保持了局部几何结构,而且也包含了自然鉴别信息.在YALE,ORL和AR人脸数据库上的实验结果表明,该算法具有很好的分类和识别性能.     

间距判别投影及其在表情识别中的应用

针对全局降维方法判别信息不足,局部降维方法对邻域关系的判定存在缺陷的问题,提出一种新的基于间距的降维方法——间距判别投影（MDP）。首先,根据类的中心均值的异类近邻关系定义描述类边缘的边界向量;在这个基础上,MDP重新定义类间离散度矩阵,同时,使用全局的方法构造类内离散度矩阵;然后,MDP借鉴判别分析思想建立衡量类间距的准则,并通过类间距最大化增强样本在投影空间中的可分性。对MDP在人脸表情数据库JAFFE和Extended Cohn-Kanade上进行表情识别实验,并且跟传统的降维方法主成分分析（PCA）、最大间距准则（MMC）和边界Fisher分析（MFA）进行对比,实验结果表明,所提算法能够有效提取更具区分性的低维特征,比其他几种方法分类精度更高。     

Extensions of LDA by PCA mixture model and class-wise features

Linear discriminant analysis (LDA) is a data discrimination technique that seeks transformation to maximize the ratio of the between-class scatter and the within-class scatter. While it has been successfully applied to several applications, it has two limitations, both concerning the underfitting problem. First, it fails to discriminate data with complex distributions since all data in each class are assumed to be distributed in the Gaussian manner. Second, it can lose class-wise information, since it produces only one transformation over the entire range of classes. We propose three extensions of LDA to overcome the above problems. The first extension overcomes the first problem by modelling the within-class scatter using a PCA mixture model that can represent more complex distribution. The second extension overcomes the second problem by taking different transformation for each class in order to provide class-wise features. The third extension combines these two modifications by representing each class in terms of the PCA mixture model and taking different transformation for each mixture component. It is shown that all our proposed extensions of LDA outperform LDA concerning classification errors for synthetic data classification, hand-written digit recognition, and alphabet recognition.     

Effective multiplicative updates for non-negative discriminative learning in multimodal dimensionality reduction

Fisher discriminant analysis gives the unsatisfactory results if points in the same class have within-class multimodality and fails to produce the non-negativity of projection vectors. In this paper, we focus on the newly formulated within and between-class scatters based supervised locality preserving dimensionality reduction problem and propose an effective dimensionality reduction algorithm, namely, Multiplicative Updates based non-negative Discriminative Learning (MUNDL), which optimally seeks to obtain two non-negative embedding transformations with high preservation and discrimination powers for two data sets in different classes such that nearby sample pairs in the original space compact in the learned embedding space, under which the projections of the original data in different classes can be appropriately separated from each other. We also show that MUNDL can be easily extended to nonlinear dimensionality reduction scenarios by employing the standard kernel trick. We verify the feasibility and effectiveness of MUNDL by conducting extensive data visualization and classification experiments. Numerical results on some benchmark UCI and real-world datasets show the MUNDL method tends to capture the intrinsic local and multimodal structure characteristics of the given data and outperforms some established dimensionality reduction methods, while being much more efficient.     

Fisher Difference Discriminant Analysis: Determining the Effective Discriminant Subspace Dimensions for Face Recognition

In most manifold learning based subspace discriminant analysis algorithms, how to construct the local neighborhood graphs and determine the effective discriminant subspace dimensions in applications are difficult but important problems. In this paper, we propose a novel supervised subspace learning method called Fisher Difference Discriminant Analysis (FDDA) for linear dimensionality reduction. FDDA introduces the local soft scatter to characterize the distributions of the data set. By combining Fisher criterion and difference criterion together, FDDA obtains the optimal discriminant subspace, on which a large margin between different classes is provided for classification. Eigenvalue analysis shows that the effective discriminant subspace dimensions of FDDA can be automatically determined by the number of positive eigenvalues and are robust to noise and invariant to rotations, rescalings and translations of the data. Comprehensive comparison and extensive experiments show that FDDA is superior to some state-of-the-art techniques in face recognition.     

基于正交SRDA和SRKDA的人脸识别

利用正交投影技术进行降维可以更好地保留与度量结构有关的信息, 提高人脸识别性能。在谱回归判别分析(SRDA)和谱回归核判别分析(SRKDA)的基础上, 提出正交SRDA(OSRDA)和正交SRKDA(OSRKDA)降维算法。首先, 给出基于Cholesky分解求解正交鉴别矢量集的方法, 然后, 通过该方法对SRDA和SRKDA投影向量作正交化处理。其简单、容易实现而且克服了迭代计算正交鉴别矢量集的方法不适应于谱回归(SR)降维的缺点。ORL、Yale和PIE库上的实验结果表明了算法的有效性和效率, 在有效降维的同时能进一步提高鉴别能力。     

基于边界判别投影的数据降维

为了提取具有较好判别性能的低维特征,提出了一种新的有监督的线性降维算法——边界判别投影,即,最小化同类样本间的最大距离,最大化异类样本间的最小距离,同时保持数据流形的几何形状.与经典的基于边界定义的算法相比,边界判别投影可以较好地保持数据流形的几何结构和判别结构等全局特性,可避免小样本问题,具有较低的计算复杂度,可应用于超高维的大数据降维.人脸数据集上的实验结果表明,边界判别分析是一种有效的降维算法,可应用于大数据上的特征提取.     

Linear discriminant analysis with worst between-class separation and average within-class compactness

Linear discriminant analysis (LDA) is one of the most popular supervised dimensionality reduction (DR) techniques and obtains discriminant projections by maximizing the ratio of average-case between-class scatter to average-case within-class scatter. Two recent discriminant analysis algorithms (DAS), minimal distance maximization (MDM) and worst-case LDA (WLDA), get projections by optimizing worst-case scatters. In this paper, we develop a new LDA framework called LDA with worst between-class separation and average within-class compactness (WSAC) by maximizing the ratio of worst-case between-class scatter to average-case within-class scatter. This can be achieved by relaxing the trace ratio optimization to a distance metric learning problem. Comparative experiments demonstrate its effectiveness. In addition, DA counterparts using the local geometry of data and the kernel trick can likewise be embedded into our framework and be solved in the same way.     

Graph embedding discriminant analysis for face recognition

This paper develops a supervised discriminant technique, called graph embedding discriminant analysis (GEDA), for dimensionality reduction of high-dimensional data in small sample size problems. GEDA can be seen as a linear approximation of a multimanifold-based learning framework in which nonlocal property is taken into account besides the marginal property and local property. GEDA seeks to find a set of perfect projections that not only can impact the samples of intraclass and maximize the margin of interclass, but also can maximize the nonlocal scatter at the same time. This characteristic makes GEDA more intuitive and more powerful than linear discriminant analysis (LDA) and marginal fisher analysis (MFA). The proposed method is applied to face recognition and is examined on the Yale, ORL and AR face image databases. The experimental results show that GEDA consistently outperforms LDA and MFA when the training sample size per class is small.     

适用于小样本的双邻接图判别分析算法

作为一种常用的降维方法,适用于小样本的监督化拉普拉斯判别分析方法通过使用图嵌入的判别近邻分析得到了很好的降维效果。但该方法在构建近邻图时,在K近邻中寻找同类和异类样本点存在数据不平衡问题;此外,在优化该方法的目标函数时,没有全面考虑到类间信息,从而会在一定程度上降低该方法的性能。针对以上两个问题,本文提出了适用于小样本的双邻接图判别分析方法。首先该方法分别在同类和异类样本中找出K个近邻点,然后使用这K个类内近邻点和K个类间近邻点来构造双邻接图,这样可以确保邻接图中既有同类样本点也有异类样本点,且数目相同。然后该方法在目标函数的推导结果中加入了类间拉普拉斯散度矩阵,从而使优化得到的投影矩阵融入更多的类间信息。在Yale和ORL人脸数据集上进行实验,并与同类方法相比,结果表明本文提出的适用于小样本的双邻接图判别分析方法能够得到更好的降维效果。