针对图像序列的谱深度学习算法

为了更好地理解图像序列的隐藏深度信息,需要分析数据的隐藏结构。目前,多采用谱流形学习算法学习高维采样数据的低维嵌入坐标,从而获取数据的隐藏结构。谱流形学习算法一般是基于所研究的高维数据分布在单个流形上的前提假设,并不支持图像序列中存在的多流形结构。结合图像序列的结构特点,提出了一种针对图像序列的谱深度学习算法(spectral deep learning,SDL)。通过建立混合多流形模型,保持流形局部变化的平滑和连续,利用流形对齐建立层次流形的映射关系,得到图像序列的深度低维嵌入坐标。最后通过实验证明了算法在混合多流形数据集和图像序列数据集上的有效性。     

高维数据流形的低维嵌入及嵌入维数研究

发现高维数据空间流形中有意义的低维嵌入是一个经典难题.Isomap是提出的一种有效的基于流形理论的非线性降维方法,它不仅能够揭示高维观察数据的内在结构,还能够发现潜在的低维参教空间.Isomap的理论基础是假设在高维数据空间和低维参数空间存在等距映射,但并没有进行证明.首先给出了高维数据的连续流形和低维参数空间之间的等距映射存在性证明,然后区分了嵌入空间维数、高维数据空间的固有维数和流形维数,并证明存在环状流形高维数据空间的参数空间维数小于嵌入空间维数.最后提出一种环状流形的发现算法,判断高维数据空间是否存在环状流形,进而估计其固有维教及潜在空间维数.在多姿态三维对象的实验中证明了算法的有效性,并得到正确的低维参数空间.     

一种在源数据稀疏情况下的数据降维算法

流形学习方法是根据流形的定义提出的一种非线性数据降维方法,主要思想是发现嵌入在高维数据空间的低维光滑流形。从分析基于流形学习理论的局部线性嵌入算法入手,针对传统的局部线性嵌入算法在源数据稀疏时会失效的缺点,提出了基于局部线性逼近思想的流形学习算法,并在S-曲线上采样测试取得良好降维效果。     

基于局部线性逼近的流形学习算法

流形学习方法是根据流形的定义提出的一种非线性数据降维方法,主要思想是发现嵌入在高维数据空间的低维光滑流形.局部线性嵌入算法是应用比较广泛的一种流形学习方法,传统的局部线性嵌入算法的一个主要缺点就是在处理稀疏源数据时会失效,而实际应用中很多情况还要面对处理源数据稀疏的问题.在分析局部线性嵌入算法的基础上提出了基于局部线性逼近思想的流形学习算法,其通过采用直接估计梯度值的方法达到局部线性逼近的目的,从而实现高维非线性数据的维数约简,最后在S-曲线上进行稀疏采样测试取得良好降维效果.     

一种在源数据稀疏情况下的流形学习算法研究

传统的流形学习算法能有效地学习出高维采样数据的低维嵌入坐标,但也存在一些不足,如不能处理稀疏的样本数据.针对这些缺点,提出了一种基于局部映射的直接求解线性嵌入算法(Solving Directly Linear Embedding,简称SDLE).通过假定低维流形的整体嵌入函数,将流形映射赋予局部光滑的约束,应用核方法将高维空间的坐标投影到特征空间,最后构造出在低维空间的全局坐标.SDLE算法解决了在源数据稀疏情况下的非线性维数约简问题,这是传统的流形学习算法没有解决的问题.通过实验说明了SDLE算法研究的有效性.     

黎曼法坐标流形学习扩展算法

LOGMAP是最近提出的一种黎曼流形学习算法,它能够有效地学习出高维数据的低维嵌入坐标.然而该算法只能处理单类数据的流形学习问题,当存在多类数据时往往不能得到理想的嵌入结果.为解决这个问题,提出了一种扩展的LOGMAP算法(Extended LOGMA PAlgorithm,简称ELOGMAP).该算法通过计算全局基准点所在类到其他类的最短距离找出各类的局部基准点,然后逐个计算各类数据相对于局部基准点的局部黎曼法坐标,最后通过扩展的全局基准点与局部基准点之间测地距离关系得到多类数据的整体嵌入坐标.实验结果验证了该算法在处理多类数据流形学习上的有效性.     

基于直接估计梯度思想的数据降维算法

高维非线性数据的降维处理对于计算机完成高复杂度的数据源分析是非常重要的。从拓扑学角度分析,维数约简的过程是挖掘嵌入在高维数据中的低维线性或非线性的流形。该文在局部嵌入思想的流形学习算法的基础上,提出直接估计梯度值的方法,从而达到局部线性误差逼近最小化,实现高维非线性数据的维数约简,并在Swiss roll曲线上采样测试取得了良好的降维效果。     

基于局部平滑性的通用增量流形学习算法

目前大多数流形学习算法无法获取高维输入空间到低维嵌入空间的映射,无法处理新增数据,因此无增量学习能力。而已有的增量流形学习算法大多是通过扩展某一特定的流形学习算法使其具备增量学习能力,不具有通用性。针对这一问题,提出了一种通用的增量流形学习(GIML)算法。该方法充分考虑流形的局部平滑性这一本质特征,利用局部主成分分析法来提取数据集的局部平滑结构,并寻找包含新增样本点的局部平滑结构到对应训练数据的低维嵌入坐标的最佳变换。最后GIML算法利用该变换计算新增样本点的低维嵌入坐标。在人工数据集和实际图像数据集上进行了系统而广泛的比较实验,实验结果表明GIML算法是一种高效通用的增量流形学习方法,且相比当前主要的增量算法,能更精确地获取增量数据的低维嵌入坐标。     

基于局部线性嵌入算法的汉语数字语音识别

语音信号转换到频域后维数较高,流行学习方法可以自主发现高维数据中潜在低维结构的规律性,提出采用流形学习的方法对高维数据降维来进行汉语数字语音识别。采用流形学习中的局部线性嵌入算法提取语音频域上高维数据的低维流形结构特征,再将低维数据输入动态时间规整识别器进行识别。仿真实验结果表明,采用局部线性嵌入算法的汉语数字语音识别相较于常用声学特征MFCC维数要少,识别率提高了1.2%,有效提高了识别速度。     

一种改进的局部线性嵌入算法

局部线性嵌入算法(Local Linear Embedding,简称LLE)是一种非线性流形学习算法,能有效地学习出高维采样数据的低维嵌入坐标,但也存在一些不足,如不能处理稀疏的样本数据.针对这些缺点,提出了一种基于局部映射的线性嵌入算法(Local Project Linear Embedding,简称LPLE).通过假定目标空间的整体嵌入函数,重新构造样本点的局部邻域特征向量,最后将问题归结为损失矩阵的特征向量问题从而构造出目标空间的全局坐标.LPLE算法解决了传统LLE算法在源数据稀疏情况下的不能有效进行降维的问题,这也是其他传统的流形学习算法没有解决的.通过实验说明了LPLE算法研究的有效性和意义.     

基于选择聚类集成的相似流形学习算法

流形学习是当今最重要的研究方向之一.约简维度的选择影响着流形学习方法的性能.当约简维度恰好是本征维度时,更容易发现原始数据的内在性质.然而,本征维度估计仍然是流形学习的一个研究难点.在此基础上,提出了一种新的无监督方法,即基于选择聚类集成的相似流形学习(SML-SCE)算法,避免了对本征维度的估计,并且性能表现良好.SML-SCE利用改进的层次平衡K-means(MBKHK)方法生成具有代表性的锚点,高效地构造相似度矩阵.随后计算得到了多个不同维度下的相似低维嵌入,这些低维嵌入是对原始数据的不同表示,而且不同低维嵌入之间的多样性有利于集成学习.因此,SML-SCE采用选择性聚类集成方法作为结合策略.对于通过K-means聚类得到的相似低维嵌入的聚类结果,采用聚类间的归一化互信息(NMI)作为权重的衡量标准.最后,舍弃权重较低的聚类,采用基于权重的选择性投票方案,得到最终的聚类结果.在多个数据集的大量实验结果表明了该方法的有效性.     

Feature extraction using maximum variance sparse mapping

In this paper, a multiple sub-manifold learning method–oriented classification is presented via sparse representation, which is named maximum variance sparse mapping. Based on the assumption that data with the same label locate on a sub-manifold and different class data reside in the corresponding sub-manifolds, the proposed algorithm can construct an objective function which aims to project the original data into a subspace with maximum sub-manifold distance and minimum manifold locality. Moreover, instead of setting the weights between any two points directly or obtaining those by a square optimal problem, the optimal weights in this new algorithm can be approached using L1 minimization. The proposed algorithm is efficient, which can be validated by experiments on some benchmark databases.     

Locality preserving embedding for face and handwriting digital recognition

Most supervised manifold learning-based methods preserve the original neighbor relationships to pursue the discriminating power. Thus, structure information of the data distributions might be neglected and destroyed in low-dimensional space in a certain sense. In this paper, a novel supervised method, called locality preserving embedding (LPE), is proposed to feature extraction and dimensionality reduction. LPE can give a low-dimensional embedding for discriminative multi-class sub-manifolds and preserves principal structure information of the local sub-manifolds. In LPE framework, supervised and unsupervised ideas are combined together to learn the optimal discriminant projections. On the one hand, the class information is taken into account to characterize the compactness of local sub-manifolds and the separability of different sub-manifolds. On the other hand, at the same time, all the samples in the local neighborhood are used to characterize the original data distributions and preserve the structure in low-dimensional subspace. The most significant difference from existing methods is that LPE takes the distribution directions of local neighbor data into account and preserves them in low-dimensional subspace instead of only preserving the each local sub-manifold’s original neighbor relationships. Therefore, LPE optimally preserves both the local sub-manifold’s original neighborhood relationships and the distribution direction of local neighbor data to separate different sub-manifolds as far as possible. The criterion, similar to the classical Fisher criterion, is a Rayleigh quotient in form, and the optimal linear projections are obtained by solving a generalized Eigen equation. Furthermore, the framework can be directly used in semi-supervised learning, and the semi-supervised LPE and semi-supervised kernel LPE are given. The proposed LPE is applied to face recognition (on the ORL and Yale face databases) and handwriting digital recognition (on the USPS database). The experimental results show that LPE consistently outperforms classical linear methods, e.g., principal component analysis and linear discriminant analysis, and the recent manifold learning-based methods, e.g., marginal Fisher analysis and constrained maximum variance mapping.     

Efficient Energy-Preserving Exponential Integrators for Multi-component Hamiltonian Systems

Graph-Laplacians and their spectral embeddings play an important role in multiple areas of machine learning. This paper is focused on graph-Laplacian dimension reduction for the spectral clustering of data as a primary application, however, it can also be applied in data mining, data manifold learning, etc. Spectral embedding provides a low-dimensional parametrization of the data manifold which makes the subsequent task (e.g., clustering with k-means or any of its approximations) much easier. However, despite reducing the dimensionality of data, the overall computational cost may still be prohibitive for large data sets due to two factors. First, computing the partial eigendecomposition of the graph-Laplacian typically requires a large Krylov subspace. Second, after the spectral embedding is complete, one still has to operate with the same number of data points, which may ruin the efficiency of the approach. For example, clustering of the embedded data is typically performed with various relaxations of k-means which computational cost scales poorly with respect to the size of data set. Also, they become prone to getting stuck in local minima, so their robustness depends on the choice of initial guess. In this work, we switch the focus from the entire data set to a subset of graph vertices (target subset). We develop two novel algorithms for such low-dimensional representation of the original graph that preserves important global distances between the nodes of the target subset. In particular, it allows to ensure that target subset clustering is consistent with the spectral clustering of the full data set if one would perform such. That is achieved by a properly parametrized reduced-order model (ROM) of the graph-Laplacian that approximates accurately the diffusion transfer function of the original graph for inputs and outputs restricted to the target subset. Working with a small target subset reduces greatly the required dimension of Krylov subspace and allows to exploit the conventional algorithms (like approximations of k-means) in the regimes when they are most robust and efficient. This was verified in the numerical clustering experiments with both synthetic and real data. We also note that our ROM approach can be applied in a purely transfer-function-data-driven way, so it becomes the only feasible option for extremely large graphs that are not directly accessible. There are several uses for our algorithms. First, they can be employed on their own for representative subset clustering in cases when handling the full graph is either infeasible or simply not required. Second, they may be used for quality control. Third, as they drastically reduce the problem size, they enable the application of more sophisticated algorithms for the task under consideration (like more powerful approximations of k-means based on semi-definite programming (SDP) instead of the conventional Lloyd’s algorithm). Finally, they can be used as building blocks of a multi-level divide-and-conquer type algorithm to handle the full graph. The latter will be reported in a separate article.

     

边界流形嵌入在特征提取中的应用

样本点的边界信息对于分类具有重要意义。针对于边界Fisher分析（MFA）和局部敏感判别分析（LSDA）构造本征图和惩罚图所利用的样本点边界信息,在一些情况下并不能很好地表征不同类样本点的可分性,提出了一种新的图嵌入降维算法——边界流形嵌入（MME）。MME算法根据样本点的标签信息,寻找距离每个样本点最近的异类边界子流形,再返回本类中寻找距离异类边界子流形最近的同类边界子流形,从而定义出不同类样本间密切联系的同类边界邻域和异类边界邻域。通过最大化所有成对的边界子流形之间的距离,MME算法可以得到更具有鉴别意义的低维特征空间。同时,MME算法能将徘徊在边界的离群点收入到边界邻域里,这对减弱离群点给算法带来的负面的影响有一定的帮助。在人脸数据库上的实验结果表明了MME算法提取的低维特征能够提升分类的准确率。     

基于流形结构的人脸民族特征研究

人脸民族特征选取与分析是人脸识别与人类学重要研究方向之一.本文建立了中国三个民族人脸数据库,通过流形结构来研究和分析人脸的民族特征.首先,在体质人类学定义的人脸几何特征指标进行流形分析,未形成按语义分布的子流形.因此本文将人脸特征扩至全部组合的长度、角度和比例特征进行分析,利用mRMR算法对2926个长度特征、21万余个角度特征、427万个比例特征中冗余特征进行筛选,加上人类学指标及混合筛选的数据集共形成5个数据集.利用LPP、Isomap、LE、PCA和LDA等流形方法分析5数据集,其中的4个数据集都形成了民族语义的子流形分布.为验证筛选特征指标的有效性,本文利用分类算法J48、SVM、RBF network、Naive Bayes、Bayes network在Weka平台对数据集以族群语义作为类别进行交叉验证实验,实验结果表明混合特征的人脸数据集族群分类平均准确率最高,且比例特征分类指标优于其他特征数据集.本文通过大量实验揭示了民族人脸数据可在子空间内形成按民族语义分布的子流形结构.中国三个民族人脸特征在低维空间存在不同民族语义的子流形,通过流形分析和特征筛选构建的人脸测量指标不仅可为人脸族群分析提供方法,同时也将丰富和补充体质人类学的相关研究工作.     

自适应主动半监督学习方法

主动学习从大量无标记样本中挑选样本交给专家标记.现有的批抽样主动学习算法主要受3个限制:(1)一些主动学习方法基于单选择准则或对数据、模型设定假设,这类方法很难找到既有不确定性又有代表性的未标记样本;(2)现有批抽样主动学习方法的性能很大程度上依赖于样本之间相似性度量的准确性,例如预定义函数或差异性衡量;(3)噪声标签问题一直影响批抽样主动学习算法的性能.提出一种基于深度学习批抽样的主动学习方法.通过深度神经网络生成标记和未标记样本的学习表示和采用标签循环模式,使得标记样本与未标记样本建立联系,再回到相同标签的标记样本.这样同时考虑了样本的不确定性和代表性,并且算法对噪声标签具有鲁棒性.在提出的批抽样主动学习方法中,算法使用的子模块函数确保选择的样本集合具有多样性.此外,自适应参数的优化,使得主动学习算法可以自动平衡样本的不确定性和代表性.将提出的主动学习方法应用到半监督分类和半监督聚类中,实验结果表明,所提出的主动学习方法的性能优于现有的一些先进的方法.     

一种深度自监督聚类集成算法

针对聚类集成中一致性函数设计问题,本文提出一种深度自监督聚类集成算法。该算法首先根据基聚类划分结果采用加权连通三元组算法计算样本之间的相似度矩阵,基于相似度矩阵表达邻接关系,将基聚类由特征空间中的数据表示变换至图数据表示;在此基础上,基聚类的一致性集成问题被转化为对基聚类图数据表示的图聚类问题。为此,本文利用图神经网络构造自监督聚类集成模型,一方面采用图自动编码器学习图的低维嵌入,依据低维嵌入似然分布估计聚类集成的目标分布;另一方面利用聚类集成目标对低维嵌入过程进行指导,确保模型获得的图低维嵌入与聚类集成结果是一致最优的。在大量数据集上进行了仿真实验,结果表明本文算法相比HGPA、CSPA和MCLA等算法可以进一步提高聚类集成结果的准确性。     

On partitioned controllability of switched linear systems

When a switched linear system is not completely controllable, the controllability subspace is not enough to describe the controllability of the system over whole state space. In this case the state space can be divided into two or three control-invariant sub-manifolds, which form a control-related partition of the state space. This paper investigates when each component is a controllable sub-manifold. First, we consider when a sub-manifold is controllable for no control input case. Then the results are used to produce a necessary and sufficient condition assuring the controllability of the partitioned control-invariant sub-manifolds of a class of switched linear systems. An example is given to demonstrate the effectiveness of the results.     

A unified framework for semi-supervised dimensionality reduction

In practice, many applications require a dimensionality reduction method to deal with the partially labeled problem. In this paper, we propose a semi-supervised dimensionality reduction framework, which can efficiently handle the unlabeled data. Under the framework, several classical methods, such as principal component analysis (PCA), linear discriminant analysis (LDA), maximum margin criterion (MMC), locality preserving projections (LPP) and their corresponding kernel versions can be seen as special cases. For high-dimensional data, we can give a low-dimensional embedding result for both discriminating multi-class sub-manifolds and preserving local manifold structure. Experiments show that our algorithms can significantly improve the accuracy rates of the corresponding supervised and unsupervised approaches.