新的连续自编码网络流形学习研究

发现高维观测数据空间的低维流形结构,是流形学习的主要目标。在前人利用神经网络进行非线性降维的基础上,提出一种新的连续自编码（Continuous Autoencoder,C-Autoencoder）网络,该方法特别采用CRBM（Continuous Restricted Boltzmann Machine）的网络结构,通过训练具有多个中间层的双向深层神经网络可将高维连续数据转换成低维嵌套并继而重构高维连续数据。特别地,这种连续自编码网络可以提供高维连续数据空间和低维嵌套结构的双向映射,不仅有效解决了大多数非线性降维方法所不具备的逆向映射问题,而且特别适用于高维连续数据的降维和重构。将C-Autoencoder用于人工连续数据的实验表明,C-Autoencoder不仅能发现嵌入在高维连续数据中的非线性流形结构,也能有效地从低维嵌套中恢复原始高维连续数据。     

基于直接估计梯度思想的数据降维算法

高维非线性数据的降维处理对于计算机完成高复杂度的数据源分析是非常重要的。从拓扑学角度分析,维数约简的过程是挖掘嵌入在高维数据中的低维线性或非线性的流形。该文在局部嵌入思想的流形学习算法的基础上,提出直接估计梯度值的方法,从而达到局部线性误差逼近最小化,实现高维非线性数据的维数约简,并在Swiss roll曲线上采样测试取得了良好的降维效果。     

基于Isomap的流形结构重建方法

已有的流形学习方法仅能建立点对点的降维嵌入,而未建立高维数据流形空间与低维表示空间之间的相互映射.此缺陷已限制了流形学习方法在诸多数据挖掘问题中的进一步应用.针对这一问题,文中提出了两种新型高效的流形结构重建算法:快速算法与稳健算法.其均以经典的Isomap方法内在运行机理为出发点,进而推导出高维流形空间与低维表示空间之间双向的显式映射函数关系,基于此函数即可实现流形映射的有效重建.理论分析与实验结果证明,所提算法在计算速度、噪音敏感性、映射表现等方面相对已有方法具有明显优势.     

Reconstruction and analysis of multi-pose face images based on nonlinear dimensionality reduction

Locally linear embedding (LLE) is a nonlinear dimensionality reduction method proposed recently. It can reveal the intrinsic distribution of data, which cannot be provided by classical linear dimensionality reduction methods. The application of LLE, however, is limited because of its lack of a parametric mapping between the observation and the low-dimensional output. And the large data set to be reduced is necessary. In this paper, we propose methods to establish the process of mapping from low-dimensional embedded space to high-dimensional space for LLE and validate their efficiency with the application of reconstruction of multi-pose face images. Furthermore, we propose that the high-dimensional structure of multi-pose face images is similar for the same kind of pose change mode of different persons. So given the structure information of data distribution which is obtained by leaning large numbers of multi-pose images in a training set, the support vector regression (SVR) method of statistical learning theory is used to learn the high-dimensional structure of someone based on small sets. The detailed learning method and algorithm are given and applied to reconstruct and synthesize face images in small set cases. The experiments prove that our idea and method is correct.     

基于流形学习的维数约简算法

介绍线性维数约简的主成分分析和多维尺度算法,描述几种经典的能发现嵌入在高维数据空间的低维光滑流形非线性维数约简算法,包括等距映射、局部线性嵌入、拉普拉斯特征映射、局部切空间排列、最大方差展开。与线性维数约简算法相比,非线性维数约简算法通过维数约简能够发现不同类型非线性高维数据的本质特征。     

基于一种连续自编码网络的图像降维和重构

针对高维连续数据的降维问题,提出一种新的非线性降维方法,称为连续自编码(Continuous autoencoder,C-autoencoder)神经网络,该方法采用限制玻耳兹曼机的连续形式(Continuous restricted Boltzmann machine,CRBM)的网络结构,通过训练具有多个中间层的双向深层神经网络将高维连续数据转换成低维嵌套并继而重构高维连续数据.这种连续自编码网络提供了高维连续数据空间和低维嵌套结构的双向映射,有效解决了大多数非线性降维方法所不具备的逆向映射问题,特别适用于高维连续数据的降维和重构.将C-autoencoder用于连续帧图像的实验表明,C-autoencoder不仅能发现嵌入在高维连续帧图像中的非线性低维结构,也能有效地从低维结构中恢复原始高维图像数据,而且还能对连续帧图像有效地进行内插重构.     

基于最近邻的随机非线性降维

针对线性降维技术应用于具有非线性结构的数据时无法得到令人满意的结果的问题,提出一种新的着重于保持高维空间局部最近邻信息的非线性随机降维算法(NNSE)。该算法首先在高维空间中通过计算样本点之间的欧氏距离找出每个样本点的最近邻点,接着在低维空间中产生一个随机的初始分布;然后通过将低维空间中的样本点不断向其最近邻点的平均位置移动,直到产生稳定的低维嵌入结果。与一种先进的非线性随机降维算法——t分布随机邻域嵌入(t-SNE)相比,NNSE算法得到的低维结果在可视化方面与t-SNE算法相差不大,但通过比较两者的量化指标可以发现,NNSE算法在保持最近邻信息方面上明显优于t-SNE算法。     

基于随机游走扩散映射的降维算法

传统数据降维算法分为线性或流形学习降维算法,但在实际应用中很难确定需要哪一类算法.设计一种综合的数据降维算法,以保证它的线性降维效果下限为主成分分析方法且在流形学习降维方面能揭示流形的数据结构.通过对高维数据构造马尔可夫转移矩阵,使越相似的节点转移概率越大,从而发现高维数据降维到低维流形的映射关系.实验结果表明,在人造...     

高维数据流形的低维嵌入及嵌入维数研究

发现高维数据空间流形中有意义的低维嵌入是一个经典难题.Isomap是提出的一种有效的基于流形理论的非线性降维方法,它不仅能够揭示高维观察数据的内在结构,还能够发现潜在的低维参教空间.Isomap的理论基础是假设在高维数据空间和低维参数空间存在等距映射,但并没有进行证明.首先给出了高维数据的连续流形和低维参数空间之间的等距映射存在性证明,然后区分了嵌入空间维数、高维数据空间的固有维数和流形维数,并证明存在环状流形高维数据空间的参数空间维数小于嵌入空间维数.最后提出一种环状流形的发现算法,判断高维数据空间是否存在环状流形,进而估计其固有维教及潜在空间维数.在多姿态三维对象的实验中证明了算法的有效性,并得到正确的低维参数空间.     

几种流形学习算法的比较研究

如何发现高维数据空间流形中有意义的低维嵌入信息是流形学习的主要目的。目前,大部分流形学习算法都是用于非线性维数约简或是数据可视化的,如等距映射（Isomap）,局部线性嵌入算法（LLE）,拉普拉斯特征映射算（laplacian Eigenmap）等等,文章对这三种流形学习算法进行实验分析与比较,目的在于了解这几种流形学习算法的特点,以便更好地进行数据的降维与分析。     

基于最优密度方向的等距映射降维算法

等距映射算法(ISOMAP)是一种典型的非线性流形降维算法,该算法可在尽量保持高维数据测地距离与低维数据空间距离对等关系的基础上实现降维.但ISOMAP容易受噪声的影响,导致数据降维后不能保持高维拓扑结构.针对这一问题,提出了一种基于最优密度方向的等距映射(ODD-ISOMAP)算法.该算法通过筛选数据的自然邻居确定每...     

Phoneme recognition using an adaptive supervised manifold learning algorithm

To effectively handle speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space, in this paper, an adaptive supervised manifold learning algorithm based on locally linear embedding (LLE) for nonlinear dimensionality reduction is proposed to extract the low-dimensional embedded data representations for phoneme recognition. The proposed method aims to make the interclass dissimilarity maximized, while the intraclass dissimilarity minimized in order to promote the discriminating power and generalization ability of the low-dimensional embedded data representations. The performance of the proposed method is compared with five well-known dimensionality reduction methods, i.e., principal component analysis, linear discriminant analysis, isometric mapping (Isomap), LLE as well as the original supervised LLE. Experimental results on three benchmarking speech databases, i.e., the Deterding database, the DARPA TIMIT database, and the ISOLET E-set database, demonstrate that the proposed method obtains promising performance on the phoneme recognition task, outperforming the other used methods.     

基于重叠片排列的流形学习算法

针对线性数据降维算法对处理非线性结构数据的降维效果不是很好,提出一种基于重叠片排列的流形学习算法,该算法根据局部的线性贴片处在非线性流形中的特性,将流形划分为线性互相重叠的局部区域贴片,且利用主成分分析方法得到局部区域贴片的低维表示,然后排列且对齐其低维坐标,以获得整体数据的低维坐标.通过仿真结果证明,基于重叠片排列的流形学习算法在应用于人脸识别和分类问题时以及在识别准确率方面要优于其他经典的流形学习算法.     

Dimensionality reduction-based spoken emotion recognition

To improve effectively the performance on spoken emotion recognition, it is needed to perform nonlinear dimensionality reduction for speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space. In this paper, a new supervised manifold learning algorithm for nonlinear dimensionality reduction, called modified supervised locally linear embedding algorithm (MSLLE) is proposed for spoken emotion recognition. MSLLE aims at enlarging the interclass distance while shrinking the intraclass distance in an effort to promote the discriminating power and generalization ability of low-dimensional embedded data representations. To compare the performance of MSLLE, not only three unsupervised dimensionality reduction methods, i.e., principal component analysis (PCA), locally linear embedding (LLE) and isometric mapping (Isomap), but also five supervised dimensionality reduction methods, i.e., linear discriminant analysis (LDA), supervised locally linear embedding (SLLE), local Fisher discriminant analysis (LFDA), neighborhood component analysis (NCA) and maximally collapsing metric learning (MCML), are used to perform dimensionality reduction on spoken emotion recognition tasks. Experimental results on two emotional speech databases, i.e. the spontaneous Chinese database and the acted Berlin database, confirm the validity and promising performance of the proposed method.     

非监督子空间学习中关联度量多维尺度分析

多维尺度分析已经在维度约减和数据挖掘领域得到了广泛应用。MDS的主要缺点是其定义在训练数据上,对于新的测试样本无法直接获得映射结果。另外,MDS基于欧氏距离度量,不适合获取相似数据中的非线性流形结构。将MDS扩展到关联度量空间,称为关联度量多维尺度分析(CMDS)。与传统MDS在训练数据中完成映射,进而缩小空间范围相比,CMDS 能够直接获得测试样本映射结果。此外,CMDS基于关联度量,能够有效学习相似数据中的非线性流形结构。理论分析表明,CMDS可以利用核方法扩展到新特征空间,解决非线性问题。实验结果表明,CMDS及其核形式KG-CMDS性能优于常用传统降维方法。     

Incremental nonlinear dimensionality reduction by manifold learning

Understanding the structure of multidimensional patterns, especially in unsupervised cases, is of fundamental importance in data mining, pattern recognition, and machine learning. Several algorithms have been proposed to analyze the structure of high-dimensional data based on the notion of manifold learning. These algorithms have been used to extract the intrinsic characteristics of different types of high-dimensional data by performing nonlinear dimensionality reduction. Most of these algorithms operate in a "batch" mode and cannot be efficiently applied when data are collected sequentially. In this paper, we describe an incremental version of ISOMAP, one of the key manifold learning algorithms. Our experiments on synthetic data as well as real world images demonstrate that our modified algorithm can maintain an accurate low-dimensional representation of the data in an efficient manner.     

基于压缩感知的社团结构深度学习方法

传统社团结构发现算法复杂度高,且只适合处理小规模低维度的社会网络数据,而无法处理大规模高维度实际网络数据。为此,提出一种基于压缩感知的社团结构深度学习方法。通过随机测量矩阵对社会网络数据进行特征降维,并使用深度信度网(DBN)对降维后的特征样本集进行无监督学习,利用带类标的小样本集进行有监督调优。仿真结果表明,随机测量方法对高维稀疏特征具有较好的降维效果,DBN对大规模数据集具有较好的处理性能,该方法适合对大规模高维度实际社会网络数据进行高效处理。     

一种新的有监督流形学习方法

提出了一种新的有监督流形学习方法,目的是提供将流形学习降维方法高效应用于有监督学习问题的全新策略.算法的核心思想是集成流形学习方法对高维流形结构数据的降维有效性与支撑向量机(SVM)在中小规模分类数据集上的优良特性实现高效有监督流形学习.算法具体实现步骤为:首先利用SVM在流形学习降维数据中选出对分类决策最重要的数据集,即支撑向量集;按标号返回可得到原空间的支撑向量集;在这个集合上再次使用SVM即可得到原空间的分类决策,从而完成有监督流形学习.在一系列人工与实际数据集上的实验验证了方法的有效性.     

Nonlinear discriminant clustering based on spectral regularization

Owing to sparseness, directly clustering high-dimensional data is still a challenge problem. Therefore, obtaining their low-dimensional compact representation by dimensional reduction is an effective method for clustering high-dimensional data. Most of existing dimensionality reduction methods, however, are developed originally for classification (such as Linear Discriminant Analysis) or recovering the geometric structure (known as manifold) of high-dimensional data (such as Locally Linear Embedding) rather than clustering purpose. Hence, a novel nonlinear discriminant clustering by dimensional reduction based on spectral regularization is proposed. The contributions of the proposed method are two folds: (1) it can obtain nonlinear low-dimensional representation that can recover the intrinsic manifold structure as well as enhance the cluster structure of the original high-dimensional data; (2) the clustering results can also be obtained in the dimensionality reduction procedure. Firstly, the desired low-dimensional coordinates are represented as linear combinations of predefined smooth vectors with respect to the data manifold, which are characterized by a weighted graph. Then, the optimal combination coefficients and the optimal cluster assignment matrix are computed by maximizing the ratio between the between-cluster scatter and the total scatter simultaneously as well as preserving the smoothness of the cluster assignment matrix with respect to the data manifold. Finally, the optimization problem is solved in an iterative procedure, which is proved to be convergent. Experiments on UCI data sets and real world data sets demonstrated the effectiveness of the proposed method for both clustering and visualization high-dimensional data set.     

基于Wasserstein距离概率分布模型的非线性降维

降维是大数据分析和可视化领域中的核心问题,其中基于概率分布模型的降维算法通过最优化高维数据模型和低维数据模型之间的代价函数来实现降维。这种策略的核心在于构建最能体现数据特征的概率分布模型。基于此,将Wasserstein距离引入降维,提出一个基于Wasserstein距离概率分布模型的非线性降维算法W-map。W-map模型在高维数据空间和其相关对应的低维数据空间建立相似的Wasserstein流,将降维转化为最小运输问题。在解决Wasserstein距离最小化的问题同时,依据数据的Wasserstein流模型在高维空间与其在低维空间相同的原则,寻找最匹配的低维数据投射。三组针对不同数据集的实验结果表明W-map相对传统概率分布模型可以产生正确性高且鲁棒性好的高维数据降维可视化结果。