图象检索中的ISOMAP算法与LLE算法及其比较

高维数据空间流形中有意义的低维嵌入是一个经典难题。ISOMAP是一种有效的基于流形理论的非线性降维方法,它不仅能够揭示高维数据的内在结构,还能够发现潜在的低维参数空间。ISOMAP的理论基础是假设在高维数据空间和低维参数空间存在等距映射,但并没有给出证明。而LLE算法能够实现高维输入数据点映射到一个全局低维坐标系,同时保留了邻接点之间的关系,这样,固有的几何结构就能够得到保留。LLE算法不仅能够有效地发现数据的非线性结构,同时还具有平移、旋转等不变特性。通过这2种算法的比较得出了结论,并提出了部分问题供后续探讨。     

基于流形学习的维数约简算法

介绍线性维数约简的主成分分析和多维尺度算法,描述几种经典的能发现嵌入在高维数据空间的低维光滑流形非线性维数约简算法,包括等距映射、局部线性嵌入、拉普拉斯特征映射、局部切空间排列、最大方差展开。与线性维数约简算法相比,非线性维数约简算法通过维数约简能够发现不同类型非线性高维数据的本质特征。     

几种流形学习算法的比较研究

如何发现高维数据空间流形中有意义的低维嵌入信息是流形学习的主要目的。目前,大部分流形学习算法都是用于非线性维数约简或是数据可视化的,如等距映射（Isomap）,局部线性嵌入算法（LLE）,拉普拉斯特征映射算（laplacian Eigenmap）等等,文章对这三种流形学习算法进行实验分析与比较,目的在于了解这几种流形学习算法的特点,以便更好地进行数据的降维与分析。     

流形距离与压缩感知核稀疏投影的局部线性嵌入算法

流形学习算法的目的是发现嵌入在高维数据空间中的低维表示,现有的流形学习算法对邻域参数k和噪声比较敏感。针对此问题,文中提出一种流形距离与压缩感知核稀疏投影的局部线性嵌入算法,其核心思想是集成局部线性嵌入算法对高维流形结构数据的降维有效性与压缩感知核稀疏投影的强鉴别性,以实现高效有降噪流形学习。首先,在选择各样本点的近邻域时,采用流形距离代替欧氏距离度量数据间相似度的方法,创建能够正确反映流形内部结构的邻域图,解决以欧氏距离作为相似性度量时对邻域参数的敏感。其次,利用压缩感知核稀疏投影作为从高维观测空间到低维嵌入空间的映射,增强算法的鉴别性。最后,利用Matlab工具对实验数据集进行仿真,进一步验证所提算法的有效性。     

基于Isomap的流形结构重建方法

已有的流形学习方法仅能建立点对点的降维嵌入,而未建立高维数据流形空间与低维表示空间之间的相互映射.此缺陷已限制了流形学习方法在诸多数据挖掘问题中的进一步应用.针对这一问题,文中提出了两种新型高效的流形结构重建算法:快速算法与稳健算法.其均以经典的Isomap方法内在运行机理为出发点,进而推导出高维流形空间与低维表示空间之间双向的显式映射函数关系,基于此函数即可实现流形映射的有效重建.理论分析与实验结果证明,所提算法在计算速度、噪音敏感性、映射表现等方面相对已有方法具有明显优势.     

基于局部线性嵌入算法的汉语数字语音识别

语音信号转换到频域后维数较高,流行学习方法可以自主发现高维数据中潜在低维结构的规律性,提出采用流形学习的方法对高维数据降维来进行汉语数字语音识别。采用流形学习中的局部线性嵌入算法提取语音频域上高维数据的低维流形结构特征,再将低维数据输入动态时间规整识别器进行识别。仿真实验结果表明,采用局部线性嵌入算法的汉语数字语音识别相较于常用声学特征MFCC维数要少,识别率提高了1.2%,有效提高了识别速度。     

基于鲁棒的全局流形学习方法

非线性降维在数据挖掘、机器学习、图像分析和计算机视觉等领域应用广泛。等距映射算法(Isomap)是一种全局流形学习方法,能有效地学习等距流形的“低维嵌入”,但它对数据中的离群样本点缺乏鲁棒性。针对这种情况,该文提出一种离群点检测方法,基于Isomap的基本思想,给出一种鲁棒的全局流形学习方法,提高Isomap处理离群样本点的能力。数值实验表明了该方法的有效性。     

基于KIsomap特征提取的软测量建模方法

来自化工生产过程的数据大多具有非线性和高维性,对数据进行特征提取是软测量建模过程的必要环节。流形学习作为一种非线性维数约简方法,可以获得高维数据在低维空间的嵌入。针对流形学习中的等距映射法(Isomap)鲁棒性差、拓扑稳定性不好等缺点,通过常数偏移的方法构造核矩阵,形成核等距映射法(KIsomap),提高了Isomap算法的鲁棒性和拓扑稳定性。运用一种将K近邻与ε-半径法相结合的方法构造邻域图,基于核等距映射法(KIsomap)对数据进行特征提取,并建立高斯过程回归软测量模型,提高了模型的泛化能力与学习效率。将该方法应用于某双酚A装置的软测量建模,仿真结果表明相比于其他特征提取的软测量建模方法,该方法具有更高的估计精度和学习效率。     

一种在源数据稀疏情况下的数据降维算法

流形学习方法是根据流形的定义提出的一种非线性数据降维方法,主要思想是发现嵌入在高维数据空间的低维光滑流形。从分析基于流形学习理论的局部线性嵌入算法入手,针对传统的局部线性嵌入算法在源数据稀疏时会失效的缺点,提出了基于局部线性逼近思想的流形学习算法,并在S-曲线上采样测试取得良好降维效果。     

基于等维度独立多流形的DC-ISOMAP算法

流形学习已经成为机器学习与数据挖掘领域中一个重要的研究课题.目前的流形学习算法都假设所研究的高维数据存在于同一个流形上,并不能支持或者应用于大量存在的采样于多流形上的高维数据.针对等维度的独立多流形DC-ISOMAP算法,首先通过从采样密集点开始扩展切空间的方法将多流形准确分解为单个流形,并逐个计算其低维嵌入,然后基于各子流形间的内部位置关系将其低维嵌入组合起来,得到最终的嵌入结果.实验结果表明,该算法在人造数据和实际的人脸图像数据上都能有效地计算出高维数据的低维嵌入结果.     

Phoneme recognition using an adaptive supervised manifold learning algorithm

To effectively handle speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space, in this paper, an adaptive supervised manifold learning algorithm based on locally linear embedding (LLE) for nonlinear dimensionality reduction is proposed to extract the low-dimensional embedded data representations for phoneme recognition. The proposed method aims to make the interclass dissimilarity maximized, while the intraclass dissimilarity minimized in order to promote the discriminating power and generalization ability of the low-dimensional embedded data representations. The performance of the proposed method is compared with five well-known dimensionality reduction methods, i.e., principal component analysis, linear discriminant analysis, isometric mapping (Isomap), LLE as well as the original supervised LLE. Experimental results on three benchmarking speech databases, i.e., the Deterding database, the DARPA TIMIT database, and the ISOLET E-set database, demonstrate that the proposed method obtains promising performance on the phoneme recognition task, outperforming the other used methods.     

新的连续自编码网络流形学习研究

发现高维观测数据空间的低维流形结构,是流形学习的主要目标。在前人利用神经网络进行非线性降维的基础上,提出一种新的连续自编码(Continuous Autoencoder,C-Autoencoder)网络,该方法特别采用CRBM(Continuous Restricted Boltzmann Machine)的网络结构,通过训练具有多个中间层的双向深层神经网络可将高维连续数据转换成低维嵌套并继而重构高维连续数据。特别地,这种连续自编码网络可以提供高维连续数据空间和低维嵌套结构的双向映射,不仅有效解决了大多数非线性降维方法所不具备的逆向映射问题,而且特别适用于高维连续数据的降维和重构。将C-Autoencoder用于人工连续数据的实验表明,C-Au-toencoder不仅能发现嵌入在高维连续数据中的非线性流形结构,也能有效地从低维嵌套中恢复原始高维连续数据。     

A comparative study of nonlinear manifold learning methods for cancer microarray data classification

The paper presents an empirical comparison of the most prominent nonlinear manifold learning techniques for dimensionality reduction in the context of high-dimensional microarray data classification. In particular, we assessed the performance of six methods: isometric feature mapping, locally linear embedding, Laplacian eigenmaps, Hessian eigenmaps, local tangent space alignment and maximum variance unfolding. Unlike previous studies on the subject, the experimental framework adopted in this work properly extends to dimensionality reduction the supervised learning paradigm, by regarding the test set as an out-of-sample set of new points which are excluded from the manifold learning process. This in order to avoid a possible overestimate of the classification accuracy which may yield misleading comparative results. The different empirical approach requires the use of a fast and effective out-of-sample embedding method for mapping new high-dimensional data points into an existing reduced space. To this aim we propose to apply multi-output kernel ridge regression, an extension of linear ridge regression based on kernel functions which has been recently presented as a powerful method for out-of-sample projection when combined with a variant of isometric feature mapping. Computational experiments on a wide collection of cancer microarray data sets show that classifiers based on Isomap, LLE and LE were consistently more accurate than those relying on HE, LTSA and MVU. In particular, under different experimental conditions LLE-based classifier emerged as the most effective method whereas Isomap algorithm turned out to be the second best alternative for dimensionality reduction.     

一种快速映射Isomap算法

传统的Isomap算法仅侧重于当前数据的分析,不能提供由高维空间到低维空间的快速直接映射,因此无法用于特征提取和高维数据检索.针对这一问题,文中提出一种基于Isornap的快速数据检索算法.该算法能够快速得到新样本的低维嵌入坐标,并基于此坐标检索与输入样本最相似的参考样本.在典型测试集上的实验结果表明,该算法在实现新样本到低维流形快速映射的同时,能较好保留样本的近邻关系.     

Dimensionality reduction-based spoken emotion recognition

To improve effectively the performance on spoken emotion recognition, it is needed to perform nonlinear dimensionality reduction for speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space. In this paper, a new supervised manifold learning algorithm for nonlinear dimensionality reduction, called modified supervised locally linear embedding algorithm (MSLLE) is proposed for spoken emotion recognition. MSLLE aims at enlarging the interclass distance while shrinking the intraclass distance in an effort to promote the discriminating power and generalization ability of low-dimensional embedded data representations. To compare the performance of MSLLE, not only three unsupervised dimensionality reduction methods, i.e., principal component analysis (PCA), locally linear embedding (LLE) and isometric mapping (Isomap), but also five supervised dimensionality reduction methods, i.e., linear discriminant analysis (LDA), supervised locally linear embedding (SLLE), local Fisher discriminant analysis (LFDA), neighborhood component analysis (NCA) and maximally collapsing metric learning (MCML), are used to perform dimensionality reduction on spoken emotion recognition tasks. Experimental results on two emotional speech databases, i.e. the spontaneous Chinese database and the acted Berlin database, confirm the validity and promising performance of the proposed method.     

Image Manifolds which are Isometric to Euclidean Space

Recently, the Isomap procedure [10] was proposed as a new way to recover a low-dimensional parametrization of data lying on a low-dimensional submanifold in high-dimensional space. The method assumes that the submanifold, viewed as a Riemannian submanifold of the ambient high-dimensional space, is isometric to a convex subset of Euclidean space. This naturally raises the question: what datasets can reasonably be modeled by this condition? In this paper, we consider a special kind of image data: families of images generated by articulation of one or several objects in a scene—for example, images of a black disk on a white background with center placed at a range of locations. The collection of all images in such an articulation family, as the parameters of the articulation vary, makes up an articulation manifold, a submanifold of L ². We study the properties of such articulation manifolds, in particular, their lack of differentiability when the images have edges. Under these conditions, we show that there exists a natural renormalization of geodesic distance which yields a well-defined metric. We exhibit a list of articulation models where the corresponding manifold equipped with this new metric is indeed isometric to a convex subset of Euclidean space. Examples include translations of a symmetric object, rotations of a closed set, articulations of a horizon, and expressions of a cartoon face. The theoretical predictions from our study are borne out by empirical experiments with published Isomap code. We also note that in the case where several components of the image articulate independently, isometry may fail; for example, with several disks in an image avoiding contact, the underlying Riemannian manifold is locally isometric to an open, connected, but not convex subset of Euclidean space. Such a situation matches the assumptions of our recently-proposed Hessian Eigenmaps procedure, but not the original Isomap procedure.     

Nonlinear Dimensionality Reduction by Locally Linear Inlaying

High-dimensional data is involved in many fields of information processing. However, sometimes, the intrinsic structures of these data can be described by a few degrees of freedom. To discover these degrees of freedom or the low-dimensional nonlinear manifold underlying a high-dimensional space, many manifold learning algorithms have been proposed. Here we describe a novel algorithm, locally linear inlaying (LLI), which combines simple geometric intuitions and rigorously established optimality to compute the global embedding of a nonlinear manifold. Using a divide-and-conquer strategy, LLI gains some advantages in itself. First, its time complexity is linear in the number of data points, and hence LLI can be implemented efficiently. Second, LLI overcomes problems caused by the nonuniform sample distribution. Third, unlike existing algorithms such as isometric feature mapping (Isomap), local tangent space alignment (LTSA), and locally linear coordination (LLC), LLI is robust to noise. In addition, to evaluate the embedding results quantitatively, two criteria based on information theory and Kolmogorov complexity theory, respectively, are proposed. Furthermore, we demonstrated the efficiency and effectiveness of our proposal by synthetic and real-world data sets.      

Unsupervised manifold learning based on multiple feature spaces

Manifold learning is a well-known dimensionality reduction scheme which can detect intrinsic low-dimensional structures in non-linear high-dimensional data. It has been recently widely employed in data analysis, pattern recognition, and machine learning applications. Isomap is one of the most promising manifold learning algorithms, which extends metric multi-dimensional scaling by using approximate geodesic distance. However, when Isomap is conducted on real-world applications, it may have some difficulties in dealing with noisy data. Although many applications represent a special sample by multiple feature vectors in different spaces, Isomap employs samples in unique observation space. In this paper, two extended versions of Isomap to multiple feature spaces problem, namely fusion of dissimilarities and fusion of geodesic distances, are presented. We have employed the advantages of several spaces and depicted the Euclidean distance on learned manifold that is more compatible to the semantic distance. To show the effectiveness and validity of the proposed method, some experiments have been carried out on the application of shape analysis on MPEG7 CE Part B and Fish data sets.