Robust kernel Isomap

Isomap is one of widely used low-dimensional embedding methods, where geodesic distances on a weighted graph are incorporated with the classical scaling (metric multidimensional scaling). In this paper we pay our attention to two critical issues that were not considered in Isomap, such as: (1) generalization property (projection property); (2) topological stability. Then we present a robust kernel Isomap method, armed with such two properties. We present a method which relates the Isomap to Mercer kernel machines, so that the generalization property naturally emerges, through kernel principal component analysis. For topological stability, we investigate the network flow in a graph, providing a method for eliminating critical outliers. The useful behavior of the robust kernel Isomap is confirmed through numerical experiments with several data sets.     

An improved local tangent space alignment method for manifold learning

Principal component analysis (PCA) is widely used in recently proposed manifold learning algorithms to provide approximate local tangent spaces. However, such approximations provided by PCA may be inaccurate when local neighborhoods of the data manifold do not lie in or close to a linear subspace. Furthermore, the approximated tangent spaces can not fit the change in data distribution density. In this paper, a new method is proposed for providing faithful approximations to the local tangent spaces of a data manifold, which is proved to be more accurate than PCA. With this new method, an improved local tangent space alignment (ILTSA) algorithm is developed, which can efficiently recover the geometric structure of data manifolds even in the case when data are sparse or non-uniformly distributed. Experimental results are presented to illustrate the better performance of ILTSA on both synthetic data and image data.     

An effective double-bounded tree-connected Isomap algorithm for microarray data classification

Isometric mapping (Isomap) is a popular nonlinear dimensionality reduction technique which has shown high potential in visualization and classification. However, it appears sensitive to noise or scarcity of observations. This inadequacy may hinder its application for the classification of microarray data, in which the expression levels of thousands of genes in a few normal and tumor sample tissues are measured. In this paper we propose a double-bounded tree-connected variant of Isomap, aimed at being more robust to noise and outliers when used for classification and also computationally more efficient. It differs from the original Isomap in the way the neighborhood graph is generated: in the first stage we apply a double-bounding rule that confines the search to at most k nearest neighbors contained within an ε-radius hypersphere; the resulting subgraphs are then joined by computing a minimum spanning tree among the connected components. We therefore achieve a connected graph without unnaturally inflating the values of k and ε. The computational experiences show that the new method performs significantly better in terms of accuracy with respect to Isomap, k-edge-connected Isomap and the direct application of support vector machines to data in the input space, consistently across seven microarray datasets considered in our tests.     

NMF和Isomap相结合的图像检索新方法*

非负矩阵分解(NMF)能够提取图像的局部特征,是一种基于局部的数据挖掘方法,在一定程度上勾勒出了相关图像在基矩阵所代表空间上的分布,然而NMF并未考虑数据的内在几何结构。提出了一种新颖的基于非负矩阵分解和非线性降维方法Isomap相结合的新方法,全局的非线性降维方法Isomap能发现数据的内在结构和相关性,使高维数据在低维空间变得可视化。将本算法应用于图像检索,实验表明,该方法能够更加准确的获取信息,提高检索的准确性。     

Incremental manifold learning by spectral embedding methods

Recent years have witnessed great success of manifold learning methods in understanding the structure of multidimensional patterns. However, most of these methods operate in a batch mode and cannot be effectively applied when data are collected sequentially. In this paper, we propose a general incremental learning framework, capable of dealing with one or more new samples each time, for the so-called spectral embedding methods. In the proposed framework, the incremental dimensionality reduction problem reduces to an incremental eigen-problem of matrices. Furthermore, we present, using this framework as a tool, an incremental version of Hessian eigenmaps, the IHLLE method. Finally, we show several experimental results on both synthetic and real world datasets, demonstrating the efficiency and accuracy of the proposed algorithm.     

Gait analysis for human identification through manifold learning and HMM

With the increasing demands of visual surveillance systems, human identification at a distance has gained more attention from the researchers recently. Gait analysis can be used as an unobtrusive biometric measure to identify people at a distance without any attention of the human subjects. We propose a novel effective method for both automatic viewpoint and person identification by using only the silhouette sequence of the gait. The gait silhouettes are nonlinearly transformed into low-dimensional embedding by Gaussian process latent variable model (GP-LVM), and the temporal dynamics of the gait sequences are modeled by hidden Markov models (HMMs). The experimental results show that our method has higher recognition rate than the other methods.     

基于Isomap的流形结构重建方法

已有的流形学习方法仅能建立点对点的降维嵌入,而未建立高维数据流形空间与低维表示空间之间的相互映射.此缺陷已限制了流形学习方法在诸多数据挖掘问题中的进一步应用.针对这一问题,文中提出了两种新型高效的流形结构重建算法:快速算法与稳健算法.其均以经典的Isomap方法内在运行机理为出发点,进而推导出高维流形空间与低维表示空间之间双向的显式映射函数关系,基于此函数即可实现流形映射的有效重建.理论分析与实验结果证明,所提算法在计算速度、噪音敏感性、映射表现等方面相对已有方法具有明显优势.     

流形学习概述

流形学习是一种新的非监督学习方法,近年来引起越来越多机器学习和认知科学工作者的重视.为了加深对流形学习的认识和理解,该文由流形学习的拓扑学概念入手,追溯它的发展过程.在明确流形学习的不同表示方法后,针对几种主要的流形算法,分析它们各自的优势和不足,然后分别引用Isomap和LLE的应用示例.结果表明,流形学习较之于传统的线性降维方法,能够有效地发现非线性高维数据的本质维数,利于进行维数约简和数据分析.最后对流形学习未来的研究方向做出展望,以期进一步拓展流形学习的应用领域.     

Stochastic neighbor projection on manifold for feature extraction

This paper develops a manifold-oriented stochastic neighbor projection (MSNP) technique for feature extraction. MSNP is designed to find a linear projection for the purpose of capturing the underlying pattern structure of observations that actually lie on a nonlinear manifold. In MSNP, the similarity information of observations is encoded with stochastic neighbor distribution based on geodesic distance metric, then the same distribution is required to be hold in feature space. This learning criterion not only empowers MSNP to extract nonlinear feature through a linear projection, but makes MSNP competitive as well by reason that distribution preservation is more workable and flexible than rigid distance preservation. MSNP is evaluated in three applications: data visualization for faces image, face recognition and palmprint recognition. Experimental results on several benchmark databases suggest that the proposed MSNP provides a unsupervised feature extraction approach with powerful pattern revealing capability for complex manifold data.     

Intrinsic dimension estimation of manifolds by incising balls

Dimensionality reduction is a very important tool in data mining. Intrinsic dimension of data sets is a key parameter for dimensionality reduction. However, finding the correct intrinsic dimension is a challenging task. In this paper, a new intrinsic dimension estimation method is presented. The estimator is derived by finding the exponential relationship between the radius of an incising ball and the number of samples included in the ball. The method is compared with the previous dimension estimation methods. Experiments have been conducted on synthetic and high dimensional image data sets and on data sets of the Santa Fe time series competition, and the results show that the new method is accurate and robust.     

一种基于加权标志点的流形学习算法

近年来出现的一系列进行维数约简的非线性方法——流形学习中等距映射（Isomap）是其中的代表,该算法高效、简单,但计算复杂度较高。基于标志点（Landmark Points）的L-Isomap减少了计算复杂度,但对于标志点的选取,大都采用随机的方法,致使该算法不稳定。考虑到样本点和近邻点相对位置,将对嵌入流形影响较大的样本点赋予较高的权重。然后根据权重大小选择标志点,同时考虑标志点之间的相对位置,使得选出的标志点不会出现过度集中的现象,近似直线分布的概率也大大降低,从而保证了算法的稳定性。实验结果表明,该算法在标志点数量较少的情况下,比L-Isomap稳定,且对缺失数据的不完整流形,也能获取和Isomap相差不大的结果。     

黎曼流形的距离均方差最小降维改进算法

TRIMAP算法重新定义了图上距离的表达形式,并用近邻点对的测地距离的误差和作为衡量投影函数好坏的标准,通过这种方法可以较好地找到所需的从高维空间到低维空间转换的媒介,但是这种衡量标准不能很好地表达出TRIMAP中定义的图上距离与投影到低维空间中两点实际距离的对比关系。针对这个不足,采用了一个新的衡量标准表达式,定义一个参数m来代表对比关系,以此来解决这个缺陷,从而更好地获得最佳投影,提高识别率。实验结果表明,在ORL人脸图像的分类识别问题中获得了较好的识别性能。     

Inductive manifold learning using structured support vector machine

Most manifold learning techniques are used to transform high-dimensional data sets into low-dimensional space. In the use of such techniques, after unseen data samples are added to the data set, retraining is usually necessary. However, retraining is a time-consuming process and no guarantee of the transformation into the exactly same coordinates, thus presenting a barrier to the application of manifold learning as a preprocessing step in predictive modeling. To solve this problem, learning a mapping from high-dimensional representations to low-dimensional coordinates is proposed via structured support vector machine. After training a mapping, low-dimensional representations of unobserved data samples can be easily predicted. Experiments on several datasets show that the proposed method outperforms the existing out-of-sample extension methods.     

保持全局和局部特性的黎曼流形改进算法

黎曼流形学习(RML)是一种全局算法,但其不能较好地保持数据局部邻域的几何性质。为解决这个问题,提出一种基于黎曼流形学习（RML）的多结构算法。先对数据集进行主成分分析（PCA）投影,再构造邻域图,然后把整个数据集分为两个部分求低维嵌入坐标,对于基准点的k近邻,采用能保持其和近邻点局部性质的权值矩阵得到低维嵌入;对于其他点仍采用RML算法,使其达到既能维持数据点的全局结构,又能最大限度地保持其局部几何性质的目的。实验结果验证了该算法的有效性和实时性。     

A new quality assessment criterion for nonlinear dimensionality reduction

A new quality assessment criterion for evaluating the performance of the nonlinear dimensionality reduction (NLDR) methods is proposed in this paper. Differing from the current quality assessment criteria focusing on the local-neighborhood-preserving performance of the NLDR methods, the proposed criterion capitalizes on a new aspect, the global-structure-holding performance, of the NLDR methods. By taking both properties into consideration, the intrinsic capability of the NLDR methods can be more faithfully reflected, and hence more rational measurement for the proper selection of NLDR methods in real-life applications can be offered. The theoretical argument is supported by experiment results implemented on a series of benchmark data sets.     

Multi-output regression on the output manifold

Multi-output regression aims at learning a mapping from an input feature space to a multivariate output space. Previous algorithms define the loss functions using a fixed global coordinate of the output space, which is equivalent to assuming that the output space is a whole Euclidean space with a dimension equal to the number of the outputs. So the underlying structure of the output space is completely ignored. In this paper, we consider the output space as a Riemannian submanifold to incorporate its geometric structure into the regression process. To this end, we propose a novel mechanism, called locally linear transformation (LLT), to define the loss functions on the output manifold. In this way, currently existing regression algorithms can be improved. In particular, we propose an algorithm under the support vector regression framework. Our experimental results on synthetic and real-life data are satisfactory.     

Formulating LLE using alignment technique

LLE is a well-known method to nonlinear dimensionality reduction. In this short paper, we present an alternative way to formulate LLE. The alignment technique is exploited to align the local coordinates on the local patches of manifolds to be the global ones. The efficient computation of embedding coordinates of LLE automatically appears in the proposed framework.     

Nonlinear fault detection based on locally linear embedding

In this paper, a new nonlinear fault detection technique based on locally linear embedding （LLE） is developed. LLE can efficiently compute the low-dimensional embedding of the data with the local neighborhood structure information preserved. In this method, a data-dependent kernel matrix which can reflect the nonlinear data structure is defined. Based on the kernel matrix, the Nystrrm formula makes the mapping extended to the testing data possible. With the kernel view of the LLE, two monitoring statistics are constructed. Together with the out of sample extensions, LLE is used for nonlinear fault detection. Simulation cases were studied to demonstrate the performance of the proposed method.