Incremental nonlinear dimensionality reduction by manifold learning

Understanding the structure of multidimensional patterns, especially in unsupervised cases, is of fundamental importance in data mining, pattern recognition, and machine learning. Several algorithms have been proposed to analyze the structure of high-dimensional data based on the notion of manifold learning. These algorithms have been used to extract the intrinsic characteristics of different types of high-dimensional data by performing nonlinear dimensionality reduction. Most of these algorithms operate in a "batch" mode and cannot be efficiently applied when data are collected sequentially. In this paper, we describe an incremental version of ISOMAP, one of the key manifold learning algorithms. Our experiments on synthetic data as well as real world images demonstrate that our modified algorithm can maintain an accurate low-dimensional representation of the data in an efficient manner.     

流形学习与非线性回归结合的头部姿态估计

流形学习的目的是发现非线性数据的内在结构,可用于非线性降维。广义回归网络是人工神经网络的一种,可用于非线性回归。基于流形学习和非线性回归,提出了用于解决头部姿态估计的ManiNLR方法。该方法首先用流形学习对图像数据进行降维,然后用非线性回归的方法将数据映射到线性可分空间,利用非线性回归的结果对人脸的头部姿态进行估计。实验结果表明,ManiNLR算法能够较好地估计图像中的头部姿态,并具有较快的速度和较高的鲁棒性。     

流形学习中非线性维数约简方法概述

较为详细地回顾了流形学习中非线性维数约简方法,分析了它们各自的优势和不足.与传统的线性维数约简方法相比较,可以发现非线性高维数据的本质维数,有利于进行维数约简和数据分析.最后展望了流形学习中非线性维数方法的未来研究方向,期望进一步拓展流形学习的应用领域.     

Semisupervised learning of hierarchical latent trait models for data visualization

Recently, we have developed the hierarchical generative topographic mapping (HGTM), an interactive method for visualization of large high-dimensional real-valued data sets. We propose a more general visualization system by extending HGTM in three ways, which allows the user to visualize a wider range of data sets and better support the model development process. 1) We integrate HGTM with noise models from the exponential family of distributions. The basic building block is the latent trait model (LTM). This enables us to visualize data of inherently discrete nature, e.g., collections of documents, in a hierarchical manner. 2) We give the user a choice of initializing the child plots of the current plot in either interactive, or automatic mode. In the interactive mode, the user selects "regions of interest", whereas in the automatic mode, an unsupervised minimum message length (MML)-inspired construction of a mixture of LTMs is employed. The unsupervised construction is particularly useful when high-level plots are covered with dense clusters of highly overlapping data projections, making it difficult to use the interactive mode. Such a situation often arises when visualizing large data sets. 3) We derive general formulas for magnification factors in latent trait models. Magnification factors are a useful tool to improve our understanding of the visualization plots, since they can highlight the boundaries between data clusters. We illustrate our approach on a toy example and evaluate it on three more complex real data sets.     

A comparative study of nonlinear manifold learning methods for cancer microarray data classification

The paper presents an empirical comparison of the most prominent nonlinear manifold learning techniques for dimensionality reduction in the context of high-dimensional microarray data classification. In particular, we assessed the performance of six methods: isometric feature mapping, locally linear embedding, Laplacian eigenmaps, Hessian eigenmaps, local tangent space alignment and maximum variance unfolding. Unlike previous studies on the subject, the experimental framework adopted in this work properly extends to dimensionality reduction the supervised learning paradigm, by regarding the test set as an out-of-sample set of new points which are excluded from the manifold learning process. This in order to avoid a possible overestimate of the classification accuracy which may yield misleading comparative results. The different empirical approach requires the use of a fast and effective out-of-sample embedding method for mapping new high-dimensional data points into an existing reduced space. To this aim we propose to apply multi-output kernel ridge regression, an extension of linear ridge regression based on kernel functions which has been recently presented as a powerful method for out-of-sample projection when combined with a variant of isometric feature mapping. Computational experiments on a wide collection of cancer microarray data sets show that classifiers based on Isomap, LLE and LE were consistently more accurate than those relying on HE, LTSA and MVU. In particular, under different experimental conditions LLE-based classifier emerged as the most effective method whereas Isomap algorithm turned out to be the second best alternative for dimensionality reduction.     

流形学习及维数约简在数据隐私保护中的应用

采用流形学习及维数约简方法可以有效保护敏感数据。针对交通事故黑点的敏感数据挖掘中隐私保护问题,提出了综合应用等距变换和微分流形两种算法来提高原始数据保密程度的方法,采用基于旋转的等距变换扰乱数据,用Laplacian Eigenmap对高维数据进行非线性降维,在保留数据内在几何结构的同时,进一步扰乱数据。该方法有效地应用于交通事故黑点数据隐私保护中,同时降低了原始数据的维数,便于后续的数据挖掘与分析。     

Riemannian manifold learning

Recently, manifold learning has been widely exploited in pattern recognition, data analysis, and machine learning. This paper presents a novel framework, called Riemannian manifold learning (RML), based on the assumption that the input high-dimensional data lie on an intrinsically low-dimensional Riemannian manifold. The main idea is to formulate the dimensionality reduction problem as a classical problem in Riemannian geometry, i.e., how to construct coordinate charts for a given Riemannian manifold? We implement the Riemannian normal coordinate chart, which has been the most widely used in Riemannian geometry, for a set of unorganized data points. First, two input parameters (the neighborhood size k and the intrinsic dimension d) are estimated based on an efficient simplicial reconstruction of the underlying manifold. Then, the normal coordinates are computed to map the input high-dimensional data into a low-dimensional space. Experiments on synthetic data as well as real world images demonstrate that our algorithm can learn intrinsic geometric structures of the data, preserve radial geodesic distances, and yield regular embeddings.     

Adaptive manifold learning

Manifold learning algorithms seek to find a low-dimensional parameterization of high-dimensional data. They heavily rely on the notion of what can be considered as local, how accurately the manifold can be approximated locally, and, last but not least, how the local structures can be patched together to produce the global parameterization. In this paper, we develop algorithms that address two key issues in manifold learning: 1) the adaptive selection of the local neighborhood sizes when imposing a connectivity structure on the given set of high-dimensional data points and 2) the adaptive bias reduction in the local low-dimensional embedding by accounting for the variations in the curvature of the manifold as well as its interplay with the sampling density of the data set. We demonstrate the effectiveness of our methods for improving the performance of manifold learning algorithms using both synthetic and real-world data sets.     

Stereo music visualization through manifold harmonics

Music visualizations are nowadays included with virtually any media player. They usually rely on harmonic analysis of each sound channel, which automatically generate parameters for procedural image generation. However, only few music visualizations make use of 3d shapes. This paper proposes to use spectral mesh processing techniques, here manifold harmonics, to produce 3d stereo music visualization. The images are generated from 3d models by deforming an initial shape, mapping the sound frequencies to the mesh harmonics. A symmetry criterion is introduced to enhance the stereo effects on the deformed shape. A concise representation of the frequency mapping is proposed to allow for an animated gallery interface with genetic reproduction. Such galleries let the user quickly navigate between visual effects. Rendering such animated galleries in real time is a challenging task, since it requires computing and rendering the deformed shapes at a very high rate. This paper introduces a direct GPU implementation of manifold harmonics filters, which allows the displaying of the animated galleries.     

Pointwise manifold regularization for semi-supervised learning

Manifold regularization(MR)provides a powerful framework for semi-supervised classification using both the labeled and unlabeled data.It constrains that similar instances over the manifold graph should share similar classification out-puts according to the manifold assumption.It is easily noted that MR is built on the pairwise smoothness over the manifold graph,i.e.,the smoothness constraint is implemented over all instance pairs and actually considers each instance pair as a single operand.However,the smoothness can be pointwise in nature,that is,the smoothness shall inherently occur“everywhereto relate the behavior of each point or instance to that of its close neighbors.Thus in this paper,we attempt to de-velop a pointwise MR(PW_MR for short)for semi-supervised learning through constraining on individual local instances.In this way,the pointwise nature of smoothness is preserved,and moreover,by considering individual instances rather than instance pairs,the importance or contribution of individual instances can be introduced.Such importance can be described by the confidence for correct prediction,or the local density,for example.PW.MR provides a different way for implementing manifold smoothness Finally,empirical results show the competitiveness of PW_MR compared to pairwise MR.     

基于流形学习的极端学习机及其在人脸识别中的应用

极端学习机以其快速高效和良好的泛化能力在模式识别领域得到了广泛应用,然而现有的ELM及其改进算法并没有充分考虑到数据维数对ELM分类性能和泛化能力的影响,当数据维数过高时包含的冗余属性及噪音点势必降低ELM的泛化能力,针对这一问题本文提出一种基于流形学习的极端学习机,该算法结合维数约减技术有效消除数据冗余属性及噪声对ELM分类性能的影响,为验证所提方法的有效性,实验使用普遍应用的图像数据,实验结果表明本文所提算法能够显著提高ELM的泛化性能。     

流形学习概述

流形学习是一种新的非监督学习方法,近年来引起越来越多机器学习和认知科学工作者的重视.为了加深对流形学习的认识和理解,该文由流形学习的拓扑学概念入手,追溯它的发展过程.在明确流形学习的不同表示方法后,针对几种主要的流形算法,分析它们各自的优势和不足,然后分别引用Isomap和LLE的应用示例.结果表明,流形学习较之于传统的线性降维方法,能够有效地发现非线性高维数据的本质维数,利于进行维数约简和数据分析.最后对流形学习未来的研究方向做出展望,以期进一步拓展流形学习的应用领域.     

Sequential manifold learning for efficient churn prediction

Nowadays, thanks to the rapid evolvement of information technology, an explosively large amount of information with very high-dimensional features for customers is being accumulated in companies. These companies, in turn, are exerting every effort to develop more efficient churn prediction models for managing customer relationships effectively. In this paper, a novel method is proposed to deal with a high-dimensional large data set for constructing better churn prediction models. The proposed method starts by partitioning a data set into small-sized data subsets, and applies sequential manifold learning to reduce high-dimensional features and give consistent results for combined data subsets. The performance of the constructed churn prediction model using the proposed method is tested using an E-commerce data set by comparing it with other existing methods. The proposed method works better and is much faster for high-dimensional large data sets without the need for retraining the original data set to reduce the dimensions of new test samples.     

Quantitative Measures for Cartogram Generation Techniques

Cartograms are used to visualize geographically distributed data by scaling the regions of a map (e.g., US states) such that their areas are proportional to some data associated with them (e.g., population). Thus the cartogram computation problem can be considered as a map deformation problem where the input is a planar polygonal map M and an assignment of some positive weight for each region. The goal is to create a deformed map M′, where the area of each region realizes the weight assigned to it (no cartographic error) while the overall map remains readable and recognizable (e.g., the topology, relative positions and shapes of the regions remain as close to those before the deformation as possible). Although several such measures of cartogram quality are well‐known, different cartogram generation methods optimize different features and there is no standard set of quantitative metrics. In this paper we define such a set of seven quantitative measures, designed to evaluate how faithfully a cartogram represents the desired weights and to estimate the readability of the final representation. We then study several cartogram‐generation algorithms and compare them in terms of these quantitative measures.     

Angle-based embedding quality assessment method for manifold learning

Neural Computing and Applications - Manifold learning (ML) is a research topic of great interest in the field of machine learning that aims to determine the appropriate low-dimensional embeddings...     

Variational Bayesian learning of nonlinear hidden state-space models for model predictive control

This paper studies the identification and model predictive control in nonlinear hidden state-space models. Nonlinearities are modelled with neural networks and system identification is done with variational Bayesian learning. In addition to the robustness of control, the stochastic approach allows for various control schemes, including combinations of direct and indirect controls, as well as using probabilistic inference for control. We study the noise-robustness, speed, and accuracy of three different control schemes as well as the effect of changing horizon lengths and initialisation methods using a simulated cart–pole system. The simulations indicate that the proposed method is able to find a representation of the system state that makes control easier especially under high noise.     

流形学习算法综述

流形学习算法作为一种新的维数降维方法工具,其目标是发现嵌入在高维数据空间中的低维流形结构,并给出一个有效的低维表示。目前,流形学习已成为模式识别、机器学习和数据挖掘领域的研究热点问题。介绍了流形学习的基本思想、一些最新研究成果及其算法分析,并提出和分析了有待进一步研究的问题。     

Image retrieval using nonlinear manifold embedding

The huge number of images on the Web gives rise to the content-based image retrieval (CBIR) as the text-based search techniques cannot cater to the needs of precisely retrieving Web images. However, CBIR comes with a fundamental flaw: the semantic gap between high-level semantic concepts and low-level visual features. Consequently, relevance feedback is introduced into CBIR to learn the subjective needs of users. However, in practical applications the limited number of user feedbacks is usually overwhelmed by the large number of dimensionalities of the visual feature space. To address this issue, a novel semi-supervised learning method for dimensionality reduction, namely kernel maximum margin projection (KMMP) is proposed in this paper based on our previous work of maximum margin projection (MMP). Unlike traditional dimensionality reduction algorithms such as principal component analysis (PCA) and linear discriminant analysis (LDA), which only see the global Euclidean structure, KMMP is designed for discovering the local manifold structure. After projecting the images into a lower dimensional subspace, KMMP significantly improves the performance of image retrieval. The experimental results on Corel image database demonstrate the effectiveness of our proposed nonlinear algorithm.     

Assess deep learning models for Egyptian exchange prediction using nonlinear artificial neural networks

Neural Computing and Applications - Financial analysis of the stock market using the historical data is the exigent demand in business and academia. This work explores the efficiency of three deep...     

Embedding new observations via sparse-coding for non-linear manifold learning

Non-linear dimensionality reduction techniques are affected by two critical aspects: (i) the design of the adjacency graphs, and (ii) the embedding of new test data—the out-of-sample problem. For the first aspect, the proposed solutions, in general, were heuristically driven. For the second aspect, the difficulty resides in finding an accurate mapping that transfers unseen data samples into an existing manifold. Past works addressing these two aspects were heavily parametric in the sense that the optimal performance is only achieved for a suitable parameter choice that should be known in advance.