基于局部平滑性的通用增量流形学习算法

目前大多数流形学习算法无法获取高维输入空间到低维嵌入空间的映射,无法处理新增数据,因此无增量学习能力。而已有的增量流形学习算法大多是通过扩展某一特定的流形学习算法使其具备增量学习能力,不具有通用性。针对这一问题,提出了一种通用的增量流形学习(GIML)算法。该方法充分考虑流形的局部平滑性这一本质特征,利用局部主成分分析法来提取数据集的局部平滑结构,并寻找包含新增样本点的局部平滑结构到对应训练数据的低维嵌入坐标的最佳变换。最后GIML算法利用该变换计算新增样本点的低维嵌入坐标。在人工数据集和实际图像数据集上进行了系统而广泛的比较实验,实验结果表明GIML算法是一种高效通用的增量流形学习方法,且相比当前主要的增量算法,能更精确地获取增量数据的低维嵌入坐标。     

基于重叠片排列的流形学习算法

针对线性数据降维算法对处理非线性结构数据的降维效果不是很好,提出一种基于重叠片排列的流形学习算法,该算法根据局部的线性贴片处在非线性流形中的特性,将流形划分为线性互相重叠的局部区域贴片,且利用主成分分析方法得到局部区域贴片的低维表示,然后排列且对齐其低维坐标,以获得整体数据的低维坐标.通过仿真结果证明,基于重叠片排列的流形学习算法在应用于人脸识别和分类问题时以及在识别准确率方面要优于其他经典的流形学习算法.     

Phoneme recognition using an adaptive supervised manifold learning algorithm

To effectively handle speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space, in this paper, an adaptive supervised manifold learning algorithm based on locally linear embedding (LLE) for nonlinear dimensionality reduction is proposed to extract the low-dimensional embedded data representations for phoneme recognition. The proposed method aims to make the interclass dissimilarity maximized, while the intraclass dissimilarity minimized in order to promote the discriminating power and generalization ability of the low-dimensional embedded data representations. The performance of the proposed method is compared with five well-known dimensionality reduction methods, i.e., principal component analysis, linear discriminant analysis, isometric mapping (Isomap), LLE as well as the original supervised LLE. Experimental results on three benchmarking speech databases, i.e., the Deterding database, the DARPA TIMIT database, and the ISOLET E-set database, demonstrate that the proposed method obtains promising performance on the phoneme recognition task, outperforming the other used methods.     

一种快速映射Isomap算法

传统的Isomap算法仅侧重于当前数据的分析,不能提供由高维空间到低维空间的快速直接映射,因此无法用于特征提取和高维数据检索.针对这一问题,文中提出一种基于Isornap的快速数据检索算法.该算法能够快速得到新样本的低维嵌入坐标,并基于此坐标检索与输入样本最相似的参考样本.在典型测试集上的实验结果表明,该算法在实现新样本到低维流形快速映射的同时,能较好保留样本的近邻关系.     

动态增殖流形学习算法

流形学习的主要目标是发现高维观测数据空间中的低维光滑流形.目前,流形学习已经成为机器学习和数据挖掘领域的研究热点.为了从高维数据流和大规模海量数据集中探索有价值的信息,迫切需要增殖地发现内在低维流形结构.但是,现有流形学习算法不具有增殖能力,并且不能有效处理海量数据集.针对这些问题,系统定义了增殖流形学习的概念,这有利于解释人脑中稳态感知流形的动态形成过程,且可以指导符合人脑增殖学习机理的流形学习算法的研究.以此为指导原则,提出了动态增殖流形学习算法,并在实验中验证了算法的有效性.     

Supervised local spline embedding for medical diagnosis

A common difficulty of intelligent medical diagnosis is the high dimensionality of medical data. Manifold learning provides an elegant way to solve this problem by mapping the high-dimensional data into the low-dimensional embedding. However, traditional manifold learning algorithms fail to fully utilize the supervised information in medical diagnosis. To overcome this problem, in this paper we propose a novel Supervised Local Spline Embedding (SLSE) algorithm, which incorporates the supervised information into the local spline manifold embedding. SLSE not only preserves the local neighborhood structure, but also utilizes the global manifold shape through spline interpolation. Moreover, SLSE leverages the supervised information by maximizing the inter-class scatterness and minimizing the intra-class scatterness in the low-dimensional embedding. The promising experimental results on real-world medical datasets illustrate the superiority of our proposed approach in comparison with the existing popular manifold learning algorithms.

     

基于非线性流形学习的3维人体运动合成

为了实现3维人体运动的有效合成,提出了一种基于非线性流形学习的3维人体运动合成框架及算法,并可应用于方便、快捷、用户可控的3维人体运动合成。该合成算法框架先采用非线性流形降维方法将高维运动样本映射到低维流形上,同时求解其本征运动语义参数空间的表达,然后将用户在低维运动语义参数空间中交互生成的样本通过逆向映射重建得到具有新运动语义特征的3维运动序列。实验结果表明该方法不仅能够对运动物理参数（如特定关节的运动位置、物理运动特征）进行较为精确的控制,还可用于合成具有高层运动语义（运动风格）的新运动数据。与现有运动合成方法比较,该方法具有用户可控、交互性强等优点,能够应用于常见3维人体运动数据的高效生成。     

Incremental Alignment Manifold Learning

A new manifold learning method,called incremental alignment method(IAM),is proposed for nonlinear dimensionality reduction of high dimensional data with intrinsic low dimensionality.The main idea is to incrementally align low-dimensional coordinates of input data patch-by-patch to iteratively generate the representation of the entire dataset. The method consists of two major steps,the incremental step and the alignment step.The incremental step incrementally searches neighborhood patch to be aligned in the next step,and the alignment step iteratively aligns the low-dimensional coordinates of the neighborhood patch searched to generate the embeddings of the entire dataset.Compared with the existing manifold learning methods,the proposed method dominates in several aspects:high efficiency,easy out-of-sample extension,well metric-preserving,and averting of the local minima issue.All these properties are supported by a series of experiments performed on the synthetic and real-life datasets.In addition,the computational complexity of the proposed method is analyzed,and its efficiency is theoretically argued and experimentally demonstrated.     

Continuum Isomap for manifold learnings

Recently, the Isomap algorithm has been proposed for learning a parameterized manifold from a set of unorganized samples from the manifold. It is based on extending the classical multidimensional scaling method for dimension reduction, replacing pairwise Euclidean distances by the geodesic distances on the manifold. A continuous version of Isomap called continuum Isomap is proposed. Manifold learning in the continuous framework is then reduced to an eigenvalue problem of an integral operator. It is shown that the continuum Isomap can perfectly recover the underlying parameterization if the mapping associated with the parameterized manifold is an isometry and its domain is convex. The continuum Isomap also provides a natural way to compute low-dimensional embeddings for out-of-sample data points. Some error bounds are given for the case when the isometry condition is violated. Several illustrative numerical examples are also provided.     

Age estimation based on improved discriminative Gaussian process latent variable model

Affected by various factors (genes, living habits and so on), different people present distinct aging patterns. To discover the underlying trend of aging patterns, we propose an effective age estimation method based on DGPLVM (Discriminative Gaussian Process Latent Variable Model). DGPLVM is a kind of discriminative latent variable method for manifold learning. It discovers the low-dimensional manifold by employing a discriminative prior distribution over the latent space. DGPLVM with KFDA (Kernel Fisher Discriminant Analysis) prior has been studied and successfully applied to face verification. Different with face verification which is a two-class problem, age estimation is a linearly inseparable multi-class problem. In this paper, DGPLVM with KFDA is reformulated to get the low-dimensional representations for age estimation. After low-dimensional representations are obtained, Gaussian process regression model is adopted to find the age regressor mapping low-dimensional representations to ages. Experimental results on two widely used databases FG-NET and MORPH show that reformulated DGPLVM with KFDA is a good application in age estimation and achieves comparable results to state-of-the arts.     

一种改进的局部切空间排列算法

局部切空间排列算法(local tangent space alignment,简称LTSA)是一种新的流形学习算法,能有效地学习出高维采样数据的低维嵌入坐标,但也存在一些不足,如不能处理样本数较大的样本集和新来的样本点.针对这些缺点,提出了一种基于划分的局部切空间排列算法(partitional local tangent space alignment,简称PLTSA).它建立在VQPCA(vector quantization principal component analysis)算法和LTSA     

Riemannian manifold learning

Recently, manifold learning has been widely exploited in pattern recognition, data analysis, and machine learning. This paper presents a novel framework, called Riemannian manifold learning (RML), based on the assumption that the input high-dimensional data lie on an intrinsically low-dimensional Riemannian manifold. The main idea is to formulate the dimensionality reduction problem as a classical problem in Riemannian geometry, i.e., how to construct coordinate charts for a given Riemannian manifold? We implement the Riemannian normal coordinate chart, which has been the most widely used in Riemannian geometry, for a set of unorganized data points. First, two input parameters (the neighborhood size k and the intrinsic dimension d) are estimated based on an efficient simplicial reconstruction of the underlying manifold. Then, the normal coordinates are computed to map the input high-dimensional data into a low-dimensional space. Experiments on synthetic data as well as real world images demonstrate that our algorithm can learn intrinsic geometric structures of the data, preserve radial geodesic distances, and yield regular embeddings.     

基于近邻元分析的半监督流形学习算法*

现有的大多数流形学习算法偏重保持流形的几何结构,并未考虑到样本点的标签信息,这在一定程度上限制了流形学习算法在数据分类中的应用.因此文中提出一种基于近邻元分析的半监督流形学习算法,采用近邻元分析学习距离度量矩阵,在距离度量方式下选择样本点的局部邻域点.基于距离度量方式构造样本点和邻域点的局部几何结构,并在样本点的低维嵌入坐标中保持这种局部几何结构不变.3个不同数据集上的分类实验验证了文中算法的有效性.     

基于局部线性嵌入算法的汉语数字语音识别

语音信号转换到频域后维数较高,流行学习方法可以自主发现高维数据中潜在低维结构的规律性,提出采用流形学习的方法对高维数据降维来进行汉语数字语音识别。采用流形学习中的局部线性嵌入算法提取语音频域上高维数据的低维流形结构特征,再将低维数据输入动态时间规整识别器进行识别。仿真实验结果表明,采用局部线性嵌入算法的汉语数字语音识别相较于常用声学特征MFCC维数要少,识别率提高了1.2%,有效提高了识别速度。     

一种基于边界约束的流形展开方法

在流形学习的谱方法中, 流形展开被表述为优化问题. 这些优化问题的解是退化的, 即所有的样本将被嵌入到同一个点. 为了避免退化解, 谱方法对嵌入坐标人为地强加了一个单位协方差矩阵约束. 然而, 该约束往往导致流形展开的失真非常明显. 本文提出一种新的流形学习方法, 彻底抛弃了人为的单位协方差矩阵约束. 主要思路是先对流形边界进行嵌入, 然后再求流形内部的嵌入; 流形边界的嵌入位置被确定后, 流形内部样本的嵌入位置将被边界拉开, 使得它们不会都收缩到一个点上, 从而避免了退化解的出现. 将流形边界的嵌入位置作为边界条件, 求解一个线性方程组来得到内部样本的嵌入; 该线性方程组反映了尽量保持邻近样本间距离不变的要求. 流形边界的嵌入由简化流形的嵌入求出; 为此, 本文还设计了一种流形边界检测算法以及一种流形简化算法. 与目前代表性的几种流形学习方法进行了比较实验, 结果表明了本文方法的有效性, 其展开失真比谱方法明显要小.     

新的连续自编码网络流形学习研究

发现高维观测数据空间的低维流形结构,是流形学习的主要目标。在前人利用神经网络进行非线性降维的基础上,提出一种新的连续自编码（Continuous Autoencoder,C-Autoencoder）网络,该方法特别采用CRBM（Continuous Restricted Boltzmann Machine）的网络结构,通过训练具有多个中间层的双向深层神经网络可将高维连续数据转换成低维嵌套并继而重构高维连续数据。特别地,这种连续自编码网络可以提供高维连续数据空间和低维嵌套结构的双向映射,不仅有效解决了大多数非线性降维方法所不具备的逆向映射问题,而且特别适用于高维连续数据的降维和重构。将C-Autoencoder用于人工连续数据的实验表明,C-Autoencoder不仅能发现嵌入在高维连续数据中的非线性流形结构,也能有效地从低维嵌套中恢复原始高维连续数据。     

High-dimensional MRI data analysis using a large-scale manifold learning approach

A novel manifold learning approach is presented to efficiently identify low-dimensional structures embedded in high-dimensional MRI data sets. These low-dimensional structures, known as manifolds, are used in this study for predicting brain tumor progression. The data sets consist of a series of high-dimensional MRI scans for four patients with tumor and progressed regions identified. We attempt to classify tumor, progressed and normal tissues in low-dimensional space. We also attempt to verify if a progression manifold exists—the bridge between tumor and normal manifolds. By identifying and mapping the bridge manifold back to MRI image space, this method has the potential to predict tumor progression. This could be greatly beneficial for patient management. Preliminary results have supported our hypothesis: normal and tumor manifolds are well separated in a low-dimensional space. Also, the progressed manifold is found to lie roughly between the normal and tumor manifolds.     

Subspace manifold learning with sample weights

Subspace manifold learning represents a popular class of techniques in statistical image analysis and object recognition. Recent research in the field has focused on nonlinear representations; locally linear embedding (LLE) is one such technique that has recently gained popularity. We present and apply a generalization of LLE that introduces sample weights. We demonstrate the application of the technique to face recognition, where a model exists to describe each face’s probability of occurrence. These probabilities are used as weights in the learning of the low-dimensional face manifold. Results of face recognition using this approach are compared against standard nonweighted LLE and PCA. A significant improvement in recognition rates is realized using weighted LLE on a data set where face occurrences follow the modeled distribution.     

基于流形的表情分解算法

提出了基于流形的表情分解算法。首先,运用保局投影将图像投影到低维的表情流形子空间,再在流形子空间里对它们进行高阶奇异值分解,最后在个人子空间和表情子空间里完成人脸和表情识别。该算法用流形学习解决了高阶奇异值分解中的图像特征值提取问题,用高阶奇异值分解解决了流形表情识别中个人模式影响表情识别的问题。是一种流形学习与高阶奇异值分解优势互补的算法。在CMU-AMP和JAFFE人脸库上的实验表明,该算法对人脸和表情识别都十分有效。     

Dimensionality reduction-based spoken emotion recognition

To improve effectively the performance on spoken emotion recognition, it is needed to perform nonlinear dimensionality reduction for speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space. In this paper, a new supervised manifold learning algorithm for nonlinear dimensionality reduction, called modified supervised locally linear embedding algorithm (MSLLE) is proposed for spoken emotion recognition. MSLLE aims at enlarging the interclass distance while shrinking the intraclass distance in an effort to promote the discriminating power and generalization ability of low-dimensional embedded data representations. To compare the performance of MSLLE, not only three unsupervised dimensionality reduction methods, i.e., principal component analysis (PCA), locally linear embedding (LLE) and isometric mapping (Isomap), but also five supervised dimensionality reduction methods, i.e., linear discriminant analysis (LDA), supervised locally linear embedding (SLLE), local Fisher discriminant analysis (LFDA), neighborhood component analysis (NCA) and maximally collapsing metric learning (MCML), are used to perform dimensionality reduction on spoken emotion recognition tasks. Experimental results on two emotional speech databases, i.e. the spontaneous Chinese database and the acted Berlin database, confirm the validity and promising performance of the proposed method.