基于非线性流形学习和支持向量机的文本分类算法

为解决文本自动分类问题,提出一种流形学习和支持向量机相结合的文本分类算法(LLE-LSSVM)。LLE-LSSVM算法利用非线性流形学习算法LEE对高维文本特征进行非线性降维,挖掘出特征内在规律与本征信息,从而得到低维特征空间,然后将其输入到LSSVM中进行学习,同时利用混沌粒子群算法对LSSVM参数进行优化,建立文本分类模型。仿真实验结果表明,LLE-LSSVM算法提高了文本分类准确率,减少了分类运行时间,是一种有效的文本分类算法。     

高维数据流形的低维嵌入及嵌入维数研究

发现高维数据空间流形中有意义的低维嵌入是一个经典难题.Isomap是提出的一种有效的基于流形理论的非线性降维方法,它不仅能够揭示高维观察数据的内在结构,还能够发现潜在的低维参教空间.Isomap的理论基础是假设在高维数据空间和低维参数空间存在等距映射,但并没有进行证明.首先给出了高维数据的连续流形和低维参数空间之间的等距映射存在性证明,然后区分了嵌入空间维数、高维数据空间的固有维数和流形维数,并证明存在环状流形高维数据空间的参数空间维数小于嵌入空间维数.最后提出一种环状流形的发现算法,判断高维数据空间是否存在环状流形,进而估计其固有维教及潜在空间维数.在多姿态三维对象的实验中证明了算法的有效性,并得到正确的低维参数空间.     

针对环状流形数据的非线性降维

近年来出现了多种新型的非线性降维方法,且在一些应用中体现出良好的效果.然而,当面对球体、柱体等环状流形产生的非线性流形数据时,这些方法往往会失效.针对这一问题,提出了针对环状流形数据的环结构检测算法与非线性降维方法.理论上,基于目前极受关注的Isomap降维方法的运行原理,给出了一个判断环状流形的充要条件;算法上利用所得的判断定理,制订了基于数据的环状流形检测算法:最后基于所找到的环结构,利用极坐标展开的思想设计了针对环状流形数据的非线性降维策略.针对一系列典型环状流形数据集的仿真实验结果表明,与其他流形学习降维方法相比,该方法对环状流形数据进行降维具有显著优势.     

摄像机镜头畸变的鲁棒校正方法

针对计算机视觉中的镜头畸变问题,设计一种鲁棒的校正方法.该方法基于空间直线的成像特性来定义畸变测度,通过非线性优化完成畸变校正.采用微粒群全局优化算法,将传统优化方法、标准微粒群算法和基于不同策略的微粒群算法的性能进行对比.实验结果表明,带变异算子基于对位学习的微粒群算法具有较强的鲁棒性,在低噪声下,微粒群算法的校正性能优于传统算法.最后通过不同畸变程度的校正实例验证了所提出方法的有效性.     

基于微粒群算法的非线性系统模型参数估计

微粒群优化(PSO)算法是一种进化算法,包含的概念简单.本文不同于传统的非线性模型参数估计方法,将微粒群优化算法应用于非线性系统模型(NSM)的参数估计,并通过重油热解三集总模型参数估计进行PSO算法效果测试.实验结果表明:微粒群算法为非线性系统模型参数估计提供了一种新方法.     

交叉耦合非线性系统的输出调节

利用中心流形定理,给出交叉耦合非线性系统的输出调节问题可解的充要条件,定义了交 叉耦合非线性系统的零误差流形.利用幂级数展开的方法,给出该零误差流形存在的充要条 件,并讨论了该零误差流形的局部收敛性.     

一种新的局部空间排列算法

局部切空间排列算法(local tangent space alignment,LTSA)是一种经典的非线性流形学习方法,能够有效地对非线性分布数据进行降维,但它无法学习局部高曲率数据集.针对此问题,给出了描述数据集局部曲率的参数,并提出一种局部最小偏差空间排列(locally minimal deviation spacealignment,LMDSA)算法.该算法考虑到局部切空间低鲁棒性的缺陷,在计算局部最小偏差空间的同时,能够发现数据的局部高曲率现象,通过参数控制及邻域间的连接信息,减少计算局部高曲率空间的可能,进而利用空间排列技术进行降维,手工流形及真实数据集的实验证实了该算法学习局部高曲率数据集的有效性.     

基于文化微粒群算法的丙烯腈收率软测量模型

丙烯腈收率是丙烯腈装置的关键指标,如何得到丙烯腈收率是厂家很关注的研究,将新型优化算法用于丙烯腈收率软测量建模是1种较好的尝试。将新型微粒群优化算法用于同样新型的文化算法种群空间的优化,设计文化微粒群优化算法。它由种群空间和信念空间2部分组成,在种群空间和信念空间分别采用各自算法并行演化,同时,2个空间又根据一定的协议相互联系。分别将该算法和基本微粒群算法用于一些常用测试函数的优化问题;结果表明,与基本微粒群算法相比,文化微粒群算法加强了全局搜索能力,更容易收敛于全局最优解。最后将文化微粒群优化算法用于优化神经网络,构成文化微粒群神经网络,并将其应用于丙烯腈收率软测量建模。结果表明,此模型精度高,应用前景广阔。     

基于黎曼流形稀疏编码的图像检索算法

针对视觉词袋（Bag-of-visual-words,BOVW）模型直方图量化误差大的缺点,提出基于稀疏编码的图像检索算法.由于大多数图像特征属于非线性流形结构,传统稀疏编码使用向量空间对其度量必然导致不准确的稀疏表示.考虑到图像特征空间的流形结构,选择对称正定矩阵作为特征描述子,构建黎曼流形空间.利用核技术将黎曼流形结构映射到再生核希尔伯特空间,非线性流形转换为线性稀疏编码,获得图像更准确的稀疏表示.实验在Corel1000和Caltech101两个数据集上进行,与已有的图像检索算法对比,提出的图像检索算法不仅提高了检索准确率,而且获得了更好的检索性能.     

增量与演化流形学习综述

流形学习的目标是发现观测数据嵌入在高维数据空间中的低维光滑流形.近年来,在线或增量地发现内在低维流形结构成为流形学习的研究热点.从增量学习和演化学习2个方面入手,对该领域已有研究进展进行综述.增量流形学习较之传统的批量流形学习方法具有动态增量的能力,而演化流形学习能够在线地发现海量动态数据的内在规律,有利于进行维数约简和数据分析.文中对主要的增量与演化流形学习算法的基本原理、特点进行了阐述,分析了各自的优点与不足,指出了该领域的开放问题,并对进一步的研究方向进行了展望.     

基于流形正则化的文档分类算法研究

基于流形正则化框架提出一种分类算法(MI_I}RI_SC),以解决高维文档分类问题。该算法通过构建训练样 本的最近部图来佑计数据空间的几何结构并将其作为流形正则化项,结合多变量线性回归获得高维文档的低维流形 结构,并采用k近部分类器对低维流形进行分类,得到针对多类问题的分类器。该算法能够充分利用训练样本的类别 信息来帮助学习以提取有效特征。通过在Rcutcrs 21578数据集上的实验,证明该算法的分类性能和运行速度比传统 分类器有较大的提高。     

基于选择关系嵌入算法的图像检索方法

结合流形学习和相关反馈技术的图像检索方法关键是结合低层可视化信息,从少量用户反馈信息中学习用户语义,以获得语义子空间流形。为获得更真实的语义子空间,文中在区分对待低层可视化和用户反馈信息的同时,基于低层可视化信息选择学习反馈信息中的类内和类间关系,提出一种选择关系嵌入算法应用于图像检索。该方法可保留更真实的语义流形结构,从而提高在低维空间中的检索精度。实验结果表明文中方法可将图像映射到更广范围的低维空间,在反馈迭代两次之后检索精度提高最高可达16。3%。     

一种基于局部流形结构的无监督特征学习方法(英文)

Unsupervised feature selection is fundamental in statistical pattern recognition, and has drawn persistent attention in the past several decades. Recently, much work has shown that feature selection can be formulated as nonlinear dimensionality reduction with discrete constraints. This line of research emphasizes utilizing the manifold learning techniques, where feature selection and learning can be studied based on the manifold assumption in data distribution. Many existing feature selection methods such as Laplacian score, SPEC(spectrum decomposition of graph Laplacian), TR(trace ratio) criterion, MSFS(multi-cluster feature selection) and EVSC(eigenvalue sensitive criterion) apply the basic properties of graph Laplacian, and select the optimal feature subsets which best preserve the manifold structure defined on the graph Laplacian. In this paper, we propose a new feature selection perspective from locally linear embedding(LLE), which is another popular manifold learning method. The main difficulty of using LLE for feature selection is that its optimization involves quadratic programming and eigenvalue decomposition, both of which are continuous procedures and different from discrete feature selection. We prove that the LLE objective can be decomposed with respect to data dimensionalities in the subset selection problem, which also facilitates constructing better coordinates from data using the principal component analysis(PCA) technique. Based on these results, we propose a novel unsupervised feature selection algorithm,called locally linear selection(LLS), to select a feature subset representing the underlying data manifold. The local relationship among samples is computed from the LLE formulation, which is then used to estimate the contribution of each individual feature to the underlying manifold structure. These contributions, represented as LLS scores, are ranked and selected as the candidate solution to feature selection. We further develop a locally linear rotation-selection(LLRS) algorithm which extends LLS to identify the optimal coordinate subset from a new space. Experimental results on real-world datasets show that our method can be more effective than Laplacian eigenmap based feature selection methods.     

Finding representative landmarks of data on manifolds

Data-driven non-parametric models, such as manifold learning algorithms, are promising data analysis tools. However, to fit an off-training-set data point in a learned model, one must first “locate” the point in the training set. This query has a time cost proportional to the problem size, which limits the model's scalability. In this paper, we address the problem of selecting a subset of data points as the landmarks helping locate the novel points on the data manifolds. We propose a new category of landmarks defined with the following property: the way the landmarks represent the data in the ambient Euclidean space should resemble the way they represent the data on the manifold. Given the data points and the subset of landmarks, we provide procedures to test whether the proposed property presents for the choice of landmarks. If the data points are organized with a neighbourhood graph, as it is often conducted in practice, we interpret the proposed property in terms of the graph topology. We also discuss the extent to which the topology is preserved for landmark set passing our test procedure. Another contribution of this work is to develop an optimization based scheme to adjust an existing landmark set, which can improve the reliability for representing the manifold data. Experiments on the synthetic data and the natural data have been done. The results support the proposed properties and algorithms.     

Image clustering based on sparse patch alignment framework

Image clustering methods are efficient tools for applications such as content-based image retrieval and image annotation. Recently, graph based manifold learning methods have shown promising performance in extracting features for image clustering. Typical manifold learning methods adopt appropriate neighborhood size to construct the neighborhood graph, which captures local geometry of data distribution. Because the density of data points’ distribution may be different in different regions of the manifold, a fixed neighborhood size may be inappropriate in building the manifold. In this paper, we propose a novel algorithm, named sparse patch alignment framework, for the embedding of data lying in multiple manifolds. Specifically, we assume that for each data point there exists a small neighborhood in which only the points that come from the same manifold lie approximately in a low-dimensional affine subspace. Based on the patch alignment framework, we propose an optimization strategy for constructing local patches, which adopt sparse representation to select a few neighbors of each data point that span a low-dimensional affine subspace passing near that point. After that, the whole alignment strategy is utilized to build the manifold. Experiments are conducted on four real-world datasets, and the results demonstrate the effectiveness of the proposed method.     

Manifold topological multi-resolution analysis method

In this paper, two significant weaknesses of locally linear embedding (LLE) applied to computer vision are addressed: “intrinsic dimension” and “eigenvector meanings”. “Topological embedding” and “multi-resolution nonlinearity capture” are introduced based on mathematical analysis of topological manifolds and LLE. The manifold topological analysis (MTA) method is described and is based on “topological embedding”. MTA is a more robust method to determine the “intrinsic dimension” of a manifold with typical topology, which is important for tracking and perception understanding. The manifold multi-resolution analysis (MMA) method is based on “multi-resolution nonlinearity capture”. MMA defines LLE eigenvectors as features for pattern recognition and dimension reduction. Both MTA and MMA are proved mathematically, and several examples are provided. Applications in 3D object recognition and 3D object viewpoint space partitioning are also described.     

统计流形扩散核的文本分类方法

提出Dirichlet混合多项式(DCM)流形,并利用DCM流形可与正半球流形建立同胚和等距关系的性质,通过拉回映射将正半球流形的测地距离映射为DCM流形的测地距离,从而在DCM流形上建立距离度量,构建统计流形上的Dirichlet混合多项式扩散核和Dirichlet混合多项式倒排文档频率(DCMIDF)扩散核。利用WebKBTop4和20Newsgroups语料库上进行实验,DCM流形能比欧氏空间更能准确地描述文本。与多项式核支持向量机算法、,负测地距离核支持向量机算法相比,实验结果显示文中基于DCM扩散核和DCMIDF扩散核的支持向量机算法可取得良好的文本分类效果。     

A Novel Statistical Manifold Algorithm for Position Estimation

In this paper, a novel statistical manifold algorithm is proposed for position estimation of sensor nodes in a wireless network, making full use of distance information available among unknown nodes and simultaneous localization of multiple unknown nodes. To begin, a ranging model including the distance information among unknown nodes is established. With the reparameterization of the natural parameter and natural statistic, the solution problem of the ranging model is transformed into a parameter estimation problem of the curved exponential family. Then, a natural gradient method is adopted to deal with the parameter estimation problem of the curved exponential family. To ensure the convergence of the proposed algorithm, a particle swarm optimization method is utilized to obtain initial values of the unknown nodes. Experimental results indicate that the proposed algorithm can improve the positioning accuracy, compared with the traditional algorithm.      

一种在源数据稀疏情况下的流形学习算法研究

传统的流形学习算法能有效地学习出高维采样数据的低维嵌入坐标,但也存在一些不足,如不能处理稀疏的样本数据.针对这些缺点,提出了一种基于局部映射的直接求解线性嵌入算法(Solving Directly Linear Embedding,简称SDLE).通过假定低维流形的整体嵌入函数,将流形映射赋予局部光滑的约束,应用核方法将高维空间的坐标投影到特征空间,最后构造出在低维空间的全局坐标.SDLE算法解决了在源数据稀疏情况下的非线性维数约简问题,这是传统的流形学习算法没有解决的问题.通过实验说明了SDLE算法研究的有效性.