一种半监督局部线性嵌入算法的文本分类方法*

针对局部线性嵌入算法（LLE）应用于非监督机器学习中的缺陷,将该算法与半监督思想相结合,提出了一种基于半监督局部线性嵌入算法的文本分类方法。通过使用文本数据的流形结构和少量的标签样本,将LLE中的距离矩阵采用分段形式进行调整;使用调整后的矩阵进行线性重建从而实现数据降维;针对半监督LLE中使用欧氏距离的缺点,采用高斯核函数将欧氏距离进行变换,并用新的核距离取代欧氏距离,提出了基于核的半监督局部线性嵌入算法;最后通过仿真实验验证了改进算法的有效性。     

改进的有监督的局部线性嵌入算法及实验演示

流形学习方法中的LLE算法可以将高维数据在保持局部邻域结构的条件下降维到低维流形子空间中．并得到与原样本集具有相似局部结构的嵌入向量集合。LLE算法在数据降维处理过程中没有考虑样本的分类信息。针对这些问题进行研究,提出改进的有监督的局部线性嵌人算法（MSLLE）,并利用MatLab对该改进算法的实现效果同LLE进行实验演示比较。通过实验演示表明,MSLLE算法较LLE算法可以有利于保持数据点本身内部结构。     

基于核函数的稳健线性嵌入方法

LLE算法是一种新的非监督学习方法,主要针对非线性降维问题。针对该算法存在的缺点,提出了一种基于核函数的稳健线性嵌入方法,该方法通过引进核函数来优化算法邻域点的求解;在特征空间中,修正权值矩阵W,进行降噪处理,经过推导,最终将实际的子空间计算归结为标准的特征值分解问题。采用最小近邻分类器估算识别率。在Yale人脸库以及AT&T人脸库的测试结果表明,在姿态、光照、表情、训练样本数目变化的情况下,改进的算法都具有较好的识别率。     

基于判别核主元空间k近邻的批次过程监视

针对批次过程非线性、多模态等特征,提出一种基于判别核主元k近邻（Dis-kPCkNN）的故障检测方法。首先,在核主元分析（kPCA）中,高斯核的窗宽参数依据样本类别标签在类内窗宽和类间窗宽中判别选取,使得核矩阵能有效提取数据的关联特征,保持数据的类别信息;其次,在核主元空间中引用k近邻规则代替传统的T²统计方法,k近邻规则可以有效处理主元空间非线性和多模态等特征的故障检测问题。数值模拟实例和半导体蚀刻工艺过程仿真实验表明:基于判别核主元k近邻方法可以有效地处理具有非线性和多模态结构特征的故障检测问题,提高计算的效率,减少内存的占用,并且故障检测率明显优于传统方法。     

一种改进的局部线性嵌入算法

局部线性嵌入算法(Local Linear Embedding,简称LLE)是一种非线性流形学习算法,能有效地学习出高维采样数据的低维嵌入坐标,但也存在一些不足,如不能处理稀疏的样本数据.针对这些缺点,提出了一种基于局部映射的线性嵌入算法(Local Project Linear Embedding,简称LPLE).通过假定目标空间的整体嵌入函数,重新构造样本点的局部邻域特征向量,最后将问题归结为损失矩阵的特征向量问题从而构造出目标空间的全局坐标.LPLE算法解决了传统LLE算法在源数据稀疏情况下的不能有效进行降维的问题,这也是其他传统的流形学习算法没有解决的.通过实验说明了LPLE算法研究的有效性和意义.     

Robust locally linear embedding

In the past few years, some nonlinear dimensionality reduction (NLDR) or nonlinear manifold learning methods have aroused a great deal of interest in the machine learning community. These methods are promising in that they can automatically discover the low-dimensional nonlinear manifold in a high-dimensional data space and then embed the data points into a low-dimensional embedding space, using tractable linear algebraic techniques that are easy to implement and are not prone to local minima. Despite their appealing properties, these NLDR methods are not robust against outliers in the data, yet so far very little has been done to address the robustness problem. In this paper, we address this problem in the context of an NLDR method called locally linear embedding (LLE). Based on robust estimation techniques, we propose an approach to make LLE more robust. We refer to this approach as robust locally linear embedding (RLLE). We also present several specific methods for realizing this general RLLE approach. Experimental results on both synthetic and real-world data show that RLLE is very robust against outliers.     

核方法与流形学习在心电识别中的研究

结合核方法和局部线性嵌入（LLE）方法,提出了一种基于核局部线性嵌入方法,该方法克服了局部线性嵌入方法由于心电特征分布不均衡造成的不稳定问题。结合支持向量机在MIT-BIH心律失常标准数据库进行实验,并利用PCA和LLE进行特征提取比较,验证了该方法的有效性及优势。     

Nonlinear process monitoring based on maximum variance unfolding projections

Kernel principal component analysis (KPCA) has recently proven to be a powerful dimensionality reduction tool for monitoring nonlinear processes with numerous mutually correlated measured variables. However, the performance of KPCA-based monitoring method largely depends on its kernel function which can only be empirically selected from finite candidates assuming that some faulty process samples are available in the off-line modeling phase. Moreover, KPCA works at high computational cost in the on-line monitoring phase due to its dense expansions in terms of kernel functions. To overcome these deficiencies, this paper proposes a new process monitoring technique comprising fault detection and identification based on a novel dimensionality reduction method named maximum variance unfolding projections (MVUP). MVUP firstly applies the recently proposed manifold learning method maximum variance unfolding (MVU) on training samples, which can be seen as a special variation of KPCA whose kernel matrix is automatically learned such that the underlying manifold structure of training samples is “unfolded” in the reduced space and hence the boundary of distribution region of training samples is preserved. Then MVUP uses linear regression to find the projection that best approximates the implicit mapping from training samples to their lower dimensional embedding learned by MVU. Simulation results on the benchmark Tennessee Eastman process show that MVUP-based process monitoring method is a good alternative to KPCA-based monitoring method.     

Nonlinear process monitoring based on kernel global–local preserving projections

A new nonlinear dimensionality reduction method called kernel global–local preserving projections (KGLPP) is developed and applied for fault detection. KGLPP has the advantage of preserving global and local data structures simultaneously. The kernel principal component analysis (KPCA), which only preserves the global Euclidean structure of data, and the kernel locality preserving projections (KLPP), which only preserves the local neighborhood structure of data, are unified in the KGLPP framework. KPCA and KLPP can be easily derived from KGLPP by choosing some particular values of parameters. As a result, KGLPP is more powerful than KPCA and KLPP in capturing useful data characteristics. A KGLPP-based monitoring method is proposed for nonlinear processes. T² and SPE statistics are constructed in the feature space for fault detection. Case studies in a nonlinear system and in the Tennessee Eastman process demonstrate that the KGLPP-based method significantly outperforms KPCA, KLPP and GLPP-based methods, in terms of higher fault detection rates and better fault sensitivity.     

Reconstruction and analysis of multi-pose face images based on nonlinear dimensionality reduction

Locally linear embedding (LLE) is a nonlinear dimensionality reduction method proposed recently. It can reveal the intrinsic distribution of data, which cannot be provided by classical linear dimensionality reduction methods. The application of LLE, however, is limited because of its lack of a parametric mapping between the observation and the low-dimensional output. And the large data set to be reduced is necessary. In this paper, we propose methods to establish the process of mapping from low-dimensional embedded space to high-dimensional space for LLE and validate their efficiency with the application of reconstruction of multi-pose face images. Furthermore, we propose that the high-dimensional structure of multi-pose face images is similar for the same kind of pose change mode of different persons. So given the structure information of data distribution which is obtained by leaning large numbers of multi-pose images in a training set, the support vector regression (SVR) method of statistical learning theory is used to learn the high-dimensional structure of someone based on small sets. The detailed learning method and algorithm are given and applied to reconstruct and synthesize face images in small set cases. The experiments prove that our idea and method is correct.     

Nonlinear process monitoring using kernel dictionary learning with application to aluminum electrolysis process

In practice, because of complex mechanism processes, such as heating process, volume heterogeneity, and various chemical reaction characteristics, there is a nonlinear relationship among variables in industrial systems. The nonlinearity brings some difficulties to process monitoring. In order to ensure that the process monitoring system can work normally in nonlinear production processes, the nonlinear relationship between variables ought to be considered. In this work, a new fault detection and isolation method based on kernel dictionary learning is presented. In detail, the linearly inseparable data is mapped to a high-dimensional space. Then, a new nonlinear dictionary learning method based on kernel method was proposed to learn the dictionary. After obtaining the dictionary, the control limit can be calculated from the training data according to the kernel density estimation (KDE) method. When new data arrive, they can be represented by the well-learned dictionary, and the kernel reconstruction error can be used as a classifier for process monitoring. As for the fault data, the iterative reconstruction based method is proposed for fault isolation. In order to evaluate the effectiveness of the proposed process monitoring method, some extensive experiments on a numerical simulation, the continuous stirred tank heater (CSTH) process, and a real industrial aluminum electrolysis process are conducted. The proposed method is compared with several state-of-the-art process monitoring methods and the experimental results show that the proposed method can provide satisfactory monitoring results, especially for some small faults, thus it is suitable for process monitoring of nonlinear industrial processes.     

KNPE算法在化工过程故障检测中的应用

化工生产过程具有维数高、非线性强等特点。针对传统的邻域保持嵌入(NPE)算法对非线性数据特征提取不足的缺陷,引入高斯核函数,将数据由非线性的输入空间转换到线性的特征空间。核邻域保持嵌入(KNPE)算法在构建局部空间特征结构的基础上,能够更好地提取数据的非线性结构。通过以田纳西-伊斯曼(TE)仿真过程为例,构造T2和SPE统计量进行故障检测,证明了KNPE方法比NPE和KPCA方法能够更快更准确的检测出非线性故障的发生。     

半监督类保持局部线性嵌入方法

为使局部线性嵌入（local linear embedding, LLE）这一无监督高维数据的非线性特征提取方法提取出的特征在分类或聚类学习上更优,提出一种半监督类保持局部线性嵌入(semi-supervised class preserving local linear embedding, SSCLLE)的非线性特征提取方法。该方法将半监督信息融入到LLE中,首先对标记样本近邻赋予伪标签,增大标记样本数量。其次,对标记样本之间的距离进行局部调整,缩小同类样本间距,扩大异类样本间距。同时在局部线性嵌入优化目标函数中增加全局同类样本间距和异类样本间距的约束项,使得提取出的低维特征可以确保同类样本点互相靠近,而异类样本点彼此分离。在一系列实验中,其聚类精确度以及可视化效果明显高于无监督LLE和现有半监督流特征提取方法,表明该方法提取出的特征具有很好的类保持特性。     

基于递推规范变量残差和核主元分析的微小故障检测

微小故障由于故障征兆不明显从而很难在故障发生早期对其进行检测. 针对该问题, 本文提出了一种基于递推规范变量残差和核主元分析(RCVD–KPCA)的微小故障检测方法. 首先构造规范变量残差, 从中提取数据的线性特征. 利用指数加权滑动平均法对规范变量残差进行递推滤波处理, 提高规范变量残差对微小故障的敏感程度;然后使用KPCA提取规范变量残差中的非线性主成分作为非线性特征, 根据提取的特征提出了两个新的故障检测统计量; 此外, 利用核密度估计确定故障检测统计量的控制限. 由于同时提取了过程数据的线性和非线性特征, 有效地提高了非线性动态过程中微小故障的可检测性. 以闭环连续搅拌釜式反应器过程为例进行了仿真分析, 仿真结果表明本文所提方法具有较好的故障检测性能.     

基于核规范变量分析的非线性故障诊断方法

提出一种基于核规范变量分析（KCVA）的非线性过程故障诊断方法．该方法使用核函数完成非线性空间到高维线性空间的映射，避免了高维空间中的数据处理和非线性映射函数的使用．在线性空间中使用规范变量分析（CVA）来辨识状态空闻模型，从数据中提取状态信息．3个监测量（Tr^2，Ts^2，Q）用来进行故障检测，同时使用贡献图分离故障变量，并判断故障原因．在CSTR系统上的仿真结果表明，KCVA方法比主元分析法（PCA）和CVA方法能更灵敏地检测到故障的发生，更有效地监控过程变化．     

Kernel-based nonlinear dimensionality reduction for electrocardiogram recognition

The human heart is a complex system that reveals many clues about its condition in its electrocardiogram (ECG) signal, and ECG supervising is the most important and efficient way of preventing heart attacks. ECG analysis and recognition are both important and tempting topics in modern medical research. The purpose of this paper is to develop an algorithm which investigates kernel method, locally linear embedding (LLE), principal component analysis (PCA), and support vector machine(SVM) algorithms for dimensionality reduction, features extraction, and classification for recognizing and classifying the given ECG signals. In order to do so, a nonlinear dimensionality reduction kernel method based LLE is proposed to reduce the high dimensions of the variational ECG signals, and the principal characteristics of the signals are extracted from the original database by means of the PCA, each signal representing a single and complete heart beat. SVM method is applied to classify the ECG data into several categories of heart diseases. Experimental results obtained demonstrated that the performance of the proposed method was similar and sometimes better when compared to other ECG recognition techniques, thus indicating a viable and accurate technique.     

基于改进LLE的Elman神经网络电路故障诊断

如今的电路越来越复杂,随之而来的就是电路系统的高故障性,所以如何定位故障发生成为一大难题。文中基于提高故障诊断性能的目的,先采用一种改进的局部线性分析算法作为初始数据处理器对输出响应序列进行降维,提取故障特征向量,然后再通过OIF-Elman神经网络搭建故障分类器,对电路进行故障检测。仿真结果表明,将改进过的局部线性分析算法和OIF-Elman神经网络应用到故障诊断中,不仅具有比传统BP神经网络更精确的故障诊断正确率,且整个网络的收敛速度也会有明显提升。     

一种基于核的半监督局部线性嵌入方法

在局部线性嵌入算法中,标签价值没有得到充分体现。针对该问题,提出一种基于核的半监督局部线性嵌入方法。考虑到欧氏距离容易破坏流形结构,将原始数据映射到高维核空间,利用高维空间中的核距离代替欧氏距离,采用半监督标签信息调整距离矩阵,通过调整后的距离矩阵对数据结构进行线性重建,从而提高算法的降维性能。在标准数据集、人脸库、字符库等数据上进行实验,结果表明,与传统局部线性嵌入算法相比,该方法的辨识率提高了2%     

Formulating LLE using alignment technique

LLE is a well-known method to nonlinear dimensionality reduction. In this short paper, we present an alternative way to formulate LLE. The alignment technique is exploited to align the local coordinates on the local patches of manifolds to be the global ones. The efficient computation of embedding coordinates of LLE automatically appears in the proposed framework.