Nonlinear Dimensionality Reduction and Data Visualization： A Review

Dimensionality reduction and data visualization are useful and important processes in pattern recognition.Many techniques have been developed in the recent years.The self-organizing map (SOM) can be an efficient method for this purpose.This paper reviews recent advances in this area and related approaches such as multidimensional scaling (MDS),nonlinear PCA,principal manifolds,as well as the connections of the SOM and its recent variant,the visualization induced SOM (ViSOM),with these approaches. The SOM is shown to produce a quantized,qualitative scaling and while the ViSOM a quantitative or metric scaling and approximates principal curve/surface.The SOM can also be regarded as a generalized MDS to relate two metric spaces by forming a topological mapping between them.The relationships among various recently proposed techniques such as ViSOM,Isomap,LLE,and eigenmap are discussed and compared.     

Color clustering and learning for image segmentation based on neural networks

An image segmentation system is proposed for the segmentation of color image based on neural networks. In order to measure the color difference properly, image colors are represented in a modified L/sup */u/sup */v/sup */ color space. The segmentation system comprises unsupervised segmentation and supervised segmentation. The unsupervised segmentation is achieved by a two-level approach, i.e., color reduction and color clustering. In color reduction, image colors are projected into a small set of prototypes using self-organizing map (SOM) learning. In color clustering, simulated annealing (SA) seeks the optimal clusters from SOM prototypes. This two-level approach takes the advantages of SOM and SA, which can achieve the near-optimal segmentation with a low computational cost. The supervised segmentation involves color learning and pixel classification. In color learning, color prototype is defined to represent a spherical region in color space. A procedure of hierarchical prototype learning (HPL) is used to generate the different sizes of color prototypes from the sample of object colors. These color prototypes provide a good estimate for object colors. The image pixels are classified by the matching of color prototypes. The experimental results show that the system has the desired ability for the segmentation of color image in a variety of vision tasks.     

Bankruptcy analysis with self-organizing maps in learning metrics

We introduce a method for deriving a metric, locally based on the Fisher information matrix, into the data space. A self-organizing map (SOM) is computed in the new metric to explore financial statements of enterprises. The metric measures local distances in terms of changes in the distribution of an auxiliary random variable that reflects what is important in the data. In this paper the variable indicates bankruptcy within the next few years. The conditional density of the auxiliary variable is first estimated, and the change in the estimate resulting from local displacements in the primary data space is measured using the Fisher information matrix. When a self-organizing map is computed in the new metric it still visualizes the data space in a topology-preserving fashion, but represents the (local) directions in which the probability of bankruptcy changes the most.     

基于自迭代主元抽取的工作模态参数识别

针对传统批处理主成分分析工作模态参数识别中存在的矩阵奇异值或特征值分解病态问题,本文提出了一种基于自迭代主元抽取的工作模态参数识别方法。与传统批处理主成分分析通过矩阵分解一次获得所有主成分不同,该方法通过自迭代逐一抽取主成分从而实现主要贡献工作模态的逐一识别。理论分析表明,该方法的时间复杂度和空间复杂度比传统批处理主成分分析工作模态参数识别方法更低。在简支梁仿真数据集上的识别结果表明,自迭代主元抽取算法可以从平稳随机响应信号中有效地识别出线性时不变结构的主要贡献模态振型和固有频率,在响应测点和采样时间较多时其时间开销较传统方法也更小。     

Adaptive nonlinear manifolds and their applications to pattern recognition

Dimensionality reduction has long been associated with retinotopic mapping for understanding cortical maps. Multisensory information is processed, fused and mapped to an essentially 2-D cortex in an information preserving manner. Data processing and projection techniques inspired by this biological mechanism are playing an increasingly important role in pattern recognition, computational intelligence, data mining, information retrieval and image recognition. Dimensionality reduction involves reduction of features or volume of data and has become an essential step of information processing in many fields. The topic of manifold learning has recently attracted a great deal of attention, and a number of advanced techniques for extracting nonlinear manifolds and reducing data dimensions have been proposed from statistics, geometry theory and adaptive neural networks. This paper provides an overview of this challenging and emerging topic and discusses various recent methods such as self-organizing map (SOM), kernel PCA, principal manifold, isomap, local linear embedding, and Laplacian eigenmap. Many of them can be considered in a learning manifold framework. The paper further elaborates on the biologically inspired SOM model and its metric preserving variant ViSOM under the framework of adaptive manifold; and their applications in dimensionality reduction with face recognition are investigated. The experiments demonstrate that adaptive ViSOM-based methods produce markedly improved performance over the others due to their metric scaling and preserving properties along the nonlinear manifold.     

Data validation and missing data reconstruction using self-organizing map for water treatment

Applications in the water treatment domain generally rely on complex sensors located at remote sites. The processing of the corresponding measurements for generating higher-level information such as optimization of coagulation dosing must therefore account for possible sensor failures and imperfect input data. In this paper, self-organizing map (SOM)-based methods are applied to multiparameter data validation and missing data reconstruction in a drinking water treatment. The SOM is a special kind of artificial neural networks that can be used for analysis and visualization of large high-dimensional data sets. It performs both in a nonlinear mapping from a high-dimensional data space to a low-dimensional space aiming to preserve the most important topological and metric relationships of the original data elements and, thus, inherently clusters the data. Combining the SOM results with those obtained by a fuzzy technique that uses marginal adequacy concept to identify the functional states (normal or abnormal), the SOM performances of validation and reconstruction process are tested successfully on the experimental data stemming from a coagulation process involved in drinking water treatment.     

Enhanced neural gas network for prototype-based clustering

In practical cluster analysis tasks, an efficient clustering algorithm should be less sensitive to parameter configurations and tolerate the existence of outliers. Based on the neural gas (NG) network framework, we propose an efficient prototype-based clustering (PBC) algorithm called enhanced neural gas (ENG) network. Several problems associated with the traditional PBC algorithms and original NG algorithm such as sensitivity to initialization, sensitivity to input sequence ordering and the adverse influence from outliers can be effectively tackled in our new scheme. In addition, our new algorithm can establish the topology relationships among the prototypes and all topology-wise badly located prototypes can be relocated to represent more meaningful regions. Experimental results¹on synthetic and UCI datasets show that our algorithm possesses superior performance in comparison to several PBC algorithms and their improved variants, such as hard c-means, fuzzy c-means, NG, fuzzy possibilistic c-means, credibilistic fuzzy c-means, hard/fuzzy robust clustering and alternative hard/fuzzy c-means, in static data clustering tasks with a fixed number of prototypes.     

Some novel classifiers designed using prototypes extracted by a newscheme based on self-organizing feature map

We propose two new comprehensive schemes for designing prototype-based classifiers. The scheme addresses all major issues (number of prototypes, generation of prototypes, and utilization of the prototypes) involved in the design of a prototype-based classifier. First we use Kohonen's self-organizing feature map (SOFM) algorithm to produce a minimum number (equal to the number of classes) of initial prototypes. Then we use a dynamic prototype generation and tuning algorithm (DYNAGEN) involving merging, splitting, deleting, and retraining of the prototypes to generate an adequate number of useful prototypes. These prototypes are used to design a "1 nearest multiple prototype (1-NMP)" classifier. Though the classifier performs quite well, it cannot reasonably deal with large variation of variance among the data from different classes. To overcome this deficiency we design a "1 most similar prototype (1-MSP)" classifier. We use the prototypes generated by the SOFM-based DYNAGEN algorithm and associate with each of them a zone of influence. A norm (Euclidean)-induced similarity measure is used for this. The prototypes and their zones of influence are fine-tuned by minimizing an error function. Both classifiers are trained and tested using several data sets, and a consistent improvement in performance of the latter over the former has been observed. We also compared our classifiers with some benchmark results available in the literature.     

基于SOM 特性和PCA 索引的三维运动检索

提出了一种基于自组织特征映射(SOM)和PCA 索引的三维运动数据检索方法.首先利用每一个运动序列来进行拓扑特性加强的SOM 的学习,其运动特性被映射到一个主曲面,然后利用主成分分析方法(PCA)提取主曲面的主成分来建立一个基于主成分的索引机制,加快检索速率.SOM 的引入避免了与原始数据的直接接触,后续的工作只是在主曲面的基础上展开,消除了不同骨架长度的位置信息对运动特性的影响.实验结果表明了算法的有效性.     

基于改进高斯核度量和KPCA的数据聚类新方法

大多数超椭球聚类（hyper-ellipsoidal clustering,HEC）算法都使用马氏距离作为距离度量,已经证明在该条件下划分聚类的代价函数是常量,导致HEC无法实现椭球聚类.本文说明了使用改进高斯核的HEC算法可以解释为寻找体积和密度都紧凑的椭球分簇,并提出了一种实用HEC算法-K-HEC,该算法能够有效地处理椭球形、不同大小和不同密度的分簇.为实现更复杂形状数据集的聚类,使用定义在核特征空间的椭球来改进K-HEC算法的能力,提出了EK-HEC算法.仿真实验证明所提出算法在聚类结果和性能上均优于K-means算法、模糊C-means算法、GMM-EM算法和基于最小体积椭球（minimum-volume ellipsoids,MVE）的马氏HEC算法,从而证明了本文算法的可行性和有效性.     

Multiple model reduction approach using gap metric and stability margin for control nonlinear systems

This paper deals with the control of nonlinear systems using multimodel approach. The main idea of this work consists on the association of the gap metric and the stability margin tools to reduce the number of models constituting the multimodel bank. In fact, the self-organisation map (SOM) algorithm is used, firstly, to develop a preliminary multimodel bank. Then, the gap metric and the stability margin are computed to determine the redundancy of the initial multimodel bank. So, the multimodel controller is elaborated based on the reduced model bank. Simulations confirm the method for selecting the appropriate number of local models which should be used in the controller design.

     

Clustering of the self-organizing map

The self-organizing map (SOM) is an excellent tool in exploratory phase of data mining. It projects input space on prototypes of a low-dimensional regular grid that can be effectively utilized to visualize and explore properties of the data. When the number of SOM units is large, to facilitate quantitative analysis of the map and the data, similar units need to be grouped, i.e., clustered. In this paper, different approaches to clustering of the SOM are considered. In particular, the use of hierarchical agglomerative clustering and partitive clustering using K-means are investigated. The two-stage procedure-first using SOM to produce the prototypes that are then clustered in the second stage-is found to perform well when compared with direct clustering of the data and to reduce the computation time.     

The effect of sample size on the extended self-organizing map network—A market segmentation application

Kohonen's self-organizing map (SOM) network maps input data to a lower dimensional output map. The extended SOM network further groups the nodes on the output map into a user specified number of clusters. Kiang, Hu and Fisher used the extended SOM network for market segmentation and showed that the extended SOM provides better results than the statistical approach that reduces the dimensionality of the problem via factor analysis and then forms segments with cluster analysis. In this study, we examined the effect of sample size on the extended SOM compared to that on the factor/cluster approach. Two sampling schemes, one with random sampling and the other one with proportionate sampling were used. Comparisons were made using the correct classification rates between the two approaches at various sample sizes. Unlike statistical models, neural networks are not dependent on statistical assumptions. Thus, the results for neural network models are stable across sample sizes but sensitive to initial weights and model specifications.     

利用U矩阵对SOM网络的处理

自组织神经网络的最大优点是能够保持原始数据的拓扑结构。但是当数据量很大的时候,自组织神经网络的神经元的数据也随之增大。因此为了更好地对数据进行分析,需要将自组织神经网络中相似的神经元进行分组,也就是聚类。在对SOM网络进行再次分析之前,为了减少“噪音”数据和孤立点对聚类结构的影响,用U矩阵的变型方法对自组织神经网络分析的结果进行预处理。     

基于改进SOM模型的高校学生学习过程评价研究

对高校学生学习过程进行准确的评价,是提升学生学习效率、改进教师教学方法、完善学校教学管理的重要环节。目前已经提出了多种数学模型来解决该问题,但这些方法均需要一定的先验知识且难以实现自学习。本文利用SOM模型能在无监督、无先验知识的状态下对样本进行自组织的特性进行学习过程的评价,同时通过主成分分析,提高了网络收敛速度和聚类准确性。实例分析表明:改进SOM模型能有效地进行学生学习过程的评价。     

3D head model retrieval in kernel feature space using HSOM

In this paper, we propose a novel 3D head model retrieval framework. Specifically, to facilitate better classification and retrieval, the original 3D head model representations are embedded into another kernel feature space in which kernel principal component analysis (kernel PCA) is then performed to search for the optimal basis representation. Based on the extracted nonlinear features, a hierarchical indexing structure for 3D model retrieval is constructed using the hierarchical self organizing map (HSOM). The proposed indexing structure clusters the database into a hierarchy so that head models are partitioned by coarse features initially and then by finer scale features at lower levels. The main motivation of adopting this approach is that subspace technique like kernel PCA provides an elegant mechanism to describe the 3D head models on multiple resolutions based on the choices for reconstruction error and the orthogonal property of the produced eigenvectors. To further enhance the performance, a fuzzy metric between the query and the feature vector associated with each node on the SOMs is adopted instead of the usual Euclidean metric. Only nodes that possess high fuzzy measure values will be considered further for retrieval. In this way, the fuzzy measure approach is able to pick up potential relevant models even though they may be distributed across a number of neighbouring nodes. In addition to model categorization, the topology-preserving property of HSOM also facilitates the exploration of the model database with the possibility for further knowledge discovery. The effectiveness of the proposed approach is verified by a set of simulation examples on a 3D head model database.     

Improve credit scoring using transfer of learned knowledge from self-organizing map

Credit scoring is important for credit risk evaluation and monitoring in the accounting and finance domain. For financial institutions, the ability to predict the business failure is crucial, as incorrect decisions have direct financial consequences. A variety of pattern recognition techniques including neural networks, decision trees, and support vector machines have been applied to predict whether the borrowers should be considered a good or bad credit risk. This paper presents a hybrid approach to building the credit scoring model and illustrates how the unsupervised learning based on self-organizing map (SOM) can improve the discriminant capability of feedforward neural network (FNN). Within the hybridization scheme, the knowledge (i.e., prototypes of clusters) found by SOM is transferred as input to the subsequent FNN model. Four real-world data sets are used in the experiments for credit approval problems. By varying the parameters, the experimental results demonstrate the predictive model built by the hybrid approach can achieve better performance than the stand-alone FNN particularly when a limited amount of labeled data is available. This gives some insights on how to construct more accurate predictive models when the data collection is difficult in some financial applications. A complete and unique graphical visualization technique is shown which better outlines the trade-off between distinct metrics and attained performance.

     

Learning prototypes and distances: A prototype reduction technique based on nearest neighbor error minimization

A prototype reduction algorithm is proposed, which simultaneously trains both a reduced set of prototypes and a suitable local metric for these prototypes. Starting with an initial selection of a small number of prototypes, it iteratively adjusts both the position (features) of these prototypes and the corresponding local-metric weights. The resulting prototypes/metric combination minimizes a suitable estimation of the classification error probability. Good performance of this algorithm is assessed through experiments with a number of benchmark data sets and with a real task consisting in the verification of images of human faces.     

Batch process monitoring with tensor factorization

A batch process monitoring method using tensor factorization, tensor locality preserving projections (TLPP), is proposed. In many existing vector-based methods on batch process monitoring such as MPCA and MLPP, a batch data is represented as a vector in high-dimensional space. But vectorizing batch data will lead to information loss. Essentially, a batch data is presented as a second order tensor, or a matrix. In this case, tensor factorization may be used to deal with the two-way batch data matrix directly instead of performing vectorizing procedure. Furthermore, tensor representation has some advantages such as low memory and storage requirements and less estimated parameters for normal operating condition (NOC) model. On the other hand, different from principal component analysis (PCA) which aims at preserving the global Euclidean structure of the data, the TLPP aims to preserve the local neighborhood information and to detect the intrinsic manifold structure of the data. Consequently, TLPP may be used to find more meaningful intrinsic information hidden in the observations. The effectiveness and advantages of the TLPP monitoring approach are tested with the data from a benchmark fed-batch penicillin fermentation and two industrial fermentation processes, penicillin and cephalosporin, respectively.