The quick dynamic clustering method for mixed-type data

This paper describes a new approach to high-dimensional mixed-type data clustering with missing values, which combines information on common nearest neighbors with classic between-vectors distances calculated by an original technique. The results are applied to form intersecting clusters for every missing value.     

基于二维GDSOM的路标动态自组织提取方法

提出一种基于距离传感器的结构化特征的动态、自组织提取方法。该方法由3个部分组成:主动感知行为的设计,时空信息的降维处理及路标的自组织提取。设计基于沿墙走的“主动感知行为”来获得高相关性的感知时空序列信息;给出基于变化检测和激活强度的活性神经元来对时空序列信息降维;最后提出一种二维动态增长自组织特征图方法,实现环境路标的自组织提取和识别。实验结果验证该方法的有效性。     

Self-Organizing Map Oversampling (SOMO) for imbalanced data set learning

Learning from imbalanced datasets is challenging for standard algorithms, as they are designed to work with balanced class distributions. Although there are different strategies to tackle this problem, methods that address the problem through the generation of artificial data constitute a more general approach compared to algorithmic modifications. Specifically, they generate artificial data that can be used by any algorithm, not constraining the options of the user. In this paper, we present a new oversampling method, Self-Organizing Map-based Oversampling (SOMO), which through the application of a Self-Organizing Map produces a two dimensional representation of the input space, allowing for an effective generation of artificial data points. SOMO comprises three major stages: Initially a Self-Organizing Map produces a two-dimensional representation of the original, usually high-dimensional, space. Next it generates within-cluster synthetic samples and finally it generates between cluster synthetic samples. Additionally we present empirical results that show the improvement in the performance of algorithms, when artificial data generated by SOMO are used, and also show that our method outperforms various oversampling methods.     

Spatio-Temporal Analysis with the Self-Organizing Feature Map

Spatio-temporal pattern recognition problems are particularly challenging. They typically involve detecting change that occurs over time in two-dimensional patterns. Analytic techniques devised for temporal data must take into account the spatial relationships among data points. An artificial neural network known as the self-organizing feature map (SOM) has been used to analyze spatial data. This paper further investigates the use of the SOM with spatio-temporal pattern recognition. The principles of the two-dimensional SOM are developed into a novel three-dimensional network and experiments demonstrate that (i) the three-dimensional network makes a better topological ordering and (ii) there is a difference in terms of the spatio-temporal analysis that can be made with the three-dimensional network. Received 21 October 1999 / Revised 11 February 2000 / Accepted 2 May 2000     

Self-Organizing Map Formation with a Selectively Refractory Neighborhood

Decreasing neighborhood with distance has been identified as one of a few conditions to achieve final states in the self-organizing map (SOM) that resemble the distribution of high-dimensional input data. In the classic SOM model, best matching units (BMU) decrease their influence area as a function of distance. We introduce a modification to the SOM algorithm in which neighborhood is contemplated from the point of view of affected units, not from the view of BMUs. In our proposal, neighborhood for BMUs is not reduced, instead the rest of the units exclude some BMUs from affecting them. Each neuron identifies, from the set of BMUs that influenced it in previous epochs, those to whom it becomes refractory to for the rest of the process. Despite that the condition of decreasing neighborhood over distance is not maintained, self-organization still persists, as shown by several experiments. The maps achieved by the proposed modification have, in many cases, a lower error measure than the maps formed by SOM. Also, the model is able to remove discontinuities (kinks) from the map in a very small number of epochs, which contrasts with the original SOM model.     

Commitment and typicality measures for the Self-Organizing Map

Soft classification using Kohonen's Self-Organizing Map (SOM) has not been explored as thoroughly as the Multi-Layer-Perceptron (MLP) neural network. In this paper, we propose two non-parametric algorithms for the SOM to provide soft classification outputs. These algorithms, which are labelling-frequency-based, are called SOM Commitment (SOM-C) and SOM Typicality (SOM-T), expressing in the first case the degree of commitment the classifier has for each class for a specific pixel and in the second case, how typical that pixel's reflectances are of those upon which the classifier was trained. To evaluate the two proposed algorithms, soft classifications of a Satellite Pour l'Observation de la Terre (SPOT) High Resolution Visible (HRV) image and an Airborne Visible Infrared Imaging Spectrometer (AVIRIS) image were undertaken. Both traditional soft classifiers, i.e. Bayesian posterior probability and Mahalanobis typicality classifier, and the most frequently used non-parametric neural network model, i.e. MLP, were used as a comparison. Principal-components analysis (PCA) was used to explore the relationship between these measures. Results indicate that great similarities exist between the SOM-C, MLP and the Bayesian posterior probability classifiers, while the SOM-T corresponds closely with Mahalanobis typicality probabilities. However, as implemented, they have the advantage of being non-parametric. The proposed measures significantly outperformed Bayesian and Mahalanobis classifiers when using the hyperspectral AVIRIS image.     

A Hierarchical Self-Organizing Map Model for Sequence Recognition

A novel neural model made up of two self-organizing maps nets – one on top of the other – is introduced and analysed experimentally. The model makes effective use of context information, and that enables it to perform sequence classification and discrimination efficiently. It was successfully applied to real sequences, taken from the third voice of the sixteenth four-part fugue in G minor of the Well-Tempered Clavier (vol. I) of J.S. Bach. The model has an application in domains which require pattern recognition, or more specifically, which demand the recognition of either a set of sequences of vectors in time or sub-sequences into a unique and large sequence of vectors in time.     

Mixing numerical and categorical data in a Self-Organizing Map by means of frequency neurons

Even though Self-Organizing Maps (SOMs) constitute a powerful and essential tool for pattern recognition and data mining, the common SOM algorithm is not apt for processing categorical data, which is present in many real datasets. It is for this reason that the categorical values are commonly converted into a binary code, a solution that unfortunately distorts the network training and the posterior analysis. The present work proposes a SOM architecture that directly processes the categorical values, without the need of any previous transformation. This architecture is also capable of properly mixing numerical and categorical data, in such a manner that all the features adopt the same weight. The proposed implementation is scalable and the corresponding learning algorithm is described in detail. Finally, we demonstrate the effectiveness of the presented algorithm by applying it to several well-known datasets.     

利用模糊神经网络实现逆向工程中的区域分割

论文提出了一种改进的模糊自组织特征映射网络(fuzzySOFM),它不仅显著加快了聚类的速度,而且算法简单。该网络采用由数据点的坐标、估算出的法矢量和曲率构成的八维特征向量作为输入,快速地实现了逆向工程中点云数据的区域分割。与现有方法相比,该方法具有以下优点:第一,具有更高的聚类速度,并可以直接处理含噪声数据;第二,聚类的结果与数据输入的顺序无关;第三,能利用数据的隶属度快速提取出特征线数据,从而将基于面的分割和基于线的分割结合起来。实验结果证明了这种方法的有效性。     

A Convolutional Deep Self-Organizing Map Feature extraction for machine learning

In this work we propose a new Unsupervised Deep Self-Organizing Map (UDSOM) algorithm for feature extraction, quite similar to the existing multi-layer SOM architectures. The principal underlying idea of using SOMs is that if a neuron is wins n times, these n inputs that activated this neuron are similar. The basic principle consists of an alternation of phases of splitting and abstraction of regions, based on a non-linear projection of high-dimensional data over a small space using Kohonen maps following a deep architecture. The proposed architecture consists of a splitting process, layers of alternating self-organizing, a rectification function RELU and an abstraction layer (convolution-pooling). The self-organizing layer is composed of a few SOMs with each map focusing on modelling a local sub-region. The most winning neurons of each SOM are then organized in a second sampling layer to generate a new 2D map. In parallel to this transmission of the winning neurons, an abstraction of the data space is obtained after the convolution-pooling module. The ReLU is then applied. This treatment is applied more than once, changing the size of the splitting window and the displacement step on the reconstructed input image each time. In this way, local information is gathered to form more global information in the upper layers by applying each time a convolution filter of the level. The architecture of the Unsupervised Deep Self-Organizing Map is unique and retains the same principle of deep learning algorithms. This architecture can be very interesting in a Big Data environment for machine learning tasks. Experiments have been conducted to discuss how the proposed architecture shows this performance.

     

The Supervised Network Self-Organizing Map for Classification of Large Data Sets

Complex application domains involve difficult pattern classification problems. The state space of these problems consists of regions that lie near class separation boundaries and require the construction of complex discriminants while for the rest regions the classification task is significantly simpler. The motivation for developing the Supervised Network Self-Organizing Map (SNet-SOM) model is to exploit this fact for designing computationally effective solutions. Specifically, the SNet-SOM utilizes unsupervised learning for classifying at the simple regions and supervised learning for the difficult ones in a two stage learning process. The unsupervised learning approach is based on the Self-Organizing Map (SOM) of Kohonen. The basic SOM is modified with a dynamic node insertion/deletion process controlled with an entropy based criterion that allows an adaptive extension of the SOM. This extension proceeds until the total number of training patterns that are mapped to neurons with high entropy (and therefore with ambiguous classification) reduces to a size manageable numerically with a capable supervised model. The second learning phase (the supervised training) has the objective of constructing better decision boundaries at the ambiguous regions. At this phase, a special supervised network is trained for the computationally reduced task of performing the classification at the ambiguous regions only. The performance of the SNet-SOM has been evaluated on both synthetic data and on an ischemia detection application with data extracted from the European ST-T database. In all cases, the utilization of SNet-SOM with supervised learning based on both Radial Basis Functions and Support Vector Machines has improved the results significantly related to those obtained with the unsupervised SOM and has enhanced the scalability of the supervised learning schemes. The highly disciplined design of the generalization performance of the Support Vector Machine allows to design the proper model for the particular training set.     

Efficient algorithms based on the k-means and Chaotic League Championship Algorithm for numeric,categorical, and mixed-type data clustering

The success rates of the expert or intelligent systems depend on the selection of the correct data clusters. The k-means algorithm is a well-known method in solving data clustering problems. It suffers not only from a high dependency on the algorithm's initial solution but also from the used distance function. A number of algorithms have been proposed to address the centroid initialization problem, but the produced solution does not produce optimum clusters. This paper proposes three algorithms (i) the search algorithm C-LCA that is an improved League Championship Algorithm (LCA), (ii) a search clustering using C-LCA (SC-LCA), and (iii) a hybrid-clustering algorithm called the hybrid of k-means and Chaotic League Championship Algorithm (KSC-LCA) and this algorithm has of two computation stages. The C-LCA employs chaotic adaptation for the retreat and approach parameters, rather than constants, which can enhance the search capability. Furthermore, to overcome the limitation of the original k-means algorithm using the Euclidean distance that cannot handle the categorical attribute type properly, we adopt the Gower distance and the mechanism for handling a discrete value requirement of the categorical value attribute. The proposed algorithms can handle not only the pure numeric data but also the mixed-type data and can find the best centroids containing categorical values. Experiments were conducted on 14 datasets from the UCI repository. The SC-LCA and KSC-LCA competed with 16 established algorithms including the k-means, k-means++, global k-means algorithms, four search clustering algorithms and nine hybrids of k-means algorithm with several state-of-the-art evolutionary algorithms. The experimental results show that the SC-LCA produces the cluster with the highest F-Measure on the pure categorical dataset and the KSC-LCA produces the cluster with the highest F-Measure for the pure numeric and mixed-type tested datasets. Out of 14 datasets, there were 13 centroids produced by the SC-LCA that had better F-Measures than that of the k-means algorithm. On the Tic-Tac-Toe dataset containing only categorical attributes, the SC-LCA can achieve an F-Measure of 66.61 that is 21.74 points over that of the k-means algorithm (44.87). The KSC-LCA produced better centroids than k-means algorithm in all 14 datasets; the maximum F-Measure improvement was 11.59 points. However, in terms of the computational time, the SC-LCA and KSC-LCA took more NFEs than the k-means and its variants but the KSC-LCA ranks first and SC-LCA ranks fourth among the hybrid clustering and the search clustering algorithms that we tested. Therefore, the SC-LCA and KSC-LCA are general and effective clustering algorithms that could be used when an expert or intelligent system requires an accurate high-speed cluster selection.     

Integrated dimensionality reduction technique for mixed-type data involving categorical values

Dimensionality reduction is a useful technique to cope with high dimensionality of the real-world data. However, traditional methods were studied in the context of datasets with only numeric attributes. With the demand of analyzing datasets involving categorical attributes, an extension to the recent dimensionality-reduction technique t-SNE is proposed. The extension facilitates t-SNE to handle mixed-type datasets. Each attribute of the data is associated with a distance hierarchy which allows the distance between numeric values and between categorical values be measured in a unified manner. More importantly, domain knowledge regarding distance considering semantics embedded in categorical values can be specified via the hierarchy. Consequently, the extended t-SNE can project the high-dimensional, mixed data to a low-dimensional space with topological order which reflects user's intuition.     

MIGSOM: Multilevel Interior Growing Self-Organizing Maps for High Dimensional Data Clustering

Understanding the inherent structure of high-dimensional datasets is a very challenging task. This can be tackled from visualization, summarizing or simply clustering points of view. The Self-Organizing Map (SOM) is a powerful and unsupervised neural network to resolve these kinds of problems. By preserving the data topology mapped onto a grid, SOM can facilitate visualization of data structure. However, classical SOM still suffers from the limits of its predefined structure. Growing variants of SOM can overcome this problem, since they have tried to define a network structure with no need an advance a fixed number of output units by dynamic growing architecture. In this paper we propose a new dynamic SOMs called MIGSOM: Multilevel Interior Growing SOMs for high-dimensional data clustering. MIGSOM present a different architecture than dynamic variants presented in the literature. Using an unsupervised training process MIGSOM has the capability of growing map size from the boundaries as well as the interior of the network in order to represent more faithfully the structure present in a data collection. As a result, MIGSOM can have three-dimensional (3-D) structure with different levels of oriented maps developed according to data direction. We demonstrate the potential of the MIGSOM with real-world datasets of high-dimensional properties in terms of topology preserving visualization, vectors summarizing by efficient quantization and data clustering. In addition, MIGSOM achieves better performance compared to growing grid and the classical SOM.     

Feature Competitive Algorithm for Dimension Reduction of the Self-Organizing Map Input Space

The self-organizing map (SOM) can classify documents by learning about their interrelationships from its input data. The dimensionality of the SOM input data space based on a document collection is generally high. As the computational complexity of the SOM increases in proportion to the dimension of its input space, high dimensionality not only lowers the efficiency of the initial learning process but also lowers the efficiencies of the subsequent retrieval and the relearning process whenever the input data is updated. A new method called feature competitive algorithm (FCA) is proposed to overcome this problem. The FCA can capture the most significant features that characterize the underlying interrelationships of the entities in the input space to form a dimensionally reduced input space without excessively losing of essential information about the interrelationships. The proposed method was applied to a document collection, consisting of 97 UNIX command manual pages, to test its feasibility and effectiveness. The test results are encouraging. Further discussions on several crucial issues about the FCA are also presented.     

Selecting the Color Space for Self-Organizing Map Based Foreground Detection in Video

Detecting foreground objects on scenes is a fundamental task in computer vision and the used color space is an important election for this task. In many situations, especially on dynamic backgrounds, neither grayscale nor RGB color spaces represent the best solution to detect foreground objects. Other standard color spaces, such as YCbCr or HSV, have been proposed for background modeling in the literature; although the best results have been achieved using diverse color spaces according to the application, scene, algorithm, etc. In this work, a color space and a color component weighting selection process are proposed to detect foreground objects in video sequences using self-organizing maps. Experimental results are also provided using well known benchmark videos.     

Recurrent Self-Structuring Machine Learning for Video Processing using Multi-Stream Hierarchical Growing Self-Organizing Maps

Multimedia Tools and Applications - The emergence of IoT and advanced multimedia information systems have undoubtedly created a proliferation of video sensor data. Although diverse machine learning...     

自组织特征映射网络在舌边数据分类中的应用

尝试利用自组织特征映射网络较强的聚类功能对分割出的舌体边缘进行分类。通过实验证明它能很好的将舌边数据分成舌根、舌尖、舌左、舌右四类点,达到预定目标。     

Using Cellular Probabilistic Self-Organizing Map in Borrowing Channel Assignment for Patterned Traffic Load

The fast growing cellular mobile systems demand more efficient and faster channel allocation techniques. Borrowing channel assignment (BCA) is a compromising technique between fixed channel allocation (FCA) and dynamic channel allocation (DCA). However, in the case of patterned traffic load, BCA is not efficient to further enhance the performance because some heavy-traffic cells are unable to borrow channels from neighboring cells that do not have unused nominal channels. The performance of the whole system can be raised if the short-term traffic load can be predicted and the nominal channels can be re-assigned for all cells. This paper describes an improved BCA scheme using traffic load prediction. The prediction is obtained by using the short-term forecasting ability of cellular probabilistic self-organizing map (CPSOM). This paper shows that the proposed CPSOM-based BCA method is able to enhance the performance of patterned traffic load compared with the traditional BCA methods. Simulation results corroborate that the proposed method delivers significantly better performance than BCA for patterned traffic load situations, and is virtually as good as BCA in the other situations analyzed.