A new approach for data clustering and visualization using self-organizing maps

A self-organizing map (SOM) is a nonlinear, unsupervised neural network model that could be used for applications of data clustering and visualization. One of the major shortcomings of the SOM algorithm is the difficulty for non-expert users to interpret the information involved in a trained SOM. In this paper, this problem is tackled by introducing an enhanced version of the proposed visualization method which consists of three major steps: (1) calculating single-linkage inter-neuron distance, (2) calculating the number of data points in each neuron, and (3) finding cluster boundary. The experimental results show that the proposed approach has the strong ability to demonstrate the data distribution, inter-neuron distances, and cluster boundary, effectively. The experimental results indicate that the effects of visualization of the proposed algorithm are better than that of other visualization methods. Furthermore, our proposed visualization scheme is not only intuitively easy understanding of the clustering results, but also having good visualization effects on unlabeled data sets.     

Topology-oriented self-organizing maps: a survey

The self-organizing map (SOM) is a prominent neural network model that has found wide application in a spectrum of domains. Accordingly, it has received widespread attention both from the communities of researchers and practitioners. As a result, several variations of the basic architecture have been devised, specifically in the early years of the SOM’s evolution, which were introduced so as to address various architectural shortcomings or to explore other structures of the basic model. The overall goal of this survey is to present a comprehensive comparison of these networks, in terms of their primitive components and properties. We dichotomize these schemes as being either tree based or non-tree based. We have embarked on this venture with the hope that since the survey is comprehensive and the bibliography extensive, it will be an asset and resource for future researchers.     

Mining sequential patterns from data streams: a centroid approach

In recent years, emerging applications introduced new constraints for data mining methods. These constraints are typical of a new kind of data: the data streams. In data stream processing, memory usage is restricted, new elements are generated continuously and have to be considered in a linear time, no blocking operator can be performed and the data can be examined only once. At this time, only a few methods has been proposed for mining sequential patterns in data streams. We argue that the main reason is the combinatory phenomenon related to sequential pattern mining. In this paper, we propose an algorithm based on sequences alignment for mining approximate sequential patterns in Web usage data streams. To meet the constraint of one scan, a greedy clustering algorithm associated to an alignment method is proposed. We will show that our proposal is able to extract relevant sequences with very low thresholds.     

Structural enhanced information and its application to improved visualization of self-organizing maps

In this paper, we propose structural enhanced information for detecting and visualizing main features in input patterns. We have so far proposed information enhancement for feature detection, where, if we want to focus upon components such as units and connection weights and interpret the functions of the components, we have only to enhance competitive units with the components. Though this information enhancement has given favorable results in feature detection, we further refine the information enhancement and propose structural enhanced information. In structural enhanced information, three types of enhanced information can be differentiated, that is, first-, second- and third-order enhanced information. The first-order information is related to the enhancement of competitive units themselves in a competitive network, and the second-order information is dependent upon the enhancement of competitive units with input patterns. Then, the third-order information is obtained by subtracting the effect of the first-order information from the second-order information. Thus, the third-order information more explicitly represents information on input patterns. With this structural enhanced information, we can estimate more detailed features in input patterns. For demonstrating explicitly and intuitively the improved performance of our method, the conventional SOM was used, and we transformed competitive unit outputs so as to improve visualization. The method was applied to the well-known Iris problem, an OECD countries classification problem and the Johns Hopkins University Ionosphere database. In all these problems, we succeeded in visualizing the detailed and important features of input patterns by using the third-order information.     

Spatial data mining and visualization based on self-organizing map

Based on the traditional spatial data analysis, a novel mode of spatial data mining and visualization is proposed which integrates the self-organizing map for the actual problem. Simulations for IRIS data show that this method （computational and visual） can collaboratively discover complex pattems in large spatial datasets, in an effective and efficient way.     

Managing complexity in large data bases using self-organizing maps

The amount of financial information in today's sophisticated large data bases is substantial and makes comparisons between company performance—especially over time—difficult or at least very time consuming. The aim of this paper is to investigate whether neural networks in the form of self-organizing maps can be used to manage the complexity in large data bases. We structure and analyze accounting numbers in a large data base over several time periods. By using self-organizing maps, we overcome the problems associated with finding the appropriate underlying distribution and the functional form of the underlying data in the structuring task that is often encountered, for example, when using cluster analysis. The method chosen also offers a way of visualizing the results. The data base in this study consists of annual reports of more than 120 world wide pulp and paper companies with data from a five year time period.     

Externally growing self-organizing maps and its application to e-mail database visualization and exploration

In this paper we present an approach to organize and classify e-mails using self-organizing maps. The aim is on the one hand to provide an intuitive visual profile of the considered mailing lists and on the other hand to offer an intuitive navigation tool, were similar e-mails are located close to each other, so that the user can scan easily for e-mails similar in content. To be able to evaluate this approach we have developed a prototypical software tool that imports messages from a mailing list and arranges/groups these e-mails based on a similarity measure. The tool combines conventional keyword search methods with a visualization of the considered e-mail collection. The prototype was developed based on externally growing self-organizing maps, which solve some problems of conventional self-organizing maps and which are computationally viable. Besides the underlying algorithms we present and discuss some system evaluations in order to show the capabilities of the approach.     

ViSOM - a novel method for multivariate data projection andstructure visualization

When used for visualization of high-dimensional data, the self-organizing map (SOM) requires a coloring scheme, such as the U-matrix, to mark the distances between neurons. Even so, the structures of the data clusters may not be apparent and their shapes are often distorted. In this paper, a visualization-induced SOM (ViSOM) is proposed to overcome these shortcomings. The algorithm constrains and regularizes the inter-neuron distance with a parameter that controls the resolution of the map. The mapping preserves the inter-point distances of the input data on the map as well as the topology. It produces a graded mesh in the data space such that the distances between mapped data points on the map resemble those in the original space, like in the Sammon mapping. However, unlike the Sammon mapping, the ViSOM can accommodate both training data and new arrivals and is much simpler in computational complexity. Several experimental results and comparisons with other methods are presented.     

Visualization of high-dimensional data using two-dimensional self-organizing piecewise-smooth Kohonen maps

To make visualization of high-dimensional data more accurate, we offer a method of approximating two-dimensional Kohonen maps lying in a multiple-dimensional space. Cubic parametric spline-based least-defect surfaces can be used as an approximation function to minimize approximation errors.     

Qualitative analysis of goat and sheep production data using self-organizing maps

Abstract: The aim of this study was to analyse the relationship between different small ruminant livestock production systems with different levels of specialization. The analysis is carried out by using the self-organizing map. This tool allows high-dimensional input spaces to be mapped into much lower-dimensional spaces, thus making it much more straightforward to understand any set of data. These representations enable the visual extraction of qualitative relationships among variables (visual data mining), converting the data to maps. The data used in this study were obtained from surveys completed by farmers who are principally dedicated to goat and sheep production. With the self-organizing map we found a relationship between qualitative and quantitative variables showing that more specialized farms have greater milk incomes per goat, highlighting farms that have a greater number of animals, better facilities (including milking machines) or animals fed with elaborated diets. The use of self-organizing maps for the analysis of this kind of data has proven to be highly valuable in extracting qualitative conclusions and in guiding improvements in farm performance.     

A new model of self-organizing neural networks and its applicationin data projection

In this paper a new model of self-organizing neural networks is proposed. An algorithm called "double self-organizing feature map" (DSOM) algorithm is developed to train the novel model. By the DSOM algorithm the network will adaptively adjust its network structure during the learning phase so as to make neurons responding to similar stimulus have similar weight vectors and spatially move nearer to each other at the same time. The final network structure allows us to visualize high-dimensional data as a two dimensional scatter plot. The resulting representations allow a straightforward analysis of the inherent structure of clusters within the input data. One high-dimensional data set is used to test the effectiveness of the proposed neural networks.     

Bankruptcy analysis with self-organizing maps in learning metrics

We introduce a method for deriving a metric, locally based on the Fisher information matrix, into the data space. A self-organizing map (SOM) is computed in the new metric to explore financial statements of enterprises. The metric measures local distances in terms of changes in the distribution of an auxiliary random variable that reflects what is important in the data. In this paper the variable indicates bankruptcy within the next few years. The conditional density of the auxiliary variable is first estimated, and the change in the estimate resulting from local displacements in the primary data space is measured using the Fisher information matrix. When a self-organizing map is computed in the new metric it still visualizes the data space in a topology-preserving fashion, but represents the (local) directions in which the probability of bankruptcy changes the most.     

Dynamic self-organizing maps with controlled growth for knowledgediscovery

The growing self-organizing map (GSOM) algorithm is presented in detail and the effect of a spread factor, which can be used to measure and control the spread of the GSOM, is investigated. The spread factor is independent of the dimensionality of the data and as such can be used as a controlling measure for generating maps with different dimensionality, which can then be compared and analyzed with better accuracy. The spread factor is also presented as a method of achieving hierarchical clustering of a data set with the GSOM. Such hierarchical clustering allows the data analyst to identify significant and interesting clusters at a higher level of the hierarchy, and continue with finer clustering of the interesting clusters only. Therefore, only a small map is created in the beginning with a low spread factor, which can be generated for even a very large data set. Further analysis is conducted on selected sections of the data and of smaller volume. Therefore, this method facilitates the analysis of even very large data sets.     

Visualization of randomly ordered numeric data sets using spherical self-organizing feature maps

Scientific data visualization requires a variety of mathematical techniques to transform multivariate data sets into simple graphical objects, or glyphs, that provide scientists and engineers with a clearer understanding of the underlying system behaviour. The spherical self-organizing feature map (SOFM) described in this paper exploits an unsupervised clustering algorithm to map randomly organized N-dimensional data into a lower three-dimensional (3D) space for visual pattern analysis. Each node on the spherical lattice corresponds to a cluster of input vectors that lie in close spatial proximity within the original feature space, and neighbouring nodes on the lattice represent cluster centres with a high degree of vector similarity. Simple metrics are used to extract associations between the cluster units and the input vectors assigned to them. These are then graphically displayed on the spherical SOFM as either surface elevations or colourized facets. The resulting colourized graphical objects are displayed and manipulated within 3D immersive virtual reality (IVR) environments for interactive data analysis. The ability of the proposed algorithm to transform arbitrarily arranged numeric strings into unique, reproducible shapes is illustrated using chaotic data generated by the Lozi, Hénon, Rössler, and Lorenz attractor functions under varying initial conditions. Implementation of the basic data visualization technique is further demonstrated using the more common Wisconsin breast cancer data and multi-spectral satellite data.     

Fractal initialization for high-quality mapping with self-organizing maps

Initialization of self-organizing maps is typically based on random vectors within the given input space. The implicit problem with random initialization is the overlap (entanglement) of connections between neurons. In this paper, we present a new method of initialization based on a set of self-similar curves known as Hilbert curves. Hilbert curves can be scaled in network size for the number of neurons based on a simple recursive (fractal) technique, implicit in the properties of Hilbert curves. We have shown that when using Hilbert curve vector (HCV) initialization in both classical SOM algorithm and in a parallel-growing algorithm (ParaSOM), the neural network reaches better coverage and faster organization.     

Bi-clustering continuous data with self-organizing map

In this paper, we present a new SOM-based bi-clustering approach for continuous data. This approach is called Bi-SOM (for Bi-clustering based on Self-Organizing Map). The main goal of bi-clustering aims to simultaneously group the rows and columns of a given data matrix. In addition, we propose in this work to deal with some issues related to this task: (1) the topological visualization of bi-clusters with respect to their neighborhood relation, (2) the optimization of these bi-clusters in macro-blocks and (3) the dimensionality reduction by eliminating noise blocks, iteratively. Finally, experiments are given over several data sets for validating our approach in comparison with other bi-clustering methods.     

PicSOM – content-based image retrieval with self-organizing maps

We have developed a novel system for content-based image retrieval in large, unannotated databases. The system is called PicSOM, and it is based on tree structured self-organizing maps (TS-SOMs). Given a set of reference images, PicSOM is able to retrieve another set of images which are similar to the given ones. Each TS-SOM is formed with a different image feature representation like color, texture, or shape. A new technique introduced in PicSOM facilitates automatic combination of responses from multiple TS-SOMs and their hierarchical levels. This mechanism adapts to the user's preferences in selecting which images resemble each other. Thus, the mechanism implements a relevance feedback technique on content-based image retrieval. The image queries are performed through the World Wide Web and the queries are iteratively refined as the system exposes more images to the user.     

ParaGro: a learning algorithm for growing parallel self-organizing maps with any input/output dimensions

Self-organizing maps (SOM) have become popular for tasks in data visualization, pattern classification or natural language processing and can be seen as one of the major contemporary concepts for artificial neural networks. The general idea is to approximate a high dimensional and previously unknown input distribution by a lower dimensional neural network structure so that the topology of the input space is mapped closely. Not only is the general topology retained but the relative densities of the input space are reflected in the final output. Kohonen maps also have the property of neighbor influence. That is, when a neuron decides to move, it pulls all of its neighbors in the same direction modified by an elasticity factor. We present a SOM that processes the whole input in parallel and organizes itself over time. The main reason for parallel input processing lies in the fact that knowledge can be used to recognize parts of patterns in the input space that have already been learned. Thus, networks can be developed that do not reorganize their structure from scratch every time a new set of input vectors is presented, but rather adjust their internal architecture in accordance with previous mappings. One basic application could be a modeling of the whole–part relationship through layered architectures.

The presented neural network model implements growing parallel SOM structure for any input and any output dimension. The advantage of the proposed algorithm is in its property of processing the whole input space in one step. All nodes of the network compute their step simultaneously, and are, therefore, able to detect known patterns without reorganizing. The simulation results support the theoretical framework presented in the following sections.     

VLSI circuit placement with rectilinear modules using three-layerforce-directed self-organizing maps

In this paper, a three-layer force-directed self-organizing map is designed to resolve the circuit placement problem with arbitrarily shaped rectilinear modules. The proposed neural model with an additional hidden layer can easily model a rectilinear module by a set of hidden neurons to correspond the partitioned rectangles. With the collective computing from hidden neurons, these rectilinear modules can correctly interact with each other and finally converge to a good placement result. In this paper, multiple contradictory criteria are accounted simultaneously during the placement process, in which, both the wire length and the module overlap are reduced. The proposed model has been successfully exploited to solve the time consuming rectilinear module placement problem. The placement results of real rectilinear test examples are presented, which demonstrate that the proposed method is better than the simulated annealing approach in the total wire length. The appropriate parameter values which yield good solutions are also investigated.     

PRSOM: a new visualization method by hybridizing multidimensional scaling and self-organizing map

Self-organizing map (SOM) is an approach of nonlinear dimension reduction and can be used for visualization. It only preserves topological structures of input data on the projected output space. The interneuron distances of SOM are not preserved from input space into output space such that the visualization of SOM can be degraded. Visualization-induced SOM (ViSOM) has been proposed to overcome this problem. However, ViSOM is derived from heuristic and no cost function is assigned to it. In this paper, a probabilistic regularized SOM (PRSOM) is proposed to give a better visualization effect. It is associated with a cost function and gives a principled rule for weight-updating. The advantages of both multidimensional scaling (MDS) and SOM are incorporated in PRSOM. Like MDS, The interneuron distances of PRSOM in input space resemble those in output space, which are predefined before training. Instead of the hard assignment by ViSOM, the soft assignment by PRSOM can be further utilized to enhance the visualization effect. Experimental results demonstrate the effectiveness of the proposed PRSOM method compared with other dimension reduction methods.