Topology-oriented self-organizing maps: a survey

The self-organizing map (SOM) is a prominent neural network model that has found wide application in a spectrum of domains. Accordingly, it has received widespread attention both from the communities of researchers and practitioners. As a result, several variations of the basic architecture have been devised, specifically in the early years of the SOM’s evolution, which were introduced so as to address various architectural shortcomings or to explore other structures of the basic model. The overall goal of this survey is to present a comprehensive comparison of these networks, in terms of their primitive components and properties. We dichotomize these schemes as being either tree based or non-tree based. We have embarked on this venture with the hope that since the survey is comprehensive and the bibliography extensive, it will be an asset and resource for future researchers.     

Mining sequential patterns from data streams: a centroid approach

In recent years, emerging applications introduced new constraints for data mining methods. These constraints are typical of a new kind of data: the data streams. In data stream processing, memory usage is restricted, new elements are generated continuously and have to be considered in a linear time, no blocking operator can be performed and the data can be examined only once. At this time, only a few methods has been proposed for mining sequential patterns in data streams. We argue that the main reason is the combinatory phenomenon related to sequential pattern mining. In this paper, we propose an algorithm based on sequences alignment for mining approximate sequential patterns in Web usage data streams. To meet the constraint of one scan, a greedy clustering algorithm associated to an alignment method is proposed. We will show that our proposal is able to extract relevant sequences with very low thresholds.     

Spatial data mining and visualization based on self-organizing map

Based on the traditional spatial data analysis, a novel mode of spatial data mining and visualization is proposed which integrates the self-organizing map for the actual problem. Simulations for IRIS data show that this method （computational and visual） can collaboratively discover complex pattems in large spatial datasets, in an effective and efficient way.     

Structural enhanced information and its application to improved visualization of self-organizing maps

In this paper, we propose structural enhanced information for detecting and visualizing main features in input patterns. We have so far proposed information enhancement for feature detection, where, if we want to focus upon components such as units and connection weights and interpret the functions of the components, we have only to enhance competitive units with the components. Though this information enhancement has given favorable results in feature detection, we further refine the information enhancement and propose structural enhanced information. In structural enhanced information, three types of enhanced information can be differentiated, that is, first-, second- and third-order enhanced information. The first-order information is related to the enhancement of competitive units themselves in a competitive network, and the second-order information is dependent upon the enhancement of competitive units with input patterns. Then, the third-order information is obtained by subtracting the effect of the first-order information from the second-order information. Thus, the third-order information more explicitly represents information on input patterns. With this structural enhanced information, we can estimate more detailed features in input patterns. For demonstrating explicitly and intuitively the improved performance of our method, the conventional SOM was used, and we transformed competitive unit outputs so as to improve visualization. The method was applied to the well-known Iris problem, an OECD countries classification problem and the Johns Hopkins University Ionosphere database. In all these problems, we succeeded in visualizing the detailed and important features of input patterns by using the third-order information.     

Externally growing self-organizing maps and its application to e-mail database visualization and exploration

In this paper we present an approach to organize and classify e-mails using self-organizing maps. The aim is on the one hand to provide an intuitive visual profile of the considered mailing lists and on the other hand to offer an intuitive navigation tool, were similar e-mails are located close to each other, so that the user can scan easily for e-mails similar in content. To be able to evaluate this approach we have developed a prototypical software tool that imports messages from a mailing list and arranges/groups these e-mails based on a similarity measure. The tool combines conventional keyword search methods with a visualization of the considered e-mail collection. The prototype was developed based on externally growing self-organizing maps, which solve some problems of conventional self-organizing maps and which are computationally viable. Besides the underlying algorithms we present and discuss some system evaluations in order to show the capabilities of the approach.     

Visualization of high-dimensional data using two-dimensional self-organizing piecewise-smooth Kohonen maps

To make visualization of high-dimensional data more accurate, we offer a method of approximating two-dimensional Kohonen maps lying in a multiple-dimensional space. Cubic parametric spline-based least-defect surfaces can be used as an approximation function to minimize approximation errors.     

ViSOM - a novel method for multivariate data projection andstructure visualization

When used for visualization of high-dimensional data, the self-organizing map (SOM) requires a coloring scheme, such as the U-matrix, to mark the distances between neurons. Even so, the structures of the data clusters may not be apparent and their shapes are often distorted. In this paper, a visualization-induced SOM (ViSOM) is proposed to overcome these shortcomings. The algorithm constrains and regularizes the inter-neuron distance with a parameter that controls the resolution of the map. The mapping preserves the inter-point distances of the input data on the map as well as the topology. It produces a graded mesh in the data space such that the distances between mapped data points on the map resemble those in the original space, like in the Sammon mapping. However, unlike the Sammon mapping, the ViSOM can accommodate both training data and new arrivals and is much simpler in computational complexity. Several experimental results and comparisons with other methods are presented.     

A new model of self-organizing neural networks and its applicationin data projection

In this paper a new model of self-organizing neural networks is proposed. An algorithm called "double self-organizing feature map" (DSOM) algorithm is developed to train the novel model. By the DSOM algorithm the network will adaptively adjust its network structure during the learning phase so as to make neurons responding to similar stimulus have similar weight vectors and spatially move nearer to each other at the same time. The final network structure allows us to visualize high-dimensional data as a two dimensional scatter plot. The resulting representations allow a straightforward analysis of the inherent structure of clusters within the input data. One high-dimensional data set is used to test the effectiveness of the proposed neural networks.     

Bankruptcy analysis with self-organizing maps in learning metrics

We introduce a method for deriving a metric, locally based on the Fisher information matrix, into the data space. A self-organizing map (SOM) is computed in the new metric to explore financial statements of enterprises. The metric measures local distances in terms of changes in the distribution of an auxiliary random variable that reflects what is important in the data. In this paper the variable indicates bankruptcy within the next few years. The conditional density of the auxiliary variable is first estimated, and the change in the estimate resulting from local displacements in the primary data space is measured using the Fisher information matrix. When a self-organizing map is computed in the new metric it still visualizes the data space in a topology-preserving fashion, but represents the (local) directions in which the probability of bankruptcy changes the most.     

Dynamic self-organizing maps with controlled growth for knowledgediscovery

The growing self-organizing map (GSOM) algorithm is presented in detail and the effect of a spread factor, which can be used to measure and control the spread of the GSOM, is investigated. The spread factor is independent of the dimensionality of the data and as such can be used as a controlling measure for generating maps with different dimensionality, which can then be compared and analyzed with better accuracy. The spread factor is also presented as a method of achieving hierarchical clustering of a data set with the GSOM. Such hierarchical clustering allows the data analyst to identify significant and interesting clusters at a higher level of the hierarchy, and continue with finer clustering of the interesting clusters only. Therefore, only a small map is created in the beginning with a low spread factor, which can be generated for even a very large data set. Further analysis is conducted on selected sections of the data and of smaller volume. Therefore, this method facilitates the analysis of even very large data sets.     

Fractal initialization for high-quality mapping with self-organizing maps

Initialization of self-organizing maps is typically based on random vectors within the given input space. The implicit problem with random initialization is the overlap (entanglement) of connections between neurons. In this paper, we present a new method of initialization based on a set of self-similar curves known as Hilbert curves. Hilbert curves can be scaled in network size for the number of neurons based on a simple recursive (fractal) technique, implicit in the properties of Hilbert curves. We have shown that when using Hilbert curve vector (HCV) initialization in both classical SOM algorithm and in a parallel-growing algorithm (ParaSOM), the neural network reaches better coverage and faster organization.     

Visualization of randomly ordered numeric data sets using spherical self-organizing feature maps

Scientific data visualization requires a variety of mathematical techniques to transform multivariate data sets into simple graphical objects, or glyphs, that provide scientists and engineers with a clearer understanding of the underlying system behaviour. The spherical self-organizing feature map (SOFM) described in this paper exploits an unsupervised clustering algorithm to map randomly organized N-dimensional data into a lower three-dimensional (3D) space for visual pattern analysis. Each node on the spherical lattice corresponds to a cluster of input vectors that lie in close spatial proximity within the original feature space, and neighbouring nodes on the lattice represent cluster centres with a high degree of vector similarity. Simple metrics are used to extract associations between the cluster units and the input vectors assigned to them. These are then graphically displayed on the spherical SOFM as either surface elevations or colourized facets. The resulting colourized graphical objects are displayed and manipulated within 3D immersive virtual reality (IVR) environments for interactive data analysis. The ability of the proposed algorithm to transform arbitrarily arranged numeric strings into unique, reproducible shapes is illustrated using chaotic data generated by the Lozi, Hénon, Rössler, and Lorenz attractor functions under varying initial conditions. Implementation of the basic data visualization technique is further demonstrated using the more common Wisconsin breast cancer data and multi-spectral satellite data.     

Bi-clustering continuous data with self-organizing map

In this paper, we present a new SOM-based bi-clustering approach for continuous data. This approach is called Bi-SOM (for Bi-clustering based on Self-Organizing Map). The main goal of bi-clustering aims to simultaneously group the rows and columns of a given data matrix. In addition, we propose in this work to deal with some issues related to this task: (1) the topological visualization of bi-clusters with respect to their neighborhood relation, (2) the optimization of these bi-clusters in macro-blocks and (3) the dimensionality reduction by eliminating noise blocks, iteratively. Finally, experiments are given over several data sets for validating our approach in comparison with other bi-clustering methods.     

VLSI circuit placement with rectilinear modules using three-layerforce-directed self-organizing maps

In this paper, a three-layer force-directed self-organizing map is designed to resolve the circuit placement problem with arbitrarily shaped rectilinear modules. The proposed neural model with an additional hidden layer can easily model a rectilinear module by a set of hidden neurons to correspond the partitioned rectangles. With the collective computing from hidden neurons, these rectilinear modules can correctly interact with each other and finally converge to a good placement result. In this paper, multiple contradictory criteria are accounted simultaneously during the placement process, in which, both the wire length and the module overlap are reduced. The proposed model has been successfully exploited to solve the time consuming rectilinear module placement problem. The placement results of real rectilinear test examples are presented, which demonstrate that the proposed method is better than the simulated annealing approach in the total wire length. The appropriate parameter values which yield good solutions are also investigated.     

PRSOM: a new visualization method by hybridizing multidimensional scaling and self-organizing map

Self-organizing map (SOM) is an approach of nonlinear dimension reduction and can be used for visualization. It only preserves topological structures of input data on the projected output space. The interneuron distances of SOM are not preserved from input space into output space such that the visualization of SOM can be degraded. Visualization-induced SOM (ViSOM) has been proposed to overcome this problem. However, ViSOM is derived from heuristic and no cost function is assigned to it. In this paper, a probabilistic regularized SOM (PRSOM) is proposed to give a better visualization effect. It is associated with a cost function and gives a principled rule for weight-updating. The advantages of both multidimensional scaling (MDS) and SOM are incorporated in PRSOM. Like MDS, The interneuron distances of PRSOM in input space resemble those in output space, which are predefined before training. Instead of the hard assignment by ViSOM, the soft assignment by PRSOM can be further utilized to enhance the visualization effect. Experimental results demonstrate the effectiveness of the proposed PRSOM method compared with other dimension reduction methods.     

PicSOM – content-based image retrieval with self-organizing maps

We have developed a novel system for content-based image retrieval in large, unannotated databases. The system is called PicSOM, and it is based on tree structured self-organizing maps (TS-SOMs). Given a set of reference images, PicSOM is able to retrieve another set of images which are similar to the given ones. Each TS-SOM is formed with a different image feature representation like color, texture, or shape. A new technique introduced in PicSOM facilitates automatic combination of responses from multiple TS-SOMs and their hierarchical levels. This mechanism adapts to the user's preferences in selecting which images resemble each other. Thus, the mechanism implements a relevance feedback technique on content-based image retrieval. The image queries are performed through the World Wide Web and the queries are iteratively refined as the system exposes more images to the user.     

Cooperative information maximization with Gaussian activation functions for self-organizing maps

In this paper, we propose a new information-theoretic method to produce explicit self-organizing maps (SOMs). Competition is realized by maximizing mutual information between input patterns and competitive units. Competitive unit outputs are computed by the Gaussian function of distance between input patterns and competitive units. A property of this Gaussian function is that, as distance becomes smaller, a neuron tends to fire strongly. Cooperation processes are realized by taking into account the firing rates of neighboring neurons. We applied our method to uniform distribution learning, chemical compound classification and road classification. Experimental results confirmed that cooperation processes could significantly increase information content in input patterns. When cooperative operations are not effective in increasing information, mutual information as well as entropy maximization is used to increase information. Experimental results showed that entropy maximization could be used to increase information and to obtain clearer SOMs, because competitive units are forced to be equally used on average.     

Topology preservation in self-organizing feature maps: exactdefinition and measurement

The neighborhood preservation of self-organizing feature maps like the Kohonen map is an important property which is exploited in many applications. However, if a dimensional conflict arises this property is lost. Various qualitative and quantitative approaches are known for measuring the degree of topology preservation. They are based on using the locations of the synaptic weight vectors. These approaches, however, may fail in case of nonlinear data manifolds. To overcome this problem, in this paper we present an approach which uses what we call the induced receptive fields for determining the degree of topology preservation. We first introduce a precise definition of topology preservation and then propose a tool for measuring it, the topographic function. The topographic function vanishes if and only if the map is topology preserving. We demonstrate the power of this tool for various examples of data manifolds.     

Narrative visualization: telling stories with data

Data visualization is regularly promoted for its ability to reveal stories within data, yet these “data stories” differ in important ways from traditional forms of storytelling. Storytellers, especially online journalists, have increasingly been integrating visualizations into their narratives, in some cases allowing the visualization to function in place of a written story. In this paper, we systematically review the design space of this emerging class of visualizations. Drawing on case studies from news media to visualization research, we identify distinct genres of narrative visualization. We characterize these design differences, together with interactivity and messaging, in terms of the balance between the narrative flow intended by the author (imposed by graphical elements and the interface) and story discovery on the part of the reader (often through interactive exploration). Our framework suggests design strategies for narrative visualization, including promising under-explored approaches to journalistic storytelling and educational media.     

Approximate reconstruction of PET data with a self-organizing neural network.

Self-organization was observed using the algorithm of Kohonen with an original "distance" adapted to stimuli resulting from coincident detections of electron-positron annihilation photon pairs. This has led to a method for approximate reconstruction of two-dimensional positron emission tomography (2-D PET) images that is totally independent of the number of detectors. To obtain meaningful information about the distribution of the radioactive tracer, a toroidal architecture must be used for the network.