Convergence Analysis of a New MaxMin-SOMO Algorithm

Machine Intelligence Research - The convergence analysis of MaxMin-SOMO algorithm is presented. The SOM-based optimization (SOMO) is an optimization algorithm based on the self-organizing map (SOM)...     

Self-Organizing and Self-Evolving Neurons: A New Neural Network for Optimization

A self-organizing and self-evolving agents (SOSENs) neural network is proposed. Each neuron of the SOSENs evolves itself with a simulated annealing (SA) algorithm. The self-evolving behavior of each neuron is a local improvement that results in speeding up the convergence. The chance of reaching the global optimum is increased because multiple SAs are run in a searching space. Optimum results obtained by the SOSENs are better in average than those obtained by a single SA. Experimental results show that the SOSENs have less temperature changes than the SA to reach the global minimum. Every neuron exhibits a self-organizing behavior, which is similar to those of the self-organizing map (SOM), particle swarm optimization (PSO), and self-organizing migrating algorithm (SOMA). At last, the computational time of parallel SOSENs can be less than the SA     

Investigation of self-organizing map for genetic algorithm

This paper describes self-organizing maps for genetic algorithm (SOM-GA) which is the combinational algorithm of a real-coded genetic algorithm (RCGA) and self-organizing map (SOM). The self-organizing maps are trained with the information of the individuals in the population. Sub-populations are defined by the help of the trained map. The RCGA is performed in the sub-populations. The use of the sub-population search algorithm improves the local search performance of the RCGA. The search performance is compared with the real-coded genetic algorithm (RCGA) in three test functions. The results show that SOM-GA can find better solutions in shorter CPU time than RCGA. Although the computational cost for training SOM is expensive, the results show that the convergence speed of SOM-GA is accelerated according to the development of SOM training.     

Self-organizing topological tree for online vector quantization and data clustering.

The self-organizing Maps (SOM) introduced by Kohonen implement two important operations: vector quantization (VQ) and a topology-preserving mapping. In this paper, an online self-organizing topological tree (SOTT) with faster learning is proposed. A new learning rule delivers the efficiency and topology preservation, which is superior of other structures of SOMs. The computational complexity of the proposed SOTT is O(log N) rather than O(N) as for the basic SOM. The experimental results demonstrate that the reconstruction performance of SOTT is comparable to the full-search SOM and its computation time is much shorter than the full-search SOM and other vector quantizers. In addition, SOTT delivers the hierarchical mapping of codevectors and the progressive transmission and decoding property, which are rarely supported by other vector quantizers at the same time. To circumvent the shortcomings of clustering performance of classical partition clustering algorithms, a hybrid clustering algorithm that fully exploit the online learning and multiresolution characteristics of SOTT is devised. A new linkage metric is proposed which can be updated online to accelerate the time consuming agglomerative hierarchical clustering stage. Besides the enhanced clustering performance, due to the online learning capability, the memory requirement of the proposed SOTT hybrid clustering algorithm is independent of the size of the data set, making it attractive for large database.     

Self-organizing map approaches for the haplotype assembly problem

Haplotype assembly is to reconstruct a pair of haplotypes from SNP values observed in a set of individual DNA fragments. In this paper, we focus on studying minimum error correction (MEC) model for the haplotype assembly problem and explore self-organizing map (SOM) methods for this problem. Specifically, haplotype assembly by MEC is formulated into an integer linear programming model. Since the MEC problem is NP-hard and thus cannot be solved exactly within acceptable running time for large-scale instances, we investigate the ability of classical SOMs to solve the haplotype assembly problem with MEC model. Then, aiming to overcome the limits of classical SOMs, a novel SOM approach is proposed for the problem. Extensive computational experiments on both synthesized and real datasets show that the new SOM-based algorithm can efficiently reconstruct haplotype pairs in a very high accuracy under realistic parameter settings. Comparison with previous methods also confirms the superior performance of the new SOM approach.     

Optimal design of structures subjected to time history loading by swarm intelligence and an advanced metamodel

This paper proposes a new metamodeling framework that reduces the computational burden of the structural optimization against the time history loading. In order to achieve this, two strategies are adopted. In the first strategy, a novel metamodel consisting of adaptive neuro-fuzzy inference system (ANFIS), subtractive algorithm (SA), self organizing map (SOM) and a set of radial basis function (RBF) networks is proposed to accurately predict the time history responses of structures. The metamodel proposed is called fuzzy self-organizing radial basis function (FSORBF) networks. In this study, the most influential natural periods on the dynamic behavior of structures are treated as the inputs of the neural networks. In order to find the most influential natural periods from all the involved ones, ANFIS is employed. To train the FSORBF, the input–output samples are classified by a hybrid algorithm consisting of SA and SOM clusterings, and then a RBF network is trained for each cluster by using the data located. In the second strategy, particle swarm optimization (PSO) is employed to find the optimum design. Two building frame examples are presented to illustrate the effectiveness and practicality of the proposed methodology. A plane steel shear frame and a realistic steel space frame are designed for optimal weight using exact and approximate time history analyses. The numerical results demonstrate the efficiency and computational advantages of the proposed methodology.     

TurSOM: A paradigm bridging Turing's unorganized machines and self-organizing maps demonstrating dual self-organization

Self-organization is a widely used technique in unsupervised learning and data analysis, largely exemplified by k-means clustering, self-organizing maps (SOM) and adaptive resonance theory.In this paper we present a new algorithm: TurSOM, inspired by Turing's unorganized machines and Kohonen's SOM. Turing's unorganized machines are an early model of neural networks characterized by self-organizing connections, as opposed to self-organizing neurons in SOM.TurSOM introduces three new mechanisms to facilitate both neuron and connection self-organization. These mechanisms are: a connection learning rate, connection reorganization, and a neuron responsibility radius.TurSOM is implemented in a 1-dimensional network (i.e. chain of neurons) to exemplify the theoretical implications of these features. In this paper we demonstrate that TurSOM is superior to the classical SOM algorithm in several ways: (1) speed until convergence; (2) independent clusters; and (3) tangle-free networks.     

Self-organizing maps by difference of convex functions optimization

We offer an efficient approach based on difference of convex functions (DC) optimization for self-organizing maps (SOM). We consider SOM as an optimization problem with a nonsmooth, nonconvex energy function and investigated DC programming and DC algorithm (DCA), an innovative approach in nonconvex optimization framework to effectively solve this problem. Furthermore an appropriate training version of this algorithm is proposed. The numerical results on many real-world datasets show the efficiency of the proposed DCA based algorithms on both quality of solutions and topographic maps.     

基于粒子群优化的自组织特征映射神经网络及应用

采用粒子群优化(PSO)算法优化权重失真指数(LW D I),提出了基于粒子群优化的SOM(PSO-SOM)训练算法.用该算法取代K ohonen提出的启发式训练算法,同时引进核函数,以加强PSO-SOM算法的非线性聚类能力.以某工厂丙烯腈反应器数据为聚类应用研究对象,研究结果表明,与启发式训练算法相比,PSO-SOM算法能够得到较优的聚类,而且该算法实现简单、便于工程应用,对丙烯腈反应器参数调整以及收率监测具有显著的指导作用.     

Identification and modeling of nonlinear dynamical systems using a novel self-organizing RBF-based approach

In this paper, a novel self-organizing radial basis function (SORBF) neural network is proposed for nonlinear identification and modeling. The proposed SORBF consists of simultaneous network construction and parameter optimization. It offers two important advantages. First, the hidden neurons in the SORBF neural network can be added or removed, based on the neuron activity and mutual information (MI), to achieve the appropriate network complexity and maintain overall computational efficiency for identification and modeling. Second, the model performance can be significantly improved through the parameter optimization. The proposed parameter-adjustment-based optimization algorithm, utilizing the forward-only computation (FOC) algorithm instead of the traditionally forward-and-backward computation, simplifies neural network training, and thereby significantly reduces computational complexity. Additionally, the convergence of the SORBF is analyzed in both the structure organizing process phase and the phase following the modification. Lastly, the proposed approach is applied to model and identify the nonlinear dynamical systems. Simulation results demonstrate its effectiveness.     

Nonlinear Dimensionality Reduction and Data Visualization： A Review

Dimensionality reduction and data visualization are useful and important processes in pattern recognition.Many techniques have been developed in the recent years.The self-organizing map (SOM) can be an efficient method for this purpose.This paper reviews recent advances in this area and related approaches such as multidimensional scaling (MDS),nonlinear PCA,principal manifolds,as well as the connections of the SOM and its recent variant,the visualization induced SOM (ViSOM),with these approaches. The SOM is shown to produce a quantized,qualitative scaling and while the ViSOM a quantitative or metric scaling and approximates principal curve/surface.The SOM can also be regarded as a generalized MDS to relate two metric spaces by forming a topological mapping between them.The relationships among various recently proposed techniques such as ViSOM,Isomap,LLE,and eigenmap are discussed and compared.     

On the performance of self-organizing maps for the non-Euclidean Traveling Salesman Problem in the polygonal domain

In this paper, two state-of-the-art algorithms for the Traveling Salesman Problem (TSP) are examined in the multi-goal path planning problem motivated by inspection planning in the polygonal domain W. Both algorithms are based on the self-organizing map (SOM) for which an application in W is not typical. The first is Somhom’s algorithm, and the second is the Co-adaptive net. These algorithms are augmented by a simple approximation of the shortest path among obstacles in W. Moreover, the competitive and cooperative rules are modified by recent adaptation rules for the Euclidean TSP, and by proposed enhancements to improve the algorithms’ performance in the non-Euclidean TSP. Based on the modifications, two new variants of the algorithms are proposed that reduce the required computational time of their predecessors by an order of magnitude, therefore making SOM more competitive with combinatorial heuristics. The results show how SOM approaches can be used in the polygonal domain so they can provide additional features over the classical combinatorial approaches based on the complete visibility graph.     

An efficient self-organizing map designed by genetic algorithms for the traveling salesman problem

As a typical combinatorial optimization problem, the traveling salesman problem (TSP) has attracted extensive research interest. In this paper, we develop a self-organizing map (SOM) with a novel learning rule. It is called the integrated SOM (ISOM) since its learning rule integrates the three learning mechanisms in the SOM literature. Within a single learning step, the excited neuron is first dragged toward the input city, then pushed to the convex hull of the TSP, and finally drawn toward the middle point of its two neighboring neurons. A genetic algorithm is successfully specified to determine the elaborate coordination among the three learning mechanisms as well as the suitable parameter setting. The evolved ISOM (eISOM) is examined on three sets of TSP to demonstrate its power and efficiency. The computation complexity of the eISOM is quadratic, which is comparable to other SOM-like neural networks. Moreover, the eISOM can generate more accurate solutions than several typical approaches for TSP including the SOM developed by Budinich, the expanding SOM, the convex elastic net, and the FLEXMAP algorithm. Though its solution accuracy is not yet comparable to some sophisticated heuristics, the eISOM is one of the most accurate neural networks for the TSP.     

利用自组织特征映射神经网络进行可视化聚类

自组织特征映射作为一种神经网络方法,在数据挖掘、机器学习和模式分类中得到了广泛的应用。它将高维输人空间的数据映射到一个低维、规则的栅格上,从而可以利用可视化技术探测数据的固有特性。该文说明了自组织特征映射神经网络的工作原理和具体实现算法,同时利用一个算例展示了利用自组织特征映射进行聚类时的可视化特性,包括聚类过程的可视化和聚类结果的可视化,这也是自组织特征映射得到广泛应用的原因之一。     

一种自动抽取图像中可判别区域的新方法

图像分割是图像处理中的一个难题，为了自动抽取图像中的可差别区域，提出了一种基于自组织图归约算法的区域抽取新方法，首先，利用包括颜色、纹理以及位置在内的多模特征抽算法，原始图像被转换成特征，接着，通过自组织映射学习算法，特征图映射成自组织图，然后，对自组织图实施归纳算法得到一族约简的自组织图谱系；最后，利用一个 综合的聚类有效性分析指标从约简的自组织图谱系中得到一个最优约简的自组织图，以此实现图像区域的分割，新方法的有效性通过两个评价实验得到了验证。     

A flexible multi-layer self-organizing map for generic processing of tree-structured data

A new multi-layer self-organizing map (MLSOM) is proposed for unsupervised processing tree-structured data. The MLSOM is an improved self-organizing map for handling structured data. By introducing multiple SOM layers, the MLSOM can overcome the computational speed and visualization problems of SOM for structured data (SOM-SD). Node data in different levels of a tree are processed in different layers of the MLSOM. Root nodes are dedicatedly processed on the top SOM layer enabling the MLSOM a better utilization of SOM map compared with the SOM-SD. Thus, the MLSOM exhibits better data organization, clustering, visualization, and classification results of tree-structured data. Experimental results on three different data sets demonstrate that the proposed MLSOM approach can be more efficient and effective than the SOM-SD.     

A neuro-immune model for discriminating and visualizing anomalies

A model that can detect anomalies, even when trained only with normal samples, and can learn from encounters with new anomalies is proposed. The model combines a negative selection algorithm and a self-organizing map (SOM) in an immune inspired architecture. One of the main advantages of the proposed system is that it is able to produce a visual representation of the self/non-self feature space, thanks to the topological two-dimensional map produced by the SOM. Experimental results with anomaly and classification data are presented and discussed.     

Cluster identification and separation in the growing self-organizing map: application in protein sequence classification

Growing self-organizing map (GSOM) has been introduced as an improvement to the self-organizing map (SOM) algorithm in clustering and knowledge discovery. Unlike the traditional SOM, GSOM has a dynamic structure which allows nodes to grow reflecting the knowledge discovered from the input data as learning progresses. The spread factor parameter (SF) in GSOM can be utilized to control the spread of the map, thus giving an analyst a flexibility to examine the clusters at different granularities. Although GSOM has been applied in various areas and has been proven effective in knowledge discovery tasks, no comprehensive study has been done on the effect of the spread factor parameter value to the cluster formation and separation. Therefore, the aim of this paper is to investigate the effect of the spread factor value towards cluster separation in the GSOM. We used simple k-means algorithm as a method to identify clusters in the GSOM. By using Davies–Bouldin index, clusters formed by different values of spread factor are obtained and the resulting clusters are analyzed. In this work, we show that clusters can be more separated when the spread factor value is increased. Hierarchical clusters can then be constructed by mapping the GSOM clusters at different spread factor values.     

SCALED SELF-ORGANIZING MAP – HIDDEN MARKOV MODEL ARCHITECTURE FOR BIOLOGICAL SEQUENCE CLUSTERING

The self-organizing hidden Markov model map (SOHMMM) introduces a hybrid integration of the self-organizing map (SOM) and the hidden Markov model (HMM). Its scaled, online gradient descent unsupervised learning algorithm is an amalgam of the SOM unsupervised training and the HMM reparameterized forward-backward techniques. In essence, with each neuron of the SOHMMM lattice, an HMM is associated. The image of an input sequence on the SOHMMM mesh is defined as the location of the best matching reference HMM. Model tuning and adaptation can take place directly from raw data, within an automated context. The SOHMMM can accommodate and analyze deoxyribonucleic acid, ribonucleic acid, protein chain molecules, and generic sequences of high dimensionality and variable lengths encoded directly in nonnumerical/symbolic alphabets. Furthermore, the SOHMMM is capable of integrating and exploiting latent information hidden in the spatiotemporal dependencies/correlations of sequences’ elements.     

SOM神经网络算法的研究与进展

自组织映射(Self-organizing Maps，SOM)算法是一种无导师学习方法，具有良好的自组织、可视化等特性，已经得到了广泛的应用和研究。该文系统地介绍了SOM算法的产生背景、基本算法。同时对SOM算法的参数设置和其不足进行了分析。重点归纳了其发展过程中的各种改进算法，并对其研究热点及应用领域作了简要的综述，最后展望了该算法的发展方向。