Constructive feedforward ART clustering networks. I

Part I of this paper proposes a definition of the adaptive resonance theory (ART) class of constructive unsupervised on-line learning clustering networks. Class ART generalizes several well-known clustering models, e.g., ART 1, improved ART 1, adaptive Hamming net (AHN), and Fuzzy ART, which are optimized in terms of memory storage and/or computation time. Next, the symmetric Fuzzy ART (S-Fuzzy ART) network is presented as a possible improvement over Fuzzy ART. As a generalization of S-Fuzzy ART, the simplified adaptive resonance theory (SART) group of ART algorithms is defined. Gaussian ART (GART), which is found in the literature, is presented as one more instance of class SART. In Part II of this work, a novel SART network, called fully self-organizing SART (FOSART), is proposed and compared with Fuzzy ART, S-Fuzzy ART, GART and other well-known clustering algorithms. Results of our comparison may easily extend to the ARTMAP supervised learning framework.     

A survey of fuzzy clustering algorithms for pattern recognition. II

For pt.I see ibid., p.775-85. In part I an equivalence between the concepts of fuzzy clustering and soft competitive learning in clustering algorithms is proposed on the basis of the existing literature. Moreover, a set of functional attributes is selected for use as dictionary entries in the comparison of clustering algorithms. In this paper, five clustering algorithms taken from the literature are reviewed, assessed and compared on the basis of the selected properties of interest. These clustering models are (1) self-organizing map (SOM); (2) fuzzy learning vector quantization (FLVQ); (3) fuzzy adaptive resonance theory (fuzzy ART); (4) growing neural gas (GNG); (5) fully self-organizing simplified adaptive resonance theory (FOSART). Although our theoretical comparison is fairly simple, it yields observations that may appear parodoxical. First, only FLVQ, fuzzy ART, and FOSART exploit concepts derived from fuzzy set theory (e.g., relative and/or absolute fuzzy membership functions). Secondly, only SOM, FLVQ, GNG, and FOSART employ soft competitive learning mechanisms, which are affected by asymptotic misbehaviors in the case of FLVQ, i.e., only SOM, GNG, and FOSART are considered effective fuzzy clustering algorithms.     

基于SOFM网络的聚类分析

基于自组织特征映射网络的聚类分析,是在神经网络基础上发展起来的一种新的非监督聚类方法,分析了基于自组织特征映射网络聚类的学习过程,分析了权系数自组织过程中邻域函数和学习步长的一般取值问题,给出了基于自组织特征映射网络聚类实现的具体算法,并通过实际示例测试,证实了算法的正确性。     

TurSOM: A paradigm bridging Turing's unorganized machines and self-organizing maps demonstrating dual self-organization

Self-organization is a widely used technique in unsupervised learning and data analysis, largely exemplified by k-means clustering, self-organizing maps (SOM) and adaptive resonance theory.In this paper we present a new algorithm: TurSOM, inspired by Turing's unorganized machines and Kohonen's SOM. Turing's unorganized machines are an early model of neural networks characterized by self-organizing connections, as opposed to self-organizing neurons in SOM.TurSOM introduces three new mechanisms to facilitate both neuron and connection self-organization. These mechanisms are: a connection learning rate, connection reorganization, and a neuron responsibility radius.TurSOM is implemented in a 1-dimensional network (i.e. chain of neurons) to exemplify the theoretical implications of these features. In this paper we demonstrate that TurSOM is superior to the classical SOM algorithm in several ways: (1) speed until convergence; (2) independent clusters; and (3) tangle-free networks.     

Quantitative object motion prediction by an ART2 and Madaline combined neural network

An ART2 and a Madaline combined neural network is applied to predicting object motions in dynamic environments. The ART2 network extracts a set of coherent patterns of the object motion by its self-organizing and unsupervised learning features. The identified patterns are directed to the Madaline network to generate a quantitative prediction of the future motion states. The method does not require any presumption of the mathematical models, and is applicable to a variety of situations.     

Centroid neural network for unsupervised competitive learning

An unsupervised competitive learning algorithm based on the classical k-means clustering algorithm is proposed. The proposed learning algorithm called the centroid neural network (CNN) estimates centroids of the related cluster groups in training date. This paper also explains algorithmic relationships among the CNN and some of the conventional unsupervised competitive learning algorithms including Kohonen's self-organizing map and Kosko's differential competitive learning algorithm. The CNN algorithm requires neither a predetermined schedule for learning coefficient nor a total number of iterations for clustering. The simulation results on clustering problems and image compression problems show that CNN converges much faster than conventional algorithms with compatible clustering quality while other algorithms may give unstable results depending on the initial values of the learning coefficient and the total number of iterations.     

Pre-synaptic lateral inhibition provides a better architecture for self-organizing neural networks

Unsupervised learning is an important ability of the brain and of many artificial neural networks. A large variety of unsupervised learning algorithms have been proposed. This paper takes a different approach in considering the architecture of the neural network rather than the learning algorithm. It is shown that a self-organizing neural network architecture using pre-synaptic lateral inhibition enables a single learning algorithm to find distributed, local, and topological representations as appropriate to the structure of the input data received. It is argued that such an architecture not only has computational advantages but is a better model of cortical self-organization.     

A survey of fuzzy clustering algorithms for pattern recognition. I

Clustering algorithms aim at modeling fuzzy (i.e., ambiguous) unlabeled patterns efficiently. Our goal is to propose a theoretical framework where the expressive power of clustering systems can be compared on the basis of a meaningful set of common functional features. Part I of this paper reviews the following issues related to clustering approaches found in the literature: relative (probabilistic) and absolute (possibilistic) fuzzy membership functions and their relationships to the Bayes rule, batch and on-line learning, prototype editing schemes, growing and pruning networks, modular network architectures, topologically perfect mapping, ecological nets and neuro-fuzziness. From this discussion an equivalence between the concepts of fuzzy clustering and soft competitive learning in clustering algorithms is proposed as a unifying framework in the comparison of clustering systems. Moreover, a set of functional attributes is selected for use as dictionary entries in the comparison of clustering algorithms, which is the subject of part II of this paper.     

基于ART2改进算法的故障聚类研究

ART2（自适应谐振理论2）算法是神经网络中一种可以对模拟输入信号或二值信号进行无监督聚类的算法,所以ART2算法能够降低数据挖掘中原始数据的预处理的复杂度,提高挖掘效率。针对ART2算法中出现的聚类中心偏移的缺点,采用ART2算法与K-均值算法相结合的方法来抑制ART2中聚类中心偏移的现象。通过仿真对该方法进行了验证。     

Neural-network front ends in unsupervised learning

Proposed is an idea of partial supervision realized in the form of a neural-network front end to the schemes of unsupervised learning (clustering). This neural network leads to an anisotropic nature of the induced feature space. The anisotropic property of the space provides us with some of its local deformation necessary to properly represent labeled data and enhance efficiency of the mechanisms of clustering to be exploited afterwards. The training of the network is completed based upon available labeled patterns-a referential form of the labeling gives rise to reinforcement learning. It is shown that the discussed approach is universal and can be utilized in conjunction with any clustering method. Experimental studies are concentrated on three main categories of unsupervised learning including FUZZY ISODATA, Kohonen self-organizing maps, and hierarchical clustering.     

基于模糊聚类神经网络的镜头突变检测算法

讨论了采用无监督的模糊竞争学习算法，并结合自组织竞争网络构成的一种新型模糊聚类神经网络模型，提出了一种基于该网络模型的镜头突变检测算法。该算法通过对线性特征空间进行由粗到细的两步模糊聚类实现镜头突变的检测。实验结果表明该算法是可行和有效的。     

The BP-SOM architecture and learning rule

For some problems, the back-propagation learning rule often used for training multilayer feedforward networks appears to have serious limitations. In this paper we describe BP-SOM, an alternative training procedure. In BP-SOM the traditional back-propagation learning rule is combined with unsupervised learning in self-organizing maps. While the multilayer feedforward network is trained, the hidden-unit activations of the feedforward network are used as training material for the accompanying self-organizing maps. After a few training cycles, the maps develop, to a certain extent, self-organization. The information in the maps is used in updating the connection weights of the feedforward network. The effect is that during BP-SOM learning, hidden-unit activations of patterns, associated with the same class, becomemore similar to each other. Results on two hard to learn classification tasks show that the BP-SOM architecture and learning rule offer a strong alternative for training multilayer feedforward networks with back-propagation.     

Comparative analysis of fuzzy ART and ART-2A network clusteringperformance

Adaptive resonance theory (ART) describes a family of self-organizing neural networks, capable of clustering arbitrary sequences of input patterns into stable recognition codes. Many different types of ART networks have been developed to improve clustering capabilities. We compare clustering performance of different types of ART networks: fuzzy ART, ART 2A with and without complement encoded input patterns, and a Euclidean ART 2A-variation. All types are tested with two- and high-dimensional input patterns in order to illustrate general capabilities and characteristics in different system environments. Based on our simulation results, fuzzy ART seems to be less appropriate whenever input signals are corrupted by addititional noise, while ART 2A-type networks keep stable in all inspected environments. Together with other examined features, ART architectures suited for particular applications can be selected.     

ART-type CMAC network classifier

In this paper, the learning process of ART 2 (adaptive resonant theory) network is applied to construct the structure of cerebellar model articulation controller (CMAC) to form an ART-type CMAC network. The proposed updating rule is in an unsupervised manner as the ART 2 network or the self-organizing map (SOM), and could equally distribute the learning information into the association memory locations as the CMAC network. If the winner fails a vigilance test, a new state is created; otherwise, the memory contents corresponding to the winner are updated according to the learning information. Like SOM, the proposed network also has a neighborhood region, but the neighborhood region is implicit in the network structure and need not be defined in advance. This paper also analyzes the convergence properties of the ART-type CMAC network. The proposed network is applied to solve data classification problems for illustration. Experiment results demonstrate the effectiveness and feasibility of the ART-type CMAC network in solving five benchmark datasets selected from the UCI repository.     

Color clustering and learning for image segmentation based on neural networks

An image segmentation system is proposed for the segmentation of color image based on neural networks. In order to measure the color difference properly, image colors are represented in a modified L/sup */u/sup */v/sup */ color space. The segmentation system comprises unsupervised segmentation and supervised segmentation. The unsupervised segmentation is achieved by a two-level approach, i.e., color reduction and color clustering. In color reduction, image colors are projected into a small set of prototypes using self-organizing map (SOM) learning. In color clustering, simulated annealing (SA) seeks the optimal clusters from SOM prototypes. This two-level approach takes the advantages of SOM and SA, which can achieve the near-optimal segmentation with a low computational cost. The supervised segmentation involves color learning and pixel classification. In color learning, color prototype is defined to represent a spherical region in color space. A procedure of hierarchical prototype learning (HPL) is used to generate the different sizes of color prototypes from the sample of object colors. These color prototypes provide a good estimate for object colors. The image pixels are classified by the matching of color prototypes. The experimental results show that the system has the desired ability for the segmentation of color image in a variety of vision tasks.     

基于深度自动编码器的脑网络状态观测矩阵降维方法

针对人脑网络状态观测矩阵高维无特征的特点,给出了一种基于深度自动编码器(DAE)的降维算法.利用深度学习网络,将高维的人脑网络空间表达映射到低维的本质特征空间中,为进一步提炼脑网络的动态性能提供了基础.实验结果证明:应用该方法可以达到有效的降维效果,且降维后脑网络状态通过自组织特征映射聚类具有一定的规律性,从而为脑网络的动态特性研究提供了基础.     

A comparison between habituation and conscience mechanism in self-organizing maps

In this letter, a preliminary study of habituation in self-organizing networks is reported. The habituation model implemented allows us to obtain a faster learning process and better clustering performances. The habituable neuron is a generalization of the typical neuron and can be used in many self-organizing network models. The habituation mechanism is implemented in a SOM and the clustering performances of the network are compared to the conscience learning mechanism that follows roughly the same principle but is less sophisticated.     

基于ART 的RBF 网络结构设计

针对径向基函数(RBF)网络隐层结构难以确定的问题,基于自适应共振理论(ART)网络良好的在线分类特性,提出一种RBF网络结构设计算法。该算法将ART网络的聚类特性用于RBF网络结构设计中,通过对输入向量与已存模式的相似度比较将输入向量进行分类,确定隐含层节点个数和初始参数,使网络具有精简的结构。对典型非线性函数逼近的仿真结果表明,所提出的结构具有快速的学习能力和良好的逼近能力。     

Learning in linear neural networks: a survey

Networks of linear units are the simplest kind of networks, where the basic questions related to learning, generalization, and self-organization can sometimes be answered analytically. We survey most of the known results on linear networks, including: 1) backpropagation learning and the structure of the error function landscape, 2) the temporal evolution of generalization, and 3) unsupervised learning algorithms and their properties. The connections to classical statistical ideas, such as principal component analysis (PCA), are emphasized as well as several simple but challenging open questions. A few new results are also spread across the paper, including an analysis of the effect of noise on backpropagation networks and a unified view of all unsupervised algorithms.     

A cortex-like learning machine for temporal hierarchical pattern clustering, detection, and recognition

A learning machine, called a clustering interpreting probabilistic associative memory (CIPAM), is proposed. CIPAM consists of a clusterer and an interpreter. The clusterer is a recurrent hierarchical neural network of unsupervised processing units (UPUs). The interpreter is a number of supervised processing units (SPUs) that branch out from the clusterer. Each processing unit (PU), UPU or SPU, comprises “dendritic encoders” for encoding inputs to the PU, “synapses” for storing resultant codes, a “nonspiking neuron” for generating inhibitory graded signals to modulate neighboring spiking neurons, “spiking neurons” for computing the subjective probability distribution (SPD) or the membership function, in the sense of fuzzy logic, of the label of said inputs to the PU and generating spike trains with the SPD or membership function as the firing rates, and a masking matrix for maximizing generalization. While UPUs employ unsupervised covariance learning mechanisms, SPUs employ supervised ones. They both also have unsupervised accumulation learning mechanisms. The clusterer of CIPAM clusters temporal and spatial data. The interpreter interprets the resultant clusters, effecting detection and recognition of temporal and hierarchical causes.