GOFAM: a hybrid neural network classifier combining fuzzy ARTMAP and genetic algorithm

Fuzzy ARTMAP (FAM), which is a supervised model from the adaptive resonance theory (ART) neural network family, is one of the conspicuous neural network classifier. The generalization/performance of FAM is affected by two important factors which are network parameters and presentation order of training data. In this paper we introduce a genetic algorithm to find a better presentation order of training data for FAM. The proposed method which is the combination of genetic algorithm with Fuzzy ARTMAP is called Genetic Ordered Fuzzy ARTMAP (GOFAM). To illustrate the effectiveness of GOFAM, several standard datasets from UCI repository of machine learning databases are experimented. The results are analyzed and compared with those from FAM and Ordered FAM which is used to determine a fixed order of training pattern presentation to FAM. Experimental results demonstrate the performance of GOFAM is much better than performance of Fuzzy ARTMAP and Ordered Fuzzy ARTMAP. In term of network size, GOFAM performs significantly better than FAM and Ordered FAM.     

A hybrid FAM‐CART model for online data classification

In this paper, an online soft computing model based on an integration between the fuzzy ARTMAP (FAM) neural network and the classification and regression tree (CART) for undertaking data classification problems is presented. Online FAM network is useful for conducting incremental learning with data samples, whereas the CART model prevails in depicting the knowledge learned explicitly in a tree structure. Capitalizing on their respective advantages, the hybrid FAM‐CART model is capable of learning incrementally while explaining its predictions with knowledge elicited from data samples. To evaluate the usefulness of FAM‐CART, 2 sets of benchmark experiments with a total of 12 problems are used in both offline and online learning modes. The results are examined and compared with those published in the literature. The experimental outcome positively indicates that the online FAM‐CART model is useful for tackling data classification tasks. In addition, a decision tree is produced to allow users in understanding the predictions, which is an important property of the hybrid FAM‐CART model in supporting decision‐making tasks.     

Parallel implementation of the fuzzy ARTMAP neural network paradigm on a hypercube

Abstract: The recent surge of interest in connectionist models arose through the availability of high speed parallel supercomputers and the advent of new learning algorithms. The computations performed on concurrent architectures are less costly than similar ones performed on sequential machines. In this paper, the design and implementation of a parallel version of fuzzy ARTMAP (Carpenter et al. 1992), which encompasses both neural and fuzzy logic, is discussed. Fuzzy ARTMAP is a supervised learning algorithm utilising two fuzzy ART modules and an associated mapping network. A simplified version of fuzzy ARTMAP (SFAM) was designed by incorporating a simplification of the match tracking concept on unsupervised fuzzy ART paradigms. The proposed simplified version consists of only one fuzzy ART module and an associated mapping network. A parallel fuzzy ARTMAP (PFAM) algorithm is then designed and implemented on a hypercube simulator (iPSC). The algorithm is parallelised for any architecture and, with the exception of issues related to communications, the implementation remains the same on any type of parallel machine. PFAM enjoys the advantage of reduced training time that makes the algorithm a successful candidate for applications that require both online testing and training. Such applications can range from underwater sonar detection and chemical plant processing control to nuclear reactor process control, flexible manufacturing and systems analysis.     

A novel fault diagnosis method of bearing based on improved fuzzy ARTMAP and modified distance discriminant technique

This paper presents a novel intelligent diagnosis method based on multiple domain features, modified distance discrimination technique and improved fuzzy ARTMAP (IFAM). The method consists of three steps. To begin with, time-domain, frequency-domain and wavelet grey moments are extracted from the raw vibration signals to demonstrate the fault-related information. Then through the modified distance discrimination technique some salient features are selected from the original feature set. Finally, the optimal feature set is input into the IFAM incorporated with similarity based on the Yu’s norm in the classification phase to identify the different fault categories. The proposed method is applied to the fault diagnosis of rolling element bearing, and the test results show that the IFAM identify the fault categories of rolling element bearing more accurately and has a better diagnosis performance compared to the FAM. Furthermore, by the application of the bootstrap method to the diagnosis results it can testify that the IFAM has more capacity of reliability and robustness.     

Application of fuzzy ARTMAP and fuzzy <Emphasis Type="Italic">c</Emphasis>-means clustering to pattern classification with incomplete data

In this paper, a hybrid neural network that is capable of incremental learning and classification of patterns with incomplete data is proposed. Fuzzy ARTMAP (FAM) is employed as the constituting network for pattern classification while fuzzy c-means (FCM) clustering is used as the underlying algorithm for processing training as well as test samples with missing features. To handle an incomplete training set, FAM is first trained using complete samples only. Missing features of the training samples are estimated and replaced using two FCM-based strategies. Then, network training is conducted using all the complete and estimated samples. To handle an incomplete test set, a non-substitution FCM-based strategy is employed so that a predicted output can be produced rapidly. The performance of the proposed hybrid network is evaluated using a benchmark problem, and its practical applicability is demonstrated using a medical diagnosis task. The results are compared, analysed and quantified statistically with the bootstrap method. Implications of the proposed network for pattern classification tasks with incomplete data are discussed.     

Incremental learning for online tool condition monitoring using Ellipsoid ARTMAP network model

In this paper, an Ellipsoid ARTMAP (EAM) network model based on incremental learning algorithm is proposed to realize online learning and tool condition monitoring. The main characteristic of EAM model is that hyper-ellipsoid is used for geometric representation of categories which can depict the sample distribution robustly and accurately. Meanwhile, adaptive resonance based strategy can realize the update of the hyper-ellipsoid node locally and monotonically. Therefore, the model has strong incremental learning ability, which guarantees the constructed classifier can learn new knowledge without forgetting the original information. Based on incremental EAM model, a tool condition monitoring system is realized. In this system, features are firstly extracted from the force and vibration signals to depict dynamic features of tool wear process. Then, fast correlation based filter (FCBF) method is introduced to select the minimum redundant features adaptively so as to decrease the feature redundancy and improve classifier robustness. Based on the selected features, EAM based incremental classifier is constructed to realize recognition of the tool wear states. To show the effectiveness of the proposed method, multi-teeth milling experiments of Ti-6Al-4V alloy were carried out. Moreover, to estimate the generation error of the classifiers accurately, a five-fold cross validation method is utilized. By comparison with the commonly used Fuzzy ARTMAP (FAM) classifier, it can be shown that the averaging recognition rate of EAM initial classifier can reach 98.67%, which is higher than FAM. Moreover, the incremental learning ability of EAM is also analyzed and compared with FAM using the new data coming from different cutting passes and tool wear category. The results show that the updated EAM classifier can get higher classification accuracy on the original knowledge while realizing effective online learning of the new knowledge.     

An online pruning strategy for supervised ARTMAP-based neural networks

Identifying an appropriate architecture of an artificial neural network (ANN) for a given task is important because learning and generalisation of an ANN is affected by its structure. In this paper, an online pruning strategy is proposed to participate in the learning process of two constructive networks, i.e. fuzzy ARTMAP (FAM) and fuzzy ARTMAP with dynamic decay adjustment (FAMDDA), and the resulting hybrid networks are called FAM/FAMDDA with temporary nodes (i.e. FAM-T and FAMDDA-T, respectively). FAM-T and FAMDDA-T possess a capability of reducing the network complexity online by removing unrepresentative neurons. The performances of FAM-T and FAMDDA-T are evaluated and compared with those of FAM and FAMDDA using a total of 13 benchmark data sets. To demonstrate the applicability of FAM-T and FAMDDA-T, a real fault detection and diagnosis task in a power plant is tested. The results from both benchmark studies and real-world application show that FAMDDA-T and FAM-T are able to yield satisfactory classification performances, with the advantage of having parsimonious network structures.     

Fuzzy ARTMAP: A neural network architecture for incrementalsupervised learning of analog multidimensional maps

A neural network architecture is introduced for incremental supervised learning of recognition categories and multidimensional maps in response to arbitrary sequences of analog or binary input vectors, which may represent fuzzy or crisp sets of features. The architecture, called fuzzy ARTMAP, achieves a synthesis of fuzzy logic and adaptive resonance theory (ART) neural networks by exploiting a close formal similarity between the computations of fuzzy subsethood and ART category choice, resonance, and learning. Four classes of simulation illustrated fuzzy ARTMAP performance in relation to benchmark backpropagation and generic algorithm systems. These simulations include finding points inside versus outside a circle, learning to tell two spirals apart, incremental approximation of a piecewise-continuous function, and a letter recognition database. The fuzzy ARTMAP system is also compared with Salzberg's NGE systems and with Simpson's FMMC system.     

The MSFAM: a modified fuzzy ARTMAP system

A fuzzy ARTMAP system is a system for incremental supervised learning of recognition categories and multi-dimensional maps in response to an arbitrary sequence of analog or binary input vectors. Fuzzy ARTMAP systems have been benchmarked against a variety of machine learning, neural networks, and genetic algorithms with considerable success. Owing to many appealing properties, fuzzy ARTMAP systems provide a natural basis for many researchers. Many different approaches have been proposed to modify fuzzy ARTMAP systems. In this paper, we propose a new approach to modifying a fuzzy ARTMAP system. We refer to the new system as the modified and simplified fuzzy ARTMAP (MSFAM) system. The aims of MSFAM systems are not only to reduce the architectural redundancy of the fuzzy ARTMAP system, but also to make extracted rules more comprehensible and concise. Four data sets were used for demonstrating the performance of the proposed MSFAM system.     

Constructive feedforward ART clustering networks. I

Part I of this paper proposes a definition of the adaptive resonance theory (ART) class of constructive unsupervised on-line learning clustering networks. Class ART generalizes several well-known clustering models, e.g., ART 1, improved ART 1, adaptive Hamming net (AHN), and Fuzzy ART, which are optimized in terms of memory storage and/or computation time. Next, the symmetric Fuzzy ART (S-Fuzzy ART) network is presented as a possible improvement over Fuzzy ART. As a generalization of S-Fuzzy ART, the simplified adaptive resonance theory (SART) group of ART algorithms is defined. Gaussian ART (GART), which is found in the literature, is presented as one more instance of class SART. In Part II of this work, a novel SART network, called fully self-organizing SART (FOSART), is proposed and compared with Fuzzy ART, S-Fuzzy ART, GART and other well-known clustering algorithms. Results of our comparison may easily extend to the ARTMAP supervised learning framework.     

A novel Euclidean quality threshold ARTMAP network and its application to pattern classification

This paper introduces a novel neural network model known as the Euclidean quality threshold ARTMAP (EQTAM) network and its application to pattern classification. The model is constructed based on fuzzy ARTMAP (FAM) and the quality threshold clustering principle. The main objective of EQTAM is to overcome the effects of training data sequences on FAM and, at the same time, to improve its classification performance. Several artificial data sets and benchmark medical data sets are used to evaluate the effectiveness of the proposed model. Performance comparisons between EQTAM and ARTMAP-based as well as other classifiers are made. From the experimental results, it can be observed that EQTAM is able to produce good results. More importantly, the performance of EQTAM is robust against the effect of training data orders or sequences.     

Evolution of heterogeneous ensembles through dynamic particle swarm optimization for video-based face recognition

In many real-world applications, pattern recognition systems are designed a priori using limited and imbalanced data acquired from complex changing environments. Since new reference data often becomes available during operations, performance could be maintained or improved by adapting these systems through supervised incremental learning. To avoid knowledge corruption and sustain a high level of accuracy over time, an adaptive multiclassifier system (AMCS) may integrate information from diverse classifiers that are guided by a population-based evolutionary optimization algorithm. In this paper, an incremental learning strategy based on dynamic particle swarm optimization (DPSO) is proposed to evolve heterogeneous ensembles of classifiers (where each classifier corresponds to a particle) in response to new reference samples. This new strategy is applied to video-based face recognition, using an AMCS that consists of a pool of fuzzy ARTMAP (FAM) neural networks for classification of facial regions, and a niching version of DPSO that optimizes all FAM parameters such that the classification rate is maximized. Given that diversity within a dynamic particle swarm is correlated with diversity within a corresponding pool of base classifiers, DPSO properties are exploited to generate and evolve diversified pools of FAM classifiers, and to efficiently select ensembles among the pools based on accuracy and particle swarm diversity. Performance of the proposed strategy is assessed in terms of classification rate and resource requirements under different incremental learning scenarios, where new reference data is extracted from real-world video streams. Simulation results indicate the DPSO strategy provides an efficient way to evolve ensembles of FAM networks in an AMCS. Maintaining particle diversity in the optimization space yields a level of accuracy that is comparable to AMCS using reference ensemble-based and batch learning techniques, but requires significantly lower computational complexity than assessing diversity among classifiers in the feature or decision spaces.     

A hybrid neural network classifier combining ordered fuzzy ARTMAP and the dynamic decay adjustment algorithm

This paper presents a novel conflict-resolving neural network classifier that combines the ordering algorithm, fuzzy ARTMAP (FAM), and the dynamic decay adjustment (DDA) algorithm, into a unified framework. The hybrid classifier, known as Ordered FAMDDA, applies the DDA algorithm to overcome the limitations of FAM and ordered FAM in achieving a good generalization/performance. Prior to network learning, the ordering algorithm is first used to identify a fixed order of training patterns. The main aim is to reduce and/or avoid the formation of overlapping prototypes of different classes in FAM during learning. However, the effectiveness of the ordering algorithm in resolving overlapping prototypes of different classes is compromised when dealing with complex datasets. Ordered FAMDDA not only is able to determine a fixed order of training patterns for yielding good generalization, but also is able to reduce/resolve overlapping regions of different classes in the feature space for minimizing misclassification during the network learning phase. To illustrate the effectiveness of Ordered FAMDDA, a total of ten benchmark datasets are experimented. The results are analyzed and compared with those from FAM and Ordered FAM. The outcomes demonstrate that Ordered FAMDDA, in general, outperforms FAM and Ordered FAM in tackling pattern classification problems.     

Classification of noisy signals using fuzzy ARTMAP neural networks

This paper describes an approach to classification of noisy signals using a technique based on the fuzzy ARTMAP neural network (FAMNN). The proposed method is a modification of the testing phase of the fuzzy ARTMAP that exhibits superior generalization performance compared to the generalization performance of the standard fuzzy ARTMAP in the presence of noise. An application to textured gray-scale image segmentation is presented. The superiority of the proposed modification over the standard fuzzy ARTMAP is established by a number of experiments using various texture sets, feature vectors and noise types. The texture sets include various aerial photos and also samples obtained from the Brodatz album. Furthermore, the classification performance of the standard and the modified fuzzy ARTMAP is compared for different network sizes. Classification results that illustrate the performance of the modified algorithm and the FAMNN are presented.     

Pattern recognition for sensor array signals using Fuzzy ARTMAP

A Fuzzy ARTMAP classifier for pattern recognition in chemical sensor array was developed based on Fuzzy Set Theory and Adaptive Resonance Theory. In contrast to most current classifiers with difficulty in detecting new analytes, the Fuzzy ARTMAP system can identify untrained analytes with comparatively high probability. And to detect presence of new analyte, the Fuzzy ARTMAP classifier does not need retraining process that is necessary for most traditional neural network classifiers. In this study, principal component analysis (PCA) was first implemented for feature extraction purpose, followed by pattern recognition using Fuzzy ARTMAP classifiers. To construct the classifier with high recognition rate, parameter sensitive analysis was applied to find critical factors and Pareto optimization was used to locate the optimum parameter setting for the classifier. The test result shows that the proposed method can not only maintain satisfactory correct classification rate for trained analytes, but also be able to detect untrained analytes at a high recognition rate. Also the Pareto optimal values of the most important parameter have been identified, which could help constructing Fuzzy ARTMAP classifiers with good classification performance in future application.     

Fuzzy classifications using fuzzy inference networks

In this paper, fuzzy inference models for pattern classifications have been developed and fuzzy inference networks based on these models are proposed. Most of the existing fuzzy rule-based systems have difficulties in deriving inference rules and membership functions directly from training data. Rules and membership functions are obtained from experts. Some approaches use backpropagation (BP) type learning algorithms to learn the parameters of membership functions from training data. However, BP algorithms take a long time to converge and they require an advanced setting of the number of inference rules. The work to determine the number of inference rules demands lots of experiences from the designer. In this paper, self-organizing learning algorithms are proposed for the fuzzy inference networks. In the proposed learning algorithms, the number of inference rules and the membership functions in the inference rules will be automatically determined during the training procedure. The learning speed is fast. The proposed fuzzy inference network (FIN) classifiers possess both the structure and the learning ability of neural networks, and the fuzzy classification ability of fuzzy algorithms. Simulation results on fuzzy classification of two-dimensional data are presented and compared with those of the fuzzy ARTMAP. The proposed fuzzy inference networks perform better than the fuzzy ARTMAP and need less training samples.     

A hybrid neural network model for rule generation and its application to process fault detection and diagnosis

In this paper, a hybrid neural network model, based on the integration of fuzzy ARTMAP (FAM) and the rectangular basis function network (RecBFN), which is capable of learning and revealing fuzzy rules is proposed. The hybrid network is able to classify data samples incrementally and, at the same time, to extract rules directly from the network weights for justifying its predictions. With regards to process systems engineering, the proposed network is applied to a fault detection and diagnosis task in a power generation station. Specifically, the efficiency of the network in monitoring the operating conditions of a circulating water (CW) system is evaluated by using a set of real sensor measurements collected from the power station. The rules extracted are analyzed, discussed, and compared with those from a rule extraction method of FAM. From the comparison results, it is observed that the proposed network is able to extract more meaningful rules with a lower degree of rule redundancy and higher interpretability within the neural network framework. The extracted rules are also in agreement with experts’ opinions for maintaining the CW system in the power generation plant.     

Fuzzy ART-based prototype classifier

Prototype classifier is based on representing every cluster by a prototype. All the input patterns that belong to that cluster will have the same label as the prototype. It should be noted that a prototype does not have to be only one data. A cluster could be represented by more than one data. In this paper, the M-dimensional rectangle of the Fuzzy ART is used as a prototype. A new tree clustering structure replaces the training phase of Fuzzy ARTMAP. The obtained clusters are used to form the prototype rectangles. These rectangles will be used in the test phase of the Fuzzy ARTMAP. This algorithm is compared to the Nearest Neighbor classifier, the Fuzzy ARTMAP, C4.5, and the fuzzy ART-Var algorithms for different values of the vigilance parameter. Databases from the UCI repository will be used for comparison. Experimental results show the good generalization capability of this new algorithm.     

Constructive feedforward ART clustering networks. II

For pt.I see ibid., p.645-61 (2002). Part I of this paper defines the class of constructive unsupervised on-line learning simplified adaptive resonance theory (SART) clustering networks. Proposed instances of class SART are the symmetric fuzzy ART (S-Fuzzy ART) and the Gaussian ART (GART) network. In Part II of our work, a third network belonging to class SART, termed fully self-organizing SART (FOSART), is presented and discussed. FOSART is a constructive, soft-to-hard competitive, topology-preserving, minimum-distance-to-means clustering algorithm capable of: 1) generating processing units and lateral connections on an example-driven basis and 2) removing processing units and lateral connections on a minibatch basis. FOSART is compared with Fuzzy ART, S-Fuzzy ART, GART and other well-known clustering techniques (e.g., neural gas and self-organizing map) in several unsupervised learning tasks, such as vector quantization, perceptual grouping and 3-D surface reconstruction. These experiments prove that when compared with other unsupervised learning networks, FOSART provides an interesting balance between easy user interaction, performance accuracy, efficiency, robustness, and flexibility     

An insect classification analysis based on shape features using quality threshold ARTMAP and moment invariant

The main objective of this paper is to investigate the use of Quality Threshold ARTMAP (QTAM) neural network in classifying the feature vectors generated by moment invariant for the insect recognition task. In this work, six different types of moment invariant technique are adopted to extract the shape features of the insect images. These moment techniques are Geometrical Moment Invariant (GMI), United Moment Invariant (UMI), Zernike Moment Invariant (ZMI), Legendre Moment Invariant (LMI), Tchebichef Moment Invariant (TMI) and Krawtchouk Moment Invariant (KMI). All the moment techniques are analyzed using the concept of intraclass and interclass analysis. In intraclass analysis, several computation methods are introduced in order to examine the invariance properties of adopted moment techniques for the same insect object. Meanwhile, the classification accuracy of neural networks is adopted to measure the interclass characteristic and the effectiveness of moment technique in extracting the shape features of insect images. Other types of neural networks are also utilized in this research work. This includes novel enhancement technique based on the Gaussian and Mahalanobis function that design to increase its prediction accuracy. All the other networks used to classify the feature vectors are based on the Fuzzy ARTMAP (FAM) neural network. The experimental results indicated that the Krawtchouk Moment Invariant technique generated the highest classification accuracy for most of the networks used and generated the smallest error for the intraclass analysis. Using different normalization technique, the Quality Threshold ARTMAP and Mahalanobis distance function (QTAM-m) network gave the highest insect recognition results when compared to other networks.