Fuzzy ARTMAP dynamic decay adjustment: An improved fuzzy ARTMAP model with a conflict resolving facility

This paper presents a hybrid neural network classifier of fuzzy ARTMAP (FAM) and the dynamic decay adjustment (DDA) algorithm. The proposed FAMDDA model is a conflict-resolving classifier that can perform stable and incremental learning while settling overlapping of hyper-rectangular prototypes of different classes in minimizing misclassification rates. The performance of FAMDDA is evaluated using a number of benchmark data sets. The results are analyzed and compared with those from FAM and a number of machine learning classifiers. The outcomes show that FAMDDA has a better generalization capability than FAM, and its performance is comparable with those from other classifiers. The effectiveness of FAMDDA is also demonstrated in an application pertaining to condition monitoring of a circulating water system in a power generation station. Implications on the effectiveness of FAMDDA from the application point of view are discussed.     

A hybrid neural network classifier combining ordered fuzzy ARTMAP and the dynamic decay adjustment algorithm

This paper presents a novel conflict-resolving neural network classifier that combines the ordering algorithm, fuzzy ARTMAP (FAM), and the dynamic decay adjustment (DDA) algorithm, into a unified framework. The hybrid classifier, known as Ordered FAMDDA, applies the DDA algorithm to overcome the limitations of FAM and ordered FAM in achieving a good generalization/performance. Prior to network learning, the ordering algorithm is first used to identify a fixed order of training patterns. The main aim is to reduce and/or avoid the formation of overlapping prototypes of different classes in FAM during learning. However, the effectiveness of the ordering algorithm in resolving overlapping prototypes of different classes is compromised when dealing with complex datasets. Ordered FAMDDA not only is able to determine a fixed order of training patterns for yielding good generalization, but also is able to reduce/resolve overlapping regions of different classes in the feature space for minimizing misclassification during the network learning phase. To illustrate the effectiveness of Ordered FAMDDA, a total of ten benchmark datasets are experimented. The results are analyzed and compared with those from FAM and Ordered FAM. The outcomes demonstrate that Ordered FAMDDA, in general, outperforms FAM and Ordered FAM in tackling pattern classification problems.     

A hybrid FAM‐CART model for online data classification

In this paper, an online soft computing model based on an integration between the fuzzy ARTMAP (FAM) neural network and the classification and regression tree (CART) for undertaking data classification problems is presented. Online FAM network is useful for conducting incremental learning with data samples, whereas the CART model prevails in depicting the knowledge learned explicitly in a tree structure. Capitalizing on their respective advantages, the hybrid FAM‐CART model is capable of learning incrementally while explaining its predictions with knowledge elicited from data samples. To evaluate the usefulness of FAM‐CART, 2 sets of benchmark experiments with a total of 12 problems are used in both offline and online learning modes. The results are examined and compared with those published in the literature. The experimental outcome positively indicates that the online FAM‐CART model is useful for tackling data classification tasks. In addition, a decision tree is produced to allow users in understanding the predictions, which is an important property of the hybrid FAM‐CART model in supporting decision‐making tasks.     

Evolutionary Fuzzy ARTMAP Neural Networks and their Applications to Fault Detection and Diagnosis

In this paper, two mutation-based evolving artificial neural networks, which are based on the Fuzzy ARTMAP (FAM) network and evolutionary programming, are proposed. The networks utilize the knowledge base extracted from a set of data to perform search and adaptation. The performances of the two networks are assessed using benchmark problems, with the results analyzed and discussed. The effects of the network parameters are evaluated through a parametric study. The applicability of the networks is also demonstrated using a real fault detection and diagnosis task in a power generation plant. The experimental results consistently indicate the usefulness of the proposed evolutionary FAM-based networks in yielding good classification performances with parsimonious network structures.     

Application of fuzzy ARTMAP and fuzzy <Emphasis Type="Italic">c</Emphasis>-means clustering to pattern classification with incomplete data

In this paper, a hybrid neural network that is capable of incremental learning and classification of patterns with incomplete data is proposed. Fuzzy ARTMAP (FAM) is employed as the constituting network for pattern classification while fuzzy c-means (FCM) clustering is used as the underlying algorithm for processing training as well as test samples with missing features. To handle an incomplete training set, FAM is first trained using complete samples only. Missing features of the training samples are estimated and replaced using two FCM-based strategies. Then, network training is conducted using all the complete and estimated samples. To handle an incomplete test set, a non-substitution FCM-based strategy is employed so that a predicted output can be produced rapidly. The performance of the proposed hybrid network is evaluated using a benchmark problem, and its practical applicability is demonstrated using a medical diagnosis task. The results are compared, analysed and quantified statistically with the bootstrap method. Implications of the proposed network for pattern classification tasks with incomplete data are discussed.     

Fuzzy ARTMAP: A neural network architecture for incrementalsupervised learning of analog multidimensional maps

A neural network architecture is introduced for incremental supervised learning of recognition categories and multidimensional maps in response to arbitrary sequences of analog or binary input vectors, which may represent fuzzy or crisp sets of features. The architecture, called fuzzy ARTMAP, achieves a synthesis of fuzzy logic and adaptive resonance theory (ART) neural networks by exploiting a close formal similarity between the computations of fuzzy subsethood and ART category choice, resonance, and learning. Four classes of simulation illustrated fuzzy ARTMAP performance in relation to benchmark backpropagation and generic algorithm systems. These simulations include finding points inside versus outside a circle, learning to tell two spirals apart, incremental approximation of a piecewise-continuous function, and a letter recognition database. The fuzzy ARTMAP system is also compared with Salzberg's NGE systems and with Simpson's FMMC system.     

GOFAM: a hybrid neural network classifier combining fuzzy ARTMAP and genetic algorithm

Fuzzy ARTMAP (FAM), which is a supervised model from the adaptive resonance theory (ART) neural network family, is one of the conspicuous neural network classifier. The generalization/performance of FAM is affected by two important factors which are network parameters and presentation order of training data. In this paper we introduce a genetic algorithm to find a better presentation order of training data for FAM. The proposed method which is the combination of genetic algorithm with Fuzzy ARTMAP is called Genetic Ordered Fuzzy ARTMAP (GOFAM). To illustrate the effectiveness of GOFAM, several standard datasets from UCI repository of machine learning databases are experimented. The results are analyzed and compared with those from FAM and Ordered FAM which is used to determine a fixed order of training pattern presentation to FAM. Experimental results demonstrate the performance of GOFAM is much better than performance of Fuzzy ARTMAP and Ordered Fuzzy ARTMAP. In term of network size, GOFAM performs significantly better than FAM and Ordered FAM.     

Accurate and Fast Off and Online Fuzzy ARTMAP-Based Image Classification With Application to Genetic Abnormality Diagnosis

We propose and investigate the fuzzy ARTMAP neural network in off and online classification of fluorescence in situ hybridization image signals enabling clinical diagnosis of numerical genetic abnormalities. We evaluate the classification task (detecting a several abnormalities separately or simultaneously), classifier paradigm (monolithic or hierarchical), ordering strategy for the training patterns (averaging or voting), training mode (for one epoch, with validation or until completion) and model sensitivity to parameters. We find the fuzzy ARTMAP accurate in accomplishing both tasks requiring only very few training epochs. Also, selecting a training ordering by voting is more precise than if averaging over orderings. If trained for only one epoch, the fuzzy ARTMAP provides fast, yet stable and accurate learning as well as insensitivity to model complexity. Early stop of training using a validation set reduces the fuzzy ARTMAP complexity as for other machine learning models but cannot improve accuracy beyond that achieved when training is completed. Compared to other machine learning models, the fuzzy ARTMAP does not loose but gain accuracy when overtrained, although increasing its number of categories. Learned incrementally, the fuzzy ARTMAP reaches its ultimate accuracy very fast obtaining most of its data representation capability and accuracy by using only a few examples. Finally, the fuzzy ARTMAP accuracy for this domain is comparable with those of the multilayer perceptron and support vector machine and superior to those of the naive Bayesian and linear classifiers.     

A novel Euclidean quality threshold ARTMAP network and its application to pattern classification

This paper introduces a novel neural network model known as the Euclidean quality threshold ARTMAP (EQTAM) network and its application to pattern classification. The model is constructed based on fuzzy ARTMAP (FAM) and the quality threshold clustering principle. The main objective of EQTAM is to overcome the effects of training data sequences on FAM and, at the same time, to improve its classification performance. Several artificial data sets and benchmark medical data sets are used to evaluate the effectiveness of the proposed model. Performance comparisons between EQTAM and ARTMAP-based as well as other classifiers are made. From the experimental results, it can be observed that EQTAM is able to produce good results. More importantly, the performance of EQTAM is robust against the effect of training data orders or sequences.     

A hybrid neural network model for rule generation and its application to process fault detection and diagnosis

In this paper, a hybrid neural network model, based on the integration of fuzzy ARTMAP (FAM) and the rectangular basis function network (RecBFN), which is capable of learning and revealing fuzzy rules is proposed. The hybrid network is able to classify data samples incrementally and, at the same time, to extract rules directly from the network weights for justifying its predictions. With regards to process systems engineering, the proposed network is applied to a fault detection and diagnosis task in a power generation station. Specifically, the efficiency of the network in monitoring the operating conditions of a circulating water (CW) system is evaluated by using a set of real sensor measurements collected from the power station. The rules extracted are analyzed, discussed, and compared with those from a rule extraction method of FAM. From the comparison results, it is observed that the proposed network is able to extract more meaningful rules with a lower degree of rule redundancy and higher interpretability within the neural network framework. The extracted rules are also in agreement with experts’ opinions for maintaining the CW system in the power generation plant.     

Dynamic multi-objective evolution of classifier ensembles for video face recognition

Due to a limited control over changing operational conditions and personal physiology, systems used for video-based face recognition are confronted with complex and changing pattern recognition environments. Although a limited amount of reference data is initially available during enrollment, new samples often become available over time, through re-enrollment, post analysis and labeling of operational data, etc. Adaptive multi-classifier systems (AMCSs) are therefore desirable for the design and incremental update of facial models. For real time recognition of individuals appearing in video sequences, facial regions are captured with one or more cameras, and an AMCS must perform fast and efficient matching against the facial model of individual enrolled to the system. In this paper, an incremental learning strategy based on particle swarm optimization (PSO) is proposed to efficiently evolve heterogeneous classifier ensembles in response to new reference data. This strategy is applied to an AMCS where all parameters of a pool of fuzzy ARTMAP (FAM) neural network classifiers (i.e., a swarm of classifiers), each one corresponding to a particle, are co-optimized such that both error rate and network size are minimized. To provide a high level of accuracy over time while minimizing the computational complexity, the AMCS integrates information from multiple diverse classifiers, where learning is guided by an aggregated dynamical niching PSO (ADNPSO) algorithm that optimizes networks according both these objectives. Moreover, pools of FAM networks are evolved to maintain (1) genotype diversity of solutions around local optima in the optimization search space and (2) phenotype diversity in the objective space. Accurate and low cost ensembles are thereby designed by selecting classifiers on the basis of accuracy, and both genotype and phenotype diversity. For proof-of-concept validation, the proposed strategy is compared to AMCSs where incremental learning of FAM networks is guided through mono- and multi-objective optimization. Performance is assessed in terms of video-based error rate and resource requirements under different incremental learning scenarios, where new data is extracted from real-world video streams (IIT-NRC and MoBo). Simulation results indicate that the proposed strategy provides a level of accuracy that is comparable to that of using mono-objective optimization and reference face recognition systems, yet requires a fraction of the computational cost (between 16% and 20% of a mono-objective strategy depending on the data base and scenario).     

A Fast Simplified Fuzzy ARTMAP Network

We present an algorithmic variant of the simplified fuzzy ARTMAP (SFAM) network, whose structure resembles those of feed-forward networks. Its difference with Kasuba's model is discussed, and their performances are compared on two benchmarks. We show that our algorithm is much faster than Kasuba's algorithm, and by increasing the number of training samples, the difference in speed grows enormously.The performances of the SFAM and the MLP (multilayer perceptron) are compared on three problems: the two benchmarks, and the Farsi optical character recognition (OCR) problem. For training the MLP two different variants of the backpropagation algorithm are used: the BPLRF algorithm (backpropagation with plummeting learning rate factor) for the benchmarks, and the BST algorithm (backpropagation with selective training) for the Farsi OCR problem.The results obtained on all of the three case studies with the MLP and the SFAM, embedded in their customized systems, show that the SFAM's convergence in fast-training mode, is faster than that of MLP, and online operation of the MLP is faster than that of the SFAM. On the benchmark problems the MLP has much better recognition rate than the SFAM. On the Farsi OCR problem, the recognition error of the SFAM is higher than that of the MLP on ill-engineered datasets, but equal on well-engineered ones. The flexible configuration of the SFAM, i.e. its capability to increase the size of the network in order to learn new patterns, as well as its simple parameter adjustment, remain unchallenged by the MLP.     

Fuzzy classifications using fuzzy inference networks

In this paper, fuzzy inference models for pattern classifications have been developed and fuzzy inference networks based on these models are proposed. Most of the existing fuzzy rule-based systems have difficulties in deriving inference rules and membership functions directly from training data. Rules and membership functions are obtained from experts. Some approaches use backpropagation (BP) type learning algorithms to learn the parameters of membership functions from training data. However, BP algorithms take a long time to converge and they require an advanced setting of the number of inference rules. The work to determine the number of inference rules demands lots of experiences from the designer. In this paper, self-organizing learning algorithms are proposed for the fuzzy inference networks. In the proposed learning algorithms, the number of inference rules and the membership functions in the inference rules will be automatically determined during the training procedure. The learning speed is fast. The proposed fuzzy inference network (FIN) classifiers possess both the structure and the learning ability of neural networks, and the fuzzy classification ability of fuzzy algorithms. Simulation results on fuzzy classification of two-dimensional data are presented and compared with those of the fuzzy ARTMAP. The proposed fuzzy inference networks perform better than the fuzzy ARTMAP and need less training samples.     

Parallel implementation of the fuzzy ARTMAP neural network paradigm on a hypercube

Abstract: The recent surge of interest in connectionist models arose through the availability of high speed parallel supercomputers and the advent of new learning algorithms. The computations performed on concurrent architectures are less costly than similar ones performed on sequential machines. In this paper, the design and implementation of a parallel version of fuzzy ARTMAP (Carpenter et al. 1992), which encompasses both neural and fuzzy logic, is discussed. Fuzzy ARTMAP is a supervised learning algorithm utilising two fuzzy ART modules and an associated mapping network. A simplified version of fuzzy ARTMAP (SFAM) was designed by incorporating a simplification of the match tracking concept on unsupervised fuzzy ART paradigms. The proposed simplified version consists of only one fuzzy ART module and an associated mapping network. A parallel fuzzy ARTMAP (PFAM) algorithm is then designed and implemented on a hypercube simulator (iPSC). The algorithm is parallelised for any architecture and, with the exception of issues related to communications, the implementation remains the same on any type of parallel machine. PFAM enjoys the advantage of reduced training time that makes the algorithm a successful candidate for applications that require both online testing and training. Such applications can range from underwater sonar detection and chemical plant processing control to nuclear reactor process control, flexible manufacturing and systems analysis.     

Probabilistic ensemble simplified fuzzy ARTMAP for sonar target differentiation

This study investigates the processing of sonar signals with ensemble neural networks for robust recognition of simple objects such as plane, corner and trapezium surface. The ensemble neural networks can differentiate the target objects with high accuracy. The simplified fuzzy ARTMAP (SFAM) and probabilistic ensemble simplified fuzzy ARTMAP (PESFAM) are compared in terms of classification accuracy. The PESFAM implements an accurate and effective probabilistic plurality voting method to combine outputs from multiple SFAM classifiers. Five benchmark data sets have been used to evaluate the applicability of the proposed ensemble SFAM network. The PESFAM achieves good accuracy based on the twofold cross-validation results. In addition, the effectiveness of the proposed ensemble SFAM is delineated in sonar target differentiation. The experiments demonstrate the potential of PESFAM classifiers in offering an optimal solution to the data-ordering problem of SFAM implementation and also as an intelligent classification tool in mobile robot application.     

Cascade ARTMAP: integrating neural computation and symbolic knowledge processing

This paper introduces a hybrid system termed cascade adaptive resonance theory mapping (ARTMAP) that incorporates symbolic knowledge into neural-network learning and recognition. Cascade ARTMAP, a generalization of fuzzy ARTMAP, represents intermediate attributes and rule cascades of rule-based knowledge explicitly and performs multistep inferencing. A rule insertion algorithm translates if-then symbolic rules into cascade ARTMAP architecture. Besides that initializing networks with prior knowledge can improve predictive accuracy and learning efficiency, the inserted symbolic knowledge can be refined and enhanced by the cascade ARTMAP learning algorithm. By preserving symbolic rule form during learning, the rules extracted from cascade ARTMAP can be compared directly with the originally inserted rules. Simulations on an animal identification problem indicate that a priori symbolic knowledge always improves system performance, especially with a small training set. Benchmark study on a DNA promoter recognition problem shows that with the added advantage of fast learning, cascade ARTMAP rule insertion and refinement algorithms produce performance superior to those of other machine learning systems and an alternative hybrid system known as knowledge-based artificial neural network (KBANN). Also, the rules extracted from cascade ARTMAP are more accurate and much cleaner than the NofM rules extracted from KBANN.     

Customer churn prediction by hybrid neural networks

As churn management is a major task for companies to retain valuable customers, the ability to predict customer churn is necessary. In literature, neural networks have shown their applicability to churn prediction. On the other hand, hybrid data mining techniques by combining two or more techniques have been proved to provide better performances than many single techniques over a number of different domain problems. This paper considers two hybrid models by combining two different neural network techniques for churn prediction, which are back-propagation artificial neural networks (ANN) and self-organizing maps (SOM). The hybrid models are ANN combined with ANN (ANN + ANN) and SOM combined with ANN (SOM + ANN). In particular, the first technique of the two hybrid models performs the data reduction task by filtering out unrepresentative training data. Then, the outputs as representative data are used to create the prediction model based on the second technique. To evaluate the performance of these models, three different kinds of testing sets are considered. They are the general testing set and two fuzzy testing sets based on the filtered out data by the first technique of the two hybrid models, i.e. ANN and SOM, respectively. The experimental results show that the two hybrid models outperform the single neural network baseline model in terms of prediction accuracy and Types I and II errors over the three kinds of testing sets. In addition, the ANN + ANN hybrid model significantly performs better than the SOM + ANN hybrid model and the ANN baseline model.     

Application of a modified fuzzy ARTMAP with feature-weight learning for the fault diagnosis of bearing

Considering different importance of the feature parameters to the fault conditions of bearing, a modified fuzzy ARTMAP (FAM) network model based on the feature-weight learning is presented in this paper. The features in time-domain, frequency-domain and wavelet-domain are extracted from the vibration signals to characterize the information relevant to the fault conditions of bearing. By the improved distance evaluation technique the optimal features are selected and the corresponding feature-weights which are assigned to the features to indicate their different importance to the fault conditions of bearing are obtained. Then they are combined with the modified FAM which is described by the weighted Manhattan distance and applied to the seven-class fault diagnosis of bearing. To assess the effectiveness and stability of the modified FAM network, bootstrapping method is employed to quantify the stability of the network performance statistically. Diagnosis results show that the modified FAM can more reliably and accurately recognize different fault classes.     

Comparing neural networks: a benchmark on growing neural gas,growing cell structures, and fuzzy ARTMAP

Compares the performance of some incremental neural networks with the well-known multilayer perceptron (MLP) on real-world data. The incremental networks are fuzzy ARTMAP (FAM), growing neural gas (GNG) and growing cell structures (GCS). The real-world datasets consist of four different datasets posing different challenges to the networks in terms of complexity of decision boundaries, overlapping between classes, and size of the datasets. The performance of the networks on the datasets is reported with respect to measure classification error, number of training epochs, and sensitivity toward variation of parameters. Statistical evaluations are applied to examine the significance of the results. The overall performance ranks in the following descending order: GNG, GCS, MLP, FAM.