Accurate and reliable diagnosis and classification using probabilistic ensemble simplified fuzzy ARTMAP

In this paper, an accurate and effective probabilistic plurality voting method to combine outputs from multiple simplified fuzzy ARTMAP (SFAM) classifiers is presented. Five ELENA benchmark problems and five medical benchmark data sets have been used to evaluate the applicability and performance of the proposed probabilistic ensemble simplified fuzzy ARTMAP (PESFAM) network. Among the five benchmark problems in ELENA project, PESFAM outperforms the SFAM and multi-layer perceptron (MLP) classifier. In addition, the effectiveness of the proposed PESFAM is delineated in medical diagnosis applications. For the medical diagnosis and classification problems, PESFAM achieves 100 percent in accuracy, specificity, and sensitivity based on the 10-fold crossvalidation and these results are superior to those from other classification algorithms. In addition, a posteri probability of the predicted class can be used to measure the prediction reliability of PESFAM. The experiments demonstrate the potential of the proposed multiple SFAM classifiers in offering an optimal solution to the data-ordering problem of SFAM implementation and also as an intelligent medical diagnosis tool.     

Parallel implementation of the fuzzy ARTMAP neural network paradigm on a hypercube

Abstract: The recent surge of interest in connectionist models arose through the availability of high speed parallel supercomputers and the advent of new learning algorithms. The computations performed on concurrent architectures are less costly than similar ones performed on sequential machines. In this paper, the design and implementation of a parallel version of fuzzy ARTMAP (Carpenter et al. 1992), which encompasses both neural and fuzzy logic, is discussed. Fuzzy ARTMAP is a supervised learning algorithm utilising two fuzzy ART modules and an associated mapping network. A simplified version of fuzzy ARTMAP (SFAM) was designed by incorporating a simplification of the match tracking concept on unsupervised fuzzy ART paradigms. The proposed simplified version consists of only one fuzzy ART module and an associated mapping network. A parallel fuzzy ARTMAP (PFAM) algorithm is then designed and implemented on a hypercube simulator (iPSC). The algorithm is parallelised for any architecture and, with the exception of issues related to communications, the implementation remains the same on any type of parallel machine. PFAM enjoys the advantage of reduced training time that makes the algorithm a successful candidate for applications that require both online testing and training. Such applications can range from underwater sonar detection and chemical plant processing control to nuclear reactor process control, flexible manufacturing and systems analysis.     

Affect classification using genetic-optimized ensembles of fuzzy ARTMAPs

Training neural networks in distinguishing different emotions from physiological signals frequently involves fuzzy definitions of each affective state. In addition, manual design of classification tasks often uses sub-optimum classifier parameter settings, leading to average classification performance. In this study, an attempt to create a framework for multi-layered optimization of an ensemble of classifiers to maximize the system's ability to learn and classify affect, and to minimize human involvement in setting optimum parameters for the classification system is proposed. Using fuzzy adaptive resonance theory mapping (ARTMAP) as the classifier template, genetic algorithms (GAs) were employed to perform exhaustive search for the best combination of parameter settings for individual classifier performance. Speciation was implemented using subset selection of classification data attributes, as well as using an island model genetic algorithms method. Subsequently, the generated population of optimum classifier configurations was used as candidates to form an ensemble of classifiers. Another set of GAs were used to search for the combination of classifiers that would result in the best classification ensemble accuracy. The proposed methodology was tested using two affective data sets and was able to produce relatively small ensembles of fuzzy ARTMAPs with excellent affect recognition accuracy.     

Dropout prediction in e-learning courses through the combination of machine learning techniques

In this paper, a dropout prediction method for e-learning courses, based on three popular machine learning techniques and detailed student data, is proposed. The machine learning techniques used are feed-forward neural networks, support vector machines and probabilistic ensemble simplified fuzzy ARTMAP. Since a single technique may fail to accurately classify some e-learning students, whereas another may succeed, three decision schemes, which combine in different ways the results of the three machine learning techniques, were also tested. The method was examined in terms of overall accuracy, sensitivity and precision and its results were found to be significantly better than those reported in relevant literature.     

Using genetic algorithm to select the presentation order of training patterns that improves simplified fuzzy ARTMAP classification performance

The presentation order of training patterns to a simplified fuzzy ARTMAP (SFAM) neural network affects the classification performance. The common method to solve this problem is to use several simulations with training patterns presented in random order, where voting strategy is used to compute the final performance. Recently, an ordering method based on min–max clustering was introduced to select the presentation order of training patterns based on a single simulation. In this paper, another single simulation method based on genetic algorithm is proposed to obtain the presentation order of training patterns for improving the performance of SFAM. The proposed method is applied to a 40-class individual classification problem using visual evoked potential signals and three other datasets from UCI repository. The proposed method has the advantages of improved classification performance, smaller network size and lower training time compared to the random ordering and min–max methods. When compared to the random ordering method, the new ordering scheme has the additional advantage of requiring only a single simulation. As the proposed method is general, it can also be applied to a fuzzy ARTMAP neural network when it is used as a classifier.     

Ensembles of ARTMAP-based neural networks: an experimental study

ARTMAP-based models are neural networks which use a match-based learning procedure. The main advantage of ARTMAP-based models over error-based models, such as Multi-Layer Perceptron, is the learning time, which is considered as significantly fast. This feature is extremely important in complex systems that require the use of several models, such as ensembles or committees, since they produce robust and fast classifiers. Subsequently, some extensions of the ARTMAP model have been proposed, such as: ARTMAP-IC, RePART, among others. Aiming to add an extra contribution to ARTMAP context, this paper presents an analysis of ARTMAP-based models in ensemble systems. As a result of this analysis, two main goals are aimed, which are: to analyze the influence of the RePART model in ensemble systems and to detect any relation between diversity and accuracy in ensemble systems in order to use this relation in the design of these systems.     

The MSFAM: a modified fuzzy ARTMAP system

A fuzzy ARTMAP system is a system for incremental supervised learning of recognition categories and multi-dimensional maps in response to an arbitrary sequence of analog or binary input vectors. Fuzzy ARTMAP systems have been benchmarked against a variety of machine learning, neural networks, and genetic algorithms with considerable success. Owing to many appealing properties, fuzzy ARTMAP systems provide a natural basis for many researchers. Many different approaches have been proposed to modify fuzzy ARTMAP systems. In this paper, we propose a new approach to modifying a fuzzy ARTMAP system. We refer to the new system as the modified and simplified fuzzy ARTMAP (MSFAM) system. The aims of MSFAM systems are not only to reduce the architectural redundancy of the fuzzy ARTMAP system, but also to make extracted rules more comprehensible and concise. Four data sets were used for demonstrating the performance of the proposed MSFAM system.     

An application to transient current signal based induction motor fault diagnosis of Fourier–Bessel expansion and simplified fuzzy ARTMAP

The start-up transient signals have been widely used for fault diagnosis of induction motor because they can reveal early defects in the development process, which are not easily detected with the signals in the steady state operation. However, transient signals are non-linear and contain multi components which need a suitable technique to process and identify the fault pattern. In this paper, the fault diagnosis problem of induction motor is conducted by a data driven framework where the Fourier–Bessel (FB) expansion is used as a tool to decompose transient current signal into series of single components. For each component, the statistical features in the time and the frequency domains are extracted to represent the characteristics of motor condition. The high dimensionality of the feature set is solved by generalized discriminant analysis (GDA) implementation to decrease the computational complexity of classification. In the meantime, with the aid of GDA, the separation of the feature clusters is increased, which enables the more classification accuracy to be achieved. Finally, the reduced dimensional features are used for classifier to perform the fault diagnosis results. The classifier used in this framework is the simplified fuzzy ARTMAP (SFAM) which belongs to a special class of neural networks (NNs) and provides a lower training time in comparison to other traditional NNs. The proposed framework is validated with transient current signals from an induction motor under different conditions including bowed rotor, broken rotor bar, eccentricity, faulty bearing, mass unbalance and phase unbalance. Additionally, this paper provides the comparative performance of (i) SFAM and support vector machine (SVM), (ii) SVM in the framework and SVM combined with wavelet transform in previous studies, (iii) the use of FB decomposition and Hilbert transform decomposition. The results show that the proposed diagnosis framework is capable of significantly improving the classification accuracy.     

Fuzzy classifications using fuzzy inference networks

In this paper, fuzzy inference models for pattern classifications have been developed and fuzzy inference networks based on these models are proposed. Most of the existing fuzzy rule-based systems have difficulties in deriving inference rules and membership functions directly from training data. Rules and membership functions are obtained from experts. Some approaches use backpropagation (BP) type learning algorithms to learn the parameters of membership functions from training data. However, BP algorithms take a long time to converge and they require an advanced setting of the number of inference rules. The work to determine the number of inference rules demands lots of experiences from the designer. In this paper, self-organizing learning algorithms are proposed for the fuzzy inference networks. In the proposed learning algorithms, the number of inference rules and the membership functions in the inference rules will be automatically determined during the training procedure. The learning speed is fast. The proposed fuzzy inference network (FIN) classifiers possess both the structure and the learning ability of neural networks, and the fuzzy classification ability of fuzzy algorithms. Simulation results on fuzzy classification of two-dimensional data are presented and compared with those of the fuzzy ARTMAP. The proposed fuzzy inference networks perform better than the fuzzy ARTMAP and need less training samples.     

ARTIE and MUSCLE models: building ensemble classifiers from fuzzy ART and SOM networks

Ensemble Learning has proven to be an efficient method to improve the performance of single classifiers. In this context, the present article introduces ARTIE (ART networks in Ensembles) and MUSCLE (Multiple SOM Classifiers in Ensembles), two novel ensemble models that use Fuzzy ART and SOM networks as base classifiers, respectively. In addition, a hybrid metaheuristic solution based on Particle Swarm Optimization and Simulated Annealing is used for parameter tuning of the base classifiers. A comprehensive performance comparison using 10 benchmarking data sets indicates that the ARTIE and MUSCLE architectures consistently outperform ensembles built from standard supervised neural networks, such as the Fuzzy ARTMAP, Learning Vector Quantization, and the Extreme Learning Machine.     

Accurate and Fast Off and Online Fuzzy ARTMAP-Based Image Classification With Application to Genetic Abnormality Diagnosis

We propose and investigate the fuzzy ARTMAP neural network in off and online classification of fluorescence in situ hybridization image signals enabling clinical diagnosis of numerical genetic abnormalities. We evaluate the classification task (detecting a several abnormalities separately or simultaneously), classifier paradigm (monolithic or hierarchical), ordering strategy for the training patterns (averaging or voting), training mode (for one epoch, with validation or until completion) and model sensitivity to parameters. We find the fuzzy ARTMAP accurate in accomplishing both tasks requiring only very few training epochs. Also, selecting a training ordering by voting is more precise than if averaging over orderings. If trained for only one epoch, the fuzzy ARTMAP provides fast, yet stable and accurate learning as well as insensitivity to model complexity. Early stop of training using a validation set reduces the fuzzy ARTMAP complexity as for other machine learning models but cannot improve accuracy beyond that achieved when training is completed. Compared to other machine learning models, the fuzzy ARTMAP does not loose but gain accuracy when overtrained, although increasing its number of categories. Learned incrementally, the fuzzy ARTMAP reaches its ultimate accuracy very fast obtaining most of its data representation capability and accuracy by using only a few examples. Finally, the fuzzy ARTMAP accuracy for this domain is comparable with those of the multilayer perceptron and support vector machine and superior to those of the naive Bayesian and linear classifiers.     

A Fast Simplified Fuzzy ARTMAP Network

We present an algorithmic variant of the simplified fuzzy ARTMAP (SFAM) network, whose structure resembles those of feed-forward networks. Its difference with Kasuba's model is discussed, and their performances are compared on two benchmarks. We show that our algorithm is much faster than Kasuba's algorithm, and by increasing the number of training samples, the difference in speed grows enormously.The performances of the SFAM and the MLP (multilayer perceptron) are compared on three problems: the two benchmarks, and the Farsi optical character recognition (OCR) problem. For training the MLP two different variants of the backpropagation algorithm are used: the BPLRF algorithm (backpropagation with plummeting learning rate factor) for the benchmarks, and the BST algorithm (backpropagation with selective training) for the Farsi OCR problem.The results obtained on all of the three case studies with the MLP and the SFAM, embedded in their customized systems, show that the SFAM's convergence in fast-training mode, is faster than that of MLP, and online operation of the MLP is faster than that of the SFAM. On the benchmark problems the MLP has much better recognition rate than the SFAM. On the Farsi OCR problem, the recognition error of the SFAM is higher than that of the MLP on ill-engineered datasets, but equal on well-engineered ones. The flexible configuration of the SFAM, i.e. its capability to increase the size of the network in order to learn new patterns, as well as its simple parameter adjustment, remain unchallenged by the MLP.     

FUZZY ARTMAP在三字词声调识别中的应用

三字词音节声调模式具有连续语音中音节声调模式的特征，声调的提取和识别远较孤立字困难。采用小波变换方法提取语音基音，用Fuzzy ARTMAP神经网络进行声调识别，获得了比BP网络更好的实验结果。分析了仿真参数对识别结果的影响，讨论了Fuzzy ARTMAP神经网络中的过拟合问题，给出了一种基于Fuzzy ARTMAP神经网络的三字词声调识别方法。     

An efficient genetic selection of the presentation order in simplified fuzzy ARTMAP patterns

Simplified fuzzy ARTMAP (SFAM) is used in numerous classification problems due to its high discriminant power and low training time. However, the performance of SFAM is affected by the presentation order of the training patterns. The genetic algorithm (GA) can be considered as a solution to the problem because the selection of the training pattern order is a complicated combinatorial problem in a large search space. In this paper, a new genetic ordering method for SFAM is proposed to improve the performance of the algorithm. Special genetic operators are employed in the genetic evolution. Compared to the conventional methods, the proposed SFAM demonstrates better classification performance since it can efficiently deliver the desirable properties of parents to their offspring. To demonstrate the performance of the proposed method, we perform experiments on various databases from the UCI repository.     

根据混合选择策略的直觉模糊核匹配追踪集成算法

为了从分类器集成系统中选择出一组差异性大的子分类器,从而提高集成系统的泛化能力,提出了一种基于混合选择策略的直觉模糊核匹配追踪算法.基本思想是通过扰动训练集和特征空间生成一组子分类器;然后采用k均值聚类算法将对所得子分类器进行修剪,删去其中的冗余分类器;最后根据实际识别目标动态选择出较高识别率的分类器组合,使选择性集成规模能够随识别目标的复杂程度而自适应地变化,并基于预期识别精度实现循环集成.实验结果表明,与其他常用的分类器选择方法相比,本文方法灵活高效,具有更好的识别效果和泛化能力.     

Evolution of heterogeneous ensembles through dynamic particle swarm optimization for video-based face recognition

In many real-world applications, pattern recognition systems are designed a priori using limited and imbalanced data acquired from complex changing environments. Since new reference data often becomes available during operations, performance could be maintained or improved by adapting these systems through supervised incremental learning. To avoid knowledge corruption and sustain a high level of accuracy over time, an adaptive multiclassifier system (AMCS) may integrate information from diverse classifiers that are guided by a population-based evolutionary optimization algorithm. In this paper, an incremental learning strategy based on dynamic particle swarm optimization (DPSO) is proposed to evolve heterogeneous ensembles of classifiers (where each classifier corresponds to a particle) in response to new reference samples. This new strategy is applied to video-based face recognition, using an AMCS that consists of a pool of fuzzy ARTMAP (FAM) neural networks for classification of facial regions, and a niching version of DPSO that optimizes all FAM parameters such that the classification rate is maximized. Given that diversity within a dynamic particle swarm is correlated with diversity within a corresponding pool of base classifiers, DPSO properties are exploited to generate and evolve diversified pools of FAM classifiers, and to efficiently select ensembles among the pools based on accuracy and particle swarm diversity. Performance of the proposed strategy is assessed in terms of classification rate and resource requirements under different incremental learning scenarios, where new reference data is extracted from real-world video streams. Simulation results indicate the DPSO strategy provides an efficient way to evolve ensembles of FAM networks in an AMCS. Maintaining particle diversity in the optimization space yields a level of accuracy that is comparable to AMCS using reference ensemble-based and batch learning techniques, but requires significantly lower computational complexity than assessing diversity among classifiers in the feature or decision spaces.     

FasART模糊神经网络用于遥感图象监督分类的研究

说明了遥感图象数据的非线性性质，目视的图象分类实践是一个模糊推理的过程，模糊神经网络遥感图象分类符合其事物的内在规律，具有理论优势，分析了模糊ART，模糊ARTMAP和FasART模型的结构和原理，详细地阐述了FasART是一种基于模糊逻辑系统的神经网络，提出了一种简化的FasART模型，改变了一般遥感数据的模糊化方法，采用中巴资源一号卫星数据进行测试实验，结果表明，该简化的FasART模型能用于遥感图象的监督分类，其分类精度高于模糊ARTMAP神经网络和K均值算法，且性能稳定，有较好的抗干扰能力，尤其具有良好的处理两组相似程度比较接近的，和同组数据模式变化较大的非线性数据的能力。     

Increasing soft classification accuracy through the use of an ensemble of classifiers

Although soft classification analyses can reduce problems such as those associated with mixed pixels that impact negatively on conventional hard classifications their accuracy is often low. One approach to increasing the accuracy of soft classifications is the use of an ensemble of classifiers, an approach which has been successful for hard classifications but rarely applied for soft classifications. Four methods for combining soft classifications to increase soft classification accuracy were assessed. These methods were based on (i) the selection of the most accurate predictions on a class‐specific basis, (ii) the average of the outputs of the individual classifications for each case, (iii) the direct combination of classifications using evidential reasoning and (iv) the adaptation of the outputs to enable the use of a conventional (hard classification) ensemble approach. These four approaches were assessed with classifications of National Oceanic and Atmospheric Administration (NOAA) Advanced Very High‐Resolution Radiometer (AVHRR) imagery of Australia. The data were classified using two neural networks and a probabilistic classifier. All four ensemble approaches applied to the outputs of these three classifiers were found to increase classification accuracy. Relative to the most accurate individual classification, the increases in overall accuracy derived ranged from 2.20% to 4.45%, increases that were statistically significant at 95% level of confidence. The results highlight that ensemble approaches may be used to significantly increase soft classification accuracy.     

具有自适应类警戒参数的模糊ARTMAP神经网络

提出了一种具有自适应类警戒参数的模糊ＡＲＴＭＡＰ神经网络,为不同的模糊ＡＲＴ的类族设置了不同的警戒测试参数,并在学习过程中进行适应调整。还提出了新的非交叠超方形以及非交叠的Ｎｅｓｔｅｄ超主形的建立与扩展学习规则。