Diversity-based combination of non-parametric classifiers for EMG signal decomposition

Non-parametric classification procedures based on a certainty measure and nearest neighbour rule for motor unit potential classification (MUP) during electromyographic (EMG) signal decomposition were explored. A diversity-based classifier fusion approach is developed and evaluated to achieve improved classification performance. The developed system allows the construction of a set of non-parametric base classifiers and then automatically chooses, from the pool of base classifiers, subsets of classifiers to form candidate classifier ensembles. The system selects the classifier ensemble members by exploiting a diversity measure for selecting classifier teams. The kappa statistic is used as the diversity measure to estimate the level of agreement between base classifier outputs, i.e., to measure the degree of decision similarity between base classifiers. The pool of base classifiers consists of two kinds of classifiers: adaptive certainty-based classifiers (ACCs) and adaptive fuzzy k-NN classifiers (AFNNCs) and both utilize different types of features. Once the patterns are assigned to their classes, by the classifier fusion system, firing pattern consistency statistics for each class are calculated to detect classification errors in an adaptive fashion. Performance of the developed system was evaluated using real and simulated EMG signals and was compared with the performance of the constituent base classifiers and the performance of the fixed ensemble containing the full set of base classifiers. Across the EMG signal data sets used, the diversity-based classifier fusion approach had better average classification performance overall, especially in terms of reducing classification errors.     

Medical diagnosis of cardiovascular diseases using an interval-valued fuzzy rule-based classification system

ObjectiveTo develop a classifier that tackles the problem of determining the risk of a patient of suffering from a cardiovascular disease within the next 10 years. The system has to provide both a diagnosis and an interpretable model explaining the decision. In this way, doctors are able to analyse the usefulness of the information given by the system.MethodsLinguistic fuzzy rule-based classification systems are used, since they provide a good classification rate and a highly interpretable model. More specifically, a new methodology to combine fuzzy rule-based classification systems with interval-valued fuzzy sets is proposed, which is composed of three steps: (1) the modelling of the linguistic labels of the classifier using interval-valued fuzzy sets; (2) the use of the K_α operator in the inference process and (3) the application of a genetic tuning to find the best ignorance degree that each interval-valued fuzzy set represents as well as the best value for the parameter α of the K_α operator in each rule.ResultsThe suitability of the new proposal to deal with this medical diagnosis classification problem is shown by comparing its performance with respect to the one provided by two classical fuzzy classifiers and a previous interval-valued fuzzy rule-based classification system. The performance of the new method is statistically better than the ones obtained with the methods considered in the comparison. The new proposal enhances both the total number of correctly diagnosed patients, around 3% with respect the classical fuzzy classifiers and around 1% vs. the previous interval-valued fuzzy classifier, and the classifier ability to correctly differentiate patients of the different risk categories.ConclusionThe proposed methodology is a suitable tool to face the medical diagnosis of cardiovascular diseases, since it obtains a good classification rate and it also provides an interpretable model that can be easily understood by the doctors.     

How good are fuzzy If-Then classifiers?

This paper gives some known theoretical results about fuzzy rule-based classifiers and offers a few new ones. The ability of Takagi-Sugeno-Kang (TSK) fuzzy classifiers to match exactly and to approximate classification boundaries is discussed. The lemma by Klawonn and Klement about the exact match of a classification boundary in R (2) is extended from monotonous to arbitrary functions. Equivalence between fuzzy rule-based and nonfuzzy classifiers (1-nn and Parzen) is outlined. We specify the conditions under which a class of fuzzy TSK classifiers turn into lookup tables. It is shown that if the rule base consists of all possible rules (all combinations of linguistic labels on the input features), the fuzzy TSK model is a lookup classifier with hyperbox cells, regardless of the type (shape) of the membership functions used. The question "why fuzzy?" is addressed in the light of these results.     

Neutrosophic classifier: An extension of fuzzy classifer

Fuzzy classification has become of great interest because of its ability to utilize simple linguistically interpretable rules and has overcome the limitations of symbolic or crisp rule based classifiers. This paper introduces an extension to fuzzy classifier: a neutrosophic classifier, which would utilize neutrosophic logic for its working. Neutrosophic logic is a generalized logic that is capable of effectively handling indeterminacy, stochasticity acquisition errors that fuzzy logic cannot handle. The proposed neutrosophic classifier employs neutrosophic logic for its working and is an extension of commonly used fuzzy classifier. It is compared with the commonly used fuzzy classifiers on the following parameters: nature of membership functions, number of rules and indeterminacy in the results generated. It is proved in the paper that extended fuzzy classifier: neutrosophic classifier; optimizes the said parameters in comparison to the fuzzy counterpart. Finally the paper is concluded with justifying that neutrosophic logic though in its nascent stage still holds the potential to be experimented for further exploration in different domains.     

基于类标签聚类的动态问题分类集成学习算法

问题分类是问答社区系统的关键技术,分析用户提出的自然语言问题,并返回一个确切而适当的问题类别.针对网络社区中问题分类标签众多(＞1 000)、有一定层次且易受时间演化影响的问题,提出了针对两种不同流动粒度的问题分类算法,运用不同时刻的数据集层次集成学习方法提高了问题分类精度和效率.同时,针对单次分类标签过多引起的特征集...     

Pattern recognition for sensor array signals using Fuzzy ARTMAP

A Fuzzy ARTMAP classifier for pattern recognition in chemical sensor array was developed based on Fuzzy Set Theory and Adaptive Resonance Theory. In contrast to most current classifiers with difficulty in detecting new analytes, the Fuzzy ARTMAP system can identify untrained analytes with comparatively high probability. And to detect presence of new analyte, the Fuzzy ARTMAP classifier does not need retraining process that is necessary for most traditional neural network classifiers. In this study, principal component analysis (PCA) was first implemented for feature extraction purpose, followed by pattern recognition using Fuzzy ARTMAP classifiers. To construct the classifier with high recognition rate, parameter sensitive analysis was applied to find critical factors and Pareto optimization was used to locate the optimum parameter setting for the classifier. The test result shows that the proposed method can not only maintain satisfactory correct classification rate for trained analytes, but also be able to detect untrained analytes at a high recognition rate. Also the Pareto optimal values of the most important parameter have been identified, which could help constructing Fuzzy ARTMAP classifiers with good classification performance in future application.     

Hierarchical distance learning by stacking nearest neighbor classifiers

We propose a two-layer decision fusion technique, called Fuzzy Stacked Generalization (FSG) which establishes a hierarchical distance learning architecture. At the base-layer of an FSG, fuzzy k-NN classifiers receive different feature sets each of which is extracted from the same dataset to gain multiple views of the dataset. At the meta-layer, first, a fusion space is constructed by aggregating decision spaces of all the base-layer classifiers. Then, a fuzzy k-NN classifier is trained in the fusion space by minimizing the difference between the large sample and N-sample classification error. In order to measure the degree of collaboration among the base-layer classifiers and the diversity of the feature spaces, a new measure called, shareability, is introduced. Shearability is defined as the number of samples that are correctly classified by at least one of the base-layer classifiers in FSG. In the experiments, we observe that FSG performs better than the popular distance learning and ensemble learning algorithms when the shareability measure is large enough such that most of the samples are correctly classified by at least one of the base-layer classifiers. The relationship between the proposed and state-of-the-art diversity measures is experimentally analyzed. The tests performed on a variety of artificial and real-world benchmark datasets show that the classification performance of FSG increases compared to that of state-of-the art ensemble learning and distance learning methods as the number of classes increases.     

Improved DS acoustic–seismic modality fusion for ground-moving target classification in wireless sensor networks

An improved DS acoustic–seismic modality fusion framework based on cascaded fuzzy classifier (CFC) is proposed to implement ground-moving target classification tasks locally at sensor nodes in wireless sensor networks (WSN). The CFC consists of three and two component binary fuzzy classifiers (BFCs) in seismic and acoustic signal channel respectively. New basic belief assignment (bba) functions are defined for component binary fuzzy classifiers (BFCs) to give out evidences instead of hard decision labels for each unclassified pattern. Available evidences are then combined into a final node classification report using a modified DS method. M-fold cross-validation experiment results show that this implementation gives significantly better performance than the implementation with a majority-voting fusion and a DS fusion implementation with a linear bba function. Performances on different terrains are also given to validate its robustness.     

Building contextual classifiers by integrating fuzzy rule based classification technique and k-nn method for credit scoring

Credit-risk evaluation is a very challenging and important problem in the domain of financial analysis. Many classification methods have been proposed in the literature to tackle this problem. Statistical and neural network based approaches are among the most popular paradigms. However, most of these methods produce so-called “hard” classifiers, those generate decisions without any accompanying confidence measure. In contrast, “soft” classifiers, such as those designed using fuzzy set theoretic approach; produce a measure of support for the decision (and also alternative decisions) that provides the analyst with greater insight. In this paper, we propose a method of building credit-scoring models using fuzzy rule based classifiers. First, the rule base is learned from the training data using a SOM based method. Then the fuzzy k-nn rule is incorporated with it to design a contextual classifier that integrates the context information from the training set for more robust and qualitatively better classification. Further, a method of seamlessly integrating business constraints into the model is also demonstrated.     

On the impact of fusion strategies on classification errors for large ensembles of classifiers

The growing availability of sensor networks brings practical situations where a large number of classifiers can be used for building a classifier ensemble. In the most general case involving sensor networks, the classifiers are fed with multiple inputs collected at different locations. However, classifier fusion is often studied within an idealized formulation where each classifier is fed with the same point in the feature space, and estimate the posterior class probability given this input. We first expand this formulation to situations where classifiers are fed with multiple inputs, demonstrating the relevance of the formulation to situations involving sensor networks, and a large number of classifiers. Following that, we determine the rate of convergence of the classification error of a classifier ensemble for three fusion strategies (average, median and maximum) when the number of classifiers becomes large. As the size of the ensemble increases, the best strategy is defined as the one that results in fastest convergence of the classification error to zero. The best strategy is analytically shown to depend on the distribution of the individual classification errors: average is the best for normal distributions; maximum is the best for uniform distributions; and median is the best for Cauchy distributions. The general effect of heavy-tailedness is also analytically investigated for the average and median strategies. The median strategy is shown to be robust to heavy-tailedness, while performance of the average strategy is shown to degrade as heavy-tailedness becomes more pronounced. The combined effects of bimodality and heavy-tailedness are also investigated when the number of classifiers become large.     

A novel prosodic-information synthesizer based on recurrent fuzzy neural network for the Chinese TTS system.

In this paper, a new technique for the Chinese text-to-speech (TTS) system is proposed. Our major effort focuses on the prosodic information generation. New methodologies for constructing fuzzy rules in a prosodic model simulating human's pronouncing rules are developed. The proposed Recurrent Fuzzy Neural Network (RFNN) is a multilayer recurrent neural network (RNN) which integrates a Self-cOnstructing Neural Fuzzy Inference Network (SONFIN) into a recurrent connectionist structure. The RFNN can be functionally divided into two parts. The first part adopts the SONFIN as a prosodic model to explore the relationship between high-level linguistic features and prosodic information based on fuzzy inference rules. As compared to conventional neural networks, the SONFIN can always construct itself with an economic network size in high learning speed. The second part employs a five-layer network to generate all prosodic parameters by directly using the prosodic fuzzy rules inferred from the first part as well as other important features of syllables. The TTS system combined with the proposed method can behave not only sandhi rules but also the other prosodic phenomena existing in the traditional TTS systems. Moreover, the proposed scheme can even find out some new rules about prosodic phrase structure. The performance of the proposed RFNN-based prosodic model is verified by imbedding it into a Chinese TTS system with a Chinese monosyllable database based on the time-domain pitch synchronous overlap add (TD-PSOLA) method. Our experimental results show that the proposed RFNN can generate proper prosodic parameters including pitch means, pitch shapes, maximum energy levels, syllable duration, and pause duration. Some synthetic sounds are online available for demonstration.     

Fuzzy multi-category proximal support vector classification via generalized eigenvalues

Given a dataset, where each point is labeled with one of M labels, we propose a technique for multi-category proximal support vector classification via generalized eigenvalues (MGEPSVMs). Unlike Support Vector Machines that classify points by assigning them to one of M disjoint half-spaces, here points are classified by assigning them to the closest of M non-parallel planes that are close to their respective classes. When the data contains samples belonging to several classes, classes often overlap, and classifiers that solve for several non-parallel planes may often be able to better resolve test samples. In multicategory classification tasks, a training point may have similarities with prototypes of more than one class. This information can be used in a fuzzy setting. We propose a fuzzy multi-category classifier that utilizes information about the membership of training samples, to improve the generalization ability of the classifier. The desired classifier is obtained by using one-from-rest (OFR) separation for each class, i.e. 1: M -1 classification. Experimental results demonstrate the efficacy of the proposed classifier over MGEPSVMs.     

Dynamic classifier selection for One-vs-One strategy: Avoiding non-competent classifiers

The One-vs-One strategy is one of the most commonly used decomposition technique to overcome multi-class classification problems; this way, multi-class problems are divided into easier-to-solve binary classification problems considering pairs of classes from the original problem, which are then learned by independent base classifiers.The way of performing the division produces the so-called non-competence. This problem occurs whenever an instance is classified, since it is submitted to all the base classifiers although the outputs of some of them are not meaningful (they were not trained using the instances from the class of the instance to be classified). This issue may lead to erroneous classifications, because in spite of their incompetence, all classifiers' decisions are usually considered in the aggregation phase.In this paper, we propose a dynamic classifier selection strategy for One-vs-One scheme that tries to avoid the non-competent classifiers when their output is probably not of interest. We consider the neighborhood of each instance to decide whether a classifier may be competent or not. In order to verify the validity of the proposed method, we will carry out a thorough experimental study considering different base classifiers and comparing our proposal with the best performer state-of-the-art aggregation within each base classifier from the five Machine Learning paradigms selected. The findings drawn from the empirical analysis are supported by the appropriate statistical analysis.     

New classification techniques for electroencephalogram (EEG) signals and a real-time EEG control of a robot

This paper studies the state-of-the-art classification techniques for electroencephalogram (EEG) signals. Fuzzy Functions Support Vector Classifier, Improved Fuzzy Functions Support Vector Classifier and a novel technique that has been designed by utilizing Particle Swarm Optimization and Radial Basis Function Networks (PSO-RBFN) have been studied. The classification performances of the techniques are compared on standard EEG datasets that are publicly available and used by brain–computer interface (BCI) researchers. In addition to the standard EEG datasets, the proposed classifier is also tested on non-EEG datasets for thorough comparison. Within the scope of this study, several data clustering algorithms such as Fuzzy C-means, K-means and PSO clustering algorithms are studied and their clustering performances on the same datasets are compared. The results show that PSO-RBFN might reach the classification performance of state-of-the art classifiers and might be a better alternative technique in the classification of EEG signals for real-time application. This has been demonstrated by implementing the proposed classifier in a real-time BCI application for a mobile robot control.     

MPEG VBR video traffic modeling and classification using fuzzytechnique

We present an approach for MPEG variable bit rate (VBR) video modeling and classification using fuzzy techniques. We demonstrate that a type-2 fuzzy membership function, i.e., a Gaussian MF with uncertain variance, is most appropriate to model the log-value of I/P/B frame sizes in MPEG VBR video. The fuzzy c-means (FCM) method is used to obtain the mean and standard deviation (std) of T/P/B frame sizes when the frame category is unknown. We propose to use type-2 fuzzy logic classifiers (FLCs) to classify video traffic using compressed data. Five fuzzy classifiers and a Bayesian classifier are designed for video traffic classification, and the fuzzy classifiers are compared against the Bayesian classifier. Simulation results show that a type-2 fuzzy classifier in which the input is modeled as a type-2 fuzzy set and antecedent membership functions are modeled as type-2 fuzzy sets performs the best of the five classifiers when the testing video product is not included in the training products and a steepest descent algorithm is used to tune its parameters     

The effects of combining classifiers with the same training statistics using Bayesian decision rules

Multiple classifier systems (MCS) are attracting increasing interest in the field of pattern recognition and machine learning. Recently, MCS are also being introduced in the remote sensing field where the importance of classifier diversity for image classification problems has not been examined. In this article, Satellite Pour l'Observation de la Terre (SPOT) IV panchromatic and multispectral satellite images are classified into six land cover classes using five base classifiers: contextual classifier, k-nearest neighbour classifier, Mahalanobis classifier, maximum likelihood classifier and minimum distance classifier. The five base classifiers are trained with the same feature sets throughout the experiments and a posteriori probability, derived from the confusion matrix of these base classifiers, is applied to five Bayesian decision rules (product rule, sum rule, maximum rule, minimum rule and median rule) for constructing different combinations of classifier ensembles. The performance of these classifier ensembles is evaluated for overall accuracy and kappa statistics. Three statistical tests, the McNemar's test, the Cochran's Q test and the Looney's F-test, are used to examine the diversity of the classification results of the base classifiers compared to the results of the classifier ensembles. The experimental comparison reveals that (a) significant diversity amongst the base classifiers cannot enhance the performance of classifier ensembles; (b) accuracy improvement of classifier ensembles can only be found by using base classifiers with similar and low accuracy; (c) increasing the number of base classifiers cannot improve the overall accuracy of the MCS and (d) none of the Bayesian decision rules outperforms the others.     

Robust fuzzy relational classifier incorporating the soft class labels

Fuzzy relational classifier (FRC) is a recently proposed two-step nonlinear classifier. At first, the unsupervised fuzzy c-means (FCM) clustering is performed to explore the underlying groups of the given dataset. Then, a fuzzy relation matrix indicating the relationship between the formed groups and the given classes is constructed for subsequent classification. It has been shown that FRC has two advantages: interpretable classification results and avoidance of overtraining. However, FRC not only lacks the robustness which is very important for a classifier, but also fails on the dataset with non-spherical distributions. Moreover, the classification mechanism of FRC is sensitive to the improper class labels of the training samples, thus leading to considerable decline in classification performance. The purpose of this paper is to develop a Robust FRC (RFRC) algorithm aiming at overcoming or mitigating all of the above disadvantages of FRC and maintaining its original advantages. In the proposed RFRC algorithm, we employ our previously proposed robust kernelized FCM (KFCM) to replace FCM to enhance its robustness against outliers and its suitability for the non-spherical data structures. In addition, we incorporate the soft class labels into the classification mechanism to improve its performance, especially for the datasets containing the improper class labels. The experimental results on 2 artificial and 11 real-life benchmark datasets demonstrate that RFRC algorithm can consistently outperform FRC in classification performance.     

Trimmed bagging

Bagging has been found to be successful in increasing the predictive performance of unstable classifiers. Bagging draws bootstrap samples from the training sample, applies the classifier to each bootstrap sample, and then averages over all obtained classification rules. The idea of trimmed bagging is to exclude the bootstrapped classification rules that yield the highest error rates, as estimated by the out-of-bag error rate, and to aggregate over the remaining ones. In this note we explore the potential benefits of trimmed bagging. On the basis of numerical experiments, we conclude that trimmed bagging performs comparably to standard bagging when applied to unstable classifiers as decision trees, but yields better results when applied to more stable base classifiers, like support vector machines.     

An experimental study of one- and two-level classifier fusion for different sample sizes

Due to the wide variety of fusion techniques available for combining multiple classifiers into a more accurate classifier, a number of good studies have been devoted to determining in what situations some fusion methods should be preferred over other ones. However, the sample size behavior of the various fusion methods has hitherto received little attention in the literature of multiple classifier systems. The main contribution of this paper is thus to investigate the effect of training sample size on their relative performance and to gain more insight into the conditions for the superiority of some combination rules.A large experiment is conducted to study the performance of some fixed and trainable combination rules for executing one- and two-level classifier fusion for different training sample sizes. The experimental results yield the following conclusions: when implementing one-level fusion to combine homogeneous or heterogeneous base classifiers, fixed rules outperform trainable ones in nearly all cases, with only one exception of merging heterogeneous classifiers for large sample size. Moreover, the best classification for any considered sample size is generally achieved by a second level of combination (namely, utilizing one fusion rule to further combine a set of ensemble classifiers with each of them constructed by fusing base classifiers). Under these circumstances, it seems that adopting different types of fusion rules (fixed or trainable) as the combiners for two levels of fusion is appropriate.     

On-line multi-stage sorting algorithm for agriculture products

This paper presents an on-line multi-stage sorting algorithm capable of adapting to different populations. The sorting algorithm selects on-line the most appropriate classifier and feature subsets for the incoming population. The sorting algorithm includes two levels, a low level for population detection and a high level for classifier selection which incorporates feature selection. Population detection is achieved by an on-line unsupervised clustering algorithm that analyzes product variability. The classifier selection uses n fuzzy kNN classifiers, each trained with different feature combinations that function as input to a fuzzy rule-based decision system. Re-training of the n fuzzy kNN classifiers occurs when the rule based system cannot assign an existing classifier with high confidence level. Classification results for synthetic and real world databases are presented.