Adaptive fuzzy rule-based classification systems

This paper proposes an adaptive method to construct a fuzzy rule-based classification system with high performance for pattern classification problems. The proposed method consists of two procedures: an error correction-based learning procedure, and an additional learning procedure. The error correction-based learning procedure adjusts the grade of certainty of each fuzzy rule by its classification performance. That is, when a pattern is misclassified by a particular fuzzy rule, the grade of certainty of that rule is decreased. On the contrary, when a pattern is correctly classified, the grade of certainty is increased. Because the error correction-based learning procedure is not meaningful after all the given patterns are correctly classified, we cannot adjust a classification boundary in such a case. To acquire a more intuitively acceptable boundary, we propose an additional learning procedure. We also propose a method for selecting significant fuzzy rules by pruning unnecessary fuzzy rules, which consists of the error correction-based learning procedure and the concept of forgetting. We can construct a compact fuzzy rule-based classification system with high performance     

Missing data imputation for fuzzy rule-based classification systems

Fuzzy rule-based classification systems (FRBCSs) are known due to their ability to treat with low quality data and obtain good results in these scenarios. However, their application in problems with missing data are uncommon while in real-life data, information is frequently incomplete in data mining, caused by the presence of missing values in attributes. Several schemes have been studied to overcome the drawbacks produced by missing values in data mining tasks; one of the most well known is based on preprocessing, formerly known as imputation. In this work, we focus on FRBCSs considering 14 different approaches to missing attribute values treatment that are presented and analyzed. The analysis involves three different methods, in which we distinguish between Mamdani and TSK models. From the obtained results, the convenience of using imputation methods for FRBCSs with missing values is stated. The analysis suggests that each type behaves differently while the use of determined missing values imputation methods could improve the accuracy obtained for these methods. Thus, the use of particular imputation methods conditioned to the type of FRBCSs is required.     

Support vector learning for fuzzy rule-based classification systems

To design a fuzzy rule-based classification system (fuzzy classifier) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, the support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high- (or even infinite) dimensional feature space. This paper investigates the connection between fuzzy classifiers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. We first show that a fuzzy classifier implicitly defines a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet a certain spectral requirement. The corresponding fuzzy classifier is named positive definite fuzzy classifier (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the "curse of dimensionality." Finally, PDFCs with different reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided.     

An ensemble method for fuzzy rule-based classification systems

Fuzzy rule-based classification systems are very useful tools in the field of machine learning as they are able to build linguistic comprehensible models. However, these systems suffer from exponential rule explosion when the number of variables increases, degrading, therefore, the accuracy of these systems as well as their interpretability. In this article, we propose to improve the comprehensibility through a supervised learning method by automatic generation of fuzzy classification rules, designated SIFCO–PAF. Our method reduces the complexity by decreasing the number of rules and of antecedent conditions, making it thus adapted to the representation and the prediction of rather high-dimensional pattern classification problems. We perform, firstly, an ensemble methodology by combining a set of simple classification models. Subsequently, each model uses a subset of the initial attributes: In this case, we propose to regroup the attributes using linear correlation search among the training set elements. Secondly, we implement an optimal fuzzy partition thanks to supervised discretization followed by an automatic membership functions construction. The SIFCO–PAF method, analyzed experimentally on various data sets, guarantees an important reduction in the number of rules and of antecedents without deteriorating the classification rates, on the contrary accuracy is even improved.     

Supervised methods for regrouping attributes in fuzzy rule-based classification systems

This paper focuses on ensemble methods for Fuzzy Rule-Based Classification Systems (FRBCS) where the decisions of different classifiers are combined in order to form the final classification model. The proposed methods reduce the FRBCS complexity and the generated rules number. We are interested in particular in ensemble methods which cluster the attributes into subgroups of attributes and treat each subgroup separately. Our work is an extension of a previous ensemble method called SIFRA. This method uses frequent itemsets mining concept in order to deduce the groups of related attributes by analyzing their simultaneous appearances in the databases. The drawback of this method is that it forms the groups of attributes by searching for dependencies between the attributes independently from the class information. Besides, since we deal with supervised learning problems, it would be very interesting to consider the class attribute when forming the attributes subgroups. In this paper, we proposed two new supervised attributes regrouping methods which take into account not only the dependencies between the attributes but also the information about the class labels. The results obtained with various benchmark datasets show a good accuracy of the built classification model.     

Performance evaluation of fuzzy rule-based classification systems obtained by multi-objective genetic algorithms

In this paper, we examine the classification performance of fuzzy if-then rules selected by a GA-based multi-objective rule selection method. This rule selection method can be applied to high-dimensional pattern classification problems with many continuous attributes by restricting the number of antecedent conditions of each candidate fuzzy if-then rule. As candidate rules, we only use fuzzy if-then rules with a small number of antecedent conditions. Thus it is easy for human users to understand each rule selected by our method. Our rule selection method has two objectives: to minimize the number of selected fuzzy if-then rules and to maximize the number of correctly classified patterns. In our multi-objective fuzzy rule selection problem, there exist several solutions (i.e., several rule sets) called “non-dominated solutions” because two conflicting objectives are considered. In this paper, we examine the performance of our GA-based rule selection method by computer simulations on a real-world pattern classification problem with many continuous attributes. First we examine the classification performance of our method for training patterns by computer simulations. Next we examine the generalization ability for test patterns. We show that a fuzzy rule-based classification system with an appropriate number of rules has high generalization ability.     

hGA: Hybrid genetic algorithm in fuzzy rule-based classification systems for high-dimensional problems

The aim of this work is to propose a hybrid heuristic approach (called hGA) based on genetic algorithm (GA) and integer-programming formulation (IPF) to solve high dimensional classification problems in linguistic fuzzy rule-based classification systems. In this algorithm, each chromosome represents a rule for specified class, GA is used for producing several rules for each class, and finally IPF is used for selection of rules from a pool of rules, which are obtained by GA. The proposed algorithm is experimentally evaluated by the use of non-parametric statistical tests on seventeen classification benchmark data sets. Results of the comparative study show that hGA is able to discover accurate and concise classification rules.     

Matrix formulation of fuzzy rule-based systems

In this paper, a matrix formulation of fuzzy rule based systems is introduced. A gradient descent training algorithm for the determination of the unknown parameters can also be expressed in a matrix form for various adaptive fuzzy networks. When converting a rule-based system to the proposed matrix formulation, only three sets of linear/nonlinear equations are required instead of set of rules and an inference mechanism. There are a number of advantages which the matrix formulation has compared with the linguistic approach. Firstly, it obviates the differences among the various architectures; and secondly, it is much easier to organize data in the implementation or simulation of the fuzzy system. The formulation will be illustrated by a number of examples.     

Improving the performance of fuzzy rule-based classification systems with interval-valued fuzzy sets and genetic amplitude tuning

Among the computational intelligence techniques employed to solve classification problems, Fuzzy Rule-Based Classification Systems (FRBCSs) are a popular tool because of their interpretable models based on linguistic variables, which are easier to understand for the experts or end-users.The aim of this paper is to enhance the performance of FRBCSs by extending the Knowledge Base with the application of the concept of Interval-Valued Fuzzy Sets (IVFSs). We consider a post-processing genetic tuning step that adjusts the amplitude of the upper bound of the IVFS to contextualize the fuzzy partitions and to obtain a most accurate solution to the problem.We analyze the goodness of this approach using two basic and well-known fuzzy rule learning algorithms, the Chi et al.’s method and the fuzzy hybrid genetics-based machine learning algorithm. We show the improvement achieved by this model through an extensive empirical study with a large collection of data-sets.     

Structure identification in complete rule-based fuzzy systems

The identification of a model is one of the key issues in the field of fuzzy system modeling and function approximation theory. There are numerous approaches to the issue of parameter optimization within a fixed fuzzy system structure but no reliable method to obtain the optimal topology of the fuzzy system from a set of input-output data. This paper presents a reliable method to obtain the structure of a complete rule-based fuzzy system for a specific approximation accuracy of the training data, i.e., it can decide which input variables must be taken into account in the fuzzy system and how many membership functions (MFs) are needed in every selected input variable in order to reach the approximation target with the minimum number of parameters     

Weighted fuzzy interpolative reasoning for sparse fuzzy rule-based systems

In this paper, we present a weighted fuzzy interpolative reasoning method for sparse fuzzy rule-based systems, where the antecedent variables appearing in the fuzzy rules have different weights. We also present a weights-learning algorithm to automatically learn the optimal weights of the antecedent variables of the fuzzy rules for the proposed weighted fuzzy interpolative reasoning method. We also apply the proposed weighted fuzzy interpolative reasoning method and the proposed weights-learning algorithm to handle the truck backer-upper control problem. The experimental results show that the proposed fuzzy interpolative reasoning method using the optimally learned weights by the proposed weights-learning algorithm gets better truck backer-upper control results than the ones by the traditional fuzzy inference system and the existing fuzzy interpolative reasoning methods. The proposed method provides us with a useful way for fuzzy rules interpolation in sparse fuzzy rule-based systems.     

Adaptation and application of multi-objective evolutionary algorithms for rule reduction and parameter tuning of fuzzy rule-based systems

Recently, multi-objective evolutionary algorithms have been applied to improve the difficult tradeoff between interpretability and accuracy of fuzzy rule-based systems. It is known that both requirements are usually contradictory, however, these kinds of algorithms can obtain a set of solutions with different trade-offs. This contribution analyzes different application alternatives in order to attain the desired accuracy/interpr-etability balance by maintaining the improved accuracy that a tuning of membership functions could give but trying to obtain more compact models. In this way, we propose the use of multi-objective evolutionary algorithms as a tool to get almost one improved solution with respect to a classic single objective approach (a solution that could dominate the one obtained by such algorithm in terms of the system error and number of rules). To do that, this work presents and analyzes the application of six different multi-objective evolutionary algorithms to obtain simpler and still accurate linguistic fuzzy models by performing rule selection and a tuning of the membership functions. The results on two different scenarios show that the use of expert knowledge in the algorithm design process significantly improves the search ability of these algorithms and that they are able to improve both objectives together, obtaining more accurate and at the same time simpler models with respect to the single objective based approach.

---

Model-free optimization of fuzzy rule-based systems using evolutionstrategies

In this paper the applicability of evolution strategies, a special kind of evolutionary algorithms, to the problem of parameter optimization in the development of fuzzy rule-based systems is demonstrated. For this aim we introduce a shell which supports the design of any kind of rule based systems employing fuzzy logic for the formalization of imprecise reasoning processes and which optimizes all numerical parameters. This method works model-free, we do not need to know implicit features of the optimizing system.     

Learning concurrently data and rule bases of Mamdani fuzzy rule-based systems by exploiting a novel interpretability index

Interpretability of Mamdani fuzzy rule-based systems (MFRBSs) has been widely discussed in the last years, especially in the framework of multi-objective evolutionary fuzzy systems (MOEFSs). Here, multi-objective evolutionary algorithms (MOEAs) are applied to generate a set of MFRBSs with different trade-offs between interpretability and accuracy. In MOEFSs interpretability has often been measured in terms of complexity of the rule base and only recently partition integrity has also been considered. In this paper, we introduce a novel index for evaluating the interpretability of MFRBSs, which takes both the rule base complexity and the data base integrity into account. We discuss the use of this index in MOEFSs, which generate MFRBSs by concurrently learning the rule base, the linguistic partition granularities and the membership function parameters during the evolutionary process. The proposed approach has been experimented on six real world regression problems and the results have been compared with those obtained by applying the same MOEA, with only accuracy and complexity of the rule base as objectives. We show that our approach achieves the best trade-offs between interpretability and accuracy.     

GP-COACH: Genetic Programming-based learning of COmpact and ACcurate fuzzy rule-based classification systems for High-dimensional problems

In this paper we propose GP-COACH, a Genetic Programming-based method for the learning of COmpact and ACcurate fuzzy rule-based classification systems for High-dimensional problems. GP-COACH learns disjunctive normal form rules (generated by means of a context-free grammar) coded as one rule per tree. The population constitutes the rule base, so it is a genetic cooperative-competitive learning approach. GP-COACH uses a token competition mechanism to maintain the diversity of the population and this obliges the rules to compete and cooperate among themselves and allows the obtaining of a compact set of fuzzy rules. The results obtained have been validated by the use of non-parametric statistical tests, showing a good performance in terms of accuracy and interpretability.     

Improving the efficiency of rule-based expert systems by rule activation

Abstract

In this paper we test a hypothesis that has shown promise in enhancing the efficiency ( run-time) of rule-based systems. The results of our experiments suggest that the use of rule activation plays an active part in improving the performance of rule bases containing conflict sets.     

A neuro-fuzzy scheme for simultaneous feature selection and fuzzy rule-based classification

Most methods of classification either ignore feature analysis or do it in a separate phase, offline prior to the main classification task. This paper proposes a neuro-fuzzy scheme for designing a classifier along with feature selection. It is a four-layered feed-forward network for realizing a fuzzy rule-based classifier. The network is trained by error backpropagation in three phases. In the first phase, the network learns the important features and the classification rules. In the subsequent phases, the network is pruned to an "optimal" architecture that represents an "optimal" set of rules. Pruning is found to drastically reduce the size of the network without degrading the performance. The pruned network is further tuned to improve performance. The rules learned by the network can be easily read from the network. The system is tested on both synthetic and real data sets and found to perform quite well.     

Knowledge-based fuzzy MLP for classification and rule generation

A new scheme of knowledge-based classification and rule generation using a fuzzy multilayer perceptron (MLP) is proposed. Knowledge collected from a data set is initially encoded among the connection weights in terms of class a priori probabilities. This encoding also includes incorporation of hidden nodes corresponding to both the pattern classes and their complementary regions. The network architecture, in terms of both links and nodes, is then refined during training. Node growing and link pruning are also resorted to. Rules are generated from the trained network using the input, output, and connection weights in order to justify any decision(s) reached. Negative rules corresponding to a pattern not belonging to a class can also be obtained. These are useful for inferencing in ambiguous cases. Results on real life and synthetic data demonstrate that the speed of learning and classification performance of the proposed scheme are better than that obtained with the fuzzy and conventional versions of the MLP (involving no initial knowledge encoding). Both convex and concave decision regions are considered in the process.     

A fuzzy reasoning approach for rule-based systems based on fuzzylogics

This paper presents a weighted fuzzy reasoning algorithm for rule-based systems based on weighted fuzzy logics. The proposed algorithm allows the truth values of the conditions appearing in the antecedent portions of the rules, the certainty factors of the rules, and the weights of the conditions appearing in the antecedent portions of the rules to be represented by trapezoidal fuzzy numbers. Given the fuzzy truth values of some conditions, the algorithm can perform weighted fuzzy reasoning to evaluate the fuzzy truth values of other conditions automatically.