Performance evaluation of fuzzy classifier systems formultidimensional pattern classification problems

We examine the performance of a fuzzy genetics-based machine learning method for multidimensional pattern classification problems with continuous attributes. In our method, each fuzzy if-then rule is handled as an individual, and a fitness value is assigned to each rule. Thus, our method can be viewed as a classifier system. In this paper, we first describe fuzzy if-then rules and fuzzy reasoning for pattern classification problems. Then we explain a genetics-based machine learning method that automatically generates fuzzy if-then rules for pattern classification problems from numerical data. Because our method uses linguistic values with fixed membership functions as antecedent fuzzy sets, a linguistic interpretation of each fuzzy if-then rule is easily obtained. The fixed membership functions also lead to a simple implementation of our method as a computer program. The simplicity of implementation and the linguistic interpretation of the generated fuzzy if-then rules are the main characteristic features of our method. The performance of our method is evaluated by computer simulations on some well-known test problems. While our method involves no tuning mechanism of membership functions, it works very well in comparison with other classification methods such as nonfuzzy machine learning techniques and neural networks.     

Performance evaluation of fuzzy rule-based classification systems obtained by multi-objective genetic algorithms

In this paper, we examine the classification performance of fuzzy if-then rules selected by a GA-based multi-objective rule selection method. This rule selection method can be applied to high-dimensional pattern classification problems with many continuous attributes by restricting the number of antecedent conditions of each candidate fuzzy if-then rule. As candidate rules, we only use fuzzy if-then rules with a small number of antecedent conditions. Thus it is easy for human users to understand each rule selected by our method. Our rule selection method has two objectives: to minimize the number of selected fuzzy if-then rules and to maximize the number of correctly classified patterns. In our multi-objective fuzzy rule selection problem, there exist several solutions (i.e., several rule sets) called “non-dominated solutions” because two conflicting objectives are considered. In this paper, we examine the performance of our GA-based rule selection method by computer simulations on a real-world pattern classification problem with many continuous attributes. First we examine the classification performance of our method for training patterns by computer simulations. Next we examine the generalization ability for test patterns. We show that a fuzzy rule-based classification system with an appropriate number of rules has high generalization ability.     

An experimental evaluation of simplicity in rule learning

While recent research on rule learning has focused largely on finding highly accurate hypotheses, we evaluate the degree to which these hypotheses are also simple, that is small. To realize this, we compare well-known rule learners, such as CN2, RIPPER, PART, FOIL and C5.0 rules, with the benchmark system SL² that explicitly aims at computing small rule sets with few literals. The results show that it is possible to obtain a similar level of accuracy as state-of-the-art rule learners using much smaller rule sets.     

正负模糊规则系统、极限学习机与图像分类

传统的图像分类一般只利用了图像的正规则,忽略了负规则在图像分类中的作用。Nguyen将负规则引入图像分类,提出将正负模糊规则相结合形成正负模糊规则系统,并将其用于遥感图像和自然图像的分类。实验证明,其在图像分类过程中取得了很好的效果。他们提出的前馈神经网络模型在调整权值时利用了梯度下降法,由于步长选择不合理或陷入局部最优从而使训练速度受到了限制。极限学习机（ELM）是一种单隐层前馈神经网络（SLFN）学习算法,具有学习速度快,泛化性能好的优点。本文证明了极限学习机与正负模糊规则系统的实质是等价的,遂将其用于图像分类。实验结果说明了极限学习机能很好的利用正负模糊规则相结合的方法对图像进行分类,实验结果较为理想。     

基于克隆选择的模糊分类规则提取算法

分类是许多研究领域的关键问题，模糊规则的提取质量对分类器的性能又有着极大影响．所提取的规则不仅在分类能力上要达到最优，同时在规则数量上也不能太多，否则会影响规则搜索和匹配的速度．结合人工免疫的克隆选择原理，采用克隆选择算法，提取通过多精度模糊分割产生的大量模糊if—then规则中的少数精华规则，从而建立了模糊分类所需要的有效规则集合，同时还对优化目标函数进行了改进．经仿真实验证明，该方法所提取的模糊规则具有分类准确率高，规则数目较少等特点。     

基于RIPPER的旋转机械故障诊断知识获取

数据挖掘技术能够从大量、不完全、有噪声、模糊、随机的实际应用数据中,提取隐含在其中的、人们事先不知道的本质的规律。为了有效地发现旋转机械故障诊断过程中的故障征兆知识,引入数据挖掘技术和方法。针对旋转机械,构建了基于重复增量修枝算法RIPPER（Repeated Incremental Pruning to Produce Error Reduction）的故障诊断知识获取系统。通过收集故障现象并整理成由故障征兆、故障类型等组成的故障信息样本,应用RIPPER算法对故障进行分析得到故障诊断规则集文件,实现故障诊断系统知识的获取和自动更新,并能对旋转机械的常见故障进行诊断,验证了算法的合理性。     

Effect of rule weights in fuzzy rule-based classification systems

This paper examines the effect of rule weights in fuzzy rule-based classification systems. Each fuzzy IF-THEN rule in our classification system has antecedent linguistic values and a single consequent class. We use a fuzzy reasoning method based on a single winner rule in the classification phase. The winner rule for a new pattern is the fuzzy IF-THEN rule that has the maximum compatibility grade with the new pattern. When we use fuzzy IF-THEN rules with certainty grades, the winner is determined as the rule with the maximum product of the compatibility grade and the certainty grade. In this paper, the effect of rule weights is illustrated by drawing classification boundaries using fuzzy IF-THEN rules with/without certainty grades. It is also shown that certainty grades play an important role when a fuzzy rule-based classification system is a mixture of general rules and specific rules. Through computer simulations, we show that comprehensible fuzzy rule-based systems with high classification performance can be designed without modifying the membership functions of antecedent linguistic values when we use fuzzy IF-THEN rules with certainty grades     

Fuzzy feature selection

In fuzzy classifier systems the classification is obtained by a number of fuzzy If–Then rules including linguistic terms such as Low and High that fuzzify each feature. This paper presents a method by which a reduced linguistic (fuzzy) set of a labeled multi-dimensional data set can be identified automatically. After the projection of the original data set onto a fuzzy space, the optimal subset of fuzzy features is determined using conventional search techniques. The applicability of this method has been demonstrated by reducing the number of features used for the classification of four real-world data sets. This method can also be used to generate an initial rule set for a fuzzy neural network.     

Fuzzy rule weight modification with particle swarm optimisation

The most challenging problem in developing fuzzy rule-based classification systems is the construction of a fuzzy rule base for the target problem. In many practical applications, fuzzy sets that are of particular linguistic meanings, are often predefined by domain experts and required to be maintained in order to ensure interpretability of any subsequent inference results. However, learning fuzzy rules using fixed fuzzy quantity space without any qualification will restrict the accuracy of the resulting rules. Fortunately, adjusting the weights of fuzzy rules can help improve classification accuracy without degrading the interpretability. There have been different proposals for fuzzy rule weight tuning through the use of various heuristics with limited success. This paper proposes an alternative approach using Particle Swarm Optimisation in the search of a set of optimal rule weights, entailing high classification accuracy. Systematic experimental studies are carried out using common benchmark data sets, in comparison to popular rule based learning classifiers. The results demonstrate that the proposed approach can boost classification performance, especially when the size of the initially built rule base is relatively small, and is competitive to popular rule-based learning classifiers.     

A NEW METHOD TO GENERATE FUZZY RULES FROM TRAINING INSTANCES FOR HANDLING CLASSIFICATION PROBLEMS

A major task in developing a fuzzy classification system is to generate a set of fuzzy rules from training instances to deal with a specific classification problem. In recent years, many methods have been developed to generate fuzzy rules from training instances. We present a new method to generate fuzzy rules from training instances to deal with the Iris data classification problem. The proposed method can discard some useless input attributes to improve the average classification accuracy rate. It can obtain a higher average classification accuracy rate and it generates fewer fuzzy rules and fewer input fuzzy sets in the generated fuzzy rules than the existing methods.     

Designing of classifiers based on immune principles and fuzzy rules

This paper proposed an algorithm to design a fuzzy classification system based on immune principles. The proposed algorithm evolves a population of antibodies based on the clonal selection and hypermutation principles. The membership function parameters and the fuzzy rule set including the number of rules inside it are evolved at the same time. Each antibody (candidate solution) corresponds to a fuzzy classification rule set. We compared our algorithm with other classification schemes on some benchmark datasets. The results demonstrated the effectiveness of the proposed immune algorithm.     

基于免疫原理和模糊规则的分类器设计

提出一种基于免疫原理的模糊分类系统的设计方法．该算法基于生物免疫系统中的克隆选择和超变异原理，通过抗体种群的演化来优化模糊分类规则集合，可以同时确定隶属度函数形状、规则集合以及规则的数目．针对典型数据集的仿真实验表明了本文方法的有效性．     

A hybrid coevolutionary algorithm for designing fuzzy classifiers

Rule learning is one of the most common tasks in knowledge discovery. In this paper, we investigate the induction of fuzzy classification rules for data mining purposes, and propose a hybrid genetic algorithm for learning approximate fuzzy rules. A novel niching method is employed to promote coevolution within the population, which enables the algorithm to discover multiple rules by means of a coevolutionary scheme in a single run. In order to improve the quality of the learned rules, a local search method was devised to perform fine-tuning on the offspring generated by genetic operators in each generation. After the GA terminates, a fuzzy classifier is built by extracting a rule set from the final population. The proposed algorithm was tested on datasets from the UCI repository, and the experimental results verify its validity in learning rule sets and comparative advantage over conventional methods.     

An ensemble method for fuzzy rule-based classification systems

Fuzzy rule-based classification systems are very useful tools in the field of machine learning as they are able to build linguistic comprehensible models. However, these systems suffer from exponential rule explosion when the number of variables increases, degrading, therefore, the accuracy of these systems as well as their interpretability. In this article, we propose to improve the comprehensibility through a supervised learning method by automatic generation of fuzzy classification rules, designated SIFCO–PAF. Our method reduces the complexity by decreasing the number of rules and of antecedent conditions, making it thus adapted to the representation and the prediction of rather high-dimensional pattern classification problems. We perform, firstly, an ensemble methodology by combining a set of simple classification models. Subsequently, each model uses a subset of the initial attributes: In this case, we propose to regroup the attributes using linear correlation search among the training set elements. Secondly, we implement an optimal fuzzy partition thanks to supervised discretization followed by an automatic membership functions construction. The SIFCO–PAF method, analyzed experimentally on various data sets, guarantees an important reduction in the number of rules and of antecedents without deteriorating the classification rates, on the contrary accuracy is even improved.     

A multi-objective genetic optimization of interpretability-oriented fuzzy rule-based classifiers

The paper presents a multi-objective genetic approach to design interpretability-oriented fuzzy rule-based classifiers from data. The proposed approach allows us to obtain systems with various levels of compromise between their accuracy and interpretability. During the learning process, parameters of the membership functions, as well as the structure of the classifier's fuzzy rule base (i.e., the number of rules, the number of rule antecedents, etc.) evolve simultaneously using a Pittsburgh-type genetic approach. Since there is no particular coding of fuzzy rule structures in a chromosome (it reduces computational complexity of the algorithm), original crossover and mutation operators, as well as chromosome-repairing technique to directly transform the rules are also proposed. To evaluate both the accuracy and interpretability of the system, two measures are used. The first one – an accuracy measure – is based on the root mean square error of the system's response. The second one – an interpretability measure – is based on the arithmetic mean of three components: (a) the average length of rules (the average number of antecedents used in the rules), (b) the number of active fuzzy sets and (c) the number of active inputs of the system (an active fuzzy set or input means a set or input used by at least one fuzzy rule). Both measures are used as objectives in multi-objective (2-objective in our case) genetic optimization approaches such as well-known SPEA2 and NSGA-II algorithms. Moreover, for the purpose of comparison with several alternative approaches, the experiments are carried out both considering the so-called strong fuzzy partitions (SFPs) of attribute domains and without them. SFPs provide more semantically meaningful solutions, usually at the expense of their accuracy. The operation of the proposed technique in various classification problems is tested with the use of 20 benchmark data sets and compared to 11 alternative classification techniques. The experiments show that the proposed approach generates classifiers of significantly improved interpretability, while still characterized by competitive accuracy.     

基于训练空间重构的多模块TSK模糊系统

利用重构训练样本空间的手段,提出一种多训练模块Takagi-Sugeno-Kang （TSK）模糊分类器H-TSK-FS.它具有良好的分类性能和较高的可解释性,可以解决现有层次模糊分类器中间层输出和模糊规则难以解释的难题.为了实现良好的分类性能,H-TSK-FS由多个优化零阶TSK模糊分类器组成.这些零阶TSK模糊分类器内部采用一种巧妙的训练方式.原始训练样本、上一层训练样本中的部分样本点以及所有已训练层中最逼近真实值的部分决策信息均被投影到当前层训练模块中,并构成其输入空间.通过这种训练方式,前层的训练结果对后层的训练起到引导和控制作用.这种随机选取样本点、在一定范围内随机选取训练特征的手段可以打开原始输入空间的流形结构,保证较好或相当的分类性能.另外,该研究主要针对少量样本点且训练特征数不是很大的数据集.在设计每个训练模块时采用极限学习机获取模糊规则后件参数.对于每个中间训练层,采用短规则表达知识.每条模糊规则则通过约束方式确定不固定的输入特征以及高斯隶属函数,目的是保证所选输入特征具有高可解释性.真实数据集和应用案例实验结果表明,H-TSK-FS具有良好的分类性能和高可解释性.     

Efficient multimatch packet classification and lookup with TCAM

Today's packet classification systems are designed to provide the highest-priority matching result, such as the longest prefix match, even if a packet matches multiple classification rules. However, new network applications demanding multimatch classification - that is, requiring all matching results instead of only the highest-priority match - are emerging. Ternary content-addressable memory is becoming a common extension to network processors, and its capability and speed make it attractive for high-speed networks. The proposed TCAM-based scheme produces multimatch classification results with about 10 times fewer memory lookups than a pure software approach. In addition, their scheme for removing negation in rule sets saves up to 95 percent of the TCAM space used by a straightforward implementation.     

Application of fuzzy Naive Bayes and a real-valued genetic algorithm in identification of fuzzy model

We present a method to identify a fuzzy model from data by using the fuzzy Naive Bayes and a real-valued genetic algorithm. The identification of a fuzzy model is comprised of the extraction of “if–then” rules that is followed by the estimation of their parameters. The involved parameters include those which determine the membership function of fuzzy sets and the certainty factors of fuzzy if–then rules. In our method, as long as the fuzzy partition in the input–output space is given, the certainty factor of each rule is computed with the fuzzy conditional probability of the consequent conditioned on the antecedent by using the fuzzy Naive Bayes, which is a generalization of Naive Bayes. The fuzzy model involves the rules characterized by the highest values of certainty factors. The certainty factor of each rule is the fuzzy conditional probability, and it reflects the inner relationship between the antecedent and the consequent. In order to improve the accuracy of the fuzzy model, the real-valued genetic algorithm is incorporated into our identification process. This process concerns the optimization of the membership functions occurring in the rules. We just involve the parameters of membership function of the fuzzy sets into the real-valued genetic algorithm, since the certainty factor of each rule can be computed automatically. The performance of the model is shown for the backing-truck problem and the prediction of Mackey–Glass time series.     

Fuzzy rule interpolation based on the ratio of fuzziness of interval type-2 fuzzy sets

In recent years, some fuzzy rule interpolation methods have been presented for sparse fuzzy rule-based systems based on interval type-2 fuzzy sets. However, the existing methods have the drawbacks that they cannot guarantee the convexity of the fuzzy interpolated result and may generate the same fuzzy interpolated results with respect to different observations. Moreover, they also cannot deal with fuzzy rule interpolation with bell-shaped interval type-2 fuzzy sets. In this paper, we present a new method for fuzzy rule interpolation for sparse fuzzy rule-based systems based on the ratio of fuzziness of interval type-2 fuzzy sets. The proposed method can overcome the drawbacks of the existing methods. First, it calculates the weights of the closest fuzzy rules with respect to the observation to obtain an intermediate consequence fuzzy set. Then, it uses the ratio of fuzziness of interval type-2 fuzzy sets to infer the fuzzy interpolated result based on the intermediate consequence fuzzy set. We also use some examples to compare the fuzzy interpolated results of the proposed method with the results by the existing methods. The experimental results show that the proposed fuzzy rule interpolation method gets more reasonable results than the existing methods.