A hybrid particle swarm optimization based fuzzy expert system for the diagnosis of coronary artery disease

This paper presents a particle swarm optimization (PSO)-based fuzzy expert system for the diagnosis of coronary artery disease (CAD). The designed system is based on the Cleveland and Hungarian Heart Disease datasets. Since the datasets consist of many input attributes, decision tree (DT) was used to unravel the attributes that contribute towards the diagnosis. The output of the DT was converted into crisp if–then rules and then transformed into fuzzy rule base. PSO was employed to tune the fuzzy membership functions (MFs). Having applied the optimized MFs, the generated fuzzy expert system has yielded 93.27% classification accuracy. The major advantage of this approach is the ability to interpret the decisions made from the created fuzzy expert system, when compared with other approaches.     

A new methodology of extraction, optimization and application ofcrisp and fuzzy logical rules

A new methodology of extraction, optimization, and application of sets of logical rules is described. Neural networks are used for initial rule extraction, local or global minimization procedures for optimization, and Gaussian uncertainties of measurements are assumed during application of logical rules. Algorithms for extraction of logical rules from data with real-valued features require determination of linguistic variables or membership functions. Contest-dependent membership functions for crisp and fuzzy linguistic variables are introduced and methods of their determination described. Several neural and machine learning methods of logical rule extraction generating initial rules are described, based on constrained multilayer perceptron, networks with localized transfer functions or on separability criteria for determination of linguistic variables. A tradeoff between accurary/simplicity is explored at the rule extraction stage and between rejection/error level at the optimization stage. Gaussian uncertainties of measurements are assumed during application of crisp logical rules, leading to "soft trapezoidal" membership functions and allowing to optimize the linguistic variables using gradient procedures. Numerous applications of this methodology to benchmark and real-life problems are reported and very simple crisp logical rules for many datasets provided.     

A novel and fast MIMO fuzzy inference system based on a class of fuzzy clustering algorithms with interpretability and complexity analysis

A data driven Fuzzy Inference System (FIS) employs Membership Functions (MFs) with adjustable parameters in its IF part to fuzzify the input data. The input space is partitioned simply by dividing universe of discourse of each input variable into some fuzzy subspaces. The MFs are then defined on the fuzzy subspaces of the input variables. Parameters of the MFs are tuned for maximum accuracy of the system (which demands high runtime) without considering the data structure which impairs interpretability of the FIS and degenerates the system into a black-box tool. Such a FIS does not represent actual structure of the data and its MFs are not necessarily in accord with the data distribution in the input space. In addition, the FIS suffers from exponential complexity of order O(T^r) where T is number of linguistic terms (number of subspaces on the universe of discourse of input variables) and r is number of input variables. This article presents a novel Multiple-Input and Multiple-Output Clustering based Fuzzy Inference System (MIMO CFIS) which is made directly from a class of fuzzy clustering algorithms to overcome these shortcomings. CFIS identifies dense regions of the input data using fuzzy clustering and then places a cluster on each of these regions. These fuzzy clusters represent actual structure of the data and serve as fuzzy rules in the rule base of CFIS and provide MFs that exactly fit the dense regions of the data that makes the system more interpretable and avoids redundant rules. These MFs are normal, convex, and continuous and have no parameter to be tuned (which makes CFIS much faster than other FISs) and fuzzify the input data according to their membership in the clusters. THEN part of CFIS is generalized form of THEN part of Takagi-Sugeno (TS) fuzzy system which accommodates any function of input variables. Despite less number of adjustable parameters, testing error of CFIS is less than that of TS system and its modified versions. Moreover, number of fuzzy rules in CFIS rule base is the same as the number of linguistic terms (or fuzzy clusters) and consequently its complexity is of orderO(T). Also, CFIS is a MIMO system and avoids inconsistent (contradictory) rules by generating well-separated fuzzy clusters whereas TS system is MISO and never guarantees generation of consistent rules. In addition, CFIS satisfies most of the interpretability criteria of FISs.     

Learning Fuzzy Rules from Data

We present an algorithm, FUZZEX, for learning fuzzy rules from a corpus of data mapping input antecedents to output consequents. The input and output spaces are first divided into a grid of cells and primitive if % then rules formulated from each occupied input cell in the role of an antecedent The distribution of output cells into which data in the input cell maps, plays the role of the consequent interpreted as a fuzzy set. Those input cells associated with sufficiently similar fuzzy output sets are then combined to form a composite rule. A concise set of rules in Disjunctive Normal Form (DNF) is formed by combining adjacent input cells belonging to the same rule, thereby simplifying the logical expression of the antecedents. Optionally, more succinctness of expression may be obtained by recruiting into a rule, adjacent input cells with (little or) no data, but which happen to simplify rule expression. Preliminary testing on testbed datasets is presented. FUZZEX can be applied effectively to problems of large dimensionality.     

Fuzzy DIFACONN-miner: A novel approach for fuzzy rule extraction from neural networks

Artificial neural networks (ANNs) are mathematical models inspired from the biological nervous system. They have the ability of predicting, learning from experiences and generalizing from previous examples. An important drawback of ANNs is their very limited explanation capability, mainly due to the fact that knowledge embedded within ANNs is distributed over the activations and the connection weights. Therefore, one of the main challenges in the recent decades is to extract classification rules from ANNs. This paper presents a novel approach to extract fuzzy classification rules (FCR) from ANNs because of the fact that fuzzy rules are more interpretable and cope better with pervasive uncertainty and vagueness with respect to crisp rules. A soft computing based algorithm is developed to generate fuzzy rules based on a data mining tool (DIFACONN-miner), which was recently developed by the authors. Fuzzy DIFACONN-miner algorithm can extract fuzzy classification rules from datasets containing both categorical and continuous attributes. Experimental research on the benchmark datasets and comparisons with other fuzzy rule based classification (FRBC) algorithms has shown that the proposed algorithm yields high classification accuracies and comprehensible rule sets.     

Incorporation of Fuzziness in ID3 and Generation of Network Architerture

This article first describes a fuzzy version of ID3, called fuzzy ID3 by incorporating fuzziness at input, output and node levels. A fuzziness measure is computed at each node, in terms of class membership, to take care of the uncertainty arising from overlapping regions. The measure is such that in the crisp (non-overlapping) case, the algorithm boils down to the classical ID3. A confidence factor is estimated at the nodes for both making a decision and determining the rule base for network mapping. In the second part, we deal with a scheme of designing a fuzzy knowledge-based network by encoding an MLP with the rules generated using fuzzy ID3, whereby the network topology is automatically determined. The frequency of samples (representative of a rule) and the confidence factors of unresolved/ambiguous nodes are taken into consideration during mapping. The effectiveness of the system, in terms of recognition scores and speed of convergence, is demonstrated on two real life data sets.       

Falcon: neural fuzzy control and decision systems using FKP and PFKP clustering algorithms.

Neural fuzzy networks proposed in the literature can be broadly classified into two groups. The first group is essentially fuzzy systems with self-tuning capabilities and requires an initial rule base to be specified prior to training. The second group of neural fuzzy networks, on the other hand, is able to automatically formulate the fuzzy rules from the numerical training data. Examples are the Falcon-ART, and the POPFNN family of networks. A cluster analysis is first performed on the training data and the fuzzy rules are subsequently derived through the proper connections of these computed clusters. This correspondence proposes two new networks: Falcon-FKP and Falcon-PFKP. They are extensions of the Falcon-ART network, and aimed to overcome the shortcomings faced by the Falcon-ART network itself, i.e., poor classification ability when the classes of input data are very similar to each other, termination of training cycle depends heavily on a preset error parameter, the fuzzy rule base of the Falcon-ART network may not be consistent Nauck, there is no control over the number of fuzzy rules generated, and learning efficiency may deteriorate by using complementarily coded training data. These deficiencies are essentially inherent to the fuzzy ART, clustering technique employed by the Falcon-ART network. Hence, two clustering techniques--Fuzzy Kohonen Partitioning (FKP) and its pseudo variant PFKP, are synthesized with the basic Falcon structure to compute the fuzzy sets and to automatically derive the fuzzy rules from the training data. The resultant neural fuzzy networks are Falcon-FKP and Falcon-PFKP, respectively. These two proposed networks have a lean and efficient training algorithm and consistent fuzzy rule bases. Extensive simulations are conducted using the two networks and their performances are encouraging when benchmarked against other neural and neural fuzzy systems.     

Eliciting transparent fuzzy model using differential evolution

In this paper a new technique for eliciting a fuzzy inference system (FIS) from data for nonlinear systems is proposed. The strategy is conducted in two phases: in the first one, subtractive clustering is applied in order to extract a set of fuzzy rules, in the second phase, the generated fuzzy rule base is refined and redundant rules are removed on the basis of an interpretability measure. Finally the centers and widths of the Membership Functions (MFs) are tuned by means differential evolution. Case studies are presented to illustrate the efficiency and accuracy of the proposed approach. The results obtained are compared and contrasted with those obtained from a conventionally neuro-fuzzy technique and the superiority of the proposed approach is highlighted.     

A new scheme for fuzzy rule-based system identification and itsapplication to self-tuning fuzzy controllers

There are many important issues that need to be resolved for identification of a fuzzy rule-based system using clustering. We address three such important issues: 1) deciding on the proper domain(s) of clustering; 2) deciding on the number of rules; and 3) getting an initial estimate of parameters of the fuzzy systems. We justify that one should start with separate clustering of X (input) and Y (output). We propose a scheme to establish correspondence between the clusters obtained in X and Y. The correspondence dictates whether further splitting/merging of clusters is needed or not. If X and Y do not exhibit strong cluster substructures, then again clustering of X* (input data augmented by the output data) exploiting the results of separate clustering of X and Y, and of the correspondence scheme is recommended. We justify that usual cluster validity indices are not suitable for finding the number of rules, and the proposed scheme does not use any cluster validity index. Three methods are suggested to get the initial estimate of membership functions (MFs). The proposed scheme is used to identify the rule base needed to realize a self-tuning fuzzy PI-type controller and its performance is found to be quite satisfactory.     

An approach to fuzzy default reasoning for function approximation

This paper discusses fuzzy reasoning for approximately realizing nonlinear functions by a small number of fuzzy if-then rules with different specificity levels. Our fuzzy rule base is a mixture of general and specific rules, which overlap with each other in the input space. General rules work as default rules in our fuzzy rule base. First, we briefly describe existing approaches to the handling of default rules in the framework of possibility theory. Next, we show that standard interpolation-based fuzzy reasoning leads to counterintuitive results when general rules include specific rules with different consequents. Then, we demonstrate that intuitively acceptable results are obtained from a non-standard inclusion-based fuzzy reasoning method. Our approach is based on the preference for more specific rules, which is a commonly used idea in the field of default reasoning. When a general rule includes a specific rule and they are both compatible with an input vector, the weight of the general rule is discounted in fuzzy reasoning. We also discuss the case where general rules do not perfectly but partially include specific rules. Then we propose a genetics-based machine learning (GBML) algorithm for extracting a small number of fuzzy if-then rules with different specificity levels from numerical data using our inclusion-based fuzzy reasoning method. Finally, we describe how our approach can be applied to the approximate realization of fuzzy number-valued nonlinear functions     

A Strategy of Dynamic Reasoning in Knowledge-Based System with Fuzzy Production Rules

Fuzzy production rules have been successfully applied to represent uncertainty in a knowledge-based system. The knowledge organized as a knowledge base is static. On the other hand, a real system such as the stock market is dynamic in nature. Therefore we need a strategy to reflect the dynamic nature of a system when we make reasoning with a knowledge-based system.This paper proposes a strategy of dynamic reasoning that can be used to takes account the dynamic behavior of decision-making with the knowledge-based system consisted of fuzzy rules. A degree of match (DM) between actual input information and antecedent of a rule is represented by a value in interval [0, 1]. Weights of relative importance of attributes in a rule are obtained by the AHP (Analytic Hierarchy Process) method. Then these weights are applied as exponents for the DM, and the DMs in a rule are combined, with the Min operator, into a single DM for the rule. In this way, the importance of attributes of a rule, which can be changed from time to time, can be reflected to reasoning in knowledge-based system with fuzzy rules.With the proposed reasoning procedure, a decision maker can take his judgment on the given decision environment into a static knowledge base with fuzzy rules when he makes decision with the knowledge base. This procedure can be automated as a pre-processing system for fuzzy expert systems. Thereby the quality of decisions could be enhanced.     

SIGNIFICANT FUZZY RULES EXTRACTION BY AN SVD–QR-BASED APPROACH

ABSTRACT

In this article, an SVD–QR-based approach is proposed to extract the important fuzzy rules from a rule base with several fuzzy rule tables to design an appropriate fuzzy system directly from some input-output data of the identified system. A fuzzy system with fuzzy rule tables is defined to approach the input-output pairs of an identified system. In the rule base of the defined fuzzy system, each fuzzy rule table corresponds to a partition of an input space. In order to extract the important fuzzy rules from the rule base of the defined fuzzy system, a firing strength matrix determined by the membership functions of the premise fuzzy sets is constructed. According to the firing strength matrix, the number of important fuzzy rules is determined by the Singular Value Decomposition SVD, and the important fuzzy rules are selected by the SVD–QR-based method. Consequently, a reconstructed fuzzy rule base composed of significant fuzzy rules is determined by the firing strength matrix. Furthermore, the recursive least-squares method is applied to determine the consequent part of the reconstructed fuzzy system according to the gathered input-output data so that a fine fuzzy system is determined by the proposed method. Finally, three nonlinear systems illustrate the efficiency of the proposed method.     

Design and analysis of a class of self-organizing and trainable fuzzy controllers

As the applications of fuzzy-controllers become more complicated, the attributes of self-organization and trainability become increasingly important. Indeed, the specification of fuzzy rules and membership functions for systems with a large number of state variables is extremely difficult. This paper introduces a new class of self-organizing and trainable fuzzy-controllers that can be designed without specific information regarding either the membership functions or the fuzzy rules. The proposed controller derives the fuzzy rules from clusters formed in the input space, through a self-organizing process. The clustering is performed through a simple method which can adaptively allocate new clusters as more date are available to the controller. Then, the membership values of crisp inputs are determined by K-nearest-neighbor (KNN) distance measures applied to the centers of the input clusters. Finally, a KNN defuzzification processes directly estimates of the crisp output of unknown input data. An adaptation procedure for the center vector of each cluster and the corresponding output value is developed. The overall design is analyzed in terms of the existence and the uniqueness of the solution of the proposed model. The performance of the proposed controller is considered through the modeling of the Mackey—Glass time-series.     

Extracting Takagi-Sugeno Fuzzy Rules with Interpretable Submodels via Regularization of Linguistic Modifiers

In this paper, a method for constructing Takagi-Sugeno (TS) fuzzy system from data is proposed with the objective of preserving TS submodel comprehensibility, in which linguistic modifiers are suggested to characterize the fuzzy sets. A good property held by the proposed linguistic modifiers is that they can broaden the cores of fuzzy sets while contracting the overlaps of adjoining membership functions (MFs) during identification of fuzzy systems from data. As a result, the TS submodels identified tend to dominate the system behaviors by automatically matching the global model (GM) in corresponding subareas, which leads to good TS model interpretability while producing distinguishable input space partitioning. However, the GM accuracy and model interpretability are two conflicting modeling objectives, improving interpretability of fuzzy models generally degrades the GM performance of fuzzy models, and vice versa. Hence, one challenging problem is how to construct a TS fuzzy model with not only good global performance but also good submodel interpretability. In order to achieve a good tradeoff between GM performance and submodel interpretability, a regularization learning algorithm is presented in which the GM objective function is combined with a local model objective function defined in terms of an extended index of fuzziness of identified MFs. Moreover, a parsimonious rule base is obtained by adopting a QR decomposition method to select the important fuzzy rules and reduce the redundant ones. Experimental studies have shown that the TS models identified by the suggested method possess good submodel interpretability and satisfactory GM performance with parsimonious rule bases.     

A Dynamically Constrained Multiobjective Genetic Fuzzy System for Regression Problems

In this paper, a multiobjective genetic fuzzy system (GFS) to learn the granularities of fuzzy partitions, tuning the membership functions (MFs), and learning the fuzzy rules is presented. It uses dynamic constraints, which enable three-parameter MF tuning to improve the accuracy while guaranteeing the transparency of fuzzy partitions. The fuzzy models (FMs) are initialized by a method that combines the benefits of Wang–Mendel (WM) and decision-tree algorithms. Thus, the initial FMs have less rules, rule conditions, and input variables than if WM initialization were to be used. Moreover, the fuzzy partitions of initial FMs are always transparent. Our approach is tested against recent multiobjective and monoobjective GFSs on six benchmark problems. It is concluded that the accuracy and interpretability of our FMs are always comparable or better than those in the comparative studies. Furthermore, on some benchmark problems, our approach clearly outperforms some comparative approaches. Suitability of our approach for higher dimensional problems is shown by studying three benchmark problems that have up to 21 input variables.      

FINDING SIMPLIFIED FUZZY IF-THEN RULES FOR FUNCTION APPROXIMATION PROBLEMS USING A FUZZY DATA MINING APPROACH

ABSTRACT

A fuzzy if-then rule whose consequent part is a real number is referred to as a simplified fuzzy rule. Since no defuzzification is required for this rule type, it has been widely used in function approximation problems. Furthermore, data mining can be used to discover useful information by exploring and analyzing data. Therefore, this paper proposes a fuzzy data mining approach to discover simplified fuzzy if-then rules from numerical data in order to approximate an unknown mapping from input to output. Since several pre-specified parameters for deriving fuzzy rules are not easily specified, they are automatically determined by the genetic algorithm with binary chromosomes. To evaluate performance of the proposed method, computer simulations are performed on various numerical data sets, showing that the fitting ability and the generalization ability of the proposed method are comparable to the known fuzzy rule-based methods.     

Simplification of a fuzzy mechanism with rule combination

 In this paper, a systematic approach to reduce the complexity of a fuzzy controller with the rule combination of a fuzzy rule base is presented. The complexity of a fuzzy controller is defined to be the computation load in this work. The proposed rule combination approach can be applied to the fuzzy mechanisms with product–sum and min–max inferences. With the input membership functions indexed in sequence for each input variable, the n-dimensional fuzzy rule table is represented as vectors so that the combination of the fuzzy rule base is realizable. Then the adjacent fuzzy rules with the same output consequent are combined to have smaller size of fuzzy rule base. The fuzzy mechanism with the combined rule table is shown to have the same output with the original fuzzy mechanism (without rule combination). Thus, in many applications, the rule combination approach presented in this paper can be used to reduce the complexity of the fuzzy mechanism without degrading the performances. Moreover, the Don't Care fuzzy rules are defined and it is indicated that the number of the necessary fuzzy rules might be decreased when the Don't Care fuzzy rules are taken into consideration. Further, the properties of the simplification approach for the fuzzy rule base of the fuzzy mechanism are discussed.     

Fuzzy rule-based reasoning approach for event detection and annotation of broadcast soccer video

This paper presents an approach for event detection and annotation of broadcast soccer video. It benefits from the fact that occurrence of some audiovisual features demonstrates remarkable patterns for detection of semantic events. However, the goal of this paper is to propose a flexible system that can be able to be used with minimum reliance on predefined sequences of features and domain knowledge derivative structures. To achieve this goal, we design a fuzzy rule-based reasoning system as a classifier which adopts statistical information from a set of audiovisual features as its crisp input values and produces semantic concepts corresponding to the occurred events. A set of tuples is created by discretization and fuzzification of continuous feature vectors derived from the training data. We extract the hidden knowledge among the tuples and correlation between the features and related events by constructing a decision tree (DT). A set of fuzzy rules is generated by traversing each path from root toward leaf nodes of constructed DT. These rules are inserted in fuzzy rule base of designed fuzzy system and employed by fuzzy inference engine to perform decision-making process and predict the occurred events in input video. Experimental results conducted on a large set of broadcast soccer videos demonstrate the effectiveness of the proposed approach.     

基于BP网络的模糊Petri网的学习能力

模糊Petri网(Fuzzy Petri Nets，FPN)是基于模糊产生式规则的知识库系统的良好建模工具，但自学习能力差是模糊系统本身的一个缺点．该文提出了适合模糊Petri网模型自学习的模糊推理算法和学习算法．在模糊推理算法中，通过对没有回路的FPN模型结构进行层次式划分以及建立变迁点燃和模糊推理的近似连续函数，从而把神经网络中的BP网络算法自然地引入到FPN模型中．在FPN模型上，用误差反传算法计算一阶梯度的方法对模糊产生式规则中的参数进行学习和训练．经过学习和训练的FPN具有很强的泛化能力和自适应功能．FPN模型经过训练得到的参数是有特定含义的，可以通过对这些参数的合法性分析，使得模糊产生式规则系统更加有效，也对知识库系统的建立、更新和维护有着重要的意义．