Additive Fuzzy Systems: From Generalized Mixtures to Rule Continua

A generalized probability mixture density governs an additive fuzzy system. The fuzzy system's if‐then rules correspond to the mixed probability densities. An additive fuzzy system computes an output by adding its fired rules and then averaging the result. The mixture's convex structure yields Bayes theorems that give the probability of which rules fired or which combined fuzzy systems fired for a given input and output. The convex structure also results in new moment theorems and learning laws and new ways to both approximate functions and exactly represent them. The additive fuzzy system itself is just the first conditional moment of the generalized mixture density. The output is a convex combination of the centroids of the fired then‐part sets. The mixture's second moment defines the fuzzy system's conditional variance. It describes the inherent uncertainty in the fuzzy system's output due to rule interpolation. The mixture structure gives a natural way to combine fuzzy systems because mixing mixtures yields a new mixture. A separation theorem shows how fuzzy approximators combine with exact Watkins‐based two‐rule function representations in a higher‐level convex sum of the combined systems. Two mixed Gaussian densities with appropriate Watkins coefficients define a generalized mixture density such that the fuzzy system's output equals any given real‐valued function if the function is bounded and not constant. Statistical hill‐climbing algorithms can learn the generalized mixture from sample data. The mixture structure also extends finite rule bases to continuum‐many rules. Finite fuzzy systems suffer from exponential rule explosion because each input fires all their graph‐cover rules. The continuum system fires only a special random sample of rules based on Monte Carlo sampling from the system's mixture. Users can program the system by changing its wave‐like meta‐rules based on the location and shape of the mixed densities in the mixture. Such meta‐rules can help mitigate rule explosion. The meta‐rules grow only linearly with the number of mixed densities even though the underlying fuzzy if‐then rules can have high‐dimensional if‐part and then‐part fuzzy sets.     

Adaptive Fuzzy Neural Networks as identifiers of discrete-time nonlinear dynamic systems

An adaptive supervised learning scheme is proposed in this paper for training Fuzzy Neural Networks (FNN) to identify discrete-time nonlinear dynamical systems. The FNN constructs are neural-network-based connectionist models consisting of several layers that are used to implement the functions of a fuzzy logic system. The fuzzy rule base considered here consists of Takagi-Sugeno IF-THEN rules, where the rule outputs are realized as linear polynomials of the input components. The FNN connectionist model is functionally partitioned into three separate parts, namely, the premise part, which provides the truth values of the rule preconditional statements, the consequent part providing the rule outputs, and the defuzzification part computing the final output of the FNN construct. The proposed learning scheme is a two-stage training algorithm that performs both structure and parameter learning, simultaneously. First, the structure learning task determines the proper fuzzy input partitions and the respective precondition matching, and is carried out by means of the rule base adaptation mechanism. The rule base adaptation mechanism is a self-organizing procedure which progressively generates the proper fuzzy rule base, during training, according to the operating conditions. Having completed the structure learning stage, the parameter learning is applied using the back-propagation algorithm, with the objective to adjust the premise/consequent parameters of the FNN so that the desired input/output representation is captured to an acceptable degree of accuracy. The structure/parameter training algorithm exhibits good learning and generalization capabilities as demonstrated via a series of simulation studies. Comparisons with conventional multilayer neural networks indicate the effectiveness of the proposed scheme.     

Learning Premises of Fuzzy Rules for Knowledge Acquisition in Classification Problems

A key issue in building fuzzy classification systems is the specification of rule conditions, which determine the structure of a knowledge base. This paper presents a new approach to automatically extract classification knowledge from numerical data by means of premise learning. A genetic algorithm is employed to search for premise structure in combination with parameters of membership functions of input fuzzy sets to yield optimal conditions of classification rules. The major advantage of our work is that a parsimonious knowledge base with a low number of rules can be achieved. The practical applicability of the proposed method is examined by computer simulations on two well-known benchmark problems of Iris Data and Cancer Data classification. Received 11 February 1999 / Revised 13 January 2001 / Accepted in revised form 13 February 2001     

Dynamic parameter optimization of evolutionary computation for on-line prediction of time series with changing dynamics

This paper introduces a dynamic evolving computation system (DECS) model, for adaptive on-line learning, and its application for dynamic time series prediction. DECS evolve through evolving clustering method and evolutionary computation for structure learning, Levenberg–Marquardt method for parameter learning, learning and accommodate new input data. DECS is created and updated during the operation of the system. At each time moment the output of DECS is calculated through a knowledge rule inference system based on m-most activated fuzzy rules which are dynamically chosen from a fuzzy rule set. An approach is proposed for a dynamic creation of a first order Takagi–Sugeno type fuzzy rule set for the DECS model. The fuzzy rules can be inserted into DECS before, or during its learning process, and the rules can also be extracted from DECS during, or after its learning process. It is demonstrated that DECS can effectively learn complex temporal sequences in an adaptive way and outperform some existing models.     

A neural fuzzy system with fuzzy supervised learning

A neural fuzzy system learning with fuzzy training data (fuzzy if-then rules) is proposed in this paper. This system is able to process and learn numerical information as well as linguistic information. At first, we propose a five-layered neural network for the connectionist realization of a fuzzy inference system. The connectionist structure can house fuzzy logic rules and membership functions for fuzzy inference. We use alpha-level sets of fuzzy numbers to represent linguistic information. The inputs, outputs, and weights of the proposed network can be fuzzy numbers of any shape. Furthermore, they can be hybrid of fuzzy numbers and numerical numbers through the use of fuzzy singletons. Based on interval arithmetics, a fuzzy supervised learning algorithm is developed for the proposed system. It extends the normal supervised learning techniques to the learning problems where only linguistic teaching signals are available. The fuzzy supervised learning scheme can train the proposed system with desired fuzzy input-output pairs which are fuzzy numbers instead of the normal numerical values. With fuzzy supervised learning, the proposed system can be used for rule base concentration to reduce the number of rules in a fuzzy rule base. Simulation results are presented to illustrate the performance and applicability of the proposed system.     

A multi-objective genetic optimization of interpretability-oriented fuzzy rule-based classifiers

The paper presents a multi-objective genetic approach to design interpretability-oriented fuzzy rule-based classifiers from data. The proposed approach allows us to obtain systems with various levels of compromise between their accuracy and interpretability. During the learning process, parameters of the membership functions, as well as the structure of the classifier's fuzzy rule base (i.e., the number of rules, the number of rule antecedents, etc.) evolve simultaneously using a Pittsburgh-type genetic approach. Since there is no particular coding of fuzzy rule structures in a chromosome (it reduces computational complexity of the algorithm), original crossover and mutation operators, as well as chromosome-repairing technique to directly transform the rules are also proposed. To evaluate both the accuracy and interpretability of the system, two measures are used. The first one – an accuracy measure – is based on the root mean square error of the system's response. The second one – an interpretability measure – is based on the arithmetic mean of three components: (a) the average length of rules (the average number of antecedents used in the rules), (b) the number of active fuzzy sets and (c) the number of active inputs of the system (an active fuzzy set or input means a set or input used by at least one fuzzy rule). Both measures are used as objectives in multi-objective (2-objective in our case) genetic optimization approaches such as well-known SPEA2 and NSGA-II algorithms. Moreover, for the purpose of comparison with several alternative approaches, the experiments are carried out both considering the so-called strong fuzzy partitions (SFPs) of attribute domains and without them. SFPs provide more semantically meaningful solutions, usually at the expense of their accuracy. The operation of the proposed technique in various classification problems is tested with the use of 20 benchmark data sets and compared to 11 alternative classification techniques. The experiments show that the proposed approach generates classifiers of significantly improved interpretability, while still characterized by competitive accuracy.     

Fuzzy modeling of fluidized catalytic cracking unit

The paper deals with the fuzzy system identification of reactor–regenerator–stripper–fractionator's (RRSF) section of a fluidized catalytic cracking unit (FCCU). The fuzzy system identification based on the data collected from an operating refinery of FCCU of capacity, 1.2 MMPTA, with a sample time of 10 min. A generalized fuzzy model (GFM) and identification of structure and model parameter for multi-input/single output is presented. The GFM has the capability of representing both the CRI model and TS model under certain conditions. The structure identification and the parameter estimation are carried out using hybrid learning approach comprising modified mountain clustering and gradient descent learning with least square estimation (LSE) for the identification of a fuzzy model. The modified mountain clustering considers every data point as a potential cluster center in x × y hyperspace. The optimum number of clusters, which leads to an optimum number of rules, is determined with the help of validity function that guides the search. The obtained result from the modified mountain clustering initializes the GFM. Further hybrid of the gradient descent technique and LSE is aimed at learning of the GFM parameters in two phases. In the first phase of an epoch of learning gradient descent tunes the premise parameter and index of fuzziness of each rule. In second phase, LSE utilizes the results of first phase for evaluating the coefficient of local linear model of corresponding rules.     

An adaptive neural fuzzy filter and its applications

A new kind of nonlinear adaptive filter, the adaptive neural fuzzy filter (ANFF), based upon a neural network's learning ability and fuzzy if-then rule structure, is proposed in this paper. The ANFF is inherently a feedforward multilayered connectionist network which can learn by itself according to numerical training data or expert knowledge represented by fuzzy if-then rules. The adaptation here includes the construction of fuzzy if-then rules (structure learning), and the tuning of the free parameters of membership functions (parameter learning). In the structure learning phase, fuzzy rules are found based on the matching of input-output clusters. In the parameter learning phase, a backpropagation-like adaptation algorithm is developed to minimize the output error. There are no hidden nodes (i.e., no membership functions and fuzzy rules) initially, and both the structure learning and parameter learning are performed concurrently as the adaptation proceeds. However, if some linguistic information about the design of the filter is available, such knowledge can be put into the ANFF to form an initial structure with hidden nodes. Two major advantages of the ANFF can thus be seen: 1) a priori knowledge can be incorporated into the ANFF which makes the fusion of numerical data and linguistic information in the filter possible; and 2) no predetermination, like the number of hidden nodes, must be given, since the ANFF can find its optimal structure and parameters automatically     

A recurrent self-organizing neural fuzzy inference network

A recurrent self-organizing neural fuzzy inference network (RSONFIN) is proposed. The RSONFIN is inherently a recurrent multilayered connectionist network for realizing the basic elements and functions of dynamic fuzzy inference, and may be considered to be constructed from a series of dynamic fuzzy rules. The temporal relations embedded in the network are built by adding some feedback connections representing the memory elements to a feedforward neural fuzzy network. Each weight as well as node in the RSONFIN has its own meaning and represents a special element in a fuzzy rule. There are no hidden nodes initially in the RSONFIN. They are created online via concurrent structure identification and parameter identification. The structure learning together with the parameter learning forms a fast learning algorithm for building a small, yet powerful, dynamic neural fuzzy network. Two major characteristics of the RSONFIN can thus be seen: 1) the recurrent property of the RSONFIN makes it suitable for dealing with temporal problems and 2) no predetermination, like the number of hidden nodes, must be given, since the RSONFIN can find its optimal structure and parameters automatically and quickly. Moreover, to reduce the number of fuzzy rules generated, a flexible input partition method, the aligned clustering-based algorithm, is proposed. Various simulations on temporal problems are done and performance comparisons with some existing recurrent networks are also made. Efficiency of the RSONFIN is verified from these results.     

Performance evaluation of fuzzy classifier systems formultidimensional pattern classification problems

We examine the performance of a fuzzy genetics-based machine learning method for multidimensional pattern classification problems with continuous attributes. In our method, each fuzzy if-then rule is handled as an individual, and a fitness value is assigned to each rule. Thus, our method can be viewed as a classifier system. In this paper, we first describe fuzzy if-then rules and fuzzy reasoning for pattern classification problems. Then we explain a genetics-based machine learning method that automatically generates fuzzy if-then rules for pattern classification problems from numerical data. Because our method uses linguistic values with fixed membership functions as antecedent fuzzy sets, a linguistic interpretation of each fuzzy if-then rule is easily obtained. The fixed membership functions also lead to a simple implementation of our method as a computer program. The simplicity of implementation and the linguistic interpretation of the generated fuzzy if-then rules are the main characteristic features of our method. The performance of our method is evaluated by computer simulations on some well-known test problems. While our method involves no tuning mechanism of membership functions, it works very well in comparison with other classification methods such as nonfuzzy machine learning techniques and neural networks.     

A Self-Evolving Interval Type-2 Fuzzy Neural Network With Online Structure and Parameter Learning

This paper proposes a self-evolving interval type-2 fuzzy neural network (SEIT2FNN) with online structure and parameter learning. The antecedent parts in each fuzzy rule of the SEIT2FNN are interval type-2 fuzzy sets and the fuzzy rules are of the Takagi–Sugeno–Kang (TSK) type. The initial rule base in the SEIT2FNN is empty, and the online clustering method is proposed to generate fuzzy rules that flexibly partition the input space. To avoid generating highly overlapping fuzzy sets in each input variable, an efficient fuzzy set reduction method is also proposed. This method independently determines whether a corresponding fuzzy set should be generated in each input variable when a new fuzzy rule is generated. For parameter learning, the consequent part parameters are tuned by the rule-ordered Kalman filter algorithm for high-accuracy learning performance. Detailed learning equations on applying the rule-ordered Kalman filter algorithm to the SEIT2FNN consequent part learning, with rules being generated online, are derived. The antecedent part parameters are learned by gradient descent algorithms. The SEIT2FNN is applied to simulations on nonlinear plant modeling, adaptive noise cancellation, and chaotic signal prediction. Comparisons with other type-1 and type-2 fuzzy systems in these examples verify the performance of the SEIT2FNN.      

基于规则熵函数法的结构自适应模糊辨识与控制

提出一种新的结构自适应模糊辨识器,其特点是采用“规则熵函数”法在线调节模糊逻辑系统参数,使规则前件参数趋于一致,从而进行规则合并,简化了系统结构。将该结构自适应模糊辨识器用于一类非线性系统的自适应控制,仿真结果验证了所提出方法的有效性。     

SIGNIFICANT FUZZY RULES EXTRACTION BY AN SVD–QR-BASED APPROACH

ABSTRACT

In this article, an SVD–QR-based approach is proposed to extract the important fuzzy rules from a rule base with several fuzzy rule tables to design an appropriate fuzzy system directly from some input-output data of the identified system. A fuzzy system with fuzzy rule tables is defined to approach the input-output pairs of an identified system. In the rule base of the defined fuzzy system, each fuzzy rule table corresponds to a partition of an input space. In order to extract the important fuzzy rules from the rule base of the defined fuzzy system, a firing strength matrix determined by the membership functions of the premise fuzzy sets is constructed. According to the firing strength matrix, the number of important fuzzy rules is determined by the Singular Value Decomposition SVD, and the important fuzzy rules are selected by the SVD–QR-based method. Consequently, a reconstructed fuzzy rule base composed of significant fuzzy rules is determined by the firing strength matrix. Furthermore, the recursive least-squares method is applied to determine the consequent part of the reconstructed fuzzy system according to the gathered input-output data so that a fine fuzzy system is determined by the proposed method. Finally, three nonlinear systems illustrate the efficiency of the proposed method.     

An ART-based fuzzy adaptive learning control network

This paper addresses the structure and an associated online learning algorithm of a feedforward multilayer neural net for realizing the basic elements and functions of a fuzzy controller. The proposed fuzzy adaptive learning control network (FALCON) can be contrasted with traditional fuzzy control systems in network structure and learning ability. An online structure/parameter learning algorithm, FALCON-ART, is proposed for constructing FALCON dynamically. It combines backpropagation for parameter learning and fuzzy ART for structure learning. FALCON-ART partitions the input state space and output control space using irregular fuzzy hyperboxes according to the data distribution. In many existing fuzzy or neural fuzzy control systems, the input and output spaces are always partitioned into “grids”. As the number of variables increases, the number of partitioned grids grows combinatorially. To avoid this problem in some complex systems, FALCON-ART partitions the I/O spaces flexibly based on data distribution. It can create and train FALCON in a highly autonomous way. In its initial form, there is no membership function, fuzzy partition, and fuzzy logic rule. They are created and begin to grow as the first training pattern arrives. Thus, the users need not give it any a priori knowledge or initial information. FALCON-ART can online partition the I/O spaces, tune membership functions, find proper fuzzy logic rules, and annihilate redundant rules dynamically upon receiving online data     

基于自适应神经模糊推理系统的视频烟雾检测

提出一种基于自适应神经模糊推理系统的视频烟雾检测算法。从视频图像中提取烟雾特征,采用减法聚类确定模糊规则数,建立初始模糊系统。通过神经网络的自学习机制调整前提参数和结论参数,确定模糊推理规则。实验结果表明,与传统BP神经网络算法及支持向量机算法相比,该算法具有较优的ROC曲线特性。     

A parallel fuzzy inference model with distributed prediction schemefor reinforcement learning

This paper proposes a three-layered parallel fuzzy inference model called reinforcement fuzzy neural network with distributed prediction scheme (RFNN-DPS), which performs reinforcement learning with a novel distributed prediction scheme. In RFNN-DPS, an additional predictor for predicting the external reinforcement signal is not necessary, and the internal reinforcement information is distributed into fuzzy rules (rule nodes). Therefore, using RFNN-DPS, only one network is needed to construct a fuzzy logic system with the abilities of parallel inference and reinforcement learning. Basically, the information for prediction in RFNN-DPS is composed of credit values stored in fuzzy rule nodes, where each node holds a credit vector to represent the reliability of the corresponding fuzzy rule. The credit values are not only accessed for predicting external reinforcement signals, but also provide a more profitable internal reinforcement signal to each fuzzy rule itself. RFNN-DPS performs a credit-based exploratory algorithm to adjust its internal status according to the internal reinforcement signal. During learning, the RFNN-DPS network is constructed by a single-step or multistep reinforcement learning algorithm based on the ART concept. According to our experimental results, RFNN-DPS shows the advantages of simple network structure, fast learning speed, and explicit representation of rule reliability.     

A Recurrent Self-Evolving Interval Type-2 Fuzzy Neural Network for Dynamic System Processing

This paper proposes a recurrent self-evolving interval type-2 fuzzy neural network (RSEIT2FNN) for dynamic system processing. An RSEIT2FNN incorporates type-2 fuzzy sets in a recurrent neural fuzzy system in order to increase the noise resistance of a system. The antecedent parts in each recurrent fuzzy rule in the RSEIT2FNN are interval type-2 fuzzy sets, and the consequent part is of the Takagi-Sugeno-Kang (TSK) type with interval weights. The antecedent part of RSEIT2FNN forms a local internal feedback loop by feeding the rule firing strength of each rule back to itself. The TSK-type consequent part is a linear model of exogenous inputs. The RSEIT2FNN initially contains no rules; all rules are learned online via structure and parameter learning. The structure learning uses online type-2 fuzzy clustering. For the parameter learning, the consequent part parameters are tuned by a rule-ordered Kalman filter algorithm to improve learning performance. The antecedent type-2 fuzzy sets and internal feedback loop weights are learned by a gradient descent algorithm. The RSEIT2FNN is applied to simulations of dynamic system identifications and chaotic signal prediction under both noise-free and noisy conditions. Comparisons with type-1 recurrent fuzzy neural networks validate the performance of the RSEIT2FNN.     

Fuzzy Identification of Nonlinear Systems via Orthogonal Transform

This paper is concerned with the application of orthogonal transforms and fuzzy competitive learning to extract fuzzy rules from data. The least square algorithm with orthogonal transforms is proposed to supervise the progress of fuzzy competitive learning. First of all, competitive learning takes place in the product space of system inputs and outputs and each cluster corresponds to a fuzzy IF–THEN rule. The fuzzy relation matrix, confirmed by fuzzy competitive learning, is studied by the orthogonal least square algorithm. The validity of fuzzy rules is obtained by analyzing the effect of orthogonal vectors in the fuzzy model, and subsequently removing less important ones. The structure identification and parameter identification of the fuzzy model are simultaneously confirmed in the proposed algorithm. Using simulation results as an example, the fuzzy model of non‐linear systems can be built by using the proposed algorithm. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

On employing fuzzy modeling algorithms for the valuation of residential premises

In this paper, we investigate fuzzy modeling techniques for predicting the prices of residential premises, based on some main drivers such as usable area of premises, age of a building, number of rooms in a flat, floor on which a flat is located, number of storeys in a building as well as the distance from the city center. Our proposed modeling techniques rely on two aspects: the first one (called SparseFIS) is a batch off-line modeling method and tries to out-sparse an initial dense rule population by optimizing the rule weights within an iterative optimization procedure subject to constrain the number of important rules; the second one (called FLEXFIS) is a single-pass incremental method which is able to build up fuzzy models in an on-line sample-wise learning context. As such, it is able to adapt former generated prediction models with new data recordings on demand and also to cope with on-line data streams. The final obtained fuzzy models provide some interpretable insight into the relations between the various features and residential prices in form of linguistically readable rules (IF-THEN conditions). Both methods will be compared with a state-of-the-art premise estimation method usually conducted by many experts and exploiting heuristic concepts such as sliding time window, nearest neighbors and averaging. The comparison is based on a two real-world data set including prices for residential premises within the years 1998-2008.     

A recurrent fuzzy-neural model for dynamic system identification

This paper presents a fuzzy modeling approach for identification of dynamic systems. In particular, a new fuzzy model, the Dynamic Fuzzy Neural Network (DFNN), consisting of recurrent TSK rules, is developed. The premise and defuzzification parts are static while the consequent parts of the fuzzy rules are recurrent neural networks with internal feedback and time delay synapses. The network is trained by means of a novel learning algorithm, named Dynamic-Fuzzy Neural Constrained Optimization Method (D-FUNCOM), based on the concept of constrained optimization. The proposed algorithm is general since it can be applied to locally as well as fully recurrent networks, regardless of their structures. An adaptation mechanism of the maximum parameter change is presented as well. The proposed dynamic model, equipped with the learning algorithm, is applied to several temporal problems, including modeling of a NARMA process and the noise cancellation problem. Performance comparisons are conducted with a series of static and dynamic systems and some existing recurrent fuzzy models. Simulation results show that DFNN compares favorably with its competing rivals and thus it can be considered for efficient system identification.