A Self-Evolving Interval Type-2 Fuzzy Neural Network With Online Structure and Parameter Learning

This paper proposes a self-evolving interval type-2 fuzzy neural network (SEIT2FNN) with online structure and parameter learning. The antecedent parts in each fuzzy rule of the SEIT2FNN are interval type-2 fuzzy sets and the fuzzy rules are of the Takagi–Sugeno–Kang (TSK) type. The initial rule base in the SEIT2FNN is empty, and the online clustering method is proposed to generate fuzzy rules that flexibly partition the input space. To avoid generating highly overlapping fuzzy sets in each input variable, an efficient fuzzy set reduction method is also proposed. This method independently determines whether a corresponding fuzzy set should be generated in each input variable when a new fuzzy rule is generated. For parameter learning, the consequent part parameters are tuned by the rule-ordered Kalman filter algorithm for high-accuracy learning performance. Detailed learning equations on applying the rule-ordered Kalman filter algorithm to the SEIT2FNN consequent part learning, with rules being generated online, are derived. The antecedent part parameters are learned by gradient descent algorithms. The SEIT2FNN is applied to simulations on nonlinear plant modeling, adaptive noise cancellation, and chaotic signal prediction. Comparisons with other type-1 and type-2 fuzzy systems in these examples verify the performance of the SEIT2FNN.      

基于增广输入变量的T-S 模糊模型建模

针对现有T-S 模糊模型建模精度与计算效率之间的矛盾, 提出一种利用增广输入变量进行T-S 模糊模型建模的方法. 对输入变量进行多项式增广处理后, 以核模糊?? 均值聚类算法配合聚类评价指标自适应获得最佳聚类数及相应的模糊划分, 并通过递推最小二乘计算得出T-S 模糊模型的后件参数. 提出可利用后件参数反推断前件结构的方法来快速有效地确定前件结构. 最后通过仿真验证了上述方法的有效性.

     

A new approach to fuzzy modeling of nonlinear dynamic systems with noise: relevance vector learning mechanism

This paper presents a new fuzzy inference system for modeling of nonlinear dynamic systems based on input and output data with measurement noise. The proposed fuzzy system has a number of fuzzy rules and parameter values of membership functions which are automatically generated using the extended relevance vector machine (RVM). The RVM has a probabilistic Bayesian learning framework and has good generalization capability. The RVM consists of the sum of product of weight and kernel function which projects input space into high dimensional feature space. The structure of proposed fuzzy system is same as that of the Takagi-Sugeno fuzzy model. However, in the proposed method, the number of fuzzy rules can be reduced under the process of optimizing a marginal likelihood by adjusting parameter values of kernel functions using the gradient ascent method. After a fuzzy system is determined, coefficients in consequent part are found by the least square method. Examples illustrate effectiveness of the proposed new fuzzy inference system.     

Learning interpretable fuzzy inference systems with FisPro

Fuzzy inference systems (FIS) are likely to play a significant part in system modeling, provided that they remain interpretable following learning from data. The aim of this paper is to set up some guidelines for interpretable FIS learning, based on practical experience with fuzzy modeling in various fields. An open source software system called FisPro has been specifically designed to provide generic tools for interpretable FIS design and learning. It can then be extended with the addition of new contributions. This work presents a global approach to design data-driven FIS that satisfy certain interpretability and accuracy criteria. It includes fuzzy partition generation, rule learning, input space reduction and rule base simplification. The FisPro implementation is discussed and illustrated through several detailed case studies.     

An online self-constructing neural fuzzy inference network and itsapplications

A self-constructing neural fuzzy inference network (SONFIN) with online learning ability is proposed in this paper. The SONFIN is inherently a modified Takagi-Sugeno-Kang (TSK)-type fuzzy rule-based model possessing neural network learning ability. There are no rules initially in the SONFIN. They are created and adapted as online learning proceeds via simultaneous structure and parameter identification. In the structure identification of the precondition part, the input space is partitioned in a flexible way according to an aligned clustering-based algorithm. As to the structure identification of the consequent part, only a singleton value selected by a clustering method is assigned to each rule initially. Afterwards, some additional significant terms selected via a projection-based correlation measure for each rule will be added to the consequent part incrementally as learning proceeds. The combined precondition and consequent structure identification scheme can set up an economic and dynamically growing network, a main feature of the SONFIN. In the parameter identification, the consequent parameters are tuned optimally by either least mean squares or recursive least squares algorithms and the precondition parameters are tuned by a backpropagation algorithm. To enhance the knowledge representation ability of the SONFIN, a linear transformation for each input variable can be incorporated into the network so that much fewer rules are needed or higher accuracy can be achieved     

Hybrid robust approach for TSK fuzzy modeling with outliers

This study proposes a hybrid robust approach for constructing Takagi–Sugeno–Kang (TSK) fuzzy models with outliers. The approach consists of a robust fuzzy C-regression model (RFCRM) clustering algorithm in the coarse-tuning phase and an annealing robust back-propagation (ARBP) learning algorithm in the fine-tuning phase. The RFCRM clustering algorithm is modified from the fuzzy C-regression models (FCRM) clustering algorithm by incorporating a robust mechanism and considering input data distribution and robust similarity measure into the FCRM clustering algorithm. Due to the use of robust mechanisms and the consideration of input data distribution, the fuzzy subspaces and the parameters of functions in the consequent parts are simultaneously identified by the proposed RFCRM clustering algorithm and the obtained model will not be significantly affected by outliers. Furthermore, the robust similarity measure is used in the clustering process to reduce the redundant clusters. Consequently, the RFCRM clustering algorithm can generate a better initialization for the TSK fuzzy models in the coarse-tuning phase. Then, an ARBP algorithm is employed to obtain a more precise model in the fine-tuning phase. From our simulation results, it is clearly evident that the proposed robust TSK fuzzy model approach is superior to existing approaches in learning speed and in approximation accuracy.     

基于自组织神经网络的非线性系统建模

针对基于T-S模糊模型的非线性系统建模问题,提出了一种基于自组织神经网络的新方法.在T-S模糊模型的建模中,目前常用的模糊C均值聚类算法存在迭代次数多,计算耗时的缺点.首先,利用竞争学习算法对输入空间进行聚类,基于此结果,借助于模糊C均值聚类算法进一步优化聚类结果,提取T-S模糊模型的规则前件隶属函数参数.然后,采用最小二乘法求得T-S模糊模型的规则后件参数,从而建立起非线性系统的T-S模糊模型.最后,仿真结果表明,该方法可以为模糊建模提供好的模型结构,并且有较高的计算效率和精度.     

Genetic reinforcement learning through symbiotic evolution forfuzzy controller design

An efficient genetic reinforcement learning algorithm for designing fuzzy controllers is proposed in this paper. The genetic algorithm (GA) adopted in this paper is based upon symbiotic evolution which, when applied to fuzzy controller design, complements the local mapping property of a fuzzy rule. Using this Symbiotic-Evolution-based Fuzzy Controller (SEFC) design method, the number of control trials, as well as consumed CPU time, are considerably reduced when compared to traditional GA-based fuzzy controller design methods and other types of genetic reinforcement learning schemes. Moreover, unlike traditional fuzzy controllers, which partition the input space into a grid, SEFC partitions the input space in a flexible way, thus creating fewer fuzzy rules. In SEFC, different types of fuzzy rules whose consequent parts are singletons, fuzzy sets, or linear equations (TSK-type fuzzy rules) are allowed. Further, the free parameters (e.g., centers and widths of membership functions) and fuzzy rules are all tuned automatically. For the TSK-type fuzzy rule especially, which put the proposed learning algorithm in use, only the significant input variables are selected to participate in the consequent of a rule. The proposed SEFC design method has been applied to different simulated control problems, including the cart-pole balancing system, a magnetic levitation system, and a water bath temperature control system. The proposed SEFC has been verified to be efficient and superior from these control problems, and from comparisons with some traditional GA-based fuzzy systems.     

A novel parametric fuzzy CMAC network and its applications

This paper shows fundamentals and applications of the novel parametric fuzzy cerebellar model articulation controller (P-FCMAC) network. It resembles a neural structure that derived from the Albus CMAC algorithm and Takagi–Sugeno–Kang parametric fuzzy inference systems. The Gaussian basis function is used to model the hypercube structure and the linear parametric equation of the network input variance is used to model the TSK-type output. A self-constructing learning algorithm, which consists of the self-clustering method (SCM) and the backpropagation algorithm, is proposed. The proposed the SCM scheme is a fast, one-pass algorithm for a dynamic estimation of the number of hypercube cells in an input data space. The clustering technique does not require prior knowledge of things such as the number of clusters present in a data set. The backpropagation algorithm is used to tune the adjustable parameters. Illustrative examples were conducted to show the performance and applicability of the proposed model.     

A self-constructing fuzzy CMAC model and its applications

This work presents a self-constructing fuzzy cerebellar model articulation controller (SC-FCMAC) model for various applications. A self-constructing learning algorithm, which consists of the self-clustering method (SCM) and the back-propagation algorithm, is presented. The proposed SCM scheme is a rapid, one-pass algorithm which dynamically estimates the number of hypercube cells in input data space. The clustering method does not require prior knowledge, such as the number of clusters in a data set. The back-propagation algorithm is applied to tune the adjustable parameters. Simulation results are obtained to show the performance and applicability of the proposed model.     

An accuracy-oriented self-splitting fuzzy classifier with support vector learning in high-order expanded consequent space

This paper proposes a self-splitting fuzzy classifier with support vector learning in expanded high-order consequent space (SFC-SVHC) for classification accuracy improvement. The SFC-SVHC expands the rule-mapped consequent space of a first-order Takagi-Sugeno (TS)-type fuzzy system by including high-order terms to enhance the rule discrimination capability. A novel structure and parameter learning approach is proposed to construct the SFC-SVHC. For structure learning, a variance-based self-splitting clustering (VSSC) algorithm is used to determine distributions of the fuzzy sets in the input space. There are no rules in the SFC-SVHC initially. The VSSC algorithm generates a new cluster by splitting an existing cluster into two according to a predefined cluster-variance criterion. The SFC-SVHC uses trigonometric functions to expand the rule-mapped first-order consequent space to a higher-dimensional space. For parameter optimization in the expanded rule-mapped consequent space, a support vector machine is employed to endow the SFC-SVHC with high generalization ability. Experimental results on several classification benchmark problems show that the SFC-SVHC achieves good classification results with a small number of rules. Comparisons with different classifiers demonstrate the superiority of the SFC-SVHC in classification accuracy.     

基于核映射的高阶Takagi-Sugeno模糊模型

本文研究规则后件为非线性函数的高阶Takagi-Sugeno(TS)模糊系统.为求解规则后件的函数表达式,首先通过一个核映射将原输入空间映射到高维特征空间,使原空间的非线性子模型转化为高维特征空间的线性子模型,获得了规则后件的非线性函数的计算公式.然后,给出了用核模糊聚类和最小二乘支持向量机设计模糊系统的一种新算法.最后通过4个公开数据集上的仿真实验验证了所提算法的逼近能力、推广能力和鲁棒性能.     

A fuzzy ensemble of parallel polynomial neural networks with information granules formed by fuzzy clustering

In this paper, we introduce a new category of fuzzy models called a fuzzy ensemble of parallel polynomial neural network (FEP²N²), which consist of a series of polynomial neural networks weighted by activation levels of information granules formed with the use of fuzzy clustering. The two underlying design mechanisms of the proposed networks rely on information granules resulting from the use of fuzzy C-means clustering (FCM) and take advantage of polynomial neural networks (PNNs).The resulting model comes in the form of parallel polynomial neural networks. In the design procedure, in order to estimate the optimal values of the coefficients of polynomial neural networks we use a weighted least square estimation algorithm. We incorporate various types of structures as the consequent part of the fuzzy model when using the learning algorithm. Among the diverse structures being available, we consider polynomial neural networks, which exhibit highly nonlinear characteristics when being viewed as local learning models.We use FCM to form information granules and to overcome the high dimensionality problem. We adopt PNNs to find the optimal local models, which can describe the relationship between the input variables and output variable within some local region of the input space.We show that the generalization capabilities as well as the approximation abilities of the proposed model are improved as a result of using polynomial neural networks. The performance of the network is quantified through experimentation in which we use a number of benchmarks already exploited within the realm of fuzzy or neurofuzzy modeling.     

核空间中智模糊聚类及图像分割应用

目的 为了更有效地提高中智模糊C-均值聚类对非凸不规则数据的聚类性能和噪声污染图像的分割效果,提出了核空间中智模糊均值聚类算法。方法 引入核函数概念。利用满足Mercer条件的非线性问题,用非线性变换把低维空间线性不可分的输入模式空间映射到一个先行可分的高维特征空间进行中智模糊聚类分割。结果 通过对大量图像添加不同的加性和乘性噪声进行分割测试获得的核空间中智模糊聚类算法提高了现有算法的对含噪声聚类的鲁棒性和分类性能。峰值信噪比至少提高0.8 dB。结论 本文算法具有显著的分割效果和良好的鲁棒性,并适应于医学,遥感图像处理需要。     

Gradient‐based back‐propagation dynamical iterative learning scheme for the neuro‐fuzzy inference system

In this paper, a gradient‐based back propagation dynamical iterative learning algorithm is proposed for structure optimization and parameter tuning of the neuro‐fuzzy system. Premise and consequent parameters of the neuro‐fuzzy model are initialized randomly and then tuned by the proposed iterative algorithm. The learning algorithm is based on the first order partial derivative of the output with respect to the structure parameters. The first order derivative of the model output with respect to the structure parameters determines the sensitivity of the model to structure parameters. The sensitivity values are then used to set the tuning factors and parameters updating step sizes. Therefore, an adaptive dynamical iterative scheme is achieved which adapts the learning procedure to the current state of the performance during the optimization process. Larger tuning step sizes make the convergence speed higher and vice versa. In this regard, this parameter is treated according to the calculated sensitivity of the model to the parameter. The proposed learning algorithm is compared with the least square back propagation method, genetic algorithm and chaotic genetic algorithm in the neuro‐fuzzy model structure optimization. Smaller mean square error and shorter learning time are sought in this paper, and the performance of the proposed learning algorithm is versified regarding these criteria.     

Adaptive fuzzy identification and predictive control for industrial processes

This paper proposes a method for adaptive identification and control for industrial applications. The learning of a T–S fuzzy model is performed from input/output data to approximate unknown nonlinear processes by a hierarchical genetic algorithm (HGA). The HGA approach is composed by five hierarchical levels where the following parameters of the T–S fuzzy system are learned: input variables and their respective time delays, antecedent fuzzy sets, consequent parameters, and fuzzy rules. In order to reduce the computational cost and increase the algorithm’s performance an initialization method is applied on HGA. To deal with nonlinear plants and time-varying processes, the T–S fuzzy model is adapted online to maintain the quality of the identification/control. The identification methodology is proposed for two application problems: (1) the design of data-driven soft sensors, and (2) the learning of a model for the Generalized predictive control (GPC) algorithm. The integration of the proposed adaptive identification method with the GPC results in an effective adaptive predictive fuzzy control methodology. To validate and demonstrate the performance and effectiveness of the proposed methodologies, they are applied on identification of a model for the estimation of the flour concentration in the effluent of a real-world wastewater treatment system; and on control of a simulated continuous stirred tank reactor (CSTR) and on a real experimental setup composed of two coupled DC motors. The results are presented, showing that the developed evolving T–S fuzzy model can identify the nonlinear systems satisfactorily and it can be used successfully as a prediction model of the process for the GPC controller.     

Support vector learning for fuzzy rule-based classification systems

To design a fuzzy rule-based classification system (fuzzy classifier) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, the support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high- (or even infinite) dimensional feature space. This paper investigates the connection between fuzzy classifiers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. We first show that a fuzzy classifier implicitly defines a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet a certain spectral requirement. The corresponding fuzzy classifier is named positive definite fuzzy classifier (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the "curse of dimensionality." Finally, PDFCs with different reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided.     

T–S fuzzy model identification based on a novel fuzzy c-regression model clustering algorithm

This paper proposes a novel approach for identification of Takagi–Sugeno (T–S) fuzzy model, which is based on a new fuzzy c-regression model (FCRM) clustering algorithm. The clustering prototype in fuzzy space partition is hyper-plane, so FCRM clustering technique is more suitable to be applied in premise parameters identification of T–S fuzzy model. A new FCRM clustering algorithm (NFCRMA) is presented, which is deduced from the fuzzy clustering objective function of FCRM with Lagrange multiplier rule, possessing integrative and concise structure. The proposed approach consists mainly of two steps: premise parameter identification and consequent parameter identification. The NFCRMA is utilized to partition the input–output data and identify the premise parameters, which can discover the real structure of the training data; on the other hand, orthogonal least square is exploited to identify the consequent parameters. Finally, some examples are given to verify the validity of the proposed modeling approach, and the results show the new approach is very efficient and of high accuracy.     

A fuzzy neural network based on fuzzy hierarchy error approach

This paper presents a novel fuzzy neural network which consists of an antecedent network and a consequent network. The antecedent network matches the premises of the fuzzy rules and the consequent network implements the consequences of the rules. In the network learning and training phase, a concise and effective algorithm based on the fuzzy hierarchy error approach is proposed to update the parameters of the network. This algorithm is simple to implement and it does not require as many calculations as some other classic neural network learning algorithms. A model reference adaptive control structure incorporating the proposed fuzzy neural network is studied. Simulation results of a cart-pole balancing system demonstrate the effectiveness of the proposed method