Takagi-Sugeno-Kang fuzzy system fusion: A survey at hierarchical,wide and stacked levels

With excellent global approximation performance and interpretability, Takagi-Sugeno-Kang (TSK) fuzzy systems have enjoyed a wide range of applications in various fields, such as smart control, medical, and finance. However, in handling high-dimensional complex data, the performance and interpretability of a single TSK fuzzy system are easily degraded by rule explosion due to the curse of dimensionality. Ensemble learning comes into play to deal with the problem by the fusion of multiple TSK fuzzy systems using appropriate ensemble learning strategies, which has shown to be effective in eliminating the issue of the curse of dimensionality curse problem and reducing the number of fuzzy rules, thereby maintaining the interpretability of fuzzy systems. To this end, this paper gives a comprehensive survey of TSK fuzzy system fusion to provide insights into further research development. First, we briefly review the fundamental concepts related to TSK fuzzy systems, including fuzzy rule structures, training methods, and interpretability, and discuss the three different development directions of TSK fuzzy systems. Next, along the direction of TSK fuzzy system fusion, we investigate in detail the current ensemble strategies for fusion at hierarchical, wide and stacked levels, and discuss their differences, merits and weaknesses from the aspects of time complexity, interpretability (model complexity) and classification performance. We then present some applications of TSK fuzzy systems in real-world scenarios. Finally, the challenges and future directions of TSK fuzzy system fusion are discussed to foster prospective research.     

A new methodology to improve interpretability in neuro-fuzzy TSK models

The present paper puts forward a methodology which allows increasing interpretability of TSK models identified by means of neuro-fuzzy techniques, although it shall also be applicable to models identified through other hybrid or different techniques. With this purpose, this paper puts forward a method which allows oriented adjustment of the rules’ precedent and consequent parameters in TSK models. The methodology extends the adaptive phase with an adjustment phase (or fine tuning phase) based on overlap ratio and overlap area, where the gradient descendent algorithm is used to adjust precisely the adapted parameters in the fuzzy model. The adjustment based on the overlap ratio is applied to the parameters defining the rules’ precedent and consequent parts. The overlap area becomes a more precise tuning of parameters of precedent part of rules. After the adaptation of the neuro-fuzzy model by means of the developed methodology, the model acquires a clear physical meaning enabling its immediate linguistic interpretation. Finally, some examples are given to prove the validity of the developed methodology.     

Improving the interpretability of TSK fuzzy models by combiningglobal learning and local learning

The fuzzy inference system proposed by Takagi, Sugeno, and Kang, known as the TSK model in fuzzy system literature, provides a powerful tool for modeling complex nonlinear systems. Unlike conventional modeling where a single model is used to describe the global behavior of a system, TSK modeling is essentially a multimodel approach in which simple submodels (typically linear models) are combined to describe the global behavior of the system. Most existing learning algorithms for identifying the TSK model are based on minimizing the square of the residual between the overall outputs of the real system and the identified model. Although these algorithms can generate a TSK model with good global performance (i.e., the model is capable of approximating the given system with arbitrary accuracy, provided that sufficient rules are used and sufficient training data are available), they cannot guarantee the resulting model to have a good local performance. Often, the submodels in the TSK model may exhibit an erratic local behavior, which is difficult to interpret. Since one of the important motivations of using the TSK model (also other fuzzy models) is to gain insights into the model, it is important to investigate the interpretability issue of the TSK model. We propose a new learning algorithm that integrates global learning and local learning in a single algorithmic framework. This algorithm uses the idea of local weighed regression and local approximation in nonparametric statistics, but remains the component of global fitting in the existing learning algorithms. The algorithm is capable of adjusting its parameters based on the user's preference, generating models with good tradeoff in terms of global fitting and local interpretation. We illustrate the performance of the proposed algorithm using a motorcycle crash modeling example     

Parameter Identification of Recurrent Fuzzy Systems With Fuzzy Finite-State Automata Representation

This paper presents the identification of nonlinear dynamical systems by recurrent fuzzy system (RFS) models. Two types of RFS models are discussed: the Takagi-Sugeno-Kang (TSK) type and the linguistic or Mamdani type. Both models are equivalent and the latter model may be represented by a fuzzy finite-state automaton (FFA). An identification procedure is proposed based on a standard general purpose genetic algorithm (GA). First, the TSK rule parameters are estimated and, in a second step, the TSK model is converted into an equivalent linguistic model. The parameter identification is evaluated in some benchmark problems for nonlinear system identification described in literature. The results show that RFS models achieve good numerical performance while keeping the interpretability of the actual system dynamics.     

GSETSK: a generic self-evolving TSK fuzzy neural network with a novel Hebbian-based rule reduction approach

Takagi–Sugeno–Kang (TSK) fuzzy systems have been widely applied for solving function approximation and regression-centric problems. Existing dynamic TSK models proposed in the literature can be broadly classified into two classes. Class I TSK models are essentially fuzzy systems that are limited to time-invariant environments. Class II TSK models are generally evolving systems that can learn in time-variant environments. This paper attempts to address the issues of achieving compact, up-to-date fuzzy rule bases and interpretable knowledge bases in TSK models. It proposes a novel rule pruning method which is simple, computationally efficient and biologically plausible. This rule pruning algorithm applies a gradual forgetting approach and adopts the Hebbian learning mechanism behind the long-term potentiation phenomenon in the brain. It also proposes a merging approach which is used to improve the interpretability of the knowledge bases. This approach can prevent derived fuzzy sets from expanding too many times to protect their semantic meanings. These two approaches are incorporated into a generic self-evolving Takagi–Sugeno–Kang fuzzy framework (GSETSK) which adopts an online data-driven incremental-learning-based approach.Extensive experiments were conducted to evaluate the performance of the proposed GSETSK against other established evolving TSK systems. GSETSK has also been tested on real world dataset using the high-way traffic flow density and Dow Jones index time series. The results are encouraging. GSETSK demonstrates its fast learning ability in time-variant environments. In addition, GSETSK derives an up-to-date and better interpretable fuzzy rule base while maintaining a high level of modeling accuracy at the same time.     

Neuro-fuzzy methods for nonlinear system identification

Most processes in industry are characterized by nonlinear and time-varying behavior. Nonlinear system identification is becoming an important tool which can be used to improve control performance and achieve robust fault-tolerant behavior. Among the different nonlinear identification techniques, methods based on neuro-fuzzy models are gradually becoming established not only in the academia but also in industrial applications. Neuro-fuzzy modeling can be regarded as a gray-box technique on the boundary between neural networks and qualitative fuzzy models. The tools for building neuro-fuzzy models are based on combinations of algorithms from the fields of neural networks, pattern recognition and regression analysis. In this paper, an overview of neuro-fuzzy modeling methods for nonlinear system identification is given, with an emphasis on the tradeoff between accuracy and interpretability.     

Complexity reduction and interpretability improvement for fuzzy rule systems based on simple interpretability measures and indices by bi-objective evolutionary rule selection

The aim of this paper is to develop a general post-processing methodology to reduce the complexity of data-driven linguistic fuzzy models, in order to reach simpler fuzzy models preserving enough accuracy and better fuzzy linguistic performance with respect to their initial values. This post-processing approach is based on rule selection via the formulation of a bi-objective problem with one objective focusing on accuracy and the other on interpretability. The latter is defined via the aggregation of several interpretability measures, based on the concepts of similarity and complexity of fuzzy systems and rules. In this way, a measure of the fuzzy model interpretability is given. Two neuro-fuzzy systems for providing initial fuzzy models, Fuzzy Adaptive System ART based and Neuro-Fuzzy Function Approximation and several case studies, data sets from KEEL Project Repository, are used to check this approach. Both fuzzy and neuro-fuzzy systems generate Mamdani-type fuzzy rule-based systems, each with its own particularities and complexities from the point of view of the fuzzy sets and the rule generation. Based on these systems and data sets, several fuzzy models are generated to check the performance of the proposal under different restrictions of complexity and fuzziness.     

结构辨识和参数优化协同学习的概率TSK模糊系统

传统Takagi-Sugeno-Kang (TSK)模糊系统的结构辨识和参数优化往往分阶段进行, 同时模糊规则数需要预先设定, 因此TSK模糊系统的逼近性能和解释性往往不理想.针对此问题, 提出了一种结构辨识和参数优化协同学习的概率TSK模糊系统(Probabilistic TSK fuzzy system, PTSK).首先, PTSK使用概率模型表示模糊回归系统, 将结构辨识和参数优化作为一个整体来考虑.其次, PTSK不借助于专家经验, 使用粒子滤波方法对规则数和前后件参数协同学习, 得到系统全部参数的最优解.实验结果表明, PTSK具有良好的逼近性能, 同时能获得较少的模糊规则数.     

Approximation of dynamic systems using recurrent neuro-fuzzy techniques

Fuzzy systems, neural networks and its combination in neuro-fuzzy systems are already well established in data analysis and system control. Especially, neuro-fuzzy systems are well suited for the development of interactive data analysis tools, which enable the creation of rule-based knowledge from data and the introduction of a-priori knowledge into the process of data analysis. In this article an architecture is presented that was designed to learn and optimize a hierarchical fuzzy rule base with feedback connections using a genetic algorithm for rule base structure learning and a gradient descent method to optimize the fuzzy sets of the learned rule base. Since this architecture is able to store information of prior system states, the model is especially suited for the analysis of dynamic systems.     

Hybrid robust approach for TSK fuzzy modeling with outliers

This study proposes a hybrid robust approach for constructing Takagi–Sugeno–Kang (TSK) fuzzy models with outliers. The approach consists of a robust fuzzy C-regression model (RFCRM) clustering algorithm in the coarse-tuning phase and an annealing robust back-propagation (ARBP) learning algorithm in the fine-tuning phase. The RFCRM clustering algorithm is modified from the fuzzy C-regression models (FCRM) clustering algorithm by incorporating a robust mechanism and considering input data distribution and robust similarity measure into the FCRM clustering algorithm. Due to the use of robust mechanisms and the consideration of input data distribution, the fuzzy subspaces and the parameters of functions in the consequent parts are simultaneously identified by the proposed RFCRM clustering algorithm and the obtained model will not be significantly affected by outliers. Furthermore, the robust similarity measure is used in the clustering process to reduce the redundant clusters. Consequently, the RFCRM clustering algorithm can generate a better initialization for the TSK fuzzy models in the coarse-tuning phase. Then, an ARBP algorithm is employed to obtain a more precise model in the fine-tuning phase. From our simulation results, it is clearly evident that the proposed robust TSK fuzzy model approach is superior to existing approaches in learning speed and in approximation accuracy.     

Fuzzy network model-based fuzzy state controller design

The performance of model-based controller design relies heavily on the quality and suitability of the utilized process model. This contribution proposes a fuzzy network based nonlinear controller design methodology. Fuzzy networks are a model approach combining high approximation quality with high interpretability. The input/output (I/O) models commonly used for identification are transformed to fuzzy state-space models. Transferring and adjusting methods from linear state-space theory a control concept consisting of a fuzzy state controller and an adaptive set-point filter for nonlinear dynamic processes is deduced. The capability of the method is demonstrated for a hydraulic drive     

Bayesian zero-order TSK fuzzy system modeling

Different from the existing TSK fuzzy system modeling methods, a novel zero-order TSK fuzzy modeling method called Bayesian zero-order TSK fuzzy system (B-ZTSK-FS) is proposed from the perspective of Bayesian inference in this paper. The proposed method B-ZTSK-FS constructs zero-order TSK fuzzy system by using the maximum a posteriori (MAP) framework to maximize the corresponding posteriori probability. First, a joint likelihood model about zero-order TSK fuzzy system is defined to derive a new objective function which can assure that both antecedents and consequents of fuzzy rules rather than only their antecedents of the most existing TSK fuzzy systems become interpretable. The defined likelihood model is composed of three aspects: clustering on the training set for antecedents of fuzzy rules, the least squares (LS) error for consequent parameters of fuzzy rules, and a Dirichlet prior distribution for fuzzy cluster memberships which is considered to not only automatically match the “sum-to-one” constraints on fuzzy cluster memberships, but also make the proposed method B-ZTSK-FS scalable for large-scale datasets by appropriately setting the Dirichlet index. This likelihood model indeed indicates that antecedent and consequent parameters of fuzzy rules can be linguistically interpreted and simultaneously optimized by the proposed method B-ZTSK-FS which is based on the MAP framework with the iterative sampling algorithm, which in fact implies that fuzziness and probability can co-jointly work for TSK fuzzy system modeling in a collaborative rather than repulsive way. Finally, experimental results on 28 synthetic and real-world datasets are reported to demonstrate the effectiveness of the proposed method B-ZTSK-FS in the sense of approximation accuracy, interpretability and scalability.     

Using adaptive neuro-fuzzy inference system for hydrological time series prediction

Conventionally, the multiple linear regression procedure has been known as the most popular models in simulating hydrological time series. However, when the nonlinear phenomenon is significant, the multiple linear will fail to develop an appropriate predictive model. Recently, intelligence system approaches such as artificial neural network (ANN) and neuro-fuzzy methods have been used successfully for time series modelling. In most instances for neural networks, multi layer perceptrons (MLPs) that are trained with the back-propagation algorithm have been used. The major shortcoming of this approach is that the knowledge contained in the trained networks is difficult to interpret. Using neuro-fuzzy approaches, which enable the information that is stored in trained networks to be expressed in the form of a fuzzy rule base, would help to overcome this issue. In the present study, a time series neuro-fuzzy model is proposed that is capable of exploiting the strengths of traditional time series approaches. The aim of this article is to investigate the potential of a neuro-fuzzy system with a Sugeno inference engine, considering different numbers of membership functions. Three rivers have been selected and daily prediction for them was applied. For better judgment, outcomes of the network have been compared to an autoregressive model.     

The equivalence between fuzzy logic systems and feedforward neuralnetworks

Demonstrates that fuzzy logic systems and feedforward neural networks are equivalent in essence. First, we introduce the concept of interpolation representations of fuzzy logic systems and several important conclusions. We then define mathematical models for rectangular wave neural networks and nonlinear neural networks. With this definition, we prove that nonlinear neural networks can be represented by rectangular wave neural networks. Based on this result, we prove the equivalence between fuzzy logic systems and feedforward neural networks. This result provides us a very useful guideline when we perform theoretical research and applications on fuzzy logic systems, neural networks, or neuro-fuzzy systems.     

A neuro-fuzzy inference system through integration of fuzzy logic and extreme learning machines.

This paper investigates the feasibility of applying a relatively novel neural network technique, i.e., extreme learning machine (ELM), to realize a neuro-fuzzy Takagi-Sugeno-Kang (TSK) fuzzy inference system. The proposed method is an improved version of the regular neuro-fuzzy TSK fuzzy inference system. For the proposed method, first, the data that are processed are grouped by the k-means clustering method. The membership of arbitrary input for each fuzzy rule is then derived through an ELM, followed by a normalization method. At the same time, the consequent part of the fuzzy rules is obtained by multiple ELMs. At last, the approximate prediction value is determined by a weight computation scheme. For the ELM-based TSK fuzzy inference system, two extensions are also proposed to improve its accuracy. The proposed methods can avoid the curse of dimensionality that is encountered in backpropagation and hybrid adaptive neuro-fuzzy inference system (ANFIS) methods. Moreover, the proposed methods have a competitive performance in training time and accuracy compared to three ANFIS methods.     

FITSK: online local learning with generic fuzzy input Takagi-Sugeno-Kang fuzzy framework for nonlinear system estimation.

Existing Takagi-Sugeno-Kang (TSK) fuzzy models proposed in the literature attempt to optimize the global learning accuracy as well as to maintain the interpretability of the local models. Most of the proposed methods suffer from the use of offline learning algorithms to globally optimize this multi-criteria problem. Despite the ability to reach an optimal solution in terms of accuracy and interpretability, these offline methods are not suitably applicable to learning in adaptive or incremental systems. Furthermore, most of the learning methods in TSK-model are susceptible to the limitation of the curse-of-dimensionality. This paper attempts to study the criteria in the design of TSK-models. They are: 1) the interpretability of the local model; 2) the global accuracy; and 3) the system dimensionality issues. A generic framework is proposed to handle the different scenarios in this design problem. The framework is termed the generic fuzzy input Takagi-Sugeno-Kang fuzzy framework (FITSK). The FITSK framework is extensible to both the zero-order and the first-order FITSK models. A zero-order FITSK model is suitable for the learning of adaptive system, and the bias-variance of the system can be easily controlled through the degree of localization. On the other hand, a first-order FITSK model is able to achieve higher learning accuracy for nonlinear system estimation. A localized version of recursive least-squares algorithm is proposed for the parameter tuning of the first-order FITSK model. The local recursive least-squares is able to achieve a balance between interpretability and learning accuracy of a system, and possesses greater immunity to the curse-of-dimensionality. The learning algorithms for the FITSK models are online, and are readily applicable to adaptive system with fast convergence speed. Finally, a proposed guideline is discussed to handle the model selection of different FITSK models to tackle the multi-criteria design problem of applying the TSK-model. Extensive simulations were conducted using the proposed FITSK models and their learning algorithms; their performances are encouraging when benchmarked against other popular fuzzy systems.     

Data mining in forecasting PVT correlations of crude oil systems based on Type1 fuzzy logic inference systems

Pressure–volume–temperature properties are very important in the reservoir engineering computations. There are many empirical approaches for predicting various PVT properties based on empirical correlations and statistical regression models. Last decade, researchers utilized neural networks to develop more accurate PVT correlations. These achievements of neural networks open the door to data mining techniques to play a major role in oil and gas industry. Unfortunately, the developed neural networks correlations are often limited, and global correlations are usually less accurate compared to local correlations. Recently, adaptive neuro-fuzzy inference systems have been proposed as a new intelligence framework for both prediction and classification based on fuzzy clustering optimization criterion and ranking. This paper proposes neuro-fuzzy inference systems for estimating PVT properties of crude oil systems. This new framework is an efficient hybrid intelligence machine learning scheme for modeling the kind of uncertainty associated with vagueness and imprecision. We briefly describe the learning steps and the use of the Takagi Sugeno and Kang model and Gustafson–Kessel clustering algorithm with K-detected clusters from the given database. It has featured in a wide range of medical, power control system, and business journals, often with promising results. A comparative study will be carried out to compare their performance of this new framework with the most popular modeling techniques, such as neural networks, nonlinear regression, and the empirical correlations algorithms. The results show that the performance of neuro-fuzzy systems is accurate, reliable, and outperform most of the existing forecasting techniques. Future work can be achieved by using neuro-fuzzy systems for clustering the 3D seismic data, identification of lithofacies types, and other reservoir characterization.     

RSPOP: rough set-based pseudo outer-product fuzzy rule identification algorithm

System modeling with neuro-fuzzy systems involves two contradictory requirements: interpretability verses accuracy. The pseudo outer-product (POP) rule identification algorithm used in the family of pseudo outer-product-based fuzzy neural networks (POPFNN) suffered from an exponential increase in the number of identified fuzzy rules and computational complexity arising from high-dimensional data. This decreases the interpretability of the POPFNN in linguistic fuzzy modeling. This article proposes a novel rough set-based pseudo outer-product (RSPOP) algorithm that integrates the sound concept of knowledge reduction from rough set theory with the POP algorithm. The proposed algorithm not only performs feature selection through the reduction of attributes but also extends the reduction to rules without redundant attributes. As many possible reducts exist in a given rule set, an objective measure is developed for POPFNN to correctly identify the reducts that improve the inferred consequence. Experimental results are presented using published data sets and real-world application involving highway traffic flow prediction to evaluate the effectiveness of using the proposed algorithm to identify fuzzy rules in the POPFNN using compositional rule of inference and singleton fuzzifier (POPFNN-CRI(S)) architecture. Results showed that the proposed rough set-based pseudo outer-product algorithm reduces computational complexity, improves the interpretability of neuro-fuzzy systems by identifying significantly fewer fuzzy rules, and improves the accuracy of the POPFNN.     

基于UKF的自适应模糊推理神经网络

如何生成最优的模糊规则数及模糊规则的自动生成和修剪是模糊神经网络训练算法研究的重点。针对这一问题,本文提出了基于UKF的自适应模糊推理神经网络(UKF-ANFIS)。首先,通过减法聚类确定UKF-ANFIS的模糊规则及其高斯隶属函数的中心和宽度参数;其次,分析了模糊神经网络的非线性动力系统表示,并用LLS和UKF分别学习线性和非线性的参数;然后,用误差下降率方法作为模糊规则修剪的策略,删除作用不大的规则;最后,通过典型的函数逼近和系统辨识实例,表明本文算法得到的模糊神经网络的结构更为紧凑,泛化性能也更佳。     

A moving-average filter based hybrid ARIMA–ANN model for forecasting time series data

A suitable combination of linear and nonlinear models provides a more accurate prediction model than an individual linear or nonlinear model for forecasting time series data originating from various applications. The linear autoregressive integrated moving average (ARIMA) and nonlinear artificial neural network (ANN) models are explored in this paper to devise a new hybrid ARIMA–ANN model for the prediction of time series data. Many of the hybrid ARIMA–ANN models which exist in the literature apply an ARIMA model to given time series data, consider the error between the original and the ARIMA-predicted data as a nonlinear component, and model it using an ANN in different ways. Though these models give predictions with higher accuracy than the individual models, there is scope for further improvement in the accuracy if the nature of the given time series is taken into account before applying the models. In the work described in this paper, the nature of volatility was explored using a moving-average filter, and then an ARIMA and an ANN model were suitably applied. Using a simulated data set and experimental data sets such as sunspot data, electricity price data, and stock market data, the proposed hybrid ARIMA–ANN model was applied along with individual ARIMA and ANN models and some existing hybrid ARIMA–ANN models. The results obtained from all of these data sets show that for both one-step-ahead and multistep-ahead forecasts, the proposed hybrid model has higher prediction accuracy.