A Self-Evolving Interval Type-2 Fuzzy Neural Network With Online Structure and Parameter Learning

This paper proposes a self-evolving interval type-2 fuzzy neural network (SEIT2FNN) with online structure and parameter learning. The antecedent parts in each fuzzy rule of the SEIT2FNN are interval type-2 fuzzy sets and the fuzzy rules are of the Takagi–Sugeno–Kang (TSK) type. The initial rule base in the SEIT2FNN is empty, and the online clustering method is proposed to generate fuzzy rules that flexibly partition the input space. To avoid generating highly overlapping fuzzy sets in each input variable, an efficient fuzzy set reduction method is also proposed. This method independently determines whether a corresponding fuzzy set should be generated in each input variable when a new fuzzy rule is generated. For parameter learning, the consequent part parameters are tuned by the rule-ordered Kalman filter algorithm for high-accuracy learning performance. Detailed learning equations on applying the rule-ordered Kalman filter algorithm to the SEIT2FNN consequent part learning, with rules being generated online, are derived. The antecedent part parameters are learned by gradient descent algorithms. The SEIT2FNN is applied to simulations on nonlinear plant modeling, adaptive noise cancellation, and chaotic signal prediction. Comparisons with other type-1 and type-2 fuzzy systems in these examples verify the performance of the SEIT2FNN.      

A type-2 neural fuzzy system learned through type-1 fuzzy rules and its FPGA-based hardware implementation

This paper first proposes a type-2 neural fuzzy system (NFS) learned through its type-1 counterpart (T2NFS-T1) and then implements the built IT2NFS-T1 in a field-programmable gate array (FPGA) chip. The antecedent part of each fuzzy rule in the T2NFS-T1 uses interval type-2 fuzzy sets, while the consequent part uses a Takagi-Sugeno-Kang (TSK) type with interval combination weights. The T2NFS-T1 uses a simplified type-reduction operation to reduce system training time and hardware implementation cost. Given a training data set, a TSK type-1 NFS is first learned through structure and parameter learning. The built type-1 fuzzy logic system (FLS) is then extended to a type-2 FLS, where highly overlapped type-1 fuzzy sets are merged into interval type-2 fuzzy sets to reduce the total number of fuzzy sets. Finally, the rule consequent and antecedent parameters in the T2NFS-T1 are tuned using a hybrid of the gradient descent and rule-ordered recursive least square (RLS) algorithms. Simulation results and comparisons with various type-1 and type-2 FLSs verify the effectiveness and efficiency of the T2NFS-T1 for system modeling and prediction problems. A new hardware circuit using both parallel-processing and pipeline techniques is proposed to implement the learned T2NFS-T1 in an FPGA chip. The T2NFS-T1 chip reduces the hardware implementation cost in comparison to other type-2 fuzzy chips.     

A Type-2 Self-Organizing Neural Fuzzy System and Its FPGA Implementation

This paper proposes a type-2 self-organizing neural fuzzy system (T2SONFS) and its hardware implementation. The antecedent parts in each T2SONFS fuzzy rule are interval type-2 fuzzy sets, and the consequent part is of Mamdani type. Using interval type-2 fuzzy sets in T2SONFS enables it to be more robust than type-1 fuzzy systems. T2SONFS learning consists of structure and parameter identification. For structure identification, an online clustering algorithm is proposed to generate rules automatically and flexibly distribute them in the input space. For parameter identification, a rule-ordered Kalman filter algorithm is proposed to tune the consequent-part parameters. The learned T2SONFS is hardware implemented, and implementation techniques are proposed to simplify the complex computation process of a type-2 fuzzy system. The T2SONFS is applied to nonlinear system identification and truck backing control problems with clean and noisy training data. Comparisons between type-1 and type-2 neural fuzzy systems verify the learning ability and robustness of the T2SONFS. The learned T2SONFS is hardware implemented in a field-programmable gate array chip to verify functionality of the designed circuits.      

An evolving recurrent interval type-2 intuitionistic fuzzy neural network for online learning and time series prediction

The prediction of time series has both the theoretical value and practical significance in reality. However, since the high nonlinear and noises in the time series, it is still an open problem to tackle with the uncertainties and fuzziness in the forecasting process. In this article, an evolving recurrent interval type-2 intuitionistic fuzzy neural network (eRIT2IFNN) is proposed for time series prediction and regression problems. The eRIT2IFNN employs interval type-2 intuitionistic fuzzy sets to enhance the modeling of uncertainties by intuitionistic evaluation and noise tolerance of the system. In the eRIT2IFNN, the antecedent part of each fuzzy rule is defined using intuitionistic interval type-2 fuzzy sets, and the consequent realizes the Takagi–Sugeno–Kang type fuzzy inference mechanism. In order to utilize the prior knowledge including intuitionistic information, a local internal feedback is established by feeding the rule firing strength of each rule to itself eRIT2IFNN is fully adaptive to the evolving of sequence data by online learning of structure and parameters. A modified density-based clustering is implemented for the structure learning, where both densities and membership degrees are involved to determine the fuzzy rules. Performance of eRIT2IFNN is evaluated using a set of benchmark problems and compared with existing fuzzy inference systems. Moreover, the eRIT2IFNN is tested for identification of dynamics under both noise-free and noisy environments. Finally, a group of practical financial price-tracking problems including high-frequency data of financial future, commodity future and precious metal are used for the evaluation of the proposed inference system.     

Recurrent type-2 fuzzy neural network using Haar wavelet energy and entropy features for speech detection in noisy environments

This paper proposes a new method to detect the boundary of speech in noisy environments. This detection method uses Haar wavelet energy and entropy (HWEE) as detection features. The Haar wavelet energy (HWE) is derived by using the robust band that shows the most significant difference between speech and nonspeech segments at different noise levels. Similarly, the wavelet energy entropy (WEE) is computed by selecting the two wavelet energy bands whose entropy shows the most significant speech/nonspeech difference. The HWEE features are fed as inputs to a recurrent self-evolving interval type-2 fuzzy neural network (RSEIT2FNN) for classification. The RSEIT2FNN is used because it uses type-2 fuzzy sets, which are more robust to noise than type-1 fuzzy sets. The recurrent structure in the RSEIT2FNN helps to remember the context information of a test frame. The RSEIT2FNN outputs are compared with a parameter threshold to determine whether it is a speech or nonspeech period. The HWEE-based RSEIT2FNN detection was applied to speech detection in different noisy environments with different noise levels. Comparisons with different detection methods verified the advantage of the proposed method of using HWEE.     

Dynamical optimal training for interval type-2 fuzzy neural network (T2FNN)

Type-2 fuzzy logic system (FLS) cascaded with neural network, type-2 fuzzy neural network (T2FNN), is presented in this paper to handle uncertainty with dynamical optimal learning. A T2FNN consists of a type-2 fuzzy linguistic process as the antecedent part, and the two-layer interval neural network as the consequent part. A general T2FNN is computational-intensive due to the complexity of type 2 to type 1 reduction. Therefore, the interval T2FNN is adopted in this paper to simplify the computational process. The dynamical optimal training algorithm for the two-layer consequent part of interval T2FNN is first developed. The stable and optimal left and right learning rates for the interval neural network, in the sense of maximum error reduction, can be derived for each iteration in the training process (back propagation). It can also be shown both learning rates cannot be both negative. Further, due to variation of the initial MF parameters, i.e., the spread level of uncertain means or deviations of interval Gaussian MFs, the performance of back propagation training process may be affected. To achieve better total performance, a genetic algorithm (GA) is designed to search optimal spread rate for uncertain means and optimal learning for the antecedent part. Several examples are fully illustrated. Excellent results are obtained for the truck backing-up control and the identification of nonlinear system, which yield more improved performance than those using type-1 FNN.     

Reinforcement Interval Type-2 Fuzzy Controller Design by Online Rule Generation and Q-Value-Aided Ant Colony Optimization

This paper proposes a new reinforcement-learning method using online rule generation and Q-value-aided ant colony optimization (ORGQACO) for fuzzy controller design. The fuzzy controller is based on an interval type-2 fuzzy system (IT2FS). The antecedent part in the designed IT2FS uses interval type-2 fuzzy sets to improve controller robustness to noise. There are initially no fuzzy rules in the IT2FS. The ORGQACO concurrently designs both the structure and parameters of an IT2FS. We propose an online interval type-2 rule generation method for the evolution of system structure and flexible partitioning of the input space. Consequent part parameters in an IT2FS are designed using Q-values and the reinforcement local-global ant colony optimization algorithm. This algorithm selects the consequent part from a set of candidate actions according to ant pheromone trails and Q-values, both of which are updated using reinforcement signals. The ORGQACO design method is applied to the following three control problems: (1) truck-backing control; (2) magnetic-levitation control; and (3) chaotic-system control. The ORGQACO is compared with other reinforcement-learning methods to verify its efficiency and effectiveness. Comparisons with type-1 fuzzy systems verify the noise robustness property of using an IT2FS.     

Adaptive Fuzzy Neural Networks as identifiers of discrete-time nonlinear dynamic systems

An adaptive supervised learning scheme is proposed in this paper for training Fuzzy Neural Networks (FNN) to identify discrete-time nonlinear dynamical systems. The FNN constructs are neural-network-based connectionist models consisting of several layers that are used to implement the functions of a fuzzy logic system. The fuzzy rule base considered here consists of Takagi-Sugeno IF-THEN rules, where the rule outputs are realized as linear polynomials of the input components. The FNN connectionist model is functionally partitioned into three separate parts, namely, the premise part, which provides the truth values of the rule preconditional statements, the consequent part providing the rule outputs, and the defuzzification part computing the final output of the FNN construct. The proposed learning scheme is a two-stage training algorithm that performs both structure and parameter learning, simultaneously. First, the structure learning task determines the proper fuzzy input partitions and the respective precondition matching, and is carried out by means of the rule base adaptation mechanism. The rule base adaptation mechanism is a self-organizing procedure which progressively generates the proper fuzzy rule base, during training, according to the operating conditions. Having completed the structure learning stage, the parameter learning is applied using the back-propagation algorithm, with the objective to adjust the premise/consequent parameters of the FNN so that the desired input/output representation is captured to an acceptable degree of accuracy. The structure/parameter training algorithm exhibits good learning and generalization capabilities as demonstrated via a series of simulation studies. Comparisons with conventional multilayer neural networks indicate the effectiveness of the proposed scheme.     

Comments on “Dynamical Optimal Training for Interval Type-2 Fuzzy Neural Network (T2FNN)”

In this comment, it will be shown that the backpropagation (BP) equations by Wang are not correct. These BP equations were used to tune the parameters of the antecedent type-2 membership functions as well as the consequent part of the interval type-2 fuzzy neural networks (T2FNNs). These incorrect equations would have led to erroneous results, and hence this might affect the comparisons and findings presented by Wang This comment will highlight the correct BP tuning equations for the T2FNN.     

Type-2 fuzzy neural networks with fuzzy clustering and differential evolution optimization

In many real-world problems involving pattern recognition, system identification and modeling, control, decision making, and forecasting of time-series, available data are quite often of uncertain nature. An interesting alternative is to employ type-2 fuzzy sets, which augment fuzzy models with expressive power to develop models, which efficiently capture the factor of uncertainty. The three-dimensional membership functions of type-2 fuzzy sets offer additional degrees of freedom that make it possible to directly and more effectively account for model’s uncertainties. Type-2 fuzzy logic systems developed with the aid of evolutionary optimization forms a useful modeling tool subsequently resulting in a collection of efficient “If-Then” rules.The type-2 fuzzy neural networks take advantage of capabilities of fuzzy clustering by generating type-2 fuzzy rule base, resulting in a small number of rules and then optimizing membership functions of type-2 fuzzy sets present in the antecedent and consequent parts of the rules. The clustering itself is realized with the aid of differential evolution.Several examples, including a benchmark problem of identification of nonlinear system, are considered. The reported comparative analysis of experimental results is used to quantify the performance of the developed networks.     

Robust Control of an LUSM-Based $Xhbox{--}Yhbox{--}theta$ Motion Control Stage Using an Adaptive Interval Type-2 Fuzzy Neural Network

The robust control of a linear ultrasonic motor based $Xhbox{--}Yhbox{--}theta$ motion control stage to track various contours is achieved by using an adaptive interval type-2 fuzzy neural network (AIT2FNN) control system in this study. In the proposed AIT2FNN control system, an IT2FNN, which combines the merits of an interval type-2 fuzzy logic system and a neural network, is developed to approximate an unknown dynamic function. Moreover, adaptive learning algorithms are derived using the Lyapunov stability theorem to train the parameters of the IT2FNN online. Furthermore, a robust compensator is proposed to confront the uncertainties including the approximation error, optimal parameter vectors, and higher order terms in Taylor series. To relax the requirement for the value of lumped uncertainty in the robust compensator, an adaptive lumped uncertainty estimation law is also investigated. In addition, the circle and butterfly contours are planned using a nonuniform rational B-spline curve interpolator. The experimental results show that the contour tracking performance of the proposed AIT2FNN is significantly improved compared with the adaptive type-1 FNN. Additionally, the robustness to parameter variations, external disturbances, cross-coupled interference, and frictional force can also be obtained using the proposed AIT2FNN.      

Comments on "Dynamical optimal training for interval type-2 fuzzy neural network (T2FNN)".

In this comment, it will be shown that the backpropagation (BP) equations by Wang et al. are not correct. These BP equations were used to tune the parameters of the antecedent type-2 membership functions as well as the consequent part of the interval type-2 fuzzy neural networks (T2FNNs). These incorrect equations would have led to erroneous results, and hence this might affect the comparisons and findings presented by Wang et al. This comment will highlight the correct BP tuning equations for the T2FNN.     

柔索驱动并联机构的二型模糊神经逆控制

由于建立精确数学模型的困难以及控制过程中各种不确定性的存在, 柔索驱动并联机构的水平调节具有一定的难度. 针对该问题, 提出了一种基于二型模糊神经网络的逆控制方案. 该控制方案中的二型模糊神经网络实现了对水平调节过程逆动态的逼近以及对各种不确定性的处理. 采用迭代最小二乘算法对二型模糊神经网络区间权重进行了优化. 最后, 将基于二型模糊神经网络的逆控制方案在实际的控制对象上进行了实验, 并与其相对应的基于一型模糊神经网络的逆控制方案进行了比较. 实验结果表明所提出的控制方案是有效的且采用二型模糊神经网络时能获得更好的控制效果.     

Bearing condition prediction considering uncertainty: An interval type-2 fuzzy neural network approach

Rolling-element bearings are critical components of rotating machinery. It is important to accurately predict in real-time the health condition of bearings so that maintenance practices can be scheduled to avoid malfunctions or even catastrophic failures. In this paper, an Interval Type-2 Fuzzy Neural Network (IT2FNN) is proposed to perform multi-step-ahead condition prediction of faulty bearings. Since the IT2FNN defines an interval type-2 fuzzy logic system in the form of a multi-layer neural network, it can integrate the merits of each, such as fuzzy reasoning to handle uncertainties and neural networks to learn from data. The interval type-2 fuzzy linguistic process in the IT2FNN enables the system to handle prediction uncertainties, since the type-2 fuzzy sets are such sets whose membership grades are type-1 fuzzy sets that can be used in failure prediction due to the difficult determination of an exact membership function for a fuzzy set. Noisy data of faulty bearings are used to validate the proposed predictor, whose performance is compared with that of a prevalent type-1 condition predictor called Adaptive Neuro-Fuzzy Inference System (ANFIS). The results show that better prediction accuracy can be achieved via the IT2FNN.     

A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks

In real life, information about the world is uncertain and imprecise. The cause of this uncertainty is due to: deficiencies on given information, the fuzzy nature of our perception of events and objects, and on the limitations of the models we use to explain the world. The development of new methods for dealing with information with uncertainty is crucial for solving real life problems. In this paper three interval type-2 fuzzy neural network (IT2FNN) architectures are proposed, with hybrid learning algorithm techniques (gradient descent backpropagation and gradient descent with adaptive learning rate backpropagation). At the antecedents layer, a interval type-2 fuzzy neuron (IT2FN) model is used, and in case of the consequents layer an interval type-1 fuzzy neuron model (IT1FN), in order to fuzzify the rule’s antecedents and consequents of an interval type-2 Takagi-Sugeno-Kang fuzzy inference system (IT2-TSK-FIS). IT2-TSK-FIS is integrated in an adaptive neural network, in order to take advantage the best of both models. This provides a high order intuitive mechanism for representing imperfect information by means of use of fuzzy If-Then rules, in addition to handling uncertainty and imprecision. On the other hand, neural networks are highly adaptable, with learning and generalization capabilities. Experimental results are divided in two kinds: in the first one a non-linear identification problem for control systems is simulated, here a comparative analysis of learning architectures IT2FNN and ANFIS is done. For the second kind, a non-linear Mackey-Glass chaotic time series prediction problem with uncertainty sources is studied. Finally, IT2FNN proved to be more efficient mechanism for modeling real-world problems.     

A PSO based integrated functional link net and interval type-2 fuzzy logic system for predicting stock market indices

This paper presents an integrated functional link interval type-2 fuzzy neural system (FLIT2FNS) for predicting the stock market indices. The hybrid model uses a TSK (Takagi-Sugano-Kang) type fuzzy rule base that employs type-2 fuzzy sets in the antecedent parts and the outputs from the Functional Link Artificial Neural Network (FLANN) in the consequent parts. Two other approaches, namely the integrated FLANN and type-1 fuzzy logic system and Local Linear Wavelet Neural Network (LLWNN) are also presented for a comparative study. Backpropagation and particle swarm optimization (PSO) learning algorithms have been used independently to optimize the parameters of all the forecasting models. To test the model performance, three well known stock market indices like the Standard's & Poor's 500 (S&P 500), Bombay stock exchange (BSE), and Dow Jones industrial average (DJIA) are used. The mean absolute percentage error (MAPE) and root mean square error (RMSE) are used to find out the performance of all the three models. Finally, it is observed that out of three methods, FLIT2FNS performs the best irrespective of the time horizons spanning from 1 day to 1 month.     

Perturbation wavelet neural sliding mode position control for a voice coil motor driver

To cope with the nonlinear electro-magneto-mechanical characteristics, this paper proposes a perturbation wavelet neural sliding mode position control (PWSPC) system for a voice coil motor (VCM) driver. A perturbed wavelet neural network (PWNN) approximator is used to online approximate the unknown nonlinear term in the VCM system dynamics. The PWNN approximator uses perturbed wavelet functions to handle the rules uncertainties like interval type-2 fuzzy sets. The structure learning ability enables the PWNN approximator to evolve its structure online. Further, the parameter learning laws and stability analysis are derived in the sense of Lyapunov function; thus, the parameter convergence and system stability can be guaranteed. Finally, the experimental results verify that the proposed PWSPC system can achieve favorable control performance such as good disturbance rejection and good tracking accuracy.

     

An online self-constructing neural fuzzy inference network and itsapplications

A self-constructing neural fuzzy inference network (SONFIN) with online learning ability is proposed in this paper. The SONFIN is inherently a modified Takagi-Sugeno-Kang (TSK)-type fuzzy rule-based model possessing neural network learning ability. There are no rules initially in the SONFIN. They are created and adapted as online learning proceeds via simultaneous structure and parameter identification. In the structure identification of the precondition part, the input space is partitioned in a flexible way according to an aligned clustering-based algorithm. As to the structure identification of the consequent part, only a singleton value selected by a clustering method is assigned to each rule initially. Afterwards, some additional significant terms selected via a projection-based correlation measure for each rule will be added to the consequent part incrementally as learning proceeds. The combined precondition and consequent structure identification scheme can set up an economic and dynamically growing network, a main feature of the SONFIN. In the parameter identification, the consequent parameters are tuned optimally by either least mean squares or recursive least squares algorithms and the precondition parameters are tuned by a backpropagation algorithm. To enhance the knowledge representation ability of the SONFIN, a linear transformation for each input variable can be incorporated into the network so that much fewer rules are needed or higher accuracy can be achieved     

Hybrid learning for interval type-2 fuzzy logic systems based on orthogonal least-squares and back-propagation methods

This paper presents a novel learning methodology based on a hybrid algorithm for interval type-2 fuzzy logic systems. Since only the back-propagation method has been proposed in the literature for the tuning of both the antecedent and the consequent parameters of type-2 fuzzy logic systems, a hybrid learning algorithm has been developed. The hybrid method uses a recursive orthogonal least-squares method for tuning the consequent parameters and the back-propagation method for tuning the antecedent parameters. Systems were tested for three types of inputs: (a) interval singleton, (b) interval type-1 non-singleton, and (c) interval type-2 non-singleton. Experiments were carried out on the application of hybrid interval type-2 fuzzy logic systems for prediction of the scale breaker entry temperature in a real hot strip mill for three different types of coil. The results proved the feasibility of the systems developed here for scale breaker entry temperature prediction. Comparison with type-1 fuzzy logic systems shows that hybrid learning interval type-2 fuzzy logic systems provide improved performance under the conditions tested.     

A Functional-Link-Based Neurofuzzy Network for Nonlinear System Control

This study presents a functional-link-based neurofuzzy network (FLNFN) structure for nonlinear system control. The proposed FLNFN model uses a functional link neural network (FLNN) to the consequent part of the fuzzy rules. This study uses orthogonal polynomials and linearly independent functions in a functional expansion of the FLNN. Thus, the consequent part of the proposed FLNFN model is a nonlinear combination of input variables. An online learning algorithm, which consists of structure learning and parameter learning, is also presented. The structure learning depends on the entropy measure to determine the number of fuzzy rules. The parameter learning, based on the gradient descent method, can adjust the shape of the membership function and the corresponding weights of the FLNN. Furthermore, results for the universal approximator and a convergence analysis of the FLNFN model are proven. Finally, the FLNFN model is applied in various simulations. Results of this study demonstrate the effectiveness of the proposed FLNFN model.