Identification using ANFIS with intelligent hybrid stable learning algorithm approaches and stability analysis of training methods

This paper proposes a novel hybrid learning algorithm with stable learning laws for Adaptive Network based Fuzzy Inference System (ANFIS) as a system identifier and studies the stability of this algorithm. The new hybrid learning algorithm is based on particle swarm optimization (PSO) for training the antecedent part and forgetting factor recursive least square (FFRLS) for training the conclusion part. Two famous training algorithms for ANFIS are the gradient descent (GD) to update antecedent part parameters and using GD or recursive least square (RLS) to update conclusion part parameters. Lyapunov stability theory is used to study the stability of the proposed algorithms. This paper, also studies the stability of PSO as an optimizer in training the identifier. Stable learning algorithms for the antecedent and consequent parts of fuzzy rules are proposed. Some constraints are obtained and simulation results are given to validate the results. It is shown that instability will not occur for the leaning rate and PSO factors in the presence of constraints. The learning rate can be calculated on-line and will provide an adaptive learning rate for the ANFIS structure. This new learning scheme employs adaptive learning rate that is determined by input–output data. Also, stable learning algorithms for two common methods are proposed based on Lyapunov stability theory and some constraints are obtained.     

Stability analysis in identification of interval type‐2 adaptive neuro‐fuzzy inference system: Contribution to a novel Lyapunov function

In recent years, control research has strongly highlighted the issue of training stability in the identification of non‐linear systems. This paper investigates the stability analysis of an interval type‐2 adaptive neuro‐fuzzy inference system (IT2ANFIS) as an identifier through a novel Lyapunov function. In so doing, stability analysis is initially conducted on the IT2ANFIS identifier, while performing the online training of both the antecedent and the consequent parameters by the gradient descent (GD) algorithm. In addition, the same stability analysis is carried out when the antecedent and the consequent parameters are trained by GD and forgetting factor recursive least square (FRLS) algorithms, respectively (GD + FRLS). A novel Lyapunov function is proposed in this study in order for the identifier stability to attain the required conditions. These conditions determine the permissible boundaries for the covariance matrix and the learning rates at every iteration of the identification procedure. Stability analysis reveals that wide range of learning rates is obtained. Furthermore, simulation results indicate that when the permissible boundaries are selected according to the proposed stability analysis, a stable identification process with appropriate performance is achieved.     

基于模糊c-means与自适应粒子群优化的模糊聚类算法

已有的粒子群模糊聚类算法需要设置粒子群参数并且收敛速度较慢,对此提出一种基于改进粒子群与模糊c-means的模糊聚类算法。首先,使用模糊c-means算法生成一组起始解,提高粒子群演化的方向性;然后,使用改进的自适应粒子群优化方法对数据进行训练与优化,训练过程中自适应地调节粒子群参数;最终,采用模糊c-means算法进行模糊聚类过程。对比实验结果表明,所提方法大幅度提高了计算速度,并获得了较高的聚类性能。     

Integration of PSO and GA for optimum design of fuzzy PID controllers in a pendubot system

In this paper, a novel auto-tuning method is proposed to design fuzzy PID controllers for asymptotical stabilization of a pendubot system. In the proposed method, a fuzzy PID controller is expressed in terms of fuzzy rules, in which the input variables are the error signals and their derivatives, while the output variables are the PID gains. In this manner, the PID gains are adaptive and the fuzzy PID controller has more flexibility and capability than the conventional ones with fixed gains. To tune the fuzzy PID controller simultaneously, an evolutionary learning algorithm integrating particle swarm optimization (PSO) and genetic algorithm (GA) methods is proposed. The simulation results illustrate that the proposed method is indeed more efficient in improving the asymptotical stability of the pendubot system. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

Adaptive fuzzy particle swarm optimization for global optimization of multimodal functions

This paper proposes an adaptive fuzzy PSO (AFPSO) algorithm, based on the standard particle swarm optimization (SPSO) algorithm. The proposed AFPSO utilizes fuzzy set theory to adjust PSO acceleration coefficients adaptively, and is thereby able to improve the accuracy and efficiency of searches. Incorporating this algorithm with quadratic interpolation and crossover operator further enhances the global searching capability to form a new variant, called AFPSO-QI. We compared the proposed AFPSO and its variant AFPSO-QI with SPSO, quadratic interpolation PSO (QIPSO), unified PSO (UPSO), fully informed particle swarm (FIPS), dynamic multi-swarm PSO (DMSPSO), and comprehensive learning PSO (CLPSO) across sixteen benchmark functions. The proposed algorithms performed well when applied to minimization problems for most of the multimodal functions considered.     

An improved fuzzy neural network based on T–S model

An improved fuzzy neural network based on Takagi–Sugeno (T–S) model is proposed in this paper. According to characteristics of samples spatial distribution the number of linguistic values of every input and the means and deviations of corresponding membership functions are determined. So the reasonable fuzzy space partition is got. Further a subtractive clustering algorithm is used to derive cluster centers from samples. With the parameters of linguistic values the cluster centers are fuzzified to get a more concise rule set with importance for every rule. Thus redundant rules in the fuzzy space are deleted. Then antecedent parts of all rules determine how a fuzzification layer and an inference layer connect. Next, weights of the defuzzification layer are initialized by a least square algorithm. After the network is built, a hybrid method combining a gradient descent algorithm and a least square algorithm is applied to tune the parameters in it. Simultaneous, an adaptive learning rate which is identified from input-state stability theory is adopted to insure stability of the network. The improved T–S fuzzy neural network (ITSFNN) has a compact structure, high training speed, good simulation precision, and generalization ability. To evaluate the performance of the ITSFNN, we experiment with two nonlinear examples. A comparative analysis reveals the proposed T–S fuzzy neural network exhibits a higher accuracy and better generalization ability than ordinary T–S fuzzy neural network. Finally, it is applied to predict markup percent of the construction bidding system and has a better prediction capability in comparison to some previous models.     

A neuro-fuzzy inference system through integration of fuzzy logic and extreme learning machines.

This paper investigates the feasibility of applying a relatively novel neural network technique, i.e., extreme learning machine (ELM), to realize a neuro-fuzzy Takagi-Sugeno-Kang (TSK) fuzzy inference system. The proposed method is an improved version of the regular neuro-fuzzy TSK fuzzy inference system. For the proposed method, first, the data that are processed are grouped by the k-means clustering method. The membership of arbitrary input for each fuzzy rule is then derived through an ELM, followed by a normalization method. At the same time, the consequent part of the fuzzy rules is obtained by multiple ELMs. At last, the approximate prediction value is determined by a weight computation scheme. For the ELM-based TSK fuzzy inference system, two extensions are also proposed to improve its accuracy. The proposed methods can avoid the curse of dimensionality that is encountered in backpropagation and hybrid adaptive neuro-fuzzy inference system (ANFIS) methods. Moreover, the proposed methods have a competitive performance in training time and accuracy compared to three ANFIS methods.     

A multi-swarm optimizer based fuzzy modeling approach for dynamic systems processing

Inspired by the phenomenon of symbiosis in natural ecosystems a multi-swarm cooperative particle swarm optimizer (MCPSO) is proposed as a new fuzzy modeling strategy for identification and control of non-linear dynamical systems. In MCPSO, the population consists of one master swarm and several slave swarms. The slave swarms execute particle swarm optimization (PSO) or its variants independently to maintain the diversity of particles, while the particles in the master swarm enhance themselves based on their own knowledge and also the knowledge of the particles in the slave swarms. With four benchmark functions, MCPSO is proved to have better performance than PSO and its variants. MCPSO is then used to automatically design the fuzzy identifier and fuzzy controller for non-linear dynamical systems. The proposed algorithm (MCPSO) is shown to outperform PSO and some other methods in identifying and controlling dynamical systems.     

基于IQPSO-GA优化ANFIS模型的服务器故障预警方法

针对服务器底层部分业务类硬件故障对系统稳定运行的影响,提出一种改进的量子行为粒子群优化(IQPSO)与遗传算法(GA)相结合的混合元启发式优化算法对自适应神经模糊推理系统(ANFIS)参数进行训练,以获得更准确的ANFIS规则进行硬件故障预警的方法。首先,通过分析服务器业务与硬件相关参数之间的映射关系,通过采集的数据集对ANFIS模型进行训练构造预测模型;其次,考虑ANFIS在梯度计算过程中存在容易陷入局部最优值的问题,设计了一种IQPSO算法结合GA中的交叉和变异算子操作混合元启发算法全局搜索ANFIS规则参数;最后,通过一组后处理样本数据集对所提方法有效性和稳定性进行了检验。实验结果表明,该方法可有效预警服务器硬件故障,基于所提混合元启发优化算法获得的ANFIS模型具备更快的收敛速度和更高的全局搜索精度,与传统ANFIS模型相比泛化精度提高了47%以上。     

A type-2 neural fuzzy system learned through type-1 fuzzy rules and its FPGA-based hardware implementation

This paper first proposes a type-2 neural fuzzy system (NFS) learned through its type-1 counterpart (T2NFS-T1) and then implements the built IT2NFS-T1 in a field-programmable gate array (FPGA) chip. The antecedent part of each fuzzy rule in the T2NFS-T1 uses interval type-2 fuzzy sets, while the consequent part uses a Takagi-Sugeno-Kang (TSK) type with interval combination weights. The T2NFS-T1 uses a simplified type-reduction operation to reduce system training time and hardware implementation cost. Given a training data set, a TSK type-1 NFS is first learned through structure and parameter learning. The built type-1 fuzzy logic system (FLS) is then extended to a type-2 FLS, where highly overlapped type-1 fuzzy sets are merged into interval type-2 fuzzy sets to reduce the total number of fuzzy sets. Finally, the rule consequent and antecedent parameters in the T2NFS-T1 are tuned using a hybrid of the gradient descent and rule-ordered recursive least square (RLS) algorithms. Simulation results and comparisons with various type-1 and type-2 FLSs verify the effectiveness and efficiency of the T2NFS-T1 for system modeling and prediction problems. A new hardware circuit using both parallel-processing and pipeline techniques is proposed to implement the learned T2NFS-T1 in an FPGA chip. The T2NFS-T1 chip reduces the hardware implementation cost in comparison to other type-2 fuzzy chips.     

A fuzzy neural network based on fuzzy hierarchy error approach

This paper presents a novel fuzzy neural network which consists of an antecedent network and a consequent network. The antecedent network matches the premises of the fuzzy rules and the consequent network implements the consequences of the rules. In the network learning and training phase, a concise and effective algorithm based on the fuzzy hierarchy error approach is proposed to update the parameters of the network. This algorithm is simple to implement and it does not require as many calculations as some other classic neural network learning algorithms. A model reference adaptive control structure incorporating the proposed fuzzy neural network is studied. Simulation results of a cart-pole balancing system demonstrate the effectiveness of the proposed method     

An efficient impulsive noise cancellation scheme for power-line communication systems using ANFIS and chaotic interleaver

Impulsive noise is one of the main disturbances that damage the data transmission over power-line communication (PLC) systems. This paper presents an adaptive noise cancellation approach based on the adaptive neuro-fuzzy inference system (ANFIS) and a chaotic interleaver, namely ANC-CI-ANFIS scheme for impulsive noise estimation and suppression from the OFDM PLC channel. The ANFIS is based on a hybrid learning algorithm to identify parameters of Sugeno-type fuzzy inference system. Accordingly, fuzzy membership function parameters are trained using a combination of both least-square and back propagation gradient descent algorithms to emulate a given training data set. Furthermore, transmitted data are managed with a chaotic interleaver to secure data transmission and give more robustness against impulsive bursts. Simulation results are carried out on an OFDM PLC transmission chain compatible with the HomePlug AV standard under different impulsive noise scenarios. The results demonstrated the scheme's ability to detect and remove the impulsive noise from the PLC channel while keeping a high security level by using the chaotic interleaver. The major advantage of this system is its ease of implementation and faster convergence rate.     

基于自适应神经模糊推理系统的噪声消除方法

针对信号处理领域噪声消除的实际问题,提出了一种基于模糊推理的自适应神经网络控制方法.通过自适应神经模糊推理系统(ANFIS)对非线性系统的结构和参数进行辨识与自学习,采用混合学习算法,对前向参数和结论参数分别辨识,在提高精度的同时可加快训练收敛的速度,使控制系统具有良好动静态性和鲁棒性,实现了消除通信系统中噪声的目标,最后对基于ANFIS的噪声消除系统进行了建模和仿真,并与自适应神经网络滤波方法的结果对比,其结果证明了该方法的有效性.     

Unsupervised adaptive neural-fuzzy inference system for solving differential equations

There has been a growing interest in combining both neural network and fuzzy system, and as a result, neuro-fuzzy computing techniques have been evolved. ANFIS (adaptive network-based fuzzy inference system) model combined the neural network adaptive capabilities and the fuzzy logic qualitative approach. In this paper, a novel structure of unsupervised ANFIS is presented to solve differential equations. The presented solution of differential equation consists of two parts; the first part satisfies the initial/boundary condition and has no adjustable parameter whereas the second part is an ANFIS which has no effect on initial/boundary conditions and its adjustable parameters are the weights of ANFIS. The algorithm is applied to solve differential equations and the results demonstrate its accuracy and convince us to use ANFIS in solving various differential equations.     

An input-output clustering approach to the synthesis of ANFIS networks

A useful neural network paradigm for the solution of function approximation problems is represented by adaptive neuro-fuzzy inference systems (ANFIS). Data driven procedures for the synthesis of ANFIS networks are typically based on clustering a training set of numerical samples of the unknown function to be approximated. Some serious drawbacks often affect the clustering algorithms adopted in this context, according to the particular data space where they are applied. To overcome such problems, we propose a new ANFIS synthesis procedure where clustering is applied in the joint input-output data space. Using this approach, it is possible to determine the consequent part of Sugeno first-order rules and therefore the hyperplanes characterizing the local structure of the function to be approximated. Successively, the fuzzy antecedent part of each rule is determined using a particular fuzzy min-max classifier, which is based on the adaptive resolution mechanism. The generalization capability of the resulting ANFIS architecture is optimized using a constructive procedure for the automatic determination of the optimal number of rules. Simulation tests and comparisons with respect to other neuro-fuzzy techniques are discussed in the paper, in order to assess the efficiency of the proposed approach.     

一种基于混沌遗传与粒子群混合优化的过程神经网络训练算法

针对时变输入/输出过程神经网络的训练问题,提出一种基于混沌遗传与带有动态惯性因子的粒子群优化相结合的学习方法。综合利用粒子群算法的经验记忆、信息共享和混沌遗传算法的混沌轨道遍历搜索性质,基于PNN训练目标函数,构建两种算法相混合的进化寻优机制,通过适应度评估和优化效率分析自适应调节混沌遗传与粒子群算法的切换,实现网络参数在可行解空间的全局优化求解。实验结果表明,该算法较大提高了PNN的训练效率。     

A hybrid simulation-adaptive network based fuzzy inference system for improvement of electricity consumption estimation

This paper presents a hybrid adaptive network based fuzzy inference system (ANFIS), computer simulation and time series algorithm to estimate and predict electricity consumption estimation. The difficulty with electricity consumption estimation modeling approach such as time series is the reason for proposing the hybrid approach of this study. The algorithm is ideal for uncertain, ambiguous and complex estimation and forecasting. Computer simulation is developed to generate random variables for monthly electricity consumption. Various structures of ANFIS are examined and the preferred model is selected for estimation by the proposed algorithm. Finally, the preferred ANFIS and time series models are selected by Granger–Newbold test. Monthly electricity consumption in Iran from 1995 to 2005 is considered as the case of this study. The superiority of the proposed algorithm is shown by comparing its results with genetic algorithm (GA) and artificial neural network (ANN). This is the first study that uses a hybrid ANFIS computer simulation for improvement of electricity consumption estimation.     

Rough set and PSO-based ANFIS approaches to modeling customer satisfaction for affective product design

Facing fierce competition in marketplaces, companies try to determine the optimal settings of design attribute of new products from which the best customer satisfaction can be obtained. To determine the settings, customer satisfaction models relating affective responses of customers to design attributes have to be first developed. Adaptive neuro-fuzzy inference systems (ANFIS) was attempted in previous research and shown to be an effective approach to address the fuzziness of survey data and nonlinearity in modeling customer satisfaction for affective design. However, ANFIS is incapable of modeling the relationships that involve a number of inputs which may cause the failure of the training process of ANFIS and lead to the ‘out of memory’ error. To overcome the limitation, in this paper, rough set (RS) and particle swarm optimization (PSO) based-ANFIS approaches are proposed to model customer satisfaction for affective design and further improve the modeling accuracy. In the approaches, the RS theory is adopted to extract significant design attributes as the inputs of ANFIS and PSO is employed to determine the parameter settings of an ANFIS from which explicit customer satisfaction models with better modeling accuracy can be generated. A case study of affective design of mobile phones is used to illustrate the proposed approaches. The modeling results based on the proposed approaches are compared with those based on ANFIS, fuzzy least-squares regression (FLSR), fuzzy regression (FR), and genetic programming-based fuzzy regression (GP-FR). Results of the training and validation tests show that the proposed approaches perform better than the others in terms of training and validation errors.     

基于PSO和BP复合算法的模糊神经网络控制器

为了克服单独应用粒子群算法（PSO）或BP算法训练模糊神经网络控制器参数时存在的缺陷,提出了一种训练模糊神经网络参数的PSO＋BP算法。该算法将二者相结合,即在PSO算法中加入一个BP算子,以充分利用PSO算法的全局寻优能力和BP算法的局部搜索能力,从而更有效地提高其收敛速度、训练效率和提高该模糊神经网络控制器的控制效果。最后的仿真实验结果验证了该基于PSO＋BP复合算法的模糊神经网络控制器的有效性和可行性。     

Adaptive fuzzy identification and predictive control for industrial processes

This paper proposes a method for adaptive identification and control for industrial applications. The learning of a T–S fuzzy model is performed from input/output data to approximate unknown nonlinear processes by a hierarchical genetic algorithm (HGA). The HGA approach is composed by five hierarchical levels where the following parameters of the T–S fuzzy system are learned: input variables and their respective time delays, antecedent fuzzy sets, consequent parameters, and fuzzy rules. In order to reduce the computational cost and increase the algorithm’s performance an initialization method is applied on HGA. To deal with nonlinear plants and time-varying processes, the T–S fuzzy model is adapted online to maintain the quality of the identification/control. The identification methodology is proposed for two application problems: (1) the design of data-driven soft sensors, and (2) the learning of a model for the Generalized predictive control (GPC) algorithm. The integration of the proposed adaptive identification method with the GPC results in an effective adaptive predictive fuzzy control methodology. To validate and demonstrate the performance and effectiveness of the proposed methodologies, they are applied on identification of a model for the estimation of the flour concentration in the effluent of a real-world wastewater treatment system; and on control of a simulated continuous stirred tank reactor (CSTR) and on a real experimental setup composed of two coupled DC motors. The results are presented, showing that the developed evolving T–S fuzzy model can identify the nonlinear systems satisfactorily and it can be used successfully as a prediction model of the process for the GPC controller.