Identification and control of dynamic systems using recurrent fuzzyneural networks

Proposes a recurrent fuzzy neural network (RFNN) structure for identifying and controlling nonlinear dynamic systems. The RFNN is inherently a recurrent multilayered connectionist network for realizing fuzzy inference using dynamic fuzzy rules. Temporal relations are embedded in the network by adding feedback connections in the second layer of the fuzzy neural network (FNN). The RFNN expands the basic ability of the FNN to cope with temporal problems. In addition, results for the FNN-fuzzy inference engine, universal approximation, and convergence analysis are extended to the RFNN. For the control problem, we present the direct and indirect adaptive control approaches using the RFNN. Based on the Lyapunov stability approach, rigorous proofs are presented to guarantee the convergence of the RFNN by choosing appropriate learning rates. Finally, the RFNN is applied in several simulations (time series prediction, identification, and control of nonlinear systems). The results confirm the effectiveness of the RFNN     

A TSK-type recurrent fuzzy network for dynamic systems processingby neural network and genetic algorithms

In this paper, a TSK-type recurrent fuzzy network (TRFN) structure is proposed. The proposal calls for the design of TRFN by either neural network or genetic algorithms depending on the learning environment. A recurrent fuzzy network is described which develops from a series of recurrent fuzzy if-then rules with TSK-type consequent parts. The recurrent property comes from feeding the internal variables, derived from fuzzy firing strengths, back to both the network input and output layers. In this configuration, each internal variable is responsible for memorizing the temporal history of its corresponding fuzzy rule. The internal variable is also combined with external input variables in each rule's consequence, which shows an increase in network learning ability. TRFN design under different learning environments is next advanced. For problems where supervised training data is directly available, TRFN with supervised learning (TRFN-S) is proposed, and a neural network (NN) learning approach is adopted for TRFN-S design. An online learning algorithm with concurrent structure and parameter learning is proposed. With flexibility of partition in the precondition part, and outcome of TSK-type, the TRFN-S displays both small network size and high learning accuracy. For problems where gradient information for NN learning is costly to obtain or unavailable, like reinforcement learning, TRFN with Genetic learning (TRFN-G) is put forward. The precondition parts of TRFN-G are also partitioned in a flexible way, and all free parameters are designed concurrently by genetic algorithm. Owing to the well-designed network structure of TRFN, TRFN-G, like TRFN-S, is characterized by high learning accuracy. To demonstrate the superior properties of TRFN, TRFN-S is applied to dynamic system identification and TRFN-G to dynamic system control. By comparing the results to other types of recurrent networks and design configurations, the efficiency of TRFN is verified     

An adaptive neural fuzzy filter and its applications

A new kind of nonlinear adaptive filter, the adaptive neural fuzzy filter (ANFF), based upon a neural network's learning ability and fuzzy if-then rule structure, is proposed in this paper. The ANFF is inherently a feedforward multilayered connectionist network which can learn by itself according to numerical training data or expert knowledge represented by fuzzy if-then rules. The adaptation here includes the construction of fuzzy if-then rules (structure learning), and the tuning of the free parameters of membership functions (parameter learning). In the structure learning phase, fuzzy rules are found based on the matching of input-output clusters. In the parameter learning phase, a backpropagation-like adaptation algorithm is developed to minimize the output error. There are no hidden nodes (i.e., no membership functions and fuzzy rules) initially, and both the structure learning and parameter learning are performed concurrently as the adaptation proceeds. However, if some linguistic information about the design of the filter is available, such knowledge can be put into the ANFF to form an initial structure with hidden nodes. Two major advantages of the ANFF can thus be seen: 1) a priori knowledge can be incorporated into the ANFF which makes the fusion of numerical data and linguistic information in the filter possible; and 2) no predetermination, like the number of hidden nodes, must be given, since the ANFF can find its optimal structure and parameters automatically     

A recurrent fuzzy-neural model for dynamic system identification

This paper presents a fuzzy modeling approach for identification of dynamic systems. In particular, a new fuzzy model, the Dynamic Fuzzy Neural Network (DFNN), consisting of recurrent TSK rules, is developed. The premise and defuzzification parts are static while the consequent parts of the fuzzy rules are recurrent neural networks with internal feedback and time delay synapses. The network is trained by means of a novel learning algorithm, named Dynamic-Fuzzy Neural Constrained Optimization Method (D-FUNCOM), based on the concept of constrained optimization. The proposed algorithm is general since it can be applied to locally as well as fully recurrent networks, regardless of their structures. An adaptation mechanism of the maximum parameter change is presented as well. The proposed dynamic model, equipped with the learning algorithm, is applied to several temporal problems, including modeling of a NARMA process and the noise cancellation problem. Performance comparisons are conducted with a series of static and dynamic systems and some existing recurrent fuzzy models. Simulation results show that DFNN compares favorably with its competing rivals and thus it can be considered for efficient system identification.     

A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks

In this paper, we present a fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks (FAOS-PFNN), where a novel structure learning algorithm incorporating a pruning strategy into new growth criteria is developed. The proposed growing procedure without pruning not only speeds up the online learning process but also facilitates a more parsimonious fuzzy neural network while achieving comparable performance and accuracy by virtue of the growing and pruning strategy. The FAOS-PFNN starts with no hidden neurons and parsimoniously generates new hidden units according to the proposed growth criteria as learning proceeds. In the parameter learning phase, all the free parameters of hidden units, regardless of whether they are newly created or originally existing, are updated by the extended Kalman filter (EKF) method. The effectiveness and superiority of the FAOS-PFNN paradigm is compared with other popular approaches like resource allocation network (RAN), RAN via the extended Kalman filter (RANEKF), minimal resource allocation network (MRAN), adaptive-network-based fuzzy inference system (ANFIS), orthogonal least squares (OLS), RBF-AFS, dynamic fuzzy neural networks (DFNN), generalized DFNN (GDFNN), generalized GAP-RBF (GGAP-RBF), online sequential extreme learning machine (OS-ELM) and self-organizing fuzzy neural network (SOFNN) on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification, chaotic time-series prediction and real-world regression problems. Simulation results demonstrate that the proposed FAOS-PFNN algorithm can achieve faster learning speed and more compact network structure with comparably high accuracy of approximation and generalization.     

脉冲GTAW熔池动态过程模糊神经网络建模与控制

展示了模糊推理与神经网络结合在脉冲GTAW熔池动态过程智能控制中的应用研究 结果.建立了脉冲GTAW平板对接动态过程特征:正反面熔池的最大宽度、长度与面积等参数 的神经网络模型,基于实验数据采用模糊辨识方法提取焊接过程的模糊控制规则,进而设计了 具有自学习适应能力的模糊神经网络控制器.建立了脉冲GTAW熔池动态过程智能控制系统, 焊接实验验证了所设计的模糊神经网络控制器具有智能控制效果.     

Prediction and identification using wavelet-based recurrent fuzzy neural networks

This paper presents a wavelet-based recurrent fuzzy neural network (WRFNN) for prediction and identification of nonlinear dynamic systems. The proposed WRFNN model combines the traditional Takagi-Sugeno-Kang (TSK) fuzzy model and the wavelet neural networks (WNN). This paper adopts the nonorthogonal and compactly supported functions as wavelet neural network bases. Temporal relations embedded in the network are caused by adding some feedback connections representing the memory units into the second layer of the feedforward wavelet-based fuzzy neural networks (WFNN). An online learning algorithm, which consists of structure learning and parameter learning, is also presented. The structure learning depends on the degree measure to obtain the number of fuzzy rules and wavelet functions. Meanwhile, the parameter learning is based on the gradient descent method for adjusting the shape of the membership function and the connection weights of WNN. Finally, computer simulations have demonstrated that the proposed WRFNN model requires fewer adjustable parameters and obtains a smaller rms error than other methods.     

A recurrent neural fuzzy network for word boundary detection invariable noise-level environments

This paper discusses the problem of automatic word boundary detection in the presence of variable-level background noise. Commonly used robust word boundary detection algorithms always assume that the background noise level is fixed. In fact, the background noise level may vary during the procedure of recording. This is the major reason that most robust word boundary detection algorithms cannot work well in the condition of variable background noise level. In order to solve this problem, we first propose a refined time-frequency (RTF) parameter for extracting both the time and frequency features of noisy speech signals. The RTF parameter extends the (time-frequency) TF parameter proposed by Junqua et al. from single band to multiband spectrum analysis, where the frequency bands help to make the distinction between speech signal and noise clear. The RTF parameter can extract useful frequency information. Based on this RTF parameter, we further propose a new word boundary detection algorithm by using a recurrent self-organizing neural fuzzy inference network (RSONFIN). Since RSONPIN can process the temporal relations, the proposed RTF-based RSONFIN algorithm can find the variation of the background noise level and detect correct word boundaries in the condition of variable background noise level. As compared to normal neural networks, the RSONFIN can always find itself an economic network size with high-learning speed. Due to the self-learning ability of RSONFIN, this RTF-based RSONFIN algorithm avoids the need for empirically determining ambiguous decision rules in normal word boundary detection algorithms. Experimental results show that this new algorithm achieves higher recognition rate than the TF-based algorithm which has been shown to outperform several commonly used word boundary detection algorithms by about 12% in variable background noise level condition, It also reduces the recognition error rate due to endpoint detection to about 23%, compared to an average of 47% obtained by the TF-based algorithm in the same condition.     

An online self-constructing neural fuzzy inference network and itsapplications

A self-constructing neural fuzzy inference network (SONFIN) with online learning ability is proposed in this paper. The SONFIN is inherently a modified Takagi-Sugeno-Kang (TSK)-type fuzzy rule-based model possessing neural network learning ability. There are no rules initially in the SONFIN. They are created and adapted as online learning proceeds via simultaneous structure and parameter identification. In the structure identification of the precondition part, the input space is partitioned in a flexible way according to an aligned clustering-based algorithm. As to the structure identification of the consequent part, only a singleton value selected by a clustering method is assigned to each rule initially. Afterwards, some additional significant terms selected via a projection-based correlation measure for each rule will be added to the consequent part incrementally as learning proceeds. The combined precondition and consequent structure identification scheme can set up an economic and dynamically growing network, a main feature of the SONFIN. In the parameter identification, the consequent parameters are tuned optimally by either least mean squares or recursive least squares algorithms and the precondition parameters are tuned by a backpropagation algorithm. To enhance the knowledge representation ability of the SONFIN, a linear transformation for each input variable can be incorporated into the network so that much fewer rules are needed or higher accuracy can be achieved     

A Recurrent Self-Evolving Interval Type-2 Fuzzy Neural Network for Dynamic System Processing

This paper proposes a recurrent self-evolving interval type-2 fuzzy neural network (RSEIT2FNN) for dynamic system processing. An RSEIT2FNN incorporates type-2 fuzzy sets in a recurrent neural fuzzy system in order to increase the noise resistance of a system. The antecedent parts in each recurrent fuzzy rule in the RSEIT2FNN are interval type-2 fuzzy sets, and the consequent part is of the Takagi-Sugeno-Kang (TSK) type with interval weights. The antecedent part of RSEIT2FNN forms a local internal feedback loop by feeding the rule firing strength of each rule back to itself. The TSK-type consequent part is a linear model of exogenous inputs. The RSEIT2FNN initially contains no rules; all rules are learned online via structure and parameter learning. The structure learning uses online type-2 fuzzy clustering. For the parameter learning, the consequent part parameters are tuned by a rule-ordered Kalman filter algorithm to improve learning performance. The antecedent type-2 fuzzy sets and internal feedback loop weights are learned by a gradient descent algorithm. The RSEIT2FNN is applied to simulations of dynamic system identifications and chaotic signal prediction under both noise-free and noisy conditions. Comparisons with type-1 recurrent fuzzy neural networks validate the performance of the RSEIT2FNN.     

A context layered locally recurrent neural network for dynamic system identification

This paper puts forward a novel recurrent neural network (RNN), referred to as the context layered locally recurrent neural network (CLLRNN) for dynamic system identification. The CLLRNN is a dynamic neural network which appears in effective in the input–output identification of both linear and nonlinear dynamic systems. The CLLRNN is composed of one input layer, one or more hidden layers, one output layer, and also one context layer improving the ability of the network to capture the linear characteristics of the system being identified. Dynamic memory is provided by means of feedback connections from nodes in the first hidden layer to nodes in the context layer and in case of being two or more hidden layers, from nodes in a hidden layer to nodes in the preceding hidden layer. In addition to feedback connections, there are self-recurrent connections in all nodes of the context and hidden layers. A dynamic backpropagation algorithm with adaptive learning rate is derived to train the CLLRNN. To demonstrate the superior properties of the proposed architecture, it is applied to identify not only linear but also nonlinear dynamic systems. The efficiency of the proposed architecture is demonstrated by comparing the results to some existing recurrent networks and design configurations. In addition, performance of the CLLRNN is analyzed through an experimental application to a dc motor connected to a load to show practicability and effectiveness of the proposed neural network. Results of the experimental application are presented to make a quantitative comparison with an existing recurrent network in the literature.     

An stable online clustering fuzzy neural network for nonlinear system identification

In this paper, we propose a online clustering fuzzy neural network. The proposed neural fuzzy network uses the online clustering to train the structure, the gradient to train the parameters of the hidden layer, and the Kalman filter algorithm to train the parameters of the output layer. In our algorithm, learning structure and parameter learning are updated at the same time, we do not make difference in structure learning and parameter learning. The center of each rule is updated to obtain the center is near to the incoming data in each iteration. In this way, it does not need to generate a new rule in each iteration, i.e., it neither generates many rules nor need to prune the rules. We prove the stability of the algorithm.     

基于修剪策略的D-FNN直接逆控制算法研究

神经网络是模拟人脑结构,它具有大规模并行及分布式信息处理能力,但是不能处理和描述模糊信息.模糊系统具有推理过程容易理解,但它很难实现自适应学习的功能.如果结合神经网络与模糊系统,可以取长补短.基于此,本文提出了一种新型动态模糊神经网络（Dynamic fuzzy neural network,D-FNN）学习算法.因为它具有结构和参数同时调整且学习速度快等优点,所以既可以在模糊逻辑系统中包含低级的神经网络学习和计算功能,也可以为神经网络提供高级的类似人的思维和推理的模糊逻辑系统.此外,本文还开发了生物医学工程应用算法程序,针对药物注射系统的直接逆控制案例进行了仿真,结果表明:D-FNN具有实时学习和控制能力强、参数估计和结构辨识同时进行等优点.     

一种新的基于神经模糊推理网络的复杂系统模糊辨识方法

针对基于输入输出数据的复杂系统的模糊辨识问题,提出了一种新的神经模糊推理网络及相应的学习算法．学习算法被应用于系统的结构辨识与参数辨识．在结构辨识阶段,介绍了一种新的直接从输入输出数据中抽取和优化模糊规则的学习算法;在参数辨识阶段,提出和推导了一种非监督学习和监督学习相结合的混合式学习算法,实现模糊隶属函数的初步调整和优化．仿真结果表明,本文的方法可以同时满足对辨识精度、收敛速度、可读性和规则数的要求．     

Adaptive fuzzy inference neural network

An adaptive fuzzy inference neural network (AFINN) is proposed in this paper. It has self-construction ability, parameter estimation ability and rule extraction ability. The structure of AFINN is formed by the following four phases: (1) initial rule creation, (2) selection of important input elements, (3) identification of the network structure and (4) parameter estimation using LMS (least-mean square) algorithm. When the number of input dimension is large, the conventional fuzzy systems often cannot handle the task correctly because the degree of each rule becomes too small. AFINN solves such a problem by modification of the learning and inference algorithm.     

特征扩展的随机向量函数链神经网络

基于宽度学习的动态模糊推理系统(broad-learning-based dynamic fuzzy inference system , BL-DFIS)能自动构建出精简的模糊规则并获得良好的分类性能. 然而, 当遇到大型复杂的数据集时, BL-DFIS因会使用较多模糊规则来试图达到令人满意的识别精度, 从而对其可解释性造成了不利影响. 对此, 提出一种兼顾分类性能和可解释性的模糊神经网络, 将其称为特征扩展的随机向量函数链神经网络(FA-RVFLNN). 在该网络中, 一个以原始数据为输入的RVFLNN被作为主体结构, BL-DFIS则用作性能补充, 这意味着FA-RVFLNN包含具有性能增强作用的直接链接. 由于主体结构的增强节点使用Sigmoid激活函数, 因此, 其推理过程可借助一种模糊逻辑算子(I-OR)来解释. 而且, 具有明确含义的原始输入数据也有助于解释主体结构的推理规则. 在直接链接的支撑下, FA-RVFLNN可利用增强节点、特征节点和模糊节点学到更丰富的有用信息. 实验表明: FA-RVFLNN既减缓了主体结构RVFLNN中过多增强节点带来的“规则爆炸”问题, 也提高了性能补充结构BL-DFIS的可解释性(平均模糊规则数降低了50%左右), 在泛化性能和网络规模上仍具有竞争力.     

A novel compensation-based recurrent fuzzy neural network and its learning algorithm

Based on detailed study on several kinds of fuzzy neural networks, we propose a novel compensation-based recurrent fuzzy neural network (CRFNN) by adding recurrent element and compensatory element to the conventional fuzzy neural network. Then, we propose a sequential learning method for the structure identification of the CRFNN in order to confirm the fuzzy rules and their correlative parameters effectively. Furthermore, we improve the BP algorithm based on the characteristics of the proposed CRFNN to train the network. By modeling the typical nonlinear systems, we draw the conclusion that the proposed CRFNN has excellent dynamic response and strong learning ability. Supported by the National High-Tech Research and Development Program of China (Grant No. 2006AA05A107) and Special Fund of Jiangsu Province for Technology Transfer (Grant No. BA2007008)     

Fuzzy wavelet neural network based on fuzzy clustering and gradient techniques for time series prediction

This paper presents the development of fuzzy wavelet neural network system for time series prediction that combines the advantages of fuzzy systems and wavelet neural network. The structure of fuzzy wavelet neural network (FWNN) is proposed, and its learning algorithm is derived. The proposed network is constructed on the base of a set of TSK fuzzy rules that includes a wavelet function in the consequent part of each rule. A fuzzy c-means clustering algorithm is implemented to generate the rules, that is the structure of FWNN prediction model, automatically, and the gradient-learning algorithm is used for parameter identification. The use of fuzzy c-means clustering algorithm with the gradient algorithm allows to improve convergence of learning algorithm. FWNN is used for modeling and prediction of complex time series and prediction of foreign-exchange rates. Exchange rates are dynamic process that changes every day and have high-order nonlinearity. The statistical data for the last 2 years are used for the development of FWNN prediction model. Effectiveness of the proposed system is evaluated with the results obtained from the simulation of FWNN-based systems and with the comparative simulation results of previous related models.     

Fault Diagnosis Based on the Fuzzy‐Recurrent Neural Network

A fuzzy‐recurrent neural network (FRNN) has been constructed by adding some feedback connections to a feedforward fuzzy neural network (FNN). The FRNN expands the modeling ability of a FNN in order to deal with temporal problems. A basic concept of the FRNN is first to use process or expert knowledge, including appropriate fuzzy logic rules and membership functions, to construct an initial structure and to then use parameter‐learning algorithms to fine‐tune the membership functions and other parameters. Its recurrent property makes it suitable for dealing with temporal problems, such as on‐line fault diagnosis. In addition, it also provides human‐understandable meaning to the normal feedforward multilayer neural network, in which the internal units are always opaque to users. In a word, the trained FRNN has good interpreting ability and one‐step‐ahead predicting ability. To demonstrate the performance of the FRNN in diagnosis, a comparison is made with a conventional feedforward network. The efficiency of the FRNN is verified by the results.     

Supervisory recurrent fuzzy neural network control of wing rock for slender delta wings

Wing rock is a highly nonlinear phenomenon in which an aircraft undergoes limit cycle roll oscillations at high angles of attack. In this paper, a supervisory recurrent fuzzy neural network control (SRFNNC) system is developed to control the wing rock system. This SRFNNC system is comprised of a recurrent fuzzy neural network (RFNN) controller and a supervisory controller. The RFNN controller is investigated to mimic an ideal controller and the supervisory controller is designed to compensate for the approximation error between the RFNN controller and the ideal controller. The RFNN is inherently a recurrent multilayered neural network for realizing fuzzy inference using dynamic fuzzy rules. Moreover, an on-line parameter training methodology, using the gradient descent method and the Lyapunov stability theorem, is proposed to increase the learning capability. Finally, a comparison between the sliding-mode control, the fuzzy sliding control and the proposed SRFNNC of a wing rock system is presented to illustrate the effectiveness of the SRFNNC system. Simulation results demonstrate that the proposed design method can achieve favorable control performance for the wing rock system without the knowledge of system dynamic functions.