Adaptive Fuzzy Neural Networks as identifiers of discrete-time nonlinear dynamic systems

An adaptive supervised learning scheme is proposed in this paper for training Fuzzy Neural Networks (FNN) to identify discrete-time nonlinear dynamical systems. The FNN constructs are neural-network-based connectionist models consisting of several layers that are used to implement the functions of a fuzzy logic system. The fuzzy rule base considered here consists of Takagi-Sugeno IF-THEN rules, where the rule outputs are realized as linear polynomials of the input components. The FNN connectionist model is functionally partitioned into three separate parts, namely, the premise part, which provides the truth values of the rule preconditional statements, the consequent part providing the rule outputs, and the defuzzification part computing the final output of the FNN construct. The proposed learning scheme is a two-stage training algorithm that performs both structure and parameter learning, simultaneously. First, the structure learning task determines the proper fuzzy input partitions and the respective precondition matching, and is carried out by means of the rule base adaptation mechanism. The rule base adaptation mechanism is a self-organizing procedure which progressively generates the proper fuzzy rule base, during training, according to the operating conditions. Having completed the structure learning stage, the parameter learning is applied using the back-propagation algorithm, with the objective to adjust the premise/consequent parameters of the FNN so that the desired input/output representation is captured to an acceptable degree of accuracy. The structure/parameter training algorithm exhibits good learning and generalization capabilities as demonstrated via a series of simulation studies. Comparisons with conventional multilayer neural networks indicate the effectiveness of the proposed scheme.     

A Comparison of Experimental Results with an Evolution Strategy and Competitive Neural Networks for Near Real-Time Color Quantization of Image Sequences

Color quantization of image sequences is a case of non-stationary clustering problem. The approach we adopt to deal with this kind of problems is to propose adaptive algorithms to compute the cluster representatives. We have studied the application of Competitive Neural Networks and Evolution Strategies to the one-pass adaptive solution of this problem. One-pass adaptation is imposed by the near real-time constraint that we try to achieve. In this paper we propose a simple and effective evolution strategy for this task. Two kinds of competitive neural networks are also applied. Experimental results show that the proposed evolution strategy can produce results comparable to that of competitive neural networks.     

一种基于混合建模的模块化神经网络控制方法

针对传统神经网络用于复杂过程系统的控制时难于收敛的问题,文章提出了基于混合建模的模块化的神经网络模型。采取运行机理建模和神经网络建模相结合的方式,把输入样本空间进行划分,实现基于混合专家网络的建模。试验结果表明,对大型燃煤锅炉供热系统,文章提出的方法可以较好地提高供热系统的稳定性和供热质量。     

Falcon: neural fuzzy control and decision systems using FKP and PFKP clustering algorithms.

Neural fuzzy networks proposed in the literature can be broadly classified into two groups. The first group is essentially fuzzy systems with self-tuning capabilities and requires an initial rule base to be specified prior to training. The second group of neural fuzzy networks, on the other hand, is able to automatically formulate the fuzzy rules from the numerical training data. Examples are the Falcon-ART, and the POPFNN family of networks. A cluster analysis is first performed on the training data and the fuzzy rules are subsequently derived through the proper connections of these computed clusters. This correspondence proposes two new networks: Falcon-FKP and Falcon-PFKP. They are extensions of the Falcon-ART network, and aimed to overcome the shortcomings faced by the Falcon-ART network itself, i.e., poor classification ability when the classes of input data are very similar to each other, termination of training cycle depends heavily on a preset error parameter, the fuzzy rule base of the Falcon-ART network may not be consistent Nauck, there is no control over the number of fuzzy rules generated, and learning efficiency may deteriorate by using complementarily coded training data. These deficiencies are essentially inherent to the fuzzy ART, clustering technique employed by the Falcon-ART network. Hence, two clustering techniques--Fuzzy Kohonen Partitioning (FKP) and its pseudo variant PFKP, are synthesized with the basic Falcon structure to compute the fuzzy sets and to automatically derive the fuzzy rules from the training data. The resultant neural fuzzy networks are Falcon-FKP and Falcon-PFKP, respectively. These two proposed networks have a lean and efficient training algorithm and consistent fuzzy rule bases. Extensive simulations are conducted using the two networks and their performances are encouraging when benchmarked against other neural and neural fuzzy systems.     

Fuzzy function approximation with ellipsoidal rules

A fuzzy rule can have the shape of an ellipsoid in the input-output state spare of a system. Then an additive fuzzy system approximates a function by covering its graph with ellipsoidal rule patches. It averages rule patches that overlap. The best fuzzy rules cover the extrema or bumps in the function. Neural or statistical clustering systems can approximate the unknown fuzzy rules from training data. Neural systems can then both tune these rules and add rules to improve the function approximation. We use a hybrid neural system that combines unsupervised and supervised learning to find and tune the rules in the form of ellipsoids. Unsupervised competitive learning finds the first-order and second-order statistics of clusters in the training data. The covariance matrix of each cluster gives an ellipsoid centered at the vector or centroid of the data cluster. The supervised neural system learns with gradient descent. It locally minimizes the mean-squared error of the fuzzy function approximation. In the hybrid system unsupervised learning initializes the gradient descent. The hybrid system tends to give a more accurate function approximation than does the lone unsupervised or supervised system. We found a closed-form model for the optimal rules when only the centroids of the ellipsoids change. We used numerical techniques to find the optimal rules in the general case.     

Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems

Artificial neural networks often achieve high classification accuracy rates, but they are considered as black boxes due to their lack of explanation capability. This paper proposes the new rule extraction algorithm RxREN to overcome this drawback. In pedagogical approach the proposed algorithm extracts the rules from trained neural networks for datasets with mixed mode attributes. The algorithm relies on reverse engineering technique to prune the insignificant input neurons and to discover the technological principles of each significant input neuron of neural network in classification. The novelty of this algorithm lies in the simplicity of the extracted rules and conditions in rule are involving both discrete and continuous mode of attributes. Experimentation using six different real datasets namely iris, wbc, hepatitis, pid, ionosphere and creditg show that the proposed algorithm is quite efficient in extracting smallest set of rules with high classification accuracy than those generated by other neural network rule extraction methods.     

BP神经网络及其在数据分类中的应用

BP神经网络模型是一种典型的前向型神经网络,具有良好的自学习、自适应、联想记忆、并行处理和非线形转换的能力,是目前应用最为广泛的一种神经网络模型。本文介绍了BP神经网络的实现以及其在数据挖掘分类方面的应用。     

Symbolic interpretation of artificial neural networks

Hybrid intelligent systems that combine knowledge-based and artificial neural network systems typically have four phases, involving domain knowledge representation, mapping of this knowledge into an initial connectionist architecture, network training and rule extraction, respectively. The final phase is important because it can provide a trained connectionist architecture with explanation power and validate its output decisions. Moreover, it can be used to refine and maintain the initial knowledge acquired from domain experts. In this paper, we present three rule extraction techniques. The first technique extracts a set of binary rules from any type of neural network. The other two techniques are specific to feedforward networks, with a single hidden layer of sigmoidal units. Technique 2 extracts partial rules that represent the most important embedded knowledge with an adjustable level of detail, while the third technique provides a more comprehensive and universal approach. A rule-evaluation technique, which orders extracted rules based on three performance measures, is then proposed. The three techniques area applied to the iris and breast cancer data sets. The extracted rules are evaluated qualitatively and quantitatively, and are compared with those obtained by other approaches     

神经网络规则抽取评估方法

神经网络规则抽取是神经网络领域的一个重要方向,但是对抽取的规则评估算法却很少.针对这一问题,提出了神经网络抽取规则评估方法.首先证明所有的规则形式都可以统一为区间的形式,然后在区间算法的基础上提出规则评估方法.评估的标准有四个:覆盖性、准确性、矛盾性,以及冗余性.由于规则的矛盾性和冗余性是规则之间的问题,所以该文仅仅研究规则的覆盖性和准确性,提出了覆盖性判断定理,并提出了覆盖性、准确性判断算法.实例证实了该算法的有效性.     

面向BSP-CNN的短文本情感倾向性分类研究

针对消费短文本评论中的情感倾向性分类问题,提出了一种BSP-CNN混合神经网络模型。模型先使用双向简单循环单元（BiSRU）对数据进行特征表示,再使用逐点卷积神经网络（P-CNN）进一步学习语义特征,并输出情感倾向性分类结果。实验结果表明,与传统的长短期记忆神经网络（LSTM）和卷积神经网络（CNN）相比,BSP-CNN混合神经网络模型有效简化了计算,缩短了运行时间,并且在不同大小和不同文本长度的数据集上均能取得更高的F1值。     

A neural composite dynamic surface control for pure‐feedback systems with unknown control gain signs and full state constraints

This paper investigates a composite neural dynamic surface control (DSC) method for a class of pure‐feedback nonlinear systems in the case of unknown control gain signs and full‐state constraints. Neural networks are utilized to approximate the compound unknown functions, and the approximation errors of neural networks are applied in the design of updated adaptation laws. Comparing the proposed composite approximation method with the conventional ones, a faster and better approximation performance result can be obtained. Combining the composite neural networks approximation with the DSC technique, an improved composite neural adaptive control approach is designed for the considered nonlinear system. Then, together with the Lyapunov stability theory, all the variables of the closed‐loop system are semiglobal uniformly ultimately bounded. The infringements of full state constraints can be avoided in the case of unknown control gain signs as well as unknown disturbances. Finally, two simulation examples show the effectiveness and feasibility of the proposed results.     

Mapping multi-spectral remote sensing images using rule extraction approach

To improve the accurate rate of mapping multi-spectral remote sensing images, in this paper we construct a class of HyperRectangular Composite Neural Networks (HRCNNs), integrating the paradigms of neural networks with the rule-based approach. The supervised decision-directed learning (SDDL) algorithm is also adopted to construct a two-layer network in a sequential manner by adding hidden nodes as needed. Thus, the classification knowledge embedded in the numerical weights of trained HRCNNs can be extracted and represented in the form of If-Then rules. The rules facilitate justification on the responses to increase accuracy of the classification. A sample of remote sensing image containing forest land, river, dam area, and built-up land is used to examine the proposed approach. The accurate recognition rate reaching over 99% demonstrates that the proposed approach is capable of dealing with image mapping.     

A two-stage hybrid credit scoring model using artificial neural networks and multivariate adaptive regression splines

The objective of the proposed study is to explore the performance of credit scoring using a two-stage hybrid modeling procedure with artificial neural networks and multivariate adaptive regression splines (MARS). The rationale under the analyses is firstly to use MARS in building the credit scoring model, the obtained significant variables are then served as the input nodes of the neural networks model. To demonstrate the effectiveness and feasibility of the proposed modeling procedure, credit scoring tasks are performed on one bank housing loan dataset using cross-validation approach. As the results reveal, the proposed hybrid approach outperforms the results using discriminant analysis, logistic regression, artificial neural networks and MARS and hence provides an alternative in handling credit scoring tasks.     

Evolutionary extreme learning machine

Extreme learning machine (ELM) [G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, Extreme learning machine: a new learning scheme of feedforward neural networks, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN2004), Budapest, Hungary, 25-29 July 2004], a novel learning algorithm much faster than the traditional gradient-based learning algorithms, was proposed recently for single-hidden-layer feedforward neural networks (SLFNs). However, ELM may need higher number of hidden neurons due to the random determination of the input weights and hidden biases. In this paper, a hybrid learning algorithm is proposed which uses the differential evolutionary algorithm to select the input weights and Moore-Penrose (MP) generalized inverse to analytically determine the output weights. Experimental results show that this approach is able to achieve good generalization performance with much more compact networks.     

一种能处理定性参数输入的推广模糊神经网络软测量建模方法

针对生产过程中存在多种类属型数据和混合型数据,而大多数软测量方法只能处理数值型数据的问题,提出了一种基于粗糙集方法的推广模糊神经网络软测量建模方法,该方法既可以接受定量参数输入,也可以接受定性参数输入.首先建立模糊-清晰混合规则的定义,对具有混合类型属性的样本集进行离散化处理后,利用粗糙集的约简算法进行规则提取,获得最小决策集.由得到的混合决策规则构建推广模糊神经网络,使用样本集训练网络参数.最后将该方法应用于蒸发器的污垢热阻值估计,取得了良好的效果.     

机器人末端臂轨迹跟踪自适应控制设计

本文提出了新颖的机器人末端臂轨迹跟踪自适应控制方法。该方法与已有的神经网络模型不同之处在于数据首先利用运动学反解求出机器人各关节旋转的角度,然后应用径向基函数自组织进行神经网络学习生成模糊规则,利用监督学习算法(SLA)、最小二乘法(LMS)、反向传播算法(BP)和聚类分析的方法在线优化控制规则以及隶属函数的参数。仿真结果表明,该方法不但规则生成的时间少,有效的防止了规则数爆炸,而且在机器人轨迹跟踪控制的应用中效果好。     

GenSoFNN: a generic self-organizing fuzzy neural network

Existing neural fuzzy (neuro-fuzzy) networks proposed in the literature can be broadly classified into two groups. The first group is essentially fuzzy systems with self-tuning capabilities and requires an initial rule base to be specified prior to training. The second group of neural fuzzy networks, on the other hand, is able to automatically formulate the fuzzy rules from the numerical training data. No initial rule base needs to be specified prior to training. A cluster analysis is first performed on the training data and the fuzzy rules are subsequently derived through the proper connections of these computed clusters. However, most existing neural fuzzy systems (whether they belong to the first or second group) encountered one or more of the following major problems. They are (1) inconsistent rule-base; (2) heuristically defined node operations; (3) susceptibility to noisy training data and the stability-plasticity dilemma; and (4) needs for prior knowledge such as the number of clusters to be computed. Hence, a novel neural fuzzy system that is immune to the above-mentioned deficiencies is proposed in this paper. This new neural fuzzy system is named the generic self-organizing fuzzy neural network (GenSoFNN). The GenSoFNN network has strong noise tolerance capability by employing a new clustering technique known as discrete incremental clustering (DIC). The fuzzy rule base of the GenSoFNN network is consistent and compact as GenSoFNN has built-in mechanisms to identify and prune redundant and/or obsolete rules. Extensive simulations were conducted using the proposed GenSoFNN network and its performance is encouraging when benchmarked against other neural and neural fuzzy systems.     

A fuzzy evolutionary framework for combining ensembles

We propose an evolutionary framework for the production of fuzzy rule bases where each rule executes an ensemble of predictors. The architecture, the rule base and the composition of the ensembles are evolved over time. To achieve this, we employ a context-free grammar within a hybrid genetic programming system using a multi-population model. As base predictors, multilayer perceptron neural networks and support vector machines are available. We apply the system to several function approximation and regression tasks and compare the results with recent research and state-of-the-art models. We conclude that the proposed architecture is competitive and has a number of very desirable features supporting automation of predictive model building and their adaptation over time. Finally, we suggest further potential research directions.     

Effective data mining using neural networks

Classification is one of the data mining problems receiving great attention recently in the database community. The paper presents an approach to discover symbolic classification rules using neural networks. Neural networks have not been thought suited for data mining because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by humans. With the proposed approach, concise symbolic rules with high accuracy can be extracted from a neural network. The network is first trained to achieve the required accuracy rate. Redundant connections of the network are then removed by a network pruning algorithm. The activation values of the hidden units in the network are analyzed, and classification rules are generated using the result of this analysis. The effectiveness of the proposed approach is clearly demonstrated by the experimental results on a set of standard data mining test problems