Fuzzy generalization of the counter-propagation neural network: a family of soft competitive basis function neural networks

 Based on combining neural network (NN) with fuzzy logical system (FLS), a new family of three-layer feedforward networks, called soft-competition basis function neural networks (SCBFs), is proposed under the framework of the counter-propagation (CP) network. The hidden layer of SCBFs is designed as competitive layer with soft competitive strategy. The output function of their hidden neuron is defined as basis function taking the form of fuzzy membership function. SCBFs possess the ability of functional approximation. They are fuzzy generalization of the CP network and functionally equivalent to TS-model of fuzzy logical system. Therefore, they can be regard as either a NN or a FLS. Their learning algorithms are also discussed in this paper. Finally, some experiments are given to test the performance of SCBFs.     

基于模糊神经网络的网络业务分类研究

该文利用神经网络的自学习能力和模糊逻辑的动态性和及时性等特点,将模糊逻辑和神经网络有机地结合起来,构造出了四层模糊神经网络,并用训练神经网络的相应学习算法训练网络,将该模型用于网络业务源特征提取与分类的研究中,并与单纯的神经网络算法相比较。计算机仿真结果表明,模糊神经网络方法比神经网络算法更优越,该文的研究结果为解决网络业务源特征提取与分类奠定了基础。     

On the design of neural-fuzzy control system

This article presents a neural–network-based fuzzy logic control (NN–FLC) system. The NN–FLC model has the learning capabilities for constructing membership functions and extracting fuzzy rules from training examples. Both unsupervised and supervised training algorithms are used to find the membership functions of the FLC. Competitive learning algorithms are employed to evaluate fuzzy logic rules. Matlab programs using both neural and fuzzy toolboxes are developed to implement the NN–FLC model. Computer simulations of the inverted pendulum controlled by NN–FLC system were conducted to illustrate the self-learning ability of the network. © 1998 John Wiley & Sons, Inc.13: 11–26, 1998     

Fuzzy adaptive single neuron NN control of brushless DC motor

Inherently, the brushless DC motor (BLDCM) is a nonlinear plant. So, it is hard to get a good performance by using the conventional PI controller for the speed control of BLDCM. In this paper, a fuzzy adaptive single neuron neural networks (NN) controller for BLDCM is developed. The fuzzy logic system (FLS) is adopted to adjust the parameter K of single neuron NN controller online. By this way, performance of the system can be improved. Performances of the proposed fuzzy adaptive single neuron NN controller are compared with the performances of conventional PI controller and normal single neuron NN controller. The experimental results demonstrate that a good control performance is achieved. The using of fuzzy adaptive single neuron NN makes the drive system robust, accurate, and insensitive to parameter variations.     

Evolutionary fuzzy hybrid neural network for project cash flow control

This paper develops an evolutionary fuzzy hybrid neural network (EFHNN) to enhance project cash flow management. The developed EFHNN combines neural networks (NN) and high order neural networks (HONN) into a hybrid neural network (HNN), which acts as the major inference engine and operates with alternating linear and nonlinear NN layer connections. Fuzzy logic is employed to sandwich the HNN between a fuzzification and defuzzification layer. The authors developed and applied the EFHNN to sequential cash flow trend problems by fusing HNN, FL, and GA. Results show that the proposed EFHNN can be deployed effectively to sequential cash flow estimation. The performance of linear and nonlinear (high order) neuron layer connectors in the EFHNN was significantly better than the performance achieved by previous models that used singular linear NN. Trained results were used for the prediction and strategic management of project cash flow. The proposed strategy can assist project managers to control project cash flows within the banana envelope of the S-curve to enhance project success.     

Prediction and identification using wavelet-based recurrent fuzzy neural networks

This paper presents a wavelet-based recurrent fuzzy neural network (WRFNN) for prediction and identification of nonlinear dynamic systems. The proposed WRFNN model combines the traditional Takagi-Sugeno-Kang (TSK) fuzzy model and the wavelet neural networks (WNN). This paper adopts the nonorthogonal and compactly supported functions as wavelet neural network bases. Temporal relations embedded in the network are caused by adding some feedback connections representing the memory units into the second layer of the feedforward wavelet-based fuzzy neural networks (WFNN). An online learning algorithm, which consists of structure learning and parameter learning, is also presented. The structure learning depends on the degree measure to obtain the number of fuzzy rules and wavelet functions. Meanwhile, the parameter learning is based on the gradient descent method for adjusting the shape of the membership function and the connection weights of WNN. Finally, computer simulations have demonstrated that the proposed WRFNN model requires fewer adjustable parameters and obtains a smaller rms error than other methods.     

The structures of optical neural nets based on new matrix-tensor equivalental models (MTEMs) and results of modeling

The structures of optical neural nets (NN) based on new matrix-tensor equivalental models (MTEMs) and algorithms are described in this article. MTE models are neuroparadigm of non-iterative type, which is a generalization of Hopfield and Hamming networks. The adaptive multi-layer networks, auto-associative and hetero-associative memory of 2-D images of high order can be built on the basis of MTEMs. The capacity of such networks in comparison with capacity of Hopfield networks is increased (including capacity for greatly correlated images). The results of modeling show that the number of neurons in neural network MTEMs is 10–20 thousand and more. The problems of training of such networks, different modifications, including networks with double adaptive-equivalental auto-weighing of weights, organization of computing process in different modes of network are discussed. The basic components of networks: matrix-tensor “equivalentors” and variants of their realization on the basis of liquid-crystal structures and optical multipliers with spatial and time integration are considered. The efficiency of proposed optical neural networks on the basis of MTEMs is evaluated for both variants on the level of 10⁹ connections per second. Modified optical connections are realized as liquid-crystal television screens.     

Optimization of type-2 fuzzy weights in backpropagation learning for neural networks using GAs and PSO

In this paper the optimization of type-2 fuzzy inference systems using genetic algorithms (GAs) and particle swarm optimization (PSO) is presented. The optimized type-2 fuzzy inference systems are used to estimate the type-2 fuzzy weights of backpropagation neural networks. Simulation results and a comparative study among neural networks with type-2 fuzzy weights without optimization of the type-2 fuzzy inference systems, neural networks with optimized type-2 fuzzy weights using genetic algorithms, and neural networks with optimized type-2 fuzzy weights using particle swarm optimization are presented to illustrate the advantages of the bio-inspired methods. The comparative study is based on a benchmark case of prediction, which is the Mackey-Glass time series (for τ = 17) problem.     

Nonlinear Systems Identification via Two Types of Recurrent Fuzzy CMAC

Normal fuzzy CMAC neural network performs well for nonlinear systems identification because of its fast learning speed and local generalization capability for approximating nonlinear functions. However, it requires huge memory and the dimension increases exponentially with the number of inputs. It is difficult to model dynamic systems with static fuzzy CMACs. In this paper, we use two types of recurrent techniques for fuzzy CMAC to overcome the above problems. The new CMAC neural networks are named recurrent fuzzy CMAC (RFCMAC) which add feedback connections in the inner layers (local feedback) or the output layer (global feedback). The corresponding learning algorithms have time-varying learning rates, the stabilities of the neural identifications are proven.     

SRNET: A Shallow Skip Connection Based Convolutional Neural Network Design for Resolving Singularities

Convolutional neural networks (CNNs) have shown tremendous progress and performance in recent years. Since emergence, CNNs have exhibited excellent performance in most of classification and segmentation tasks. Currently, the CNN family includes various architectures that dominate major vision-based recognition tasks. However, building a neural network (NN) by simply stacking convolution blocks inevitably limits its optimization ability and introduces overfitting and vanishing gradient problems. One of the key reasons for the aforementioned issues is network singularities, which have lately caused degenerating manifolds in the loss landscape. This situation leads to a slow learning process and lower performance. In this scenario, the skip connections turned out to be an essential unit of the CNN design to mitigate network singularities. The proposed idea of this research is to introduce skip connections in NN architecture to augment the information flow, mitigate singularities and improve performance. This research experimented with different levels of skip connections and proposed the placement strategy of these links for any CNN. To prove the proposed hypothesis, we designed an experimental CNN architecture, named as Shallow Wide ResNet or SRNet, as it uses wide residual network as a base network design. We have performed numerous experiments to assess the validity of the proposed idea. CIFAR-10 and CIFAR-100, two well-known datasets are used for training and testing CNNs. The final empirical results have shown a great many of promising outcomes in terms of performance, efficiency and reduction in network singularities issues.

     

一种有效的增量顺序学习模糊神经网络

利用误差下降率定义输入数据对系统输出的敏感性,并以此作为规则产生标准,提出一种有效增量顺序学习模糊神经网络。将修剪策略引入规则产生过程,因此该算法产生的模糊神经网络不需要进行修剪。通过仿真实验,本算法在达到与其他算法相当性能的情况下,能够获得更高的准确率和更简单的结构。     

Fuzzy classifications using fuzzy inference networks

In this paper, fuzzy inference models for pattern classifications have been developed and fuzzy inference networks based on these models are proposed. Most of the existing fuzzy rule-based systems have difficulties in deriving inference rules and membership functions directly from training data. Rules and membership functions are obtained from experts. Some approaches use backpropagation (BP) type learning algorithms to learn the parameters of membership functions from training data. However, BP algorithms take a long time to converge and they require an advanced setting of the number of inference rules. The work to determine the number of inference rules demands lots of experiences from the designer. In this paper, self-organizing learning algorithms are proposed for the fuzzy inference networks. In the proposed learning algorithms, the number of inference rules and the membership functions in the inference rules will be automatically determined during the training procedure. The learning speed is fast. The proposed fuzzy inference network (FIN) classifiers possess both the structure and the learning ability of neural networks, and the fuzzy classification ability of fuzzy algorithms. Simulation results on fuzzy classification of two-dimensional data are presented and compared with those of the fuzzy ARTMAP. The proposed fuzzy inference networks perform better than the fuzzy ARTMAP and need less training samples.     

神经网络自学习模糊控制及其在合成氨生产中的应用

提出一类基于神经网络的模糊控制设计方案。控制系统中包括两个神经网络,一是利用神经网络进行模糊推理,实现控制规则的推理过程;二是采用另一个神经网络对系统的动态进行跟踪,以实现前向通道中语言变量的模糊区间的优化,从而使控制效果更加理想。合成氨控制系统的实例验证了该算法的有效性     

Robust H∞ performance for discrete time T-S fuzzy switched memristive stochasticneural networks with mixed time-varying delays

ABSTRACT

In this paper, we study the robust H _∞ performance for discrete-time T-S fuzzy switched memristive stochastic neural networks with mixed time-varying delays and switching signal design. The neural network under consideration is subject to time-varying and norm bounded parameter uncertainties. Decomposing of the delay interval approach is employed in both the discrete delays and distributed delays. By constructing a proper Lyapunov-Krasovskii functional (LKF) with triple summation terms and using an improved summation inequality techniques. Sufficient conditions are derived in terms of linear matrix inequalities (LMIs) to guarantee the considered discrete-time neural networks to be exponentially stable. Finally, numerical examples with simulation results are given to illustrate the effectiveness of the developed theoretical results.     

A Method of Face Recognition Based on Fuzzy c-Means Clustering and Associated Sub-NNs

The face is a complex multidimensional visual model and developing a computational model for face recognition is difficult. In this paper, we present a method for face recognition based on parallel neural networks. Neural networks (NNs) have been widely used in various fields. However, the computing efficiency decreases rapidly if the scale of the NN increases. In this paper, a new method of face recognition based on fuzzy clustering and parallel NNs is proposed. The face patterns are divided into several small-scale neural networks based on fuzzy clustering and they are combined to obtain the recognition result. In particular, the proposed method achieved a 98.75% recognition accuracy for 240 patterns of 20 registrants and a 99.58% rejection rate for 240 patterns of 20 nonregistrants. Experimental results show that the performance of our new face-recognition method is better than those of the backpropagation NN (BPNN) system, the hard c-means (HCM) and parallel NNs system, and the pattern-matching system     

一般二维模糊控制器的等效神经网络建模与验证

针对模糊控制器的计算复杂性,实时性能差,易产生维度灾难等问题,利用神经网络的万能函数逼近能力,构建一个神经网络模型,精确的逼近已知的模糊控制器,从而减少运算量,实现实时控制.以一个已知的二输入单输出模糊控制器为例,建立一个与之等效的神经网络,通过训练,使得精确的逼近模糊控制系统.最后,给定输入信号,分别用模糊控制器和神经网络控制同一个被控对象.结果表明,用一个与模糊控制器等效的神经网络来控制同一个对象,控制效果非常相似.因此,用模糊控制器的等效神经网络模型代替,在实现环节上可以减少计算复杂性,维度灾难,提高实时性能.     

Constructing gene regulatory networks from microarray data using GA/PSO with DTW

Recently, many methods have been proposed for constructing gene regulatory networks (GRNs). However, most of the existing methods ignored the time delay regulatory relation in the GRN predictions. In this paper, we propose a hybrid method, termed GA/PSO with DTW, to construct GRNs from microarray datasets. The proposed method uses test of correlation coefficient and the dynamic time warping (DTW) algorithm to determine the existence of a time delay relation between two genes. In addition, it uses the particle swarm optimization (PSO) to find thresholds for discretizing the microarray dataset. Based on the discretized microarray dataset and the predicted types of regulatory relations among genes, the proposed method uses a genetic algorithm to generate a set of candidate GRNs from which the predicted GRN is constructed. Three real-life sub-networks of yeast are used to verify the performance of the proposed method. The experimental results show that the GA/PSO with DTW is better than the other existing methods in terms of predicting sensitivity and specificity.     

Neural net robot controller: Structure and stability proofs

A multilayer neural net (NN) controller for a general serial-link robot arm is developed. The structure of the NN controller is derived using a filtered error approach. It is argued that standard backpropagation tuning, when used for real-time closed-loop control, can yield unbounded NN weights if: (1) the net can not exactly reconstruct a certain required control function, (2) there are bounded unknown disturbances in the robot dynamics, or (3) the robot arm has more than one link (i.e. nonlinear case). On-line weight tuning algorithms including correction terms to backpropagation, plus an added robustifying signal, guarantee tracking as well as bounded weights. The correction terms involve a second-orderforward-propagated wave in the backprop network.     

Training fuzzy logic based software components by combining adaptation algorithms

 A training framework of an effective method for off-line training of a class of control software components (e.g., for first-order nonlinear feedback control systems) using combinations of three kinds of adaptation algorithms is presented. Each control software component is represented at the abstract level by means of a set of adaptive fuzzy logic (FL) rules and at the concrete level by means of fuzzy membership functions (MBFs). At the concrete representation level adaptation algorithms specified for use in adapting MBFs are: genetic algorithms, neural net algorithms, and Monte Carlo algorithms. We specify effective combinations of these three existing adaptation algorithms to train a faulty FL rule-based software component for the tracker problem. In the framework, training consists of two phases: testing and adapting. In the testing phase, a test driver generates an effective fault scenario ( fs) and locates the faulty fuzzy elements (FFEs) by using each or a combination of three adaptation algorithms. In the adapting phase, for each fault scenario adaptation algorithms and their combinations are used to modify the MBFs of the component. Effectiveness of the two phase training is determined in terms of testability, flexibility, adaptability, and stability. An initial design of the simulation environment is presented. In the experiment, for a given circumstance (environment and fuzzy rules) we apply a combination of a genetic algorithm GA) and a neural network (NN) with an error back-propagation algorithm (BP) in the testing phase for generating fault scenarios. Then we apply GA-only method in the adapting phase for adapting the faulty software component. Simulation results on effectiveness and efficiency are discussed.