Diagonal recurrent neural networks for dynamic systems control

A new neural paradigm called diagonal recurrent neural network (DRNN) is presented. The architecture of DRNN is a modified model of the fully connected recurrent neural network with one hidden layer, and the hidden layer comprises self-recurrent neurons. Two DRNN's are utilized in a control system, one as an identifier called diagonal recurrent neuroidentifier (DRNI) and the other as a controller called diagonal recurrent neurocontroller (DRNC). A controlled plant is identified by the DRNI, which then provides the sensitivity information of the plant to the DRNC. A generalized dynamic backpropagation algorithm (DBP) is developed and used to train both DRNC and DRNI. Due to the recurrence, the DRNN can capture the dynamic behavior of a system. To guarantee convergence and for faster learning, an approach that uses adaptive learning rates is developed by introducing a Lyapunov function. Convergence theorems for the adaptive backpropagation algorithms are developed for both DRNI and DRNC. The proposed DRNN paradigm is applied to numerical problems and the simulation results are included.     

An on-line adaptive neural network for speech recognition

In this paper, we present an on-line learning neural network model, Dynamic Recognition Neural Network (DRNN), for real-time speech recognition. The property of accumulative learning of the DRNN makes it very suitable for real-time speech recognition with on-line learning. A comparison between the DRNN and Hidden Markov Model (HMM) shows that the computational complexity of the former is lower than that of the latter in both training and recognition. Encouraging results are obtained when the DRNN is tested on a BUPT digit database (Mandarin) and on the on-line learning of twenty isolated English computer command words.     

对角神经网络用于环形激光陀螺消噪建模

对角神经网络(DRNN)为非全反馈式动态神经网络。应用DRNN对处于静止和运动状态下的环形激光陀螺(RLG)进行了消噪建模,并应用A llan方差方法对消噪后的结果进行了对比分析。结果表明:使用DRNN对RLG进行消噪建模是可行的。同时,将DRNN与反向传播神经网络的消噪结果进行了比较,得到动态网络的消噪能力要优于静态网络的结论。所用方法对研究RLG的误差补偿及快速启动是有实际意义的。     

应用DRNN和ARIMA组合模型的时空集成预测方法

在评析目前时空预测研究现状的基础上,提出基于动态回归神经网络(DRNN)和自回归集成移动平均(ARIMA)组合模型的时空集成预测方法.该方法先用ARIMA模型对时空数据的时序进行预测,再用DRNN捕获时空数据间隐藏的空间关系,最后用线性回归将二者整合起来,得到集成预测结果.案例实验结果表明:该方法比不考虑空间影响的预测方法或单一的预测方法有更高的精度;该方法具有良好的动态处理和计算能力,对跨空间的动态过程的预测有效可行.     

Robust air/fuel ratio control with adaptive DRNN model and AD tuning

Current production engines use look-up table and proportional and integral (PI) feedback control to regulate air/fuel ratio (AFR), which is time-consuming for calibration and is not robust to engine parameter uncertainty and time varying dynamics. This paper investigates engine modelling with the diagonal recurrent neural network (DRNN) and such a model-based predictive control for AFR. The DRNN model is made adaptive on-line to deal with engine time varying dynamics, so that the robustness in control performance is greatly enhanced. The developed strategy is evaluated on a well-known engine benchmark, a simulated mean value engine model (MVEM). The simulation results are also compared with the PI control.     

基于深度循环神经网络和改进SMOTE算法的组合式入侵检测模型

已有入侵检测模型普遍只针对网络入侵行为的静态特征进行分析检测,造成检测率低及误报率高等缺陷,且无法有效应用低频攻击。为此提出一种新的基于深度循环神经网络（DRNN）和区域自适应合成过采样算法（RA-SMOTE）的组合式入侵检测模型（DRRS）。首先,RA-SMOTE 对数据集中低频攻击样本进行自适应区域划分,实现差别样本增量,从数据层面提升低频攻击样本数量;其次,利用 DRNN 特有的层间反馈单元,完成多阶段分类特征的时序积累学习,同时多隐层网络结构实现对原始数据分布的最优非线性拟合;最后,使用训练好的DRRS模型完成入侵检测。实验结果表明,相比已有入侵检测模型,DRRS在改善整体检测效果的同时显著提高了低频攻击检测率,且对未知新型攻击具有一定检出率,适用于实际网络环境。     

聚合物分子量分布的多变量动态系统模型

为了解决聚合产品分子量分布控制的难题,将神经网络引入对其进行了无需任何系统内部先验知识的黑箱建模。所使用的神经网络是由B样条神经网络和非线性递归神经网络(DRNN)组合而成,并使用误差反传算法对网络进行训练和学习,从而建立了多变量动态系统的分子量分布模型。在模型建立中将控制变量与分布参数的函数关系利用非线性递归神经网络描述,分子量分布函数使用B样条神经网络表示,仿真研究结果证明该方法取得了预期的建模效果,具有一定的推广实用价值。     

基于动态递归神经网络模型的混沌时间序列预测

提出了一种动态递归神经网络模型进行混沌时间序列预测，以最佳延迟时间为间隔的最小嵌入维数作为递归神经网络的输入维数，并按预测相点步进动态递归的生成训练数据，利用混沌特性处理样本及优化网络结构，用递归神经网络映射混沌相空间相点演化的非线性关系，提高了预测精度和稳定性。将该模型应用于Lorenz系统数据仿真以及沪市股票综合指数预测，其结果与已有网络模型预测的结果相比较，精度有很大提高。因此，证明了该预测模型在实际混沌时间序列预测领域的有效性和实用性。     

A diagonal recurrent neural network-based hybrid direct adaptiveSPSA control system

A direct adaptive simultaneous perturbation stochastic approximation (DA SPSA) control system with a diagonal recurrent neural network (DRNN) controller is proposed. The DA SPSA control system with DRNN has simpler architecture and parameter vector size that is smaller than a feedforward neural network (FNN) controller. The simulation results show that it has a faster convergence rate than FNN controller. It results in a steady-state error and is sensitive to SPSA coefficients and termination condition. For trajectory control purpose, a hybrid control system scheme with a conventional PID controller is proposed     

Training of a discrete recurrent neural network for sequence classification by using a helper FNN

This research is concerned with a gradient descent training algorithm for a target network that makes use of a helper feed-forward network (FFN) to represent the cost function required for training the target network. A helper FFN is trained because the cost relation for the target is not differentiable. The transfer function of the trained helper FFN provides a differentiable cost function of the parameter vector for the target network allowing gradient search methods for finding the optimum values of the parameters. The method is applied to the training of discrete recurrent networks (DRNNs) that are used as a tool for classification of temporal sequences of characters from some alphabet and identification of a finite state machine (FSM) that may have produced all the sequences. Classification of sequences that are input to the DRNN is based on the terminal state of the network after the last element in the input sequence has been processed. If the DRNN is to be used for classifying sequences the terminal states for class 0 sequences must be distinct from the terminal states for class 1 sequences. The cost value to be used in training must therefore be a function of this disjointedness and no more. The outcome of this is a cost relationship that is not continuous but discrete and therefore derivative free methods have to be used or alternatively the method suggested in this paper. In the latter case the transform function of the helper FFN that is trained using the cost function is a differentiable function that can be used in the training of the DRNN using gradient descent.Acknowledgement. This work was supported by a discovery grant from the Government of Canada. The comments made by the reviewers are also greatly appreciated and have proven to be quite useful.     

A Simplified Dual Neural Network for Quadratic Programming With Its KWTA Application

The design, analysis, and application of a new recurrent neural network for quadratic programming, called simplified dual neural network, are discussed. The analysis mainly concentrates on the convergence property and the computational complexity of the neural network. The simplified dual neural network is shown to be globally convergent to the exact optimal solution. The complexity of the neural network architecture is reduced with the number of neurons equal to the number of inequality constraints. Its application to k-winners-take-all (KWTA) operation is discussed to demonstrate how to solve problems with this neural network     

Identification of a golf swing robot using soft computing approach

Golf swing robots have been recently developed in an attempt to simulate the ultra high-speed swing motions of golfers. Accurate identification of a golf swing robot is an important and challenging research topic, which has been regarded as a fundamental basis in the motion analysis and control of the robots. But there have been few studies conducted on the golf swing robot identification, and comparative analyses using different kinds of soft computing methodologies have not been found in the literature. This paper investigates the identification of a golf swing robot based on four kinds of soft computing methods, including feedforward neural networks (FFNN), dynamic recurrent neural networks (DRNN), fuzzy neural networks (FNN) and dynamic recurrent fuzzy neural networks (DRFNN). The performance comparison is evaluated based on three sets of swing trajectory data with different boundary conditions. The sensitivity of the results to the changes in system structure and learning rate is also investigated. The results suggest that both FNN and DRFNN can be used as a soft computing method to identify a golf robot more accurately than FFNN and DRNN, which can be used in the motion control of the robot.     

Dynamics identification and control of nonlinear MIMO coupled plant using supervised neural gas and comparison with recurrent neural controller

The dynamics identification and subsequent control of a nonlinear system is not a trivial issue. The application of a neural gas network that is trained with a supervised batch version of the algorithm can produce identification models in a robust way. In this paper, the neural model identifies each local transfer function, demonstrating that the local linear approximation can be done. Moreover, other parameters are analyzed in order to obtain a correct modeling. Furthermore, the algorithm is applied to control a nonlinear multi-input multi-output system composed of tanks. In addition, this plant is a coupled system where the manipulated input variables are influencing all the output variables. The aim of the work is to demonstrate that the supervised neural gas algorithm is able to obtain linear models to be used in a state space design scenario to control nonlinear coupled systems and guarantee a robust control method. The results are compared with the common approach of using a recurrent neural controller trained with a dynamic backpropagation algorithm. Regarding the steady-state errors in disturbance rejection, reference tracking and sensitivity to simple process changes, the proposed approach shows an interesting application to control nonlinear plants.

     

A two-stage algorithm for identification of nonlinear dynamic systems

This paper investigates the two-stage stepwise identification for a class of nonlinear dynamic systems that can be described by linear-in-the-parameters models, and the model has to be built from a very large pool of basis functions or model terms. The main objective is to improve the compactness of the model that is obtained by the forward stepwise methods, while retaining the computational efficiency. The proposed algorithm first generates an initial model using a forward stepwise procedure. The significance of each selected term is then reviewed at the second stage and all insignificant ones are replaced, resulting in an optimised compact model with significantly improved performance. The main contribution of this paper is that these two stages are performed within a well-defined regression context, leading to significantly reduced computational complexity. The efficiency of the algorithm is confirmed by the computational complexity analysis, and its effectiveness is demonstrated by the simulation results.     

一种基于PSO的自适应神经网络预测控制

针对非线性系统,提出了一种基于微粒群优化(PSO)的自适应神经网络预测控制方法.采用对角递归网络(DRNN)对非线性系统进行建模,并利用扩展卡尔曼滤波(EKF)递推估计算法在线计算网络模型参数的Jacobian矩阵以实现模型参数的自适应.利用PSO算法在线优化求解非线性系统的预测控制律,以克服传统基于梯度法的非线性规划方法求解预测控制律时对初始条件非常敏感的缺点.生化发酵过程的仿真结果表明,所提出的控制方法具有良好的跟踪能力和抗干扰能力.     

Deep convolutional neural network based disease identification in grapevine leaf images

Grapevine (Vitis vinifera L.) is a major fruit crop with commercial importance worldwide. Black rot, Black measles, and Leaf blight are three diseases commonly found in the grapevine. The timely and accurate diagnosis is crucial in preventing the spread of the disease and reducing loss in production. The advancement in deep learning has opened doors for new diagnostic algorithms in the domain of plant disease identification. In this paper, we propose a grapevine disease identification method using a convolutional neural network (CNN). A light weight 6-layer CNN model was designed from scratch and trained using an open repository with 3 disease classes and 1 healthy leaf image dataset. The dataset contained a total of 3423 grapevine leaf images. The model was trained with a 70–30 train-test ratio. Image augmentation and early stopping techniques were used to avoid overfitting of the model. The proposed model achieved 98.4% classification accuracy on the test dataset. Additionally, the key feature of the proposed 6-layer model is that it has lesser number of trainable parameters which reduces its computational complexity as compared to the existing pre-trained models.

     

Reducing the complexity of an adaptive radial basis function network with a histogram algorithm

In this paper, a constructive training technique known as the dynamic decay adjustment (DDA) algorithm is combined with an information density estimation method to develop a new variant of the radial basis function (RBF) network. The RBF network trained with the DDA algorithm (i.e. RBFNDDA) is able to learn information incrementally by creating new hidden units whenever it is necessary. However, RBFNDDA exhibits a greedy insertion behaviour that absorbs both useful and non-useful information during its learning process, therefore increasing its network complexity unnecessarily. As such, we propose to integrate RBFNDDA with a histogram (HIST) algorithm to reduce the network complexity. The HIST algorithm is used to compute distribution of information in the trained RBFNDDA network. Then, hidden nodes with non-useful information are identified and pruned. The effectiveness of the proposed model, namely RBFNDDA-HIST, is evaluated using a number of benchmark data sets. A performance comparison study between RBFNDDA-HIST and other classification methods is conducted. The proposed RBFNDDA-HIST model is also applied to a real-world condition monitoring problem in a power generation plant. The results are analysed and discussed. The outcome indicates that RBFNDDA-HIST not only can reduce the number of hidden nodes significantly without requiring a long training time but also can produce promising accuracy rates.

     

可控受限多变量耦合系统的智能控制

针对可控受限多变量耦合系统,提出了一种基于对角递归神经网络(DRNN)整定的PID混合解耦控制。采用对角递归神经网络来辨识系统模型,进而对PID控制器参数进行整定,实现多变量解耦控制。通过对多变量耦合控制系统的设计和实时控制,实际控制结果达到了解耦控制的要求,并具有无超调、响应速度快、控制精度高等特点。     

A Wiener-type recurrent neural network and its control strategy for nonlinear dynamic applications

This paper presents a Wiener-type recurrent neural network with a systematic identification algorithm and a control strategy for the identification and control of unknown dynamic nonlinear systems. The proposed Wiener-type recurrent network resembles the conventional Wiener model that consists of a dynamic linear subsystem cascaded with a static nonlinear subsystem. The novelties of our network include: (1) the two subsystems are integrated into a single network whose output is expressed by a nonlinear transformation of a linear state-space equation; (2) the characteristics of the trained network can be analyzed by its associated state-space equation using the well-developed theory of linear systems; and (3) the size of the network structure is determined by the number of state variables (or the system order) of the unknown systems to be identified. To effectively identify a given unknown system from its input–output data, we have developed a systematic identification algorithm that consists of an order determination procedure, a parameterization procedure, and an online learning procedure. The false nearest neighbors algorithm was adopted to acquire a minimal embedding dimension from the input–output data as the system order, and then the eigensystem realization algorithm (ERA) was used to initialize a best-fit state-space representation according to the acquired system order. To improve the overall identification performance, we have derived an online parameter learning algorithm based on an ordered derivatives and momentum terms. Subsequently, a simple feedback linear controller was designed to control the unknown dynamic nonlinear systems without much complexity. Computer simulations and comparisons with some existing recurrent networks have conducted to confirm the effectiveness and superiority of the proposed Wiener-type network, identification algorithm and control strategy.     

Deep recurrent neural network for mobile human activity recognition with high throughput

In this paper, we propose a method of human activity recognition with high throughput from raw accelerometer data applying a deep recurrent neural network (DRNN), and investigate various architectures and its combination to find the best parameter values. The “high throughput” refers to short time at a time of recognition. We investigated various parameters and architectures of the DRNN by using the training dataset of 432 trials with 6 activity classes from 7 people. The maximum recognition rate was 95.42% and 83.43% against the test data of 108 segmented trials each of which has single activity class and 18 multiple sequential trials, respectively. Here, the maximum recognition rates by traditional methods were 71.65% and 54.97% for each. In addition, the efficiency of the found parameters was evaluated using additional dataset. Further, as for throughput of the recognition per unit time, the constructed DRNN was requiring only 1.347 ms, while the best traditional method required 11.031 ms which includes 11.027 ms for feature calculation. These advantages are caused by the compact and small architecture of the constructed real time oriented DRNN.