Simultaneous recurrent neural network trained with non-recurrent backpropagation algorithm for static optimisation

This paper explores feasibility of employing the non-recurrent backpropagation training algorithm for a recurrent neural network, Simultaneous Recurrent Neural network, for static optimisation. A simplifying observation that maps the recurrent network dynamics, which is configured to operate in relaxation mode as a static optimizer, to feedforward network dynamics is leveraged to facilitate application of a non-recurrent training algorithm such as the standard backpropagation and its variants. A simulation study that aims to assess feasibility, optimizing potential, and computational efficiency of training the Simultaneous Recurrent Neural network with non-recurrent backpropagation is conducted. A comparative computational complexity analysis between the Simultaneous Recurrent Neural network trained with non-recurrent backpropagation algorithm and the same network trained with the recurrent backpropagation algorithm is performed. Simulation results demonstrate that it is feasible to apply the non-recurrent backpropagation to train the Simultaneous Recurrent Neural network. The optimality and computational complexity analysis fails to demonstrate any advantage on behalf of the non-recurrent backpropagation versus the recurrent backpropagation for the optimisation problem considered. However, considerable future potential that is yet to be explored exists given that computationally efficient versions of the backpropagation training algorithm, namely quasi-Newton and conjugate gradient descent among others, are also applicable for the neural network proposed for static optimisation in this paper.     

Design of Adaptive Robot Control System Using Recurrent Neural Network

The use of a new Recurrent Neural Network (RNN) for controlling a robot manipulator is presented in this paper. The RNN is a modification of Elman network. In order to solve load uncertainties, a fast-load adaptive identification is also employed in a control system. The weight parameters of the network are updated using the standard Back-Propagation (BP) learning algorithm. The proposed control system is consisted of a NN controller, fast-load adaptation and PID-Robust controller. A general feedforward neural network (FNN) and a Diagonal Recurrent Network (DRN) are utilised for comparison with the proposed RNN. A two-link planar robot manipulator is used to evaluate and compare performance of the proposed NN and the control scheme. The convergence and accuracy of the proposed control scheme is proved.     

Adaptive air-fuel ratio control with MLP network

This paper presents an application of adaptive neural network model-based predictive control (MPC) to the air-fuel ratio of an engine simulation. A multi-layer perceptron (MLP) neural network is trained using two on-line training algorithms: a back propagation algorithm and a recursive least squares (RLS) algorithm. It is used to model parameter uncertainties in the nonlinear dynamics of internal combustion (IC) engines. Based on the adaptive model, an MPC strategy for controlling air-fuel ratio is realized, and its control performance compared with that of a traditional PI controller. A reduced Hessian method, a newly developed sequential quadratic programming (SQP) method for solving nonlinear programming (NLP) problems, is implemented to speed up nonlinear optimization in the MPC.     

基于CNN和GRU的混合股指预测模型研究

针对股票数据共线性和非线性的特点,提出了一种基于卷积神经网络（Convolutional Neural Network,CNN）和门控循环单元（Gated Recurrent Unit,GRU）神经网络的混合预测模型,并对沪深300指数、上证综指和深证成指进行了预测。该模型首先采用CNN提取特征向量,对原始数据进行降维,然后利用GRU神经网络学习特征动态变化规律进行股指预测。仿真结果表明,与GRU神经网络、长短时记忆（Long-Short-Term Memory,LSTM）神经网络和CNN相比,该模型能够挖掘历史数据中蕴含的信息,有效提高股指预测的准确率,并可为股指交易提供一些参考。     

Dynamic neural controllers for induction motor

The paper reports application of recently developed adaptive control techniques based on neural networks to the induction motor control. This case study represents one of the more difficult control problems due to the complex, nonlinear, and time-varying dynamics of the motor and unavailability of full-state measurements. A partial solution is first presented based on a single input-single output (SISO) algorithm employing static multilayer perceptron (MLP) networks. A novel technique is subsequently described which is based on a recurrent neural network employed as a dynamical model of the plant. Recent stability results for this algorithm are reported. The technique is applied to multiinput-multioutput (MIMO) control of the motor. A simulation study of both methods is presented. It is argued that appropriately structured recurrent neural networks can provide conveniently parameterized dynamic models for many nonlinear systems for use in adaptive control.     

基于结构自适应滤波方法的非线性系统辨识

针对非线性系统辨识中定结构参数辨识局限性高和辨识率低的问题,将结构自适应引入辨识的优化,提出一种基于子系统的结构自适应滤波（SSAF）方法。该方法的模型由若干子系统级联而成,每一个子系统均为线性-非线性混合结构。子系统的线性部分是一个一阶或二阶可选的无限脉冲响应滤波器（IIR）,非线性部分则是一个静态的非线性函数。初始化中,子系统的参数随机产生,生成的若干子系统按照设定的连接规则进行随机连接,而不含反馈的连接机制确保了非线性系统的有效性。采用一种自适应多精英引导的复合差分进化（AMECoDEs）算法用于自适应模型循环优化,直至找到最优的结构和参数,即全局最优。实验结果表明,SSAF方法在非线性测试函数以及真实数据集上的表现优异,辨识率高且收敛性好,与聚焦时滞递归神经网络（FTLRNN）相比,它所用参数的个数仅为FTLRNN的1/10,且适应值精度提高了7%,验证了所提方法的有效性。     

Joint state and parameter estimation in particle filtering and stochastic optimization

In this paper, an adaptive estimation algorithm is proposed for non-linear dynamic systems with unknown static parameters based on combination of particle filtering and Simultaneous Perturbation Stochastic Approxi- mation （SPSA） technique. The estimations of parameters are obtained by maximum-likelihood estimation and sampling within particle filtering framework, and the SPSA is used for stochastic optimization and to approximate the gradient of the cost function. The proposed algorithm achieves combined estimation of dynamic state and static parameters of nonlinear systems. Simulation result demonstrates the feasibilitv and efficiency of the proposed algorithm     

Optimum design of aerospace structural components using neural networks

The application of artificial neural networks to capture structural design expertise is demonstrated. The principal advantage of a trained neural network is that it requires a trivial computational effort to produce an acceptable new design. For the class of problems addressed, the development of a conventional expert system would be extremely difficult. In the present effort, a structural optimization code with multiple nonlinear programming algorithms and an artificial neural network code NETS were used. A set of optimum designs for a ring and two aircraft wings for static and dynamic constraints were generated using the optimization codes. The optimum design data were processed to obtain input and output pairs, which were used to develop a trained artificial neural network using the code NETS. Optimum designs for new design conditions were predicted using the trained network. Neural net prediction of optimum designs was found to be satisfactory for the majority of the output design parameters. However, results from the present study indicate that caution must be exercised to ensure that all design variables are within selected error bounds.     

MRNN:一种新的基于改进型递归神经网络的WSN动态建模方法:应用于故障检测

提出了一种适用于无线传感器网络WSN的故障检测方法,该方法运用改进的递归神经网络MRNN为WSN的节点、节点的动态特性以及节点间的关系建立相关模型,对WSN节点进行识别和故障检测。MRNN的输入选择建模节点的先前输出值及其邻居节点的当前及先前输出值,模型基于一种新的改进的反向传播型神经网络,该神经网络的输入以及传感器网络的拓扑结构基于通用的非线性传感器模型。仿真实验将MRNN方法与卡尔曼滤波法进行了全面的比较。实验表明,MRNN在置信因子较小的情况下与卡尔曼滤波方法相比有较高的故障检测精度。     

Joint state and parameter estimation in particle filtering and stochastic optimization

In this paper, an adaptive estimation algorithm is proposed for non-linear dynamic systems with unknown static parameters based on combination of particle filtering and Simultaneous Perturbation Stochastic Approximation (SPSA) technique. The estimations of parameters are obtained by maximum-likelihood estimation and sampling within particle filtering framework, and the SPSA is used for stochastic optimization and to approximate the gradient of the cost function. The proposed algorithm achieves combined estimation of dynamic state and static parameters of nonlinear systems. Simulation result demonstrates the feasibility and efficiency of the proposed algorithm.     

基于循环卷积神经网络的POMDP值迭代算法

基于卷积神经网络的部分可观测马尔科夫决策过程(POMDP)值迭代算法QMDP-net在无先验知识的情况下具有较好的性能表现,但其存在训练效果不稳定、参数敏感等优化难题.提出基于循环卷积神经网络的POMDP值迭代算法RQMDP-net,使用门控循环单元网络实现值迭代更新,在保留输入和递归权重矩阵卷积特性的同时增强网络时序...     

基于动态递归神经网络模型的混沌时间序列预测

提出了一种动态递归神经网络模型进行混沌时间序列预测,以最佳延迟时间为间隔的最小嵌入维数作为递归神经网络的输入维数,并按预测相点步进动态递归的生成训练数据,利用混沌特性处理样本及优化网络结构,用递归神经网络映射混沌相空间相点演化的非线性关系,提高了预测精度和稳定性。将该模型应用于Lorenz系统数据仿真以及沪市股票综合指数预测,其结果与已有网络模型预测的结果相比较,精度有很大提高。因此,证明了该预测模型在实际混沌时间序列预测领域的有效性和实用性。     

基于动态递归网络的PID自适应控制器的设计与应用

本文分析了改进的ELMAN网络的结构,并讨论了神经网络的学习算法,针对BP算法的缺陷,提出了用遗传算法修正网络权值的学习算法。本文不仅将采用遗传算法进行训练的改进ELMAN网络应用于汽车磷化加热系统的建模,而且针对该系统的特点提出了一种带预测模型的神经网络PID自适应控制器,并最后将该控制器应用于磷化温度控制,取得了良好的控制效果。     

Fast tunable gradient RBF networks for online modeling of nonlinear and nonstationary dynamic processes

Since most real-world processes exhibit both nonlinear and time-varying characteristics, there exists a need for accurate and efficient models that can adapt in nonstationary environments. Also for adaptive control purpose, it is vital that an adaptive model has a fixed small model size. In this paper, we propose an adaptive tunable gradient radial basis function (GRBF) network for online modeling of nonlinear dynamic processes, which meets these practical requirements. Specifically, a compact GRBF model is constructed by the orthogonal least squares algorithm in training, which is capable of modeling variations of local mean and trend in the data well. During online operation, the adaptive GRBF model tacks the time-varying process’s dynamics by replacing a worst performing node with a new node which encodes the current new data. By exploiting the local predictor property of the GRBF node, the new node optimization can be done extremely efficiently. The proposed approach combining the advantages of both the GRBF network structure and fast tunable node mechanism is capable of tracking the time-varying nonlinear dynamics accurately and effectively. Extensive simulation results demonstrate that the proposed fast tunable GRBF network significantly outperforms the existing state-of-the-art methods, in terms of both adaptive modeling accuracy and online computational complexity.     

粒子群优化算法在神经网络控制中的应用

考虑粒子群优化算法在不确定系统的自适应控制中的应用。神经网络在不确定系统的自适应控制中起着重要作用。但传统的梯度下降法训练神经网络时收敛速度慢,容易陷入局部极小,且对网络的初始权值等参数极为敏感。为了克服这些缺点,提出了一种基于粒子群算法优化的RBF神经网络整定PID的控制策略。首先,根据粒子群算法的基本原理提出了优化得到RBF神经网络输出权、节点中心和节点基宽参数的初值的算法。其次,再利用梯度下降法对控制器参数进一步调节。将传统的神经网络控制与基于粒子群优化的神经网络控制进行了对比,结果表明,后者有更好逼近精度。以PID控制器参数整定为例,对一类非线性控制系统进行了仿真。仿真结果表明基于粒子群优化的神经网络控制具有较强的鲁棒性和自适应能力。     

Neural network-based adaptive controller design of roboticmanipulators with an observer

A neural network (NN)-based adaptive controller with an observer is proposed for the trajectory tracking of robotic manipulators with unknown dynamics nonlinearities. It is assumed that the robotic manipulator has only joint angle position measurements. A linear observer is used to estimate the robot joint angle velocity, while NNs are employed to further improve the control performance of the controlled system through approximating the modified robot dynamics function. The adaptive controller for robots with an observer can guarantee the uniform ultimate bounds of the tracking errors and the observer errors as well as the bounds of the NN weights. For performance comparisons, the conventional adaptive algorithm with an observer using linearity in parameters of the robot dynamics is also developed in the same control framework as the NN approach for online approximating unknown nonlinearities of the robot dynamics. Main theoretical results for designing such an observer-based adaptive controller with the NN approach using multilayer NNs with sigmoidal activation functions, as well as with the conventional adaptive approach using linearity in parameters of the robot dynamics are given. The performance comparisons between the NN approach and the conventional adaptation approach with an observer is carried out to show the advantages of the proposed control approaches through simulation studies     

Filtering adaptive neural network controller for multivariable nonlinear systems with mismatched uncertainties

This paper synthesizes a filtering adaptive neural network controller for multivariable nonlinear systems with mismatched uncertainties. The multivariable nonlinear systems under consideration have both matched and mismatched uncertainties, which satisfy the semiglobal Lipschitz condition. The nonlinear uncertainties are approximated by a Gaussian radial basis function (GRBF)‐based neural network incorporated with a piecewise constant adaptive law, where the adaptive law will generate adaptive parameters by solving the error dynamics between the real system and the state predictor with the neglection of unknowns. The combination of GRBF‐based neural network and piecewise constant adaptive law relaxes hardware limitations (CPU). A filtering control law is designed to handle the nonlinear uncertainties and deliver a good tracking performance with guaranteed robustness. The matched uncertainties are cancelled directly by adopting their opposite in the control signal, whereas a dynamic inversion of the system is required to eliminate the effect of the mismatched uncertainties on the output. Since the virtual reference system defines the best performance that can be achieved by the closed‐loop system, the uniform performance bounds are derived for the states and control signals via comparison. To validate the theoretical findings, comparisons between the model reference adaptive control method and the proposed filtering adaptive neural network control architecture with the implementation of different sampling time are carried out.     

基于神经网络的非线性预测控制的进一步研究

介绍了适于具有纯滞后对象的基于人工神经网络的非线性多步预测控制，提出了具有自补偿功能的非线性预测反馈校正方法。仿真实验证明此方法大大增强了非线性预测控制的鲁棒性，改善了控制系统的动态和静态性能。     

Upper bounds for approximation of continuous-time dynamics using delayed outputs and feedforward neural networks

The problem of approximation of unknown dynamics of a continuous-time observable nonlinear system is considered using a feedforward neural network, operating over delayed sampled outputs of the system. Error bounds are derived that explicitly depend upon the sampling time interval and network architecture. The main result of this note broadens the class of nonlinear dynamical systems for which adaptive output feedback control and state estimation problems are solvable.     

基于BP算法PID控制器的研究

文中主要研究了基于BP算法的PID控制器在非线性系统中的控制效果以及对权值整定初始化的优化。在介绍BP网络基本原理的基础上以非线性控制系统Simulink仿真为例,使用基于BP算法的PID控制器对该系统进行优化和整定,并结合Nguyen-Widrow初始化算法为层产生初始权重和偏置值,使得每层神经元的活动区域能大致平坦地分布在输入空间,实现了对PID参数的实时调节,并且使神经网络的学习和收敛速度加快,大大改善系统的初始运行的稳定性。仿真结果表明,基于BP算法的PID控制器在非线性控制系统中对其参数优化整定具有良好的效果。