Model-free adaptive control design for nonlinear discrete-time processes with reinforcement learning techniques

This paper deals with the model-free adaptive control (MFAC) based on the reinforcement learning (RL) strategy for a family of discrete-time nonlinear processes. The controller is constructed based on the approximation ability of neural network architecture, a new actor-critic algorithm for neural network control problem is developed to estimate the strategic utility function and the performance index function. More specifically, the novel RL-based MFAC scheme is reasonable to design the controller without need to estimate y(k+1) information. Furthermore, based on Lyapunov stability analysis method, the closed-loop systems can be ensured uniformly ultimately bounded. Simulations are shown to validate the theoretical results.     

具有未知负载扰动的水井钻机电液伺服系统无模型自适应控制

针对电液伺服系统在水井钻机推进工况下存在的参数不确定以及未知负载扰动突变等非线性因素,提出了基于径向基(RBF)神经网络扰动观测器的无模型自适应控制方法.首先,通过改进的无模型自适应控制动态线性化方法,将被控系统线性化为与输入输出相关的增量形式,并将未知负载扰动合并到一个非线性项中;然后,设计了径向基神经网络扰动观测器对含有未知负载扰动的非线性项进行估计,作为对未知扰动的补偿;最后,设计了时变参数估计律,通过在线调整伪偏导数,给出了电液伺服系统的控制更新律.仿真结果表明,所设计的控制器能够对未知负载扰动突变进行补偿,并能确保跟踪误差有界收敛.     

Non-affine nonlinear adaptive control of decentralized large-scale systems using neural networks

This paper introduces a new decentralized adaptive neural network controller for a class of large-scale nonlinear systems with unknown non-affine subsystems and unknown interconnections represented by nonlinear functions. A radial basis function neural network is used to represent the controller’s structure. The stability of the closed loop system is guaranteed through Lyapunov stability analysis. The effectiveness of the proposed decentralized adaptive controller is illustrated by considering two nonlinear systems: a two-inverted pendulum and a turbo generator. The simulation results verify the merits of the proposed controller.     

Optimal and robust control of a class of nonlinear systems using dynamically re-optimised single network adaptive critic design

Following the philosophy of adaptive optimal control, a neural network-based state feedback optimal control synthesis approach is presented in this paper. First, accounting for a nominal system model, a single network adaptive critic (SNAC) based multi-layered neural network (called as NN₁) is synthesised offline. However, another linear-in-weight neural network (called as NN₂) is trained online and augmented to NN₁ in such a manner that their combined output represent the desired optimal costate for the actual plant. To do this, the nominal model needs to be updated online to adapt to the actual plant, which is done by synthesising yet another linear-in-weight neural network (called as NN₃) online. Training of NN₃ is done by utilising the error information between the nominal and actual states and carrying out the necessary Lyapunov stability analysis using a Sobolev norm based Lyapunov function. This helps in training NN₂ successfully to capture the required optimal relationship. The overall architecture is named as ‘Dynamically Re-optimised single network adaptive critic (DR-SNAC)’. Numerical results for two motivating illustrative problems are presented, including comparison studies with closed form solution for one problem, which clearly demonstrate the effectiveness and benefit of the proposed approach.     

小型无人直升机的无模型自适应鲁棒控制设计

本文针对小型无人直升机的姿态控制问题,考虑到现有基于模型的控制方法对系统动力学模型的依赖性,以及未建模动态对系统控制性能的影响,设计了一种新的基于数据驱动的无模型自适应鲁棒控制律.通过基于数据驱动的设计方法,降低了控制器对直升机动力学模型先验知识的依赖,补偿了未建模不确定性的影响.仅利用无人直升机的输入输出数据,即可实现对无人直升机系统的稳定姿态控制.然后本文结合离散滑模控制设计,补偿了未知外界扰动的影响,提高了系统的鲁棒性,并通过理论证明了控制误差的收敛性和闭环系统的稳定性.最后,在本研究组自主开发的无人直升机飞行控制实验平台上,进行了无人机实时控制实验.实验结果表明,本文所提出的控制算法取得了很好的姿态控制效果,并对系统不确定性和外界风扰动具有良好的鲁棒性.     

Stable decentralized adaptive control design of robot manipulators using neural network approximations

In this paper, we present a decentralized neural network (NN) adaptive technique for control of robot manipulators in the presence of unknown non-linear functions. Radial basis function NNs are used to approximate the non-linear functions to include the case of both parametric and dynamic uncertainty in each subsystem. The robustifying terms are added to the controllers to overcome the effects of the interconnections. The stability can be guaranteed by using a rigid proof. Finally, simulation is given to illustrate the effectiveness of the proposed algorithm.     

Adaptive neural sliding mode compensator for a class of nonlinear systems with unmodeled uncertainties

This paper addresses the problem of adaptive neural sliding mode control for a class of multi-input multi-output nonlinear system. The control strategy is an inverse nonlinear controller combined with an adaptive neural network with sliding mode control using an on-line learning algorithm. The adaptive neural network with sliding mode control acts as a compensator for a conventional inverse controller in order to improve the control performance when the system is affected by variations in its entire structure (kinematics and dynamics). The controllers are obtained by using Lyapunov's stability theory. Experimental results of a case study show that the proposed method is effective in controlling dynamic systems with unexpected large uncertainties.     

Observer-based adaptive control for nonlinear input-delay systems with unknown control directions

This paper investigates the problem of adaptive control for strict-feedback nonlinear systems with input delay and unknown control directions. The Nussbaum function is utilised to deal with the unknown control directions and a novel compensation system is introduced to handle the time-varying input delay. By using neural network(NN) approximation and backstepping approaches, an adaptive NN controller is designed which can guarantee the semi-global boundedness of all the signals in the closed-loop system. Two simulation examples are also given to illustrate the effectiveness of the proposed method.     

非线性系统神经网络自适应控制的发展现状及展望

简要回顾了神经网络控制及其应用的发展历程,重点论述了人们在连续、离散时间非线性系统的神经网络以及神经模糊稳定自适应控制研究方面所取得的主要进展,探讨了神经网络自适应控制研究方面存在的主要问题及解决问题的基本途径.作为当前解决神经网络自适应控制问题的途径之一,介绍了近来人们对二阶模糊神经网络以及量子神经网络的研究.最后,总结并指出了这一领域下一步的发展方向和有待解决的新课题.     

Neural network approximation for periodically disturbed functions and applications to control design

This paper addresses the approximation problem of functions affected by unknown periodically time-varying disturbances. By combining Fourier series expansion into multilayer neural network or radial basis function neural network, we successfully construct two kinds of novel approximators, and prove that over a compact set, the new approximators can approximate a continuously and periodically disturbed function to arbitrary accuracy. Then, we apply the proposed approximators to disturbance rejection in the first-order nonlinear control systems with periodically time-varying disturbances, but it is straightforward to extend the proposed design methods to higher-order systems by using adaptive backstepping technique. A simulation example is provided to illustrate the effectiveness of control schemes designed in this paper.     

Novel adaptive neural control design for nonlinear MIMO time-delay systems

In this paper, we address the problem of adaptive neural control for a class of multi-input multi-output (MIMO) nonlinear time-delay systems in block-triangular form. Based on a neural network (NN) online approximation model, a novel adaptive neural controller is obtained by constructing a novel quadratic-type Lyapunov-Krasovskii functional, which not only efficiently avoids the controller singularity, but also relaxes the restriction on unknown virtual control coefficients. The merit of the suggested controller design scheme is that the number of online adapted parameters is independent of the number of nodes of the neural networks, which reduces the number of the online adaptive learning laws considerably. The proposed controller guarantees that all closed-loop signals remain bounded, while the output tracking error dynamics converges to a neighborhood of the origin. A simulation example is given to illustrate the design procedure and performance of the proposed method.     

Robust adaptive neural control of uncertain pure-feedback nonlinear systems

In this paper, a robust adaptive neural control design approach is presented for a class of uncertain pure-feedback nonlinear systems. To reduce the complexity of the both controller structure and computation, only one neural network is used to approximate the lumped unknown function of the system at the last step of the recursive design process. By this approach, the complexity growing problem existing in conventional methods can be eliminated completely. Stability analysis shows that all the closed-loop system signals are uniformly ultimately bounded, and the steady state tracking error can be made arbitrarily small by appropriately choosing control parameters. Simulation results demonstrate the effectiveness and merits of the proposed approach.     

基于反推方法的一类自适应神经网络容错控制

针对一类可控标准型基础上添加非线性模型误差与故障项的MIMO非线性系统,结合反推技术,提出了神经网络自适应控制方案,对模型误差与故障项进行在线估计。文中鲁棒项用于补偿逼近模型误差,当检测出系统故障时,通过调整各步骤的虚拟控制量来补偿故障项,消除故障项对系统的影响。通过理论证明实现了提出的控制方法使得各残差信号一致有界,并最终收敛到一个小的邻域内。实例仿真表明该方案的可行性。     

基于自适应AGBFN的不确定非线性系统的跟踪控制

针对一类具有未知不确定性的非线性系统,提出了一种基于观测器的自适应不对称高斯基函数网络(AGBFN)跟踪控制方案.当系统只有输出可以测量时,通过设计观测器对其进行在线状态估计,进而构造反馈控制律和自适应控制律.所提出的完全自适应AGBFN,可以在线更新网络所有参数,克服了传统RBF网络对称性约束,提高了网络的适应性和学习能力,可以有效地对消系统未知不确定项的影响.证明了闭环系统所有误差信号最终一致有界,且系统输出较好地跟踪参考模型输出.仿真结果表明了所提出方法的有效性.     

采用自适应自回归小波神经网络的单步预测控制

针对非线性系统的控制问题,提出一种基于神经网络辨识的单步预测控制算法。算法在自回归小波神经网络的基础上,利用混沌机制消除了神经网络易陷入局部极值的缺点．采用自适应性学习率,提高神经网络的收敛能力和速度．以该神经网络为预测模型,引入输出反馈和偏差校正克服预测误差,以此构造一步加权预测控制性能指标。然后采用Brent一维搜索方法求取控制律,Brent法无需任何相关的导数信息,需调整的参数少,使得Brent法适合实时控制．仿真研究说明了该非线性预测控制器的有效性。     

Control of chaotic dynamical systems using radial basis function network approximators

This paper presents a general control method based on radial basis function networks (RBFNs) for chaotic dynamical systems. For many chaotic systems that can be decomposed into a sum of a linear and a nonlinear part, under some mild conditions the RBFN can be used to well approximate the nonlinear part of the system dynamics. The resulting system is then dominated by the linear part, with some small or weak residual nonlinearities due to the RBFN approximation errors. Thus, a simple linear state-feedback controller can be devised, to drive the system response to a desirable set-point. In addition to some theoretical analysis, computer simulations on two representative continuous-time chaotic systems (the Duffing and the Lorenz systems) are presented to demonstrate the effectiveness of the proposed method.     

Model-free control

‘Model-free control’and the corresponding ‘intelligent’ PID controllers (iPIDs), which already had many successful concrete applications, are presented here for the first time in an unified manner, where the new advances are taken into account. The basics of model-free control is now employing some old functional analysis and some elementary differential algebra. The estimation techniques become quite straightforward via a recent online parameter identification approach. The importance of iPIs and especially of iPs is deduced from the presence of friction. The strange industrial ubiquity of classic PIDs and the great difficulty for tuning them in complex situations is deduced, via an elementary sampling, from their connections with iPIDs. Several numerical simulations are presented which include some infinite-dimensional systems. They demonstrate not only the power of our intelligent controllers but also the great simplicity for tuning them.     

Computational intelligence approach to PID controller design using the universal model

Despite the popularity of PID (Proportional-Integral-Derivative) controllers, their tuning aspect continues to present challenges for researches and plant operators. Various control design methodologies have been proposed in the literature, such as auto-tuning, self-tuning, and pattern recognition. The main drawback of these methodologies in the industrial environment is the number of tuning parameters to be selected. In this paper, the design of a PID controller, based on the universal model of the plant, is derived, in which there is only one parameter to be tuned. This is an attractive feature from the viewpoint of plant operators. Fuzzy and neural approaches - bio-inspired methods in the field of computational intelligence - are used to design and assess the efficiency of the PID controller design based on differential evolution optimization in nonlinear plants. The numerical results presented herein indicate that the proposed bio-inspired design is effective for the nonlinear control of nonlinear plants.     

Ｓｔｅｗａｒｔ主动隔振平台的神经网络自适应控制

针对Stewart主动隔振平台,提出一种基于径向基函数(RBF)神经网络的多输入多输出自适应隔振控制方法.考虑外界振动对Stewart主动隔振平台动态特性的影响,建立了隔振平台在工作空间中的动力学模型.推导出RBF神经网络的权值矩阵、高斯基函数中心和宽度的在线自适应调节律,以使神经网络快速逼近系统的非线性动态函数.应用Lyapunov稳定性理论,证明了在扰动力和神经网络逼近误差有界的条件下,闭环控制系统滤波误差和RBF神经网络各调节参数估计误差的一致最终有界.仿真结果表明,该控制方法能有效地抑制不同方向的低频有界振动.     

自适应CRBF非线性滤波器及其改进学习算法

传统的随机梯度算法由于采用基于二阶统计量的平方误差代价函数,因此含有的信息量较少,难以实现更高的精度。针对此问题,以基于高阶统计量的指数平方误差作为代价函数,结合基于两层RBF网络凸组合的非线性自适应滤波器,提出了最小指数平方误差自适应学习算法。非线性系统辨识和非线性信道均衡的实验仿真结果表明,该改进算法的收敛性能明显优于传统的随机梯度算法。