Model-Free control performance improvement using virtual reference feedback tuning and reinforcement Q-learning

This paper proposes the combination of two model-free controller tuning techniques, namely linear virtual reference feedback tuning (VRFT) and nonlinear state-feedback Q-learning, referred to as a new mixed VRFT-Q learning approach. VRFT is first used to find stabilising feedback controller using input-output experimental data from the process in a model reference tracking setting. Reinforcement Q-learning is next applied in the same setting using input-state experimental data collected under perturbed VRFT to ensure good exploration. The Q-learning controller learned with a batch fitted Q iteration algorithm uses two neural networks, one for the Q-function estimator and one for the controller, respectively. The VRFT-Q learning approach is validated on position control of a two-degrees-of-motion open-loop stable multi input-multi output (MIMO) aerodynamic system (AS). Extensive simulations for the two independent control channels of the MIMO AS show that the Q-learning controllers clearly improve performance over the VRFT controllers.     

Joint acceleration feedback control for robots: analysis, sensing and experiments

Independent joint control for robots is enhanced to suppress the dynamic couplings by incorporating an acceleration feedback loop that is designed in terms of its stability and ability in resisting the dynamic coupling disturbances. The sensing and modeling of joint acceleration via linear accelerometers is dealt from the viewpoint of practical implementation. Extensive experiments are conducted on a three-link direct drive robot, to mainly investigate the ability of the independent joint controller with the acceleration loop in resisting the coupling torque, and demonstrating the enhanced trajectory tracking performance. Results are given against the ones obtained by conventional independent joint control without the acceleration feedback control.     

Robust design of feedback feed-forward iterative learning control based on 2D system theory for linear uncertain systems

For a class of linear discrete-time uncertain systems, a feedback feed-forward iterative learning control (ILC) scheme is proposed, which is comprised of an iterative learning controller and two current iteration feedback controllers. The iterative learning controller is used to improve the performance along the iteration direction and the feedback controllers are used to improve the performance along the time direction. First of all, the uncertain feedback feed-forward ILC system is presented by an uncertain two-dimensional Roesser model system. Then, two robust control schemes are proposed. One can ensure that the feedback feed-forward ILC system is bounded-input bounded-output stable along time direction, and the other can ensure that the feedback feed-forward ILC system is asymptotically stable along time direction. Both schemes can guarantee the system is robust monotonically convergent along the iteration direction. Third, the robust convergent sufficient conditions are given, which contains a linear matrix inequality (LMI). Moreover, the LMI can be used to determine the gain matrix of the feedback feed-forward iterative learning controller. Finally, the simulation results are presented to demonstrate the effectiveness of the proposed schemes.     

Control of a pneumatic power active lower-limb orthosis with filter-based iterative learning control

A filter-based iterative learning control (FILC) scheme is developed in this paper, which consists in a proportional–derivative (PD) feedback controller and a feedforward filter. Moreover, based on two-dimensional system theory, the stability of the FILC system is proven. The design criteria for a wavelet transform filter (WTF) – chosen as the feedforward filter – and the PD feedback controller are also given. Finally, using a pneumatic power active lower-limb orthosis (PPALO) as the controlled plant, the wavelet-based iterative learning control (WILC) implementation and the orchestration of a trajectory tracking control simulation are given in detail and the overall tracking performance is validated.     

基于LMI可靠跟踪控制器设计

针对具有不确定性的线性定常系统,提出了考虑执行器故障的可靠跟踪控制器设计问 题.在更一般、更实际的执行器故障模型的基础上,给出了系统输出信号渐近跟踪参考输入信 号可靠跟踪控制存在的充分条件.通过求解线性矩阵不等式(LMI)完成状态反馈可靠跟踪控制 器设计.利用仿真数例验证了文中提出的设计方法的可行性,并且通过对可靠跟踪控制系统与 不考虑故障的标准跟踪控制系统的比较,进一步说明对系统进行可靠跟踪控制的必要性.     

Optimal LQ feedforward tracking with preview: Practical design for rigid body motion control

For reference-tracking motion control, preview-based linear quadratic (LQ) design methods provide an effective means to balance tracking performance with available actuation capacity. This paper considers a control structure for which the optimal feedforward controller is independent of the feedback controller. In this way, explicit implementation formulas for feedforward controllers are derived that can be applied to a range of rigid-body motion systems. Key aspects of the optimal LQ solutions are identified, particularly how the choice of design weightings affect steady-state error for polynomial tracking. A redesign procedure for finite preview-time is proposed that preserves exact polynomial tracking properties and control bandwidth of the optimal solutions. Comparative experimental results are presented for a motor-driven linear motion stage.     

Linear learning control of robot motion

For the trajectory following problem of a robot manipulator, a new linear learning control law, consisting of the conventional proportional-integral-differential (PID) control law, with respect to position tracking error, and an iterative learning term is provided. The learning part is a linear feedback control of position, velocity, and acceleration errors (PDD²). It has been shown that, under the proposed learning control, the position, velocity, and acceleration tracking errors are asymptotically stable in the presence of highly nonlinear dynamics. The proposed control is robust in the sense that exact knowledge about nonlinear dynamics is not required except for the bounding functions on their magnitudes. Further, neither is linear approximation of nonlinear dynamics nor repeatability of robot motion required.     

局部对称积分型迭代学习控制

提出了一个新的迭代学习控制（ＩＬＣ）更新律用于连续线性系统的有限时间区间跟踪控制,迭代学习控制作为一个前馈控制,迭代学习控制作为一个前馈控制器加在已有的反馈控制器之上,对于上倥 的反馈控制信号作局部对称积分,所提出的迭代学习控制更新律具备较简单的形式且仅含有两个设计参数,即：学习增益和局部积分的区间长度,给出了收敛性分析以及设计步骤。     

基于反馈控制的迭代学习控制器设计

针对具有不确定项或干扰项的重复非线性时变系统,提出了基于反馈控制的迭代学习控制器,其中迭代学习控制器设计为高阶PD型,它以前馈的形式作用于对象,在满足一定的收敛性条件下,证明了该控制器的跟踪误差界是系统初始状态误差界和系统输出干扰项界的线性函数,同时改变反馈增益可以调整系统的最终跟踪误差界,仿真与实验均表明了该方法的有效性。     

A globally stable saturated desired compensation adaptive robust control for linear motor systems with comparative experiments

The recently proposed saturated adaptive robust controller is integrated with desired trajectory compensation to achieve global stability with much improved tracking performance. The algorithm is tested on a linear motor drive system which has limited control effort and is subject to parametric uncertainties, unmodeled nonlinearities, and external disturbances. Global stability is achieved by employing back-stepping design with bounded (virtual) control input in each step. A guaranteed transient performance and final tracking accuracy is achieved by incorporating the well-developed adaptive robust controller with effective parameter identifier. Signal noise that affects the adaptation function is alleviated by replacing the noisy velocity signal with the cleaner position feedback. Furthermore, asymptotic output tracking can be achieved when only parametric uncertainties are present.     

Control of an Industrial Robot using Acceleration Feedback

A controller using acceleration feedback has been applied to a flexible robot for which the position and velocity of the load are not measured. It is shown that acceleration feedback allows an exact tracking of the motor position, irrespective of the non-linear flexibilities of the axes and of the measurement disturbances. This easy-to-tune algorithm whose main control parameters are the modal masses of the motor and load part, and only consists of a positive acceleration feedback plus a PD controller, has been validated on an industrial robot with orthogonal axes.     

T–S fuzzy model-based tracking control of a one-dimensional manipulator actuated by pneumatic artificial muscles

Pneumatic artificial muscle (PAM) has highly nonlinear and time-varying behavior due to gas compression and nonlinear elasticity of the bladder containers. Hence, it is difficult to achieve excellent tracking performance when using classical control methods. This study proposes a Takagi–Sugeno (T–S) fuzzy model-based control for improving control performance. The proposed approach decomposes the model of a nonlinear system into a set of linear subsystems. This allows, the T–S fuzzy model-based controller to use simple linear control techniques providing a systematic framework for the design of a state feedback controller. Stability analysis is carried out using Lyapunov direct method. The powerful LMI Toolbox in MATLAB is employed to solve linear matrix inequalities (LMIs) to obtain the controller gains. Experimental results verified that the proposed controller can achieve excellent tracking performance under different disturbances.     

基于神经网络的水下机器人三维航迹跟踪控制

本文研究了水下机器人三维航迹跟踪控制问题.在充分考虑了模型中不确定水动力系数和外界海流干扰的基础上,提出了基于神经网络的自适应输出反馈控制方法.控制器由3部分组成:基于动态补偿器的输出反馈控制项、神经网络自适应控制项和鲁棒控制项.神经网络所需的自适应学习信号由线性观测器提供.基于Lyapunov稳定性理论证明了控制系统的稳定性.最后针对某AUV进行了空间三维航迹跟踪控制仿真实验,结果表明设计的控制器可以较好地克服时变非线性水动力阻尼对系统的影响,并对外界海流干扰有较好的抑制作用,可以实现三维航迹的精确跟踪.     

Iterative Learning Control Via Weighted Local-Symmetrical-Integration

A new iterative learning control (ILC) updating law is proposed for tracking control of continuous linear system over a finite time interval. The ILC is applied as a feedforward controller to the existing feedback controller. By using the weighted local symmetrical integral (WLSI) of feedback control signal of previous iteration, the ILC updating law takes a simple form with only two design parameters: the learning gain and the range of local integration. Convergence analysis is presented together with a design procedure. A set of experimental results are presented to illustrate the effectiveness of the proposed WLSI-ILC scheme.     

Robust iterative learning control for multi-phase batch processes: an average dwell-time method with 2D convergence indexes

In order to cope with system disturbances in multi-phase batch processes with different dimensions, a hybrid robust control scheme of iterative learning control combined with feedback control is proposed in this paper. First, with a hybrid iterative learning control law designed by introducing the state error, the tracking error and the extended information, the multi-phase batch process is converted into a two-dimensional Fornasini–Marchesini (2D-FM) switched system with different dimensions. Second, a switching signal is designed using the average dwell-time method integrated with the related switching conditions to give sufficient conditions ensuring stable running for the system. Finally, the minimum running time of the subsystems and the control law gains are calculated by solving the linear matrix inequalities. Meanwhile, a compound 2D controller with robust performance is obtained, which includes a robust extended feedback control for ensuring the steady-state tracking error to converge rapidly. The application on an injection molding process displays the effectiveness and superiority of the proposed strategy.     

Decoupling and tracking control for elastic joint robots with coupled joint structure

The paper deals with the modeling, identification, and control of a flexible joint robot developed for medical applications at the German Aerospace Center (DLR). In order to design anthropomorphic kinematics, the robot uses a coupled joint structure realized by a differential gearbox, which however leads to strong mechanical couplings inside the coupled joints and must be taken into account. Therefore, a regulation MIMO state feedback controller based on modal analysis is developed for each coupled joint pair, which consists of full state feedback (motor position, link side torque, as well as their derivatives). Furthermore, in order to improve position accuracy and simultaneously keep good dynamic behavior of the MIMO state feedback controller, a cascaded tracking control scheme is proposed, based on the MIMO state feedback controller with additional feedforward terms (desired motor velocity, desired motor acceleration, derivative of the desired torque), which are computed in a computed torque controller and take the whole rigid body dynamics into account. Stability analysis is shown for the complete controlled robot. Finally, experimental results with the DLR medical robot are presented to validate the practical efficiency of the approaches.     

Perfect tracking control of discrete-time quadratic TS fuzzy systems via feedback linearisation

Perfect tracking control is an important and frequently encountered requirement in various industries (e.g. robotic control). We developed a novel systematic framework for designing a fuzzy controller via feedback linearisation to control a class of discrete-time Takagi–Sugeno (TS) fuzzy systems with quadratic rule consequents to achieve such tracking. We established a necessary condition for its local stability and a necessary and sufficient condition for the boundedness of the controller. The feedback linearisation is known to fail to work in certain systems due to the unboundedness of the tracking controller output. To address this issue, we developed a method to check whether any given quadratic TS fuzzy system will cause such a failure. We developed a scheme to ensure that the output of the controller designed for any failure-causing system will be bounded and the resulting controller will attain nearly perfect tracking performance. Applying feedback linearisation to the quadratic fuzzy systems is innovative relative to the literature exclusively dealing with the TS fuzzy systems with linear rule consequents (including our previous results), which are now generalised by the new findings. Two numerical examples are provided to illustrate the effectiveness and utility of our new theoretical results.     

On the design of approximate non‐linear parametric controllers

This paper focuses on the design of non‐linear parametric controllers, around a nominal input/output trajectory of a discrete‐time non‐linear system. The main result provided herein is a relationship between the tracking performance of the closed‐loop control system in the neighbourhood of a nominal trajectory, and some local features (the first‐order linear approximations about the nominal trajectory) of the non‐linear mappings which characterize the plant and the feedback controller. Such a result can be used to predict the dynamic behaviour of the control system, and to reduce the computational complexity of the optimization task associated with the tuning of the parametric feedback controller. Copyright © 2000 John Wiley & Sons, Ltd.     

A structured matrix approach to efficient calculation of LQG repetitive learning controllers in the lifted setting

This article discusses the use of repetitive control for output reference tracking in linear time-varying discrete time systems with both repetitive and non-repetitive noise components. The design of such controllers is formulated as a lifted linear stochastic output feedback problem on which the mature techniques of discrete linear control may be applied. In many modern applications, the large size of the system matrices in such a control problem inhibits the application of standard solvers and optimisation techniques. For linear quadratic Gaussian (LQG) problems, the matrices of the lifted feedback problem can be fitted into the recently developed sequentially semi-separable structure. Innovative numerical solutions are developed that have 𝒪(N) computational complexity (where N is the trial length) in both controller synthesis and implementation, comparable to that of many non-lifted and Fourier transform based learning control methods. Moreover, within this formulation, the system is allowed to vary over the learning cycle, closed-loop stability is guaranteed, and stochastic noise and disturbances are handled in an LQG sense.     

Adaptive recurrent neural network control using a structure adaptation algorithm

This paper proposes an adaptive recurrent neural network control (ARNNC) system with structure adaptation algorithm for the uncertain nonlinear systems. The developed ARNNC system is composed of a neural controller and a robust controller. The neural controller which uses a self-structuring recurrent neural network (SRNN) is the principal controller, and the robust controller is designed to achieve L ² tracking performance with desired attenuation level. The SRNN approximator is used to online estimate an ideal tracking controller with the online structuring and parameter learning algorithms. The structure learning possesses the ability of both adding and pruning hidden neurons, and the parameter learning adjusts the interconnection weights of neural network to achieve favorable approximation performance. And, by the L ² control design technique, the worst effect of approximation error on the tracking error can be attenuated to be less or equal to a specified level. Finally, the proposed ARNNC system with structure adaptation algorithm is applied to control two nonlinear dynamic systems. Simulation results prove that the proposed ARNNC system with structure adaptation algorithm can achieve favorable tracking performance even unknown the control system dynamics function.