Iterative learning control design with high‐order internal model for discrete‐time nonlinear systems

In this paper, a high‐order internal model (HOIM)‐based iterative learning control (ILC) scheme is proposed for discrete‐time nonlinear systems to tackle the tracking problem under iteration‐varying desired trajectories. By incorporating the HOIM that is utilized to describe the variation of desired trajectories in the iteration domain into the ILC design, it is shown that the system output can converge to the desired trajectory along the iteration axis within arbitrarily small error. Furthermore, the learning property in the presence of state disturbances and output noise is discussed under HOIM‐based ILC with an integrator in the iteration axis. Two simulation examples are given to demonstrate the effectiveness of the proposed control method. Copyright © 2016 John Wiley & Sons, Ltd.     

PDα‐type iterative learning control for fractional‐order linear continuous‐time switched systems

For a class of fractional‐order linear continuous‐time switched systems specified by an arbitrary switching rule, this paper proposes a PD^α‐type fractional‐order iterative learning control algorithm. For systems disturbed by bounded measurement noise, the robustness of PD^α‐type algorithm is first discussed in the iteration domain and the tracking performance is analyzed. Next, a sufficient condition for monotone convergence of the algorithm is studied when external noise is absent. The results of analysis and simulation illustrate the feasibility and effectiveness of the proposed control algorithm.     

An E‐HOIM Based Data‐Driven Adaptive TILC of Nonlinear Discrete‐Time Systems for Non‐Repetitive Terminal Point Tracking

This work proposes a new adaptive terminal iterative learning control approach based on the extended concept of high‐order internal model, or E‐HOIM‐ATILC, for a nonlinear non‐affine discrete‐time system. The objective is to make the system state or output at the endpoint of each operation track a desired target value. The target value varies from one iteration to another. Before proceeding to the data‐driven design of the proposed approach, an iterative dynamical linearization is performed for the unknown nonlinear systems by using the gradient of the nonlinear system with regard to the control input as the iteration‐and‐time‐varying parameter vector of the equivalent linear I/O data model. By virtue of the basic idea of the internal model, the inverse of the parameter vector is approximated by a high‐order internal model. The proposed E‐HOIM‐ATILC does not use measurements of any intermediate points except for the control input and terminal output at the endpoint. Moreover, it is data‐driven and needs merely the terminal I/O measurements. By incorporating additional control knowledge from the known portion of the high order internal model into the learning control law, the control performance of the proposed E‐HOIM‐ATILC is improved. The convergence is shown by rigorous mathematical proof. Simulations through both a batch reactor and a coupled tank system demonstrate the effectiveness of the proposed method.     

Sampled‐data iterative learning control for continuous‐time nonlinear systems with iteration‐varying lengths

In this work, sampled‐data iterative learning control (ILC) method is extended to a class of continuous‐time nonlinear systems with iteration‐varying trial lengths. In order to propose a unified ILC algorithm, the tracking errors will be redefined when the trial length is shorter or longer than the desired one. Based on the modified tracking errors, 2 sampled‐data ILC schemes are proposed to handle the randomly varying trial lengths. Sufficient conditions are derived rigorously to guarantee the convergence of the nonlinear system at each sampling instant. To verify the effectiveness of the proposed ILC laws, simulations for a nonlinear system are performed. The simulation results show that if the sampling period is set to be small enough, the convergence of the learning algorithms can be achieved as the iteration number increases.     

一类非线性时变系统的迭代学习控制

针对一类具有任意初态的不确定非线性时变系统,应用校正期望轨迹方法把任意初态问题转换为零初始误差的变期望轨迹的迭代学习控制问题,提出了求解校正期望轨迹的过渡轨迹的计算方法.然后,针对变期望轨迹问题提出了一种新的迭代学习控制算法,在算法中引入了期望轨迹的高阶导数来克服期望轨迹的变化,并通过设计稳定的跟踪误差滑动面来处理系统中非线性时变不确定性.论文给出了相关定理,并应用类Lyapunov方法给出了详细证明.仿真结果表明所提出的算法是有效的,该算法不需要系统的模型结构信息,比自适应迭代学习控制算法具有更宽的适用范围.     

Recursive Direct Learning of Control Efforts for Trajectories with Different Magnitude Scales

Direct learning control (DLC) schemes have been developed recently to address non‐repeatable trajectory tracking problems. Unlike conventional iterative learning schemes, DLC schemes learn a set of unknown basis function vectors which can be used to generate the desired control profile of a new trajectory. DLC schemes use all available trajectory tracking information to obtain the unknown basis function vectors in a Least Squares and pointwise manner. A drawback of DLC is that the inverse matrix calculation is inevitable, which is time consuming and may result in singularities due to the batch processing nature. A Recursive Direct Learning Control method is proposed which learns the basis function vectors meanwhile overcomes the implementation difficulties in DLC schemes. The focus of this paper is on learning the control profile of trajectories with same operation period but different magnitude scales. The recursive learning method makes use of one trajectory information at a time, thus avoids the batch processing. The scheme is first developed for a class of nonlinear time varying systems and then extended to cover more general classes of nonlinear systems including robotic manipulator dynamics. Extensive simulation results on a two‐link robotic model are provided to confirm the features of the proposed algorithm.     

Dynamic intermittent Q‐learning–based model‐free suboptimal co‐design of ‐stabilization

This paper proposes an intermittent model‐free learning algorithm for linear time‐invariant systems, where the control policy and transmission decisions are co‐designed simultaneously while also being subjected to worst‐case disturbances. The control policy is designed by introducing an internal dynamical system to further reduce the transmission rate and provide bandwidth flexibility in cyber‐physical systems. Moreover, a Q‐learning algorithm with two actors and a single critic structure is developed to learn the optimal parameters of a Q‐function. It is shown by using an impulsive system approach that the closed‐loop system has an asymptotically stable equilibrium and that no Zeno behavior occurs. Furthermore, a qualitative performance analysis of the model‐free dynamic intermittent framework is given and shows the degree of suboptimality concerning the optimal continuous updated controller. Finally, a numerical simulation of an unknown system is carried out to highlight the efficacy of the proposed framework.     

Data‐driven high‐order terminal iterative learning control with a faster convergence speed

In this paper, a novel high‐order optimal terminal iterative learning control (high‐order OTILC) is proposed via a data‐driven approach for nonlinear discrete‐time systems with unknown orders in the input and output. The objective is to track the desired values at the endpoint of the operation cycle. The terminal tracking errors over more than one previous iterations are used to enhance the high‐order OTILC's performance with faster convergence. From rigor of the analysis, the monotonic convergence of the terminal tracking error is proved along the iteration direction. More importantly, the condition for a high‐order OTILC to outperform the low‐order ones is first established by this work. The learning gain is not fixed but iteratively updated by using the input and output (I/O) data, which enhances the flexibility of the proposed controller for modifications and expansions. The proposed method is data‐driven in which no explicit models are used except for the input and output data. The applications to a highly nonlinear continuous stirred tank reactor and a highly nonlinear fed‐batch fermentater demonstrate the effectiveness of the proposed high‐order OTILC design.     

Formation control for a class of nonlinear multiagent systems using model‐free adaptive iterative learning

In this paper, the problem of formation control is considered for a class of unknown nonaffine nonlinear multiagent systems under a repeatable operation environment. To achieve the formation objective, the unknown nonlinear agent's dynamic is first transformed into a compact form dynamic linearization model along the iteration axis. Then, a distributed model‐free adaptive iterative learning control scheme is designed to ensure that all agents can keep their desired deviations from the reference trajectory over the whole time interval. The main results are given for the multiagent systems with fixed communication topologies and the extension to the switching topologies case is also discussed. The feature of this design is that formation control can be solved only depending on the input/output data of each agent. An example is given to demonstrate the effectiveness of the proposed method.     

Stochastic Point‐to‐Point Iterative Learning Control Based on Stochastic Approximation

An iterative learning control algorithm with iteration decreasing gain is proposed for stochastic point‐to‐point tracking systems. The almost sure convergence and asymptotic properties of the proposed recursive algorithm are strictly proved. The selection of learning gain matrix is given. An illustrative example shows the effectiveness and asymptotic trajectory properties of the proposed approach.     

H∞ tracking control for linear discrete‐time systems via reinforcement learning

In this paper, the H_∞ tracking control of linear discrete‐time systems is studied via reinforcement learning. By defining an improved value function, the tracking game algebraic Riccati equation with a discount factor is obtained, which is solved by iteration learning algorithms. In particular, Q‐learning based on value iteration is presented for H_∞ tracking control, which does not require the system model information and the initial allowable control policy. In addition, to improve the practicability of algorithm, the convergence analysis of proposed algorithm with a discount factor is given. Finally, the feasibility of proposed algorithms is verified by simulation examples.     

Synchronization of Heterogeneous Multi‐Agent Systems by Adaptive Iterative Learning Control

In this work, under a repeatable control environment, an adaptive iterative learning control method is applied to synchronize a group of uncertain heterogeneous agents. The agent dynamics are modeled by nonlinear equations, which contain both parametric and non‐parametric uncertainties. Furthermore, the uncertainties are assumed to be general nonlinear terms instead of the global Lipschitz functions. The communication among the followers is depicted by an undirected and connected graph, meanwhile, the virtual leader's trajectory is only accessible to a small portion of the followers. The proposed learning rules enable all the followers to learn and handle both parametric and non‐parametric uncertainties based on the local information such that the followers can synchronize their trajectories to the desired one. In comparison with the existing literature, most works assume first or second order nonlinear systems, and perfect initial conditions. In order to mitigate the identical initialization condition, the applicability of alignment condition and initial rectifying action are further explored. In addition, our developed algorithms can be applied to general high order nonlinear systems. Finally, synchronization examples of networked robotic manipulators are presented to demonstrate the effectiveness of the developed methods.     

Iterative Learning Control for Nonlinear Systems: A Bounded‐Error Algorithm

This paper presents a nonlinear iterative learning control (NILC) for nonlinear time‐varying systems. An algorithm of a new strategy for the NILC implementation is proposed. This algorithm ensures that trajectory‐tracking errors of the proposed NILC, when implemented, are bounded by a given error norm bound. A special feature of the algorithm is that the trial‐time interval is finite but not fixed as it is for the other iterative learning algorithms. A sufficient condition for convergence and robustness of the bounded‐error learning procedure is derived. With respect to the bounded‐error and standard learning processes applied to a virtual robot, simulation results are presented in order to verify maximal tracking errors, convergence and applicability of the proposed learning control.     

Online solution of nonlinear two‐player zero‐sum games using synchronous policy iteration

The two‐player zero‐sum (ZS) game problem provides the solution to the bounded L₂‐gain problem and so is important for robust control. However, its solution depends on solving a design Hamilton–Jacobi–Isaacs (HJI) equation, which is generally intractable for nonlinear systems. In this paper, we present an online adaptive learning algorithm based on policy iteration to solve the continuous‐time two‐player ZS game with infinite horizon cost for nonlinear systems with known dynamics. That is, the algorithm learns online in real time an approximate local solution to the game HJI equation. This method finds, in real time, suitable approximations of the optimal value and the saddle point feedback control policy and disturbance policy, while also guaranteeing closed‐loop stability. The adaptive algorithm is implemented as an actor/critic/disturbance structure that involves simultaneous continuous‐time adaptation of critic, actor, and disturbance neural networks. We call this online gaming algorithm ‘synchronous’ ZS game policy iteration. A persistence of excitation condition is shown to guarantee convergence of the critic to the actual optimal value function. Novel tuning algorithms are given for critic, actor, and disturbance networks. The convergence to the optimal saddle point solution is proven, and stability of the system is also guaranteed. Simulation examples show the effectiveness of the new algorithm in solving the HJI equation online for a linear system and a complex nonlinear system. Copyright © 2011 John Wiley & Sons, Ltd.     

High‐precision formation control of nonlinear multi‐agent systems with switching topologies: A learning approach

Arbitrary high precision is considered one of the most desirable control objectives in the relative formation for many networked industrial applications, such as flying spacecrafts and mobile robots. The main purpose of this paper is to present design guidelines of applying the iterative schemes to develop distributed formation algorithms in order to achieve this control objective. If certain conditions are met, then the control input signals can be learned by the developed algorithms to accomplish the desired formations with arbitrary high precision. The systems under consideration are a class of multi‐agent systems under directed networks with switching topologies. The agents have discrete‐time affine nonlinear dynamics, but their state functions do not need to be identical. It is shown that the learning processes resulting from the relative output formation of multi‐agent systems can converge exponentially fast with the increase of the iteration number. In particular, this work induces a distributed algorithm that can simultaneously achieve the desired relative output formation between agents and regulate the movement of multi‐agent formations as desired along the time axis. The illustrative numerical simulations are finally performed to demonstrate the effectiveness and performance of the proposed distributed formation algorithms. Copyright © 2014 John Wiley & Sons, Ltd.     

On iterative learning control design for tracking iteration-varying trajectories with high-order internal model

In this paper, iterative learning control (ILC) design is studied for an iteration-varying tracking problem in which reference trajectories are generated by high-order internal models (HOIM). An HOIM formulated as a polynomial operator between consecutive iterations describes the changes of desired trajectories in the iteration domain and makes the iterative learning problem become iteration varying. The classical ILC for tracking iteration-invariant reference trajectories, on the other hand, is a special case of HOIM where the polynomial renders to a unity coefficient or a special first-order internal model. By inserting the HOIM into P-type ILC, the tracking performance along the iteration axis is investigated for a class of continuous-time nonlinear systems. Time-weighted norm method is utilized to guarantee validity of proposed algorithm in a sense of data-driven control.     

Iterative learning control for nonlinear differential inclusion systems

In this paper, we propose an iterative learning control strategy to track a desired trajectory for a class of uncertain systems governed by nonlinear differential inclusions. By imposing Lipschitz continuous condition on a set‐valued mapping described by a closure of the convex hull of a set and using D‐type and PD‐type updating laws with initial iterative learning, we establish the iterative learning process and give a new convergence analysis with the help of Steiner‐type selector. Finally, numerical examples are provided to verify the effectiveness of the proposed method with suitable selection of set‐valued mappings. An application to the speed control of robotic fish is also given.     

Global asymptotic tracking for nonminimum‐phase nonlinear systems in output feedback form

The problem of global asymptotic tracking by output feedback is studied for a class of nonminimum‐phase nonlinear systems in output feedback form. It is proved that the problem is solvable by an n‐dimensional output feedback controller under the two conditions: (a) the nonminimum‐phase nonlinear system can be rendered minimum‐phase by a virtual output; and (b) the internal dynamics of the nonlinear system driven by a desired signal and its derivatives has a bounded solution trajectory. With the help of a new coordinate transformation, a constructive method is presented for the design of a dynamic output tracking controller. An example is given to validate the proposed output feedback tracking control scheme.     

D‐type iterative learning control for one‐sided Lipschitz nonlinear systems

In this paper, the problem of iterative learning control for a class of nonlinear systems is studied. Here, the nonlinear functions satisfy the one‐sided Lipschitz and quadratically inner‐bounded conditions. For such nonlinear systems, open‐loop and closed‐loop D‐type learning algorithms are adopted, respectively, and furthermore, the convergence conditions of the D‐type learning algorithms are established. It is shown that both algorithms can ensure that the system output converges to the desired trajectory on the whole time interval. Finally, the validity of the presented D‐type learning algorithms is verified by a numerical example.     

Iterative Learning Control for Linear Discrete‐Time Systems with Unknown High‐Order Internal Models: A Time‐Frequency Analysis Approach

This work focuses on the iterative learning control (ILC) for linear discrete‐time systems with unknown initial state and disturbances. First, multiple high‐order internal models (HOIMs) are introduced for the reference, initial state, and disturbances. Both the initial state and disturbance consist of two components, one strictly satisfies HOIM and the other is random bounded. Then, an ILC scheme is constructed according to an augmented HOIM that is the aggregation of all HOIMs. For all known HOIMs, an ILC design criterion is introduced to achieve satisfactory tracking performance based on the 2‐D theory. Next, the case with unknown HOIMs is discussed, where a time‐frequency‐analysis (TFA)‐based ILC algorithm is proposed. In this situation, it is shown that the tracking error inherits the unknown augmented HOIM that is an aggregation of all unknown HOIMs. Then, a TFA‐based method, e.g., the short‐time Fourier transformation (STFT), is employed to identify the unknown augmented HOIM, where the STFT could ignore the effect of the random bounded initial state and disturbances. A new ILC law is designed for the identified unknown augmented HOIM, which has the ability to reject the unknown the initial state and disturbances that strictly satisfy HOIMs. Finally, a gantry robot system with iteration‐invariant or slowly‐varying frequencies is given to illustrate the efficiency of the proposed TFA‐based ILC algorithm.