Online optimal control of nonlinear discrete-time systems using approximate dynamic programming

In this paper,the optimal control of a class of general affine nonlinear discrete-time(DT) systems is undertaken by solving the Hamilton Jacobi-Bellman(HJB) equation online and forward in time.The proposed approach,referred normally as adaptive or approximate dynamic programming(ADP),uses online approximators(OLAs) to solve the infinite horizon optimal regulation and tracking control problems for affine nonlinear DT systems in the presence of unknown internal dynamics.Both the regulation and tracking contro...     

Legendre–Galerkin spectral approximation for a nonlocal elliptic Kirchhof-type problem

We propose a numerical scheme to obtain an approximate solution of a nonlocal elliptic Kirchhof-type problem. We first reduce the problem to a nonlinear finite dimensional system by a Legendre–Galerkin spectral method and then solve it by an iterative process. Convergence of the iterative process and an error estimation of the approximate solution is provided. Numerical experiments are conducted to illustrate the performance of the proposed method.     

APPLICATION OF A MODEL‐BASED ITERATIVE LEARNING TECHNIQUE TO TRACKING CONTROL OF A PIEZOELECTRIC SYSTEM

This paper presents the application of a model‐based iterative learning control technique to position tracking of a piezoelectric system. Identification of the closed‐loop piezoelectric system was undertaken first, and then an iterative learning control methodology based on the identified model was implemented for dynamic tracking control of the actuator. The methodology differs from the conventional iterative learning scheme in that it takes into account the difference of the one‐step‐ahead predictive input between two successive iterations. The methodology compensates for the predictive input difference as well as the causal error in the previous iteration. The results of the experiments prove the excellence of this technique for precision tracking control of the piezoelectric actuator.     

离散非线性时变系统开闭环PI型迭代学习控制律及其收敛性

对于具有重复运动性质的对象,迭代学习控制是一种有效的控制方法.针对一类 离散非线性时变系统在有限时域上的精确轨迹跟踪问题,提出了一种开闭环PI型迭代学习 控制律.这种迭代律同时利用系统当前的跟踪误差和前次迭代控制的跟踪误差修正控制作 用.给出了所提出的学习控制律收敛的充分必要条件,并采用归纳法进行了证明.最后用仿真 结果对收敛条件进行了验证.     

An Iterative Method for Optimal Feedback Control and Generalized HJB Equation

In this paper, a new iterative method is proposed to solve the generalized Hamilton-Jacobi-Bellman (GHJB) equation through successively approximate it. Firstly, the GHJB equation is converted to an algebraic equation with the vector norm, which is essentially a set of simultaneous nonlinear equations in the case of dynamic systems. Then, the proposed algorithm solves GHJB equation numerically for points near the origin by considering the linearization of the non-linear equations under a good initial control guess. Finally, the procedure is proved to converge to the optimal stabilizing solution with respect to the iteration variable. In addition, it is shown that the result is a closed-loop control based on this iterative approach. Illustrative examples show that the update control laws will converge to optimal control for nonlinear systems.      

Neuro-optimal tracking control for a class of discrete-time nonlinear systems via generalized value iteration adaptive dynamic programming approach

In this paper, a novel value iteration adaptive dynamic programming (ADP) algorithm, called “generalized value iteration ADP” algorithm, is developed to solve infinite horizon optimal tracking control problems for a class of discrete-time nonlinear systems. The developed generalized value iteration ADP algorithm permits an arbitrary positive semi-definite function to initialize it, which overcomes the disadvantage of traditional value iteration algorithms. Convergence property is developed to guarantee that the iterative performance index function will converge to the optimum. Neural networks are used to approximate the iterative performance index function and compute the iterative control policy, respectively, to implement the iterative ADP algorithm. Finally, a simulation example is given to illustrate the performance of the developed algorithm.     

基于Jacobi方法的线性离散时间系统的迭代学习控制

提出线性离散时间系统基于Jacobi方法的迭代学习控制问题.通过构建线性迭代学习控制问题与线性方程组之间的联系,将Jacobi方法引入到迭代学习控制中,并由此构建得到迭代学习控制律.借助于矩阵运算,证明这种学习律能使得系统的输出跟踪误差经有限次迭代后为零.数值例子说明了算法的可适用性.     

Output regulation for linear distributed parameter systems

This work extends the geometric theory of output regulation to linear distributed parameter systems with bounded input and output operator, in the case when the reference signal and disturbances are generated by a finite dimensional exogenous system. In particular it is shown that the full state feedback and error feedback regulator problems are solvable, under the standard assumptions of stabilizability and detectability, if and only if a pair of regulator equations is solvable. For linear distributed parameter systems this represents an extension of the geometric theory of output regulation developed in Francis (1997) and Isidori and Byrnes (1990). We also provide simple criteria for solvability of the regulator equations based on the eigenvalues of the exosystem and the system transfer function. Examples are given of periodic tracking, set point control, and disturbance attenuation for parabolic systems and periodic tracking for a damped hyperbolic system     

A feedforward–feedback controller for infinite‐dimensional systems and regulation of bounded uniformly continuous signals

We design a controller for infinite‐dimensional linear systems (with bounded control, observation and feedthrough operators) which, under certain assumptions, achieves asymptotic tracking of arbitrary bounded uniformly continuous reference signals in the presence of disturbances. The proposed controller is of feedforward–feedback type: The dynamic feedback part is used to stabilize the closed‐loop system consisting of the plant and the controller, whereas the feedforward part is tuned using the regulator equations to achieve the regulation of desired signals. We also completely solve the regulator equations for SISO systems, and we discuss robustness properties of the proposed controller. A useful feature in our design is that the feedforward part of the controller can be designed independently of the feedback part. This automatically leads to a degree of robustness in the stabilizing part of the controller, which is not present in the existing state feedback controllers solving the same output regulation problem. Copyright © 2006 John Wiley & Sons, Ltd.     

Stable iterative adaptive dynamic programming algorithm with approximation errors for discrete-time nonlinear systems

In this paper, a novel iterative adaptive dynamic programming (ADP) algorithm is developed to solve infinite horizon optimal control problems for discrete-time nonlinear systems. When the iterative control law and iterative performance index function in each iteration cannot be accurately obtained, it is shown that the iterative controls can make the performance index function converge to within a finite error bound of the optimal performance index function. Stability properties are presented to show that the system can be stabilized under the iterative control law which makes the present iterative ADP algorithm feasible for implementation both on-line and off-line. Neural networks are used to approximate the iterative performance index function and compute the iterative control policy, respectively, to implement the iterative ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the present method.     

Prescribed performance adaptive neural output feedback dynamic surface control for a class of strict‐feedback uncertain nonlinear systems with full state constraints and unmodeled dynamics

This paper studies the output feedback tracking control problem for a class of strict‐feedback uncertain nonlinear systems with full state constraints and unmodeled dynamics using a prescribed performance adaptive neural dynamic surface control design approach. A nonlinear mapping technique is employed to address the state constraints. Radial basis function neural networks are utilized to approximate the unknown nonlinear functions. The unmodeled dynamics is addressed by introducing an available dynamic signal. Subsequently, we construct the controller and parameter adaptive laws using a backstepping technique. Based on Lyapunov stability theory, it is shown that all signals in the closed‐loop system are semiglobally uniformly ultimately bounded and that the tracking error always remains within the prescribed performance bound. Simulation results are presented to demonstrate the effectiveness of the proposed control scheme.     

Mixed ℋ︁2/ℋ︁∞ nonlinear filtering

In this paper, we consider the mixed ??₂/??_∞ filtering problem for affine nonlinear systems. Sufficient conditions for the solvability of this problem with a finite‐dimensional filter are given in terms of a pair of coupled Hamilton–Jacobi–Isaacs equations (HJIEs). For linear systems, it is shown that these conditions reduce to a pair of coupled Riccati equations similar to the ones for the control case. Both the finite‐horizon and the infinite‐horizon problems are discussed. Simulation results are presented to show the usefulness of the scheme, and the results are generalized to include other classes of nonlinear systems. Copyright © 2008 John Wiley & Sons, Ltd.     

Approximate finite‐time control for a class of uncertain nonlinear systems with dynamic compensation

In this work, a new robust nonlinear feedback control method with dynamic active compensation is proposed; the active control method has been applied to an integral series of finite‐time single‐input single‐output nonlinear system with uncertainty. In further tracking the closed‐loop stability and nonlinear uncertainty, an extended state observer has been employed to solve the immeasurability and nonlinear uncertainty within a nonlinear system. A singular perturbation theory has been used to solve for the finite‐time stability of the closed‐loop system; furthermore, numerical simulations showed that the robust nonlinear feedback controller tracked the uncertainty in a nonlinear Duffing‐type oscillator and has proven the effectiveness of the approximate finite‐time control strategy proposed. By using an approximate finite‐time control approach with active compensation, the uncertainty in a nonlinear system could be accurately estimated and controlled. Copyright © 2017 John Wiley & Sons, Ltd.     

Operator-splitting methods for the 2D convective Cahn–Hilliard equation

In this work, we present operator-splitting methods for the two-dimensional nonlinear fourth-order convective Cahn–Hilliard equation with specified initial condition and periodic boundary conditions. The full problem is split into hyperbolic, nonlinear diffusion and linear fourth-order problems. We prove that the semi-discrete approximate solution obtained from the operator-splitting method converges to the weak solution. Numerical methods are then constructed to solve each sub equations sequentially. The hyperbolic conservation law is solved by efficient finite volume methods and dimensional splitting method, while the one-dimensional hyperbolic conservation laws are solved using front tracking algorithm. The front tracking method is based on the exact solution and hence has no stability restriction on the size of the time step. The nonlinear diffusion problem is solved by a linearized implicit finite volume method, which is unconditionally stable. The linear fourth-order equation is solved using a pseudo-spectral method, which is based on an exact solution. Finally, some numerical experiments are carried out to test the performance of the proposed numerical methods.     

Data-Based Optimal Tracking of Autonomous Nonlinear Switching Systems

In this paper,a data-based scheme is proposed to solve the optimal tracking problem of autonomous nonlinear switching systems.The system state is forced to track the reference signal by minimizing the performance function.First,the problem is transformed to solve the corresponding Bellman optimality equation in terms of the Q-function(also named as action value function).Then,an iterative algorithm based on adaptive dynamic programming(ADP)is developed to find the optimal solution which is totally based on sampled data.The linear-in-parameter(LIP)neural network is taken as the value function approximator.Considering the presence of approximation error at each iteration step,the generated approximated value function sequence is proved to be boundedness around the exact optimal solution under some verifiable assumptions.Moreover,the effect that the learning process will be terminated after a finite number of iterations is investigated in this paper.A sufficient condition for asymptotically stability of the tracking error is derived.Finally,the effectiveness of the algorithm is demonstrated with three simulation examples.     

Reinforcement Learning Based Controller Synthesis for Flexible Aircraft Wings

Aeroelastic study of flight vehicles has been a subject of great interest and research in the last several years. Aileron reversal and flutter related problems are due in part to the elasticity of a typical airplane. Structural dynamics of an aircraft wing due to its aeroelastic nature are characterized by partial differential equations. Controller design for these systems is very complex as compared to lumped parameter systems defined by ordinary differential equations. In this paper, a stabilizing statefeedback controller design approach is presented for the heave dynamics of a wing-fuselage model. In this study, a continuous actuator in the spatial domain is assumed. A control methodology is developed by combining the technique of "proper orthogonal decomposition" and approximate dynamic programming. The proper orthogonal decomposition technique is used to obtain a low-order nonlinear lumped parameter model of the infinite dimensional system. Then a near optimal controller is designed using the single-network-adaptive-critic technique. Furthermore, to add robustness to the nominal single-network-adaptive-critic controller against matched uncertainties, an identifier based adaptive controller is proposed. Simulation results demonstrate the effectiveness of the single-network-adaptive-critic controller augmented with adaptive controller for infinite dimensional systems.      

Approximating solutions for the systems of strongly accretive operator equations

The purpose of this paper is to study a strong convergence of multi-step iterative scheme to a common fixed point for a finite family of strongly pseudo-contractive mappings. As a consequence, the strong convergence theorems for the modified multi-step iterative sequence to a solution of systems of strongly accretive operator equations are obtained in real uniformly smooth Banach spaces. The results presented in this paper improve and extend the corresponding Xu [Y. Xu, Ishikawa and Mann iterative processes with errors for nonlinear strongly accretive operator equations, J. Math. Anal. Appl. 224 (1998) 91–101] and Noor, Rassias and Huang [M.A. Noor, T.M. Rassias, Z. Huang, Three-step iterations for nonlinear accretive operator equations, J. Math. Anal. Appl. 274 (2002) 59–68], and many others.     

Identification of finite dimensional models of infinite dimensional dynamical systems

The identification of finite dimensional discrete-time models of deterministic linear and nonlinear infinite dimensional systems from pointwise observations is investigated. The input and output observations are used to construct finite dimensional approximations of the solution and the forcing function which are expanded in terms of a finite element basis. An algorithm to determine a minimal basis to approximate the data is introduced. Subsequently, the resulting coordinate vectors are used to identify a finite dimensional discrete-time model. Theoretical results concerning the existence, stability and convergence of the finite dimensional representation are established. Numerical results involving identification of finite dimensional models for both linear and nonlinear infinite dimensional systems are presented.     

Convergence Acceleration for Hyperbolic Systems Using Semicirculant Approximations

The iterative solution of systems of equations arising from systems of hyperbolic, time-independent partial differential equations (PDEs) is studied. The PDEs are discretized using a finite volume or finite difference approximation on a structured grid. A convergence acceleration technique where a semicirculant approximation of the spatial difference operator is employed as preconditioner is considered. The spectrum of the preconditioned coefficient matrix is analyzed for a model problem. It is shown that, asymptotically, the time step for the forward Euler method could be chosen as a constant, which is independent of the number of grid points and the artificial viscosity parameter. By linearizing the Euler equations around an approximate solution, a system of linear PDEs with variable coefficients is formed. When utilizing the semicirculant (SC) preconditioner for this problem, which has properties very similar to the full nonlinear equations, numerical experiments show that the favorable convergence properties hold also here. We compare the results for the SC method to those of a multigrid (MG) scheme. The number of iterations and the arithmetic complexities are considered, and it is clear that the SC method is more efficient for the problems studied. Also, the MG scheme is sensitive to the amount of artificial dissipation added, while the SC method is not.     

Output feedback stabilization and approximate and restricted tracking for a class of cascaded systems

In this note, we discuss the problems of output feedback stabilization for a class of cascaded systems and of (approximate and restricted) output regulation for general nonlinear systems. It is shown that (global) output feedback stabilization for a class of systems in feedforward form can be achieved with a dynamic feedback law, yielding bounded control, and relying on the introduction of a reduced-order observer. The above result, together with standard tools borrowed from the output regulator theory, is instrumental to construct dynamic control laws achieving (approximate) disturbance rejection and output tracking in the presence of (small) disturbance/reference signals generated by a known exosystem.