Variants of the Uzawa method for saddle point problem

In this paper, we first propose three variants of the Uzawa method for solving the saddle point problem, and then we provide convergence results for the three proposed methods. Numerical experiments show that our proposed methods with three parameters perform about twice as fast as the GSOR (Generalized SOR) method with two parameters since the proposed methods have less workload per iteration than the GSOR.     

Preconditioned accelerated generalized successive overrelaxation method for solving complex symmetric linear systems

In this paper, by adopting the preconditioned technique for the accelerated generalized successive overrelaxation method (AGSOR) proposed by Edalatpour et al. (2015), we establish the preconditioned AGSOR (PAGSOR) iteration method for solving a class of complex symmetric linear systems. The convergence conditions, optimal iteration parameters and corresponding optimal convergence factor of the PAGSOR iteration method are determined. Besides, we prove that the spectral radius of the PAGSOR iteration method is smaller than that of the AGSOR one under proper restrictions, and its optimal convergence factor is smaller than that of the preconditioned symmetric block triangular splitting (PSBTS) one put forward by Zhang et al. (2018) recently. The spectral properties of the preconditioned PAGSOR matrix are also proposed. Numerical experiments illustrate the correctness of the theories and the effectiveness of the proposed iteration method and the preconditioner for the generalized minimal residual (GMRES) method.     

Two-parameter GSOR method for the augmented system

A new iterative method is given for the augmented system of equations. Similarly to the GSOR and SOR-like methods, the new method involves two iteration parameters and one preconditioning matrix. The convergence analysis and the determination of the optimum parameters are studied first. Then the explicit formulae for the optimum parameters and the associated spectral radius are derived. Finally, numerical computations are presented which show clearly that the new method has a very good numerical performance and is much faster than the SOR-like method.     

GSOR Method for the Equality Constrained Least Squares Problems and the Generalized Least Squares Problems

In a recent paper [4], Li et al . gave a generalized successive overrelaxation (GSOR) method for the least squares problems. In this paper, we show that the GSOR method can be applied to the equality constrained least squares (LSE) problems and the generalized least squares (GLS) problems.     

广义修正HSS迭代法的超松弛加逮

通过推广修正埃尔米特和反埃尔米特（MHSS）迭代法,我们进一步得到了求解大型稀疏非埃尔米特正定线性方程组的广义MHSS（GMHSS）迭代法．基于不动点方程,我们还将超松弛（SOR）技术运用到了GMHSS迭代法,得到了关于GMHSS迭代法的SOR加速,并分析了它的收敛性．数值算例表明,SOR技术能够大大提高加速GMHSS迭代法的收敛效率．     

Preconditioned Conjugate Gradient Method and Generalized Successive Overrelaxation Method for the Least Squares Problems

In a recent paper [4], Li et al . gave a generalized successive overrelaxation (GSOR) method for the least squares problems. In this paper, the connection between the GSOR method and the preconditioned conjugate gradient (PCG) method for the normal equations is investigated. It is shown that the PCG method is at least as fast as the GSOR method. Numerical examples demonstrates that the PCG method is much faster than the GSOR method.     

An efficient two-step iterative method for solving a class of complex symmetric linear systems

In this paper, a new two-step iterative method called the two-step parameterized (TSP) iteration method for a class of complex symmetric linear systems is developed. We investigate its convergence conditions and derive the quasi-optimal parameters which minimize the upper bound of the spectral radius of the iteration matrix of the TSP iteration method. Meanwhile, some more practical ways to choose iteration parameters for the TSP iteration method are proposed. Furthermore, comparisons of the TSP iteration method with some existing ones are given, which show that the upper bound of the spectral radius of the TSP iteration method is smaller than those of the modified Hermitian and skew-Hermitian splitting (MHSS), the preconditioned MHSS (PMHSS), the combination method of real part and imaginary part (CRI) and the parameterized variant of the fixed-point iteration adding the asymmetric error (PFPAE) iteration methods proposed recently. Inexact version of the TSP iteration (ITSP) method and its convergence properties are also presented. Numerical experiments demonstrate that both TSP and ITSP are effective and robust when they are used either as linear solvers or as matrix splitting preconditioners for the Krylov subspace iteration methods and they have comparable advantages over some known ones for the complex symmetric linear systems.     

Online solution of nonlinear two‐player zero‐sum games using synchronous policy iteration

The two‐player zero‐sum (ZS) game problem provides the solution to the bounded L₂‐gain problem and so is important for robust control. However, its solution depends on solving a design Hamilton–Jacobi–Isaacs (HJI) equation, which is generally intractable for nonlinear systems. In this paper, we present an online adaptive learning algorithm based on policy iteration to solve the continuous‐time two‐player ZS game with infinite horizon cost for nonlinear systems with known dynamics. That is, the algorithm learns online in real time an approximate local solution to the game HJI equation. This method finds, in real time, suitable approximations of the optimal value and the saddle point feedback control policy and disturbance policy, while also guaranteeing closed‐loop stability. The adaptive algorithm is implemented as an actor/critic/disturbance structure that involves simultaneous continuous‐time adaptation of critic, actor, and disturbance neural networks. We call this online gaming algorithm ‘synchronous’ ZS game policy iteration. A persistence of excitation condition is shown to guarantee convergence of the critic to the actual optimal value function. Novel tuning algorithms are given for critic, actor, and disturbance networks. The convergence to the optimal saddle point solution is proven, and stability of the system is also guaranteed. Simulation examples show the effectiveness of the new algorithm in solving the HJI equation online for a linear system and a complex nonlinear system. Copyright © 2011 John Wiley & Sons, Ltd.     

关于非对称代数Riccati方程的ALI迭代算法收敛速率的讨论

交替线性化隐式迭代法(ALI)是求非对称代数Riccati方程最小非负解的一种十分有效的算法.其中所包含的一个参数能够显著影响其收敛速率.本文将讨论该参数的选择以及使收敛达到最快的参数最优值.     

Modified alternating direction-implicit iteration method for linear systems from the incompressible Navier–Stokes equations

In order to solve the large sparse systems of linear equations arising from numerical solutions of two-dimensional steady incompressible viscous flow problems in primitive variable formulation, Ran and Yuan [On modified block SSOR iteration methods for linear systems from steady incompressible viscous flow problems, Appl. Math. Comput. 217 (2010), pp. 3050–3068] presented the block symmetric successive over-relaxation (BSSOR) and the modified BSSOR iteration methods based on the special structures of the coefficient matrices. In this study, we present the modified alternating direction-implicit (MADI) iteration method for solving the linear systems. Under suitable conditions, we establish convergence theorems for the MADI iteration method. In addition, the optimal parameter involved in the MADI iteration method is estimated in detail. Numerical experiments show that the MADI iteration method is a feasible and effective iterative solver.     

非线性非仿射离散时间系统的两阶段最优迭代学习控制

On the basis of a new dynamic linearization technology along the iteration axis,a dual-stage optimal iterative learning control is presented for nonlinear and non-affine discrete-time systems.Dual-stage indicates that two optimal learning stages are designed respectively to improve control input sequence and the learning gain iteratively.The main feature is that the controller design and convergence analysis only depend on the I/O data of the dynamical system.In other words,we can easily select the control parameters without knowing any other knowledge of the system.Simulation study illustrates the geometrical convergence of the presented method along the iteration axis,in which an example of freeway traffic iterative learning control is noteworthy for its intrinsic engineering importance.     

Stability and monotone convergence of generalised policy iteration for discrete-time linear quadratic regulations

In this paper, we analyse the convergence and stability properties of generalised policy iteration (GPI) applied to discrete-time linear quadratic regulation problems. GPI is one kind of the generalised adaptive dynamic programming methods used for solving optimal control problems, and is composed of policy evaluation and policy improvement steps. To analyse the convergence and stability of GPI, the dynamic programming (DP) operator is defined. Then, GPI and its equivalent formulas are presented based on the notation of DP operator. The convergence of the approximate value function to the exact one in policy evaluation is proven based on the equivalent formulas. Furthermore, the positive semi-definiteness, stability, and the monotone convergence (PI-mode and VI-mode convergence) of GPI are presented under certain conditions on the initial value function. The online least square method is also presented for the implementation of GPI. Finally, some numerical simulations are carried out to verify the effectiveness of GPI as well as to further investigate the convergence and stability properties.     

非线性非仿射离散时间系统的两阶段最优迭代学习控制

针对非仿射非线性离散时间系统, 基于一种新的沿迭代轴的动态线性化技术, 提出了双层最优迭代学习控制算法. 双层意味着分别设计了两个最优学习层, 迭代的改进控制输入序列和学习增益. 其主要特点是控制器的设计和收敛性分析只依赖于动态系统的 I/O 数据. 换句话说, 不需要知道系统的任何其他信息就可以很容易的选取控制器参数. 仿真研究表明了提出的算法沿迭代轴具有几何收敛性, 这一特点在快速路交通迭代学习控制中具有重要的工程意义.     

Two-parameter generalized Hermitian and skew-Hermitian splitting iteration method

It is known that the Hermitian and skew-Hermitian splitting (HSS) iteration method is an efficient solver for non-Hermitian positive-definite linear system of equations. Benzi [A generalization of the Hermitian and skew-Hermitian splitting iteration, SIAM J. Matrix Anal. Appl. 31 (2009), pp. 360–374] proposed a generalized HSS (GHSS) iteration method. In this paper, we present a two-parameter version of the GHSS (TGHSS) method and investigate its convergence properties. To show the effectiveness of the proposed method the TGHSS iteration method is applied to image restoration and convection–diffusion problems and the results are compared with those of the HSS and GHSS methods.     

Efficient preconditioned NHSS iteration methods for solving complex symmetric linear systems

Based on the new HSS (NHSS) iteration method introduced by Pour and Goughery (2015), we propose a preconditioned variant of NHSS (P*NHSS) and an efficient parameterized P*NHSS (PPNHSS) iteration methods for solving a class of complex symmetric linear systems. The convergence properties of the P*NHSS and the PPNHSS iteration methods show that the iterative sequences are convergent to the unique solution of the linear system for any initial guess when the parameters are properly chosen. Moreover, we discuss the quasi-optimal parameters which minimize the upper bounds for the spectral radius of the iteration matrices. Numerical results show that the PPNHSS iteration method is superior to several iteration methods whether the experimental optimal parameters are used or not.     

Data‐driven–based attitude control of combined spacecraft with noncooperative target

In this paper, the attitude control of combined spacecraft with noncooperative target is studied. For the linearized system of the attitude control system, which possesses uncertainties on both its control and system matrices caused by the noncooperative target, this paper proposes an approximated iteration method to obtain the optimal controller such that the closed‐loop system possesses a prescribed convergence rate. An explicit stabilizing gain is established to initiate the iteration method. Based on the proposed iteration method and the initial stabilizing gain, a data‐driven algorithm is proposed to obtain the optimal controller for the attitude control system without using the system parameter information. A simulation is given to verify the effectiveness of the proposed method.     

Online policy iteration algorithm for optimal control of linear hyperbolic PDE systems

In this paper, the linear quadratic (LQ) optimal control problem is considered for a class of linear distributed parameter systems described by first-order hyperbolic partial differential equations (PDEs). Reinforcement learning (RL) technique is introduced for adaptive optimal control design from the design-then-reduce (DTR) framework. Initially, a policy iteration (PI) algorithm is proposed, which learns the solution of the space-dependent Riccati differential equation (SDRDE) online without requiring the internal system dynamics of the PDE system. To prove its convergence, the PI algorithm is shown to be equivalent to an iterative procedure of a sequence of space-dependent Lyapunov differential equations (SDLDEs). Then, the convergence is established by showing that the solutions of SDLDEs are a monotone non-increasing sequence that converges to the solution of the SDRDE. For implementation purpose, an online least-square method is developed for the approximation of the solutions of the SDLDEs. Finally, the proposed design method is applied to the distributed control of a steam-jacketed tubular heat exchanger to illustrate its effectiveness.     

Two-loop reinforcement learning algorithm for finite-horizon optimal control of continuous-time affine nonlinear systems

This article proposes three novel time-varying policy iteration algorithms for finite-horizon optimal control problem of continuous-time affine nonlinear systems. We first propose a model-based time-varying policy iteration algorithm. The method considers time-varying solutions to the Hamiltonian–Jacobi–Bellman equation for finite-horizon optimal control. Based on this algorithm, value function approximation is applied to the Bellman equation by establishing neural networks with time-varying weights. A novel update law for time-varying weights is put forward based on the idea of iterative learning control, which obtains optimal solutions more efficiently compared to previous works. Considering that system models may be unknown in real applications, we propose a partially model-free time-varying policy iteration algorithm that applies integral reinforcement learning to acquiring the time-varying value function. Moreover, analysis of convergence, stability, and optimality is provided for every algorithm. Finally, simulations for different cases are given to verify the convenience and effectiveness of the proposed algorithms.     

On parameterized matrix splitting preconditioner for the saddle point problems

In this paper, we present a parameterized matrix splitting (PMS) preconditioner for the large sparse saddle point problems. The preconditioner is based on a parameterized splitting of the saddle point matrix, resulting in a fixed-point iteration. The convergence theorem of the new iteration method for solving large sparse saddle point problems is proposed by giving the restrictions imposed on the parameter. Based on the idea of the parameterized splitting, we further propose a modified PMS preconditioner. Some useful properties of the preconditioned matrix are established. Numerical implementations show that the resulting preconditioner leads to fast convergence when it is used to precondition Krylov subspace iteration methods such as generalized minimal residual method.     

Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design

This paper addresses the model-free nonlinear optimal control problem based on data by introducing the reinforcement learning (RL) technique. It is known that the nonlinear optimal control problem relies on the solution of the Hamilton–Jacobi–Bellman (HJB) equation, which is a nonlinear partial differential equation that is generally impossible to be solved analytically. Even worse, most practical systems are too complicated to establish an accurate mathematical model. To overcome these difficulties, we propose a data-based approximate policy iteration (API) method by using real system data rather than a system model. Firstly, a model-free policy iteration algorithm is derived and its convergence is proved. The implementation of the algorithm is based on the actor–critic structure, where actor and critic neural networks (NNs) are employed to approximate the control policy and cost function, respectively. To update the weights of actor and critic NNs, a least-square approach is developed based on the method of weighted residuals. The data-based API is an off-policy RL method, where the “exploration” is improved by arbitrarily sampling data on the state and input domain. Finally, we test the data-based API control design method on a simple nonlinear system, and further apply it to a rotational/translational actuator system. The simulation results demonstrate the effectiveness of the proposed method.