Characterizing all optimal controls for an indefinite stochasticlinear quadratic control problem

This paper is concerned with a stochastic linear quadratic (LQ) control problem in the infinite-time horizon, with indefinite state and control weighting matrices in the cost function. It is shown that the solvability of this problem is equivalent to the existence of a so-called static stabilizing solution to a generalized algebraic Riccati equation. Moreover, another algebraic Riccati equation is introduced and all the possible optimal controls, including the ones in state feedback form, of the underlying LQ problem are explicitly obtained in terms of the two Riccati equations     

带有随机跳跃干扰的线性二次随机最优控制问题

给出一类布朗运动和泊松过程混合驱动的正倒向随机微分方程解的存在唯一性结果, 应用这一结果研究带有随机跳跃干扰的线性二次随机最优控制问题,并得到最优控制的显式形 式,可以证明最优控制是唯一的.然后,引入和研究一类推广的黎卡提方程系统,讨论该方程系统 的可解性并由该方程的解得到带有随机跳跃干扰的线性二次随机最优控制问题最优的线性反馈.     

Linear matrix inequalities, Riccati equations, and indefinitestochastic linear quadratic controls

This paper deals with an optimal stochastic linear-quadratic (LQ) control problem in infinite time horizon, where the diffusion term in dynamics depends on both the state and the control variables. In contrast to the deterministic case, we allow the control and state weighting matrices in the cost functional to be indefinite. This leads to an indefinite LQ problem, which may still be well posed due to the deep nature of uncertainty involved. The problem gives rise to a stochastic algebraic Riccati equation (SARE), which is, however, fundamentally different from the classical algebraic Riccati equation as a result of the indefinite nature of the LQ problem. To analyze the SARE, we introduce linear matrix inequalities (LMIs) whose feasibility is shown to be equivalent to the solvability of the SARE. Moreover, we develop a computational approach to the SARE via a semi-definite programming associated with the LMIs. Finally, numerical experiments are reported to illustrate the proposed approach     

Infinite horizon indefinite stochastic linear quadratic control for discrete-time systems

This paper discusses discrete-time stochastic linear quadratic (LQ) problem in the infinite horizon with state and control dependent noise, where the weighting matrices in the cost function are assumed to be indefinite. The problem gives rise to a generalized algebraic Riccati equation (GARE) that involves equality and inequality constraints. The well-posedness of the indefinite LQ problem is shown to be equivalent to the feasibility of a linear matrix inequality (LMI). Moreover, the existence of a stabilizing solution to the GARE is equivalent to the attainability of the LQ problem. All the optimal controls are obtained in terms of the solution to the GARE. Finally, we give an LMI -based approach to solve the GARE via a semidefinite programming.     

基于策略迭代的连续时间系统的随机线性二次最优控制

针对模型参数部分未知的随机线性连续时间系统, 通过策略迭代算法求解无限时间随机线性二次(LQ) 最优控制问题. 求解随机LQ最优控制问题等价于求随机代数Riccati 方程(SARE) 的解. 首先利用伊藤公式将随机微分方程转化为确定性方程, 通过策略迭代算法给出SARE 的解序列; 然后证明SARE 的解序列收敛到SARE 的解, 而且在迭代过程中系统是均方可镇定的; 最后通过仿真例子表明策略迭代算法的可行性.

     

Well-posedness and attainability of indefinite stochastic linear quadratic control in infinite time horizon

This paper is concerned with a stochastic linear-quadratic (LQ) problem in an infinite time horizon with multiplicative noises both in the state and the control. A distinctive feature of the problem under consideration is that the cost weighting matrices for the state and the control are allowed to be indefinite. A new type of algebraic Riccati equation – called a generalized algebraic Riccati equation (GARE) – is introduced which involves a matrix pseudo-inverse and two additional algebraic equality/inequality constraints. It is then shown that the well-posedness of the indefinite LQ problem is equivalent to a linear matrix inequality (LMI) condition, whereas the attainability of the LQ problem is equivalent to the existence of a “stabilizing solution” to the GARE. Moreover, all possible optimal controls are identified via the solution to the GARE. Finally, it is proved that the solution to the GARE can be obtained via solving a convex optimization problem called semidefinite programming.     

线性离散奇异系统的不定二次最优控制问题: Krein空间方法

The finite time horizon indefinite linear quadratic(LQ) optimal control problem for singular linear discrete time-varying systems is discussed. Indefinite LQ optimal control problem for singular systems can be transformed to that for standard state-space systems under a reasonable assumption. It is shown that the indefinite LQ optimal control problem is dual to that of projection for backward stochastic systems. Thus, the optimal LQ controller can be obtained by computing the gain matrices of Kalman filter. Necessary and sufficient conditions guaranteeing a unique solution for the indefinite LQ problem are given. An explicit solution for the problem is obtained in terms of the solution of Riccati difference equations.     

一类双线性随机系统M量测不定LQ控制

研究了带有乘性噪声和受扰动观测的离散时间随机系统不定线性二次(Linear quadratic, LQ) 最优输出反馈控制问题. 对此类问题而言,二次成本函数的加权矩阵不定号,并且最优控制具有对偶效果.为在最优性和计算复杂度间 进行折衷,本文采用了一种M量测反馈控制设计方法.基于动态规划方法,将未来的测量结合到当前控制 计算当中的M量测反馈控制可以通过倒向求解一类与原系统维数相同的广义差分Riccati方程(Generalized difference Riccati equation, GDRE)得到.仿真结果 表明本文提出的算法与目前普遍采用的确定等价性方法相比具有优越性.     

Linear Quadratic Optimal Control of Time-Varying Systems with Indefinite Costs on Hilbert Spaces

In this paper we consider time-varying linear systems on Hilbert spaces and study the linear quadratic optimal control problem with an indefinite performance criterion over an infinite time interval. An example shows that in contrast to the finite-dimensional situation, in general, the solvability of the optimal control problem does not imply the solvability of the associated integral Riccati equation. Using an operator theoretic approach towards the time-varying integral Riccati equation, we derive equivalent conditions for the solvability of both problems. Date received: October 8, 1997. Date revised: August 10, 1998.     

Discrete time mean-field stochastic linear-quadratic optimal control problems

This paper firstly presents necessary and sufficient conditions for the solvability of discrete time, mean-field, stochastic linear-quadratic optimal control problems. Secondly, the optimal control within a class of linear feedback controls is investigated using a matrix dynamical optimization method. Thirdly, by introducing several sequences of bounded linear operators, the problem is formulated as an operator stochastic linear-quadratic optimal control problem. By the kernel-range decomposition representation of the expectation operator and its pseudo-inverse, the optimal control is derived using solutions to two algebraic Riccati difference equations. Finally, by completing the square, the two Riccati equations and the optimal control are also obtained.     

Optimistic Value Model of Indefinite LQ Optimal Control for Discrete‐Time Uncertain Systems

Uncertainty theory is a branch of mathematics which provides a new tool to deal with the human uncertainty. Based on uncertainty theory, this paper proposes an optimistic value model of discrete‐time linear quadratic (LQ) optimal control, whereas the state and control weighting matrices in the cost function are indefinite, the system dynamics are disturbed by uncertain noises. With the aid of the Bellman's principle of optimality in dynamic programming, we first present a recurrence equation. Then, a necessary condition for the state feedback control of the indefinite LQ problem is derived by using the recurrence equation. Moreover, a sufficient condition of well‐posedness for the indefinite LQ optimal control is given. Finally, a numerical example is presented by using the obtained results.     

Infinite time horizon nonzero-sum linear quadratic stochastic differential games with state and control-dependent noise

This paper discusses the infinite time horizon nonzero-sum linear quadratic （LQ） differential games of stochastic systems governed by Itoe＇s equation with state and control-dependent noise. First, the nonzero-sum LQ differential games are formulated by applying the results of stochastic LQ problems. Second, under the assumption of mean-square stabilizability of stochastic systems, necessary and sufficient conditions for the existence of the Nash strategy are presented by means of four coupled stochastic algebraic Riccati equations. Moreover, in order to demonstrate the usefulness of the obtained results, the stochastic H-two/H-infinity control with state, control and external disturbance-dependent noise is discussed as an immediate application.     

The generalized continuous algebraic Riccati equation and impulse-free continuous-time LQ optimal control

The purpose of this paper is to investigate the role that the so-called constrained generalized Riccati equation plays within the context of continuous-time singular linear–quadratic (LQ) optimal control. This equation has been defined following the analogy with the discrete-time setting. However, while in the discrete-time case the connections between this equation and the linear–quadratic optimal control problem has been thoroughly investigated, to date very little is known on these connections in the continuous-time setting. This note addresses this point. We show, in particular, that when the continuous-time constrained generalized Riccati equation admits a solution, the corresponding linear–quadratic problem admits an impulse-free optimal control. We also address the corresponding infinite-horizon LQ problem for which we establish a similar result under the additional constraint that there exists a control input for which the cost index is finite.     

线性Markov 切换系统的随机Nash 微分博弈及混合H2/H∞ 控制

研究线性Markov切换系统的随机Nash微分博弈问题。首先借助线性Markov切换系统随机最优控制的相关结果,得到了有限时域和无线时域Nash均衡解的存在条件等价于其相应微分(代数) Riccati方程存在解,并给出了最优解的显式形式;然后应用相应的微分博弈结果分析线性Markov切换系统的混合H2/H∞控制问题;最后通过数值算例验证了所提出方法的可行性。     

Solvability and asymptotic behavior of generalized Riccatiequations arising in indefinite stochastic LQ controls

The optimal control problem in a finite time horizon with an indefinite quadratic cost function for a linear system subject to multiplicative noise on both the state and control can be solved via a constrained matrix differential Riccati equation. In this paper, we provide general necessary and sufficient conditions for the solvability of this generalized differential Riccati equation. Furthermore, its asymptotic behavior is investigated along with its connection to the generalized algebraic Riccati equation associated with the linear quadratic control problem in finite time horizon. Examples are presented to illustrate the results established     

Feedback control on Nash equilibrium for discrete-time stochastic systems with Markovian jumps: Finite-horizon case

In this paper, we consider the feedback control on nonzero-sum linear quadratic (LQ) differential games in finite horizon for discrete-time stochastic systems with Markovian jump parameters and multiplicative noise. Four-coupled generalized difference Riccati equations (GDREs) are obtained, which are essential to find the optimal Nash equilibrium strategies and the optimal cost values of the LQ differential games. Furthermore, an iterative algorithm is given to solve the four-coupled GDREs. Finally, a suboptimal solution of the LQ differential games is proposed based on a convex optimization approach and a simplification of the suboptimal solution is given. Simulation examples are presented to illustrate the effectiveness of the iterative algorithm and the suboptimal solution.     

一类带随机跳跃的完全耦合的线性二次随机控制问题

研究一类带随机跳跃的完全耦合的线性二次随机控制问题. 得到了最优控制的显式解, 并可以证明最优控制是唯一的. 引入了一类推广的黎卡提方程并讨论了其可解性. 利用这一类推广的黎卡提方程的解, 得到了上述带随机跳跃的最优控制问题的线性状态反馈调节器.