Linear‐Quadratic Optimal Control Problems for Mean‐Field Stochastic Differential Equations with Jumps

In this paper, we study a linear‐quadratic optimal control problem for mean‐field stochastic differential equations driven by a Poisson random martingale measure and a one‐dimensional Brownian motion. Firstly, the existence and uniqueness of the optimal control is obtained by the classic convex variation principle. Secondly, by the duality method, the optimality system, also called the stochastic Hamilton system which turns out to be a linear fully coupled mean‐field forward‐backward stochastic differential equation with jumps, is derived to characterize the optimal control. Thirdly, applying a decoupling technique, we establish the connection between two Riccati equations and the stochastic Hamilton system and then prove the optimal control has a state feedback representation.     

Closed‐form solution to finite‐horizon suboptimal control of nonlinear systems

Finite‐horizon optimal control of input‐affine nonlinear systems with fixed final time is considered in this study. It is first shown that the associated Hamilton–Jacobi–Bellman partial differential equation to the problem is reducible to a state‐dependent differential Riccati equation after some approximations. With a truncation in the control equation, a near optimal solution to the problem is obtained, and the global onvergence properties of the closed‐loop system are analyzed. Afterwards, an approximate method, called Finite‐horizon State‐Dependent Riccati Equation (Finite‐SDRE), is suggested for solving the differential Riccati equation, which renders the origin a locally exponentially stable point. The proposed method provides online feedback solution for controlling different initial conditions. Finally, through some examples, the performance of the resulting controller in finite‐horizon control is analyzed. Copyright © 2014 John Wiley & Sons, Ltd.     

A risk‐sensitive stochastic maximum principle for fully coupled forward‐backward stochastic differential equations with applications

In this paper, we are interested in the problem of optimal control where the system is given by a fully coupled forward‐backward stochastic differential equation with a risk‐sensitive performance functional. As a preliminary step, we use the risk neutral which is an extension of the initial control system where the admissible controls are convex, and an optimal solution exists.Then, we study the necessary as well as sufficient optimality conditions for risk sensitive performance. At the end of this work, we illustrate our main result by giving an example that deals with an optimal portfolio choice problem in financial market, specifically the model of control cash flow of a firm or project where, for instance, we can set the model of pricing and managing an insurance contract.     

Discrete‐time mixed ℋ︁2/ℋ︁∞ nonlinear filtering

In this paper, we consider the discrete‐time mixed ??₂/??_∞ filtering problem for affine nonlinear systems. Necessary and sufficient conditions for the solvability of this problem with a finite‐dimensional filter are given in terms of a pair of coupled discrete‐time Hamilton–Jacobi‐Isaac's equations (DHJIE) with some side‐conditions. For linear systems, it is shown that these conditions reduce to a pair of coupled discrete‐time algebraic‐Riccati‐equations (DAREs) or a system of linear matrix inequalities (LMIs) similar to the ones for the control case. Both the finite‐horizon and infinite‐horizon problems are discussed. Moreover, sufficient conditions for approximate solvability of the problem are also derived. These solutions are especially useful for computational purposes, considering the difficulty of solving the coupled DHJIEs. An example is also presented to demonstrate the approach. Copyright © 2010 John Wiley & Sons, Ltd.     

Discrete‐time invariant extended Kalman filter on matrix Lie groups

In this article, we derive symmetry preserving discrete‐time invariant extended Kalman filters (IEKF) on matrix Lie groups. These Kalman filters offer an advantage over classical extended Kalman filters as the error dynamics for such filters are independent of the group configuration which, in turn, provides a uniform estimate of the region of convergence. In contrast to existing techniques in the literature, the discrete‐time IEKF is derived using minimal tools from differential geometry which simplifies the derivation and the representation of IEKF. In our technique, the linearized error dynamics is defined on the Lie algebra directly using variational approaches, unlike conventional approaches where the error dynamics is translated to an Euclidean space using the logarithm map before its linearization. Moreover, the Kalman gains and its associated difference Riccati equations are derived in operator spaces by setting a discrete‐time optimal control problem and solving it with discrete‐time Pontryagin's maximum principle. The proposed discrete‐time IEKF is implemented for the attitude dynamics of the rigid body, which is a benchmark problem in control. It is observed from the numerical studies that the IEKF is computationally less intensive and provides better performance than the classical extended Kalman filter.     

Recursive state estimation for discrete‐time nonlinear systems with event‐triggered data transmission,norm‐bounded uncertainties and multiple missing measurements

In this paper, we consider the recursive state estimation problem for a class of discrete‐time nonlinear systems with event‐triggered data transmission, norm‐bounded uncertainties, and multiple missing measurements. The phenomenon of event‐triggered communication mechanism occurs only when the specified event‐triggering condition is violated, which leads to a reduction in the number of excessive signal transmissions in a network. A sequence of independent Bernoulli random variables is employed to model the multiple measurements missing in the transmission. The norm‐bounded uncertainties that could be considered as external disturbances which lie in a bounded set. The purpose of the addressed filtering problem is to obtain an optimal robust recursive filter in the minimum‐variance sense such that with the simultaneous presence of event‐triggered data transmission, norm‐bounded uncertainties, and multiple missing measurements; the filtering error is minimized at each sampling time. By solving two Riccati‐like difference equations, the filter gain is calculated recursively. Based on the stochastic analysis theory, it is proved that the estimation error is bounded under certain conditions. Finally, two numerical examples are presented to demonstrate the effectiveness of the proposed algorithm. Copyright © 2016 John Wiley & Sons, Ltd.     

Necessary and sufficient conditions of risk‐sensitive optimal control and differential games for stochastic differential delayed equations

In this paper, we consider risk‐sensitive optimal control and differential games for stochastic differential delayed equations driven by Brownian motion. The problems are related to robust stochastic optimization with delay due to the inherent feature of the risk‐sensitive objective functional. For both problems, by using the logarithmic transformation of the associated risk‐neutral problem, the necessary and sufficient conditions for the risk‐sensitive maximum principle are obtained. We show that these conditions are characterized in terms of the variational inequality and the coupled anticipated backward stochastic differential equations (ABSDEs). The coupled ABSDEs consist of the first‐order adjoint equation and an additional scalar ABSDE, where the latter is induced due to the nonsmooth nonlinear transformation of the adjoint process of the associated risk‐neutral problem. For applications, we consider the risk‐sensitive linear‐quadratic control and game problems with delay, and the optimal consumption and production game, for which we obtain explicit optimal solutions.     

Optimal Control of Forward‐Backward Stochastic Jump‐Diffusion Differential Systems with Observation Noises: Stochastic Maximum Principle

This paper is concerned with a partially observed optimal control problem for a controlled forward‐backward stochastic system with correlated noises between the system and the observation, which generalizes the result of a previous work to a jump‐diffusion system. Under some convexity assumptions, necessary and sufficient optimality conditions for such an optimal control are established in the form of Pontryagin type maximum principle in a unified way by means of duality analysis and convex variational techniques     

无线通讯网络功率和流量的预联合控制

To mitigate the loop delay in distributed wireless networks, a predictive power and rate control scheme is proposed for the system model that also accounts for the congestion levels and input delay instead of state-delayed in a network. A measurement feedback control problem with input delay is formulated by minimizing the energy of the difference between the actual and the desired signal-to-interference-plus-noise ratio (SNR) levels, as well as the energy of the control sequence. To solve this problem, we present two Riccati equations for the control and the estimation for the time delay systems. A complete analytical optimal controller is obtained by using the separation principle and solving two Riccati equations, where one is backward equation for stochastic linear quadratic regulation and the other is the standard filtering Riccati equation. Simulation results illustrate the performance of the proposed power and the rate control scheme.     

Data‐driven optimal event‐triggered consensus control for unknown nonlinear multiagent systems with control constraints

This paper considers optimal consensus control problem for unknown nonlinear multiagent systems (MASs) subjected to control constraints by utilizing event‐triggered adaptive dynamic programming (ETADP) technique. To deal with the control constraints, we introduce nonquadratic energy consumption functions into performance indices and formulate the Hamilton‐Jacobi‐Bellman (HJB) equations. Then, based on the Bellman's optimality principle, constrained optimal consensus control policies are designed from the HJB equations. In order to implement the ETADP algorithm, the critic networks and action networks are developed to approximate the value functions and consensus control policies respectively based on the measurable system data. Under the event‐triggered control framework, the weights of the critic networks and action networks are only updated at the triggering instants which are decided by the designed adaptive triggered conditions. The Lyapunov method is used to prove that the local neighbor consensus errors and the weight estimation errors of the critic networks and action networks are ultimately bounded. Finally, a numerical example is provided to show the effectiveness of the proposed ETADP method.     

Linear quadratic tracking problem for discrete‐time systems with multiple delays in single input channel

The linear quadratic tracking problem for discrete‐time systems with multiple delays in single input channel is considered. In this paper, we provide an approach without resorting to system state augmentation. The optimal tracking control is given in terms of the current state, the previous inputs, and the output of an auxiliary backward deterministic delay system which is formulated for the first time in this paper. The solution relies on a Riccati difference equation of the same dimension as the plant (ignoring the delays). The key to our development is the establishment of a duality between the optimal tracking control and the optimal smoothing estimation of an associated stochastic backward system as well as the introduction of the auxiliary backward deterministic delay system. An analysis of the computational complexity of the proposed approach and its comparison with that of the augmentation method, which is to incorporate the delayed inputs into the augmented state, are provided. An example is given to demonstrate the effectiveness of the results. Copyright © 2009 John Wiley & Sons, Ltd.     

Decentralized linear–quadratic–Gaussian control of networked control systems with asymmetric information

The decentralized linear–quadratic–Gaussian (LQG) control problem for networked control systems (NCSs) with asymmetric information is investigated, where controller 1 shares its historical information with controller 2, and not vice versa. The asymmetry of the information structure leads to the coupling between controller 2 and estimator 1, and hence the classical separation principle fails. Through the assumption of linear control strategy, the coupling between controller 2 and estimator 1 (CCE) is decoupled, but the estimation gain is still coupled with the control gain. It is noted that the control gain conforms to the backward Riccati equation while estimation gain abides by the forward equation, which is computationally challenging. Applying the stochastic maximum principle, the solvability of the decentralized LQG control problem is reduced to that of corresponding forward and backward stochastic difference equations (FBSDEs). Further, necessary and sufficient conditions for the solvability of optimal control problem are presented by two Riccati equations, one of which is nonsymmetric. Moreover, a novel iterative forward method is proposed to calculate the coupled backward control gain and forward estimation gain.     

Linear−quadratic optimal control and nonzero‐sum differential game of forward−backward stochastic system

An existence and uniqueness result for one kind of forward–backward stochastic differential equations with double dimensions was obtained under some monotonicity conditions. Then this result was applied to the linear‐quadratic stochastic optimal control and nonzero‐sum differential game of forward–backward stochastic system. The explicit forms of the optimal control and the Nash equilibrium point are obtained respectively. We note that our method is effective in studying the uniqueness of Nash equilibrium point. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Optimal Control Problem for Risk‐Sensitive Mean‐Field Stochastic Delay Differential Equation with Partial Information

This paper deals with the risk‐sensitive control problem for mean‐field stochastic delay differential equations (MF‐SDDEs) with partial information. Firstly, under the assumptions that the control domain is not convex and the value function is non‐smooth, we establish a stochastic maximum principle (SMP). Then, by means of Itô's formula and some continuous dependence, we prove the existence and uniqueness results for another type of MF‐SDDEs. Meanwhile, the verification theorem for the MF‐SDDEs is obtained by using a clever construction of the Hamiltonian function. Finally, based on our verification theorem, a linear‐quadratic system is investigated and the optimal control is also derived by the stochastic filtering technique.     

H∞ Control of Linear Discrete‐Time Systems With Norm‐Bounded Nonlinear Uncertainties

In this paper, robust H_∞ control of a class of discrete‐time uncertain systems in state‐space form with linear nominal parts and norm‐bounded nonlinear uncertainties in both state and output equations is discussed. Such systems have a unique characterisic; that is, the two norm‐bounded nonlinear uncertainties have the equivalent representation by means of time‐varying and norm‐bounded linear uncertainties. To overcome the conservativenss of [5], the two nonlinear uncertainty sets are considered to be different. Then, by converting such systems into related discrete‐time linear systems with time‐varying and norm‐bounded linear uncertainties, we obtain that a sufficient condition for robust H_∞ control of such systems is equivalent to the solvability of the same problem of the related linear uncertain systems, which is solvable by means of a linear algebraic Riccati inequality.     

线性离散奇异系统的不定二次最优控制问题: Krein空间方法

The finite time horizon indefinite linear quadratic(LQ) optimal control problem for singular linear discrete time-varying systems is discussed. Indefinite LQ optimal control problem for singular systems can be transformed to that for standard state-space systems under a reasonable assumption. It is shown that the indefinite LQ optimal control problem is dual to that of projection for backward stochastic systems. Thus, the optimal LQ controller can be obtained by computing the gain matrices of Kalman filter. Necessary and sufficient conditions guaranteeing a unique solution for the indefinite LQ problem are given. An explicit solution for the problem is obtained in terms of the solution of Riccati difference equations.     

A further result of the nonlinear mixed H2/H∞ tracking control problem for robotic systems

The design objective of a mixed H₂/H_∞ control is to find the H₂ optimal tracking control law under a prescribed disturbance attenuation level. With the help of the technique of completing the squares, a further result of the mixed H₂/H_∞ optimal tracking control problem is presented, by combining it with standard LQ optimal control technique. In this paper, only a nonlinear time‐varying Riccati equation is required to solve the problem in the design procedure—instead of two coupled nonlinear time‐varying Riccati equations, or two coupled linear algebraic Riccati‐Iike equations—with some assumptions made regarding the weighting matrices in the existing results. A closed‐form controller for the mixed H₂/H_∞ robotic tracking problem is simply constructed with a matrix inequality check. Moreover, it shows that the existing results are the special cases of these results. Finally, detailed comparison is performed by numerical simulation of a two‐link robotic manipulator. © 2002 John Wiley & Sons, Inc.     

Optimal robot‐environment interaction using inverse differential Riccati equation

An optimal robot‐environment interaction is designed by transforming an environment model into an optimal control problem. In the optimal control, the inverse differential Riccati equation is introduced as a fixed‐end‐point closed‐loop optimal control over a specific time interval. Then, the environment model, including interaction force, is formulated in a state equation, and the optimal trajectory is determined by minimizing a cost function. Position control is proposed, and the stability of the closed‐loop system is investigated using the Lyapunov direct method. Finally, theoretical developments are verified through numerical simulation.     

H2/H∞ control for stochastic systems with delay

This paper is concerned with the H₂/H_∞ control problem for stochastic linear systems with delay in state, control and external disturbance-dependent noise. A necessary and sufficient condition for the existence of a unique solution to the control problem is derived. The resulting solution is characterised by a kind of complex generalised forward–backward stochastic differential equations with stochastic delay equations as forward equations and anticipated backward stochastic differential equations as backward equations. Especially, we present the equivalent feedback solution via a new type of Riccati equations. To explain the theoretical results, we apply them to a population control problem.