On the quantum master equation under feedback control

The nature of the quantum trajectories, described by stochastic master equations, may be jump-like or diffusive, depending upon different measurement processes. There are many different unravelings corresponding to different types of stochastic master equations for a given master equation. In this paper, we study the relationship between the quantum stochastic master equations and the quantum master equations in the Markovian case under feedback control. We show that the corresponding unraveling no longer exists when we further consider feedback control besides measurement. It is due to the fact that the information gained by the measurement plays an important role in the control process. The master equation governing the evolution of ensemble average cannot be restored simply by eliminating the noise term unlike the case without a control term. By establishing a fundamental limit on performance of the master equation with feedback control, we demonstrate the differences between the stochastic master equation and the master equation via theoretical proof and simulation, and show the superiority of the stochastic master equation for feedback control.     

Nonquadratic cost and nonlinear feedback control

Nonlinear controllers offer significant advantages over linear controllers in a variety of circumstances. Hence there has been significant interest in extending linear-quadratic synthesis to nonlinear-nonquadratic problems. The purpose of this paper is to review the current status of such efforts and to present, in a simplified and tutorial manner, some of the basic ideas underlying these results. Our approach focuses on the role of the Lyapunov function in guaranteeing stability for autonomous systems on an infinite horizon. Sufficient conditions for optimality are given in a form that corresponds to a steady-state version of the Hamilton-Jacobi-Bellman equation. These results are used to provide a simplified derivation of the nonlinear feedback controller obtained by Bass and Webber (1966)³⁸ and to obtain a deterministic variation of the stochastic nonlinear feedback controller developed by Speyer (1976).⁴⁵.     

On discrete-time linear quadratic control

The discrete-infinite time stochastic control system with complete observation is considered with quadratic cost functional when the coefficients of the system and cost functional are not time-invariant. It has been shown that the optimal control law has the form of time invariant feedback under the assumption that the coefficients have limits as time tends to infinity. In addition, asymptotic property of the solution of the difference Riccati equation with time-varying coefficients are established.     

A solvable stochastic control problem in hyperbolic three space

A stochastic optimal control problem in hyperbolic three space is described and explicitly solved. The solution is obtained by finding a smooth solution to the Hamilton-Jacobi or dynamic programming equation.     

The effects of redundant control inputs in optimal control

For a stabilizable system, the extension of the control inputs has no use for stabilizability, but it is important for optimal control. In this paper, a necessary and sufficient condition is presented to strictly decrease the quadratic optimal performance index after control input extensions. A similar result is also provided for H ₂ optimal control problem. These results show an essential difference between single-input and multi-input control systems. Several examples are taken to illustrate related problems.     

Stochastic linear-quadratic optimal control problems — Recent developments

In this paper, we present a brief survey for some recent developments of stochastic linear-quadratic optimal controls. We mainly concentrate on the results obtained by the authors and their collaborators in the last decay.     

Simultaneous control problems for systems of elastic strings and beams

Controllability problems for N strings or beams controlled from a common endpoint are studied. We give necessary and sufficient conditions for simultaneous spectral and approximate controllability and describe the space of simultaneously reachable states as a function of the position of the endpoint. For each type of controllability result the sharp controllability time is obtained. The proofs are based on new results in nonharmonic Fourier series which describe Riesz bases of exponential divided differences.     

Necessary integral optimality conditions for quasi-singular controls in one control problem

An optimal control problem with variable structure described by a system of nonlinear integral Volterra-type equations is considered in the article. A necessary optimality condition is obtained in the form of a linearized maximum condition. A case of degeneracy of the linearized maximum principle is further studied. Integral necessary optimality conditions of quasi-singular controls are proved.     

自平衡控制系统的平衡性仿真

自平衡两轮移动式倒立摆是三个自由度的强耦合和非线性的自然不稳定系统。虽然现代控制论中的状态反馈的极点配置对两轮移动式倒立摆的平衡起到了一定的控制作用,但是在现实多干扰的外界环境中,控制论中极点配置算法的控制作用就很难满足实际控制需要。针对两轮移动式倒立摆数学模型,给出详细的力学推导并在Simulink环境下,对状态反馈的极点配置法控制的两轮移动式倒立摆处于不同的坡度平衡情况进行了仿真和分析。     

A risk-sensitive control dual approach to a large deviations control problem

We consider a control problem of an ergodic process where the objective is to maximize over long term the probability to overperform a given level. This is formulated as a large deviations control problem for which the standard dynamic programming methods may not be applied directly. We solve this problem by adopting a duality approach leading to a risk-sensitive ergodic control problem. In a continuous-time diffusion setting, we state a verification theorem in terms of partial differential equations for this dual problem. We then turn back to the primal problem by means of large deviations techniques. We derive the optimal rate function and nearly optimal controls for the large deviations optimization problem. Finally, explicit solutions are provided in a linear-quadratic case.     

Linear-quadratic optimal control revisited

We offer a new proof of the relationship between the solution of a matrix Riccati equation and the optimal solution of a linear-quadratic regulator problem in the presence of linear terminal constraints.     

Stochastic maximum principle for SPDEs with noise and control on the boundary

In this paper we prove necessary conditions for optimality of a stochastic control problem for a class of stochastic partial differential equations that is controlled through the boundary. This kind of problem can be interpreted as a stochastic control problem for an evolution system in a Hilbert space. The regularity of the solution of the adjoint equation, that is a backward stochastic equation in infinite dimension, plays a crucial role in the formulation of the maximum principle.     

比例–积分控制加广义预测控制算法及其应用

针对比例–积分(proportional-integral, PI)控制因不能预测未来输出而提前改变控制量使其用于光电稳定伺服系统时往往响应剧烈的问题,研究了光电稳定伺服系统的广义预测控制(generalized predictive control, GPC).首先通过证明受控自回归积分滑动平均(controlled auto-regressive integral moving-average, CARIMA)模型的直接递推预测与Diophantine方程预测等价,提出了预测较快的模型等价预测GPC算法,其预测复杂度比原GPC降低了一个阶次.其次通过对PI和GPC的特点进行分析,综合考虑两者的优缺点,提出了一种新型的基于PI增量和GPC增量加权的比例积分控制加广义预测控制(proportional-integral control plus generalized predictive control, PI+GPC)算法,实现了基于历史、当前和未来偏差计算控制量,并给出了算法设计流程和参数选取规则.最后通过仿真并在某光电稳定伺服平台上验证后得出, PI+GPC和PI相比稳定精度有所提高,且平稳性和快速性大为改善.     

Sparse event-triggered control of linear systems

In event-triggered control, a situation where the control input must be sparse often arises. Therefore, in this study, we propose sparse event-triggered control, meaning that the control input is sparse and updated in an event-triggered manner. First, we present a model-based method for sparse event-triggered control of linear systems, where the event condition is defined by a Lyapunov function. The resulting control input is proven to be sparse and the control system is confirmed to be asymptotically stable. Second, we extend it to a data-driven version, where the event condition is adaptively updated from online data on the state trajectory. Finally, we discuss the possibility of extending our framework to two cases of disturbance and nonlinear dynamics.     

Singular control of stochastic linear systems with recursive utility

We formulate a class of singular stochastic control problem with recursive utility where the cost function is determined by a backward stochastic differential equation. Some characteristics of the value function of the control problem are obtained by the method of approximation via penalization, and the optimal control process is constructed.     

Non-collocated sliding mode control of partial differential equations for soil irrigation

In this paper, sliding mode control (SMC) is developed to regulate the water content in the soil. The movement of water in the soil is modeled by the Richards equation, which is a parabolic partial differential equation (PDE). The non-collocated SMC of PDEs is considered, that is, where the sensor and the actuator are at different positions. Two control objectives are considered: the mean water content control, and the water content control at a specified depth. For these purposes, non-standard sliding manifolds are proposed. The existence of the sliding mode in this control system is proved using a Lyapunov function, and the stability properties are analyzed using a frequency stability criterion. Simulation results indicate the excellent transient performance and the complete rejection of external disturbances caused by water evaporation, outperforming on–off and PI controllers.     

The optimal robust control policy for uncertain semi-Markov control processes

The optimization problems of Markov control processes (MCPs) with exact knowledge of system parameters, in the form of transition probabilities or infinitesimal transition rates, can be solved by using the concept of Markov performance potential which plays an important role in the sensitivity analysis of MCPs. In this paper, by using an equivalent infinitesimal generator, we first introduce a definition of discounted Poisson equations for semi-Markov control processes (SMCPs), which is similar to that for MCPs, and the performance potentials of SMCPs are defined as solution of the equation. Some related optimization techniques based on performance potentials for MCPs may be extended to the optimization of SMCPs if the system parameters are known with certainty. Unfortunately, exact values of the distributions of the sojourn times at some states or the transition probabilities of the embedded Markov chain for a large-scale SMCP are generally difficult or impossible to obtain, which leads to the uncertainty of the semi-Markov kernel, and thereby to the uncertainty of equivalent infinitesimal transition rates. Similar to the optimization of uncertain MCPs, a potential-based policy iteration method is proposed in this work to search for the optimal robust control policy for SMCPs with uncertain infinitesimal transition rates that are represented as compact sets. In addition, convergence of the algorithm is discussed.     

一类控制量非负的新型脉冲随机控制模型

提出并研究了一类新型的脉冲随机控制模型,其状态结构由关于半鞅的线性随机微分方程所确定,其控制费用函数为关于控制前状态与控制量的二元函数且其控制量保持非负.首先建立了一类新型的变分方程并证明了其解的存在性.通过对变分方程的解函数进行一系列随机分析处理,证明了最佳控制的存在性且对其结构进行了深入分析.此外,由于本文模型与以往文献中的随机控制模型有着重大差异,因而在分析手法上与以往文献相比颇多差异之处.可以预期,本文不仅在随机控制的研究中将具有重要的理论意义,而且在金融控制及证券管理方面也将有着广泛的应用价值.     

Policy iteration based feedback control

It is well known that stochastic control systems can be viewed as Markov decision processes (MDPs) with continuous state spaces. In this paper, we propose to apply the policy iteration approach in MDPs to the optimal control problem of stochastic systems. We first provide an optimality equation based on performance potentials and develop a policy iteration procedure. Then we apply policy iteration to the jump linear quadratic problem and obtain the coupled Riccati equations for their optimal solutions. The approach is applicable to linear as well as nonlinear systems and can be implemented on-line on real world systems without identifying all the system structure and parameters.