Infinite horizon optimal control of linear discrete time systems with stochastic parameters

The infinite horizon optimal control problem is considered in the general case of linear discrete time systems and quadratic criteria, both with stochastic parameters which are independent with respect to time. A stronger stabilizability property and a weaker observability property than usual for deterministic systems are introduced. It is shown that the infinite horizon problem has a solution if the system has the first property. If in addition the problem has the second property the solution is unique and the control system is stable in the mean square sense. A simple necessary and sufficient condition, explicit in the system matrices, is given for the system to have the stronger stabilizability property. This condition also holds for deterministic systems to be stabilizable in the usual sense. The stronger stabilizability and weaker observability properties coincide with the usual ones if the parameters are deterministic.     

Active fault detection and control: Unified formulation and optimal design

A new unified formulation of the active fault detection and control problem for discrete-time stochastic systems and its optimal solution are proposed. The problem formulation stems from the optimal stochastic control problem and includes important special cases: an active detector and controller, an active detector and input signal generator, and an active detector with a given input signal generator. The optimal solution is derived using the so-called closed loop information processing strategy. This strategy respects the influence of the current decision and/or input on the future behavior of the observed system, allows penalizing future wrong decisions, and improves the quality of fault detection. The proposed formulation and obtained solution also provide better understanding of the active fault detection and its relation to the optimal stochastic control. The results are illustrated in numerical examples.     

The return difference for discrete-time optimal feedback systems

This brief paper deals with the return difference function and the inverse optimal control problem for linear single-input discrete-time systems with a quadratic performance criterion. It is pointed out that there are essential differences with respect to the case of continuous-time systems. The effect of these differences on the stability margins and the sensitivity properties of optimal feedback discrete-time systems is analysed.     

Dynamic programming for constrained optimal control of discrete-time linear hybrid systems

In this paper we study the solution to optimal control problems for constrained discrete-time linear hybrid systems based on quadratic or linear performance criteria. The aim of the paper is twofold. First, we give basic theoretical results on the structure of the optimal state-feedback solution and of the value function. Second, we describe how the state-feedback optimal control law can be constructed by combining multiparametric programming and dynamic programming.     

Robustness issues in optimal control of unstable plants

This paper presents a robustness analysis of the optimal solution of the step tracking problem in a one-degree of freedom control architecture, for the discrete-time multivariable (MIMO) case. It is shown that the resulting closed loop becomes arbitrarily fragile, i.e. the sensitivity peak becomes arbitrarily large, for certain reference directions. The cause of this problem is that the optimal controller depends on the reference direction. The paper shows that this dependence can be eliminated with an alternative formulation of the optimal problem where the cost is averaged over all reference directions.     

An efficient algorithm for optimal control of PWA systems with polyhedral performance indices

We present an algorithm for the computation of explicit optimal control laws for piecewise affine (PWA) systems with polyhedral performance indices. The algorithm is based on dynamic programming (DP) and represents an extension of ideas initially proposed in Kerrigan and Mayne [(2003). Optimal control of constrained, piecewise affine systems with bounded disturbances. In Proceedings of the 41st IEEE conference on decision and control, Las Vegas, Nevada, USA, December], and Baoti? et al. [(2003). A new algorithm for constrained finite time optimal control of hybrid systems with a linear performance index. In Proceedings of European control conference, Cambridge, UK, September]. Specifically, we show how to exploit the underlying geometric structure of the optimization problem in order to significantly improve the efficiency of the off-line computations. An extensive case study is provided, which clearly indicates that the algorithm proposed in this paper may be preferable to other schemes published in the literature.     

Solving nonlinear optimal control problems using a hybrid IPSO-SQP algorithm

A hybrid algorithm by integrating an improved particle swarm optimization (IPSO) with successive quadratic programming (SQP), namely IPSO-SQP, is proposed for solving nonlinear optimal control problems. The particle swarm optimization (PSO) is showed to converge rapidly to a near optimum solution, but the search process will become very slow around global optimum. On the contrary, the ability of SQP is weak to escape local optimum but can achieve faster convergent speed around global optimum and the convergent accuracy can be higher. Hence, in the proposed method, at the beginning stage of search process, a PSO algorithm is employed to find a near optimum solution. In this case, an improved PSO (IPSO) algorithm is used to enhance global search ability and convergence speed of algorithm. When the change in fitness value is smaller than a predefined value, the searching process is switched to SQP to accelerate the search process and find an accurate solution. In this way, this hybrid algorithm may find an optimum solution more accurately. To validate the performance of the proposed IPSO-SQP approach, it is evaluated on two optimal control problems. Results show that the performance of the proposed algorithm is satisfactory.     

Approximations for optimal stopping of a piecewise-deterministic process

This paper deals with approximation techniques for the optimal stopping of a piecewise-deterministic Markov process (P.D.P.). Such processes consist of a mixture of deterministic motion and random jumps. In the first part of the paper (Section 3) we study the optimal stopping problem with lower semianalytic gain function; our main result is the construction of ε-optimal stopping times. In the second part (Section 4) we consider a P.D.P. satisfying some smoothness conditions, and forN integer we construct a discretized P.D.P. which retains the main characteristics of the original process. By iterations of the single jump operator from ℝ ^N to ℝ ^N , each iteration consisting ofN one-dimensional minimizations, we can calculate the payoff function of the discretized process. We demonstrate the convergence of the payoff functions, and for the case when the state space is compact we construct ε-optimal stopping times for the original problem using the payoff function of the discretized problem. A numerical example is presented.     

Stochastic iterative dynamic programming: a Monte Carlo approach to dual control

Practical exploitation of optimal dual control (ODC) theory continues to be hindered by the difficulties involved in numerically solving the associated stochastic dynamic programming (SDPs) problems. In particular, high-dimensional hyper-states coupled with the nesting of optimizations and integrations within these SDP problems render their exact numerical solution computationally prohibitive. This paper presents a new stochastic dynamic programming algorithm that uses a Monte Carlo approach to circumvent the need for numerical integration, thereby dramatically reducing computational requirements. Also, being a generalization of iterative dynamic programming (IDP) to the stochastic domain, the new algorithm exhibits reduced sensitivity to the hyper-state dimension and, consequently, is particularly well suited to solution of ODC problems. A convergence analysis of the new algorithm is provided, and its benefits are illustrated on the problem of ODC of an integrator with unknown gain, originally presented by Åström and Helmersson (Computers and Mathematics with Applications 12A (1986) 653-662).     

Multistage linear programming for discrete optimal control with distributed-lags

A multistage decomposition scheme is developed for optimizing discrete-time dynamic systems, which include distributed and/or multiple pure delays. The discrete optimal control problem in this paper consists of a system dynamics described by a multidimensional linear difference equation of high-order which is called the distributed-lag model, a linear objective function, and linear state and control constraints. This problem may be solved as a linear program by, for example, a revised simplex method. However, this leads to excessive storage requirement for large problems. Instead, by taking advantage of the staircase-structure of equality constraints (system equation), Dantzig-Wolfe decomposition principle is applied repeatedly in each stage, and an effective multistage decomposition algorithm for distributed-lag models is obtained. Significant advantage of the optimization technique in this paper is that it can handle any number of delay terms in the system without reducing the multidimensional high-order system equation to a conventional larger dimensional first-order system equation (state equation of normal form). Therefore, a substantial reduction of computational burden, the so called curse of dimensionality, in the existing discrete optimal control algorithms, is obtained. A numerical example of a congested urban road traffic control problem with many delays is included.     

基于自适应动态规划的多设定值跟踪控制方法

针对一类典型的带有控制约束的非线性离散时间系统,提出了一种基于自适应动态规划(adaptive dynamic programmmg,ADP)算法的多设定值跟踪控制方法,并对其收敛性和稳定性做了严格分析.在ADP迭代跟踪控制的基础上,根据多模型控制的思想,设置阶梯状的参考轨迹,使得系统状态逐渐地跟踪到最终设定值,保证了系统的稳定性,极大地减小超调量,加快了响应时间,改善控制品质;同时由于控制器约束的存在,引入非二次型的性能指标函数,使得控制量始终在有界的范围内变化.最后对仿真结果进行了分析,结果表明了此方法的可行性和有效性.     

Convergence of semidiscrete approximations to optimal control problems in hilbert spaces: A counterexample

In this paper we give and discuss a counterexample related to the approximation of non-quadratic optimal control problems for linear distributed parameter systems. More precisely, we show that the assumption (1.7), which has been proved in a previous paper to be sufficient, is also necessary for the convergence (at least in the most general setting of the problem).     

Lossless convexification of a class of optimal control problems with non-convex control constraints

We consider a class of finite time horizon optimal control problems for continuous time linear systems with a convex cost, convex state constraints and non-convex control constraints. We propose a convex relaxation of the non-convex control constraints, and prove that the optimal solution of the relaxed problem is also an optimal solution for the original problem, which is referred to as the lossless convexification of the optimal control problem. The lossless convexification enables the use of interior point methods of convex optimization to obtain globally optimal solutions of the original non-convex optimal control problem. The solution approach is demonstrated on a number of planetary soft landing optimal control problems.     

Dynamic programming and viscosity solutions for the optimal control of quantum spin systems

The purpose of this paper is to describe the application of the notion of viscosity solutions to solve the Hamilton-Jacobi-Bellman (HJB) equation associated with an important class of optimal control problems for quantum spin systems. The HJB equation that arises in the control problems of interest is a first-order nonlinear partial differential equation defined on a Lie group. Hence we employ recent extensions of the theory of viscosity solutions to Riemannian manifolds in order to interpret possibly non-differentiable solutions to this equation. Results from differential topology on the triangulation of manifolds are then used develop a finite difference approximation method for numerically computing the solution to such problems. The convergence of these approximations is proven using viscosity solution methods. In order to illustrate the techniques developed, these methods are applied to an example problem.     

An improved state-space model structure and a corresponding predictive functional control design with improved control performance

Conventional state-space model predictive control requires a state estimator/observer to access the state information for feedback controller design. Its drawbacks are the numerical convergence stability of the observer and closed-loop control performance deterioration with activated plant input/output constraints. The recent direct use of measured input and output variables to formulate a non-minimal state-space (NMSS) model overcomes these problems, but the subsequent controller is too sensitive to model mismatch. In this article, an improved structure of NMSS model that incorporates the output-tracking error is first formulated and then a subsequent predictive functional control design is proposed. The proposed controller is tested on both model match and model mismatch cases for comparison with previous controllers. Results show that control performance is improved. In addition, a linear programming method for constraints dealing and a closed form of transfer function representation of the control system are provided for further insight into the proposed method.     

An optimal regulation strategy with disturbance rejection for energy management of hybrid electric vehicles

The energy management problem of finding the optimal split between the different sources of energy in a charge-sustaining parallel HEV, ensuring stability and optimality with respect to a performance objective (fuel consumption minimization over a driving cycle), is addressed in this paper. The paper develops a generic stability and optimality framework within which the energy management problem is cast in the form of a nonlinear optimal regulation (with disturbance rejection) problem and a control Lyapunov function is used to design the control law. Two theorems ensuring optimality and asymptotic stability of the energy management strategy are proposed and proved. The sufficient conditions for optimality and stability are used to derive an analytical expression for the control law as a function of the battery state of charge/state of energy and system parameters. The control law is implemented in a simplified backward vehicle simulator and its performance is evaluated against the global optimal solution obtained from dynamic programming. The strategy performs within 4% of the benchmark solution while guaranteeing optimality and stability for any driving cycle.     

Stability Margin Determination of Linear Discrete‐Time Systems Described By Fuzzy Convex Sets Uncertainty

This paper proposes a new method for determining the stability margin of uncertain linear discrete‐time systems with uncertainty described by fuzzy convex sets. Nonsymmetrical triangular membership functions describing the uncertainty of characteristic polynomial coefficients are considered. This method is based on transformation of the uncertain discrete systems to the continuous time area and generalization of Kharitonov theorem for determining the uncommon minimum confidence level that guarantees stability of the system.     

Application of optimal control theory to analysis of cancer chemotherapy regimens

Cancer chemotherapy with application of one drug is studied. The negative and inhibiting effect of the tumor on normal cells is taken into account. Under certain assumptions, we determine the optimal regimen that minimizes the tumor burden at the end of a fixed period of therapy while maintaining the normal cell population above a prescribed level.     

Analysis and synthesis of networked control systems: Topological entropy, observability, robustness and optimal control

This paper extends the concept of topological to the case of uncertain dynamical systems. We address problems of observability and optimal control via limited capacity digital communication channels. The main results are given in terms of inequalities between date rate of the communication channel and topological entropy of the open-loop system. Topological entropy is calculated for some classes of uncertain dynamical systems.