Embedding adaptive JLQG into LQ martingale control with acompletely observable stochastic control matrix

With jump linear quadratic Gaussian (JLQG) control, one refers to the control under a quadratic performance criterion of a linear Gaussian system, the coefficients of which are completely observable, while they are jumping according to a finite-state Markov process. With adaptive JLQG, one refers to the more complicated situation that the finite-state process is only partially observable. Although many practically applicable results have been developed, JLQG and adaptive JLQG control are lagging behind those for linear quadratic Gaussian (LQG) and adaptive LQG. The aim of this paper is to help improve the situation by introducing an exact transformation which embeds adaptive JLQG control into LQM (linear quadratic Martingale) control with a completely observable stochastic control matrix. By LQM control, the authors mean the control of a martingale driven linear system under a quadratic performance criterion. With the LQM transformation, the adaptive JLQG control can be studied within the framework of robust or minimax control without the need for the usual approach of averaging or approximating the adaptive JLQG dynamics. To show the effectiveness of the authors' transformation, it is used to characterize the open-loop-optimal feedback (OLOF) policy for adaptive JLQG control     

IS A MIXED DESIGN OF OBSERVER‐CONTROLLERS FOR TIME‐DELAY SYSTEMS INTERESTING?

This paper deals with H∞ observer‐based feedback control for linear time‐delay systems, in the framework of delay independent stability. We will propose a new LMI solution to observer‐controller design that ensures a disturbance attenuation level for the controlled output as well as for the state estimation error, which is an open problem. This will be compared with a well‐known solution and with a usual strategy in control which consists in designing the observer and the controller separately. Our aim is to try to bring a positive answer to the following question: is there an interest to solve the problem in a single (unique) formulation or should we design separately the observer and the controller? An application to a wind tunnel model is provided to emphasize the interest of the given results, particularly in comparison with existing results on H∞ observer‐based control.     

Observer‐based mode‐independent integral sliding mode controller design for phase‐type semi‐Markov jump singular systems

This paper considers the observer‐based integral sliding mode controller design problem of semi‐Markovian jumping singular systems with time‐varying delays. Firstly, by using a plant transformation and supplementary variable technique in the work of Hou et al, the discussed phase‐type semi‐Markov jump singular system is equivalently transformed into its associated Markov jump singular system. Secondly, an observer‐based sliding mode controller design problem is investigated for the associated singular Markov jump systems. The highlight of this paper is that we construct an observer‐based mode‐independent integral sliding mode surface function, which is different from the mode‐dependant sliding mode surface function in the previous literatures. Based on this, an observer‐based sliding mode controller is designed to guarantee that the associated singular Markov jump system meets the reachable condition. Finally, a practical example is presented to demonstrate the efficiency and effectiveness of our obtained results.     

Policy Gradient Approach of Event‐Based Optimization and Its Online Implementation

In the theory of event‐based optimization (EBO), the decision making is triggered by events, which is different from the traditional state‐based control in Markov decision processes (MDP). In this paper, we propose a policy gradient approach of EBO. First, an equation of performance gradient in the event‐based policy space is derived based on a fundamental quantity called Q‐factors of EBO. With the performance gradient, we can find the local optimum of EBO using the gradient‐based algorithm. Compared to the policy iteration approach in EBO, this policy gradient approach does not require restrictive conditions and it has a wider application scenario. The policy gradient approach is further implemented based on the online estimation of Q‐factors. This approach does not require the prior information about the system parameters, such as the transition probability. Finally, we use an EBO model to formulate the admission control problem and demonstrate the main idea of this paper. Such online algorithm provides an effective implementation of the EBO theory in practice.     

Observer‐based finite‐time stabilization for extended Markov jump systems

In this paper, we study the problem of observer‐based finite‐time stabilization for a class of extended Markov jump systems with norm‐bounded uncertainties and external disturbances. The stochastic character under consideration is governed by a finite‐state Markov process, but with only partial information on the transition jump rates. Based on the finite‐time stability analysis, sufficient conditions for the existence of the observer‐based controller are derived via a linear matrix inequality approach such that the closed‐loop system trajectory stays within a prescribed bound in a fixed time interval. When these conditions are satisfied, the designed observer‐based controller gain matrices can be obtained by solving a convex optimization problem. Simulation results demonstrate the effectiveness of the approaches proposed in this paper. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Stochastic stability and stabilization of a class of piecewise‐homogeneous Markov jump linear systems with mixed uncertainties

This paper investigates the stochastic stability and stabilization problem for a general class of uncertain, continuous‐time Markov jump linear systems (MJLSs). The system under consideration is a piecewise‐homogenous Markovian structure subject to piecewise‐constant time‐varying transition rates (TRs). The time variation of the TRs is characterized by a high‐level Markovian signal, which is independent from the low‐level Markovian mechanism that governs the switching between the system dynamics. It is assumed that the structure is subject to mixed uncertainties in the form of additive norm‐bounded terms. The uncertainties help to consider the effect of imperfections induced by modeling errors for the system dynamics and the TRs of Markovian signals of both levels. This new uncertain, two‐level Markovian jump linear system is a more general model than the existing ones and is applicable to more practical situations. Besides, it is capable of being specialized to uncertain piecewise‐homogeneous MJLS with arbitrarily varying TRs, as well as the uncertain time‐homogeneous MJLS. The stability/stabilizability of this system is first examined by constructing a Lyapunov function which depends on both switching signals. Then, based on the analysis results, the corresponding robust controller gains are synthesized through solving a set of linear matrix inequalities (LMIs). Finally, simulation results for an industrial stirred tank reactor (CSTR) are used to demonstrate the applicability and potentials of the proposed theoretical method. Comparative simulations are also provided to show the superiority of the presented approach to the existing ones. Copyright © 2016 John Wiley & Sons, Ltd.     

Support vector machines‐based generalized predictive control

In this study, we propose a novel control methodology that introduces the use of support vector machines (SVMs) in the generalized predictive control (GPC) scheme. The SVM regression algorithms have extensively been used for modelling nonlinear systems due to their assurance of global solution, which is achieved by transforming the regression problem into a convex optimization problem in dual space, and also their higher generalization potential. These key features of the SVM structures lead us to the idea of employing a SVM model of an unknown plant within the GPC context. In particular, the SVM model can be employed to obtain gradient information and also it can predict future trajectory of the plant output, which are needed in the cost function minimization block. Simulations have confirmed that proposed SVM‐based GPC scheme can provide a noticeably high control performance, in other words, an unknown nonlinear plant controlled by SVM‐based GPC can accurately track the reference inputs with different shapes. Moreover, the proposed SVM‐based GPC scheme maintains its control performance under noisy conditions. Copyright © 2006 John Wiley & Sons, Ltd.     

Global robust practical output regulation for nonlinear systems in output feedback form by output‐based event‐triggered control

In this paper, we study the event‐triggered global robust practical output regulation problem for a class of nonlinear systems in output feedback form with any relative degree. Our approach consists of the following three steps. First, we design an internal model and an observer to form the so‐called extended augmented system. Second, we convert the original problem into the event‐triggered global robust practical stabilization problem of the extended augmented system. Third, we design an output‐based event‐triggered control law and a Zeno‐free output‐based event‐triggered mechanism to solve the stabilization problem, which, in turn, leads to the solvability of the original problem. Finally, we apply our result to the controlled hyperchaotic Lorenz systems.     

Robust stability,stabilization and ℋ∞ control of time‐delay systems with Markovian jump parameters

In this paper, the problems of stochastic stability and stabilization for a class of uncertain time‐delay systems with Markovian jump parameters are investigated. The jumping parameters are modelled as a continuous‐time, discrete‐state Markov process. The parametric uncertainties are assumed to be real, time‐varying and norm‐bounded that appear in the state, input and delayed‐state matrices. The time‐delay factor is constant and unknown with a known bound. Complete results for both delay‐independent and delay‐dependent stochastic stability criteria for the nominal and uncertain time‐delay jumping systems are developed. The control objective is to design a state feedback controller such that stochastic stability and a prescribed ?_∞‐performance are guaranteed. We establish that the control problem for the time‐delay Markovian jump systems with and without uncertain parameters can be essentially solved in terms of the solutions of a finite set of coupled algebraic Riccati inequalities or linear matrix inequalities. Extension of the developed results to the case of uncertain jumping rates is also provided. Copyright © 2003 John Wiley & Sons, Ltd.     

A GRADIENT BASED ADAPTIVE CONTROL ALGORITHM FOR DUAL‐RATE SYSTEMS

In this paper, using a polynomial transformation technique, we derive a mathematical model for dual‐rate systems. Based on this model, we use a stochastic gradient algorithm to estimate unknown parameters directly from the dual‐rate input‐output data, and then establish an adaptive control algorithm for dual‐rate systems. We prove that the parameter estimation error converges to zero under persistent excitation, and the parameter estimation based control algorithm can achieve virtually asymptotically optimal control and ensure the closed‐loop systems to be stable and globally convergent. The simulation results are included.     

Strictly dissipative stabilization of multiple‐memory Markov jump systems with general transition rates: A novel event‐triggered control strategy

This paper investigates the strictly dissipative stabilization problem for multiple‐memory Markov jump systems with network communication protocol. Firstly, for reducing data transmission, we put forward a novel mode‐dependent event‐triggered communication scheme based on aperiodically sampled data. Secondly, a Markov jump system with general transition rates is considered to make the result more applicable, where the transition rates of some jumping modes allow to be completely known, or partially known, or even completely unknown. Thirdly, a less restrictive Lyapunov‐Krasovskii functional, which is only required to be positive definite at end points of each subinterval of the holding intervals, is first introduced for event‐triggered control issue. Based on the above methods, a sufficient condition with less conservatism is obtained to ensure the stochastic stability and dissipativity of the resulting closed‐loop system. Meanwhile, an explicit design method of the desired controller is achieved. Finally, two numerical examples are presented to demonstrate the effectiveness and advantage of the proposed method.     

Integrated fault detection and control for two‐dimensional Markovian jump systems

This paper is concerned with the problem of integrated fault detection and control for a class of two‐dimensional (2D) discrete‐time Markovian jump systems. The mathematical model of 2D Markovian jump systems is established upon the well‐known Roesser model, and a faults detection filter/controller is proposed to detect faults and meet some control specifications simultaneously. In this strategy, it takes into account both the fault detection objective and the control objective simultaneously through certain performance levels. The integrated design problem is then formulated as a multi‐objective optimization problem, which is nonconvex in essence. Furthermore, a two‐step algorithm is developed to solve this nonconvex problem. Sufficient conditions for existence of the desired fault detection filter/controller are established in terms of LMIs. A numerical example is used to demonstrate the effectiveness of the proposed method.     

Feedback predictive control for constrained fuzzy systems with Markovian jumps

In this paper, a feedback model predictive control method is presented to tackle control problems with constrained multivariables for uncertain discrete‐time nonlinear Markovian jump systems. An uncertain Markovian jump fuzzy system (MJFS) is obtained by employing the Takagi‐Sugeno (T‐S) fuzzy model to represent a discrete‐time nonlinear system with norm bounded uncertainties and Markovain jump parameters. To achieve more generality, the transition probabilities of the Markov chain are assumed to be partly unknown and partly accessible. The predictive formulation adopts an on‐line optimization paradigm that utilizes the closed‐loop state feedback controller and is solved using the standard semi‐definite programming (SDP). To reduce the on‐line computational burden, a mode independent control move is calculated at every sampling time based on a stochastic fuzzy Lyapunov function (FLF) and a parallel distributed compensation (PDC) scheme. The robust mean square stability, performance minimization and constraint satisfaction properties are guaranteed under the control move for all admissible uncertainties. A numerical example is given to show the efficiency of the developed approach. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Jump linear quadratic regulator with controlled jump rates

Deals with the class of continuous-time linear systems with Markovian jumps. We assume that jump rates are controlled. Our purpose is to study the jump linear quadratic (JLQ) regulator of the class of systems. The structure of the optimal controller is established. For a one-dimensional (1-D) system, an algorithm for solving the corresponding set of coupled Riccati equations of this optimal control problem is provided. Two numerical examples are given to show the usefulness of our results     

Recursive methods for estimating the radial basis function‐based state‐dependent autoregressive model

Identifying a nonlinear radial basis function‐based state‐dependent autoregressive (RBF‐AR) time series model is the basis for solving the corresponding prediction and control problems. This paper studies some recursive parameter estimation algorithms for the RBF‐AR model. Considering the difficulty of the nonlinear optimal problem arising in estimating the RBF‐AR model, an overall forgetting gradient algorithm is deduced based on the negative gradient search. A numerical method with a forgetting factor is provided to solve the problem of determining the optimal convergence factor. In order to improve the parameter estimation accuracy, the multi‐innovation identification theory is applied to develop an overall multi‐innovation forgetting gradient (O‐MIFG) algorithm. The simulation results indicate that the estimation model based on the O‐MIFG algorithm can capture the dynamics of the RBF‐AR model very well.     

Augmented Lagrangian Method for Generalized TV-Stokes Model

In this paper, we propose a general form of TV-Stokes models and provide an efficient and fast numerical algorithm based on the augmented Lagrangian method. The proposed model and numerical algorithm can be used for a number of applications such as image inpainting, image decomposition, surface reconstruction from sparse gradient, direction denoising, and image denoising. Comparing with properties of different norms in regularity term and fidelity term, various results are investigated in applications. We numerically show that the proposed model recovers jump discontinuities of a data and discontinuities of the data gradient while reducing stair-case effect.     

Two‐stage auxiliary model gradient‐based iterative algorithm for the input nonlinear controlled autoregressive system with variable‐gain nonlinearity

This article focuses on the parameter estimation problem of the input nonlinear system where an input variable‐gain nonlinear block is followed by a linear controlled autoregressive subsystem. The variable‐gain nonlinearity is described analytical by using an appropriate switching function. According to the gradient search technique and the auxiliary model identification idea, an auxiliary model‐based stochastic gradient algorithm with a forgetting factor is presented. For the sake of improving the parameter estimation accuracy, an auxiliary model gradient‐based iterative algorithm is proposed by utilizing the iterative identification theory. To further optimize the performance of the algorithm, we decompose the identification model of the system into two submodels and derive a two‐stage auxiliary model gradient‐based iterative (2S‐AM‐GI) algorithm by using the hierarchical identification principle. The simulation results confirm the effectiveness of the proposed algorithms and show that the 2S‐AM‐GI algorithm has higher identification efficiency compared with the other two algorithms.     

Codesign of controller,routing and scheduling in WirelessHART networked control systems

In this paper, we provide a novel methodology to co‐design controller, scheduling and routing in a wireless control network compliant with the WirelessHART protocol. We both provide a modeling framework and derive a novel suboptimal solution to the linear‐quadratic regulator problem for a class of systems that extends Markov jump linear system considering both continuous and discrete inputs. To allow that, our results can be directly implemented in a real WirelessHART network, we setup a receding horizon optimization problem that takes into account the constraint for compliance with WirelessHART and validate our solution on a batch reactor control loop.     

Optimal Control of Forward‐Backward Stochastic Jump‐Diffusion Differential Systems with Observation Noises: Stochastic Maximum Principle

This paper is concerned with a partially observed optimal control problem for a controlled forward‐backward stochastic system with correlated noises between the system and the observation, which generalizes the result of a previous work to a jump‐diffusion system. Under some convexity assumptions, necessary and sufficient optimality conditions for such an optimal control are established in the form of Pontryagin type maximum principle in a unified way by means of duality analysis and convex variational techniques     

Robust sampled‐data control for Itô stochastic Markovian jump systems with state delay

In this paper, the problem of robust sampled‐data control for Itô stochastic Markovian jump systems (Itô SMJSs) with state delay is investigated. Using parameters‐dependent Lyapunov functionals and some stochastic equations, we give stochastic sufficient stability criteria for polytopic uncertain Itô SMJSs. As a corollary, stochastic sufficient stability criteria are given for nominal Itô SMJSs. For this two cases of Itô SMJSs, based on the obtained stochastic stability criteria, their time‐independent sampled‐data controllers are designed, respectively. Then, for designing a time‐dependent sampled‐data controller for Itô SMJSs, a parameters‐dependent time‐scheduled Lyapunov functional is developed. New stochastic sufficient stability criteria are obtained for polytopic uncertain Itô SMJSs and nominal Itô SMJSs. Furthermore, their time‐dependent sampled‐data controllers are designed, respectively. Lastly, a numerical example is provided to illustrate the effectiveness of the proposed method.