Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses

In this paper, the stochastic optimal control of linear networked control system (NCS) with uncertain system dynamics and in the presence of network imperfections such as random delays and packet losses is derived. The proposed stochastic optimal control method uses an adaptive estimator (AE) and ideas from Q-learning to solve the infinite horizon optimal regulation of unknown NCS with time-varying system matrices. Next, a stochastic suboptimal control scheme which uses AE and Q-learning is introduced for the regulation of unknown linear time-invariant NCS that is derived using certainty equivalence property. Update laws for online tuning the unknown parameters of the AE to obtain the Q-function are derived. Lyapunov theory is used to show that all signals are asymptotically stable (AS) and that the estimated control signals converge to optimal or suboptimal control inputs. Simulation results are included to show the effectiveness of the proposed schemes. The result is an optimal control scheme that operates forward-in-time manner for unknown linear systems in contrast with standard Riccati equation-based schemes which function backward-in-time.     

OPTIMAL REJECTION WITH ZERO STEADY‐STATE ERROR OF SINUSOIDAL DISTURBANCES FOR TIME‐DELAY SYSTEMS

This paper studies the problem of optimal rejection with zero steady‐state error of sinusoidal disturbances for linear systems with time‐delay. Based on the internal model principle, a disturbance compensator is constructed to counterbalance the external sinusoidal disturbances, so that the original system can be transformed into an augmented system without disturbances. Then, with the introduction of a sensitivity parameter and expanding power series around it, the optimal disturbance rejection problem can be simplified to the problem of solving an infinite sum of a linear optimal control series without time‐delay or disturbance. The optimal control law for disturbance rejection with zero steady‐state error consists of accurate linear state feedback terms and a time‐delay compensating term, which is an infinite sum of an adjoint vector series. In the presented approach, iteration is required only for the time‐delay compensation series. By intercepting a finite sum of the compensation series, we obtain an approximate physically realizable optimal control law that avoids complex calculation. A numerical simulation shows that the algorithm is effective and easy to implement.     

An Optimal Hybrid Learning Approach for Attack Detection in Linear Networked Control Systems

A novel learning-based attack detection and estimation scheme is proposed for linear networked control systems (NCS), wherein the attacks on the communication network in the feedback loop are expected to increase network induced delays and packet losses, thus changing the physical system dynamics. First, the network traffic flow is modeled as a linear system with uncertain state matrix and an optimal Q-learning based control scheme over finite-horizon is utilized to stabilize the flow. Next, an adaptive observer is proposed to generate the detection residual, which is subsequently used to determine the onset of an attack when it exceeds a predefined threshold, followed by an estimation scheme for the signal injected by the attacker. A stochastic linear system after incorporating network-induced random delays and packet losses is considered as the uncertain physical system dynamics. The attack detection scheme at the physical system uses the magnitude of the state vector to detect attacks both on the sensor and the actuator. The maximum tolerable delay that the physical system can tolerate due to networked induced delays and packet losses is also derived. Simulations have been performed to demonstrate the effectiveness of the proposed schemes.      

Sampled‐data H∞ control for networked systems with random packet dropouts

This paper is concerned with the H_∞ control problem for networked control systems (NCSs) with random packet dropouts. The NCS is modeled as a sampled‐data system which involves a continuous plant, a digital controller, an event‐driven holder and network channels. In this model, two types of packet dropouts in the sensor‐to‐controller (S/C) side and controller‐to‐actuator (C/A) side are both considered, and are described by two mutually independent stochastic variables satisfying the Bernoulli binary distribution. By applying an input/output delay approach, the sampled‐data NCS is transformed into a continuous time‐delay system with stochastic parameters. An observer‐based control scheme is designed such that the closed‐loop NCS is stochastically exponentially mean‐square stable and the prescribed H_∞ disturbance attenuation level is also achieved. The controller design problem is transformed into a feasibility problem for a set of linear matrix inequalities (LMIs). A numerical example is given to illustrate the effectiveness of the proposed design method. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Robust optimal tracking control for multiplayer systems by off‐policy Q‐learning approach

In this article, a novel off‐policy cooperative game Q‐learning algorithm is proposed for achieving optimal tracking control of linear discrete‐time multiplayer systems suffering from exogenous dynamic disturbance. The key strategy, for the first time, is to integrate reinforcement learning, cooperative games with output regulation under the discrete‐time sampling framework for achieving data‐driven optimal tracking control and disturbance rejection. Without the information of state and input matrices of multiplayer systems, as well as the dynamics of exogenous disturbance and command generator, the coordination equilibrium solution and the steady‐state control laws are learned using data by a novel off‐policy Q‐learning approach, such that multiplayer systems have the capability of tolerating disturbance and follow the reference signal via the optimal approach. Moreover, the rigorous theoretical proofs of unbiasedness of coordination equilibrium solution and convergence of the proposed algorithm are presented. Simulation results are given to show the efficacy of the developed approach.     

Active disturbance rejection control for nonlinear fractional‐order systems

This paper investigates active disturbance rejection control involving the fractional‐order tracking differentiator, the fractional‐order PID controller with compensation and the fractional‐order extended state observer for nonlinear fractional‐order systems. Firstly, the fractional‐order optimal‐time control scheme is studied to propose the fractional‐order tracking differentiator by the Hamilton function and fractional‐order optimal conditions. Secondly, the linear fractional‐order extend state observer is offered to acquire the estimated value of the sum of nonlinear functions and disturbances existing in the investigated nonlinear fractional‐order plant. For the disturbance existing in the feedback output, the effect of the disturbance is discussed to choose a reasonable parameter in fractional‐order extended state observer. Thirdly, by this observed value, the nonlinear fractional‐order plant is converted into a linear fractional‐order plant by adding the compensation in the controller. With the aid of real root boundary, complex root boundary, and imaginary boot boundary, the approximate stabilizing boundary with respect to the integral and differential coefficients is determined for the given proportional coefficient, integral order and differential order. By choosing the suitable parameters, the fractional‐order active disturbance rejection control scheme can deal with the unknown nonlinear functions and disturbances. Finally, the illustrative examples are given to verify the effectiveness of fractional‐order active disturbance rejection control scheme. Copyright © 2015 John Wiley & Sons, Ltd.     

Robust stabilization for discrete‐time nonlinear singular systems with mixed time delays

This paper investigates the robust stability of discrete‐time singular systems involving nonlinear disturbance and mixed time delays. The mixed time delays are comprised of both discrete and distributed delays. The interval of discrete time delays can be divided into several subintervals, firstly. Then, in terms of linear matrix inequality (LMI), a suitable state feedback controller is designed for discrete‐time singular systems with nonlinear disturbance and mixed time delays, and the overall closed‐loop system is regular, causal and mean square asymptotically stable. Numerical examples are provided to show the usefulness and effectiveness of the proposed methods, and the results derived from our approaches are less conservative than existing ones. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

A periodic approach for input‐delay problems: Application to network controlled systems affected by polytopic uncertainties

This paper deals with robust stability and stabilization of linear discrete‐time systems subject to uncertainties and network constraints. In network control systems, the control loop is closed over a network, which induces additional dynamics to the original control loop such as delays, sampling, and quantization among many others. This paper focuses on networked induced delays due to unreliable network for which packet losses may occur. An equivalent periodic‐like representation of the resulting system is proposed. This allows first to revisit existing results in this framework and second to take model uncertainties into account by analyzing the closed‐loop model by means of a recent method based on robust control for discrete‐time time‐varying systems. Stability analysis and dynamic state‐feedback stabilization are characterized via new conditions, whose conservatism is extensively discussed. Effectiveness of the proposed methodology is illustrated by numerical examples. Copyright © 2015 John Wiley & Sons, Ltd.     

Worst‐case optimal control for an electrical drive system with time‐delay

This paper considers an optimal control developed for an electrical drive system with a DC motor. Since it is a linear control system with input time‐delay subject to unknown but bounded disturbances, we construct a worst‐case feedback control policy, which can guarantee that, for all admissible uncertain disturbances, the real system state should be in a prescribed neighborhood of a desired value at the given final time, and the cost functional takes the best guarantee value. The worst‐case feedback control policy is allowed to be corrected at a given set of correction points between the initial and the final time, which is equivalent to solving a (m‐1)‐level min‐max problem. Since the min‐max problem at each stage does not yield a simple analytical solution, construction of the optimal policy is computationally prohibitive. This is why we consider an approximate control policy which is more convenient for computation. The simulation results illustrate that this proposed approach is feasible. Copyright © 2009 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Global adaptive stabilization for high‐order uncertain time‐varying nonlinear systems with time‐delays

This paper focuses on the adaptive stabilization problem for a class of high‐order nonlinear systems with time‐varying uncertainties and unknown time‐delays. Time‐varying uncertain parameters are compensated by combining a function gain with traditional adaptive technique, and unknown multiple time‐delays are manipulated by the delicate choice of an appropriate Lyapunov function. With the help of homogeneous domination idea and recursive design, a continuous adaptive state‐feedback controller is designed to guarantee that resulting closed‐loop systems are globally uniformly stable and original system states converge to zero. The effectiveness of the proposed control scheme is illustrated by the stabilization of delayed neural network systems. Copyright © 2016 John Wiley & Sons, Ltd.     

Asymptotic Stability and Finite‐Time Stability of Networked Control Systems: Analysis and Synthesis

This paper focuses on the problems of asymptotic stability and finite‐time stability (FTS) analysis, along with the state feedback controller design for networked control systems (NCSs) with consideration of both network‐induced delay and packet dropout. The closed‐loop NCS is modeled as a discrete‐time linear system with a time‐varying, bounded state delay. Sufficient conditions for the asymptotic stability and the FTS of the closed‐loop NCS are provided, respectively. Based on the stability analysis results, a mixed controller design method, which guarantees the asymptotic stability of the closed‐loop NCS in the usual case and the FTS of the closed‐loop NCS in the unusual case (that is, in some particular time intervals, large state delay occurs), is presented. A numerical example is provided to illustrate the effectiveness of the proposed mixed controller design method.     

Robust H∞ control for networked systems with transmission delays and successive packet dropouts under stochastic sampling

This paper is concerned with the H_∞ performance analysis for networked control systems with transmission delays and successive packet dropouts under stochastic sampling. The parameter uncertainties are time‐varying norm‐bounded and appear in both the state and input matrices. If packet loss is considered the same as time delay, when models the networked control systems with successive packet dropouts and delays as ordinary linear system with input‐delay approach, due to sampling period is stochastic, then the delay caused by packet losses is a stochastic variable, which leads to difficulties in the stability analysis of the considered system. However, if we can transform the system with stochastic delay into a continuous system with stochastic parameter, we can solve the problem. In this paper, by assuming that the network packet loss rate and employing the information of probabilistic distribution of the time delays, the stochastic sampling system is transformed into a continuous‐time model with stochastic variable, which satisfies a Bernoulli distribution. By linear matrix inequality approach, sufficient conditions are obtained, which guarantee the robust mean‐square exponential stability of the system with an H_∞ performance. What's more, an H_∞ controller design procedure is then proposed, and a less conservative result is obtained by taking the probability into consideration. Finally, a numerical simulation example is employed to show the effectiveness of the obtained results. Copyright © 2016 John Wiley & Sons, Ltd.     

Adaptive/robust time‐varying stabilization of second‐order non‐holonomic chained form with input uncertainties

Adaptive and robust time‐varying control schemes are constructed to stabilize second‐order non‐holonomic chained form in the presence of input uncertainties. The proposed control schemes guarantee that all the state variables converge to zero asymptotically in spite of input uncertainties, and are applied to the stabilization of a planar rigid body driven by active force and torque with unknown inertia and geometric parameters. The basic idea of the proposed stabilization schemes is to first convert the non‐holonomic system into a linear time‐varying form by time‐varying co‐ordinate transformation, and then design control laws to stabilize the converted linear time‐varying system. Copyright © 2002 John Wiley & Sons, Ltd.     

On designing network‐based H ∞ controllers for stochastic systems

This paper is concerned with network‐based H _∞ stabilization for stochastic systems, where network‐induced delays, packet dropouts, and packet disorders are taken into account simultaneously. The packet disorders arising from both the sampler‐to‐controller channel and the controller‐to‐actuator channel are considered by introducing a logic controller and a logic zero‐order hold. The network‐induced delays and packet dropouts are modeled as a constant delay plus a non‐differentiable time‐varying delay in the input. By employing Lyapunov–Krasovskii functional approach, we establish results that parallel well‐known bounded real Lemmas. More specifically, these results provide conditions to bound the H _∞ level of the system, which means the worst case energy of the output of the system when subjected to a unitary norm deterministic disturbance signal. On the basis of these results, suitable network‐based H _∞ controllers are designed by using cone complementary linearization method. An air vehicle system is finally taken as an example to show the effectiveness of the proposed method. Copyright © 2013 John Wiley & Sons, Ltd.     

Fundamental Issues in Networked Control Systems

This paper provides a survey on modeling and theories of networked control systems (NCS). In the first part, modeling of the different types of imperfections that affect NCS is discussed. These imperfections are quantization errors, packet dropouts, variable sampling/transmission intervals, variable transmission delays, and communication constraints. Then follows in the second part a presentation of several theories that have been applied for controlling networked systems. These theories include: input delay system approach, Markovian system approach, switched system approach, stochastic system approach, impulsive system approach, and predictive control approach. In the last part, some advanced issues in NCS including decentralized and distributed NCS, cloud control system, and co-design of NCS are reviewed.      

Sampled‐data control of networked nonlinear systems with variable delays and drops

This paper investigates the stabilization problem of the nonlinear networked control systems (NCSs) with drops and variable delays. The NCS is modeled as a sampled‐data system. For such a sampled‐data NCS, the stability properties are studied for delay that can be both shorter and longer than one sampling period, respectively. The exponential stability conditions are derived in terms of the parameters of the plant and time delay. On the other hand, a model‐based control scheme based on an approximate discrete‐time model of the plant is presented to guarantee the stability of the closed‐loop system subject to variable time delays and packet losses. The performance of the proposed control schemes are examined through numerical simulations of an automated rendezvous and docking of spacecraft system. Moreover, the simulations show that by employing the model‐based controller, a higher closed‐loop control performance can be achieved. Copyright © 2013 John Wiley & Sons, Ltd.     

Network‐based H∞ control for stochastic systems

This paper investigates the problem of network‐based control for stochastic plants. A new model of stochastic time‐delay systems is presented where both network‐induced delays and packet dropouts are taken into consideration for a sampled‐data network‐based control system. This model consists of two successive delay components in the state, and we solve the network‐based H_∞ control problem based on this model by a new stochastic delay system approach. The controller design for the sampled‐data systems is carried out in terms of linear matrix inequalities. Finally, we illustrate the methodology by applying these results to an air vehicle control problem. Copyright © 2008 John Wiley & Sons, Ltd.     

Almost Disturbance Decoupling for a Class of Nonlinear Systems Subject to Time‐Delays Via Sampled‐Data Output Feedback Control

This paper considers the problem of almost disturbance decoupling (ADD) via sampled‐data output feedback control for a class of uncertain nonlinear systems subject to time‐delays. Based on output feedback domination approach, a sampled‐data output feedback controller is designed to globally stabilize the system under a lower‐triangular linear growth condition. Gronwall‐Bellman‐like inequality and inductive method are introduced to estimate the state growth in the presence of time‐delays, uncertain nonlinearities and unknown disturbances. The proposed controller can attenuate the influence of disturbances on the output to an arbitrary degree in the L₂ gain sense. Finally, simulation results show the effectiveness of the control method.     

Stabilization of linear discrete‐time networked control systems via protocol and controller co‐design

This paper investigates the stabilization problem for networked control systems (NCSs) with communication constraint and packet loss. The communication constraint considered is that only one network node is allowed to access a shared communication channel during one time‐slot, and a feedback control is performed with only partially available measurements and control inputs. By taking random packet loss into consideration, a stochastic switched system model is presented to describe the NCS. A sufficient condition is derived for the NCS to be mean‐square exponentially stable, and it is shown that the system performance specified by the exponential decay rate critically depends on the network accessing rates (NARs) of the network nodes and the packet loss probability. The state feedback controller and scheduling protocol, which allocates the NARs, are co‐designed such that the NCS achieves a minimal decay rate. Finally, an illustrative example is given to show the effectiveness of the proposed design approach. Copyright © 2014 John Wiley & Sons, Ltd.     

Polynomial filtering for nonlinear stochastic systems with state‐ and disturbance‐dependent noises

This article is concerned with the polynomial filtering problem for a class of nonlinear stochastic systems governed by the Itô differential equation. The system under investigation involves polynomial nonlinearities, unknown‐but‐bounded disturbances, and state‐ and disturbance‐dependent noises ((x,d)‐dependent noises for short). By expanding the polynomial nonlinear functions in Taylor series around the state estimate, a new polynomial filter design method is developed with hope to reduce the conservatism of the existing results. In virtue of stochastic analysis and inequality technique, sufficient conditions in terms of parameter‐dependent linear matrix inequalities (PDLMIs) are derived to guarantee that the estimation error system is input‐to‐state stable in probability. Moreover, the desired polynomial matrix can be obtained by solving the PDLMIs via the sum‐of‐squares approach. The effectiveness and applicability of the proposed method are illustrated by two numerical examples with one concerning the permanent magnet synchronous motor.