Stochastic algorithms for robustness of control performances

In recent years, there has been a growing interest in developing statistical learning methods to provide approximate solutions to “difficult” control problems. In particular, randomized algorithms have become a very popular tool used for stability and performance analysis as well as for design of control systems. However, as randomized algorithms provide an efficient solution procedure to the “intractable” problems, stochastic methods bring closer to understanding the properties of the real systems. The topic of this paper is the use of stochastic methods in order to solve the problem of control robustness: the case of parametric stochastic uncertainty is considered. Necessary concepts regarding stochastic control theory and stochastic differential equations are introduced. Then a convergence analysis is provided by means of the Chernoff bounds, which guarantees robustness in mean and in probability. As an illustration, the robustness of control performances of example control systems is computed.     

Attraction, stability and robustness for stochastic functional differential equations with infinite delay

This paper establishes the stochastic LaSalle theorem to locate limit sets for stochastic functional differential equations with infinite delay, from which some criteria on attraction, boundedness, stability and robustness are obtained. To illustrate the applications of our results clearly, this paper considers a scalar stochastic integro-differential equation with infinite delay as an example.     

A note on robustness tolerances for combinatorial optimization problems

We consider so-called generic combinatorial optimization problem, where the set of feasible solutions is some family of nonempty subsets of a finite ground set with specified positive initial weights of elements, and the objective function represents the total weight of elements of the feasible solution. We assume that the set of feasible solutions is fixed, but the weights of elements may be perturbed or are given with errors. All possible realizations of weights form the set of scenarios.A feasible solution, which for a given set of scenarios guarantees the minimum value of the worst-case relative regret among all the feasible solutions, is called a robust solution. The maximum percentage perturbation of a single weight, which does not destroy the robustness of a given solution, is called the robustness tolerance of this weight with respect to the solution considered.In this paper we give formulae for computing the robustness tolerances with respect to an optimal solution obtained for some initial weights and we show that this can be done in polynomial time whenever the optimization problem is polynomially solvable itself.     

Transition probability bounds for the stochastic stability robustness of continuous- and discrete-time Markovian jump linear systems

This paper considers the robustness of stochastic stability of Markovian jump linear systems in continuous- and discrete-time with respect to their transition rates and probabilities, respectively. The continuous-time (discrete-time) system is described via a continuous-valued state vector and a discrete-valued mode which varies according to a Markov process (chain). By using stochastic Lyapunov function approach and Kronecker product transformation techniques, sufficient conditions are obtained for the robust stochastic stability of the underlying systems, which are in terms of upper bounds on the perturbed transition rates and probabilities. Analytical expressions are derived for scalar systems, which are straightforward to use. Numerical examples are presented to show the potential of the proposed techniques.     

A generalized certainty-equivalence result in stochastic control

The certainty-equivalence property, known to hold under certain assumptions for the linear-quadratic stochastic control problem, is extended to more complex situations, when the system noise variables are related to a complicated (nonlinear) system, whose state can be partially observed.     

Equivalent stochastic control problems

A decentralized stochastic control problem is called static if the observations available for any one decision do not depend on the other decisions. Otherwise it is called dynamic. We consider only problems with a finite number of decisions. A notion of equivalence between problems, suitable for complexity analysis, is defined. It turns out that a large class of dynamic problems can be reduced to equivalent static problems. The class includes all sequential discrete variable problems and some of the most studied continuous variable problems.     

Suboptimal stochastic linear feedback control of linear systems with state- and control-dependent noise: The incomplete information case

The stochastic regulation problem for linear systems with state- and control-dependent noise and a noisy linear output equation is considered. The optimal quadratic cost output-feedback control law in a class of linear controllers is found. This problem was first addressed in the early 1970s and solved, in the complete information case, by Wonham. In this paper we give the solution of the problem in the incomplete information case, that is, for a linear output equation corrupted by Gaussian noise. Moreover, a different method is used here, giving the solution in a more direct way even in the complete information case.     

A new vertex result for robustness problems with interval matrix uncertainty

This paper¹ addresses a family of robustness problems in which the system under consideration is affected by interval matrix uncertainty. The main contribution of the paper is a new vertex result that drastically reduces the number of extreme realizations required to check robust feasibility. This vertex result allows one to solve, in a deterministic way and without introducing conservatism, the corresponding robustness problem for small and medium size problems. For example, consider quadratic stability of an autonomous n_x dimensional system. In this case, instead of checking vertices, we show that it suffices to check 2^2n_x specially constructed systems. This solution is still exponential, but this is not surprising because the problem is NP-hard. Finally, vertex extensions to multiaffine interval families and some sufficient conditions (in LMI form) for robust feasibility are presented. Some illustrative examples are also given.     

Robustness analysis of identification and adaptive control for stochastic systems

This paper gives a preliminary robustness analysis for stochastic systems with unmodelled dynamics. The extended least squares algorithm and stochastic adaptive control for tracking are defined on the basis of the modelled part of the system, and it is shown that the parameter estimate is closed to the true one and the tracking error is small if the unmodelled dynamics is small.     

A transformation method for stochastic control problems with partial observations

Two stochastic control problems with partial observations are studied, one where the policy or control law depends only on the latest observation (the controller does not have recall of observations), and the other with the standard partial observations model. The equivalent full observation problems are formulated, and the equivalence is proven using a new method. The results are illustrated with a simple example.     

Discrete-time stochastic control systems: A continuous Lyapunov function implies robustness to strictly causal perturbations

Discrete-time stochastic systems employing possibly discontinuous state-feedback control laws are addressed. Allowing discontinuous feedbacks is fundamental for stochastic systems regulated, for instance, by optimization-based control laws. We introduce generalized random solutions for discontinuous stochastic systems to guarantee the existence of solutions and to generate enough solutions to get an accurate picture of robustness with respect to strictly causal perturbations. Under basic regularity conditions, the existence of a continuous stochastic Lyapunov function is sufficient to establish that asymptotic stability in probability for the closed-loop system is robust to sufficiently small, state-dependent, strictly causal, worst-case perturbations. Robustness of a weaker stochastic stability property called recurrence is also shown in a global sense in the case of state-dependent perturbations, and in a semiglobal practical sense in the case of persistent perturbations. An example shows that a continuous stochastic Lyapunov function is not sufficient for robustness to arbitrarily small worst-case disturbances that are not strictly causal. Our positive results are also illustrated by examples.     

Performance and robustness trade-offs in PID control

Control design is a rich problem which requires consideration of many issues such as load disturbance attenuation, set-point tracking, robustness with respect to process variations and model uncertainty, and effects of measurement noise. The purpose of this paper is to provide insight into the trade-offs between performance and robustness explicitly. This is accomplished by introducing plots that show the trade-offs for PI and PID control. These also provide valuable understanding of design compromises used for common PI design methods.     

The zero divisor problem of multivariable stochastic adaptive control

Stochastic adaptive minimum variance control algorithms require a division by a function of a recursively computed parameter estimate at each instant of time. In order that the analysis of these algorithms is valid, zero divisions must be events of probability zero. This property is established for the stochastic gradient adaptive control algorithm under the condition that the initial state of the system and all finite segments of its random disturbance process have a joint distribution which is absolutely continuous with respect to Lebesgue measure. This result is deduced from the following general result established in this paper: a non-constant rational function of a finite set of random variables {x₁},x_n} is absolutely continuous with respect to Lebesgue measure if the joint distribution function of {x₁,…,x_n} has this property.     

Suboptimal stochastic H-two/H-infinity design with spectrum constraint

This paper studies the problem of suboptimal state-feedback H-two/H-infinity control of stochastic systems with spectrum constraint. Concretely speaking, a mixed suboptimal H-two/H-infinity controller synthesis together with placing the spectrum into a strip region is considered, which is achieved via solving a convex optimization problem.     

Moving least-squares approximations for linearly-solvable stochastic optimal control problems

Nonlinear stochastic optimal control problems are fundamental in control theory.A general class of such problems can be reduced to computing the principal eigenfunction of a linear operator.Here,we describe a new method for finding this eigenfunction using a moving least-squares function approximation.We use efficient iterative solvers that do not require matrix factorization,thereby allowing us to handle large numbers of basis functions.The bases are evaluated at collocation states that change over iterations of the algorithm,so as to provide higher resolution at the regions of state space that are visited more often.The shape of the bases is automatically defined given the collocation states,in a way that avoids gaps in the coverage.Numerical results on test problems are provided.     

Internal stability of dynamic quantised control for stochastic linear plants

Existing analyses of ‘zooming’ quantisation schemes for bit-rate-limited control systems rely on the encoder and controller being initialised with identical internal states. Due to the quantiser discontinuity and the plant instability, it was not clear if closed-loop stability was possible if the encoder and controller commenced from different initial conditions. In this article, we consider partially observed, unstable linear time-invariant plants, with unbounded and possibly non-Gaussian noise, and propose a modified zooming-like scheme with finite-dimensional internal encoder and controller states that may not initially be identical. Using a stochastic pseudo-norm, we prove that this scheme yields mean-square stability in all closed-loop state variables, not just the plant state, under a sufficient condition involving this initial error, the plant dynamics and the channel data rate. With diminishing initial error, this condition approaches a known universal lower bound on data rates and becomes tight. Furthermore, we show that the scheme automatically corrects itself, in the sense that the errors between the internal states of the encoder and controller tend to zero stochastically with time. This suggests that the policy will maintain stability in the presence of channel errors, for sufficiently low bit error rates. We support these conclusions with simulations.     

A novel approach to output feedback control of fuzzy stochastic systems

This paper investigates the problem of Hankel-norm output feedback controller design for a class of T–S fuzzy stochastic systems. The full-order output feedback controller design technique with the Hankel-norm performance is proposed by the fuzzy-basis-dependent Lyapunov function approach and the conversion on the Hankel-norm controller parameters. Sufficient conditions are established to design the controllers such that the resulting closed-loop system is stochastically stable and satisfies a prescribed performance. The desired output feedback controller can be obtained by solving a convex optimization problem, which can be efficiently solved by standard numerical algorithms. Finally, a Henon map system is used to illustrate the effectiveness of the proposed techniques.     

A stochastic control model of economic growth with environmental disaster prevention

This paper proposes a capital accumulation model with a random stopping time corresponding to the occurrence of an environmental catastrophe. Depending on the preventive capital stock accumulated at the time of the catastrophe, the damage cost associated with the catastrophe varies. The long-term behavior of the optimal accumulation path is analyzed using turnpike theory. The case where the catastrophe process is uncontrolled is distinguished from the case where there is an anthropogenic effect on the probability of an occurrence. Intergenerational equity issues are discussed. Numerical experiments with an adaptation of the integrated assessment model DICE94 are proposed to explore the model responses.     

A stochastic control model for optimal timing of climate policies

A stochastic control model is proposed as a paradigm for the design of optimal timing of greenhouse gas (GHG) emission abatement. The resolution of uncertainty concerning climate sensitivity and the technological breakthrough providing access to a carbon-free production economy are modeled as controlled stochastic jump processes. The optimal policy is characterized using the dynamic programming solution to a piecewise deterministic optimal control problem. A numerical illustration is developed with a set of parameters calibrated on recently proposed models for integrated assessment of climate policies. The results are interpreted and the insights they provide on the timing issue of climate policy are discussed.