Two-time-scale distributions and singular perturbations

Two-time-scale (TTS) distributions are introduced. For a class of stable systems, it is shown that every TTS distribution has a two-frequency-scale (TFS) Laplace transform. Conversely, it is shown that the impulse response of any stable TFS transfer function, and hence any stable (standard) singularly perturbed system, can be characterized in terms of a stable TTS distribution. A time domain decomposition for TTS distributions is obtained which parallels the slow and fast decomposition of singularly perturbed systems and also the frequency domain decomposition of TFS transfer functions. It is shown that every stable TTS distribution can be decomposed in terms of two simpler distributions represented in two different time scales. A composite distribution is constructed from these two which approximates the TTS distribution arbitrarily closely in the L ₁ norm.     

Stability of dynamic systems with aftereffect with due regard for Markov perturbations

Lyapunov-Krasovskii functionals and infinitesimal operators are employed to analyze a system for asymptotic stochastic stability on the whole and asymptotic stability on the whole. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 6, pp. 134–146, November–December 2007.     

Stability in impulsive systems with Markov perturbations in averaging scheme. I. Averaging principle for impulsive Markov systems

The Bogoliubov–Mitropolsky small parameter method is used to study the behavior of stochastic differential systems in the analysis of the corresponding properties of solutions of averaged systems.     

A new parameter estimate in singular perturbations

A new upper bound is obtained for the singular perturbation parameter of an asymptotically stable singularly perturbed system. General time-invariant systems with a single small parameter are considered. The paper employs a Riccati equation whose solution is known to facilitate the exact decoupling of fast and slow dynamics. An application of the Brouwer fixed point theorem to the Riccati equation and of Liapunov's direct method to the fast and slow subsystems results in the desired upper bound. Computation of the estimate requires only the solution of two Liapunov matrix equations.     

High-accuracy control systems with singular perturbations

In this paper, a linear quadratic problem for a continuous scalar control plant with stationary perturbations is considered. A class of perturbations is selected, in which arbitrary high accuracy of control by the criterion of steady-state response variance is ensured. An algorithm for synthesizing a stabilizing controller providing the desired control performance is proposed.     

Robustness of exponential stability to singular perturbations and delays

Singularly perturbed nonlinear differential equations with small time delays in the slow variables are considered. Averages of the fast variables are used in order to obtain a sufficient condition under which the exponential stability of the slow subsystem is robust to singular perturbations and delays.     

Adaptive control of constrained Markov chains

Adaptive control of finite-state Markov chains is discussed, The optimal performance is characterized through the minimization of a long-run average cost functional, subject to constraints on several other such functionals. Under mild structural and feasibility conditions, two explicit adaptive control policies are exhibited for the case where the transition probabilities are unknown. The policies are optimal under the constrained optimization criterion. They rely on a powerful estimation scheme which provides consistent estimators for the transition probabilities. This scheme is of independent interest, as it provides strong consistency under a large number of adaptive schemes and is independent of any identifiability conditions. As an application, an optimal adaptive policy is derived for a system of K competing queues with countable state space, for which the constrained criteria arise naturally in the context of communication networks     

Control of linear systems with multiparameter singular perturbations

The singular perturbation theory is extended to systems with several small parameters which can change the system order. Difficulties arising in testing the boundary layer stability in multiparameter linear problems are discussed. The theory is applied to linear quadratic optimal control and Nash game problems.     

Robustness of policies in constrained Markov decision processes

We consider the optimization of finite-state, finite-action Markov decision processes (MDPs), under constraints. Cost and constraints are discounted. We introduce a new method for investigating the continuity, and a certain type of robustness, of the optimal cost and the optimal policy under changes in the constraints. This method is also applicable for other cost criteria such as finite horizon and infinite horizon average cost.     

Adaptive control of linear systems with Markov perturbations

The stochastic model considered is a linear jump diffusion process X for which the coefficients and the jump processes depend on a Markov chain Z with finite state space. First, we study the optimal filtering and control problem for these systems with non-Gaussian initial conditions, given noisy observations of the state X and perfect measurements of Z. We derive a new sufficient condition which ensures the existence and the uniqueness of the solution of the nonlinear stochastic differential equations satisfied by the output of the filter. We study a quadratic control problem and show that the separation principle holds. Next, we investigate an adaptive control problem for a state process X defined by a linear diffusion for which the coefficients depend on a Markov chain, the processes X and Z being observed in independent white noises. Suboptimal estimates for the process X, Z and approximate control law are investigated for a large class of probability distributions of the initial state. Asymptotic properties of these filters and this control law are obtained. Upper bounds for the corresponding error are given     

Stability and stabilization of discrete‐time singular Markov jump systems with time‐varying delay

The stochastic stability and stochastic stabilization of time‐varying delay discrete‐time singular Markov jump systems are discussed. For full and partial knowledge of transition probabilities cases, delay‐dependent linear matrix inequalities (LMIs) conditions for the systems to be regular, causal and stochastically stable are given. Sufficient conditions are proposed for the existence of state feedback controller in terms of LMIs. Finally, two numerical examples to illustrate the effectiveness of the method are given. Copyright © 2009 John Wiley & Sons, Ltd.     

Robust stability of systems with application to singular perturbations

In this paper we will give a simple approach to determining conditions for stability of feedback systems subject to perturbations in the operators describing these systems. The approach is based on techniques used in functional analysis, and provides an alternative development and generalization of some conditions for the linear time-invariant case that have appeared in the literature very recently. As an example of the application of the conditions, we consider the determination of finite regions of stability for singularly perturbed systems.     

Control of flexible manipulators via singular perturbations and distributed vibration damping

A composite control strategy for a two-link flexible manipulator is analyzed which combines hub actuation with distributed vibration control. The hub actuation is based upon an integral manifold approach in which the system dynamics are approximately linearized to any order of a small parameter E representing stiffness of the robot arms. A polymer film is proposed as a distributed actuator to dampen vibrations due to elasticity in the links. Simulation results are provided which show that the addition of the distributed actuator significantly reduces the displacement and velocity of the first flexible mode in each link compared to hub actuation alone. Editor: T. Vincent     

On singular perturbations due to fast actuators in hybrid control systems

A stability result is given for hybrid control systems singularly perturbed by fast but continuous actuators. If a hybrid control system has a compact set globally asymptotically stable when the actuator dynamics are omitted, or equivalently, are infinitely fast, then the same compact set is semiglobally practically asymptotically stable in the finite speed of the actuator dynamics. This result, which generalizes classical results for differential equations, justifies using a simplified plant model that ignores fast but continuous actuator dynamics, even when using a hybrid feedback control algorithm.     

An actor-critic algorithm for constrained Markov decision processes

An actor-critic type reinforcement learning algorithm is proposed and analyzed for constrained controlled Markov decision processes. The analysis uses multiscale stochastic approximation theory and the envelope theorem' of mathematical economics.     

Multiparameter singular perturbations of linear systems with multiple time scales

A joint multitime scale-multiparameter singular perturbation is formulated and resolved in the context of linear time-varying systems. An interesting feature of the solution procedure is a hierarchial scheme of aggregating and arranging of the groups of small parameters according to their order. The scheme provides a suitable framework for establishing qualitative properties of multitime scale systems.     

Regulation of linear continuous-time singular systems with constrained states and controls

This article considers the constrained regulation problem (CRP) for linear continuous-time singular systems. The study consists in finding for a completely controllable singular system a linear state feedback control law that eliminates the impulsive behavior of a system and transfers asymptotically to the origin all initial states belonging to some polyhedral subset of the state space while respecting the linear constraints on state and control vectors. The proposed method gives a solution to the CRP for a singular system from a transformation that leads to a problem of positive invariance for a reduced order nonsingular one. Conditions guaranteeing the positive invariance for the whole domain of admissible controls are deduced. It is shown that the solution to the CRP is that of a nonlinear algebraic matrix equation.     

New stability criteria for singular systems with time-varying delay and nonlinear perturbations

This paper concerns the stability analysis for singular systems with time-varying delay and nonlinear perturbations. Two cases of time-varying delay, which is differentiable (Case 1) or not differentiable (Case 2), are considered. The considered nonlinear perturbations includes the norm-bounded uncertainties as a special case. Some delay-dependent stability criteria are derived by using a delay decomposition approach. In the delay decomposition approach, the entire delay interval is divided into multiple sub-intervals for which different energy functions are defined for building new Lyapunov–Krasovskii functional. Some numerical and practical examples are given to show the effectiveness and significant improvement of the proposed method.     

离散广义区间动力系统的稳定性

利用范数理论,研究离散广义区间动力系统的稳定性问题,给出其区间动力系统的正则、因果且稳定的充分条件.在此基础上,进一步考虑该系统的区间极点集的配置问题,给出了使离散广义区间动力系统的极点落在预先给定园盘内的判定定理.数值算例表明了该方法的可行性.     

Score-based methods for learning Markov boundaries by searching in constrained spaces

Within probabilistic classification problems, learning the Markov boundary of the class variable consists in the optimal approach for feature subset selection. In this paper we propose two algorithms that learn the Markov boundary of a selected variable. These algorithms are based on the score+search paradigm for learning Bayesian networks. Both algorithms use standard scoring functions but they perform the search in constrained spaces of class-focused directed acyclic graphs, going through the space by means of operators adapted for the problem. The algorithms have been validated experimentally by using a wide spectrum of databases, and their results show a performance competitive with the state-of-the-art.