Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems

In this work, probabilistic reachability over a finite horizon is investigated for a class of discrete time stochastic hybrid systems with control inputs. A suitable embedding of the reachability problem in a stochastic control framework reveals that it is amenable to two complementary interpretations, leading to dual algorithms for reachability computations. In particular, the set of initial conditions providing a certain probabilistic guarantee that the system will keep evolving within a desired ‘safe’ region of the state space is characterized in terms of a value function, and ‘maximally safe’ Markov policies are determined via dynamic programming. These results are of interest not only for safety analysis and design, but also for solving those regulation and stabilization problems that can be reinterpreted as safety problems. The temperature regulation problem presented in the paper as a case study is one such case.     

Reachability for partially observable discrete time stochastic hybrid systems

When designing optimal controllers for any system, it is often the case that the true state of the system is unknown to the controller. Imperfect state information must be taken into account in the controller’s design in order to preserve its optimality. The same is true when performing reachability calculations. To estimate the probability that the state of a stochastic system reaches, or stays within, some set of interest in a given time horizon, it is necessary to find a controller that drives the system to that set with maximum probability, given the controller’s knowledge of the true state of the system. To date, little work has been done on stochastic reachability calculations with partially observable states. The work that has been done relies on converting the reachability optimization problem to one with an additive cost function, for which theoretical results are well known. Our approach is to preserve the multiplicative cost structure when deriving a sufficient statistic that reduces the problem to one of perfect state information. Our transformation includes a change of measure that simplifies the distribution of the sufficient statistic conditioned on its previous value. We develop a dynamic programming recursion for the solution of the equivalent perfect information problem, proving that the recursion is valid, an optimal solution exists, and results in the same solution as to the original problem. We also show that our results are equivalent to those for the reformulated additive cost problem, and so such a reformulation is not required.     

Safety verification and reachability analysis for hybrid systems

Safety verification and reachability analysis for hybrid systems is a very active research domain. Many approaches that seem quite different, have been proposed to solve this complex problem. This paper presents an overview of various approaches for autonomous, continuous-time hybrid systems and presents them with respect to basic problems related to verification.     

Stability and switching control design issues for a class of discrete time hybrid systems

For the class of systems considered, necessary and sufficient stabilizability conditions are unknown. However, by considering the same systems with unknown but bounded exogenous disturbances, we give finitely computable conditions, sufficient for stabilizability without disturbances, yet necessary for stabilizability with disturbances.     

Specifications for decidable hybrid games

We introduce STORMED hybrid games (SHG), a generalization of STORMED hybrid systems, Vladimerou et al. (2008) [33], which have natural specifications that allow rich continuous dynamics and various decidable properties. We solve the control problem for SHG using a reduction to bisimulation on finite game graphs. This generalizes to a greater family of games, which includes o-minimal hybrid games, Bouyer et al. (2006) [6]. We also solve the optimal-cost reachability problem for Weighted SHG and prove decidability of WCTL for Weighted STORMED hybrid systems.     

Sliding mode control of singular stochastic hybrid systems

This paper is concerned with the sliding mode control (SMC) of nonlinear singular stochastic systems with Markovian switching. An integral sliding surface function is designed, and the resulting sliding mode dynamics is a full-order Markovian jump singular stochastic system. By introducing some specified matrices, a new sufficient condition is proposed in terms of strict linear matrix inequality (LMI), which guarantees the stochastic stability of the sliding mode dynamics. Then, a SMC law is synthesized for reaching motion. Moreover, when there exists an external disturbance, the ?₂ disturbance attenuation performance is analyzed for the sliding mode dynamics. Some related sufficient conditions are also established.     

On hybrid control of a class of stochastic non-linear Markovian switching systems

In this paper, we address the problem of hybrid control for a class of stochastic non-linear Markovian switching systems. First, a hybrid controller is introduced for the systems. Then under some appropriate assumptions, the stabilization condition for the systems under pure impulsive control is given. Further under impulsive control, the output feedback stabilization problem of the systems is discussed and linear output feedback controllers are designed. Finally a numerical example is provided to illustrate the effectiveness of the proposed methods.     

Span-reachability and observability of bilinear hybrid systems

The paper presents an algebraic characterization of observability and span-reachability of bilinear hybrid systems without guards, i.e. hybrid systems whose continuous dynamics are determined by bilinear control systems, and whose discrete transitions are triggered externally. The proposed characterization provides necessary and sufficient conditions for observability and span-reachability in terms of rank conditions for generalized observability and controllability matrices. This characterization is useful for the system theory of bilinear hybrid systems, in particular, it is used in the realization theory of bilinear hybrid systems, (Petreczky, 2006).     

Sporadic event-based control of first-order linear stochastic systems

The standard approach in computer-controlled systems is to sample and control periodically. In certain applications, such as networked control systems or energy-constrained systems, it could be advantageous to instead use event-based control schemes. Aperiodic event-based control of first-order stochastic systems has been investigated in previous work. In any real implementation, however, it is necessary to have a well-defined minimum inter-event time. In this paper, we explore two such sporadic control schemes for first-order linear stochastic systems and compare the achievable performance to both periodic and aperiodic control. The results show that sporadic control can give better performance than periodic control in terms of both reduced process state variance and reduced control action frequency.     

A control synthesis approach for time discrete event systems

In this paper, we introduce a control synthesis method for discrete event systems whose behavior is dependent on explicit values of time. Our goal is to control the occurrence dates of the controllable events so that the functioning of the system respects given specifications. The system to be controlled is modeled by a time Petri net. In a previous work we proposed a systematic method to build the timed automaton which models the exact behavior of a time Petri net. Furthermore, the forbidden behaviors of the system are modeled by forbidden timed automaton locations. This paper focuses on the control synthesis method, which consists in computing new firing conditions for the timed automaton transitions so that the forbidden locations are no longer reachable.     

Near optimal control for a class of stochastic hybrid systems

To address a computationally intractable optimal control problem for a class of stochastic hybrid systems, this paper proposes a near optimal state feedback control scheme, which is constructed by using a statistical prediction method based on approximate numerical solution that samples over the entire state space. A numerical example illustrates the potential of the approach.     

Minimax games for stochastic systems subject to relative entropy uncertainty: applications to SDEs on Hilbert spaces

In this paper, we consider minimax games for stochastic uncertain systems with the pay-off being a nonlinear functional of the uncertain measure where the uncertainty is measured in terms of relative entropy between the uncertain and the nominal measure. The maximizing player is the uncertain measure, while the minimizer is the control which induces a nominal measure. Existence and uniqueness of minimax solutions are derived on suitable spaces of measures. Several examples are presented illustrating the results. Subsequently, the results are also applied to controlled stochastic differential equations on Hilbert spaces. Based on infinite dimensional extension of Girsanov’s measure transformation, martingale solutions are used in establishing existence and uniqueness of minimax strategies. Moreover, some basic properties of the relative entropy of measures on infinite dimensional spaces are presented and then applied to uncertain systems described by a stochastic differential inclusion on Hilbert space. An explicit expression for the worst case measure representing the maximizing player (adversary) is found.     

Static output feedback control for stochastic hybrid systems: LMI approach

This paper deals with the stabilization problem of the class of uncertain stochastic hybrid systems. The uncertainties are norm bounded type. Under the complete access to the system mode a static output feedback controller that makes the closed-loop dynamics of this class of systems regular, impulse-free and stochastically stable is designed. The gains of this controller are the solution of some linear matrix inequalities (LMIs).     

Connections between stochastic control and dynamic games

We consider duality relations between risk-sensitive stochastic control problems and dynamic games. They are derived from two basic duality results, the first involving free energy and relative entropy and resulting from a Legendre-type transformation, the second involving power functions. Our approach allows us to treat, in essentially the same way, continuous- and discrete-time problems, with complete and partial state observation, and leads to a very natural formal justification of the structure of the cost functional of the dual. It also allows us to obtain the solution of a stochastic game problem by solving a risk-sensitive control problem.     

Kalman filter for controlled hybrid systems

The linear partially observed discrete-continuous (hybrid) stochastic controllable system described by differential equations with measures is considered. The optimal filtering equations in the form of generalized Kalman filter are obtained in the case of non-anticipating control. This result could be a theoretical basis for the optimal control in stochastic hybrid systems with incomplete information.     

Fault diagnosis in discrete time hybrid systems - A case study

A method of analysing diagnosability of discrete time hybrid systems (DTHS), which are similar to the simple n-rate timed automata [R. Alur, C. Courcoubetis, T.A. Henzinger, P. Ho, Hybrid automata: an algorithmic approach to the specification and verification of hybrid systems, in: Hybrid Systems, LNCS 736, Springer Verlag, 1993, pp. 209-229], has been proposed. A state based fault modeling formalism is used. The properties of the DTHS model, under measurement limitations due to inadequacy or non-availability of sensors, are discussed. A definition of diagnosability for DTHS models has been adopted from the one proposed in [M. Sampath, R. Sengupta, S. Lafortune, K. Sinnamohideen, D. Teneketzis, Diagnosability of discrete-event systems, IEEE Transactions on Automatic Control 40 (9) (1995) 1555-1575] for discrete-event system (DES) models. Based on the measurement limited DTHS models, an algorithm for construction of a diagnoser is presented. It is next demonstrated through an example of a chemical reaction chamber that the diagnosability condition (over the diagnoser), which has been shown to be necessary and sufficient for DES diagnosability, fails to hold for many systems. This is so because the abstraction employed in DES modeling obliterates an important feature of the transitions namely fairness. Exploiting the explicit continuous dynamics of the DTHS models, the fairness of transitions is identified and used to demonstrate diagnosability. The diagnosability condition over the diagnoser is suitably modified to encompass the situations typified by the example.     

Reachability analysis of complex planar hybrid systems

Hybrid systems are systems that exhibit both discrete and continuous behavior. Reachability, the question of whether a system in one state can reach some other state, is undecidable for hybrid systems in general. In this paper we are concerned with GSPDIs, 2-dimensional systems generalizing SPDIs (planar hybrid systems based on “simple polygonal differential inclusions”), for which reachability have been shown to be decidable. GSPDIs are useful to approximate 2-dimensional control systems, allowing the verification of safety properties of such systems.     

Control for discrete singular hybrid systems

The problems of stability, state feedback control and static output feedback control for a class of discrete-time singular hybrid systems are investigated in this paper. A new sufficient and necessary condition for a class of discrete-time singular hybrid systems to be regular, causal and stochastically stable is proposed in terms of a set of coupled strict linear matrix inequalities (LMIs). Sufficient conditions are proposed for the existence of state feedback controller and static output feedback controller in terms of a set of coupled strict LMIs, respectively. Finally, two illustrative examples are provided to demonstrate the applicability of the proposed approach.     

Soft-constrained stochastic Nash games for weakly coupled large-scale systems

In this paper, we discuss infinite-horizon soft-constrained stochastic Nash games involving state-dependent noise in weakly coupled large-scale systems. First, we formulate linear quadratic differential games in which robustness is attained against model uncertainty. It is noteworthy that this is the first time conditions for the existence of robust equilibria have been derived based on the solutions of sets of cross-coupled stochastic algebraic Riccati equations (CSAREs). After establishing an asymptotic structure with positive definiteness for CSAREs solutions, we derive a recursive algorithm by means of Newton’s method so that it can be used to obtain solutions for CSAREs. As another important feature, we propose a high-order approximate Nash strategy based on iterative solutions. Finally, we provide a numerical example to verify the efficiency of the proposed algorithms.