General Smoothing Formulas for Markov-Modulated Poisson Observations

In this paper, we compute general smoothing dynamics for partially observed dynamical systems generating Poisson observations. We consider two model classes, each Markov modulated Poisson processes, whose stochastic intensities depend upon the state of an unobserved Markov process. In one model class, the hidden state process is a continuously-valued ItÔ process, which gives rise to a continuous sample-path stochastic intensity. In the other model class, the hidden state process is a continuous-time Markov chain, giving rise to a pure jump stochastic intensity. To compute filtered estimates of state process, we establish dynamics, whose solutions are unnormalized marginal probabilities; however, these dynamics include Lebesgue–Stieltjes stochastic integrals. By adapting the transformation techniques introduced by J. M. C. Clark, we compute filter dynamics which do not include these stochastic integrals. To construct smoothers, we exploit a duality between our forward and backward transformed dynamics and thereby completely avoid the technical complexities of backward evolving stochastic integral equations. The general smoother dynamics we present can readily be applied to specific smoothing algorithms, referred to in the literature as: Fixed point smoothing, fixed lag smoothing and fixed interval smoothing. It is shown that there is a clear motivation to compute smoothers via transformation techniques similar to those presented by J. M. C. Clark, that is, our smoothers are easily obtained without recourse to two sided stochastic integration. A computer simulation is included.     

White noise theory of robust nonlinear filtering with correlated state and observation noises

In the existing ‘direct’ white noise theory of nonlinear filtering, the state process is still modelled as a Markov process satisfying an Itô stochastic differential equation, while a ‘finitely additive’ white noise is used to model the observation noise. We remove this asymmetry by modelling the state process as the solution of a (stochastic) differential equation with a ‘finitely additive’ white noise as the input. This enables us to introduce correlation between the state and observation noises, and to obtain robust nonlinear filtering equations in the correlated noise case.     

Optimal preview control for a linear continuous-time stochastic control system in finite-time horizon

This paper discusses the design of the optimal preview controller for a linear continuous-time stochastic control system in finite-time horizon, using the method of augmented error system. First, an assistant system is introduced for state shifting. Then, in order to overcome the difficulty of the state equation of the stochastic control system being unable to be differentiated because of Brownian motion, the integrator is introduced. Thus, the augmented error system which contains the integrator vector, control input, reference signal, error vector and state of the system is reconstructed. This leads to the tracking problem of the optimal preview control of the linear stochastic control system being transformed into the optimal output tracking problem of the augmented error system. With the method of dynamic programming in the theory of stochastic control, the optimal controller with previewable signals of the augmented error system being equal to the controller of the original system is obtained. Finally, numerical simulations show the effectiveness of the controller.     

A phase-space formulation and Gaussian approximation of the filtering equations for nonlinear quantum stochastic systems

This paper is concerned with a filtering problem for a class of nonlinear quantum stochastic systems with multichannel nondemolition measurements. The system-observation dynamics are governed by a Markovian Hudson-Parthasarathy quantum stochastic differential equation driven by quantum Wiener processes of bosonic fields in vacuum state. The Hamiltonian and system-field coupling operators, as functions of the system variables, are assumed to be represented in a Weyl quantization form. Using the Wigner-Moyal phase-space framework, we obtain a stochastic integro-differential equation for the posterior quasi-characteristic function (QCF) of the system conditioned on the measurements. This equation is a spatial Fourier domain representation of the Belavkin-Kushner-Stratonovich stochastic master equation driven by the innovation process associated with the measurements. We discuss a specific form of the posterior QCF dynamics in the case of linear system-field coupling and outline a Gaussian approximation of the posterior quantum state.     

含参数不确定性的马尔可夫跳变过程鲁棒正实控制

讨论一类具有随机跳变参数的线性系统正实控制问题,其跳变参数的跃迁由有限状 态的马尔可夫过程描述.基于随机李亚普诺夫函数的方法,并结合线性矩阵不等式,分别提出依 赖于模态的状态反馈和输出反馈控制,以保证相应闭环系统的严格正实性.进一步针对系统含 参数不确定性的情形,引入鲁棒正实性分析,得到鲁棒正实控制器存在的充分条件和设计方法.     

GPU based generation of state transition models using simulations for unmanned surface vehicle trajectory planning

This paper describes GPU based algorithms to compute state transition models for unmanned surface vehicles (USVs) using 6 degree of freedom (DOF) dynamics simulations of vehicle–wave interaction. A state transition model is a key component of the Markov Decision Process (MDP), which is a natural framework to formulate the problem of trajectory planning under motion uncertainty. The USV trajectory planning problem is characterized by the presence of large and somewhat stochastic forces due to ocean waves, which can cause significant deviations in their motion. Feedback controllers are often employed to reject disturbances and get back on the desired trajectory. However, the motion uncertainty can be significant and must be considered in the trajectory planning to avoid collisions with the surrounding obstacles. In case of USV missions, state transition probabilities need to be generated on-board, to compute trajectory plans that can handle dynamically changing USV parameters and environment (e.g., changing boat inertia tensor due to fuel consumption, variations in damping due to changes in water density, variations in sea-state, etc.). The 6 DOF dynamics simulations reported in this paper are based on potential flow theory. We also present a model simplification algorithm based on temporal coherence and its GPU implementation to accelerate simulation computation performance. Using the techniques discussed in this paper we were able to compute state transition probabilities in less than 10 min. Computed transition probabilities are subsequently used in a stochastic dynamic programming based approach to solve the MDP to obtain trajectory plan. Using this approach, we are able to generate dynamically feasible trajectories for USVs that exhibit safe behaviors in high sea-states in the vicinity of static obstacles.     

Perturbation of multivariable linear quadratic systems with jumpparameters

Considers the problem of the perturbation of a class of linear quadratic systems where the change from one structure (for the dynamics and costs) to another is governed by a finite-state Markov process. The problem above leads to the analysis of some perturbed linearly coupled sets of Riccati equations. We show that the matrix obtained as the solution of the equations, which determines the optimal value and control, has a Taylor expansion in the perturbation parameter. We compute explicitly the terms of this expansion     

On stabilization of bilinear uncertain time-delay stochasticsystems with Markovian jumping parameters

In this paper, we investigate the stochastic stabilization problem for a class of bilinear continuous time-delay uncertain systems with Markovian jumping parameters. Specifically, the stochastic bilinear jump system under study involves unknown state time-delay, parameter uncertainties, and unknown nonlinear deterministic disturbances. The jumping parameters considered here form a continuous-time discrete-state homogeneous Markov process. The whole system may be regarded as a stochastic bilinear hybrid system that includes both time-evolving and event-driven mechanisms. Our attention is focused on the design of a robust state-feedback controller such that, for all admissible uncertainties as well as nonlinear disturbances, the closed-loop system is stochastically exponentially stable in the mean square, independent of the time delay. Sufficient conditions are established to guarantee the existence of desired robust controllers, which are given in terms of the solutions to a set of either linear matrix inequalities (LMIs), or coupled quadratic matrix inequalities. The developed theory is illustrated by numerical simulation     

Identification of Systems With Regime Switching and Unmodeled Dynamics

This paper is concerned with persistent identification of systems that involve deterministic unmodeled dynamics and stochastic observation disturbances, and whose unknown parameters switch values (possibly large jumps) that can be represented by a Markov chain. Two classes of problems are considered. In the first class, the switching parameters are stochastic processes modeled by irreducible and aperiodic Markov chains with transition rates sufficiently faster than adaptation rates of the identification algorithms. In this case, tracking real-time parameters by output observations becomes impossible and we show that an averaged behavior of the parameter process can be derived from the stationary measure of the Markov chain and can be estimated with periodic inputs and least-squares type algorithms. Upper and lower error bounds are established that explicitly show impact of unmodeled dynamics. In contrast, the second class of problems represents systems whose state transitions occur infrequently. An adaptive algorithm with variable step sizes is introduced for tracking the time-varying parameters. Convergence and error bounds are derived. Numerical results are presented to illustrate the performance of the algorithm.     

Feedback stabilization of discrete-time quantum systems subject to non-demolition measurements with imperfections and delays

We consider a controlled quantum system whose finite dimensional state is governed by a discrete-time nonlinear Markov process. In open-loop, the measurements are assumed to be quantum non-demolition (QND). The eigenstates of the measured observable are thus the open-loop stationary states: they are used to construct a closed-loop supermartingale playing the role of a strict control Lyapunov function. The parameters of this supermartingale are calculated by inverting a Metzler matrix that characterizes the impact of the control input on the Kraus operators defining the Markov process. The resulting state feedback scheme, taking into account a known constant delay, provides the almost sure convergence to the target state. This convergence is ensured even in the case where the filter equation results from imperfect measurements corrupted by random errors with conditional probabilities given as a left stochastic matrix. Closed-loop simulations corroborated by experimental data illustrate the interest of such nonlinear feedback scheme for the photon box, a cavity quantum electrodynamics system.     

Distributed control of uncertain systems using superpositions of linear operators

Control in the natural environment is difficult in part because of uncertainty in the effect of actions. Uncertainty can be due to added motor or sensory noise, unmodeled dynamics, or quantization of sensory feedback. Biological systems are faced with further difficulties, since control must be performed by networks of cooperating neurons and neural subsystems. Here, we propose a new mathematical framework for modeling and simulation of distributed control systems operating in an uncertain environment. Stochastic differential operators can be derived from the stochastic differential equation describing a system, and they map the current state density into the differential of the state density. Unlike discrete-time Markov update operators, stochastic differential operators combine linearly for a large class of linear and nonlinear systems, and therefore the combined effects of multiple controllable and uncontrollable subsystems can be predicted. Design using these operators yields systems whose statistical behavior can be specified throughout state-space. The relationship to Bayesian estimation and discrete-time Markov processes is described.     

A method of parameter identification for linear distributed parameter systems

A method is presented for estimating unknown parameters in distributed parameter systems. The system considered is assumed to be modeled by a stochastic partial differential equation whose form is known to be linear and unknown parameters are contained in exciting terms. Unknown parameters are assumed to be a set of random constants whose a priori probabilities are known. First, the estimation process of unknown parameters is given by the Bayesian approach in the Markovian framework. The dynamics of the state estimation is also given, which is simultaneously required in the parameter identification scheme. Secondly, the computing procedure is presented, circumventing tedious calculations of the covariance function between the system state and unknown parameters. Finally, two numerical examples are shown, emphasizing that the dynamics of the observation mechanisms adopted plays an important role in both the state estimation and parameter identification.     

Robust Stability of a Class of Uncertain Markov Jump Nonlinear Systems

This note addresses the problem of robust stability analysis for a class of Markov jump nonlinear systems subject to polytopic-type parameter uncertainty. A condition for robust local exponential mean square stability in terms of linear matrix inequalities is developed. An estimate of a robust domain of attraction of the origin is also provided. The approach is based on a stochastic Lyapunov function with polynomial dependence on the system state and uncertain parameters. A numerical example illustrates the proposed result     

Lumpable hidden Markov models-model reduction and reducedcomplexity filtering

This paper is concerned with filtering of hidden Markov processes (HMP) which possess (or approximately possess) the property of lumpability. This property is a generalization of the property of lumpability of a Markov chain which has been previously addressed by others. In essence, the property of lumpability means that there is a partition of the (atomic) states of the Markov chain into aggregated sets which act in a similar manner as far as the state dynamics and observation statistics are concerned. We prove necessary and sufficient conditions on the HMP for exact lumpability to hold. For a particular class of hidden Markov models (HMM), namely finite output alphabet models, conditions for lumpability of all HMP representable by a specified HMM are given. The corresponding optimal filter algorithms for the aggregated states are then derived. The paper also describes an approach to efficient suboptimal filtering for HMP which are approximately lumpable. By this we mean that the HMM generating the process may be approximated by a lumpable HMM. This approach involves directly finding a lumped HMM which approximates the original HMM well, in a matrix norm sense. An alternative approach for model reduction based on approximating a given HMM by an exactly lumpable HMM is also derived. This method is based on the alternating convex projections algorithm. Some simulation examples are presented which illustrate the performance of the suboptimal filtering algorithms     

Partially‐observable stochastic hybrid systems (poshss) state estimation and optimal control

This paper discusses the state estimation and optimal control problem of a class of partially‐observable stochastic hybrid systems (POSHS). The POSHS has interacting continuous and discrete dynamics with uncertainties. The continuous dynamics are given by a Markov‐jump linear system and the discrete dynamics are defined by a Markov chain whose transition probabilities are dependent on the continuous state via guard conditions. The only information available to the controller are noisy measurements of the continuous state. To solve the optimal control problem, a separable control scheme is applied: the controller estimates the continuous and discrete states of the POSHS using noisy measurements and computes the optimal control input from the state estimates. Since computing both optimal state estimates and optimal control inputs are intractable, this paper proposes computationally efficient algorithms to solve this problem numerically. The proposed hybrid estimation algorithm is able to handle state‐dependent Markov transitions and compute Gaussian‐ mixture distributions as the state estimates. With the computed state estimates, a reinforcement learning algorithm defined on a function space is proposed. This approach is based on Monte Carlo sampling and integration on a function space containing all the probability distributions of the hybrid state estimates. Finally, the proposed algorithm is tested via numerical simulations.     

Combined identification of the input-output and noise dynamics of a closed-loop controlled linear system

The open-loop input—output dynamics and the noise dynamics of a feedback controlled linear system perturbed by coloured noise admitting a Markov representation are identified in state variable form using a two-stage algorithm. This system is equivalent to an augmented system driven by white noise.

First the input—output dynamics are identified through a stochastic approximation algorithm using superimposed white noise. Subtracting the model output from the system output yields correlated residuals which are then used to identify the noise dynamics using stochastic realization theory. An innovations representation is obtained that is equivalent to the above defined augmented system. The two stages are combined by a judicious coordinate transformation. The method can be applied on an operating feedback controlled process, regardless of the structure of the unknown suboptimal regulator.     

Robust H-infinity fuzzy control for uncertainMarkovian jump nonlinear singular systems withwiener process

This paper deals with the problems of robust stochastic stabilization and H-infinity control for Markovian jump nonlinear singular systems with Wiener process via a fuzzy-control approach. The Takagi-Sugeno (T-S) fuzzy model is employed to represent a nonlinear singular system. The purpose of the robust stochastic stabilization problem is to design a state feedback fuzzy controller such that the closed-loop fuzzy system is robustly stochastically stable for all admissible uncertainties. In the robust H-infinity control problem, in addition to the stochastic stability requirement, a prescribed performance is required to be achieved. Linear matrix inequality (LMI) sufficient conditions are developed to solve these problems, respectively. The expressions of desired state feedback fuzzy controllers are given. Finally, a numerical simulation is given to illustrate the effectiveness of the proposed method.     

Magnetic sensor-based simultaneous state and parameter estimation using a nonlinear observer

This paper presents an observer design technique for a newly developed non-intrusive position estimation system based on magnetic sensors. Typically, the magnetic field of an object as a function of position needs to be represented by a highly nonlinear measurement equation. Previous results on observer design for nonlinear systems have mostly assumed that the measurement equation is linear, even if the process dynamics are nonlinear. Hence, a new nonlinear observer design method for a Wiener system composed of a linear process model together with a nonlinear measurement equation is developed in this paper. First, the design of a two degree-of-freedom nonlinear observer is proposed that relies on a Lure system representation of the observer error dynamics. To improve the performance in the presence of parametric uncertainty in the measurement model, the nonlinear observer is augmented to estimate both the state and unknown parameters simultaneously. A rigorous nonlinear observability analysis is also presented to show that a dual sensor configuration is a sufficient and necessary condition for simultaneous state and parameter estimation. Finally, the developed observer design technique is applied to non-intrusive position estimation of the piston inside a pneumatic cylinder. Experimental results show that both position and unknown parameters can be reliably estimated in this application.     

A singular system approach to robust sliding mode control for uncertain Markov jump systems

The robust sliding mode control for Markov jump systems with parameter uncertainties and an unknown nonlinear function is discussed. Based on a singular system approach and linear matrix inequality (LMI), a sufficient condition which guarantees the existence of linear switching surface and the stochastic stability of sliding mode dynamics is given. A sliding mode controller is designed such that the closed-loop system is convergent to the switching surface in finite time. A numerical example is given to show the effectiveness of the proposed method.     

Robust fusion time‐varying Kalman estimators for multisensor networked systems with mixed uncertainties

This paper addresses the problem of designing robust fusion time‐varying Kalman estimators for a class of multisensor networked systems with mixed uncertainties including multiplicative noises, missing measurements, packet dropouts, and uncertain‐variance linearly correlated measurement and process white noises. By the augmented approach, the original system is converted into a stochastic parameter system with uncertain noise variances. Furthermore, applying the fictitious noise approach, the original system is converted into one with constant parameters and uncertain noise variances. According to the minimax robust estimation principle, based on the worst‐case system with the conservative upper bounds of the noise variances, the five robust fusion time‐varying Kalman estimators (predictor, filter, and smoother) are presented by using a unified design approach that the robust filter and smoother are designed based on the robust Kalman predictor, which include three robust weighted state fusion estimators with matrix weights, diagonal matrix weights, and scalar weights, a modified robust covariance intersection fusion estimator, and robust centralized fusion estimator. Their robustness is proved by using a combination method, which consists of Lyapunov equation approach, augmented noise approach, and decomposition approach of nonnegative definite matrix, such that their actual estimation error variances are guaranteed to have the corresponding minimal upper bounds for all admissible uncertainties. The accuracy relations among the robust local and fused time‐varying Kalman estimators are proved. A simulation example is shown with application to the continuous stirred tank reactor system to show the effectiveness and correctness of the proposed results.