Performance sensitivities for parameterized Markov systems

It is known that the performance potentials (or equivalently, perturbation realization factors) can be used as building blocks for performance sensitivities of Markov systems. In parameterized systerns, the changes in parameters may only affect some states, and the explicit transition probability matrix may not be known. In this paper, we use an example to show that we can use potentials to construct performance sensitivities m a more flexible way; only the potentials at the affected states need to be estimated, and the transition probability matrix need not be known. Policy iteration algorithms, which are simpler than the standard one, can be established.     

随机马尔可夫跳变系统的变结构控制设计

采用滑动扇区方法,研究了不确定随机马尔可夫跳变系统的变结构控制设计问题。首先给出随机马尔可夫跳变系统滑动扇区的定义,然后基于线性矩阵不等式技术,提出一种滑动扇区及变结构控制律设计方法。经过理论证明该控制律能够确保随机马尔可夫跳变不确定系统二次稳定,并有效地抑制抖振。最后数值仿真算例验证了控制方案的有效性。     

一类线性离散时间结构随机跳变系统的逼近滤波算法

提出了具有条件马尔可夫跳变结构的离散时间随机系统的条件滤波方法, 应用随机变结构系统的性质对滤波算法进行简化处理, 并将后验概率密度函数用条件高斯函数来逼近, 得到具有条件马尔可夫结构离散随机系统的逼近最优滤波算法, 最后给出滤波算法的计算步骤并仿真验证了算法的正确性.     

The IBMAP approach for Markov network structure learning

In this work we consider the problem of learning the structure of Markov networks from data. We present an approach for tackling this problem called IBMAP, together with an efficient instantiation of the approach: the IBMAP-HC algorithm, designed for avoiding important limitations of existing independence-based algorithms. These algorithms proceed by performing statistical independence tests on data, trusting completely the outcome of each test. In practice tests may be incorrect, resulting in potential cascading errors and the consequent reduction in the quality of the structures learned. IBMAP contemplates this uncertainty in the outcome of the tests through a probabilistic maximum-a-posteriori approach. The approach is instantiated in the IBMAP-HC algorithm, a structure selection strategy that performs a polynomial heuristic local search in the space of possible structures. We present an extensive empirical evaluation on synthetic and real data, showing that our algorithm outperforms significantly the current independence-based algorithms, in terms of data efficiency and quality of learned structures, with equivalent computational complexities. We also show the performance of IBMAP-HC in a real-world application of knowledge discovery: EDAs, which are evolutionary algorithms that use structure learning on each generation for modeling the distribution of populations. The experiments show that when IBMAP-HC is used to learn the structure, EDAs improve the convergence to the optimum.     

An LMI-based variable structure control for a class of uncertain singular Markov switched systems

An H-infinifty variable structure control is presented for singular Markov switched systems with mismatched norm-bounded uncertainties and mismatched norm-bounded external disturbances. It is shown that the sliding mode dynamics on the given switching surface is regular, impulse-free, and stochastically stable and satisfies H-infinity performance. A variable structure controller is designed to guarantee that the system trajectory converges to the linear switching surface in some finite time. Finally, a numerical example is solved to show the effectiveness and validness of the theoretical results.     

An LMI-based variable structure control for a class of uncertain singular Markov switched systems

An H-infinifty variable structure control is presented for singular Markov switched systems with mismatched norm-bounded uncertainties and mismatched norm-bounded external disturbances. It is shown that the sliding mode dynamics on the given switching surface is regular, impulse-free, and stochastically stable and satisfies H-infinity performance. A variable structure controller is designed to guarantee that the system trajectory converges to the linear switching surface in some finite time. Finally, a numerical example is solved to show the effectiveness and validness of the theoretical results.     

The ripple minimization problems in sample data systems

The problem of minimizing the state value between the sampling times of a sampled data regulator is considered in this paper and is termed the ripple minimization problem. It is found that this problem is equivalent to the discrete problem of minimizing the state at the sampling times with a cost functional that includes in addition to the quadratic terms others which are bilinear in x and u. The results of the solution of the discrete optimal problem are used to solve the ripple minimization problem for both time-variant and time-invariant sampled data system. It is found that an additional condition on the rank of the controller matrices is necessary which is to some extent analogous to the condition necessary for preservation of controllability of sampled data system (Kalman et at., Bar-Ness 1975).     

Mean characteristics of Markov queueing systems

Consideration is given to queueing systems described by nonstationary birth-death processes with rates close to periodic. Questions connected with the existence and design of limiting mean characteristics are studied. Some examples of designing the means for concrete queueing systems are considered.     

Differentiable structure of realizable Markov parameters

The geometry of the finite sequences of p×m$p\times m$ matrices that are realizable by systems of minimal order d$d$ is investigated. They can be stratified according to their partial row and column Kronecker indices. The case when the sum of the number of non-zero row and column indices is smaller than the number of elements in the sequence of parameters is considered. It is shown that, in this case, the whole set of Markov parameters and each stratum can be endowed with structures of differentiable manifolds, and their dimensions are computed.     

On the optimal control of a stochastic system with discontinuous sample paths

This paper deals with the optimal control of a stochastic n-order system. It is assumed that the system is subjected to two different kinds of perturbations. The first kind of perturbation is represented by a vector of independent standard Wiener processes and the second kind by a vector of a generalized type of Poisson process. By applying the calculus of variations necessary conditions on the optimal controls are derived. These conditions are given by a pair of coupled non-linear partial integro-differential equations, A stochastic second-order system is given, as a test case, and a numerical method for the computation of its optimal controls is suggested. The efficiency and applicability of this method ore demonstrated with examples.     

The sample complexity of worst-case identification of FIR linear systems

We consider the problem of identification of linear systems in the presence of measurement noise which is unknown but bounded in magnitude by some δ > 0. We focus on the case of linear systems with a finite impulse response. It is known that the optimal identification error is related (within a factor of 2) to the diameter of a so-called uncertainty set and that the latter diameter is upper-bounded by 2δ, if a sufficiently long identification experiment is performed. We establish that, for any K 1, the minimal length of an identification experiment that is guaranteed to lead to a diameter bounded by 2Kδ behaves like 2^Nf(1/K), when N is large, where N is the length of the impulse response and is a positive function known in closed form. While the framework is entirely deterministic, our results are proved using probabilistic tools.     

Decomposition of systems governed by Markov chains

This paper applies the Dantzig-Wolfe decomposition technique to control systems governed by Markov chains, and the three usual types of costs: 1) the average cost attained until a target state is reached, 2) discounted cost, 3) average cost per unit time. Additional systems constraints are allowed. A technique for subdividing or "essentially" decomposing the problem is developed, and a Markov interpretation is given to each subsystem. The special significance, for this problem, of the extreme points and rays of the subproblem, is discussed.     

On the comparison of the sensitivities of open-loop and closed-loop optimal control systems

This paper derives, via Pontryagin's minimum principle, the invariance of the sensitivity of cost for optimal systems with respect to open-loop and closed-loop control law implementation when system parameters undergo small variations. The result obtained here is more general than that of Pagurek[1] and somewhat easier to apply than that of Witsenhausen.[2] The cost of the sensitivity vector is simply related to the gradient of the Hamiltonian function.     

Adjoint network method applied to the performance sensitivities of microwave amplifiers

This work focuses on the performance sensitivities of microwave amplifiers using the “adjoint network and adjoint variable” method, via “wave” approaches, which includes sensitivities of the transducer power gain, noise figure, and magnitudes and phases of the input and output reflection coefficients. The method can be extended to sensitivities of the other performance measure functions. The adjoint‐variable methods for design‐sensitivity analysis offer computational speed and accuracy. They can be used for efficiency‐based gradient optimization, in tolerance and yield analyses. In this work, an arbitrarily configured microwave amplifier is considered: firstly, each element in the network is modeled by the scattering matrix formulation, then the topology of the network is taken into account using the connection scattering‐matrix formulation. The wave approach is utilized in the evaluation of all the performance‐measurement functions, then sensitivity invariants are formulated using Tellegen's theorem. Performance sensitivities of the T‐ and Π‐types of distributed‐parameter amplifiers are considered as a worked example. The numerical results of T‐ and Π‐type amplifiers for the design targets of noise figure F_req = 0.46 dB ? 1,12 and V_ireq = 1, G_Treq = 12 dB ? 15.86 in the frequency range 2–11 GHz are given in comparison to each other. Furthermore, analytical methods of the “gain factorisation” and “chain sensitivity parameter” are applied to the gain and noise sensitivities as well. In addition, “numerical perturbation” is applied to calculation of all the sensitivities. © 2006 Wiley Periodicals, Inc. Int J RF and Microwave CAE, 2006.     

Stability of Markov modulated discrete-time dynamic systems

Motivated by various control applications, this paper develops stability analysis of discrete-time systems with regime switching, in which the dynamics are modulated by Markov chains with two time scales. Using the high contrast of the different transition rates among the Markovian states, singularly perturbed Markov chains are used in the formulations. Taking into consideration of the regime changes and time-scale separation makes the stability analysis a difficult task. In this work, asymptotic stability analysis is carried out using perturbed Liapunov function techniques. It is demonstrated that if the limit system is stable, then the original system is also stable. In addition, we examine path excursion, derive bounds on mean recurrence time, and obtain the associated probability bounds.     

Adaptive control of linear Markov jump systems

Stabilization of linear Markov jump systems via adaptive control is considered in this paper. The switching law is assumed to be unobservable Markov process. A sufficient condition is obtained for the stochastic stabilizability based on common quadratic Lyapunov functions (QLFs). The constructive proof provides a method to construct a sampling adaptive stabilizer. An example is used to describe the design of adaptive control, which stabilizes the system.