Continuous stochastic approximation with semi-Markov switchings in the diffusion approximation scheme

The sufficient conditions of convergence are obtained for a continuous stochastic approximation procedure in the diffusion approximation scheme under balance conditions imposed on singular perturbations of the regression function in a semi-Markov environment. To this end, a singular perturbation problem is solved for the asymptotic representation of a compensating operator of a Markov renewal process. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 170–178, July–August 2007.     

Convergence of a jump procedure in a semi-Markov environment in diffusion-approximation scheme

The sufficient convergence conditions are obtained for a jump stochastic approximation procedure in a semi-Markov environment in a diffusion approximation scheme with balance conditions for a singular perturbation of the regression function. To this end, a singular perturbation problem is solved for the asymptotic representation of the compensating operator of an augmented Markov renewal process. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 6, pp. 124–133, November–December 2007.     

Generalized H2 control of discrete-time semi-Markov jump systems

This paper mainly studies the generalized H₂ control of discrete-time semi-Markov jump systems with bounded sojourn-time. By constructing the mode-dependent Lyapunov function related to the elapsed-time and the semi-Markov process, we obtain sufficient conditions to ensure the σ-error mean square stable and generalized H₂ performance of the semi-Markov jump system. Moreover, by introducing slack variables, the generalized H₂ performance index is reduced and the feasible region is expanded. Finally, through numerical calculations, we confirm the effectiveness of the design methods.     

Evaluation of the Probability of Functional Failure of a Redundant System by Importance Sampling Method

A redundant system whose structure is determined by the fault tree with efficiencies is considered. External load on the system is the function of state of a semi-Markov process. A fast simulation method enabling to evaluate the probability of functional failure when the real system efficiency becomes less than required ones is proposed. The conditions that guarantee the boundedness of the relative error estimate as the reliability of elements increases are found. The numerical example illustrates the the accuracy of estimates obtained by the proposed method.     

不确定离散半马尔可夫跳变系统的有限时间鲁棒$H_infty$控制

研究一类不确定离散时间半马尔可夫跳变系统的有限时间鲁棒H_∞控制问题.首先,应用半马尔可夫核方法建模离散时间系统的半马尔可夫跳变过程,其中驻留时间的概率密度函数依赖系统当前模态和下一模态,使得所提出理论可考虑不同的驻留时间概率分布类型,针对随机跳变系统有限时间分析中的跳变次数问题,提出估计跳变次数最大值的方法;然后,考虑系统模态驻留时间的上下界,提出新的有限时间有界概念,并保证基于半马尔可夫核方法给出的不确定离散时间半马尔可夫随机跳变系统有限时间有界判据数值可解;接着,在此基础上,分析系统的H_∞性能指标,设计模态依赖的状态反馈控制律保证闭环系统的有限时间鲁棒H_∞性能;最后,通过2个算例仿真验证所提出理论的可行性和有效性.     

A difference stochastic optimization procedure with impulse perturbation

The sufficient conditions are obtained for the convergence of the difference stochastic optimization procedure with impulse perturbations in a Markov environment under the conditions of exponential stability of the averaged system and smooth regression function of the source system. To this end, an asymptotic representation of the perturbed procedure generator is obtained.     

Discrete Time Non-Homogeneous Semi-Markov Reliability Transition Credit Risk Models and the Default Distribution Functions

This paper shows how to apply discrete time non-homogeneous semi-Markov processes (DTNHSMP) with an age index to credit risk. The idea is to consider the credit risk problem as a reliability model indexed by the age and in this way, many semi-Markov results could be adapted to describe credit risk problem. The default state is seen as a “non working state”. As the semi-Markov process is a generalization of Markov process, the presented model can be seen as a more general migration model. In fact, in semi-Markov environment the distribution function (d.f.) of the waiting time before a transition can be of any type and without the strong constraints of the Markov model. Furthermore, some results on the asymptotic behavior of the presented model are given. This permits the construction of the d.f. of the default random variable for each “working state”. An example, constructed manipulating some Standard & Poor’s (S&P) data, is presented.     

H∞ fuzzy control of semi-Markov jump nonlinear systems under σ-error mean square stability

This paper is concerned with the problem of time-varying H_∞ fuzzy control for a class of semi-Markov jump nonlinear systems in the sense of σ-error mean square stability. The nonlinear plant is described via the Takagi–Sugeno fuzzy model. By defining a time-varying mode-dependent Lyapunov function, a set of sufficient stability and stabilisation criteria for non-disturbance case is first derived and then applied to the investigation of H_∞ performance analysis and H_∞ fuzzy controller design problems of semi-Markov jump nonlinear systems. Different from the traditional stochastic switching system framework, the probability density function of sojourn time is exploited to circumvent the complex computation of transition probabilities. The derived conditions can cover the time-invariant mode-dependent and time-invariant mode-independent H_∞ fuzzy control schemes as special cases. A classic cart-pendulum system is presented to demonstrate the effectiveness and advantages of the proposed theoretical results.     

Optimal maintenance and replacement via a semi-Markov decision model

Optimal bang-coast maintenance policies for a machine, subject to failure, are considered. The approach utilizes a semi-Markov model for the system. A simplified model for modifying the probability of machine failure with maintenance is employed. A numerical example is presented to illustrate the procedure and results.     

Robust control synthesis for discrete-time uncertain semi-Markov jump systems

This paper studies the control synthesis for uncertain semi-Markov jump systems in a discrete-time domain subjected to external disturbance. The switching between modes is determined by a function of the transition probability and the sojourn-time distribution between two neighbouring modes. Based on the σ-error mean square stability criterion, time-varying controllers are designed to stabilise the system. By constructing a holding time dependent Lyapunov function, time-varying state-feedback controllers are obtained that meet a set of sufficient conditions in the form of linear matrix inequalities. Two examples, including a DC motor system, are presented to show the validity of the proposed control scheme.     

Stochastic Stability, ℒ<Subscript>1</Subscript>-gain and Control Synthesis for Positive Semi-Markov Jump Systems

This paper treats the problems of stochastic stability, ?₁-gain and control synthesis for positive semi-Markov jump systems (S-MJSs). The system under consideration involves semi-Markov stochastic process related to Weibull distribution. The main motivation for this paper is that the positive condition sometimes needs to be considered in S-MJSs and the controller design methods in the existing works have some conservation. To deal with these problems, some sufficient conditions for stochastic stability of positive S-MJSs are established by implying the linear co-positive Lyapunov function. Then, some sufficient conditions for ?₁-gain constraint are also presented, upon which, a state feedback controller is designed by decomposing the controller gain matrix such that the resulting closed-loop system is positive and stochastically stable with ?₁-gain performance in the form of standard linear programming (LP). The advantages of the new framework lie in the following facts: (1) the weak infinitesimal operator is derived for S-MJSs under the constraint of positive condition and (2) the less conservative stabilizing controller is designed to achieve the desired control performance. Finally, numerical examples are given to demonstrate the validity of the main results.     

Continuous procedure of stochastic approximation with singular perturbation under balance conditions

Sufficient conditions for the convergence of a continuous stochastic approximation procedure are established for the case where the regression function depends on a Markovian environment and has a singular perturbation that satisfies balance conditions. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 133–139, May–June 2006.     

tdc.msm: an R library for the analysis of multi-state survival data

The aim of this paper is to present an R library, called tdc.msm, developed to analyze multi-state survival data. In this library, the time-dependent regression model and multi-state models are included as two possible approaches for such data. For the multi-state modelling five different models are considered, allowing the user to choose between Markov and semi-Markov property, as well as to use homogeneous or non-homogeneous models. Specifically, the following multi-state models in continuous time were implemented: Cox Markov model; Cox semi-Markov model; homogeneous Markov model; non-homogeneous piecewise model and non-parametric Markov model. This software can be used to fit multi-state models with one initial state (e.g., illness diagnosis), a finite number of intermediate states, representing, for example, a change of treatment, and one absorbing state corresponding to a terminal event of interest. Graphical output includes survival estimates, transition probabilities estimates and smooth log hazard for continuous covariates.     

Passage-time computation and aggregation strategies for large semi-Markov processes

High-level semi-Markov modelling paradigms such as semi-Markov stochastic Petri nets and process algebras are used to capture realistic performance models of computer and communication systems but often have the drawback of generating huge underlying semi-Markov processes. Extraction of performance measures such as steady-state probabilities and passage-time distributions therefore relies on sparse matrix-vector operations involving very large transition matrices. Previous studies have shown that exact state-by-state aggregation of semi-Markov processes can be applied to reduce the number of states. This can, however, lead to a dramatic increase in matrix density caused by the creation of additional transitions between remaining states. Our paper addresses this issue by presenting the concept of state space partitioning for aggregation.We present a new deterministic partitioning method which we term barrier partitioning. We show that barrier partitioning is capable of splitting very large semi-Markov models into a number of partitions such that first passage-time analysis can be performed more quickly and using up to 99% less memory than existing algorithms.     

Queuing system evolution in a diffusion approximation scheme

An asymptotic diffusion approximation scheme is investigated as applied to the requirement evolution in semi-Markov queuing systems. In proving the diffusion approximation theorem, the compensating operator of the corresponding extended Markov process is used. This problem is solved with the help of a phase merging procedure. Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 136–145, May–June 2009. Original article submitted August 19, 2008.     

Robust bandwidth selection in semiparametric partly linear regression models: Monte Carlo study and influential analysis

In this paper, under a semiparametric partly linear regression model with fixed design, we introduce a family of robust procedures to select the bandwidth parameter. The robust plug-in proposal is based on nonparametric robust estimates of the νth derivatives and under mild conditions, it converges to the optimal bandwidth. A robust cross-validation bandwidth is also considered and the performance of the different proposals is compared through a Monte Carlo study. We define an empirical influence measure for data-driven bandwidth selectors and, through it, we study the sensitivity of the data-driven bandwidth selectors. It appears that the robust selector compares favorably to its classical competitor, despite the need to select a pilot bandwidth when considering plug-in bandwidths. Moreover, the plug-in procedure seems to be less sensitive than the cross-validation in particular, when introducing several outliers. When combined with the three-step procedure proposed by Bianco and Boente [2004. Robust estimators in semiparametric partly linear regression models. J. Statist. Plann. Inference 122, 229-252] the robust selectors lead to robust data-driven estimates of both the regression function and the regression parameter.     

Fluctuations of queuing processes with semi-Markov (impulse) flows

The problem of diffusion approximation of a queuing process with a semi-Markov flow is solved. In proving a theorem, the solution to a singular perturbation problem is used that is obtained for a compensative operator of a semi-Markov process.     

Analysis of the distribution of a random process with differential semi-Markov walk with delaying screen at zero

Given a four-dimensional sequence of independent identically located and positive random variables, a process with differential semi-Markov walk delayed by a screen at zero is considered. An explicit form of the Laplace transform in time, the Laplace–Stieltjes transform of the ergodic distribution of a process with differential semi-Markov walk with delaying screen at zero is found.     

基于隐半马尔可夫模型的用户兴趣特征提取

针对网络用户兴趣行为特征的抽取,提出了一种基于隐半马尔可夫模型的用户兴趣特征提取模型,通过用状态驻留时间的概率来控制用户浏览行为,使描述兴趣特征的隐状态和时间的相关性更紧密地结合起来,并且根据隐半马尔可夫模型可以产生多观察值序列的特性,把文本信息划分成多个文本块子区域,使每个子区域的特征和其中一个观察值序列对应起来。实验结果表明,利用隐半马尔可夫模型进行特征提取比HMM方法有更高的准确率和召回率。