Convergence of a jump procedure in a semi-Markov environment in diffusion-approximation scheme

The sufficient convergence conditions are obtained for a jump stochastic approximation procedure in a semi-Markov environment in a diffusion approximation scheme with balance conditions for a singular perturbation of the regression function. To this end, a singular perturbation problem is solved for the asymptotic representation of the compensating operator of an augmented Markov renewal process. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 6, pp. 124–133, November–December 2007.     

Continuous procedure of stochastic approximation with singular perturbation under balance conditions

Sufficient conditions for the convergence of a continuous stochastic approximation procedure are established for the case where the regression function depends on a Markovian environment and has a singular perturbation that satisfies balance conditions. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 133–139, May–June 2006.     

Queuing system evolution in a diffusion approximation scheme

An asymptotic diffusion approximation scheme is investigated as applied to the requirement evolution in semi-Markov queuing systems. In proving the diffusion approximation theorem, the compensating operator of the corresponding extended Markov process is used. This problem is solved with the help of a phase merging procedure. Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 136–145, May–June 2009. Original article submitted August 19, 2008.     

Stability of solutions of stochastic functional-differential equations with poisson switchings and entire prehistory

The Markov properties of the solutions of Ito-Skorokhod stochastic functional-differential equations (SFDEs) with entire prehistory are considered, the concept of a weak infinitesimal operator is introduced for a Markov process that is a solution of an SFDE, and the strong solution of the SFDE is analyzed for stability. Translated from Kibernetika i Sistemnyi Analiz, No. 1, pp. 123–134, January–February 2009.     

Asymptotic behavior of asynchronous stochastic approximation

The pathwise convergence of a distributed, asynchronous stochastic approximation (SA) scheme is analyzed. The conditions imposed on the step size and noise are the weakest in comparison with the existing ones. The step sizes in different processors are allowed to be different, and the time-delays between processors are also allowed to be different and even time-varying.     

New combinatorial direction stochastic approximation algorithms

The stochastic approximation problem is to find some roots or minimizers of a nonlinear function whose expression is unknown and whose evaluations are contaminated with noise. In order to accelerate the classical RM algorithm, this paper proposes a new three-term combinatorial direction stochastic approximation algorithm and its general framework which employ a weighted combination of the current noisy gradient and several previous noisy gradients as the iterative direction. Both the almost sure convergence and the asymptotic rate of convergence of the new algorithms are established. Numerical experiments show that the new algorithm outperforms the RM algorithm and another existing combined direction algorithm.     

Convergence analysis of dynamic stochastic approximation

Stochastic approximation method concerns the sequential estimation of the root or the extreme of a function observed with noise. The idea is then extended to the moving root case by Dupa , and it is called as the dynamic stochastic approximation method. Its convergence properties are derived here by using a randomly varying truncation technique under weaker conditions in comparison with the previous work: the growth rate restriction on the function has been removed and the condition on the observation noises has been weakened to the possibly weakest ones.     

On the Hankel-norm approximation of linear stochastic systems

We derive ordering and interlocking properties for the singular values of the block-Hankel matrices corresponding to different spectral factors of a given spectral density matrix. The results are then applied to Hankel-norm approximation of SISO stochastic systems. In particular we show that the minimum phase model may be approximated in Hankel norm by systems of a certain prescribed dimension with better accuracy than any other model with the same output process. We also provide upper bounds on the gain in accuracy obtained by choosing the minimum phase model over some other model.     

Asynchronous control of discrete-time stochastic bilinear systems with Markovian switchings

This paper is concerned with the problem of asynchronous control for a class of discrete-time Markov systems with multiplicative stochastic white noises. Based on a stability analysis scheme developed from mode-dependent Lyapunov function method, we first derive testable conditions in linear matrix inequality (LMI) setting to ensure the robust stability of the closed-loop system. We then recast the proposed stability conditions into equivalent forms that are later utilised to design a multi-mode asynchronous state-feedback controller (ASFC) that makes the closed-loop system stable. An extension to the case of deficient mode information (i.e. transition rates of the system and the controller are not fully accessible) is also presented. Finally, a model of networked control with DC devices is given to demonstrate the efficacy of the proposed design scheme.     

Asymptotic stability in probability of singular stochastic systems with Markovian switchings

This paper investigates the problem of asymptotic stability in probability for singular stochastic systems with Markovian switchings. A stochastic Lyapunov theorem on asymptotic stability in probability for the considered systems is provided. Also, we show that the original system has the same stability property as its difference‐algebraic form based on singular value decomposition. By utilizing the earlier results, a sufficient condition is obtained in terms of linear matrix inequalities, which is easy to check by using standard software. Copyright © 2017 John Wiley & Sons, Ltd.     

Stability of linear stochastic delay differential equations with infinite Markovian switchings

This paper investigates the stability of linear stochastic delay differential equations with infinite Markovian switchings. Some novel exponential stability criteria are first established based on the generalized It formula and linear matrix inequalities. Then, a new sufficient condition is proposed for the equivalence of 4 stability definitions, namely, asymptotic mean square stability, stochastic stability, exponential mean square stability with conditioning, and exponential mean square stability. In particular, our results generalize and improve some of the previous results. Finally, two examples are given to illustrate the effectiveness of the proposed results.     

Optimal control of stochastic dynamic systems with past history and poison switchings

An optimal control problem for systems of stochastic differential-functional linear equations with past history and Poisson switchings is formulated. The Bellman equation is solved for this problem.Translated from Kibernetika i Sistemnyi Analiz, No. 6, pp. 112–118, November–December 2004.This revised version was published online in April 2005 with a corrected cover date.     

Random evolution in a scheme of asymptotically small diffusion with markov switchings

A limit exponential generator is constructed for random evolution in an asymptotically small diffusion scheme using a small parameter of series and the solution to the singular perturbation problem for a uniformly ergodic Markov process.     

A phase-space formulation and Gaussian approximation of the filtering equations for nonlinear quantum stochastic systems

This paper is concerned with a filtering problem for a class of nonlinear quantum stochastic systems with multichannel nondemolition measurements. The system-observation dynamics are governed by a Markovian Hudson-Parthasarathy quantum stochastic differential equation driven by quantum Wiener processes of bosonic fields in vacuum state. The Hamiltonian and system-field coupling operators, as functions of the system variables, are assumed to be represented in a Weyl quantization form. Using the Wigner-Moyal phase-space framework, we obtain a stochastic integro-differential equation for the posterior quasi-characteristic function (QCF) of the system conditioned on the measurements. This equation is a spatial Fourier domain representation of the Belavkin-Kushner-Stratonovich stochastic master equation driven by the innovation process associated with the measurements. We discuss a specific form of the posterior QCF dynamics in the case of linear system-field coupling and outline a Gaussian approximation of the posterior quantum state.     

Sampling average approximation method for a class of stochastic Nash equilibrium problems

We consider a class of stochastic Nash equilibrium problems (SNEP). Under some mild conditions, we reformulate the SNEP as a stochastic mixed complementarity problem (SMCP). We apply the well-known sample average approximation (SAA) method to solve the SMCP. We further introduce a semismooth Newton method to solve the SAA problems. The comprehensive convergence analysis is given as well. In addition, we demonstrate the proposed approach on a stochastic Nash equilibrium model in the wholesale gas–oil markets.     

A stochastic approximation algorithm for option pricing model calibration with a switchable market

This paper is concerned with option pricing under a regime-switching model. The switching process takes two different modes, and the underlying stock price evolves in accordance with the two modes dictated by a continuous-time, finite-state Markov chain. At any given instance, the price follows either a geometric Brownian motion model or a mean-reversion model, depending on its market mode. Stochastic approximation/optimization algorithms are developed for model calibration. Convergence of the algorithm is proved; rate of convergence is also provided. Option market data are used to predict the future market mode.     

Exponential stability of stochastic singular delay systems with general Markovian switchings

In recent years, Markovian jump systems have received much attention. However, there are very few results on the stability of stochastic singular systems with Markovian switching. In this paper, the discussed system is the stochastic singular delay system with general transition rate matrix in terms of uncertain and partially unknown transition rate matrix. The aim is to answer the question whether there are conditions guaranteeing the underlying system having a unique solution and being exponentially admissible simultaneously. The proposed results show that all the features of the underlying system such as time delay, diffusion, and general Markovian switchings play important roles in the system analysis of exponential admissibility. A numerical example is used to demonstrate the effectiveness of the proposed methods. Copyright © 2014 John Wiley & Sons, Ltd.     

Simulation based optimization of stochastic systems with integer design variables by sequential multipoint linear approximation

Optimization problems are considered for which objective function and constraints are defined as expected values of stochastic functions that can only be evaluated at integer design variable levels via a computationally expensive computer simulation. Design sensitivities are assumed not to be available. An optimization approach is proposed based on a sequence of linear approximate optimization subproblems. Within each search subregion a linear approximate optimization subproblem is built using response surface model building. To this end, N simulation experiments are carried out in the search subregion according to a D-optimal experimental design. The linear approximate optimization problem is solved by integer linear programming using corrected constraint bounds to account for any uncertainty due to the stochasticity. Each approximate optimum is evaluated on the basis of M simulation replications with respect to objective function change and feasibility of the design. The performance of the optimization approach and the influence of parameters N and M is illustrated via two analytical test problems. A third example shows the application to a production flow line simulation model. Received April 28, 2000     

Nonlinear stochastic receding horizon control: stability,robustness and Monte Carlo methods for control approximation

This work considers the stability of nonlinear stochastic receding horizon control when the optimal controller is only computed approximately. A number of general classes of controller approximation error are analysed including deterministic and probabilistic errors and even controller sample and hold errors. In each case, it is shown that the controller approximation errors do not accumulate (even over an infinite time frame) and the process converges exponentially fast to a small neighbourhood of the origin. In addition to this analysis, an approximation method for receding horizon optimal control is proposed based on Monte Carlo simulation. This method is derived via the Feynman–Kac formula which gives a stochastic interpretation for the solution of a Hamilton–Jacobi–Bellman equation associated with the true optimal controller. It is shown, and it is a prime motivation for this study, that this particular controller approximation method practically stabilises the underlying nonlinear process.