Fluctuations of a stochastic system under an asymptotic diffusive perturbation

A stochastic singularly perturbed system is considered. Sufficient conditions of the asymptotic normality are obtained for the dynamic system in the vicinity of its equilibrium point. To this end, a solution to the singular perturbation problem is constructed for an asymptotic representation of the generator of a Markov renewal process. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 5, pp. 104–109, September–October 2008.     

Maximal solution to algebraic Riccati equations linked to infinite Markov jump linear systems

In this paper, we deal with a perturbed algebraic Riccati equation in an infinite dimensional Banach space. Besides the interest in its own right, this class of equations appears, for instance, in the optimal control problem for infinite Markov jump linear systems (from now on iMJLS). Here, infinite or finite has to do with the state space of the Markov chain being infinite countable or finite (see Fragoso and Baczynski in SIAM J Control Optim 40(1):270–297, 2001). By using a certain concept of stochastic stability (a sort of L ₂-stability), we prove the existence (and uniqueness) of maximal solution for this class of equation and provide a tool to compute this solution recursively, based on an initial stabilizing controller. When we recast the problem in the finite setting (finite state space of the Markov chain), we recover the result of de Souza and Fragoso (Syst Control Lett 14:233–239, 1999) set to the Markovian jump scenario, now free from an inconvenient technical hypothesis used there, originally introduced in Wonham in (SIAM J Control 6(4):681–697). Research supported by grants CNPq 520367-97-9, 300662/2003-3 and 474653/2003-0, FAPERJ 171384/2002, PRONEX and IM-AGIMB.     

Superposition of Markov Renewal Processes in Stationary Phase Integration

Heuristic principles of the phase integration of a superposition of Markov renewal process are substantiated. The limiting processes of phase integration in a series scheme for stationary integrated processes are proved to be the same as the limiting processes of phase integration for a superposition of Markov renewal processes. A stationary phase integration of a superposition of Markov renewal processes is constructed. The probability of failure of a two-lift system in a series scheme is calculated for the time to failure tending to zero. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 5, pp. 119–135, September–October 2005.     

Characteristic of exit moments and models of enlargement of states for finite markov chains in terms of global memory functionals

A notion of local- and global-memory functionate for discrete-type distributions is introduced by analogy with the notion of the memory for continuous-type distributions introduced in Muth’s papers. In the class of PH-distributions (i.e., of distributions of the time of Markov chain exit from a subset of states) the necessary and sufficient conditions are obtained for the case where the exit time has an exponential (continuous-time) or a geometric (discrete time) distribution. A new notion of a global memory functional for decomposition of the state space of a finite Markov chain is introduced. Its properties as a measure of quality of decomposition and enlargement of a state space are studied. The asymptotic optimality is proved. Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 110–120, May–June, 2000.     

Optimal pattern recognition procedures and their application

Results on Bayesian classification procedures, optimal on structures such as Markov chain and independent features, are reviewed. Numerical results of predicting protein secondary structure based on Bayesian classification procedures on non-stationary Markov chains are discussed. Complementarity relations for encoding bases in one DNA strand are presented. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 6, pp. 41–54, November–December 2007.     

Convergence of a jump procedure in a semi-Markov environment in diffusion-approximation scheme

The sufficient convergence conditions are obtained for a jump stochastic approximation procedure in a semi-Markov environment in a diffusion approximation scheme with balance conditions for a singular perturbation of the regression function. To this end, a singular perturbation problem is solved for the asymptotic representation of the compensating operator of an augmented Markov renewal process. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 6, pp. 124–133, November–December 2007.     

Detecting duplicate biological entities using Markov random field-based edit distance

Detecting duplicate entities in biological data is an important research task. In this paper, we propose a novel and context-sensitive Markov random field-based edit distance (MRFED) for this task. We apply the Markov random field theory to the Needleman–Wunsch distance and combine MRFED with TFIDF, a token-based distance algorithm, resulting in SoftMRFED. We compare SoftMRFED with other distance algorithms such as Levenshtein, SoftTFIDF, and Monge–Elkan for two matching tasks: biological entity matching and synonym matching. The experimental results show that SoftMRFED significantly outperforms the other edit distance algorithms on several test data collections. In addition, the performance of SoftMRFED is superior to token-based distance algorithms in two matching tasks.     

Applying foster’s criteria to a GI / PH /1 queuing system

Foster’s criteria are used to obtain necessary and sufficient conditions ensuring the recurrence and ergodicity of an embedded Markov chain for a GI / PH / 1 queuing system. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 148–156, May–June 2006.     

The Performance of German Firms in the Business-Related Service Sectors Revisited: Differential Evolution Markov Chain Estimation of the Multinomial Probit Model

We outline a new estimation method for the multinomial probit model (MNP). The method is a differential evolution Markov chain algorithm that employs a Metropolis-within-Gibbs sampler with data augmentation and the Geweke–Hajivassiliou–Keane (GHK) probability simulator. The method lifts the curse of dimensionality in numerical integration as it neither requires simulation of the whole likelihood function nor the computation of its analytical or numerical derivatives. The method is applied to an unbalanced panel dataset of firms from the German business-related service sector over the period 1994–2000. In spite of its less restricted character, the MNP model is found not to provide more accurate estimates for explaining the performance of these firms than the multinomial logit model.     

Stability of diffusion stochastic functional differential equations with Markov parameters

The paper justifies the second Lyapunov method for diffusion stochastic functional differential equations with Markov parameters, which generalize stochastic diffusion equations without aftereffect. Analogs of Lyapunov stability theorems, which generalize the results for systems with finite aftereffect, are proved. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 1, pp. 74–88, January–February 2008.     

Continuous stochastic approximation with semi-Markov switchings in the diffusion approximation scheme

The sufficient conditions of convergence are obtained for a continuous stochastic approximation procedure in the diffusion approximation scheme under balance conditions imposed on singular perturbations of the regression function in a semi-Markov environment. To this end, a singular perturbation problem is solved for the asymptotic representation of a compensating operator of a Markov renewal process. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 170–178, July–August 2007.     

Reduced Stochastic Models for Complex Molecular Systems

We present a new numerical method for the identification of the most important metastable states of a system with complicated dynamical behavior from time series information. The approach is based on the representation of the effective dynamics of the full system by a Markov jump process between metastable states, and the dynamics within each of these metastable states by rather simple stochastic differential equations (SDEs). Its algorithmic realization exploits the concept of hidden Markov models (HMMs) with output behavior given by SDEs. A first complete algorithm including an explicit Euler–Maruyama-based likelihood estimator has already been presented in Horenko et al. (MMS, 2006a). Herein, we present a semi-implicit exponential estimator that, in contrast to the Euler–Maruyama-based estimator, also allows for reliable parameter optimization for time series where the time steps between single observations are large. The performance of the resulting method is demonstrated for some generic examples, in detail compared to the Euler–Maruyama-based estimator, and finally applied to time series originating from a 100 ns B-DNA molecular dynamics simulation.Dedicated to Peter Deuflhard on the occassion of his sixtieth birthday.Supported by the SfB 450 and DFG research center “Mathematics for key technologies” (FZT 86) in Berlin.     

Queuing system evolution in a diffusion approximation scheme

An asymptotic diffusion approximation scheme is investigated as applied to the requirement evolution in semi-Markov queuing systems. In proving the diffusion approximation theorem, the compensating operator of the corresponding extended Markov process is used. This problem is solved with the help of a phase merging procedure. Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 136–145, May–June 2009. Original article submitted August 19, 2008.     

Modeling appraisal in theory of mind reasoning

Cognitive appraisal theories, which link human emotional experience to their interpretations of events happening in the environment, are leading approaches to model emotions. Cognitive appraisal theories have often been used both for simulating “real emotions” in virtual characters and for predicting the human user’s emotional experience to facilitate human–computer interaction. In this work, we investigate the computational modeling of appraisal in a multi-agent decision-theoretic framework using Partially Observable Markov Decision Process-based (POMDP) agents. Domain-independent approaches are developed for five key appraisal dimensions (motivational relevance, motivation congruence, accountability, control and novelty). We also discuss how the modeling of theory of mind (recursive beliefs about self and others) is realized in the agents and is critical for simulating social emotions. Our model of appraisal is applied to three different scenarios to illustrate its usages. This work not only provides a solution for computationally modeling emotion in POMDP-based agents, but also illustrates the tight relationship between emotion and cognition—the appraisal dimensions are derived from the processes and information required for the agent’s decision-making and belief maintenance processes, which suggests a uniform cognitive structure for emotion and cognition.     

A framework for exploring numerical solutions of advection–reaction–diffusion equations using a GPU-based approach

In this paper we describe a general purpose, graphics processing unit (GP-GPU)-based approach for solving partial differential equations (PDEs) within advection–reaction–diffusion models. The GP-GPU-based approach provides a platform for solving PDEs in parallel and can thus significantly reduce solution times over traditional CPU implementations. This allows for a more efficient exploration of various advection–reaction–diffusion models, as well as, the parameters that govern them. Although the GPU does impose limitations on the size and accuracy of computations, the PDEs describing the advection–reaction–diffusion models of interest to us fit comfortably within these constraints. Furthermore, the GPU technology continues to rapidly increase in speed, memory, and precision, thus applying these techniques to larger systems should be possible in the future. We chose to solve the PDEs using two numerical approaches: for the diffusion, a first-order explicit forward Euler solution and a semi-implicit second order Crank–Nicholson solution; and, for the advection and reaction, a first-order explicit solution. The goal of this work is to provide motivation and guidance to the application scientist interested in exploring the use of the GP-GPU computational framework in the course of their research. In this paper, we present a rigorous comparison of our GPU-based advection–reaction–diffusion code model with a CPU-based analog, finding that the GPU model out-performs the CPU implementation in one-to-one comparisons.     

Fokker–Planck approximation of the master equation in molecular biology

The master equation of chemical reactions is solved by first approximating it by the Fokker–Planck equation. Then this equation is discretized in the state space and time by a finite volume method. The difference between the solution of the master equation and the discretized Fokker–Planck equation is analyzed. The solution of the Fokker–Planck equation is compared to the solution of the master equation obtained with Gillespie’s Stochastic Simulation Algorithm (SSA) for problems of interest in the regulation of cell processes. The time dependent and steady state solutions are computed and for equal accuracy in the solutions, the Fokker–Planck approach is more efficient than SSA for low dimensional problems and high accuracy.     

Choosing the parameter of a modified additive-averaged splitting algorithm

The problem-solving time and the solution accuracy are expressed as functions of a parameter. An optimization problem of choosing the parameter using a “time cost–solution accuracy” criterion is considered. A Pareto-optimal set of solutions is obtained. The best value of the parameter is chosen by the ideal-point method. Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 98–105, July–August 2009.     

A tool for model-checking Markov chains

Markov chains are widely used in the context of the performance and reliability modeling of various systems. Model checking of such chains with respect to a given (branching) temporal logic formula has been proposed for both discrete [34, 10] and continuous time settings [7, 12]. In this paper, we describe a prototype model checker for discrete and continuous-time Markov chains, the Erlangen–Twente Markov Chain Checker E⊢MC², where properties are expressed in appropriate extensions of CTL. We illustrate the general benefits of this approach and discuss the structure of the tool. Furthermore, we report on successful applications of the tool to some examples, highlighting lessons learned during the development and application of E⊢MC². Published online: 19 November 2002 Correspondence to: Holger Hermanns     

Multichannel image segmentation using adaptive finite elements

We present an adaptive finite element algorithm for segmentation with denoising of multichannel images in two dimensions, of which an extension to three dimensional images is straight forward. It is based on a level set formulation of the Mumford–Shah approach proposed by Chan and Vese in (JVCIR 11:130–141,(2000); IEEE Trans Image Proces 10(2):266–277, (2001); Int J Comp Vis 50(3):271–293, (2002)) In case of a minimal partition problem an exact solution is given and convergence of the discrete solution towards this solution is numerically verified.     

New prioritized value iteration for Markov decision processes

The problem of solving large Markov decision processes accurately and quickly is challenging. Since the computational effort incurred is considerable, current research focuses on finding superior acceleration techniques. For instance, the convergence properties of current solution methods depend, to a great extent, on the order of backup operations. On one hand, algorithms such as topological sorting are able to find good orderings but their overhead is usually high. On the other hand, shortest path methods, such as Dijkstra’s algorithm which is based on priority queues, have been applied successfully to the solution of deterministic shortest-path Markov decision processes. Here, we propose an improved value iteration algorithm based on Dijkstra’s algorithm for solving shortest path Markov decision processes. The experimental results on a stochastic shortest-path problem show the feasibility of our approach.