Fastest random walk on a path

In this paper, we consider two convex optimisation problems in order to maximise the mixing rate of a Markov chain on an undirected path. In the first formulation, the holding probabilities of vertices are identical and the transition probabilities from a vertex to its neighbours are equal, whereas the second formulation is the more general reversible Markov chain with the same degree proportional stationary distribution. We derive analytical results on the solutions of the optimisation problems and compare the spectra of the associated transition probability matrices.     

Estimation of Finite Probabilities via Aggregation of Markov Chains

An aggregated (trivial) chain with fewer number of states than for the initial Markov chain is constructed such that the finite probabilities of aggregated states equal the finite probabilities of the corresponding states of the initial Markov chain. A method is developed for determining the upper and lower estimates of finite probabilities of aggregated states from data defining the initial Markov chain. These estimates are related with the necessary and sufficient conditions for the classical aggregation of Markov chains. An example on computations is given.     

Adaptive estimation for a linear system with interrupted observation

This short paper is concerned with the Bayesian estimation problem for a linear system with the interrupted observation mechanism that is expressed in terms of the stationary two-state Markov chain with unknown transition probabilities. Derived is the approximate minimum variance adaptive estimator algorithm coupled with the estimation of the unknown transition probabilities.     

Slotted Aloha as a game with partial information

This paper studies distributed choice of retransmission probabilities in slotted ALOHA. Both the cooperative team problem as well as the noncooperative game problem are considered. Unlike some previous work, we assume that mobiles do not know the number of backlogged packets at other nodes. A Markov chain analysis is used to obtain optimal and equilibrium retransmission probabilities and throughput. We then investigate the impact of adding retransmission costs (which may represent the disutility for power consumption) on the equilibrium and show how this pricing can be used to make the equilibrium throughput coincide with the optimal team throughput.     

Generating transition probabilities to support model‐based software testing

Markov chain usage models support test planning, test automation, and analysis of test results. In practice, transition probabilities for Markov chain usage models are often specified using a cycle of assigning, verifying, and revising specific values for individual transition probabilities. For large systems, such an approach can be difficult for a variety of reasons. We describe an improved approach that represents transition probabilities by explicitly preserving the information concerning test objectives and the relationships between transition probabilities in a format that is easy to maintain and easy to analyze. Using mathematical programming, transition probabilities are automatically generated to satisfy test management objectives and constraints. A more mathematical treatment of this approach is given in References [ 1 ] (Poore JH, Walton GH, Whittaker JA. A constraint‐based approach to the representation of software usage models. Information and SoftwareTechnology 2000; at press) and [ 2 ] (Walton GH. Generating transition probabilities for Markov chain usage models. PhD Thesis, University of Tennessee, Knoxville, TN, May 1995.). In contrast, this paper is targeted at the software engineering practitioner, software development manager, and test manager. This paper also adds to the published literature on Markov chain usage modeling and model‐based testing by describing and illustrating an iterative process for usage model development and optimization and by providing some recommendations for embedding model‐based testing activities within an incremental development process. Copyright © 2000 John Wiley & Sons, Ltd.     

Control of discrete-time HMM partially observed under fractional Gaussian noises

A discrete-time control problem of a finite-state hidden Markov chain partially observed in a fractional Gaussian process is discussed using filtering. The control problem is then recast as a separated problem with information variables given by the unnormalized conditional probabilities of the whole path of the hidden Markov chain. A dynamic programming result and a minimum principle are obtained.     

Performance assessment for single echelon airport spare part management

This study compares approximation techniques for the estimation of the operational availability of a corrective maintenance system. The assessment is based on the practical maintenance of safety equipments in operation in 38 Italian Airports. A single echelon one-for-one ordering policy with complete pooling is analyzed, with a deterministic rule for lateral transshipments. With this policy, the state probabilities of the associated Markov model cannot be expressed in product form. Since the exact computation of the state probabilities is not practical as the number of states in the Markov chain increases, this study describes three approximation techniques and assesses their performance in terms of computational effort, memory requirement and error with respect to the exact value. The first two techniques are based on a method by Alfredsson and Verrijdt and on the Equivalent Random Traffic method, respectively. The idea of both methods is to approximate the state probabilities with a product form, so that the Markov chain can be decomposed. The third technique is based on the multi-dimensional scaling down approach, which studies an equivalent reduced Markov chain rather than decomposing the original one.     

Enumerating the decomposable neighbors of a decomposable graph under a simple perturbation scheme

Given a decomposable graph, we characterize and enumerate the set of pairs of vertices whose connection or disconnection results in a new graph that is also decomposable. We discuss the relevance of these results to Markov chain Monte Carlo methods that sample or optimize over the space of decomposable graphical models according to probabilities determined by a posterior distribution given observed multivariate data.     

Waiting time distribution of generalized later patterns

In this paper the concept of later waiting time distributions for patterns in multi-state trials is generalized to cover a collection of compound patterns that must all be counted pattern-specific numbers of times, and a practical method is given to compute the generalized distribution. The solution given applies to overlapping counting and two types of non-overlapping counting, and the underlying sequences are assumed to be Markovian of a general order. Patterns are allowed to be weighted so that an occurrence is counted multiple times, and patterns may be completely included in longer patterns. Probabilities are computed through an auxiliary Markov chain. As the state space associated with the auxiliary chain can be quite large if its setup is handled in a naïve fashion, an algorithm is given for generating a “minimal” state space that leaves out states that can never be reached. For the case of overlapping counting, a formula that relates probabilities for intersections of events to probabilities for unions of subsets of the events is also used, so that the distribution is also computed in terms of probabilities for competing patterns. A detailed example is given to illustrate the methodology.     

Feature Enhancement for Noisy Speech Recognition With a Time-Variant Linear Predictive HMM Structure

This paper presents a new approach for speech feature enhancement in the log-spectral domain for noisy speech recognition. A switching linear dynamic model (SLDM) is explored as a parametric model for the clean speech distribution. Each multivariate linear dynamic model (LDM) is associated with the hidden state of a hidden Markov model (HMM) as an attempt to describe the temporal correlations among adjacent frames of speech features. The state transition on the Markov chain is the process of activating a different LDM or activating some of them simultaneously by different probabilities generated by the HMM. Rather than holding a transition probability for the whole process, a connectionist model is employed to learn the time variant transition probabilities. With the resulting SLDM as the speech model and with a model for the noise, speech and noise are jointly tracked by means of switching Kalman filtering. Comprehensive experiments are carried out using the Aurora2 database to evaluate the new algorithm. The results show that the new SLDM approach can further improve the speech feature enhancement performance in terms of noise-robust recognition accuracy, since the transition probabilities among the LDMs can be described more precisely at each time point.     

Comments on "Analysis, stability, and optimization of slotted ALOHA with a finite number of buffered users"

The model of a buffered slotted ALOHA scheme presented by Saadawi and Ephremides in the above paper is improved by refining the USER and SYSTEM Markov chains. New expressions are proposed for the transition probabilities in the USER chain. The SYSTEM chain is also modified to account for the influence of an arriving packet on all possible transitions. The principle of flow balance at equilibrium (the net packet input flow rate should be equal to the rate of successful packet transmissions) is employed to justify these changes. Incorporating the proposed modifications, the packet flow equivalence condition is established analytically for the two-user, single-buffer case and numerically for systems with a larger number of users, each one having a single, finite, or an infinite buffer.     

马尔科夫链的粒子群优化算法全局收敛性分析

本文对粒子群优化算法的全局收敛性进行了分析,给出了粒子速度和位置的一步转移概率,然后从粒子状态所构成的马尔科夫链着手,分析了此马尔科夫链的一系列性质,证明了粒子状态空间的可约性和非齐次性,并验证粒子状态空间是非常返态的,最后表明马尔科夫链不存在平稳过程的条件,继而从转移概率的角度证明了算法不是全局收敛的.     

Exact computation of pattern probabilities in random sequences generated by Markov chains

Observed patterns in macromolecular sequences are often considered as words and compared with their probabilities of occurring in random sequences. Calculation of these probabilities, however, often lacks rigour. We have developed an algorithm for exact computation of such probabilities for stochastic sequences that follow a Markov chain model. The method is applicable to the case that a random sequence contains one out of two given patterns P and Q, or both simultaneously. Another application yields the probability function P(x) that a sequence contains pattern P exactly x times. An application to patterns that include wild-card characters yields probabilities for homonucleotide clusters of a given length. We prove the probability of multiple runs of single nucleotides in the SV40 genome to be in accordance with the dinucleotide composition of the sequence, although it is in conflict with mononucleotide composition.     

Markov analysis of qualitative dynamics1

Common sense sometimes predicts events to be likely or unlikely rather than merely possible. We extend methods of qualitative reasoning to predict the relative likelihoods of possible qualitative behaviors by viewing the dynamics of a system as a Markov chain over its transition graph. This involves adding qualitative or quantitative estimates of transition probabilities to each of the transitions and applying the standard theory of Markov chains to distinguish persistent states from transient states and to calculate recurrence times, settling times, and probabilities for ending up in each state. Much of the analysis depends solely on qualitative estimates of transition probabilities, which follow directly from theoretical considerations and which lead to qualitative predictions about entire classes of systems. Quantitative estimates for specific systems are derived empirically and lead to qualitative and quantitative conclusions, most of which are insensitive to small perturbations in the estimated transition probabilities. The algorithms are straightforward and efficient.     

A further result on the Markov chain model of genetic algorithmsand its application to a simulated annealing-like strategy

This paper shows a theoretical property on the Markov chain of genetic algorithms: the stationary distribution focuses on the uniform population with the optimal solution as mutation and crossover probabilities go to zero and some selective pressure defined in this paper goes to infinity. Moreover, as a result, a sufficient condition for ergodicity is derived when a simulated annealing-like strategy is considered. Additionally, the uniform crossover counterpart of the Vose-Liepins formula is derived using the Markov chain model.     

A constraint-based approach to the representation of software usage models

Software usage models are the basis for statistical testing. They derive their structure from specifications and their probabilities from evolving knowledge about the intended use of the software product. The evolving knowledge comes from developers, customers and testers of the software system in the form of relationships that should hold among the parameters of a model. When software usage models are encoded as Markov chains, their structure can be represented by a system of linear constraints, and many of the evolving relationships among model parameters can be represented by convex constraints. Given a Markov chain usage model as a system of convex constraints, mathematical programming can be used to generate the Markov chain transition probabilities that represent a specific software usage model.     

无线Ad hoc网络MAC层吞吐量分析

MAC层吞吐量分析是无线Ad hoc网络容量分析的基础。对CSMA协议特别是IEEE802.11DCF协议建立了一个Markov链分析模型。分析得出状态间的转移概率,通过建立状态方程得出稳态概率的线性方程。通过数值方法得出稳态解,从而得到无线Ad hoc网络MAC层吞吐量,并与公认的网络容量分析结果做了比较。通过吞吐量与节点数及分组大小的关系曲线,为网络性能的优化提供了理论基础。     

基于重要抽样的软件统计测试加速

本文提出一种基于重要抽样的软件统计测试加速方法,该方法通过调整软件Markov链使用模型的迁移概率,在根据统计测试结果得到软件可靠性无偏估计的前提下,可以有效提高安全攸关软件的测试效率,部分解决了安全攸关软件统计测试时间和费用开销过大的问题。同时,本文给出了计算优化迁移概率的模拟退火算法。实验仿真结果表明,该方法可以有效地提高安全攸关软件统计测试的效率。     

基于Monte Carlo方法的自适应多模型诊断

多模型混合系统的模型切换服从有限状态的Markov链,其转移概率通常假定是已知的.当模型转移概率未知的时候,本文基于Monte Carlo粒子滤波器给出了混合系统状态估计的一种自适应算法.该算法假定未知的转移概率先验分布为Dirichlet分布,首先通过采样得到一组模型序列的随机样本,利用其中状态的转移次数计算先验转移概率,使用量测信息对样本更新选择后,获得模型转移概率的一种迭代的后验估计值,同时由粒子滤波器得到系统状态和模型概率的后验估计.将该方法用于混合系统的状态监测和故障诊断,仿真结果表明了算法的有效性.     

Adjusting the outputs of a classifier to new a priori probabilities: a simple procedure.

It sometimes happens (for instance in case control studies) that a classifier is trained on a data set that does not reflect the true a priori probabilities of the target classes on real-world data. This may have a negative effect on the classification accuracy obtained on the real-world data set, especially when the classifier's decisions are based on the a posteriori probabilities of class membership. Indeed, in this case, the trained classifier provides estimates of the a posteriori probabilities that are not valid for this real-world data set (they rely on the a priori probabilities of the training set). Applying the classifier as is (without correcting its outputs with respect to these new conditions) on this new data set may thus be suboptimal. In this note, we present a simple iterative procedure for adjusting the outputs of the trained classifier with respect to these new a priori probabilities without having to refit the model, even when these probabilities are not known in advance. As a by-product, estimates of the new a priori probabilities are also obtained. This iterative algorithm is a straightforward instance of the expectation-maximization (EM) algorithm and is shown to maximize the likelihood of the new data. Thereafter, we discuss a statistical test that can be applied to decide if the a priori class probabilities have changed from the training set to the real-world data. The procedure is illustrated on different classification problems involving a multilayer neural network, and comparisons with a standard procedure for a priori probability estimation are provided. Our original method, based on the EM algorithm, is shown to be superior to the standard one for a priori probability estimation. Experimental results also indicate that the classifier with adjusted outputs always performs better than the original one in terms of classification accuracy, when the a priori probability conditions differ from the training set to the real-world data. The gain in classification accuracy can be significant.