Non-stationary fuzzy Markov chain

This paper deals with a recent statistical model based on fuzzy Markov random chains for image segmentation, in the context of stationary and non-stationary data. On one hand, fuzzy scheme takes into account discrete and continuous classes through the modeling of hidden data imprecision and on the other hand, Markovian Bayesian scheme models the uncertainty on the observed data. A non-stationary fuzzy Markov chain model is proposed in an unsupervised way, based on a recent Markov triplet approach. The method is compared with the stationary fuzzy Markovian chain model. Both stationary and non-stationary methods are enriched with a parameterized joint density, which governs the attractiveness of the neighbored states. Segmentation task is processed with Bayesian tools, such as the well known MPM (Mode of Posterior Marginals) criterion. To validate both models, we perform and compare the segmentation on synthetic images and raw optical patterns which present diffuse structures.     

输入为Markov序列时的多用户检测算法

考虑当用户序列存在时间相关性时的多用户检测，并假设这种相关性可以用Markov链描述，在传统的线性最大似然检测器中嵌入一个隐Markov模型估计过程。因为输入序列是Markov链，检测器的输出可以看成是被噪声污染的Markov序列，Markov模型估计子用于估计用户序列及其转移概率，而估计得到的用户序列用来更新检测器的估计。因此，检测器和用户序列可以通过迭代的方式求解。仿真结果显示本文算法能充分利用信道输入的时间相关性．效果优于传统的最大似然线性检测器。     

Hidden Markov models for the stimulus-response relationships of multistate neural systems

Given recent experimental results suggesting that neural circuits may evolve through multiple firing states, we develop a framework for estimating state-dependent neural response properties from spike train data. We modify the traditional hidden Markov model (HMM) framework to incorporate stimulus-driven, non-Poisson point-process observations. For maximal flexibility, we allow external, time-varying stimuli and the neurons' own spike histories to drive both the spiking behavior in each state and the transitioning behavior between states. We employ an appropriately modified expectation-maximization algorithm to estimate the model parameters. The expectation step is solved by the standard forward-backward algorithm for HMMs. The maximization step reduces to a set of separable concave optimization problems if the model is restricted slightly. We first test our algorithm on simulated data and are able to fully recover the parameters used to generate the data and accurately recapitulate the sequence of hidden states. We then apply our algorithm to a recently published data set in which the observed neuronal ensembles displayed multistate behavior and show that inclusion of spike history information significantly improves the fit of the model. Additionally, we show that a simple reformulation of the state space of the underlying Markov chain allows us to implement a hybrid half-multistate, half-histogram model that may be more appropriate for capturing the complexity of certain data sets than either a simple HMM or a simple peristimulus time histogram model alone.     

A M/D/1 vacation queue model for a signalized intersection

We consider a queueing system that arises in the modeling of isolated signalized intersections in a urban transportation network. In this system, the server alternates in two states, attended or removed, in respect to the queue, while in each state, the server will spend a constant time period with different value. It is assumed that the server is able to disperse up to r(r≥1) customers during a constant service cycle. The evolution of this queueing system can be characterized by a Markov chain embedded at equally spaced time epochs along the time axis. Transition matrix of this Markov chain is of the M/G/1 type introduced by Neuts so that matrix analytical method can be applied to obtain the necessary and sufficient criterion for ergodicity of this Markov chain as well as to compute its stationary distribution. Furthermore, the queue length and waiting time distributions with other performance measures are also given in this paper.     

HMM-Based Concept Learning for a Mobile Robot

We are developing an intelligent robot and attempting to teach it language. While there are many aspects of this research, for the purposes here the most important are the following ideas. Language is primarily based on semantics, not syntax, which is still the focus in speech recognition research these days. To truly learn meaning, a language engine cannot simply be a computer program running on a desktop computer analyzing speech. It must be part of a more general, embodied intelligent system, one capable of using associative learning to form concepts from the perception of experiences in the world, and further capable of manipulating those concepts symbolically. In this paper, we present a general cascade model for learning concepts, and explore the use of hidden Markov models (HMMs) as part of the cascade model. HMMs are capable of automatically learning and extracting the underlying structure of continuous-valued inputs and representing that structure in the states of the model. These states can then be treated as symbolic representations of the inputs. We show how a cascade of HMMs can be embedded in a small mobile robot and used to find correlations among sensory inputs to learn a set of symbolic concepts, which are used for decision making and could eventually be manipulated linguistically     

An Empirical Study of Two Approaches to Sequence Learning for Anomaly Detection

This paper introduces the computer security domain of anomaly detection and formulates it as a machine learning task on temporal sequence data. In this domain, the goal is to develop a model or profile of the normal working state of a system user and to detect anomalous conditions as long-term deviations from the expected behavior patterns. We introduce two approaches to this problem: one employing instance-based learning (IBL) and the other using hidden Markov models (HMMs). Though not suitable for a comprehensive security solution, both approaches achieve anomaly identification performance sufficient for a low-level focus of attention detector in a multitier security system. Further, we evaluate model scaling techniques for the two approaches: two clustering techniques for the IBL approach and variation of the number of hidden states for the HMM approach. We find that over both model classes and a wide range of model scales, there is no significant difference in performance at recognizing the profiled user. We take this invariance as evidence that, in this security domain, limited memory models (e.g., fixed-length instances or low-order Markov models) can learn only part of the user identity information in which we're interested and that substantially different models will be necessary if dramatic improvements in user-based anomaly detection are to be achieved.     

基于函数型数据时间序列建模的单传感器日常行为识别

在基于惯性传感器的人体行为识别中,传统算法常忽略行为的周期性与时序性,对提取特征的滑动窗口大小也有相应要求.文中基于单个腰部传感器分析人体日常行为,提出面向周期行为的函数型数据分析方法和隐马尔可夫模型结合的行为识别算法.首先,使用函数型数据分析方法,拟合周期性日常行为的动作捕捉数据,提取拟合后的单个周期数据.然后基于此周期时间序列数据建立描述各个日常行为过程的隐马尔可夫模型.最后,使用最大似然估计判别行为,得到识别结果.该算法通过单个腰部传感器即可快速有效地识别8种日常行为,在基于用户依赖策略和用户独立策略时识别率较高.     

基于SVM-HMM混合模型的说话人确认

提出一个文本无关的说话人确认的算法。该算法将支持向量机（SVM）的输出通过Sigmoid函数和高斯模型转化为概率，并作为隐式马尔可夫模型（HMM）中各个隐状态的输出概率。由于HMM适于处理连续信号，SVM适于处理分类问题；同时，HMM更多地表达了类别内部的相似性，而SVM则很大程度上反映了类别间的差异，因而根据两者不同的侧重点，使其组合获得了很好的效果。     

Mixed Memory Markov Models: Decomposing Complex Stochastic Processes as Mixtures of Simpler Ones

We study Markov models whose state spaces arise from the Cartesian product of two or more discrete random variables. We show how to parameterize the transition matrices of these models as a convex combination—or mixture—of simpler dynamical models. The parameters in these models admit a simple probabilistic interpretation and can be fitted iteratively by an Expectation-Maximization (EM) procedure. We derive a set of generalized Baum-Welch updates for factorial hidden Markov models that make use of this parameterization. We also describe a simple iterative procedure for approximately computing the statistics of the hidden states. Throughout, we give examples where mixed memory models provide a useful representation of complex stochastic processes.     

基于Shell命令和DTMC模型的用户行为异常检测新方法

提出一种新的基于离散时间Markov链模型的用户行为异常检测方法,主要用于以shell命令为审计数据的入侵检测系统。该方法在训练阶段充分考虑了用户行为复杂多变的特点和审计数据的短时相关性,将shell命令序列作为基本数据处理单元,依据其出现频率利用阶梯式的数据归并方法来确定Markov链的状态,同现有方法相比提高了用户行为轮廓描述的准确性和对用户行为变化的适应性,并且大幅度减少了状态个数,节约了存储成本。在检测阶段,针对检测实时性和准确度需求,通过计算状态序列的出现概率分析用户行为异常程度,并提供了基于固定窗长度和可变窗长度的两种均值滤噪处理及行为判决方案。实验表明,该方法具有很高的检测性能,其可操作性也优于同类方法。     

A new disk-based technique for solving the largeness problem of stochastic modeling formalisms

Stochastic modeling formalisms such as stochastic Petri nets, generalized stochastic Petri nets, and stochastic reward nets can be used to model and evaluate the dynamic behavior of realistic computer systems. Once we translate the stochastic system model to the underlying corresponding Markov Chain (MC), the developed MC grows wildly to several hundred thousands states. This problem is known as the largeness problem. To tolerate the largeness problem of Markov models, several iterative and direct methods have been proposed in the literature. Although the iterative methods provide a feasible solution for most realistic systems, a major problem appears when these methods fail to reach a solution. Unfortunately, the direct method represents an undesirable numerical technique for tolerating large matrices due to the fill-in problem. In order to solve such problem, in this paper, we develop a disk-based segmentation (DBS) technique based on modifying the Gauss Elimination (GE) technique. The proposed technique has the capability of solving the consequences of the fill-in problem without making assumptions about the underlying structure of the Markov processes of the developed model. The DBS technique splits the matrix into a number of vertical segments and uses the hard disk to store these segments. Using the DBS technique, we can greatly reduce the memory required as compared to that of the GE technique. To minimize the increase in the solution time due to the disk accessing processes, the DBS utilizes a clever management technique for such processes. The effectiveness of the DBS technique has been demonstrated by applying it to a realistic model for the Kanban manufacturing system.     

A note on the attractor-property of infinite-state Markov chains

In the past 5 years, a series of verification algorithms has been proposed for infinite Markov chains that have a finite attractor, i.e., a set that will be visited infinitely often almost surely starting from any state. In this paper, we establish a sufficient criterion for the existence of an attractor. We show that if the states of a Markov chain can be given levels (positive integers) such that the expected next level for states at some level n>0 is less than n−Δ for some positive Δ, then the states at level 0 constitute an attractor for the chain. As an application, we obtain a direct proof that some probabilistic channel systems combining message losses with duplication and insertion errors have a finite attractor.     

基于加权马尔可夫链的主动用户行为预测模型

建立有效的用户行为预测模型,准确地预测用户的上网行为,是当前网络主动管理地关键,传统的Markov模型是一种简单而有效的预测模型,但它存在测准确率低、预测覆盖率低以及存储复杂度高等缺点.提出了基于加权马尔可夫链模型,通过分析用户行为特征和最优状态分类的方法,预测网络用户行为.最后通过实验结果表明了该模型的可行性和实用性...     

Continuous-time hidden Markov models for network performance evaluation

In this paper, we study the use of continuous-time hidden Markov models (CT-HMMs) for network protocol and application performance evaluation. We develop an algorithm to infer the CT-HMM from a series of end-to-end delay and loss observations of probe packets. This model can then be used to simulate network environments for network performance evaluation. We validate the simulation method through a series of experiments both in ns and over the Internet. Our experimental results show that this simulation method can represent a wide range of real network scenarios. It is easy to use, accurate and time efficient.     

网络用户兴趣的智能挖掘方法研究

目前网络上的重要应用都是围绕对用户兴趣的研究和发现而展开和完善的,主要的方式是借助于对用户的Web访问数据进行相关挖掘。该研究主要是通过建立一个从底层数据获取到上层数据处理的原型系统,对真实捕获的网络数据利用小世界网络模型提取中文文档关键字后处理为用户兴趣,再将用户的访问兴趣通过隐马尔可夫模型抽象成一种时间序列,依次反映用户兴趣的序列性,从而利用GSP算法得到用户的兴趣并供后续处理。实验证明,该原型系统从数据获取到最终处理,可以得到比较满意的结果。     

基于隐马尔可夫模型的兴趣迁移模式发现

Web挖掘的一个重要研究方向是发现用户的迁移模式。一般来说,用户的迁移具有某种目的性。这种目的性表现为用户对某种概念的兴趣。文中提出基于隐马尔可夫模型的兴趣迁移模式发现方法,用于发现这种带有某种兴趣的用户迁移模式,这种模式实质上是一种特殊的关联规则。在这种方法中,作者首先根据用户的访问记录定义一个隐马尔可夫模型,然后提出一种新的增量发现算法Increase_R用于发现兴趣迁移模式,同时给出了证明以说明该算法可以发现所有的兴趣迁移模式。     

A new parallel block aggregated algorithm for solving Markov chains

In this paper, we propose a new scalable parallel block aggregated iterative method (PBA) for computing the stationary distribution of a Markov chain. The PBA technique is based on aggregation of groups (block) of Markov chain states. Scalability of the PBA algorithm depends on varying the number of blocks and their size, assigned to each processor. PBA solves the aggregated blocks very efficiently using a modified LU factorization technique. Some Markov chains have been tested to compare the performance of PBA algorithm with other block techniques such as parallel block Jacobi and block Gauss–Seidel. In all the tested models PBA outperforms the other parallel block methods.     

基于Adaboost-Markov模型的移动用户位置预测方法

针对Markov模型在位置预测中存在预测精度不高及匹配稀疏等问题,提出了一种基于Adaboost-Markov模型的移动用户位置预测方法。首先,通过基于转角偏移度与距离偏移量的轨迹划分方法对原始轨迹数据进行预处理,提取出特征点,并采用密度聚类算法将特征点聚类为用户的各个兴趣区域,把原始轨迹数据离散化为由兴趣区域组成的轨迹序列;然后,根据前缀轨迹序列与历史轨迹序列模式树的匹配程度来自适应地确定模型阶数k;最后,采用Adaboost算法根据1~k阶Markov模型的重要程度为其赋予相应的权重系数,组成多阶融合Markov模型,从而实现对移动用户未来兴趣区域的预测。在大规模真实用户轨迹数据集上的实验结果表明,与1阶Markov模型、2阶Markov模型、权重系数平均的多阶融合Markov模型相比,Adaboost-Markov模型的平均预测准确率分别提高了20.83%、11.3%以及5.38%,且具有良好的普适性与多步预测性能。