隐马尔可夫模型在自然语言处理中的应用

隐马尔可夫模型是序列数据处理和统计学习的一种重要概率模型,最近几年已经被成功应用到许多关于自然语言处理的任务中.简要介绍了隐马尔可夫模型,对其在词性标注应用中的难点、模型的建立,Viterbi算法等问题进行了详细论述,给出了基于隐马尔可夫模型的中文科研论文头部信息抽取过程以及模型结构的学习和参数的训练等关键问题的解决办法.     

隐马尔可夫模型及其在自然语言处理中的应用

基于隐马尔可夫模型(HMM)对汉语文本进行了词性标注,首先介绍隐马尔可夫模型的基本概念,然后着重介绍了隐马尔可夫模型的三个基本问题以及解决问题的基本算法,最后演示了隐马尔可夫模型在词性标注中的简单应用.     

隐马尔可夫模型在维吾尔语词性标注中的应用

本文提出了将三阶隐马尔可夫模型运用到维吾尔语词性标注中的方法。运用改进的Baum-Welch方法训练模型参数。并且采用改良的动态规划方法:viterbi算法,找出最优标注序列。     

隐马尔可夫模型在维吾尔语词性标注中的应用

本文提出了将三阶隐马尔可夫模型运用到维吾尔语词性标注中的方法。运用改进的Baum-Welch方法训练模型参数。并且采用改良的动态规划方法：viterbi算法，找出最优标注序列。     

基于上下文的二阶隐马尔可夫模型

为体现上下文信息对当前词汇词性的影响,在传统隐马尔可夫模型的基础上提出一种基于上下文的二阶隐马尔可夫模型,并应用于中文词性标注中。针对改进后的统计模型中由于训练数据过少而出现的数据稀疏问题,给出基于指数线性插值改进平滑算法,对参数进行有效平滑。实验表明,基于上下文的二阶隐马尔可夫模型比传统的隐马尔可夫模型具有更高的词性标注正确率和消歧率。     

BioTrHMM:基于迁移学习的生物医学命名实体识别算法

传统的生物医学命名实体识别方法需要大量目标领域的标注数据,但是标注数据代价高昂。为了降低生物医学文本中命名实体识别对目标领域标注数据的需求,将生物医学文本中的命名实体识别问题化为基于迁移学习的隐马尔可夫模型问题。对要进行命名实体识别的目标领域数据集无须进行大量数据标注,通过迁移学习的方法实现对目标领域的识别分类。以相关领域数据为辅助数据集,利用数据引力的方法评估辅助数据集的样本在目标领域学习中的贡献程度,在辅助数据集和目标领域数据集上计算权值进行迁移学习。基于权值学习模型,构建基于迁移学习的隐马尔可夫模型算法BioTrHMM。在GENIA语料库的数据集上的实验表明,BioTrHMM算法比传统的隐马尔可夫模型算法具有更好的性能;仅需要少量的目标领域标注数据,即可具有较好的命名实体识别性能。     

基于二阶隐马尔可夫模型的连续手语识别

在传统的一阶隐马尔可夫模型(HMM1)中, 状态序列中的每一个状态被假设只与前一个状态有关, 这样虽然可以简单、有效地推导出模型的学习和识别算法, 但也丢失了许多从上文传递下来的信息. 因此, 在传统一阶隐马尔可夫模型的基础上, 为了解决手语识别困难、正确率低的问题, 提出了一种基于二阶隐马尔可夫模型(HMM2)的连续手语识别方法. 该方法利用滑动窗口算法使手语视频切分成多个手语短视频, 通过三维卷积模型得到手语短视频和手语词汇视频的特征向量, 由此计算出二阶隐马尔可夫模型的相关参数, 并运用Viterbi算法实现连续手语的识别. 实验证明, 基于二阶隐马尔可夫模型的手语识别取得了88.6%的识别准确率, 高于传统的一阶隐马尔可夫模型.     

基于改进粒子群算法的隐马尔可夫模型训练

针对隐马尔可夫模型传统训练算法易收敛于局部极值的问题,提出一种带极值扰动的自适应调整惯性权重和加速系数的粒子群算法,将改进后的粒子群优化算法引入到隐马尔可夫模型的训练中,分别对隐马尔可夫模型的状态数与参数进优化.通过对手写数字识别的实验说明,提出的基于改进粒子群优化算法的隐马尔可夫模型训练算法与传统隐马尔可夫模型训练算法Baum-Welch算法相比,能有效地跳出局部极值,从而使训练后的隐马尔可夫模型具有较高的识别能力.     

基于遗传算法的隐马尔可夫模型在名词短语识别中的应用研究

为了进一步提高名词短语的识别精度,针对遗传算法和隐马尔可夫模型各自的特点,提出一种基于遗传算法的隐马尔可夫模型识别方法。该方法是在高准确率词性标注的基础上实现的。在训练阶段,用遗传算法获取HMM参数;识别阶段先用一种改进的Viterbi算法进行动态规划,识别同层名词短语,然后用逐层扫描算法和改进Viterbi算法相结合来识别嵌套名词短语。实验结果表明,此联合算法达到了94.78%的准确率和94.29%的召回率,充分融合了遗传算法和隐马尔可夫模型的优点,证明它较单一的隐马尔可夫模型识别法具有更好的识别效果。     

隐马尔可夫模型的多序列比对研究

研究一种关于隐马尔可夫模型的多序列比对,利用值和特征序列的保守性,通过增加频率因子,改进传统隐马尔可夫模型算法的不足。实验表明,新算法不但提高了模型的稳定性,而且应用于蛋白质家族识别,平均识别率比传统隐马尔可夫算法提高了3.3个百分点。     

一种改进的隐马尔可夫模型在语音识别中的应用

提出了一种新的马尔可夫模型——异步隐马尔可夫模型．该模型针对噪音环境下语音识别过程中出现丢失帧的情况，通过增加新的隐藏时间标示变量Ck，估计出实际观察值对应的状态序列，实现对不规则或者不完整采样数据的建模．详细介绍了适合异步HMM的前后向算法以及用于训练的EM算法，并且对转移矩阵的计算进行了优化．最后通过实验仿真，分别使用经典HMM和异步HMM对相同的随机抽取帧的语音数据进行识别,识别结果显示在抽取帧相同情况下异步HMM比经典HMM的识别错误率低．     

SCALED SELF-ORGANIZING MAP – HIDDEN MARKOV MODEL ARCHITECTURE FOR BIOLOGICAL SEQUENCE CLUSTERING

The self-organizing hidden Markov model map (SOHMMM) introduces a hybrid integration of the self-organizing map (SOM) and the hidden Markov model (HMM). Its scaled, online gradient descent unsupervised learning algorithm is an amalgam of the SOM unsupervised training and the HMM reparameterized forward-backward techniques. In essence, with each neuron of the SOHMMM lattice, an HMM is associated. The image of an input sequence on the SOHMMM mesh is defined as the location of the best matching reference HMM. Model tuning and adaptation can take place directly from raw data, within an automated context. The SOHMMM can accommodate and analyze deoxyribonucleic acid, ribonucleic acid, protein chain molecules, and generic sequences of high dimensionality and variable lengths encoded directly in nonnumerical/symbolic alphabets. Furthermore, the SOHMMM is capable of integrating and exploiting latent information hidden in the spatiotemporal dependencies/correlations of sequences’ elements.     

基于MLP-HMM的跨站脚本攻击检测

针对隐马尔科夫模型(HMM)在跨站脚本检测中对初始先验假设估计不准确和以极大似然准则规定的HMM参数分类能力差的缺陷,提出了一种基于MLP-HMM的跨站脚本检测模型。首先,使用自然语言处理(NLP)方法解决数据高维复杂性问题。然后,通过多层感知机(MLP)神经网络学习对整个模型进行权值微调得到初始观察矩阵。最后,将该观察矩阵代入HMM中,增强HMM参数构建能力和分类能力。结果表明,结合MLP的HMM相比于原始HMM以及传统算法在跨站脚本检测上检测率有显著提高,并缩短了检测时间。     

基于序列标注的全词消歧方法

全词消歧(All-Words Word Sense Disambiguation)可以看作一个序列标注问题,该文提出了两种基于序列标注的全词消歧方法,它们分别基于隐马尔可夫模型(Hidden Markov Model, HMM)和最大熵马尔可夫模型(Maximum Entropy Markov Model, MEMM)。首先,我们用HMM对全词消歧进行建模。然后,针对HMM只能利用词形观察值的缺点,我们将上述HMM模型推广为MEMM模型,将大量上下文特征集成到模型中。对于全词消歧这类超大状态问题,在HMM和MEMM模型中均存在数据稀疏和时间复杂度过高的问题,我们通过柱状搜索Viterbi算法和平滑策略来解决。最后,我们在Senseval-2和Senseval-3的数据集上进行了评测,该文提出的MEMM方法的F1值为0.654,超过了该评测上所有的基于序列标注的方法。     

Increasing mapping based hidden Markov model for dynamic process monitoring and diagnosis

Hidden Markov models (HMMs) perform parameter estimation based on the forward–backward (FB) procedure and the Baum–Welch (BW) algorithm. The two algorithms together may increase the computational complexity and the difficulty to understand the algorithm structure of HMMs clearly. In this study, an increasing mapping based hidden Markov model (IMHMM) is proposed. Between the observation sequence and possible state sequence an increasing mapping is established. The re-estimation formulas for the model parameters are derived straightforwardly based on these mappings instead of FB variables. The IMHMM has simpler algorithm structure and lower storage requirement than the HMM. Based on IMHMM, an expandable process monitoring and fault diagnosis framework for large-scale dynamical process is developed. To characterize the dynamic process, a novel index considering serial correlation is used to evaluate process state. The presented methodology is carried out in Tennessee Eastman process (TEP). The results show improvement over HMM in terms of memory complexity and training time of the model. Also, the power of IMHMM can be observed compared with principal component analysis (PCA) based methods.     

Hidden Markov models with patterns to learn Boolean vectorsequences and applications to the built-in self-test for integratedcircuits

We present a new model, derived from the hidden Markov model (HMM), to learn Boolean vector sequences. Our HMM with patterns (HMMP) is a simple, hybrid, and interpretable model that uses Boolean patterns to define emission probability distributions attached to states. Vectors consistent with a given pattern are equally probable, while inconsistent ones have probability zero to be emitted. We define an efficient learning algorithm for this model, which relies on the maximum likelihood principle, and proceeds by iteratively simplifying the structure and updating the parameters of an initial specific HMMP that represents the learning sequences. HMMPs and our learning algorithm are applied to the built-in self-test (BIST) for integrated circuits, which is one of the key microelectronic problems. An HMMP is learned from a test sequence set that covers most of the potential faults of the circuit at hand. Then, this HMMP is used as test sequence generator. The experiments carried out show that learned HMMPs have a very high fault coverage     

Locating Human Faces in a Cluttered Scene

In this paper, we present two new schemes for finding human faces in a photograph. The first scheme adopts a distribution-based model approach to face-finding. Distributions of the face and the face-like manifolds are approximated using higher order statistics (HOS) by deriving a series expansion of the density function in terms of the multivariate Gaussian and the Hermite polynomials in an attempt to get a better approximation to the unknown original density function. An HOS-based data clustering algorithm is then proposed to facilitate the decision process. The second scheme adopts a hidden Markov model (HMM) based approach to the face-finding problem. This is an unsupervised scheme in which face-to-nonface and nonface-to-face transitions are learned by using an HMM. The HMM learning algorithm estimates the HMM parameters corresponding to a given photograph and the faces are located by examining the optimal state sequence of the HMM. We present experimental results on the performance of both schemes. A training data base of face images was constructed in the laboratory. The performances of both the proposed schemes are found to be quite good when measured with respect to several standard test face images.     

输入为Markov序列时的多用户检测算法

考虑当用户序列存在时间相关性时的多用户检测，并假设这种相关性可以用Markov链描述，在传统的线性最大似然检测器中嵌入一个隐Markov模型估计过程。因为输入序列是Markov链，检测器的输出可以看成是被噪声污染的Markov序列，Markov模型估计子用于估计用户序列及其转移概率，而估计得到的用户序列用来更新检测器的估计。因此，检测器和用户序列可以通过迭代的方式求解。仿真结果显示本文算法能充分利用信道输入的时间相关性．效果优于传统的最大似然线性检测器。     

Harmony search-based hidden Markov model optimization for online classification of single trial eegs during motor imagery tasks

This paper presents an improved method based on single trial EEG data for the online classification of motor imagery tasks for brain-computer interface (BCI) applications. The ultimate goal of this research is the development of a novel classification method that can be used to control an interactive robot agent platform via a BCI system. The proposed classification process is an adaptive learning method based on an optimization process of the hidden Markov model (HMM), which is, in turn, based on meta-heuristic algorithms. We utilize an optimized strategy for the HMM in the training phase of time-series EEG data during motor imagery-related mental tasks. However, this process raises important issues of model interpretation and complexity control. With these issues in mind, we explore the possibility of using a harmony search algorithm that is flexible and thus allows the elimination of tedious parameter assignment efforts to optimize the HMM parameter configuration. In this paper, we illustrate a sequential data analysis simulation, and we evaluate the optimized HMM. The performance results of the proposed BCI experiment show that the optimized HMM classifier is more capable of classifying EEG datasets than ordinary HMM during motor imagery tasks.