基于HMM-FNN模型的复杂动态手势识别

复杂动态手势识别是利用视频手势进行人机交互的关键问题.提出一种HMM-FNN模型结构.它整合了隐马尔可夫模型对时序数据的建模能力与模糊神经网络的模糊规则构建与推理能力,并将其应用到复杂动态手势的识别中.复杂动态手势具备两大特点:运动特征的可分解性与定义描述的模糊性.针对这两种特性,复杂手势被分解为手形变化、2D平面运动与Z轴方向运动3个子部分,分别利用HMM进行建模,HMM模型对观察子序列的似然概率被作为FNN的模糊隶属度,通过模糊规则推理,最终得到手势的分类类别.HMM-FNN方法将高维手势特征分解为低维子特征序列,降低了模型的复杂度.此外,它还可以充分利用人的经验辅助模型结构的创建与优化.实验表明,该方法是一种有效的复杂动态手势识别方法,并且优于传统的HMM模型方法.     

一种基于判别随机场模型的联机行为识别方法

提出了一种基于判别随机场模型的联机行为识别方法,将传统的随机场模型和隐藏条件随机场模型的特点相结合,构建一个针对于运动序列帧数据建模的帧-隐藏条件随机场模型,并将该模型应用于数据驱动的行为建模,利用传统条件随机场模型对行为间的运动特性进行建模;通过引入隐藏特征函数,设计有效的特征模板来表示行为中子姿态的联系,实现对行为的内在运动特性进行建模.通过对比实验表明,该模型对于联机处理行为序列具有更强的识别能力.     

基于条件随机域的上下文人类动作识别

提出一种新的基于条件随机域和隐马尔可夫模型（HMM）的人类动作识别方法——HMCRF。目前已有的动作识别方法均使用隐马尔可夫模型及其变型,这些模型一个最突出的不足就是要求观察值相互独立。条件模型很容易表示上下文相关性,且可使用动态规划做到有效且精确的推论,它的参数可以通过凸函数优化训练得到。把条件图形模型应用于动作识别之上,并通过大量的实验表明,所提出的方法在识别正确率方面明显优于一般线性结构的CRF和HMM。     

基于OpenCV的摄像头动态手势轨迹识别及其应用

利用OpenCV计算机视觉库在vs2008平台上设计了一个基于实时摄像头的集动态手势检测、动态手势跟踪、动态手势轨迹识别的应用.首先,该应用基于静止的背景更新,利用背景差分检测运动手势,再结合颜色直方图的粒子滤波进行动态手势跟踪,最后利用隐马尔可夫模型(HMM)进行运动轨迹识别.在运动检测部分结合了背景差分图与通过颜色直方图获得的反投影图,达到比较满意的实时运动检测效果;在运动手势跟踪部分,改进的颜色直方图的粒子跟踪能够在经过类肤色人脸的干扰后迅速地找回运动手势,基本达到了跟踪的要求,但是同时对于HMM识别轨迹时需要的运动轨迹序列采集造成了影响;在识别轨迹部分,HMM的训练达到了识别的要求,但是识别的效果主要取决于实时运动轨迹序列的采集工作与采集方法的优化.     

A probabilistic model for state sequence analysis in hidden Markov model for hand gesture recognition

The role of gesture recognition is significant in areas like human‐computer interaction, sign language, virtual reality, machine vision, etc. Among various gestures of the human body, hand gestures play a major role to communicate nonverbally with the computer. As the hand gesture is a continuous pattern with respect to time, the hidden Markov model (HMM) is found to be the most suitable pattern recognition tool, which can be modeled using the hand gesture parameters. The HMM considers the speeded up robust feature features of hand gesture and uses them to train and test the system. Conventionally, the Viterbi algorithm has been used for training process in HMM by discovering the shortest decoded path in the state diagram. The recursiveness of the Viterbi algorithm leads to computational complexity during the execution process. In order to reduce the complexity, the state sequence analysis approach is proposed for training the hand gesture model, which provides a better recognition rate and accuracy than that of the Viterbi algorithm. The performance of the proposed approach is explored in the context of pattern recognition with the Cambridge hand gesture data set.     

Hand gesture recognition based on dynamic Bayesian network framework

In this paper, we propose a new method for recognizing hand gestures in a continuous video stream using a dynamic Bayesian network or DBN model. The proposed method of DBN-based inference is preceded by steps of skin extraction and modelling, and motion tracking. Then we develop a gesture model for one- or two-hand gestures. They are used to define a cyclic gesture network for modeling continuous gesture stream. We have also developed a DP-based real-time decoding algorithm for continuous gesture recognition. In our experiments with 10 isolated gestures, we obtained a recognition rate upwards of 99.59% with cross validation. In the case of recognizing continuous stream of gestures, it recorded 84% with the precision of 80.77% for the spotted gestures. The proposed DBN-based hand gesture model and the design of a gesture network model are believed to have a strong potential for successful applications to other related problems such as sign language recognition although it is a bit more complicated requiring analysis of hand shapes.     

A model-based hand gesture recognition system

This paper introduces a model-based hand gesture recognition system, which consists of three phases: feature extraction, training, and recognition. In the feature extraction phase, a hybrid technique combines the spatial (edge) and the temporal (motion) information of each frame to extract the feature images. Then, in the training phase, we use the principal component analysis (PCA) to characterize spatial shape variations and the hidden Markov models (HMM) to describe the temporal shape variations. A modified Hausdorff distance measurement is also applied to measure the similarity between the feature images and the pre-stored PCA models. The similarity measures are referred to as the possible observations for each frame. Finally, in recognition phase, with the pre-trained PCA models and HMM, we can generate the observation patterns from the input sequences, and then apply the Viterbi algorithm to identify the gesture. In the experiments, we prove that our method can recognize 18 different continuous gestures effectively. Received: 19 May 1999 / Accepted: 4 September 2000     

基于一种改进禁忌搜索算法优化离散隐马尔可夫模型

隐马尔可夫模型(HMM,HiddenMarkovModel)是语音识别和手势识别中广泛使用的统计模式识别方法。文章提出了一种改进的禁忌搜索(ITS,ImprovedTabuSearch)优化HMM的参数。传统的TabuSearch(TS)与局部搜索算法(极大似然法)交替进行,从而加快了算法的收敛速度,并得到优化解。分别用TS及ITS训练隐马尔可夫模型进行动态手势识别。结果表明ITS可获得更高的识别率,且能达到全局优化。     

Simulating a Smartboard by Real-Time Gesture Detection in Lecture Videos

Gesture plays an important role for recognizing lecture activities in video content analysis. In this paper, we propose a real-time gesture detection algorithm by integrating cues from visual, speech and electronic slides. In contrast to the conventional “complete gesture” recognition, we emphasize detection by the prediction from “incomplete gesture”. Specifically, intentional gestures are predicted by the modified hidden Markov model (HMM) which can recognize incomplete gestures before the whole gesture paths are observed. The multimodal correspondence between speech and gesture is exploited to increase the accuracy and responsiveness of gesture detection. In lecture presentation, this algorithm enables the on-the-fly editing of lecture slides by simulating appropriate camera motion to highlight the intention and flow of lecturing. We develop a real-time application, namely simulated smartboard, and demonstrate the feasibility of our prediction algorithm using hand gesture and laser pen with simple setup without involving expensive hardware.      

Fuzzy Qualitative Human Motion Analysis

This paper proposes a fuzzy qualitative approach to vision-based human motion analysis with an emphasis on human motion recognition. It achieves feasible computational cost for human motion recognition by combining fuzzy qualitative robot kinematics with human motion tracking and recognition algorithms. First, a data-quantization process is proposed to relax the computational complexity suffered from visual tracking algorithms. Second, a novel human motion representation, i.e., qualitative normalized template, is developed in terms of the fuzzy qualitative robot kinematics framework to effectively represent human motion. The human skeleton is modeled as a complex kinematic chain, and its motion is represented by a series of such models in terms of time. Finally, experiment results are provided to demonstrate the effectiveness of the proposed method. An empirical comparison with conventional hidden Markov model (HMM) and fuzzy HMM (FHMM) shows that the proposed approach consistently outperforms both HMMs in human motion recognition.      

Hybrid continuous speech recognition systems by HMM,MLP and SVM: a comparative study

This paper presents a new hybrid method for continuous Arabic speech recognition based on triphones modelling. To do this, we apply Support Vectors Machine (SVM) as an estimator of posterior probabilities within the Hidden Markov Models (HMM) standards. In this work, we describe a new approach of categorising Arabic vowels to long and short vowels to be applied on the labeling phase of speech signals. Using this new labeling method, we deduce that SVM/HMM hybrid model is more efficient then HMMs standards and the hybrid system Multi-Layer Perceptron (MLP) with HMM. The obtained results for the Arabic speech recognition system based on triphones are 64.68 % with HMMs, 72.39 % with MLP/HMM and 74.01 % for SVM/HMM hybrid model. The WER obtained for the recognition of continuous speech by the three systems proves the performance of SVM/HMM by obtaining the lowest average for 4 tested speakers 11.42 %.     

基于手势识别的机器人人机交互技术研究

研究了基于视觉的动态手势识别技术,采用基于肤色的高斯模型与改进的光流场跟踪算法结合的方 法,实现了复杂背景下实时的手势跟踪,具有快速和准确的特点,且具有较好的鲁棒性．对于动态手势识别器,采 用了隐马尔可夫模型（HMM）作为训练识别算法．考虑到动态手势特征本身的一些特点,对HMM 参数优化算法重 估式加以修正,调整了算法比例因子,从而推导了最佳状态链的确定算法、HMM 参数优化算法．最后将研究开发 的动态手势识别算法成功地应用到了基于网络的远程机器人控制系统中．     

基于Leap Motion的动态手势识别方法

伴随虚拟现实（Virtual Reality,VR）技术的发展,以及人们对人机交互性能和体验感的要求提高,手势识别作为影响虚拟现实中交互操作的重要技术之一,其精确度急需提升^[1].针对当前手势识别方法在一些动作类似的手势识别中表现欠佳的问题,提出了一种多特征动态手势识别方法.该方法首先使用体感控制器Leap Motion追踪动态手势获取数据,然后在特征提取过程中增加对位移向量角度和拐点判定计数的提取,接着进行动态手势隐马尔科夫模型（Hidden Markov Model,HMM）的训练,最后根据待测手势与模型的匹配率进行识别.从实验结果中得出,该多特征识别方法能够提升相似手势的识别率.     

Customer credit scoring based on HMM/GMDH hybrid model

Hidden Markov model (HMM) has made great achievements in many fields such as speech recognition and engineering. However, due to its assumption of state conditional independence between observations, HMM has a very limited capacity for recognizing complex patterns involving more than first-order dependencies in customer relationships management. Group Method of Data Handling (GMDH) could overcome the drawbacks of HMM, so we propose a hybrid model by combining the HMM and GMDH to score customer credit. There are three phases in this model: training HMM with multiple observations, adding GMDH into HMM and optimizing the hybrid model. The proposed hybrid model is compared with other exiting methods in terms of average accuracy, Type I error, Type II error and AUC. Experimental results show that the proposed method has better performance than HMM/ANN in two credit scoring datasets. The implementation of HMM/GMDH hybrid model allows lenders and regulators to develop techniques to measure customer credit risk.     

Real-time 3D pointing gesture recognition for mobile robots with cascade HMM and particle filter

In this paper, we present a real-time 3D pointing gesture recognition algorithm for mobile robots, based on a cascade hidden Markov model (HMM) and a particle filter. Among the various human gestures, the pointing gesture is very useful to human-robot interaction (HRI). In fact, it is highly intuitive, does not involve a-priori assumptions, and has no substitute in other modes of interaction. A major issue in pointing gesture recognition is the difficultly of accurate estimation of the pointing direction, caused by the difficulty of hand tracking and the unreliability of the direction estimation.The proposed method involves the use of a stereo camera and 3D particle filters for reliable hand tracking, and a cascade of two HMMs for a robust estimate of the pointing direction. When a subject enters the field of view of the camera, his or her face and two hands are located and tracked using particle filters. The first stage HMM takes the hand position estimate and maps it to a more accurate position by modeling the kinematic characteristics of finger pointing. The resulting 3D coordinates are used as input into the second stage HMM that discriminates pointing gestures from other types. Finally, the pointing direction is estimated for the pointing state.The proposed method can deal with both large and small pointing gestures. The experimental results show gesture recognition and target selection rates of better than 89% and 99% respectively, during human-robot interaction.     

Principal motion components for one-shot gesture recognition

This paper introduces principal motion components (PMC), a new method for one-shot gesture recognition. In the considered scenario a single training video is available for each gesture to be recognized, which limits the application of traditional techniques (e.g., HMMs). In PMC, a 2D map of motion energy is obtained per each pair of consecutive frames in a video. Motion maps associated to a video are processed to obtain a PCA model, which is used for recognition under a reconstruction-error approach. The main benefits of the proposed approach are its simplicity, easiness of implementation, competitive performance and efficiency. We report experimental results in one-shot gesture recognition using the ChaLearn Gesture Dataset; a benchmark comprising more than 50,000 gestures, recorded as both RGB and depth video with a Kinect?camera. Results obtained with PMC are competitive with alternative methods proposed for the same data set.     

基于Kinect的动态手势识别

针对现有的动态手势识别率低,识别手势少等不足,利用Kinect设备提出了动态手势识别算法.首先利用Kinect捕获人的手部区域,采用基于像素分类的指尖检测算法找到指尖的个数,并以左右手的手指个数作为动态手势的开始和结束;对人手的运动轨迹进行分析,针对运动轨迹的运动方向的变化,提取了该动态手势的运动方向变化角度作为特征;采用隐马尔科夫模型训练和识别各个手势.实验结果表明:方法能够识别16个大写手写英文字母,且效果较好.     

ANN／HMM混合模型在语音识别中的应用

结合HMM较强的处理时间序列的能力以及ANN的学习能力强、识别速度快等特点提出了一种ANN／HMM混合模型,该模型具有较强的处理时问序列的能力。本文主要介绍了该模型的结构以及模型的训练算法。在此基础上将其应用于语音识别的建模,并通过相应实验验证了该模型的可行性。