复杂背景下的手势分割与识别

目前在基于单目视觉的手势识别中,手势分割技术几乎都是基于简单的背景或者要求 手势者带有特殊颜色的手套,给人机交互增加了一定的限制.本文融合人手颜色信息和手势运 动信息,两次利用种子算法对复杂背景下的手势进行分割.根据分割出的手区域大大加速了运动 特征参数的提取,并结合手区域的形状特征,建立手势的时空表观模型.识别时,采用独立分布的 多状态高斯概率模型,进行时间规整.手势训练集和测试集的识别率分别为97.8%和95.6%.     

基于特征线条的手势识别

单目视觉的手势识别系统,通常把人手建模成一个像素或者一块,从整体上分析手势的运动参数并提取表观特征.从细微之处着手,融合颜色、运动和边缘等多种信息以提取能够反应人手结构特性的特征线条,并将特征线条分割成小的曲线段,跟踪这些曲线段的运动.采用平面模型对手势图像表观的变化建模,根据各个曲线段的运动,融合成手的整体运动分量.同时分析了图像坐标系的选取和对运动参数的影响,提出了随手运动的坐标系,以抽取平移不变的平面模型参数,进行手势识别.     

基于肤色检测与卷积神经网络的手势识别

针对光照变化、背景噪声等复杂环境对手势识别的影响,提出了一种基于YCb Cr空间肤色分割去除背景结合卷积神经网络进行手势识别方法。首先根据人体肤色在YCb Cr颜色空间中的聚类效果,采用基于椭圆模型的肤色检测方法进行手势分割;然后对分割后的手势图像提取骨架与边缘相融合的手势特征图;再通过深层次的Alex Net卷积神经网络结构,对经过融合的手势特征图进行识别。实验结果表明,针对复杂的背景环境,该算法具有较强的鲁棒性,在不同数据集下对手势的平均识别率提升了4%,可以达到99.93%。     

基于时空上下文的手势跟踪与识别

针对复杂背景下手势运动过程中出现的手势形态变化、遮挡、光照变化等问题,提出了一种基于时空上下文的手势跟踪与识别方法。使用机器学习方法离线训练手势样本分类器,实现对手势的检测和定位;利用时空上下文跟踪算法对动态手势进行跟踪,同时为了避免跟踪过程中出现的漂移、目标丢失等情况,使用手势检测算法对手势位置信息进行实时校准;根据手势运动轨迹对手势运动进行跟踪与识别。实验表明,提出的方法可以实现对手势运动快速、准确、连续识别,满足人机交互的要求。     

基于表观的动态孤立手势识别

给出一种基于表观的动态孤立手势识别技术.借助于图像运动的变阶参数模型和鲁棒回归分析,提出一种基于运动分割的图像运动估计方法.基于图像运动参数,构造了两种表观变化模型分别作为手势的表观特征,利用最大最小优化算法来创建手势参考模板,并利用基于模板的分类技术进行识别.对120个手势样本所做的大量实验表明,这种动态孤立手势识别技术具有识别率高、计算量小、算法稳定性好等优点.     

复杂环境中多信息融合的手势识别

针对复杂环境中的手势识别问题,提出了一种融合深度信息和红外信息的手势识别方法。首先利用Kinect摄像头的深度信息进行动态实时手势分割,然后融合红外图像复原手势区域。解决了实时手势分割和利用手势的空间分布特征进行手势识别时由于分割的手势区域有缺损或有人脸干扰时识别率低的问题。经实验验证,提出的方法不仅不受环境光线的影响,而且可以识别区分度较小的手势,对旋转、缩放、平移的手势识别也具有鲁棒性。对于区分度较大的手势,识别率高达100%。     

复杂背景下基于空间分布特征的手势识别算法

为实现复杂背景下的手势识别,根据手势图像的区域形状特征提出一种基于手势空间分布特征的手势识别算法.利用复杂背景下基于亮度高斯模型的手势分割算法分割出肤色区域,利用"搜索窗口"筛选当前肤色区域实现手势定位,并提取包括空间相对密度特征和指节相对间距特征的手势空间分布特征,最后综合手势的2个手势特征向量计算总的相似性来识别手势.通过引入随机采样机制提高识别速度,并引入搜索窗口机制实现肤色干扰时的手势识别.实验结果表明,在环境光照相对稳定的条件下,文中算法能够实现鲁棒的实时手势识别,且具有很好的旋转、平移、缩放不变性,对于差异较大的手势识别率高达98%.     

复杂背景下BP神经网络的手势识别方法

针对手势图像的肤色特点,结合肤色在RGB空间的阈值分割和YCbCr颜色空间上的聚簇特性,以及背景模型的应用,有效减少了背景中类肤色的干扰,完成了手部图像在复杂背景下的检测和分割;并采用图像的7个不变Hu矩描述子来表征不同二值化的手势轮廓;最后采用BP神经网络进行手势识别。实验结果表明该方法有较好的识别率和鲁棒性。     

复杂背景下基于傅立叶描述子的手势识别

人的手势是人们日常生活中最广泛使用的一种交流方式。由于在人机交互界面和虚拟现实环境中的应用,手势识别的研究受到了越来越广泛的关注。但是目前基于单目视觉的手势识别技术中,手势分割要求背景简单或者要求识别者戴着笨重的数据手套。而该文结合了运动信息和基于KL变换的肤色模型,在复杂背景下进行手势分割,与传统的基于RGB肤色模型的手势分割相比,在复杂背景环境下得到了很好的分割效果。在对分割的手势区域进行预处理后,该文使用了一种归一化的傅立叶描述子进行手势的特征提取,相比传统的傅立叶描述子更加准确,最后采用了传统的三层BP网络作为模式识别器,手势训练集和测试集的识别率分别达到了95．9％和95％。     

基于OpenCV的摄像头动态手势轨迹识别及其应用

利用OpenCV计算机视觉库在vs2008平台上设计了一个基于实时摄像头的集动态手势检测、动态手势跟踪、动态手势轨迹识别的应用.首先,该应用基于静止的背景更新,利用背景差分检测运动手势,再结合颜色直方图的粒子滤波进行动态手势跟踪,最后利用隐马尔可夫模型(HMM)进行运动轨迹识别.在运动检测部分结合了背景差分图与通过颜色直方图获得的反投影图,达到比较满意的实时运动检测效果;在运动手势跟踪部分,改进的颜色直方图的粒子跟踪能够在经过类肤色人脸的干扰后迅速地找回运动手势,基本达到了跟踪的要求,但是同时对于HMM识别轨迹时需要的运动轨迹序列采集造成了影响;在识别轨迹部分,HMM的训练达到了识别的要求,但是识别的效果主要取决于实时运动轨迹序列的采集工作与采集方法的优化.     

A Real-Time Approach to the Spotting, Representation, and Recognition of Hand Gestures for Human–Computer Interaction

Aiming at the use of hand gestures for human–computer interaction, this paper presents a real-time approach to the spotting, representation, and recognition of hand gestures from a video stream. The approach exploits multiple cues including skin color, hand motion, and shape. Skin color analysis and coarse image motion detection are joined to perform reliable hand gesture spotting. At a higher level, a compact spatiotemporal representation is proposed for modeling appearance changes in image sequences containing hand gestures. The representation is extracted by combining robust parameterized image motion regression and shape features of a segmented hand. For efficient recognition of gestures made at varying rates, a linear resampling technique for eliminating the temporal variation (time normalization) while maintaining the essential information of the original gesture representations is developed. The gesture is then classified according to a training set of gestures. In experiments with a library of 12 gestures, the recognition rate was over 90%. Through the development of a prototype gesture-controlled panoramic map browser, we demonstrate that a vocabulary of predefined hand gestures can be used to interact successfully with applications running on an off-the-shelf personal computer equipped with a home video camera.     

Recognition of dynamic hand gestures

This paper is concerned with the problem of recognition of dynamic hand gestures. We have considered gestures which are sequences of distinct hand poses. In these gestures hand poses can undergo motion and discrete changes. However, continuous deformations of the hand shapes are not permitted. We have developed a recognition engine which can reliably recognize these gestures despite individual variations. The engine also has the ability to detect start and end of gesture sequences in an automated fashion. The recognition strategy uses a combination of static shape recognition (performed using contour discriminant analysis), Kalman filter based hand tracking and a HMM based temporal characterization scheme. The system is fairly robust to background clutter and uses skin color for static shape recognition and tracking. A real time implementation on standard hardware is developed. Experimental results establish the effectiveness of the approach.     

基于傅立叶描述子和HMM的手势识别

针对家庭服务机器人平台中人机交互的问题,提出基于视觉的手势识别作为人与机器人交互的方式,研究利用傅立叶描述子对手势形状进行描述,并结合支持向量机和隐马尔可夫模型分别对静态手势和动态手势进行分类,实现了静态手势和动态手势的识别。该系统基于新型传感器Kinect,在图像分割阶段结合图像深度信息,可以有效的将手势区域提取出来,在一定范围内具有较强的鲁棒性,特征提取阶段基于傅立叶描述子,使手势识别具有旋转、缩放、平移不变性。针对七种常见静态手势和四种动态手势进行测试,平均识别率分别达到98.8%和96.7%,实验结果表明该系统具有较高的准确度。     

Hand gesture recognition based on dynamic Bayesian network framework

In this paper, we propose a new method for recognizing hand gestures in a continuous video stream using a dynamic Bayesian network or DBN model. The proposed method of DBN-based inference is preceded by steps of skin extraction and modelling, and motion tracking. Then we develop a gesture model for one- or two-hand gestures. They are used to define a cyclic gesture network for modeling continuous gesture stream. We have also developed a DP-based real-time decoding algorithm for continuous gesture recognition. In our experiments with 10 isolated gestures, we obtained a recognition rate upwards of 99.59% with cross validation. In the case of recognizing continuous stream of gestures, it recorded 84% with the precision of 80.77% for the spotted gestures. The proposed DBN-based hand gesture model and the design of a gesture network model are believed to have a strong potential for successful applications to other related problems such as sign language recognition although it is a bit more complicated requiring analysis of hand shapes.     

基于视觉的多特征手势识别

手势是一种自然直观的交互方式,基于视觉的手势识别是实现新一代人机交互的关键技术。本文在已有的手势识别技术基础上,从手势分割及手势表示两方面着手,提出了一种单目视觉下的手势识别方法。利用颜色特征检测肤色区域,成功分割出人手;利用人手的轮廓及凸缺陷检测指尖,再利用指尖的数目和方位来表示一个手势,进而结合轮廓长度和面积等几何特征完成手势识别。传统的指尖检测方法需要遍历并扫描手掌外轮廓,计算量大,本文通过凸缺陷检测指尖,减少了计算量,提高了指尖检测的速度。实验结果表明,本文的方法具有很好的鲁棒性及实时性,能适应环境的变化。     

Recognizing expressions from face and body gesture by temporal normalized motion and appearance features

Recently, recognizing affects from both face and body gestures attracts more attentions. However, it still lacks of efficient and effective features to describe the dynamics of face and gestures for real-time automatic affect recognition. In this paper, we combine both local motion and appearance feature in a novel framework to model the temporal dynamics of face and body gesture. The proposed framework employs MHI-HOG and Image-HOG features through temporal normalization or bag of words to capture motion and appearance information. The MHI-HOG stands for Histogram of Oriented Gradients (HOG) on the Motion History Image (MHI). It captures motion direction and speed of a region of interest as an expression evolves over the time. The Image-HOG captures the appearance information of the corresponding region of interest. The temporal normalization method explicitly solves the time resolution issue in the video-based affect recognition. To implicitly model local temporal dynamics of an expression, we further propose a bag of words (BOW) based representation for both MHI-HOG and Image-HOG features. Experimental results demonstrate promising performance as compared with the state-of-the-art. Significant improvement of recognition accuracy is achieved as compared with the frame-based approach that does not consider the underlying temporal dynamics.