基于Petri网和BPNN的多重触控手势识别

为解决多重触控技术的手势识别问题,提出一个多重触控手势描述与识别框架,给出其描述和识别方法。多重触控手势可分为原子手势和组合手势,在手势描述过程中,利用BP网络对原子手势进行建模,然后在将用户的意图映射为原子手势逻辑、时序和空间关系关联而成的组合手势,并在Petri网引入逻辑、时序和空间关系描述符对组合手势进行描述。在手势识别过程中,根据BP网络分类器检测出原子手势,并触发组合手势Petri网模型的转移,实现组合手势的识别。实验结果表明该方法对不同用户操作习惯有鲁棒性,能有效解决多重触控手势识别问题。     

多点触控的沙画手势识别

针对多点触控手势间接指令问题,提出了基于多点触控的沙画手势识别系统,该识别系统由时间、空间、形状信息控制。提出一种手势图形建模方法,测量手势的笔划之间的空间和时间关系。采用聚类算法标记手势图形中笔划的形状信息作为局部形状特征;利用基准方法HBF49特征提取全局形状特征。通过一组有10种不同多点触控的沙画手势的数据集评估基于多点触控的沙画手势识别系统,使用图嵌入方法和SVM分类进行手势识别,识别的准确率达到94.75%。实验结果证明,此研究对完成基于多点触控的沙画虚拟系统有重要作用。     

多点触摸手势分析及识别算法的研究

针对目前多点触控系统缺乏对触摸手势含义的理想描述和解析,提出了一种通用的多点触摸手势分析与设计框架,研究了高性能的算法合理解析并优化多点触控指令,使其更符合用户的思维与认知。设置触点位移和时间函数双阈值来提高触点识别的精确性,防止突增杂点的误判断,减缓过快操作产生的抖动;采用RAF神经网络模型解决动态手势识别的问题,并引入基于欧氏距离函数的聚类统计量作为网络的特征参数,大大提高了多点触摸手势识别的效率和精确度。     

多点触控手势识别算法的研究

本文针对当下流行的多点触控设备,对多点触控手势识别算法进行了研究,通过类比计算机中的快捷键设计出了一系列的触控手势,并结合曲线识别算法提出了一种多点触控手势识别算法。实验在当下流行的Android设备上进行,手势检测成功表明了识别算法的有效性。     

基于Leap Motion的动态手势识别

动态手势识别作为人机交互的一个重要方向，在各个领域具有广泛的需求。相较于静态手势，动态手势的变化更为复杂，对其特征的充分提取与描述是准确识别动态手势的关键。为了解决对动态手势特征描述不充分的问题，利用高精度的Leap Motion传感器对手部三维坐标信息进行采集，提出了一种包含手指姿势和手掌位移的特征在内的、能够充分描述复杂动态手势的特征序列，并结合长短期记忆网络模型进行动态手势识别。实验结果表明，提出的方法在包含16种动态手势的数据集上的识别准确率为98.50%；与其他特征序列的对比实验表明，提出的特征序列，能更充分准确地描述动态手势特征。     

融合关节旋转特征和指尖距离特征的手势识别

作为人机交互的重要方式,手势交互和识别由于其具有的高自由度而成为计算机图形学、虚拟现实与人机交互等领域的研究热点.传统直接提取手势轮廓或手部关节点位置信息的手势识别方法,其提取的特征通常难以准确表示手势之间的区别.针对手势识别中不同手势具有的高自由度以及由于手势图像分辨率低、背景杂乱、手被遮挡、手指形状尺寸不同、个体差异性导致手势特征表示不准确等问题,本文提出了一种新的融合关节旋转特征和指尖距离特征的手势特征表示与手势识别方法.首先从手势深度图中利用手部模板并将手部看成链段结构提取手部20个关节点的3D位置信息;然后利用手部关节点位置信息提取四元数关节旋转特征和指尖距离特征,该表示构成了手势特征的内在表示;最后利用一对一支持向量机对手势进行有效识别分类.本文不仅提出了一种新的手势特征表示与提取方法,该表示融合了关节旋转信息和指尖距离特征;而且从理论上证明了该特征表示能唯一地表征手势关节点的位置信息;同时提出了基于一对一SVM多分类策略进行手势分类与识别.对ASTAR静态手势深度图数据集中8类中国数字手势和21类美国字母手势数据集分别进行了实验验证,其分类识别准确率分别为99.71%和85.24%.实验结果表明,本文提出的基于关节旋转特征和指尖距离特征的融合特征能很好地表示不同手势的几何特征,能准确地表征静态手势并进行手势识别.     

结合CSS与傅里叶描述子的手势特征提取

目前常用的基于视觉的静态手势特征提取方法只从单一方面进行描述,缺乏全局信息和局部信息的有效结合。为此,提出一种结合CSS形状描述子与傅里叶描述子的手势特征提取方法。将CSS形状描述子与傅里叶描述子相结合,以此作为一种融合手势局部特征和全局特征的新的静态手势特征。实验结果表明,与传统方法相比,该方法的正确率更高,达到98.3%。     

基于加锁机制的静态手势识别方法

基于RGB-D（RGB-Depth）的静态手势识别的速度高于其动态手势识别,但是存在冗余手势和重复手势而导致识别准确性不高的问题。针对该问题,提出了一种基于加锁机制的静态手势识别方法来识别运动中的手势。首先,将通过Kinect设备获取RGB数据流和Depth数据流融合成人体骨骼数据流;然后,在静态手势方法中引入加锁机制,并与之前建立好的骨骼点特征模型手势库进行比对计算;最后,设计一款“程序员进阶之路”益智类网页游戏进行应用与实验。实验验证在6种不同运动手势情况下,该方法与纯静态手势识别方法相比,平均识别准确率提高了14.4%;与动态手势识别相比,识别速度提高了14%。实验结果表明,提出的基于加锁机制的静态手势识别方法,既保留了静态识别的速率,实现了实时识别;又能很好地剔除冗余手势和重复手势,提高了识别正确性。     

Simultaneous gesture segmentation and recognition based on forward spotting accumulative HMMs

Existing gesture segmentations use the backward spotting scheme that first detects the end point, then traces back to the start point and sends the extracted gesture segment to the hidden Markov model (HMM) for gesture recognition. This makes an inevitable time delay between the gesture segmentation and recognition and is not appropriate for continuous gesture recognition. To solve this problem, we propose a forward spotting scheme that executes gesture segmentation and recognition simultaneously. The start and end points of gestures are determined by zero crossing from negative to positive (or from positive to negative) of a competitive differential observation probability that is defined by the difference of observation probability between the maximal gesture and the non-gesture. We also propose the sliding window and accumulative HMMs. The former is used to alleviate the effect of incomplete feature extraction on the observation probability and the latter improves the gesture recognition rate greatly by accepting all accumulated gesture segments between the start and end points and deciding the gesture type by a majority vote of all intermediate recognition results. We use the predetermined association mapping to determine the 3D articulation data, which reduces the feature extraction time greatly. We apply the proposed simultaneous gesture segmentation and recognition method to recognize the upper-body gestures for controlling the curtains and lights in a smart home environment. Experimental results show that the proposed method has a good recognition rate of 95.42% for continuously changing gestures.     

一种基于加速度特征提取的手势识别方法

随着智能手机等移动电子设备的发展,基于MEMS加速度传感器的手势识别成为移动设备人机交互的研究热点。由于准确率及实时性的限制,目前的手势识别方法仍不足以推向实用。针对这一问题,提出了一种简单有效的手势识别方法:在手势定义阶段根据语义及操作的相似性将10个手势分为4个类别,通过提取反映各类手势运动学特征的加速度特征量,利用决策树分类器对手势进行预分类,然后根据各类手势的加速度变化规律识别具体的手势;同时通过严格的特征量阈值,有效地去除了无意识的误动作。该方法在15位实验者中获得了95.2%的平均准确率,识别时间小于0.01 s,对基于MEMS加速度传感器的手势识别研究具有一定参考价值。     

基于Hu-GLCM的手势识别方法研究

针对手势识别过程中单一手势特征对手势描述的不足,提出了一种基于改进Hu矩和灰度共生矩阵GLCM的手势识别方法 Hu-GLCM。首先利用肤色模型对采集的图像分割出手势区域;其次采用数学形态学和多边形拟合的方法提取手势的单连通轮廓,利用改进Hu-GLCM算法提取手势的几何形状特征和纹理特征并建立模板数据库;最后通过扩展的Canberra距离对手势图像进行识别和分类。实验结果表明,该改进算法对7种手势的平均识别率达到95%以上,且计算速度快,能够满足实时性的需求。     

A Real-Time Approach to the Spotting, Representation, and Recognition of Hand Gestures for Human–Computer Interaction

Aiming at the use of hand gestures for human–computer interaction, this paper presents a real-time approach to the spotting, representation, and recognition of hand gestures from a video stream. The approach exploits multiple cues including skin color, hand motion, and shape. Skin color analysis and coarse image motion detection are joined to perform reliable hand gesture spotting. At a higher level, a compact spatiotemporal representation is proposed for modeling appearance changes in image sequences containing hand gestures. The representation is extracted by combining robust parameterized image motion regression and shape features of a segmented hand. For efficient recognition of gestures made at varying rates, a linear resampling technique for eliminating the temporal variation (time normalization) while maintaining the essential information of the original gesture representations is developed. The gesture is then classified according to a training set of gestures. In experiments with a library of 12 gestures, the recognition rate was over 90%. Through the development of a prototype gesture-controlled panoramic map browser, we demonstrate that a vocabulary of predefined hand gestures can be used to interact successfully with applications running on an off-the-shelf personal computer equipped with a home video camera.     

User identification approach based on simple gestures

We present an intuitive, implicit, gesture based identification system suited for applications such as the user login to home multimedia services, with less strict security requirements. The term “implicit gesture” in this work refers to a natural physical hand manipulation of the control device performed by the user, who picks it up from its neutral motionless position or shakes it. For reference with other related systems, explicit and well defined identification gestures were used. Gestures were acquired by an accelerometer sensor equipped device in a form of the Nintendo WiiMote remote controller. A dynamic time warping method is used at the core of our gesture based identification system. To significantly increase the computational efficiency and temporal stability, the “super-gesture” concept was introduced, where acceleration features of multiple gestures are combined in only one super-gesture template per each user. User evaluation spanning over a period of 10 days and including 10 participants was conducted. User evaluation study results show that our algorithm ensures nearly 100 % recognition accuracy when using explicit identification signature gestures and between 88 % and 77 % recognition accuracy when the system needs to distinguish between 5 and 10 users, using the implicit “pick-up” gesture. Performance of the proposed system is comparable to the results of other related works when using explicit identification gestures, while showing that implicit gesture based identification is also possible and viable.     

Human–computer interaction using vision-based hand gesture recognition systems: a survey

Considerable effort has been put toward the development of intelligent and natural interfaces between users and computer systems. In line with this endeavor, several modes of information (e.g., visual, audio, and pen) that are used either individually or in combination have been proposed. The use of gestures to convey information is an important part of human communication. Hand gesture recognition is widely used in many applications, such as in computer games, machinery control (e.g., crane), and thorough mouse replacement. Computer recognition of hand gestures may provide a natural computer interface that allows people to point at or to rotate a computer-aided design model by rotating their hands. Hand gestures can be classified into two categories: static and dynamic. The use of hand gestures as a natural interface serves as a motivating force for research on gesture taxonomy, its representations, and recognition techniques. This paper summarizes the surveys carried out in human--computer interaction (HCI) studies and focuses on different application domains that use hand gestures for efficient interaction. This exploratory survey aims to provide a progress report on static and dynamic hand gesture recognition (i.e., gesture taxonomies, representations, and recognition techniques) in HCI and to identify future directions on this topic.     

Design and evaluation of finger-count interaction: Combining multitouch gestures and menus

Selecting commands on multi-touch displays is still a challenging problem. While a number of gestural vocabularies have been proposed, these are generally restricted to one or two fingers or can be difficult to learn. We introduce Finger-Count gestures, a coherent set of multi-finger and two-handed gestures. Finger-Count gestures are simple, robust, expressive and fast to perform. In order to make these gestures self-revealing and easy to learn, we propose the Finger-Count menu, a menu technique and teaching method for implicitly learning Finger-Count gestures. We discuss the properties, advantages and limitations of Finger-Count interaction from the gesture and menu technique perspectives as well as its integration into three applications. We present alternative designs to increase the number of commands and to enable multi-user scenarios. Following a study which shows that Finger-Count is as easy to learn as radial menus, we report the results of an evaluation investigating which gestures are easier to learn and which finger chords people prefer. Finally, we present Finger-Count for in-the-air gestures. Thereby, the same gesture set can be used from a distance as well as when touching the surface.     

MTBuilder:一个多触点交互桌面界面工具

针对基于WIMP范式的图形用户界面工具不能解决多触点交互桌面的多指手势识别、界面组件朝向等问题,提出基于自然用户界面通用隐喻OCGM(objects,containers,gestures and manipulations)的多触点交互桌面界面工具箱——MTBuilder.首先用层次化多触点数据表示模型存储多触点数据,然后对多指手势识别器进行动态管理以加速识别处理,最后基于OCGM设计并实现界面组件库.通过多人信息浏览、城市规划等原型系统的开发和实验评估可以看出,MTBuilder能够为交互桌面界面构造与快速原型系统开发提供强有力的支持.     

Vision-based arm gesture recognition for a long-range human–robot interaction

This paper proposes a vision-based human arm gesture recognition method for human–robot interaction, particularly at a long distance where speech information is not available. We define four meaningful arm gestures for a long-range interaction. The proposed method is capable of recognizing the defined gestures only with 320×240 pixel-sized low-resolution input images captured from a single camera at a long distance, approximately five meters from the camera. In addition, the system differentiates the target gestures from the users’ normal actions that occur in daily life without any constraints. For human detection at a long distance, the proposed approach combines results from mean-shift color tracking, short- and long-range face detection, and omega shape detection. The system then detects arm blocks using a background subtraction method with a background updating module and recognizes the target gestures based on information about the region, periodical motion, and shape of the arm blocks. From experiments using a large realistic database, a recognition rate of 97.235% is achieved, which is a sufficiently practical level for various pervasive and ubiquitous applications based on human gestures.     

Static hand gesture recognition using neural networks

This paper presents a novel technique for hand gesture recognition through human–computer interaction based on shape analysis. The main objective of this effort is to explore the utility of a neural network-based approach to the recognition of the hand gestures. A unique multi-layer perception of neural network is built for classification by using back-propagation learning algorithm. The goal of static hand gesture recognition is to classify the given hand gesture data represented by some features into some predefined finite number of gesture classes. The proposed system presents a recognition algorithm to recognize a set of six specific static hand gestures, namely: Open, Close, Cut, Paste, Maximize, and Minimize. The hand gesture image is passed through three stages, preprocessing, feature extraction, and classification. In preprocessing stage some operations are applied to extract the hand gesture from its background and prepare the hand gesture image for the feature extraction stage. In the first method, the hand contour is used as a feature which treats scaling and translation of problems (in some cases). The complex moment algorithm is, however, used to describe the hand gesture and treat the rotation problem in addition to the scaling and translation. The algorithm used in a multi-layer neural network classifier which uses back-propagation learning algorithm. The results show that the first method has a performance of 70.83% recognition, while the second method, proposed in this article, has a better performance of 86.38% recognition rate.