基于肤色检测与卷积神经网络的手势识别

针对光照变化、背景噪声等复杂环境对手势识别的影响,提出了一种基于YCb Cr空间肤色分割去除背景结合卷积神经网络进行手势识别方法。首先根据人体肤色在YCb Cr颜色空间中的聚类效果,采用基于椭圆模型的肤色检测方法进行手势分割;然后对分割后的手势图像提取骨架与边缘相融合的手势特征图;再通过深层次的Alex Net卷积神经网络结构,对经过融合的手势特征图进行识别。实验结果表明,针对复杂的背景环境,该算法具有较强的鲁棒性,在不同数据集下对手势的平均识别率提升了4%,可以达到99.93%。     

Hand gesture recognition using topological features

Multimedia Tools and Applications - Hand Gestures Recognition (HGR) is one of the main areas of research for Human Computer Interaction applications. Most existing approaches are based on local or...     

Hand gesture recognition based on dynamic Bayesian network framework

In this paper, we propose a new method for recognizing hand gestures in a continuous video stream using a dynamic Bayesian network or DBN model. The proposed method of DBN-based inference is preceded by steps of skin extraction and modelling, and motion tracking. Then we develop a gesture model for one- or two-hand gestures. They are used to define a cyclic gesture network for modeling continuous gesture stream. We have also developed a DP-based real-time decoding algorithm for continuous gesture recognition. In our experiments with 10 isolated gestures, we obtained a recognition rate upwards of 99.59% with cross validation. In the case of recognizing continuous stream of gestures, it recorded 84% with the precision of 80.77% for the spotted gestures. The proposed DBN-based hand gesture model and the design of a gesture network model are believed to have a strong potential for successful applications to other related problems such as sign language recognition although it is a bit more complicated requiring analysis of hand shapes.     

Static hand gesture recognition using neural networks

This paper presents a novel technique for hand gesture recognition through human–computer interaction based on shape analysis. The main objective of this effort is to explore the utility of a neural network-based approach to the recognition of the hand gestures. A unique multi-layer perception of neural network is built for classification by using back-propagation learning algorithm. The goal of static hand gesture recognition is to classify the given hand gesture data represented by some features into some predefined finite number of gesture classes. The proposed system presents a recognition algorithm to recognize a set of six specific static hand gestures, namely: Open, Close, Cut, Paste, Maximize, and Minimize. The hand gesture image is passed through three stages, preprocessing, feature extraction, and classification. In preprocessing stage some operations are applied to extract the hand gesture from its background and prepare the hand gesture image for the feature extraction stage. In the first method, the hand contour is used as a feature which treats scaling and translation of problems (in some cases). The complex moment algorithm is, however, used to describe the hand gesture and treat the rotation problem in addition to the scaling and translation. The algorithm used in a multi-layer neural network classifier which uses back-propagation learning algorithm. The results show that the first method has a performance of 70.83% recognition, while the second method, proposed in this article, has a better performance of 86.38% recognition rate.     

Hand gesture recognition using Leap Motion via deterministic learning

With the development of multimedia technology, traditional interactive tools, such as mouse and keyboard, cannot satisfy users’ requirements. Touchless interaction has received considerable attention in recent years with benefit of removing barriers of physical contact. Leap Motion is an interactive device which can be used to collect information of dynamic hand gestures, including coordinate, acceleration and direction of fingers. The aim of this study is to develop a new method for hand gesture recognition using jointly calibrated Leap Motion via deterministic learning. Hand gesture features representing hand motion dynamics, including spatial position and direction of fingers, are derived from Leap Motion. Hand motion dynamics underlying motion patterns of different gestures which represent Arabic numbers (0-9) and capital English alphabets (A-Z) are modeled by constant radial basis function (RBF) neural networks. Then, a bank of estimators is constructed by the constant RBF networks. By comparing the set of estimators with a test gesture pattern, a set of recognition errors are generated. The average L₁ norms of the errors are taken as the recognition measure according to the smallest error principle. Finally, experiments are carried out to demonstrate the high recognition performance of the proposed method. By using the 2-fold, 10-fold and leave-one-person-out cross-validation styles, the correct recognition rates for the Arabic numbers are reported to be 94.2%, 95.1% and 90.2%, respectively, for the English alphabets are reported to be 89.2%, 92.9% and 86.4%, respectively.     

基于改进RCE和RBF神经网络的静态手势识别

针对手势识别的手区域分割、手势特征提取和手势分类的三个过程,提出了一种新的静态手势识别方法。改进了传统的RCE神经网络用于手区域的分割,具有更高的运行速度和更强的抗噪能力。依Freeman链码方向提取手的边缘到掌心的距离作为手势的特征向量。将上一步得到的手势特征向量作为RBF神经网络的输入,进行网络的训练和分类。实验验证了该方法的有效性和可行性,并用其实现了人和仿人机器人的剪刀石头布的猜拳游戏。     

Hand posture and gesture recognition technology

Hand gestures that are performed by one or two hands can be categorized according to their applications into different categories including conversational, controlling, manipulative and communicative gestures. Generally, hand gesture recognition aims to identify specific human gestures and use them to convey information. The process of hand gesture recognition composes mainly of four stages: hand gesture images collection, gesture image preprocessing using some techniques including edge detection, filtering and normalization, capture the main characteristics of the gesture images and the evaluation (or classification) stage where the image is classified to its corresponding gesture class. There are many methods that have been used in the classification stage of hand gesture recognition such as Artificial Neural Networks, template matching, Hidden Markov Models and Dynamic Time Warping. This exploratory survey aims to provide a progress report on hand posture and gesture recognition technology.     

复杂背景下的手势识别

目的 手势识别是人机交互领域的热点问题。针对传统手势识别方法在复杂背景下识别率低,以及现有基于深度学习的手势识别方法检测时间长等问题,提出了一种基于改进TinyYOLOv3算法的手势识别方法。方法 对TinyYOLOv3主干网络重新进行设计,增加网络层数,从而确保网络提取到更丰富的语义信息。使用深度可分离卷积代替传统卷积,并对不同网络层的特征进行融合,在保证识别准确率的同时,减小网络模型的大小。采用CIoU（complete intersection over union）损失对原始的边界框坐标预测损失进行改进,将通道注意力模块融合到特征提取网络中,提高了定位精度和识别准确率。使用数据增强方法避免训练过拟合,并通过超参数优化和先验框聚类等方法加快网络收敛速度。结果 改进后的网络识别准确率达到99.1%,网络模型大小为27.6 MB,相比原网络（TinyYOLOv3）准确率提升了8.5%,网络模型降低了5.6 MB,相比于YOLO（you only look once）v3和SSD（single shot multibox detector）300算法,准确率略有降低,但网络模型分别减小到原来的1/8和1/3左右,相比于YOLO-lite和MobileNet-SSD等轻量级网络,准确率分别提升61.12%和3.11%。同时在自制的复杂背景下的手势数据集对改进后的网络模型进行验证,准确率达到97.3%,充分证明了本文算法的可行性。结论 本文提出的改进Tiny-YOLOv3手势识别方法,对于复杂背景下的手势具有较高的识别准确率,同时在检测速度和模型大小方面都优于其他算法,可以较好地满足在嵌入式设备中的使用要求。     

Hand gesture recognition in real world scenarios using approximate string matching

New interaction paradigms combined with emerging technologies have produced the creation of diverse Natural User Interface (NUI) devices in the market. These devices enable the recognition of body gestures allowing users to interact with applications in a more direct, expressive, and intuitive way. In particular, the Leap Motion Controller (LMC) device has been receiving plenty of attention from NUI application developers because it allows them to address limitations on gestures made with hands. Although this device is able to recognize the position of several parts of the hands, developers are still left with the difficult task of recognizing gestures. For this reason, several authors approached this problem using machine learning techniques. We propose a classifier based on Approximate String Matching (ASM). In short, we encode the trajectories of the hand joints as character sequences using the K-means algorithm and then we analyze these sequences with ASM. It should be noted that, when using the K-means algorithm, we select the number of clusters for each part of the hands by considering the Silhouette Coefficient. Furthermore, we define other important factors to take into account for improving the recognition accuracy. For the experiments, we generated a balanced dataset including different types of gestures and afterwards we performed a cross-validation scheme. Experimental results showed the robustness of the approach in terms of recognizing different types of gestures, time spent, and allocated memory. Besides, our approach achieved higher performance rates than well-known algorithms proposed in the current state-of-art for gesture recognition.

     

基于运动历史图像和椭圆拟合的手势分割

针对当前常用的手势分割方法较难适应复杂的光照环境,提出了一种新的方法。该方法先从视频序列获取运动历史图像（Motion History Image,MHI）,对MHI进行运动区域分割,然后在这些运动区域筛选出手势区域。为了克服手势区域分割偏大的问题,提出了利用该区域内的当前运动轮廓做椭圆拟合,进而得到精度更高的手势分割结果。实验结果表明,提出的方法能够有效地分割出手势,并且和传统方法相比较更能适应不同的光照环境。     

Hand number gesture recognition using recognized hand parts in depth images

In this paper, we present a novel approach of recognizing hand number gestures using the recognized hand parts in a depth image. Our proposed approach is divided into two stages: (i) hand parts recognition by random forests (RFs) and (ii) rule-based hand number gestures recognition. In the first stage, we create a database (DB) of synthetic hand depth silhouettes and their corresponding hand parts-labeled maps and then train RFs with the DB. Via the trained RFs, we recognize or label the hand parts in a depth silhouette. In the second stage, based on the information of the recognized or labeled hand parts, hand number gestures are recognized according to our derived rules. In our experiments, we quantitatively and qualitatively evaluated our hand parts recognition system with synthetic and real data. Then, we tested our hand number gesture recognition system with real data. Our results show the average recognition rate of 97.80 % over the ten hand number gestures from five different subjects.     

Convolutional neural network with spatial pyramid pooling for hand gesture recognition

Neural Computing and Applications - Hand gesture provides a means for human to interact through a series of gestures. While hand gesture plays a significant role in human–computer...     

双通道卷积神经网络在静态手势识别中的应用

针对静态手势识别任务中,传统基于人工提取特征方法耗时耗力,识别率较低,现有卷积神经网络依赖单一卷积核提取特征不够充分的问题,提出双通道卷积神经网络模型。输入手势图片通过两个相互独立的通道进行特征提取,双通道具有尺度不同的卷积核,能够提取输入图像中不同尺度的特征,然后在全连接层进行特征融合,最后经过softmax分类器进行分类。在Thomas Moeslund和Jochen Triesch手势数据库上进行实验验证,结果表明该模型提高了静态手势识别的准确率,增强了卷积神经网络的泛化能力。     

基于卷积神经网络的表面肌电信号手势识别

运用卷积神经网络原理,实现一维多通道的表面肌电信号的手势识别,避免了复杂的前期表面信号的预处理,以及手工特征提取阶段。文中分别采集右手的握拳、向左、向右和展拳4种手势的表面肌电信号。然后将采集的四种不同手势的肌电信号进行切割与标记,生成不同信号长度的八通道信号的训练集与测试集,运用卷积神经网络的原理,分别对其进行卷积、下采样。经过试验研究发现,运用卷积神经网络处理一维多通道表面肌电信号,从而实现手势识别的算法是可行的,并且能够得到较高的识别率。     

一种基于深度图像的静态手势神经网络识别方法

静态手势识别是以手势驱动的人机交互系统的核心技术。针对静态手势识别问题,提出了一种基于深度图像进行静态手势识别的方法。为了消除静态手势识别过程中的平移、旋转和缩放不变性,提取手势轮廓的Hu不变矩,并以Hu不变矩作为特征构建静态手势深度感知神经网络模型,以此实现对静态手势进行分类识别。在VisualStudio的开发环境下实现了对该方法的验证,取得了良好的效果,并与传统的模板匹配法与基于卷积神经网络的深度学习方法作比较,静态手势识别准确率总体可达95%,识别效率高,能满足实时性要求。     

基于卷积神经网络的手势识别算法设计与实现

为了克服传统手势识别方法复杂的人工提取特征值操作,引入卷积神经网络进行手势识别,该算法可以直接对原始图像进行处理,具有局部感知域、权值共享和池化等特点,可以有效提取图像特征。使用Marcel手势识别数据集对框架进行训练,采用交叉验证的方法对系统进行评估,实验结果表明该方法可以识别经过训练的手势,且精确度高,鲁棒性强。     

基于Bag of Features的手势识别

基于视觉的手势识别中,手势的识别效果易受手势旋转,光照亮度的影响,针对该问题,借鉴了目标识别和图像检索领域的Bag of Features(特征袋)算法,将Bag of Features算法应用到手势识别领域.通过SURF(加速鲁棒性特征)算法提取手势图像的特征描述符,使手势对尺度、旋转、光照具有很强的适应力,再应用Bag of Features算法把SURF特征描述符映射到一个统一维度的向量,即Bag of Features特征向量,再用支持向量机对图像得到的特征向量进行训练分类.实验结果表示,该方法不仅具有较高的时间效率,满足手势识别的实时性,而且即使在很大角度的旋转以及亮度的变化下,仍能达到较高的识别率.