Real-time dynamic gesture recognition and hand servo tracking using PTZ camera

Multimedia Tools and Applications - A technology of real-time dynamic gesture recognition and hand tracking using a Pan-Tilt-Zoom (PTZ) camera was presented in this study. It was aimed to achieve...     

Human gesture recognition system for TV viewing using time-of-flight camera

We developed a new device-free user interface for TV viewing that uses a human gesture recognition technique. Although many motion recognition technologies have been reported, no man–machine interface that recognizes a large enough variety of gestures has been developed. The difficulty was the lack of spatial information that could be acquired from normal video sequences. We overcame the difficulty by using a time-of-flight camera and novel action recognition techniques. The main functions of this system are gesture recognition and posture measurement. The former is performed using the bag-of-features approach, which uses key-point trajectories as features. The use of 4-D spatiotemporal trajectory features is the main technical contribution of the proposed system. The latter is obtained through face detection and object tracking technology. The interface is useful because it does not require any contact-type devices. Several experiments proved the effectiveness of our proposed method and the usefulness of the system.     

Real-time robust vision-based hand gesture recognition using stereo images

This paper presents a real-time and robust approach to recognize two types of gestures consisting of seven motional gestures and six finger spelling gestures. This approach utilizes stereo images captured by a stereo webcam to achieve robust recognition under realistic lighting conditions and in various backgrounds. It incorporates several existing computationally efficient techniques and introduces a rule-based approach to merge the information from a pair of stereo images leading to an improved hand detection compared to using single images. The results obtained indicate that high recognition rates under realistic conditions are obtained in real-time on PC platforms at the rate of 30 frames per second. It is shown that its outcome is comparable to two existing approaches while it is computationally more efficient than these approaches.     

基于视觉的动态手势识别

采用隐马尔可夫模型（Hidden Markov Model）算法的缺点，采用纠错算法对其修正，提高了识别率。了对机器人控制的目的，优化了人机交互的接口。训练并识别手势样本，针对HMM的经典训练算法Baum-Welch将识别结果应用于“基于Internet远程机器人控制”项目，达到了对机器人控制的目的，优化了人机交互的接口。     

Grassmann manifold based dynamic hand gesture recognition using depth data

Multimedia Tools and Applications - In this paper, we propose a novel Grassmann manifold based framework for dynamic hand gesture recognition from depth data. Automated dynamic hand gesture...     

A novel approach for dynamic hand gesture recognition using contour-based similarity images

A novel approach is proposed for the recognition of moving hand gestures based on the representation of hand motions as contour-based similarity images (CBSIs). The CBSI was constructed by calculating the similarity between hand contours in different frames. The input CBSI was then matched with CBSIs in the database to recognize the hand gesture. The proposed continuous hand gesture recognition algorithm can simultaneously divide the continuous gestures into disjointed gestures and recognize them. No restrictive assumptions were considered for the motion of the hand between the disjointed gestures. The proposed algorithm was tested using hand gestures from American Sign Language and the results showed a recognition rate of 91.3% for disjointed gestures and 90.4% for continuous gestures. The experimental results illustrate the efficiency of the algorithm for noisy videos.     

Static hand gesture recognition using neural networks

This paper presents a novel technique for hand gesture recognition through human–computer interaction based on shape analysis. The main objective of this effort is to explore the utility of a neural network-based approach to the recognition of the hand gestures. A unique multi-layer perception of neural network is built for classification by using back-propagation learning algorithm. The goal of static hand gesture recognition is to classify the given hand gesture data represented by some features into some predefined finite number of gesture classes. The proposed system presents a recognition algorithm to recognize a set of six specific static hand gestures, namely: Open, Close, Cut, Paste, Maximize, and Minimize. The hand gesture image is passed through three stages, preprocessing, feature extraction, and classification. In preprocessing stage some operations are applied to extract the hand gesture from its background and prepare the hand gesture image for the feature extraction stage. In the first method, the hand contour is used as a feature which treats scaling and translation of problems (in some cases). The complex moment algorithm is, however, used to describe the hand gesture and treat the rotation problem in addition to the scaling and translation. The algorithm used in a multi-layer neural network classifier which uses back-propagation learning algorithm. The results show that the first method has a performance of 70.83% recognition, while the second method, proposed in this article, has a better performance of 86.38% recognition rate.     

结合手指检测和HOG特征的分层静态手势识别

目的 基于手势的交互方式在人机交互中发挥着越来越重要的作用,手势识别是大多数手势交互系统的核心技术.当手势种类较多时,目前已有的大多数手势识别方法往往无法获得足够高的识别率.为此,提出了一种结合手指检测和梯度方向直方图(HOG)特征的分层静态手势识别方法.方法 提出一种基于形态学操作的手指检测算法作为手势识别方法的基础.首先由肤色模型从输入图像中提取出手部区域,然后利用手指检测算法识别出手势包含的手指个数,并根据手指个数从事先训练好的支持向量机分类器集合中选取一个,最后提取手部区域的HOG特征,并利用选择好的分类器完成识别任务.结果 对25种常用手势进行了识别实验,将本文方法与单独使用HOG特征的方法进行对比.本文方法可以将传统HOG方法的识别率提高20%左右.结论 基于手指个数的分层识别策略可以有效地解决传统单层识别方法在手势种类较多时识别率不高的问题.在手部区域能被成功检测的情况下,提出的结合手指检测和HOG特征的方法可以取得较理想的手势识别结果,且能达到实时性要求.     

利用柔性神经树的实时肌电信号手势识别模型

传统的手势识别通常采用数字信号处理(Digital Signal Processing,DSP)芯片或者集合方法(Ensem-ble Methods)研究实时识别问题。这些方法易导致数学模型参数繁多、硬件连接复杂和实时识别率较低。提出一种基于表面肌电信号与柔性神经树(Flexible Neural Trees,FNT)模型的实时手势识别模型。表面肌电信号(surface Electromyography,sEMG)具有非入侵式、易于采集特点,故被广泛应用于行为识别和诊断等领域。柔性神经树模型通过简单的预定义来构建,能够解决人工神经网络(Artificial Neural Network,ANN)的结构依赖性高的问题。柔性神经树模型不仅能够避免复杂的计算和电路连接,还具有较高的实时识别率和较低的方均根误差(Root Mean Square Error,RMSE)。实验针对六名参与者的六种手势进行测试,结果表明该模型实时识别率较高,实际应用也证明该算法可行。     

DeReFNet: Dual-stream Dense Residual Fusion Network for static hand gesture recognition

Vision-based hand gesture recognition (HGR) system provides the most effective and natural way of interaction between humans and machines. However, the recognition performance of such an HGR system is challenging due to the variations in illumination, complex backgrounds, the shape of the user’s hand, and inter-class similarity. This work proposes a compact dual-stream dense residual fusion network (DeReFNet) to address the above challenges. The proposed convolutional neural network architecture mainly utilizes the strength of global features from each residual block of the residual stream and spatial information from the other stream using dense connectivity. Both the streams are fused to gather enriched information using the feature concatenation module. The efficacy of the DeReFNet is validated using a subject-independent cross-validation technique on four publicly available benchmark datasets. Furthermore, the qualitative and quantitative analysis of the benchmarked datasets illustrates that the DeReFNet outperforms state-of-the-art methods in terms of accuracy and computational time.     

Real-time fingertip tracking and gesture recognition

Augmented desk interfaces and other virtual reality systems depend on accurate, real-time hand and fingertip tracking for seamless integration between real objects and associated digital information. We introduce a method for discerning fingertip locations in image frames and measuring fingertip trajectories across image frames. We also propose a mechanism for combining direct manipulation and symbolic gestures based on multiple fingertip motions. Our method uses a filtering technique, in addition to detecting fingertips in each image frame, to predict fingertip locations in successive image frames and to examine the correspondences between the predicted locations and detected fingertips. This lets us obtain multiple complex fingertip trajectories in real time and improves fingertip tracking. This method can track multiple fingertips reliably even on a complex background under changing lighting conditions without invasive devices or color markers.     

Novel Haar features for real-time hand gesture recognition using SVM

Journal of Real-Time Image Processing - Due to the effect of lighting and complex background, most visual hand gesture recognition systems work only under restricted environments. Here, we propose...     

Real-time hand posture recognition using range data

A hand posture recognition system using 3D data is described. The system relies on a novel 3D sensor that generates a dense range image of the scene. The main advantage of the proposed system, compared to other gesture recognition techniques, is the capability for robust unconstrained recognition of complex hand postures such as those encountered in sign language alphabets. This is achieved by explicitly utilizing 3D hand geometry. Moreover, the proposed approach does not rely on color information, and guarantees robust segmentation of the hand under varying illumination conditions, and scene content. Several novel 3D image analysis algorithms are presented, covering the complete processing chain: 3D image acquisition, arm segmentation, hand–forearm segmentation, hand pose estimation, 3D feature extraction, and gesture classification. The proposed system is extensively evaluated.     

Markerless tracking and gesture recognition using polar correlation of camera optical flow

We present a novel, real-time, markerless vision-based tracking system, employing a rigid orthogonal configuration of two pairs of opposing cameras. Our system uses optical flow over sparse features to overcome the limitation of vision-based systems that require markers or a pre-loaded model of the physical environment. We show how opposing cameras enable cancellation of common components of optical flow leading to an efficient tracking algorithm that captures five degrees of freedom including direction of translation and angular velocity. Experiments comparing our device with an electromagnetic tracker show that its average tracking accuracy is 80 % over 185 frames, and it is able to track large range motions even in outdoor settings. We also present how our tracking system can be used for gesture recognition by combining it with a simple linear classifier over a set of 15 gestures. Experimental results show that we are able to achieve 86.7 % gesture recognition accuracy.     

Hand number gesture recognition using recognized hand parts in depth images

In this paper, we present a novel approach of recognizing hand number gestures using the recognized hand parts in a depth image. Our proposed approach is divided into two stages: (i) hand parts recognition by random forests (RFs) and (ii) rule-based hand number gestures recognition. In the first stage, we create a database (DB) of synthetic hand depth silhouettes and their corresponding hand parts-labeled maps and then train RFs with the DB. Via the trained RFs, we recognize or label the hand parts in a depth silhouette. In the second stage, based on the information of the recognized or labeled hand parts, hand number gestures are recognized according to our derived rules. In our experiments, we quantitatively and qualitatively evaluated our hand parts recognition system with synthetic and real data. Then, we tested our hand number gesture recognition system with real data. Our results show the average recognition rate of 97.80 % over the ten hand number gestures from five different subjects.