Recognizing hand gestures using the weighted elastic graph matching (WEGM) method

This paper proposes a weighted scheme for elastic graph matching hand posture recognition. Visual features scattered on the elastic graph are assigned corresponding weights according to their relative ability to discriminate between gestures. The weights' values are determined using adaptive boosting. A dictionary representing the variability of each gesture class is expressed in the form of a bunch graph. The positions of the nodes in the bunch graph are determined using three techniques: manually, semi-automatically, and automatically. Experimental results also show that the semi-automatic annotation method is efficient and accurate in terms of three performance measures; assignment cost, accuracy, and transformation error. In terms of the recognition accuracy, our results show that the hierarchical weighting on features has more significant discriminative power than the classic method (uniform weighting). The hierarchical elastic graph matching (WEGM) approach was used to classify a lexicon of ten hand postures, and it was found that the poses were recognized with a recognition accuracy of 97.08% on average. Using the weighted scheme, computing cycles can be decreased by only computing the features for those nodes whose weight is relatively high and ignoring the remaining nodes. It was found that only 30% of the nodes need to be computed to obtain a recognition accuracy of over 90%.     

Classification of hand postures against complex backgrounds using elastic graph matching

A system for person-independent classification of hand postures against complex backgrounds in video images is presented. The system employs elastic graph matching, which has already been successfully applied for object and face recognition. We use the bunch graph technique to model variance in hand posture appearance between different subjects and variance in backgrounds. Our system does not need a separate segmentation stage but closely integrates finding the object boundaries with posture classification.     

Estimation of the fundamental matrix from uncalibrated stereo hand images for 3D hand gesture recognition

All 3D hand models employed for hand gesture recognition so far use kinematic models of the hand. We propose to use computer vision models of the hand, and recover hand gestures using 3D reconstruction techniques. In this paper, we present a new method to estimate the epipolar geometry between two uncalibrated cameras from stereo hand images. We first segmented hand images using the RCE neural network based color segmentation algorithm and extracted edge points of fingers as points of interest, then match them based on the topological features of the hand. The fundamental matrix is estimated using a combination of techniques such as input data normalization, rank-2 constraint, linear criterion, nonlinear criterion as well as M-estimator. This method has been tested with real calibrated and uncalibrated images. The experimental comparison demonstrates the effectiveness and robustness of the method.     

Feature fusion for 3D hand gesture recognition by learning a shared hidden space

Hand gesture recognition has been intensively applied in various human-computer interaction (HCI) systems. Different hand gesture recognition methods were developed based on particular features, e.g., gesture trajectories and acceleration signals. However, it has been noticed that the limitation of either features can lead to flaws of a HCI system. In this paper, to overcome the limitations but combine the merits of both features, we propose a novel feature fusion approach for 3D hand gesture recognition. In our approach, gesture trajectories are represented by the intersection numbers with randomly generated line segments on their 2D principal planes, acceleration signals are represented by the coefficients of discrete cosine transformation (DCT). Then, a hidden space shared by the two features is learned by using penalized maximum likelihood estimation (MLE). An iterative algorithm, composed of two steps per iteration, is derived to for this penalized MLE, in which the first step is to solve a standard least square problem and the second step is to solve a Sylvester equation. We tested our hand gesture recognition approach on different hand gesture sets. Results confirm the effectiveness of the feature fusion method.     

DeReFNet: Dual-stream Dense Residual Fusion Network for static hand gesture recognition

Vision-based hand gesture recognition (HGR) system provides the most effective and natural way of interaction between humans and machines. However, the recognition performance of such an HGR system is challenging due to the variations in illumination, complex backgrounds, the shape of the user’s hand, and inter-class similarity. This work proposes a compact dual-stream dense residual fusion network (DeReFNet) to address the above challenges. The proposed convolutional neural network architecture mainly utilizes the strength of global features from each residual block of the residual stream and spatial information from the other stream using dense connectivity. Both the streams are fused to gather enriched information using the feature concatenation module. The efficacy of the DeReFNet is validated using a subject-independent cross-validation technique on four publicly available benchmark datasets. Furthermore, the qualitative and quantitative analysis of the benchmarked datasets illustrates that the DeReFNet outperforms state-of-the-art methods in terms of accuracy and computational time.     

基于视觉的动态手势识别

采用隐马尔可夫模型（Hidden Markov Model）算法的缺点，采用纠错算法对其修正，提高了识别率。了对机器人控制的目的，优化了人机交互的接口。训练并识别手势样本，针对HMM的经典训练算法Baum-Welch将识别结果应用于“基于Internet远程机器人控制”项目，达到了对机器人控制的目的，优化了人机交互的接口。     

Inexact graph matching for model-based recognition: Evaluation and comparison of optimization algorithms

A method for segmentation and recognition of image structures based on graph homomorphisms is presented in this paper. It is a model-based recognition method where the input image is over-segmented and the obtained regions are represented by an attributed relational graph (ARG). This graph is then matched against a model graph thus accomplishing the model-based recognition task. This type of problem calls for inexact graph matching through a homomorphism between the graphs since no bijective correspondence can be expected, because of the over-segmentation of the image with respect to the model. The search for the best homomorphism is carried out by optimizing an objective function based on similarities between object and relational attributes defined on the graphs. The following optimization procedures are compared and discussed: deterministic tree search, for which new algorithms are detailed, genetic algorithms and estimation of distribution algorithms. In order to assess the performance of these algorithms using real data, experimental results on supervised classification of facial features using face images from public databases are presented.     

Symbol recognition in electrical diagrams using probabilistic graph matching

In this paper a method is proposed to recognize symbols in electrical diagrams based on probabilistic matching. The skeletons of the symbols are represented by graphs. After finding the pose of the graph (orientation, translation, scale) by a bounded search for a minimum error transformation, the observed graph is matched to the class models and the likelihood of the match is calculated. Results are given for computer-generated symbols and hand drawn symbols with and without a template. Error rates range from <1% to 8%.     

Iconic and multi-stroke gesture recognition

Many handwritten gestures, characters, and symbols comprise multiple pendown strokes separated by penup strokes. In this paper, a large number of features known from the literature are explored for the recognition of such multi-stroke gestures. Features are computed from a global gesture shape. From its constituent strokes, the mean and standard deviation of each feature are computed. We show that using these new stroke-based features, significant improvements in classification accuracy can be obtained between 10% and 50% compared to global feature representations. These results are consistent over four different databases, containing iconic pen gestures, handwritten symbols, and upper-case characters. Compared to two other multi-stroke recognition techniques, improvements between 25% and 39% are achieved, averaged over all four databases.     

A structure-preserved local matching approach for face recognition

In this paper, a novel local matching method called structure-preserved projections (SPP) is proposed for face recognition. Unlike most existing local matching methods which neglect the interactions of different sub-pattern sets during feature extraction, i.e., they assume different sub-pattern sets are independent; SPP takes the holistic context of the face into account and can preserve the configural structure of each face image in subspace. Moreover, the intrinsic manifold structure of the sub-pattern sets can also be preserved in our method. With SPP, all sub-patterns partitioned from the original face images are trained to obtain a unified subspace, in which recognition can be performed. The efficiency of the proposed algorithm is demonstrated by extensive experiments on three standard face databases (Yale, Extended YaleB and PIE). Experimental results show that SPP outperforms other holistic and local matching methods.     

Eigen-deformations for elastic matching based handwritten character recognition

Deformations in handwritten characters have category-dependent tendencies. In this paper, the estimation and the utilization of such tendencies called eigen-deformations are investigated for the better performance of elastic matching based handwritten character recognition. The eigen-deformations are estimated by the principal component analysis of actual deformations automatically collected by the elastic matching. From experimental results it was shown that typical deformations of each category can be extracted as the eigen-deformations. It was also shown that the recognition performance can be improved significantly by using the eigen-deformations for the detection of overfitting, which is the main cause of the misrecognition in the elastic matching based recognition methods.     

The discriminant elastic graph matching algorithm applied to frontal face verification

In this paper a generalized framework for face verification is proposed employing discriminant techniques in all phases of elastic graph matching. The proposed algorithm is called discriminant elastic graph matching (DEGM) algorithm. In the first step of the proposed method, DEGM, discriminant techniques at the node feature vectors are used for feature selection. In the sequel, the two local similarity values, i.e., the similarity measure for the projected node feature vector and the node deformation, are combined in a discriminant manner in order to form the new local similarity measure. Moreover, the new local similarity values at the nodes of the elastic graph are weighted by coefficients that are derived as well from discriminant analysis in order to form a total similarity measure between faces. The proposed method exploits the individuality of the human face and the discriminant information of elastic graph matching in order to improve the verification performance of elastic graph matching. We have applied the proposed scheme to a modified morphological elastic graph matching algorithm. All experiments have been conducted in the XM2VTS database resulting in very low error rates for the test sets.     

A model-based hand gesture recognition system

This paper introduces a model-based hand gesture recognition system, which consists of three phases: feature extraction, training, and recognition. In the feature extraction phase, a hybrid technique combines the spatial (edge) and the temporal (motion) information of each frame to extract the feature images. Then, in the training phase, we use the principal component analysis (PCA) to characterize spatial shape variations and the hidden Markov models (HMM) to describe the temporal shape variations. A modified Hausdorff distance measurement is also applied to measure the similarity between the feature images and the pre-stored PCA models. The similarity measures are referred to as the possible observations for each frame. Finally, in recognition phase, with the pre-trained PCA models and HMM, we can generate the observation patterns from the input sequences, and then apply the Viterbi algorithm to identify the gesture. In the experiments, we prove that our method can recognize 18 different continuous gestures effectively. Received: 19 May 1999 / Accepted: 4 September 2000     

多点触控的沙画手势识别

针对多点触控手势间接指令问题,提出了基于多点触控的沙画手势识别系统,该识别系统由时间、空间、形状信息控制。提出一种手势图形建模方法,测量手势的笔划之间的空间和时间关系。采用聚类算法标记手势图形中笔划的形状信息作为局部形状特征;利用基准方法HBF49特征提取全局形状特征。通过一组有10种不同多点触控的沙画手势的数据集评估基于多点触控的沙画手势识别系统,使用图嵌入方法和SVM分类进行手势识别,识别的准确率达到94.75%。实验结果证明,此研究对完成基于多点触控的沙画虚拟系统有重要作用。     

基于Hu-GLCM的手势识别方法研究

针对手势识别过程中单一手势特征对手势描述的不足,提出了一种基于改进Hu矩和灰度共生矩阵GLCM的手势识别方法 Hu-GLCM。首先利用肤色模型对采集的图像分割出手势区域;其次采用数学形态学和多边形拟合的方法提取手势的单连通轮廓,利用改进Hu-GLCM算法提取手势的几何形状特征和纹理特征并建立模板数据库;最后通过扩展的Canberra距离对手势图像进行识别和分类。实验结果表明,该改进算法对7种手势的平均识别率达到95%以上,且计算速度快,能够满足实时性的需求。     

A graph isomorphism algorithm for object recognition

We present an algorithm to solve the graph isomorphism problem for the purpose of object recognition. Objects, such as those which exist in a robot workspace, may be represented by labelled graphs (graphs with attributes on their nodes and/or edges). Thereafter, object recognition is achieved by matching pairs of these graphs. Assuming that all objects are sufficiently different so that their corresponding representative graphs are distinct, then given a new graph, the algorthm efficiently finds the isomorphic stored graph (if it exists). The algorithm consists of three phases: preprocessing, link construction, and ambiguity resolution. Results from experiments on a wide variety and sizes of graphs are reported. Results are also reported for experiments on recognising graphs that represent protein molecules. The algorithm works for all types of graphs except for a class of highly ambiguous graphs which includes strongly regular graphs. However, members of this class are detected in polynomial time, which leaves the option of switching to a higher complexity algorithm if desired.     

基于深度信息的实时手势识别和虚拟书写系统

鉴于无接触体感交互技术在人机交互领域的成功应用,提出了一种基于Kinect深度相机的实时隔空虚拟书写方法。结合颜色和深度数据检测和分割出手掌区域;进一步,通过修改的圆扫描转换算法获得手指的个数,以识别不同的手势指令;根据指尖检测从指尖的运动轨迹分割出独立的字符或汉字运动轨迹,并采用随机森林算法识别该字符或汉字。这种基于深度信息的手势检测和虚拟书写方法可以克服光照和肤色重叠的影响,可靠实时地检测和识别手势和隔空书写的文字,其识别率达到93.25%,识别速度达到25 frame/s。     

基于多相机的人脸姿态识别

主动形状模型(ASM)算法被用来进行人脸特征点的精确定位,然后在多相机测量的图像中进行特征点的立体匹配,利用双目视觉和相机三维测距技术可以确定人脸特征点的空间三维位置,从而利用这些特征点的相对位置确定出人脸的姿态。实验结果显示,用该方法进行人脸姿态识别能取得比二维识别更高的精确度。