Recognizing hand gestures using the weighted elastic graph matching (WEGM) method

This paper proposes a weighted scheme for elastic graph matching hand posture recognition. Visual features scattered on the elastic graph are assigned corresponding weights according to their relative ability to discriminate between gestures. The weights' values are determined using adaptive boosting. A dictionary representing the variability of each gesture class is expressed in the form of a bunch graph. The positions of the nodes in the bunch graph are determined using three techniques: manually, semi-automatically, and automatically. Experimental results also show that the semi-automatic annotation method is efficient and accurate in terms of three performance measures; assignment cost, accuracy, and transformation error. In terms of the recognition accuracy, our results show that the hierarchical weighting on features has more significant discriminative power than the classic method (uniform weighting). The hierarchical elastic graph matching (WEGM) approach was used to classify a lexicon of ten hand postures, and it was found that the poses were recognized with a recognition accuracy of 97.08% on average. Using the weighted scheme, computing cycles can be decreased by only computing the features for those nodes whose weight is relatively high and ignoring the remaining nodes. It was found that only 30% of the nodes need to be computed to obtain a recognition accuracy of over 90%.     

Classification of hand postures against complex backgrounds using elastic graph matching

A system for person-independent classification of hand postures against complex backgrounds in video images is presented. The system employs elastic graph matching, which has already been successfully applied for object and face recognition. We use the bunch graph technique to model variance in hand posture appearance between different subjects and variance in backgrounds. Our system does not need a separate segmentation stage but closely integrates finding the object boundaries with posture classification.     

Sketch recognition by fusion of temporal and image-based features

The increasing availability of pen-based hardware has recently resulted in a parallel growth in sketch-based user interfaces. Sketch-based user interfaces aim to combine the expressive power of free-hand sketching with the processing power of computers. Most sketch-based systems require intelligent ink processing capabilities, which makes the development of robust sketch recognition algorithms a primary concern in the field. So far, the research in sketch recognition has produced various independent approaches to recognition, each of which uses a particular kind of information (e.g., geometric and spatial constraints, image-based features, temporal stroke-ordering patterns). These methods were designed in isolation as stand-alone algorithms, and there has been little work treating various recognition methods as alternative sources of information that can be combined to increase sketch recognition accuracy. In this paper, we focus on two such methods and fuse an image-based method with a time-based method in an attempt to combine the knowledge of how objects look (image data) with the knowledge of how they are drawn (temporal data). In the course of combining spatial and temporal information, we also introduce a mathematically well founded fusion method for combining recognizers. Our combination method can be used for isolated sketch recognition as well as full diagram recognition. Our evaluation with two databases shows that fusing image-based and temporal features yields higher recognition rates. These results are the first to confirm the complementary nature of image-based and temporal recognition methods for full sketch recognition, which has long been suggested, but never supported by data.     

Sensor Network Structure Recognition Based on P-law

A sensor graph network is a sensor network model organized according to graph network structure. Structural unit and signal propagation of core nodes are the basic characteristics of sensor graph networks. In sensor networks, network structure recognition is the basis for accurate identification and effective prediction and control of node states. Aiming at the problems of difficult global structure identification and poor interpretability in complex sensor graph networks, based on the characteristics of sensor networks, a method is proposed to firstly unitize the graph network structure and then expand the unit based on the signal transmission path of the core node. This method which builds on unit patulousness and core node signal propagation (called p-law) can rapidly and effectively achieve the global structure identification of a sensor graph network. Different from the traditional graph network structure recognition algorithms such as modularity maximization and spectral clustering, the proposed method reveals the natural evolution process and law of graph network subgroup generation. Experimental results confirm the effectiveness, accuracy and rationality of the proposed method and suggest that our method can be a new approach for graph network global structure recognition.     

A face recognition system based on automatically determined facial fiducial points

In this paper, a completely automatic face recognition system is presented. The method works on color images: after having localized the face and the facial features, it determines 24 facial fiducial points, and characterizes them applying a bank of Gabor filters which extract the peculiar texture around them (jets). Recognition is realized measuring the similarity between the different jets. The system is inspired by the elastic bunch graph method, while it does no assumption on the scale, pose, and the background. Comparison with standard algorithms is presented and discussed.     

Clique-perfectness and balancedness of some graph classes

A graph is clique-perfect if the maximum size of a clique-independent set (a set of pairwise disjoint maximal cliques) and the minimum size of a clique-transversal set (a set of vertices meeting every maximal clique) coincide for each induced subgraph. A graph is balanced if its clique-matrix contains no square submatrix of odd size with exactly two ones per row and column. In this work, we give linear-time recognition algorithms and minimal forbidden induced subgraph characterizations of clique-perfectness and balancedness of P₄-tidy graphs and a linear-time algorithm for computing a maximum clique-independent set and a minimum clique-transversal set for any P₄-tidy graph. We also give a minimal forbidden induced subgraph characterization and a linear-time recognition algorithm for balancedness of paw-free graphs. Finally, we show that clique-perfectness of diamond-free graphs can be decided in polynomial time by showing that a diamond-free graph is clique-perfect if and only if it is balanced.     

Graph-optimized locality preserving projections

Locality preserving projections (LPP) is a typical graph-based dimensionality reduction (DR) method, and has been successfully applied in many practical problems such as face recognition. However, LPP depends mainly on its underlying neighborhood graph whose construction suffers from the following issues: (1) such neighborhood graph is artificially defined in advance, and thus does not necessary benefit subsequent DR task; (2) such graph is constructed using the nearest neighbor criterion which tends to work poorly due to the high-dimensionality of original space; (3) it is generally uneasy to assign appropriate values for the neighborhood size and heat kernel parameter involved in graph construction. To address these problems, we develop a novel DR algorithm called Graph-optimized Locality Preserving Projections (GoLPP). The idea is to integrate graph construction with specific DR process into a unified framework, which results in an optimized graph rather than predefined one. Moreover, an entropy regularization term is incorporated into the objective function for controlling the uniformity level of the edge weights in graph, so that a principled graph updating formula naturally corresponding to conventional heat kernel weights can be obtained. Finally, the experiments on several publicly available UCI and face data sets show the feasibility and effectiveness of the proposed method with encouraging results.     

Central Clustering of Attributed Graphs

Partitioning a data set of attributed graphs into clusters arises in different application areas of structural pattern recognition and computer vision. Despite its importance, graph clustering is currently an underdeveloped research area in machine learning due to the lack of theoretical analysis and the high computational cost of measuring structural proximities. To address the first issue, we introduce the concept of metric graph spaces that enables central (or center-based) clustering algorithms to be applied to the domain of attributed graphs. The key idea is to embed attributed graphs into Euclidean space without loss of structural information. In addressing the second issue of computational complexity, we propose a neural network solution of the K-means algorithm for structures (KMS). As a distinguishing feature to improve the computational time, the proposed algorithm classifies the data graphs according to the principle of elimination of competition where the input graph is assigned to the winning model of the competition. In experiments we investigate the behavior and performance of the neural KMS algorithm.     

Commute time guided transformation for feature extraction

This paper presents a random-walk-based feature extraction method called commute time guided transformation (CTG) in the graph embedding framework. The paper contributes to the corresponding field in two aspects. First, it introduces the usage of a robust probability metric, i.e., the commute time (CT), to extract visual features for face recognition via a manifold way. Second, the paper designs the CTG optimization to find linear orthogonal projections that would implicitly preserve the commute time of high dimensional data in a low dimensional subspace. Compared with previous CT embedding algorithms, the proposed CTG is a graph-independent method. Existing CT embedding methods are graph-dependent that could only embed the data on the training graph in the subspace. Differently, CTG paradigm can be used to project the out-of-sample data into the same embedding space as the training graph. Moreover, CTG projections are robust to the graph topology that it can always achieve good recognition performance in spite of different initial graph structures. Owing to these positive properties, when applied to face recognition, the proposed CTG method outperforms other state-of-the-art algorithms on benchmark datasets. Specifically, it is much efficient and effective to recognize faces with noise.     

Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds

A system for the detection, segmentation and recognition of multi-class hand postures against complex natural backgrounds is presented. Visual attention, which is the cognitive process of selectively concentrating on a region of interest in the visual field, helps human to recognize objects in cluttered natural scenes. The proposed system utilizes a Bayesian model of visual attention to generate a saliency map, and to detect and identify the hand region. Feature based visual attention is implemented using a combination of high level (shape, texture) and low level (color) image features. The shape and texture features are extracted from a skin similarity map, using a computational model of the ventral stream of visual cortex. The skin similarity map, which represents the similarity of each pixel to the human skin color in HSI color space, enhanced the edges and shapes within the skin colored regions. The color features used are the discretized chrominance components in HSI, YCbCr color spaces, and the similarity to skin map. The hand postures are classified using the shape and texture features, with a support vector machines classifier. A new 10 class complex background hand posture dataset namely NUS hand posture dataset-II is developed for testing the proposed algorithm (40 subjects, different ethnicities, various hand sizes, 2750 hand postures and 2000 background images). The algorithm is tested for hand detection and hand posture recognition using 10 fold cross-validation. The experimental results show that the algorithm has a person independent performance, and is reliable against variations in hand sizes and complex backgrounds. The algorithm provided a recognition rate of 94.36 %. A comparison of the proposed algorithm with other existing methods evidences its better performance.     

基于傅立叶描述子和HMM的手势识别

针对家庭服务机器人平台中人机交互的问题,提出基于视觉的手势识别作为人与机器人交互的方式,研究利用傅立叶描述子对手势形状进行描述,并结合支持向量机和隐马尔可夫模型分别对静态手势和动态手势进行分类,实现了静态手势和动态手势的识别。该系统基于新型传感器Kinect,在图像分割阶段结合图像深度信息,可以有效的将手势区域提取出来,在一定范围内具有较强的鲁棒性,特征提取阶段基于傅立叶描述子,使手势识别具有旋转、缩放、平移不变性。针对七种常见静态手势和四种动态手势进行测试,平均识别率分别达到98.8%和96.7%,实验结果表明该系统具有较高的准确度。     

HEGM: A hierarchical elastic graph matching for hand gesture recognition

A hierarchical scheme for elastic graph matching applied to hand gesture recognition is proposed. The proposed algorithm exploits the relative discriminatory capabilities of visual features scattered on the images, assigning the corresponding weights to each feature. A boosting algorithm is used to determine the structure of the hierarchy of a given graph. The graph is expressed by annotating the nodes of interest over the target object to form a bunch graph. Three annotation techniques, manual, semi-automatic, and automatic annotation are used to determine the position of the nodes. The scheme and the annotation approaches are applied to explore the hand gesture recognition performance. A number of filter banks are applied to hand gestures images to investigate the effect of using different feature representation approaches. Experimental results show that the hierarchical elastic graph matching (HEGM) approach classified the hand posture with a gesture recognition accuracy of 99.85% when visual features were extracted by utilizing the Histogram of Oriented Gradient (HOG) representation. The results also provide the performance measures from the aspect of recognition accuracy to matching benefits, node positions correlation and consistency on three annotation approaches, showing that the semi-automatic annotation method is more efficient and accurate than the other two methods.     

Inexact graph matching for model-based recognition: Evaluation and comparison of optimization algorithms

A method for segmentation and recognition of image structures based on graph homomorphisms is presented in this paper. It is a model-based recognition method where the input image is over-segmented and the obtained regions are represented by an attributed relational graph (ARG). This graph is then matched against a model graph thus accomplishing the model-based recognition task. This type of problem calls for inexact graph matching through a homomorphism between the graphs since no bijective correspondence can be expected, because of the over-segmentation of the image with respect to the model. The search for the best homomorphism is carried out by optimizing an objective function based on similarities between object and relational attributes defined on the graphs. The following optimization procedures are compared and discussed: deterministic tree search, for which new algorithms are detailed, genetic algorithms and estimation of distribution algorithms. In order to assess the performance of these algorithms using real data, experimental results on supervised classification of facial features using face images from public databases are presented.     

A system for person-independent hand posture recognition againstcomplex backgrounds

A computer vision system for person-independent recognition of hand postures against complex backgrounds is presented. The system is based on the elastic graph matching, which was extended to allow for combinations of different feature types at the graph nodes     

多尺度特征融合的图嵌入方法

针对结构模式识别领域中通用图嵌入方法缺乏且计算复杂度较高的问题,基于空间句法理论提出一种融合多尺度特征的图嵌入方法。通过提取图的节点数、边数和智能度等全局特征、节点拓扑特征、边领域特征差异度和边拓扑特征差异度等局部特征和节点与边上的数值属性和符号属性等细节特征,利用多尺度直方图统计的方法构造描述图特征的特征向量,以此将桥梁将结构模式识别问题转化为统计模式识别问题,进而借助支持向量机(SVM)实现图的分类识别。实验结果表明,所提出的图嵌入方法在不同的图数据集上均具有较高的分类识别率。与其他图嵌入方法相比,该方法对图的拓扑表达能力强,并且可融合图的领域方面的非拓扑特征,通用性较好,计算复杂度较低。     

面向智能家居的挥手识别算法

目的 用手势控制家电是智能家居发展的趋势之一,传统的静态手势识别算法难以适应复杂的居家环境,特别当使用广角相机或环境干扰大时,为此提出一种动态的挥手识别算法,可以对视频序列中的挥手动作做出响应,以达到控制家电的目的。方法 挥手动作具有周期性且频率相对稳定,算法首先调整长滤波器和短滤波器使其检测到视频内周期性运动的区域,然后利用人手识别算法对周期性运动区域进行验证并确认人手。结果 通过与主流的手势识别算法的对比,在复杂环境下,本文算法将成功次数提高了3%,误触发次数降低了44%,响应时间也降低了近0.4 s。结论 实验结果表明,算法能够满足实际应用需求。此外,算法不基于运动目标检测,运算量极低,可以在较高的图像分辨率下实时运行,并能被移植到嵌入式平台下。     

The design of hand gestures for human–computer interaction: Lessons from sign language interpreters

The design and selection of 3D modeled hand gestures for human–computer interaction should follow principles of natural language combined with the need to optimize gesture contrast and recognition. The selection should also consider the discomfort and fatigue associated with distinct hand postures and motions, especially for common commands. Sign language interpreters have extensive and unique experience forming hand gestures and many suffer from hand pain while gesturing. Professional sign language interpreters (N=24) rated discomfort for hand gestures associated with 47 characters and words and 33 hand postures. Clear associations of discomfort with hand postures were identified. In a nominal logistic regression model, high discomfort was associated with gestures requiring a flexed wrist, discordant adjacent fingers, or extended fingers. These and other findings should be considered in the design of hand gestures to optimize the relationship between human cognitive and physical processes and computer gesture recognition systems for human–computer input.     

3D object recognition technique using multiple 2D views for Arabic sign language

Some objects in specific poses cannot be distinguished using a single view. A model is proposed and developed for 3D object recognition based on multiple-views; it was applied on hand postures recognition. A pulse-coupled neural network is used to generate features vector for single view. Two views with different view angles are used; each view generates its features’ vector. The two 2D-vectors are then linearly combined into one 3D vector. The hand postures are then combined to construct a dynamic gesture (word). The reconstruction is performed using best-match search algorithm. The experiment was conducted on 50 words and the result was 96% recognition accuracy confirming objects dataset offline extendibility.