Hand gesture recognition using Leap Motion via deterministic learning

With the development of multimedia technology, traditional interactive tools, such as mouse and keyboard, cannot satisfy users’ requirements. Touchless interaction has received considerable attention in recent years with benefit of removing barriers of physical contact. Leap Motion is an interactive device which can be used to collect information of dynamic hand gestures, including coordinate, acceleration and direction of fingers. The aim of this study is to develop a new method for hand gesture recognition using jointly calibrated Leap Motion via deterministic learning. Hand gesture features representing hand motion dynamics, including spatial position and direction of fingers, are derived from Leap Motion. Hand motion dynamics underlying motion patterns of different gestures which represent Arabic numbers (0-9) and capital English alphabets (A-Z) are modeled by constant radial basis function (RBF) neural networks. Then, a bank of estimators is constructed by the constant RBF networks. By comparing the set of estimators with a test gesture pattern, a set of recognition errors are generated. The average L₁ norms of the errors are taken as the recognition measure according to the smallest error principle. Finally, experiments are carried out to demonstrate the high recognition performance of the proposed method. By using the 2-fold, 10-fold and leave-one-person-out cross-validation styles, the correct recognition rates for the Arabic numbers are reported to be 94.2%, 95.1% and 90.2%, respectively, for the English alphabets are reported to be 89.2%, 92.9% and 86.4%, respectively.     

基于Leap Motion和支持向量机的手势识别

为了使手势识别在更多的领域得到推广及应用,提出了基于Leap Motion体感设备实时跟踪技术获取手势三维空间坐标信息的方法,并从中分别提取角度信息和相对坐标信息,构建手势特征数据,建立手势识别模型.对特征数据进行归一化处理后,利用支持向量机(SVM)分类器进行训练、建模和分类,实现手势识别.实验结果表明:以角度数据和坐标数据作为手势特征的方法可行,平均识别率分别为96.6%和91.8%.通过对比可以得出:以角度数据作为特征值具有较高的准确性和鲁棒性,并避免了单纯依照一种特征值产生的局限性.     

基于Leap Motion指尖位置的手势提取和识别技术研究

Leap Motion是最近推出的一款比较新颖的手部信息采集设备,它能够高精度、高帧率地跟踪捕获手部信息,基于此特性,本文阐述了一种基于指尖位置和方向信息进行手势提取和识别的研究方案。采用Leap Motion传感器进行手势的三维空间坐标信息采集,从中提取指尖坐标及方向向量信息,建立手势识别模型,构建手势特征数据。对特征数据进行归一化处理后输入到支持向量机进行训练,实现对特定手势的识别。实验结果表明,提出的手势识别方案平均识别精度达到97.33%,具有较高的准确性和鲁棒性。     

基于深度学习的虹膜人脸多特征融合识别

传统多生物特征融合识别方法中人工设计特征提取存在盲目性和差异性,特征融合存在空间不匹配或维度过高等问题,为此提出一种基于深度学习的多生物特征融合识别方法。通过卷积神经网络(convolutional neural networks,CNN)提取人脸和虹膜特征、参数化t-SNE算法特征降维和支持向量机(support vector machine,SVM)分类组合进行融合识别。实验结果表明,该融合识别方法与单一生物特征识别以及其它融合识别方法相比,鲁棒性增强,识别性能提升明显。     

Motion estimation of 3-D objects using multisensor data fusion

This article presents an approach to estimate the general 3-D motion of a polyhedral object using multiple sensor data some of which may not provide sufficient information for the estimation of object motion. Motion can be estimated continuously from each sensor through the analysis of the instantaneous state of an object. The instantaneous state of an object is specified by the rotation, which is defined by a rotation axis and rotation angle, and the displacement of the center of rotation. We have introduced a method based on Moore-Penrose pseudoinverse theory to estimate the instantaneous state of an object, and a linear feedback estimation algorithm to approach the motion estimation. The motion estimated from each sensor is fused to provide more accurate and reliable information about the motion of an unknown object. The techniques of multisensor data fusion can be categorized into three methods: averaging, decision, and guiding. We present a fusion algorithm which combines averaging and decision. With the assumption that the motion is smooth, our approach can handle the data sequences from multiple sensors with different sampling times. We can also predict the next immediate object position and its motion. The simulation results show our proposed approach is advantageous in terms of accuracy, speed, and versatility.     

Human gait recognition using extraction and fusion of global motion features

This paper proposes a novel computer vision approach that processes video sequences of people walking and then recognises those people by their gait. Human motion carries different information that can be analysed in various ways. The skeleton carries motion information about human joints, and the silhouette carries information about boundary motion of the human body. Moreover, binary and gray-level images contain different information about human movements. This work proposes to recover these different kinds of information to interpret the global motion of the human body based on four different segmented image models, using a fusion model to improve classification. Our proposed method considers the set of the segmented frames of each individual as a distinct class and each frame as an object of this class. The methodology applies background extraction using the Gaussian Mixture Model (GMM), a scale reduction based on the Wavelet Transform (WT) and feature extraction by Principal Component Analysis (PCA). We propose four new schemas for motion information capture: the Silhouette-Gray-Wavelet model (SGW) captures motion based on grey level variations; the Silhouette-Binary-Wavelet model (SBW) captures motion based on binary information; the Silhouette–Edge-Binary model (SEW) captures motion based on edge information and the Silhouette Skeleton Wavelet model (SSW) captures motion based on skeleton movement. The classification rates obtained separately from these four different models are then merged using a new proposed fusion technique. The results suggest excellent performance in terms of recognising people by their gait.     

Design and implementation of somatosensory interactive game based on Leap Motion

Recently, virtual reality and interactive somatosensory technology has become one of the hot issues in the research of computer applications. Leap Motion is a new type of interactive somatosensory devices which bring users senses of immersion efficiently. This paper studies a interactive somatosensory game model based on Leap Motion and implemented with Unity. Based on the two core technology philosophy of Leap Motion, i.e., virtual reality technology and body sense of interactive technology, the design implementation of each sub module of the system and Leap Motion game algorithm are thoroughly addressed. This paper has certain significance for future application of Leap Motion in film, television, and interactive games.     

EEG based emotion recognition using fusion feature extraction method

Multimedia Tools and Applications - As a high-level function of the human brain, emotion is the external manifestation of people’s psychological characteristics. The emotion has a great...     

Motion codeword generation using selective subsequence clustering for human action recognition

The understanding of human activity is one of the key research areas in human-centered robotic applications. In this paper, we propose complexity-based motion features for recognizing human actions. Using a time-series-complexity measure, the proposed method evaluates the amount of useful information in subsequences to select meaningful temporal parts in a human motion trajectory. Based on these meaningful subsequences, motion codewords are learned using a clustering algorithm. Motion features are then generated and represented as a histogram of the motion codewords. Furthermore, we propose a multiscaled sliding window for generating motion codewords to solve the sensitivity problem of the performance to the fixed length of the sliding window. As a classification method, we employed a random forest classifier. Moreover, to validate the proposed method, we present experimental results of the proposed approach based on two open data sets: MSR Action 3D and UTKinect data sets.     

基于虚拟演播室的Motion Capture研究

在介绍虚拟演播室的基础上,分析了Motion Capture的技术和应用,提出了一种基于虚拟演播室的系统实现,并探讨了其在节目制作中的应用。     

Automatic speaker age and gender recognition using acoustic and prosodic level information fusion

The paper presents a novel automatic speaker age and gender identification approach which combines seven different methods at both acoustic and prosodic levels to improve the baseline performance. The three baseline subsystems are (1) Gaussian mixture model (GMM) based on mel-frequency cepstral coefficient (MFCC) features, (2) Support vector machine (SVM) based on GMM mean supervectors and (3) SVM based on 450-dimensional utterance level features including acoustic, prosodic and voice quality information. In addition, we propose four subsystems: (1) SVM based on UBM weight posterior probability supervectors using the Bhattacharyya probability product kernel, (2) Sparse representation based on UBM weight posterior probability supervectors, (3) SVM based on GMM maximum likelihood linear regression (MLLR) matrix supervectors and (4) SVM based on the polynomial expansion coefficients of the syllable level prosodic feature contours in voiced speech segments. Contours of pitch, time domain energy, frequency domain harmonic structure energy and formant for each syllable (segmented using energy information in the voiced speech segment) are considered for analysis in subsystem (4). The proposed four subsystems have been demonstrated to be effective and able to achieve competitive results in classifying different age and gender groups. To further improve the overall classification performance, weighted summation based fusion of these seven subsystems at the score level is demonstrated. Experiment results are reported on the development and test set of the 2010 Interspeech Paralinguistic Challenge aGender database. Compared to the SVM baseline system (3), which is the baseline system suggested by the challenge committee, the proposed fusion system achieves 5.6% absolute improvement in unweighted accuracy for the age task and 4.2% for the gender task on the development set. On the final test set, we obtain 3.1% and 3.8% absolute improvement, respectively.     

维修仿真中虚拟手动作生成方法研究

在维修仿真中,虚拟维修人员的手部操作起着重要的作用,对维修活动中手部动作的实时、连贯生成问题进行研究。首先提出了一种内含解剖结构的虚拟手模型,然后在分析总结各类虚拟人运动控制算法的基础上,提出一种优先级阻尼IK方法,通过加入阻尼系数来克服传统IK算法无法处理奇异性、多目标冲突性的难题,并利用雅可比矩阵的正交投影来处理多目标约束条件下的手部动作生成,实现IK算法的进一步扩展。最后对虚拟手的指向、握和捏动作进行实际验证,证明所提方法的有效性和健壮性。     

Classification and recognition of computed tomography images using image reconstruction and information fusion methods

In this paper, we propose a diagnosis and classification method of hydrocephalus computed tomography (CT) images using deep learning and image reconstruction methods. The proposed method constructs pathological features differing from the other healthy tissues. This method tries to improve the accuracy of pathological images identification and diagnosis. Identification of pathological features from CT images is an essential subject for the diagnosis and treatment of diseases. However, it is difficult to accurately distinguish pathological features owing to the variability of appearances, fuzzy boundaries, heterogeneous densities, shapes and sizes of lesions, etc. Some study results reported that the ResNet network has a better classification and diagnosis performance than other methods, and it has broad application prospectives in the identification of CT images. We use an improved ResNet network as a classification model with our proposed image reconstruction and information fusion methods. First, we evaluate a classification experiment using the hydrocephalus CT image datasets. Through the comparative experiments, we found that gradient features play an important role in the classification of hydrocephalus CT images. The classification effect of CT images with small information entropy is excellent in the evaluation of hydrocephalus CT images. A reconstructed image containing two channels of gradient features and one channel of LBP features is very effective in classification. Second, we apply our proposed method in classification experiments on CT images of colonography polyps for an evaluation. The experimental results have consistency with the hydrocephalus classification evaluation. It shows that the method is universal and suitable for classification of CT images in these two applications for the diagnosis of diseases. The original features of CT images are not ideal characteristics in classification, and the reconstructed image and information fusion methods have a great effect on CT images classification for pathological diagnosis.

     

Deep neural network-based fusion model for emotion recognition using visual data

The Journal of Supercomputing - In this study, we present a fusion model for emotion recognition based on visual data. The proposed model uses video information as its input and generates emotion...     

Hand gesture recognition using two-level speed normalization,feature selection and classifier fusion

Hand gesture recognition provides an alternative way to many devices for human computer interaction. In this work, we have developed a classifier fusion based dynamic free-air hand gesture recognition system to identify the isolated gestures. Different users gesticulate at different speed for the same gesture. Hence, when comparing different samples of the same gesture, variations due to difference in gesturing speed should not contribute to the dissimilarity score. Thus, we have introduced a two-level speed normalization procedure using DTW and Euclidean distance-based techniques. Three features such as ‘orientation between consecutive points’, ‘speed’ and ‘orientation between first and every trajectory points’ were used for the speed normalization. Moreover, in feature extraction stage, 44 features were selected from the existing literatures. Use of total feature set could lead to overfitting, information redundancy and may increase the computational complexity due to higher dimension. Thus, we have tried to overcome this difficulty by selecting optimal set of features using analysis of variance and incremental feature selection techniques. The performance of the system was evaluated using this optimal set of features for different individual classifiers such as ANN, SVM, k-NN and Naïve Bayes. Finally, the decisions of the individual classifiers were combined using classifier fusion model. Based on the experimental results it may be concluded that classifier fusion provides satisfactory results compared to other individual classifiers. An accuracy of 94.78 % was achieved using the classifier fusion technique as compared to baseline CRF (85.07 %) and HCRF (89.91 %) models.     

Motion recognition using nonparametric image motion models estimated from temporal and multiscale co-occurrence statistics

A new approach for motion characterization in image sequences is presented. It relies on the probabilistic modeling of temporal and scale co-occurrence distributions of local motion-related measurements directly computed over image sequences. Temporal multiscale Gibbs models allow us to handle both spatial and temporal aspects of image motion content within a unified statistical framework. Since this modeling mainly involves the scalar product between co-occurrence values and Gibbs potentials, we can formulate and address several fundamental issues: model estimation according to the ML criterion (hence, model training and learning) and motion classification. We have conducted motion recognition experiments over a large set of real image sequences comprising various motion types such as temporal texture samples, human motion examples, and rigid motion situations.     

2D and 3D palmprint fusion and recognition using PCA plus TPTSR method

This paper employs both two-dimensional (2D) and three-dimensional (3D) features of palmprint for recognition. While 2D palmprint image contains plenty of texture information, 3D palmprint image contains the depth information of the palm surface. Using two different features, we can achieve higher recognition accuracy than using only one of them. In addition, we can improve the robustness. To recognize palmprints, we use two-phase test sample representation (TPTSR) which is proved to be successful in face recognition. Before TPTSR, we perform principal component analysis to extract global features from the 2D and 3D palmprint images. We make decision based on the fusion of 2D and 3D features matching scores. We perform experiments on the PolyU 2D + 3D palmprint database which contains 8,000 samples and achieve satisfying recognition performance.