Hand gesture recognition using Leap Motion via deterministic learning

With the development of multimedia technology, traditional interactive tools, such as mouse and keyboard, cannot satisfy users’ requirements. Touchless interaction has received considerable attention in recent years with benefit of removing barriers of physical contact. Leap Motion is an interactive device which can be used to collect information of dynamic hand gestures, including coordinate, acceleration and direction of fingers. The aim of this study is to develop a new method for hand gesture recognition using jointly calibrated Leap Motion via deterministic learning. Hand gesture features representing hand motion dynamics, including spatial position and direction of fingers, are derived from Leap Motion. Hand motion dynamics underlying motion patterns of different gestures which represent Arabic numbers (0-9) and capital English alphabets (A-Z) are modeled by constant radial basis function (RBF) neural networks. Then, a bank of estimators is constructed by the constant RBF networks. By comparing the set of estimators with a test gesture pattern, a set of recognition errors are generated. The average L₁ norms of the errors are taken as the recognition measure according to the smallest error principle. Finally, experiments are carried out to demonstrate the high recognition performance of the proposed method. By using the 2-fold, 10-fold and leave-one-person-out cross-validation styles, the correct recognition rates for the Arabic numbers are reported to be 94.2%, 95.1% and 90.2%, respectively, for the English alphabets are reported to be 89.2%, 92.9% and 86.4%, respectively.     

基于Leap Motion和支持向量机的手势识别

为了使手势识别在更多的领域得到推广及应用,提出了基于Leap Motion体感设备实时跟踪技术获取手势三维空间坐标信息的方法,并从中分别提取角度信息和相对坐标信息,构建手势特征数据,建立手势识别模型.对特征数据进行归一化处理后,利用支持向量机(SVM)分类器进行训练、建模和分类,实现手势识别.实验结果表明:以角度数据和坐标数据作为手势特征的方法可行,平均识别率分别为96.6%和91.8%.通过对比可以得出:以角度数据作为特征值具有较高的准确性和鲁棒性,并避免了单纯依照一种特征值产生的局限性.     

基于Leap Motion的虚拟课堂手势交互方法

随着虚拟现实技术的飞速发展, Leap Motion等体感传感器出现并被广泛地应用在人机交互中.针对Leap Motion体感控制器在识别范围边缘识别率低且识别速度慢的问题提出了一种基于深度神经网络的Leap Motion手势交互方法.该方法在定义的交互手势基础上,设计了三维交互系统并应用到虚拟场景中.系统首先通过Leap Motion进行数据捕捉,对获取到的红外图像采用深度神经网络进行特征提取并实现对手势的分类识别,然后结合Leap Motion获取的手部坐标前后帧的变化来判断动态手势,最终结合动态手势完成虚拟场景中的交互功能.经过实验验证,本文手势识别方法无论是在识别速度还是识别精度上都优于Leap Motion自带的手势识别方法,同时在Leap Motion识别范围边界处仍能保持较高的识别率.     

基于Leap Motion指尖位置的手势提取和识别技术研究

Leap Motion是最近推出的一款比较新颖的手部信息采集设备,它能够高精度、高帧率地跟踪捕获手部信息,基于此特性,本文阐述了一种基于指尖位置和方向信息进行手势提取和识别的研究方案。采用Leap Motion传感器进行手势的三维空间坐标信息采集,从中提取指尖坐标及方向向量信息,建立手势识别模型,构建手势特征数据。对特征数据进行归一化处理后输入到支持向量机进行训练,实现对特定手势的识别。实验结果表明,提出的手势识别方案平均识别精度达到97.33%,具有较高的准确性和鲁棒性。     

基于Leap Motion和Unity3D的虚拟沙画手势识别及交互

虚拟仿真技术的快速发展及体感设备的不断更新为沙画动画这一全新的艺术创作形式带来新的灵感。针对沙画现场作画工序复杂的问题,结合Leap Motion设备和Unity3D开发环境完成手势识别并驱动虚拟手实现虚拟沙画效果。首先,依据Leap Motion捕捉到的手势坐标及方向信息提取手部关键点;然后提出角域划分的方法并引入新的特征向量,将其与提取信息串联作为手势分类依据;最后,根据自行定义的沙画手势语义驱动虚拟手完成虚拟沙画创作。实验证明,利用Leap Motion完成近距离手势识别效果较其他方法结果更加精准,实时性较高,手势跟踪稳定,虚拟沙画绘画过程沉浸感强。     

基于加速度传感器的运动模式识别

提出一种用于识别人体运动行为模式的算法,该算法仅需从智能手机加速传感器获取信号数据,对信号进行频域滤波,采用改进的DBSCAN算法进行聚类和识别出运动模式。实验结果表明该算法具有较高的准确率和实用性。     

基于深度学习的虹膜人脸多特征融合识别

传统多生物特征融合识别方法中人工设计特征提取存在盲目性和差异性,特征融合存在空间不匹配或维度过高等问题,为此提出一种基于深度学习的多生物特征融合识别方法。通过卷积神经网络(convolutional neural networks,CNN)提取人脸和虹膜特征、参数化t-SNE算法特征降维和支持向量机(support vector machine,SVM)分类组合进行融合识别。实验结果表明,该融合识别方法与单一生物特征识别以及其它融合识别方法相比,鲁棒性增强,识别性能提升明显。     

Motion estimation of 3-D objects using multisensor data fusion

This article presents an approach to estimate the general 3-D motion of a polyhedral object using multiple sensor data some of which may not provide sufficient information for the estimation of object motion. Motion can be estimated continuously from each sensor through the analysis of the instantaneous state of an object. The instantaneous state of an object is specified by the rotation, which is defined by a rotation axis and rotation angle, and the displacement of the center of rotation. We have introduced a method based on Moore-Penrose pseudoinverse theory to estimate the instantaneous state of an object, and a linear feedback estimation algorithm to approach the motion estimation. The motion estimated from each sensor is fused to provide more accurate and reliable information about the motion of an unknown object. The techniques of multisensor data fusion can be categorized into three methods: averaging, decision, and guiding. We present a fusion algorithm which combines averaging and decision. With the assumption that the motion is smooth, our approach can handle the data sequences from multiple sensors with different sampling times. We can also predict the next immediate object position and its motion. The simulation results show our proposed approach is advantageous in terms of accuracy, speed, and versatility.     

Evaluation of visual feedback techniques for virtual grasping with bare hands using Leap Motion and Oculus Rift

Virtual Reality - Bare hand interaction (BHI) allows users to use their hands and fingers to interact with digital content without any attached devices or accessories. For BHI to realize widespread...     

使用二次特征选择及核融合的语音情感识别

为提高语音情感识别精度,对基本声学特征构建的多维特征集合,采用二次特征选择方法综合考虑特征参数与情感类别之间的内在特性,从而建立优化的、具有有效情感可分性的特征子集;在语音情感识别阶段,设计二叉树结构的多分类器以综合考虑系统整体性能与复杂度,采用核融合方法改进SVM模型,使用多核SVM识别混淆度最大的情感。算法在Berlin情感语音库五种情感状态的样本上进行验证,实验结果表明二次特征选择与核融合相结合的方法在有效提高情感识别精度的同时,对噪声具有一定的鲁棒性。     

Design and implementation of somatosensory interactive game based on Leap Motion

Recently, virtual reality and interactive somatosensory technology has become one of the hot issues in the research of computer applications. Leap Motion is a new type of interactive somatosensory devices which bring users senses of immersion efficiently. This paper studies a interactive somatosensory game model based on Leap Motion and implemented with Unity. Based on the two core technology philosophy of Leap Motion, i.e., virtual reality technology and body sense of interactive technology, the design implementation of each sub module of the system and Leap Motion game algorithm are thoroughly addressed. This paper has certain significance for future application of Leap Motion in film, television, and interactive games.     

Human gait recognition using extraction and fusion of global motion features

This paper proposes a novel computer vision approach that processes video sequences of people walking and then recognises those people by their gait. Human motion carries different information that can be analysed in various ways. The skeleton carries motion information about human joints, and the silhouette carries information about boundary motion of the human body. Moreover, binary and gray-level images contain different information about human movements. This work proposes to recover these different kinds of information to interpret the global motion of the human body based on four different segmented image models, using a fusion model to improve classification. Our proposed method considers the set of the segmented frames of each individual as a distinct class and each frame as an object of this class. The methodology applies background extraction using the Gaussian Mixture Model (GMM), a scale reduction based on the Wavelet Transform (WT) and feature extraction by Principal Component Analysis (PCA). We propose four new schemas for motion information capture: the Silhouette-Gray-Wavelet model (SGW) captures motion based on grey level variations; the Silhouette-Binary-Wavelet model (SBW) captures motion based on binary information; the Silhouette–Edge-Binary model (SEW) captures motion based on edge information and the Silhouette Skeleton Wavelet model (SSW) captures motion based on skeleton movement. The classification rates obtained separately from these four different models are then merged using a new proposed fusion technique. The results suggest excellent performance in terms of recognising people by their gait.     

基于VGG-NET的特征融合面部表情识别

为了解决在面部表情特征提取过程中卷积神经网络CNN和局部二值模式LBP只能提取面部表情图像的单一特征,难以提取与面部变化高度相关的精确特征的问题,提出了一种基于深度学习的特征融合的表情识别方法。该方法将LBP特征和CNN卷积层提取的特征通过加权的方式结合在改进的VGG-16网络连接层中,最后将融合特征送入Softmax分类器获取各类特征的概率,完成基本的6种表情分类。实验结果表明,所提方法在CK+和JAFFE数据集上的平均识别准确率分别达到了97.5%和97.62%,利用融合特征得到的识别结果明显优于利用单一特征识别的效果。与其他方法相比较,该方法能有效提高表情识别准确率,对光照变化更加鲁棒。     

EEG based emotion recognition using fusion feature extraction method

Multimedia Tools and Applications - As a high-level function of the human brain, emotion is the external manifestation of people’s psychological characteristics. The emotion has a great...     

基于多尺度特征融合CNN模型的车辆精细型号识别

车辆精细型号是车辆识别的主要线索之一,也是智能交通系统的重要组成部分。针对车辆精细型号种类繁多、车辆所处环境复杂多变等因素,提出一种基于多尺度特征融合的车辆精细型号识别方法。该方法基于传统的卷积神经网络,通过提取并融合来自网络底层和高层的车辆特征,完成对车辆精细型号的识别。与其他基于卷积神经网络的车辆精细型号识别方法相比,该方法在提高分类准确率的同时还大幅度降低了整体网络的参数规模。实验结果表明,在公开数据集CompCars的监控场景下其识别精度达到了98.43%,且模型参数大小仅为3.93 MB,平均每张图片只需0.83 ms的分类时间。     

Emotion-relevant activity recognition based on smart cushion using multi-sensor fusion

More and more common activities are leading to a sedentary lifestyle forcing us to sit several hours every day. In-seat actions contain significant hidden information, which not only reflects the current physical health status but also can report mental states. Considering this, we design a system, based on body-worn inertial sensors (attached to user’s wrists) combined with a pressure detection module (deployed on the seat), to recognise and monitor in-seat activities through sensor- and feature-level fusion techniques. Specifically, we focus on four common basic emotion-relevant activities (i.e. interest-, frustration-, sadness- and happiness-related). Our results show that the proposed method, by fusion of time- and frequency-domain feature sets from all the different deployed sensors, can achieve high accuracy in recognising the considered activities.     

基于支持向量机分数等级融合的虹膜识别方法

提出了一种使用支持向量机（Support Vector Machine,SVM）的分数等级融合的虹膜识别方法。通过对虹膜纹理采用小波包分解,选择最高能量区域和次高能量区域提取特征向量,与注册入库的虹膜特征向量计算出海明距离。最后融合两个海明距离输入SVM进行识别。该方法减少输入支持向量机的维数。实验结果表明,该法提高了识别率,能够有效地应用到身份鉴别系统中。     

Motion codeword generation using selective subsequence clustering for human action recognition

The understanding of human activity is one of the key research areas in human-centered robotic applications. In this paper, we propose complexity-based motion features for recognizing human actions. Using a time-series-complexity measure, the proposed method evaluates the amount of useful information in subsequences to select meaningful temporal parts in a human motion trajectory. Based on these meaningful subsequences, motion codewords are learned using a clustering algorithm. Motion features are then generated and represented as a histogram of the motion codewords. Furthermore, we propose a multiscaled sliding window for generating motion codewords to solve the sensitivity problem of the performance to the fixed length of the sliding window. As a classification method, we employed a random forest classifier. Moreover, to validate the proposed method, we present experimental results of the proposed approach based on two open data sets: MSR Action 3D and UTKinect data sets.