Multimodal hand gesture recognition combining temporal and pose information based on CNN descriptors and histogram of cumulative magnitudes |
| |
| Affiliation: | 1. Department of Electronic Engineering, National Taipei University of Technology, Taipei City 10608, Taiwan;2. Department of Communication Engineering, National Central University, Taoyuan City 320, Taiwan;1. Haian Senior School of Jiangsu Province, Nantong 226600, China;2. College of Physical Education, China University of Mining and Technology, Xuzhou 221000, China;1. University of São Paulo, Institute of Mathematics and Statistics, São Paulo, SP, Brazil;2. Universidade Federal de São Paulo, Instituto de Ciência e Tecnologia, São José dos Campos, SP, Brazil;3. University of Campinas, Institute of Computing, Campinas, SP, Brazil;1. Yango University, Fuzhou 350015, China;2. Department of Public Finance and Taxation, National Kaohsiung University of Science and Technology , Kaohsiung City, 80778, Taiwan;3. Department of International Business, National Kaohsiung University of Science and Technology, Kaohsiung City 80778, Taiwan;1. School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo 454001, China;2. School of Computer Science, Shaanxi Normal University, Xi’an 710119, China;3. School of Computer Science and Technology, Nanjing Normal University, Nanjing 210046, China;4. College of Electronics and Information Engineering, Shaanxi University of Science and Technology, Xi’an 710021, China |
| |
| Abstract: | ![]()
In this paper, we present a new approach for dynamic hand gesture recognition. Our goal is to integrate spatiotemporal features extracted from multimodal data captured by the Kinect sensor. In case the skeleton data is not provided, we apply a novel skeleton estimation method to compute temporal features. Furthermore, we introduce an effective method to extract a fixed number of keyframes to reduce the processing time. To extract pose features from RGB-D data, we take advantage of two different approaches: (1) Convolutional Neural Networks and (2) Histogram of Cumulative Magnitudes. We test different integration methods to fuse the extracted spatiotemporal features to boost recognition performance in a linear SVM classifier. Extensive experiments prove the effectiveness and feasibility of the proposed framework for hand gesture recognition. |
| |
| Keywords: | Hand gesture recognition Spherical coordinates Keyframe extraction Pose and motion information Convolucional neuronal networks Histogram of cumulative magnitudes Fusion schemes |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
