| 基于RGB-D视频的多模态手势识别 |
| |
| 引用本文： | 马正文,蔡坚勇,刘磊,欧阳乐峰,李楠.基于RGB-D视频的多模态手势识别[J].计算机系统应用,2018,27(12):234-239. |
| |
| 作者姓名： | 马正文 蔡坚勇 刘磊 欧阳乐峰 李楠 |
| |
| 作者单位： | 福建师范大学 光电与信息工程学院, 福州 350007,福建师范大学 光电与信息工程学院, 福州 350007;福建师范大学 医学光电科学与技术教育部重点实验室, 福州 350007;福建师范大学 福建省光子技术重点实验室, 福州 350007;福建师范大学 福建省光电传感应用工程技术研究中心, 福州 350007;福建师范大学 智能光电系统工程研究中心, 福州 350007,福建师范大学 光电与信息工程学院, 福州 350007,福建师范大学 光电与信息工程学院, 福州 350007,福建师范大学 光电与信息工程学院, 福州 350007 |
| |
| 基金项目： | 福建省自然科学基金（2017J01744） |
| |
| 摘 要： | 本文是对SKIG RGB-D多模态的孤立手势视频进行手势识别研究.首先将RGB和Depth两种单模态视频提取成图片的形式保存，然后采样成长度为32帧的手势序列分别输入到本文提出的稠密连接的3DCNN组件学习短期的时空域特征，然后将提取的时空域特征输入到卷积GRU网络进行长期的时空域特征学习，最终对单模态训练好的网络进行多模态融合，提升网络识别准确率.本文在SKIG数据集上取得了99.07%的识别准确率，达到了极高的准确率，证明了本文提出的网络模型的有效性.
|
| 关 键 词： | 手势识别 稠密连接的3DCNN 卷积GRU 时空域特征 |
| 收稿时间： | 2018/5/2 0:00:00 |
| 修稿时间： | 2018/5/24 0:00:00 |
Multimodal Gesture Recognition Based on RGB-D Video |
| |
| MA Zheng-Wen,CAI Jian-Yong,LIU Lei,OUYANG Le-Feng and LI Nan.Multimodal Gesture Recognition Based on RGB-D Video[J].Computer Systems& Applications,2018,27(12):234-239. |
| |
| Authors: | MA Zheng-Wen CAI Jian-Yong LIU Lei OUYANG Le-Feng and LI Nan |
| |
| Affiliation: | College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China,College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China;Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Normal University, Fuzhou 350007, China;Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou 350007, China;Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou 350007, China;Intelligent Optoelectronic Systems Research Center, Fujian Normal University, Fuzhou 350007, China,College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China,College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China and College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou 350007, China |
| |
| Abstract: | In this study, the gesture recognition based on SKIG RGB-D multimodal isolated gesture video is studied. The RGB and depth videos are extracted into the form of images. Then the sampled 32 frames from images are input to the densely connected 3DCNN component to learn short-term spatiotemporal features, after that the features input to the convolutional GRU to learn long-term spatiotemporal features. Finally, the trained networks for single modal are used to multimodal fusion to improve the recognition accuracy. 99.07% recognition accuracy is obtained on the SKIG dataset, which achieves high accuracy and proves the validity of the network model proposed in this study. |
| |
| Keywords: | gesture recognition densely connected 3DCNN convolutional GRU spatiotemporal features |
|
| 点击此处可从《计算机系统应用》浏览原始摘要信息 |
|
点击此处可从《计算机系统应用》下载全文 |
|
