基于语义特征立方体切片的人体动作识别

基于深度学习的人体动作识别近几年取得了良好的识别效果,尤其是二维卷积神经网络可以较充分的学习人体动作的空间特征,但在捕获长时间的运动信息上仍存在问题。针对此问题,提出了基于语义特征立方体切片的人体动作识别模型来联合地学习动作的表观和运动特征。该模型在时序分割网络(Temporal Segment Networks,TSN)的基础上,选取InceptionV4作为骨干网络提取人体动作的表观特征,将得到的三维特征图立方体分为二维的空间上和时间上的特征图切片。另外设计一个时空特征融合模块协同的学习多维度切片的权重分配,从而得到人体动作的时空特征,由此实现了网络的端到端训练。与TSN模型相比,该模型在UCF101和 HMDB51数据集上的准确率均有所提升。实验结果表明,该模型在不显著增加网络参数量的前提下,能够捕获更丰富的运动信息,使人体动作的识别结果提高。 

Human Action Recognition Based on Semantic Feature Cuboid Slicing

Human action recognition based on deep learning has achieved great success in recent years. Especially the 2D convolutional neural network can learn the spatial features of human action well, but there are still problems in capturing long-term motion information. In order to solve this problem, the human action recognition model based on the semantic feature cuboid slicing is proposed to jointly learn the appearance and motion features of action. On the basis of temporal segment networks(TSN), the model adopts InceptionV4 as the backbone network to extract the appearance features of human action, and divides the 3D feature cuboids into 2D spatio-slices and 2D temporal-slices. A spatiotemporal feature fusion module is also proposed to comprehensively learn the weight distribution of multi-dimensional slices, so as to obtain the spatiotemporal features of human action, and an end-to-end model is trained in this way. The accuracy of our model improves in UCF101 and HMDB51 compared with TSN. The experimental result shows that the model can capture more motion information and improve the recognition results of human action without significantly increasing the network parameters.