基于融合几何特征时空图卷积网络的动作识别

最近,基于骨架的动作识别研究受到了广泛关注.因为图卷积网络可以更好地建模非规则数据的内部依赖,ST-GCN (spatial temporal graph convolutional network)已经成为该领域的首选网络框架.针对目前大多数基于ST-GCN的改进方法忽视了骨架序列所蕴含的几何特征.本文利用骨架关节几何特征,作为ST-GCN框架的特征补充,其具有视觉不变性和无需添加额外参数学习即可获取的优势,进一步地,利用时空图卷积网络建模骨架关节几何特征和早期特征融合方法,构成了融合几何特征的时空图卷积网络框架.最后,实验结果表明,与ST-GCN、2s-AGCN和SGN等动作识别模型相比,我们提出的框架在NTU-RGB+D数据集和NTU-RGB+D 120数据集上都取得了更高准确率的效果.

Spatio-temporal GCN with Geometric Features Fusion for Action Recognition

Recently, the research on skeleton-based action recognition has attracted a lot of attention. As the graph convolutional networks can better model the internal dependencies of non-regular data, the spatio-temporal graph convolutional network (ST-GCN) has become the preferred network framework in this field. However, most of the current improvement methods based on the ST-GCN framework ignore the geometric features contained in the skeleton sequences. In this study, we exploit the geometric features of the skeleton joint as the feature enhancement of the ST-GCN framework, which has the advantage of visual invariance without additional parameters. Further, we integrate the geometric feature of the skeleton joint with earlier features to develop ST-GCN with geometric features. Finally, the experimental results show that the proposed framework achieves higher accuracy on both NTU-RGB+D dataset and NTU-RGB+D 120 dataset than other action recognition models such as ST-GCN, 2s-AGCN, and SGN.