基于运动学动态图的人体动作识别方法

为了识别RGB-D视频中的人体动作,针对视频中运动信息利用不充分的问题,提出了一种基于运动学动态图的人体动作识别方法。首先利用RGB视频序列和对应的深度图序列生成场景流特征图,基于场景流特征图计算运动学特征图序列,其中包含丰富的运动信息;使用分层排序池化将运动学特征图序列编码为运动学动态图,同时将RGB视频序列编码为外观动态图,最后将运动学动态图和外观动态图输入到双流卷积网络进行人体动作识别。结果表明:基于运动学动态图和双流卷积网络的人体动作识别方法融合了外观信息和运动信息,不仅充分表征了视频的动态,而且使用了视频中具有丰富运动信息的运动学特征;在公开的数据集上对本方法进行验证,在M2I数据集和SBU Kinect Interaction数据集的动作识别率分别为91.8%和95.2%。

Human action recognition based on kinematic dynamic image

To recognize human action in RGB-D video,aiming at the problem of insufficient using of motion information in the RGB-D video,a human action recognition method was proposed based on kinematic dynamic image.Firstly,RGB sequence and its depth maps sequence were used to generate scene flow feature map,which is used to get motion feature sequence and it has more motion information.Hierarchical rank pooling is used to encode kinematic feature maps sequence into a kinematic dynamic image,and to encode RGB sequence into appearance dynamic image.Finally,the kinematics dynamic image and the appearance dynamic image were fed into the dual-stream convolution network for human action recognition.The experiment results show that the proposed method combines the evolution of appearance and motion information,which can fully represent the dynamics and explore more motion information.This method is evaluated on public RGB-D datasets and the action recognition accurate on the M2I dataset and the SBU Kinect Interaction dataset are 91.8%and 95.2%,respectively.