基于时空权重姿态运动特征的人体骨架行为识别研究

人体行为识别在视觉领域的广泛应用使得它在过去的几十年里一直都是备受关注的研究热点.近些年来,深度传感器的普及以及基于深度图像实时骨架估测算法的提出,使得基于骨架序列的人体行为识别研究越来越吸引人们的注意.已有的研究工作大部分提取帧内骨架不同关节点的空间域信息和帧间骨架关节点的时间域信息来表征行为序列,但没有考虑到不同关节点和姿态对判定行为类别所起作用是不同的.因此本文提出了一种基于时空权重姿态运动特征的行为识别方法,采用双线性分类器迭代计算得到关节点和静止姿态相对于该类别动作的权重,确定那些信息量大的关节点和姿态;同时,为了对行为特征进行更好的时序分析,本文引入了动态时间规整和傅里叶时间金字塔算法进行时序建模,最后采用支持向量机完成行为分类.在多个数据集上的实验结果表明,该方法与其它一些方法相比,表现出了相当大的竞争力,甚至更好的识别效果.

Spatio-Temporal Weighted Posture Motion Features for Human Skeleton Action Recognition Research

In recent years,computer vision related applications(e.g.behavior surveillance,human-computer interaction,electronic games,and health care)have gained increasing popularity,and the key technology of these interactive applications is how to make the machine understand human movements,which is also known as human action recognition.Although experts have done a lot of research before,how to accurately identify human action from traditional RGB videos is still a challenging problem due to various interference factors,such as lighting changes,view changes,occlusion and background clutter.Latterly,the popularity of depth sensors and real-time skeleton estimation algorithm based on depth image have brought opportunities for human behavior recognition research.The depth map provides additional depth information,which can easily segment the desired targets from complex scenes,significantly improve the background clutter,and greatly simplify the behavior recognition model.And all of this boosts the development of skeleton-based action recognition.The skeleton estimation algorithm defines skeleton as a graphical model composed of human trunk,head and limbs position.It can quickly and accurately estimate the 3D position information of skeleton joints from depth images at the speed of 200 frames per second on Xbox 360 GPU.Existing skeleton based behavior recognition methods can be roughly divided into two categories:joint-based methods and body-based methods.The human skeleton is regarded as a set of joints and described by the location correlation feature of the joints in the joint-based methods.These features include joint position features,relative joint location features,joint orientation features in a fixed coordinate system and so on.On the other hand,part-based methods regard the human skeleton as a set of rigid segments and use joint angle features,bioinspired 3D features and geometric relationship features of different rigid body parts to represent the human skeleton.Most of these researches focus on extracting the spatial information of different body joints in single frame and temporal information of body joints between adjacent frames to represent action sequence,but these works don’t take into consideration that the importance of different body joints and postures may vary in terms of deciding which type of action class the sample belongs to.Therefore,an action recognition method based on spatio-temporal weighted posture motion features is proposed in this paper.Since each 3D video sequence can be regarded as a set of ordered static gestures,and the static pose can be regarded as a set of joints.Based on this,the author first deals with the spatial relationships of all joints contained in each static pose to obtain the spatial domain characteristics of the video sequence,and then calculate location relationships of the same joint between adjacent frames to get the temporal features.And normalization scheme is introduced to obtain the final representation of the skeleton sequence.The author also adopts bilinear classifier to calculate the weights of the joints and static postures for the action class to determine the informative joints and postures.Meanwhile,dynamic time warping(DTW)algorithm and Fourier temporal pyramid(FTP)representation are introduced to process temporal modeling for better temporal analyses,and SVM is finally used for the action classification.The experimental results on three challenging datasets demonstrate that our approach achieves competitive,even the best performance compared with the state-of-the-art methods.