Grenander时间结构学习与推理优化下的行为识别

针对现有基于视频整体时间结构建模的行为识别方法中,存在的时间噪声信息和歧义信息干扰现象,从而引起行为类别识别错误的问题,提出一种新型的Grenander推理优化下时间图模型(temporal graph model with Grenander inference, TGM-GI).首先,构建3D CNN-LSTM模块,其中3D CNN用于行为的动态特征提取, LSTM模块用于该特征的时间依赖关系优化.其次,在深度模块基础上,利用Grenander理论构建了行为识别的时间图模型,并设计了两个模块分别处理慢行为时间冗余和异常行为干扰问题,实现了时间噪声抑制下的时间结构提议.随后,设计融合特征约束和语义约束的Grenander测度,并提出一种时序增量形式的Viterbi算法,修正了行为时间模式中的歧义信息.最后,采用基于动态时间规划的模式匹配方法,完成了基于时间模式的行为识别任务.在UCF101和Olympic Sports两个公认数据集上,与现有多种基于深度学习的行为识别方法进行比较,该方法获得了最好的行为识别正确率.该方法优于基准的3D CNN-LSTM方法,在UCF101数据集上识别...

Temporal Structure Learning with Grenander Inference for Action Recognition

Action recognition is one crucial and very challenging task in computer vision. Most of the existing methods use the temporal structure of the whole video and ignore its temporal noise and ambiguity feature, which leads to failure in action recognition. To address this problem, a novel temporal graph model is proposed with Grenander inference, namely, TGM-GI. First, a 3D CNN+ LSTM module is constructed to learn deep features, in which 3D CNN extracts the dynamic feature of video clips and LSTM optimizes the time dependence between features of two clips. Second, a temporal graph model is constructed with these deep features which use the generator space of Grenander theory. The original temporal pattern is modified using two operations, in which combination operation can remove redundancy clips like slow motion and denoise operation can remove low-frequency clips like abnormal motion. Third, an incremental Viterbi algorithm is proposed for temporal pattern learning with Grenander inference, in which a Grenander measure is designed with both feature bond and semantic bond. Finally, the dynamic time warping is used to match the Grenander temporal pattern of test video with the Grenander temporal pattern of the training set and the label of the test video is predicted. The experimental results show that the proposed TGM-GI outperforms the state-of-the-art methods on two acknowledge databases. The TGM-GI is superior to the baseline method of 3D CNN-LSTM, and its accuracy improves 6.41% on the UCF101 dataset and 5.67% on the Olympic Sports dataset respectively.