Recognizing 50 human action categories of web videos

Action recognition on large categories of unconstrained videos taken from the web is a very challenging problem compared to datasets like KTH (6 actions), IXMAS (13 actions), and Weizmann (10 actions). Challenges like camera motion, different viewpoints, large interclass variations, cluttered background, occlusions, bad illumination conditions, and poor quality of web videos cause the majority of the state-of-the-art action recognition approaches to fail. Also, an increased number of categories and the inclusion of actions with high confusion add to the challenges. In this paper, we propose using the scene context information obtained from moving and stationary pixels in the key frames, in conjunction with motion features, to solve the action recognition problem on a large (50 actions) dataset with videos from the web. We perform a combination of early and late fusion on multiple features to handle the very large number of categories. We demonstrate that scene context is a very important feature to perform action recognition on very large datasets. The proposed method does not require any kind of video stabilization, person detection, or tracking and pruning of features. Our approach gives good performance on a large number of action categories; it has been tested on the UCF50 dataset with 50 action categories, which is an extension of the UCF YouTube Action (UCF11) dataset containing 11 action categories. We also tested our approach on the KTH and HMDB51 datasets for comparison.     

动作切分和流形度量学习的视频动作识别

目的 为了提高视频中动作识别的准确度,提出基于动作切分和流形度量学习的视频动作识别算法。方法 首先利用基于人物肢体伸展程度分析的动作切分方法对视频中的动作进行切分,将动作识别的对象具体化;然后从动作片段中提取归一化之后的全局时域特征和空域特征、光流特征、帧内的局部旋度特征和散度特征,构造一种7×7的协方差矩阵描述子对提取出的多种特征进行融合;最后结合流形度量学习方法有监督式地寻找更优的距离度量算法提高动作的识别分类效果。结果 对Weizmann公共视频集的切分实验统计结果表明本文提出的视频切分方法具有很好的切分能力,能够作好动作识别前的预处理;在Weizmann公共视频数据集上进行了流形度量学习前后的识别效果对比,结果表明利用流形度量学习方法对动作识别效果提升2.8%;在Weizmann和KTH两个公共视频数据集上的平均识别率分别为95.6%和92.3%,与现有方法的比较表明,本文提出的动作识别方法有更好的识别效果。结论 多次实验结果表明本文算法在预处理过程中动作切分效果理想,描述动作所构造协方差矩阵对动作的表达有良好的多特征融合能力,而且光流信息和旋度、散度信息的加入使得人体各部位的运动方向信息具有了更多细节的描述,有效提高了协方差矩阵的描述能力,结合流形度量学习方法对动作识别的准确性有明显提高。     

Human action recognition using three orthogonal planes with unsupervised deep convolutional neural network

Deep learning models have attained great success for an extensive range of computer vision applications including image and video classification. However, the complex architecture of the most recently developed networks imposes certain memory and computational resource limitations, especially for human action recognition applications. Unsupervised deep convolutional neural networks such as PCANet can alleviate these limitations and hence significantly reduce the computational complexity of the whole recognition system. In this work, instead of using 3D convolutional neural network architecture to learn temporal features of video actions, the unsupervised convolutional PCANet model is extended into (PCANet-TOP) which effectively learn spatiotemporal features from Three Orthogonal Planes (TOP). For each video sequence, spatial frames (XY) and temporal planes (XT and YT) are utilized to train three different PCANet models. Then, the learned features are fused after reducing their dimensionality using whitening PCA to obtain spatiotemporal feature representation of the action video. Finally, Support Vector Machine (SVM) classifier is applied for action classification process. The proposed method is evaluated on four benchmarks and well-known datasets, namely, Weizmann, KTH, UCF Sports, and YouTube action datasets. The recognition results show that the proposed PCANet-TOP provides discriminative and complementary features using three orthogonal planes and able to achieve promising and comparable results with state-of-the-art methods.

     

基于骨骼特征Hough变换的行为识别研究

为有效地表征人体行为的时空特征,将骨骼特征通过Hough变换后建立人体的动作表示.具体来说,采用OpenPose获取视频帧人体骨骼关键点,之后构建骨骼关节并映射到Hough空间,将骨骼关节轨迹转换为点迹,然后角度和轨迹特征的FV(Fisher vector)编码融合作为线性SVM分类器的输入.在经典公共数据集KTH、Weizmann、KARD和Drone-Action上,实验结果表明Hough变换提升了特征的鲁棒性,提高了人体行为识别的性能.     

Exploring trace transform for robust human action recognition

Machine based human action recognition has become very popular in the last decade. Automatic unattended surveillance systems, interactive video games, machine learning and robotics are only few of the areas that involve human action recognition. This paper examines the capability of a known transform, the so-called Trace, for human action recognition and proposes two new feature extraction methods based on the specific transform. The first method extracts Trace transforms from binarized silhouettes, representing different stages of a single action period. A final history template composed from the above transforms, represents the whole sequence containing much of the valuable spatio-temporal information contained in a human action. The second, involves Trace for the construction of a set of invariant features that represent the action sequence and can cope with variations usually appeared in video capturing. The specific method takes advantage of the natural specifications of the Trace transform, to produce noise robust features that are invariant to translation, rotation, scaling and are effective, simple and fast to create. Classification experiments performed on two well known and challenging action datasets (KTH and Weizmann) using Radial Basis Function (RBF) Kernel SVM provided very competitive results indicating the potentials of the proposed techniques.     

基于人体骨架特征编码的健身动作识别方法

健身动作识别是智能健身系统的核心环节。为了提高健身动作识别算法的精度和速度,并减少健身动作中人体整体位移对识别结果的影响,提出了一种基于人体骨架特征编码的健身动作识别方法。该方法包括三个步骤：首先,构建精简的人体骨架模型,并利用人体姿态估计技术提取骨架模型中各关节点的坐标信息;其次,利用人体中心投影法提取动作特征区域以消除人体整体位移对动作识别的影响;最后,将特征区域编码作为特征向量并输入多分类器进行动作识别,同时通过优化特征向量长度使识别率和速度达到最优。实验结果表明,本方法在包含28种动作的自建健身数据集上的动作识别率为97.24%,证明该方法能够有效识别各类健身动作;在公开的KTH和Weizmann数据集上,所提方法的动作识别率分别为91.67%和90%,优于其他同类型方法。     

一种基于改进PLSA和案例推理的行为识别算法

针对行为人发生的行为因遮挡或者自遮挡可能导致行为歧义性的问题,提出基于改进PLSA和案例推理算法的行为识别方法。该算法既可以克服传统PLSA算法中生成式模型对观察特征序列的独立性假设会导致过拟合的缺点,又可以消除由于遮挡等原因引起的歧义性带来的识别精度降低问题。实验表明该方法能有效地提高人体行为识别准确率。     

基于混合特征的人体动作识别改进算法

运动特征的选择直接影响人体动作识别方法的识别效果.单一特征往往受到人体外观、环境、摄像机设置等因素的影响不同,其适用范围不同,识别效果也是有限的.在研究人体动作的表征与识别的基础上,充分考虑不同特征的优缺点,提出一种结合全局的剪影特征和局部的光流特征的混合特征,并用于人体动作识别.实验结果表明,该算法得到了理想的识别结果,对于Weizmann数据库中的动作可以达到100％的正确识别率.     

Part-based motion descriptor image for human action recognition

This paper presents a novel and efficient framework for human action recognition based on modeling the motion of human body-parts. Intuitively, a collective understanding of human body-part movements can lead to better understanding and representation of any human action. In this paper, we propose a generative representation of the motion of human body-parts to learn and classify human actions. The proposed representation combines the advantages of both local and global representations, encoding the relevant motion information as well as being robust to local appearance changes. Our work is motivated by the pictorial structures model and the framework of sparse representations for recognition. Human body-part movements are represented efficiently through quantization in the polar space. The key discrimination within each action is efficiently encoded by sparse representation for classification. The proposed framework is evaluated on both the KTH and the UCF Sport action datasets and results compared against several state-of-the-art methods.     

Silhouette-based human action recognition using sequences of key poses

In this paper, a human action recognition method is presented in which pose representation is based on the contour points of the human silhouette and actions are learned by making use of sequences of multi-view key poses. Our contribution is twofold. Firstly, our approach achieves state-of-the-art success rates without compromising the speed of the recognition process and therefore showing suitability for online recognition and real-time scenarios. Secondly, dissimilarities among different actors performing the same action are handled by taking into account variations in shape (shifting the test data to the known domain of key poses) and speed (considering inconsistent time scales in the classification). Experimental results on the publicly available Weizmann, MuHAVi and IXMAS datasets return high and stable success rates, achieving, to the best of our knowledge, the best rate so far on the MuHAVi Novel Actor test.     

基于局部时空特征的人体行为软分类识别

针对全局运动特征难以准确提取的问题,本文采用局部时空特征对人体行为进行表征。针对传统词袋中硬分类的方法量化误差大的不足,本文借鉴模糊聚类的思想,提出软分类的方法。根据兴趣点检测算法从视频中提取出视觉词汇,用K means算法对其进行聚类,建立码本。在计算分类特征时,首先计算待分类视觉词汇到码本中各个码字的距离,根据距离计算这个视觉词汇隶属于各个码字的概率,最后统计得到每个视频中各码字出现的频率。在Weizmann和KTH数据库对本文提出的人体行为识别算法进行验证,Weizmann库的识别率比传统的词袋算法提高8%,KTH库的识别率比传统的词袋算法提高9%,因此本文提出的算法能更有效地对人体行为进行识别。     

基于局部时空特征方向加权的人体行为识别

目的 对人体行为的描述是行为识别中的关键问题,为了能够充分利用训练数据从而保证特征对行为的高描述性,提出了基于局部时空特征方向加权的人体行为识别方法。方法 首先,将局部时空特征的亮度梯度特征分解为3个方向(X、Y、Z)分别来描述行为, 通过直接构造视觉词汇表分别得到不同行为3方向特征描述子集合的标准视觉词汇码本,并利用训练视频得到每个行为的标准3方向词汇分布;进而,根据不同行为3方向特征描述子集合的标准视觉词汇码本,分别计算测试视频相应的3方向的词汇分布,并利用与各行为标准3方向词汇分布的加权相似性度量进行行为识别;结果 在Weizmann数据库和KTH数据库中进行实验,Weizmann数据库中的平均识别率高达96.04%,KTH数据库中的平均识别率也高达96.93%。结论 与其他行为识别方法相比可以明显提高行为平均识别率。     

Human action recognition using boosted EigenActions

This paper proposes a boosting EigenActions algorithm for human action recognition. A spatio-temporal Information Saliency Map (ISM) is calculated from a video sequence by estimating pixel density function. A continuous human action is segmented into a set of primitive periodic motion cycles from information saliency curve. Each cycle of motion is represented by a Salient Action Unit (SAU), which is used to determine the EigenAction using principle component analysis. A human action classifier is developed using multi-class Adaboost algorithm with Bayesian hypothesis as the weak classifier. Given a human action video sequence, the proposed method effectively locates the SAUs in the video, and recognizes the human actions by categorizing the SAUs. Two publicly available human action databases, namely KTH and Weizmann, are selected for evaluation. The average recognition accuracy are 81.5% and 98.3% for KTH and Weizmann databases, respectively. Comparative results with two recent methods and robustness test results are also reported.     

基于多特征融合的运动人体行为识别

提出了一种基于特征级融合的运动人体行为识别方法。应用背景差分法和阴影消除技术获得运动人体区域和人体轮廓;采用R变换提取人体区域特征,采用小波描述子提取人体轮廓特征;然后将这两种具有一定互补性的特征采用K-L变换进行融合,得到一个分类能力更强的特征;最后,在传统支持向量机的基础上,结合模糊聚类技术和决策树构建多级二叉树分类器,从而实现行为多类分类。该方法在Weizmann行为数据库上进行了实验,实验结果表明所提出的识别方法具有较高的识别性能。     

面向智能监控的行为识别

目的 为了进一步提高智能监控场景下行为识别的准确率和时间效率,提出了一种基于YOLO（you only look once：unified,real-time object detection）并结合LSTM（long short-term memory）和CNN（convolutional neural network）的人体行为识别算法LC-YOLO（LSTM and CNN based on YOLO）。方法 利用YOLO目标检测的实时性,首先对监控视频中的特定行为进行即时检测,获取目标大小、位置等信息后进行深度特征提取;然后,去除图像中无关区域的噪声数据;最后,结合LSTM建模处理时间序列,对监控视频中的行为动作序列做出最终的行为判别。结果 在公开行为识别数据集KTH和MSR中的实验表明,各行为平均识别率达到了96.6%,平均识别速度达到215 ms,本文方法在智能监控的行为识别上具有较好效果。结论 提出了一种行为识别算法,实验结果表明算法有效提高了行为识别的实时性和准确率,在实时性要求较高和场景复杂的智能监控中有较好的适应性和广泛的应用前景。