基于时空单词的两人交互行为识别方法

文中提出一种基于时空单词的两人交互行为识别方法,该方法从行为视频中提取丰富的时空兴趣点,基于人体剪影的连通性分析和时空兴趣点的历史信息,把时空兴趣点划分给不同的人体,并在兴趣点样本空间聚类生成时空码本(spatial-temporal codebook).对于给定的时空兴趣点集,通过投票得到表示单人原子行为的时空单词(spatial-temporal words).采用条件随机场模型建模单人原子行为,在两人交互行为的语义建模过程中,人工建立表示领域知识(domain knowledge)的一阶逻辑知识库,并训练马尔可夫逻辑网用以两人交互行为的推理.两人交互行为库上的实验结果证明了该方法的有效性.  

动作识别中局部时空特征的运动表示方法研究

近年来，基于局部时空特征的运动表征方法已被越来越多地运用于视频中的动作识别问题，相关研究人员已经提出了多种特征检测和描述方法，并取得了良好的效果。但上述方法在适应摄像头移动、光照以及穿着变化等方面还存在明显不足。为此，提出了基于时空兴趣点局部时空特征的运动表示方法，实现了基于时空单词的动作识别。首先采用基于Gabor滤波器和Gaussian滤波器相结合的检测算法从视频中提取时空兴趣点，然后抽取兴趣点的静态特征、运动特征和时空特征，并分别对运动进行表征，最后利用基于时空码本的动作分类器对动作进行分类识别。在Weizmann和KTH两个行为数据集进行了测试，实验结果表明：基于时空特征的运动表示能够更好地适应摄像头移动、光照变化以及施动者的穿着和动作差异等环境因素的影响，取得更好的识别效果。  

表情和姿态的双模态情感识别

多模态情感识别是当前情感计算研究领域的重要内容,针对人脸表情和动作姿态开展双模态情感识别研究,提出一种基于双边稀疏偏最小二乘的表情和姿态的双模态情感识别方法.首先,从视频图像系列中分别提取表情和姿态两种模态的空时特征作为情感特征矢量.然后,通过双边稀疏偏最小二乘(BSPLS)的数据降维方法来进一步提取两组模态中的情感特征,并组合成新的情感特征向量.最后,采用了两种分类器来进行情感的分类识别.以国际上广泛采用的FABO表情和姿态的双模态情感数据库为实验数据,并与多种子空间方法(主成分分析、典型相关分析、偏最小二乘回归)进行对比实验来评估本文方法的识别性能.实验结果表明,两种模态融合后相比单模态更加有效,双边稀疏偏最小二乘(BSPLS)算法在几种方法中得到最高的情感识别率.  

一种高效的三维运动检索方法

随着运动捕获设备的普及,大量的运动数据可以直接得到,从而使得大规模的运动数据库的建立成为可能.在此背景下,研究以检索为核心的运动捕获数据处理技术就显得十分重要了.本文提出了一种对运动捕获数据中的人体的各个关节点提取一种基于三维空间变换规律的3D时空特征的方法,并基于时空运动连续性引入了关键空间的概念.针对各关节点时空特征相对保持独立的特性,本文用每个关节点作为索引,并通过数据驱动决策树的学习方法去分析关节点对运动相似的不同影响,最终实现了一个高效的运动检索系统.  

基于可达标识列计时Petri网的时态特征及可控性研究

应用时态逻辑提出计量Ｐｅｔｒｉ网的形式化分析方法，基于可达标识列研究受控系统的时态特征及其可控性与控制不变性，给出控制逻辑存在的充要条件，提出了时态公式分解方法，并讨论了禁止状态避免问题。  

Reconstruction of 3D human body pose from stereo image sequences based on top-down learning

This paper presents a novel method for reconstructing a 3D human body pose from stereo image sequences based on a top-down learning method. However, it is inefficient to build a statistical model using all training data. Therefore, the training data is hierarchically divided into several clusters to reduce the complexity of the learning problem. In the learning stage, the human body model database is hierarchically constructed by classifying the training data into several sub-clusters with silhouette images. The data of each cluster in the bottom level is represented by a linear combination of examples. In the reconstruction stage, the proposed method hierarchically searches a cluster for the best matching silhouette image using a silhouette history image (SHI). Then, the 3D human body pose is reconstructed from a depth image using a linear combination of examples method. By using depth information to reconstruct 3D human body pose, the similar poses in silhouette images are estimated as different 3D human body poses. The experimental results demonstrate that the proposed method is efficient and effective for reconstructing 3D human body poses.  

A tensor-based deep learning framework

This paper presents an unsupervised deep learning framework that derives spatio-temporal features for human–robot interaction. The respective models extract high-level features from low-level ones through a hierarchical network, viz. the Hierarchical Temporal Memory (HTM), providing at the same time a solution to the curse of dimensionality in shallow techniques. The presented work incorporates the tensor-based framework within the operation of the nodes and, thus, enhances the feature derivation procedure. This is due to the fact that tensors allow the preservation of the initial data format and their respective correlation and, moreover, attain more compact representations. The computational nodes form spatial and temporal groups by exploiting the multilinear algebra and subsequently express the samples according to those groups in terms of proximity. This generic framework may be applied in a diverse of visual data, while it has been examined on sequences of color and depth images, exhibiting remarkable performance.  

Detecting bipedal motion from correlated probabilistic trajectories

This paper is about detecting bipedal motion in video sequences by using point trajectories in a framework of classification. Given a number of point trajectories, we find a subset of points which are arising from feet in bipedal motion by analysing their spatio-temporal correlation in a pairwise fashion. To this end, we introduce probabilistic trajectories as our new features which associate each point over a sufficiently long time period in the presence of noise. They are extracted from directed acyclic graphs whose edges represent temporal point correspondences and are weighted with their matching probability in terms of appearance and location. The benefit of the new representation is that it practically tolerates inherent ambiguity for example due to occlusions. We then learn the correlation between the motion of two feet using the probabilistic trajectories in a decision forest classifier. The effectiveness of the algorithm is demonstrated in experiments on image sequences captured with a static camera, and extensions to deal with a moving camera are discussed.  

Structured learning of local features for human action classification and localization

Human action recognition is a promising yet non-trivial computer vision field with many potential applications. Current advances in bag-of-feature approaches have brought significant insights into recognizing human actions within complex context. It is, however, a common practice in literature to consider action as merely an orderless set of local salient features. This representation has been shown to be oversimplified, which inherently limits traditional approaches from robust deployment in real-life scenarios. In this work, we propose and show that, by taking into account global configuration of local features, we can greatly improve recognition performance. We first introduce a novel feature selection process called Sparse Hierarchical Bayes Filter to select only the most contributive features of each action type based on neighboring structure constraints. We then present the application of structured learning in human action analysis. That is, by representing human action as a complex set of local features, we can incorporate different spatial and temporal feature constraints into the learning tasks of human action classification and localization. In particular, we tackle the problem of action localization in video using structured learning with two alternatives: one is Dynamic Conditional Random Field from probabilistic perspective; the other is Structural Support Vector Machine from max-margin point of view. We evaluate our modular classification-localization framework on various testbeds, in which our proposed framework is proven to be highly effective and robust compared against bag-of-feature methods.  

基于时空深度特征的人体行为识别算法

针对底层局部时空特征数量少以及中层特征表达能力弱的问题，结合时空深度特征，提出一种人体行为识别算法。依据运动剧烈区域在行为识别中提供更多判别信息的思想，利用视频图像的深度信息确定人体运动显著性区域，通过计算区域内光流特征作为度量区域活跃度的能量函数，依据能量函数对运动显著性区域进行高斯取样，使样本点分布于运动剧烈区域。将采集到的样本点作为动作底层特征描述人体行为，结合词袋模型，采用支持向量机分类器对行为进行识别。实验结果表明，在SwustDepth数据集中，基于时空深度特征的人体行为识别算法的平均行为识别准确率达到92%，且具有较高的鲁棒性。