A real-time system for motion retrieval and interpretation

This paper proposes a new examplar-based method for real-time human motion recognition using Motion Capture (MoCap) data. We have formalized streamed recognizable actions, coming from an online MoCap engine, into a motion graph that is similar to an animation motion graph. This graph is used as an automaton to recognize known actions as well as to add new ones. We have defined and used a spatio-temporal metric for similarity measurements to achieve more accurate feedbacks on classification. The proposed method has the advantage of being linear and incremental, making the recognition process very fast and the addition of a new action straightforward. Furthermore, actions can be recognized with a score even before they are fully completed. Thanks to the use of a skeleton-centric coordinate system, our recognition method has become view-invariant. We have successfully tested our action recognition method on both synthetic and real data. We have also compared our results with four state-of-the-art methods using three well known datasets for human action recognition. In particular, the comparisons have clearly shown the advantage of our method through better recognition rates.     

Averaging of motion capture recordings for movements’ templates generation

In this paper we propose, describe and evaluate the novel motion capture (MoCap) data averaging framework. It incorporates hierarchical kinematic model, angle coordinates’ preprocessing methods, that recalculate the original MoCap recording making it applicable for further averaging algorithms, and finally signals averaging processing. We have tested two signal averaging methods namely Kalman Filter (KF) and Dynamic Time Warping barycenter averaging (DBA). The propose methods have been tested on MoCap recordings of elite Karate athlete, multiple champion of Oyama karate knockdown kumite who performed 28 different karate techniques repeated 10 times each. The proposed methods proved to have not only high effectiveness measured with root-mean-square deviation (4.04 ± 5.03 degrees for KF and 5.57 ± 6.27 for DBA) and normalized Dynamic Time Warping distance (0.90 ± 1.58 degrees for KF and 0.93 ± 1.23 for DBA), but also the reconstruction and visualization of those recordings persists all crucial aspects of those complicated actions. The proposed methodology has many important applications in classification, clustering, kinematic analysis and coaching. Our approach generates an averaged full body motion template that can be practically used for example for human actions recognition. In order to prove it we have evaluated templates generated by our method in human action classification tasks using DTW classifier. We have made two experiments. In first leave - one - out cross - validation we have obtained 100% correct recognitions. In second experiment when we classified recordings of one person using templates of another recognition rate 94.2% was obtained.     

Forward non-rigid motion tracking for facial MoCap

For the existing motion capture (MoCap) data processing methods, manual interventions are always inevitable, most of which are derived from the data tracking process. This paper addresses the problem of tracking non-rigid 3D facial motions from sequences of raw MoCap data in the presence of noise, outliers and long time missing. We present a novel dynamic spatiotemporal framework to automatically solve the problem. First, based on a 3D facial topological structure, a sophisticated non-rigid motion interpreter (SNRMI) is put forward; together with a dynamic searching scheme, it cannot only track the non-missing data to the maximum extent but recover missing data (it can accurately recover more than five adjacent markers under long time (about 5 seconds) missing) accurately. To rule out wrong tracks of the markers labeled in open structures (such as mouth, eyes), a semantic-based heuristic checking method was raised. Second, since the existing methods have not taken the noise propagation problem into account, a forward processing framework is presented to solve the problem. Another contribution is the proposed method could track facial non-rigid motions automatically and forward, and is believed to greatly reduce even eliminate the requirements of human interventions during the facial MoCap data processing. Experimental results proved the effectiveness, robustness and accuracy of our system.     

基于混合式协同训练的人体动作识别算法研究

人体动作识别是计算机视觉研究中备受关注的课题。现有的动作识别方法大多属于监督学习,需要大量的有标记数据来训练识别模型。然而,在现实应用中有标记的数据成本较高,而无标记数据很容易获取。提出一种基于混合式协同训练的新型人体动作识别算法——Co-KNN-SVM,该算法利用动作识别领域不同类型的方法来构建基分类器,并进行迭代的相互训练以提高泛化性能,可以降低标注成本,并实现不同识别方法的优势互补。此外,还改进了协同训练中对伪标记数据的选择方法和迭代训练策略,有效控制了伪标记数据的噪声影响,提高了协同训练的识别效果。实验结果表明,所提算法可以有效地识别视频中的人体动作。     

MEMS热电堆传感器的红外探测系统

为了实现远距离红外目标的运动和静止状态的识别,设计了基于热电堆红外传感器的红外探测系统,系统包括梯析(GRIN)透镜、微系统(MEMS)热电堆传感器、信号调理电路、数据采集电路和识别算法.探测结果表明:在相同光路系统的情况下,探测系统实现了比热释电系统更远的探测距离,实现了动态目标和静态目标的识别,并基于探测目标温度、辐射面积的区别,实现了人、车辆和其他红外目标的分类识别,可为红外目标的探测识别提供一种新的解决方案.     

分层树结构字典编码的行为识别

目的 基于学习字典的稀疏编码能够自适应地表示信号。然而,传统学习字典的原子之间缺少关联,信号的相似性在编码后缺失。考虑到结构化稀疏表示的鲁棒性和判别性能力,结构化字典的构建成为一个重要的任务。方法 依据标准的凸优化字典学习算法,引入数据点编码路径的约束（由上层原子激活的索引规划下层的索引）,构思了一种树结构字典学习框架。结果 实验结果表明,局部描述符的稀疏表示具有较好的鲁棒性和判别性,同时在KTH数据库上人体行为识别实验与其他类似文献方法相比获得了较高的识别精度,其中,时空梯度方向直方图（HOG3D）的编码识别结果达到97.99%。结论 通过实验结果,观察到采用本文构建的字典编码信号具有较好的鲁棒性和判别性,更好的适合分类任务。     

分级语音识别研究

分级识别的策略在模式识别领域中提出相当长的时间了。尽管人类可以训练地使用这个策略进行识别,但对语音识别而言,缺少一个有效的系统化的方法来实现它。本文给出了我们最近在这方面做的一些研究工作,使用了子空间划分原理来实现一个分级识别器,并用树型结构来组织多个识别器。实验结果表明,该方法与传统方法相比,误识率降低10%。我们将在未来的研究工作中,测试全部汉语音节,并将该方法扩展到连续语音识别。     

基于视觉的人体行为识别算法研究综述*

人体行为识别应用广泛,是人工智能领域研究的热点问题,针对人体行为识别算法进行归纳总结,具有很重要的参考价值。以行为识别为核心,同时包含数据集、动作分割等内容。引言部分主要讲述人体行为识别的基础流程,数据集部分归纳了人体行为识别常用的数据集,动作分割方法总结了时域分割的发展现状和常用的方法,传统方法讲解了人体行为识别比较经典的方法,深度学习方法归纳了人体行为识别最新最热的深度学习方法。引入了动作分割,再结合行为识别,能够实现连续的人体行为识别,使得行为识别适用于实际场景,而不再是对经过人工剪辑好的单个视频进行识别,这在实际应用中意义重大。     

改进的HOG和Gabor,LBP性能比较

为了实现复杂环境下的人脸特征有效表达,提出一种改进的梯度方向直方图(HOG)人脸识别方法.首先以人脸图像网格作为采样窗口并在其上提取HOG特征;然后将所有网格HOG特征向量进行组合,实现整个人脸特征表达;最后采用最近邻分类器进行识别.另外,比较了该方法与Gabor小波和局部二值模式(LBP)2种著名的人脸局部特征表示方法的优劣.实验结果表明,在调优的HOG参数下,在具有光照和时间环境等复杂变化的FERET人脸库中,较少维数的HOG特征比LBP特征有更好的表现,而且HOG特征提取时间和特征向量维数比Gabor小波方法更具有优势.     

Linguistic properties based on American Sign Language isolated word recognition with artificial neural networks using a sensory glove and motion tracker

Sign language (SL), which is a highly visual–spatial, linguistically complete, and natural language, is the main mode of communication among deaf people. Described in this paper are two different American Sign Language (ASL) word recognition systems developed using artificial neural networks (ANN) to translate the ASL words into English. Feature vectors of signing words taken at five time instants were used in the first system, while histograms of feature vectors of signing words were used in the second system. The systems use a sensory glove, Cyberglove™, and a Flock of Birds^® 3-D motion tracker to extract the gesture features. The finger joint angle data obtained from strain gauges in the sensory glove define the hand shape, and the data from the tracker describe the trajectory of hand movement. In both systems, the data from these devices were processed by two neural networks: a velocity network and a word recognition network. The velocity network uses hand speed to determine the duration of words. Signs are defined by feature vectors such as hand shape, hand location, orientation, movement, bounding box, and distance. The second network was used as a classifier to convert ASL signs into words based on features or histograms of these features. We trained and tested our ANN models with 60 ASL words for a different number of samples. These methods were compared with each other. Our test results show that the accuracy of recognition of these two systems is 92% and 95%, respectively.     

基于深度数据的人体动作识别方法

本文提出了一个基于流形学习的动作识别框架,用来识别深度图像序列中的人体行为。本文从Kinect设备获得的深度信息中评估出人体的关节点信息,并用相对关节点位置差作为人体特征表达。在训练阶段,本文利用Lapacian eigenmaps(LE)流形学习对高维空间下的训练集进行降维,得到低维隐空间下的运动模型。在识别阶段,本文用最近邻差值方法将测试序列映射到低维流形空间中去,然后进行匹配计算。在匹配过程中,通过使用改进的Hausdorff距离对低维空间下测试序列和训练运动集的吻合度和相似度进行度量。本文用Kinect设备捕获的数据进行了实验,取得了良好的效果;同时本文也在MSR Action3D数据库上进行了测试,结果表明在训练样本较多情况下,本文识别效果优于以往方法。实验结果表明本文所提的方法适用于基于深度图像序列的人体动作识别。     

A group of novel approaches and a toolkit for motion capture data reusing

Now more and more motion capture (MoCap) systems are used to acquire realistic and highly detailed motion data which are widely used for producing animations of human-like characters in a variety of applications, such as simulations, video games and animation films. And recently large MoCap databases are available. As a kind of emerging multimedia data, 3D human motion has its own specific data form and standard format. But to the best of our knowledge, only a few approaches have been explored for 3D MoCap data feature representation and reusing. This paper proposes a group of novel approaches for posture feature representation, motion sequence segmentation, key-frame extraction and content-based motion retrieval, which are all very important for MoCap data reusing and benefit to the efficient animation production. To validate these approaches, we set up a MoCap database and implemented a prototype toolkit. The experiments show that the proposed algorithms could achieve the approvable results.     

Gait recognition based on joint distribution of motion angles

Gait as a biometric trait has the ability to be recognized in remote monitoring. In this article, a method based on joint distribution of motion angles is proposed for gait recognition. The new feature of the motion angles of lower limbs is defined and extracted from either 2D video database or 3D motion capture database, and the corresponding angles of right leg and left leg are joined together to work out the joint distribution spectrums. Based on the joint distribution of these angles, we build a feature histogram individually. In the stage of distance measurement, three types of distance vector are defined and utilized to measure the similarity between the histograms, and then a classifier is built to implement the classification. Experiments has been carried out both on CASIA Gait Database and CMU motion capture database, which show that our method can achieve a good recognition performance.     

时空域融合的骨架动作识别与交互研究

在人体骨架结构动作识别方法中,很多研究工作在提取骨架结构上的空间信息和运动信息后进行融合,没有对具有复杂时空关系的人体动作进行高效表达。本文提出了基于姿态运动时空域融合的图卷积网络模型(PM-STFGCN)。对于在时域上存在大量的干扰信息,定义了一种基于局部姿态运动的时域关注度模块(LPM-TAM),用于抑制时域上的干扰并学习运动姿态的表征。设计了基于姿态运动的时空域融合模块(PM-STF),融合时域运动和空域姿态特征并进行自适应特征增强。通过实验验证,本文提出的方法是有效性的,与其他方法相比,在识别效果上具有很好的竞争力。设计的人体动作交互系统,验证了在实时性和准确率上优于语音交互系统。     

Tree-structured image difference for fast histogram and distance between histograms computation

In this paper we present a new method for fast histogram computing and its extension to bin to bin histogram distance computing. The idea consists in using the information of spatial differences between images, or between regions of images (a current one and a reference one), and encoding it into a specific data structure: a tree. The histogram of the current image or of one of its regions is then computed by updating the histogram of the reference one using the temporal data stocked into the tree. With this approach, we never need to store any of the current histograms, except the reference image ones, as a preprocessing step. We compare our approach with the well-known Integral Histogram one, and obtain better results in terms of processing time while reducing the memory footprint. We show theoretically and with experimental results the superiority of our approach in many cases. We also extend our idea to the computation of the Bhattacharyya distance between two histograms, using a similar incremental approach that also avoid current histogram computations: we just need histograms of the reference image, and spatial differences between the reference and the current image to compute this distance using an updating process. Finally, we demonstrate the advantages of our approach on a real visual tracking application using a particle filter framework by improving its correction step computation time.     

基于流形学习的人体动作识别

目的 提出了一个基于流形学习的动作识别框架,用来识别深度图像序列中的人体行为。方法 从Kinect设备获得的深度信息中评估出人体的关节点信息,并用相对关节点位置差作为人体特征表达。在训练阶段,利用LE（Lalpacian eigenmaps）流形学习对高维空间下的训练集进行降维,得到低维隐空间下的运动模型。在识别阶段,用最近邻差值方法将测试序列映射到低维流形空间中去,然后进行匹配计算。在匹配过程中,通过使用改进的Hausdorff距离对低维空间下测试序列和训练运动集的吻合度和相似度进行度量。结果 用Kinect设备捕获的数据进行了实验,取得了良好的效果;同时也在MSR Action3D数据库上进行了测试,结果表明在训练样本较多情况下,本文方法识别效果优于以往方法。结论 实验结果表明本文方法适用于基于深度图像序列的人体动作识别。     

自适应加权完全局部二值模式的表情识别

为了有效地提取局部特征和全局特征以提高表情识别的性能,提出自适应加权的完全局部二值模式（Adaptively Weighted Compound Local Binary Pattern,AWCLBP）的人脸表情识别算法。首先对人脸表情图像进行预处理分离出表情子区域,与此同时生成表情子区域的贡献度图谱（Contribution Map,CM）;然后对表情子区域和整幅表情图像做完全局部二值模式变换提取三种特征（差值符号特征CLBP_S、差值幅值特征CLBP_M、中心像素特征CLBP_C）并连接三种特征生成级联直方图,并根据CM对表情子区域的级联直方图进行加权和整张图像的直方图进行融合;最后用卡方距离和最近邻方法进行分类识别。本算法在JAFFE库上做了实验并和LBP、Gabor小波、活动外观模型进行了比较,验证了本算法的有效性。     

基于轻量级图卷积的人体骨架动作识别方法

视频中的人体动作识别在计算机视觉领域得到广泛关注,基于人体骨架的动作识别方法可以明确地表现人体动作,因此已逐渐成为该领域的重要研究方向之一。针对多数主流人体动作识别方法网络参数量大、计算复杂度高等问题,设计一种融合多流数据的轻量级图卷积网络,并将其应用于人体骨架动作识别任务。在数据预处理阶段,利用多流数据融合方法对4种特征数据流进行融合,通过一次训练就可得到最优结果,从而降低网络参数量。设计基于图卷积网络的非局部网络模块,以捕获图像的全局信息从而提高动作识别准确率。在此基础上,设计空间Ghost图卷积模块和时间Ghost图卷积模块,从网络结构上进一步降低网络参数量。在动作识别数据集NTU60 RGB+D和NTU120 RGB+D上进行实验,结果表明,与近年主流动作识别方法ST-GCN、2s AS-GCN、2s AGCN等相比,基于该轻量级图卷积网络的人体骨架动作识别方法在保持较低网络参数量的情况下能够取得较高的识别准确率。     

A multi-level approach to highly efficient recognition of Chinese spam short messages

The problem of spam short message (SMS) recognition involves many aspects of natural language processing. A good solution to solving the problem can not only improve the quality of people experiencing the mobile life, but also has a positive role on promoting the analysis of short text occurring in current mobile applications, such as Webchat and microblog. As spam SMSes have characteristics of sparsity, transformation and real-timedness, we propose three methods at different levels, i.e., recognition based on symbolic features, recognition based on text similarity, and recognition based on pattern matching. By combining these methods, we obtain a multi-level approach to spam SMS recognition. In order to enrich the pattern base to reduce manual labor and time, we propose a quasi-pattern learning method, which utilizes quasi-pattern matching results in the pattern matching process. The method can learn many interesting and new patterns from the SMS corpus. Finally, a comprehensive analysis indicates that our spam SMS recognition approach achieves a precision rate as high as 95.18%, and a recall rate of 95.51%.