基于主题隐马尔科夫模型的人体异常行为识别

针对基于监控视频的人体异常行为识别问题,提出了基于主题隐马尔科夫模型的人体异常行为识别方法,即通过无任何人工标注的视频训练集自动学习人体行为模型,并能够应用学到的人体行为模型实时检测异常行为和识别正常行为。这一方法主要围绕"低层视频表示-中层语义行为建模-高层语义分类"3个方面进行:1)基于时-空间兴趣点构建了一种紧凑的和有效的视频表示方法。2)提出一种新颖的语义主题模型(Topic Model,TM)——主题隐马尔科夫模型(Topic Hidden Markov Model,THMM),它能够自然地分组视频中检测到的人体行为。主题隐马尔科夫模型基于已有的马尔科夫模型和主题模型构造,不但聚类运动词汇成简单动作,而且聚类简单动作成全局行为,同时建模了行为时间上的相关性。THMM是一个4层贝叶斯主题模型,它将视频序列建模为行为的马尔科夫链,同时行为是视频序列中某些视频剪辑(Clip)的概率分布;将视频剪辑建模为动作的随机组合,同时动作是视频剪辑中运动词汇的概率分布。克服了传统隐马尔科夫模型和主题模型在人体复杂行为建模过程中精度、鲁棒性和计算效率上的不足。3)提出运行时累积的异常性测度及其在线异常行为检测方法和基于在线似然比检验(Likelihood Ratio Test,LRT)的实时正常行为分类方法,从而克服了实时行为识别过程中由于缺乏充分的视觉证据而引发的行为类型歧义,能完较好地完成监控场景中实时异常行为检测和在线正常行为识别的任务。取自实际监控场景的实验数据集上的实验结果证明了本方法的有效性。     

基于傅里叶-隐马尔科夫模型的人体行为识别

研究人行为识别的正确性,针对提高对人体行为序列图像进行识别的能力,隐马尔科夫模型(HMM)是一种统计分析模型,具有时序模式分析能力.为了增加图像信息的有效性,提出了一种傅里叶与隐马尔科夫模型相结合的方法人体行为识别方法.通过获得各种人体行为的二值图像序列,对待识别的序列图像提取具有旋转、平移和尺度不变性的傅立叶特征,采用了一种基于中心距的傅里叶描述子,利用改进的隐马尔科夫模型对提取的特征向量进行分类,得到人体行为的识别结果.试验结果表明,系统的识别率与HMM的状态数和观察值数有关,方法是有效且可行的,设计适当的HMM分类器能使系统的识别率达到90%以上,实际应用效果满足要求.     

基于高斯混合隐马尔科夫模型的自由换道识别

驾驶辅助系统被认为是解决交通安全问题的有效手段, 开发驾驶辅助系统的基础是对车辆的行为进行准确的识别, 以应用于车辆安全预警, 路径规划, 智能导航等方面. 目前存在的基于支持向量机模型, 隐马尔科夫模型, 卷积神经网络等行为识别方法还存在计算量与精度平衡的问题. 本文结合了隐马尔科夫模型与高斯混合模型, 提出了高斯混合隐马尔科夫模型, 利用美国联邦公路管理局NGSIM数据集对此方法进行了实验验证, 结果表明该方法对自由换道行为识别具有较高的精度. 本文还对高斯混合隐马尔科夫模型的实验参数进行了优化, 以期达到最好的识别效果, 为未来智能驾驶的车辆行为识别提供了参考.     

基于静电信号的人体动作识别

提出一种通过检测人体行为动作产生的静电信号进行人体动作识别的方法.在分析人体荷电特性的基础上,设计静电信号检测系统采集被测人员的5种典型动作（行走、踏步、坐下、拿取物品、挥手）的静电感应信号.对采集的5种动作的静电信号进行特征参量提取和显著性差异分析,优化用于分类的特征参数.基于Weka平台使用3种分类算法（支持向量机、决策树C4.5和随机森林）分别对采集到的250组样本数据通过10折交叉验证进行了分类识别,结果显示随机森林算法的识别效果最好,正确率可达99.6%.研究表明本文提出的单人环境下基于人体静电信号的动作分类识别方法能够有效地对典型人体动作进行识别.     

井下人员人脸识别方法研究

针对光照突变、视角变化以及遮挡等因素容易导致井下人员人脸识别精确度低的问题,提出了一种隐马尔科夫模型下基于SIFT特征的人脸识别方法。该方法将提取的SIFT特征作为隐马尔科夫模型的训练样本,经训练建立精准匹配库,对人脸部位和非人脸部位的SIFT特征进行分类,从而实现人脸图像的识别。实验结果表明,该方法识别速度快、准确率高,平均识别准确率达97.14%。     

一种基于消除能量偏差的双层环境声识别模型

相比Mel倒谱系数(MFCC),基于能量偏差移除和幂函数的声音特征(PNCC)具有较强的抗噪能力.首先,将PNCC和MFCC组成混合特征矩阵,在隐马尔科夫模型(HMM)、高斯混合模型(GMM)和支持向量机(SVM)下对混合特征和传统特征做对比实验.其次,先选取实验结果较好的HMM模型过滤测试样本,再分别选取GMM和SVM做二次分类,并测试两种双层模型的识别正确率.结果表明在噪声环境下使用HMM/GMM双层模型和混合特征可取得较好的识别效果.     

基于空间分布特征的人体动作动态建模识别

人体动作的表面特征和动态特征无统一性描述,导致无法精确地区分人体动作.文中提出基于空间分布特征的人体动作动态建模识别方法(DMRSD).利用相对极坐标划分特征的空间区域,统计局部区域非零信息点数目,形成空间分布特征描述表面特征.然后利用自回归滑动平均动态模型建模空间特征序列,形成模型参数特征描述动态时间结构特征.最后通过各参数特征的相似矩阵线性关系假设,结构融合表面特征和动态运动特征,形成统一性描述特征.用最近邻识别人体动作.在Weizmann和KTH库中比对当前方法的识别结果,文中方法获得较好的识别性能.     

基于Kinect的运动训练辅助系统

针对当下人们自主运动训练时出现的动作不标准且无人监督指导的问题，设计一种基于Kinect传感器的运动训练辅助系统。利用Kinect捕捉人体的关节点坐标提取特征，采用动态时间规整算法实现运动动作的识别，将模板动作与实时动作进行匹配，根据匹配的程度给予评价。实验结果表明该系统对于运动动作的平均正确识别率达到91.25%，正确评价率达到95.9%，能及时有效地反馈评价，起到一定运动训练辅助作用。     

基于流形学习与隐条件随机场的人体动作识别

提出了一种基于流形学习与隐条件随机场(hidden conditional random fields,简称HCRF)的动作识别方法.算法提取人体剪影作为输入特征,采用有监督的保持邻域嵌入(neighborhood preserving embedding,简称NPE)的子空间学习算法获得高维运动特征的低维流形表示,基于HCRF建模运动特征与动作语义之间的映射关系.在降维过程中,通过保持数据的局部邻接关系,NPE可以获取动作特征在低维流形空间上的本质分布特性.与HMM(hidden Markov model)等产生式模型相比,HCRF侧重从样本数据中抽取共有特征以获取正确的分类边界,不需要假定观测过程条件独立,可以更加自然地对动作的时空邻域关系进行建模.实验结果表明,即便对于特征差异较大或存在噪声干扰的动作序列,算法也能取得较好的识别效果.     

基于DBN-HMM的人体动作识别

动作识别使得机器能够对人体动作的意图进行判别理解，进而实现高效的人机交互。提出一种肢体角度模型，实现在三维空间中对人体动作进行表示，该模型具有一定的不变性，计算复杂度低。针对传统的基于混合高斯的隐马尔可夫模型（GMM-HMM）的动作识别，提出深度置信网络模型（DBN）和隐马尔可夫模型相结合的动作识别模型，构建了一种非线性的基于条件限制玻尔兹曼机（CRBM）的DBN深度学习模型，深层次结构使其建模能力更强，且能够结合历史信息建模，更适用于动作识别。实验表明该算法具有较高的识别结果。     

基于多尺度特征的双层隐马尔可夫模型及其在行为识别中的应用

借鉴人类视觉感知所具有的多尺度、多分辨性的特性,针对智能视频监控系统的人体运动行为识别,提出了一种基于多尺度特征的双层隐马尔可夫模型.根据人体行为关键姿态数确定HMM的状态数目,发掘人体运动行为隐藏的多尺度结构间的关系,将运动轨迹和人体姿态边缘小波矩2个不同尺度特征应用于2层HMM,提供更为丰富的行为尺度间的相关信息.分别用Weizmann人体行为数据库和自行拍摄的室内视频,对人体运动行为识别进行仿真实验,结果表明,五状态HMM模型更符合人体运动行为特点,基于多尺度特征的五状态双层隐马尔可夫模型具有较高的识别率.     

Bilateral remote teaching and autonomous task execution with task progress feedback

This paper describes an approach to estimating the progress in a task executed by a humanoid robot and to synthesizing motion based on the current progress so that the robot can achieve the task. The robot observes a human performing whole body motion for a specific task, and encodes these motions into a hidden Markov model (HMM). The current observation is compared with the motion generated by the HMM, and the task progress can be estimated during the robot performing the motion. The robot subsequently uses the estimate of the task progress to generate a motion appropriate to the current situation with the feedback rule. We constructed a bilateral remote control system with humanoid robot HRP-4 and haptic device Novint Falcon, and we made the humanoid robot push a button. Ten trial motions of pushing a button were recorded for the training data. We tested our proposed approach on the autonomous execution of the pushing motion by the humanoid robot, and confirmed the effectiveness of our task progress feedback method.     

Affective State Estimation for Human–Robot Interaction

In order for humans and robots to interact in an effective and intuitive manner, robots must obtain information about the human affective state in response to the robot's actions. This secondary mode of interactive communication is hypothesized to permit a more natural collaboration, similar to the "body language" interaction between two cooperating humans. This paper describes the implementation and validation of a hidden Markov model (HMM) for estimating human affective state in real time, using robot motions as the stimulus. Inputs to the system are physiological signals such as heart rate, perspiration rate, and facial muscle contraction. Affective state was estimated using a two- dimensional valence-arousal representation. A robot manipulator was used to generate motions expected during human-robot interaction, and human subjects were asked to report their response to these motions. The human physiological response was also measured. Robot motions were generated using both a nominal potential field planner and a recently reported safe motion planner that minimizes the potential collision forces along the path. The robot motions were tested with 36 subjects. This data was used to train and validate the HMM model. The results of the HMM affective estimation are also compared to a previously implemented fuzzy inference engine. [All rights reserved Elsevier].     

Learning,Generation and Recognition of Motions by Reference-Point-Dependent Probabilistic Models

This paper presents a novel method for learning object manipulation such as rotating an object or placing one object on another. In this method, motions are learned using reference-point-dependent probabilistic models, which can be used for the generation and recognition of motions. The method estimates (i) the reference point, (ii) the intrinsic coordinate system type, which is the type of coordinate system intrinsic to a motion, and (iii) the probabilistic model parameters of the motion that is considered in the intrinsic coordinate system. Motion trajectories are modeled by a hidden Markov model (HMM), and an HMM-based method using static and dynamic features is used for trajectory generation. The method was evaluated in physical experiments in terms of motion generation and recognition. In the experiments, users demonstrated the manipulation of puppets and toys so that the motions could be learned. A recognition accuracy of 90% was obtained for a test set of motions performed by three subjects. Furthermore, the results showed that appropriate motions were generated even if the object placement was changed.     

基于多路sEMG时序分析的人体运动模式识别方法

针对主动康复训练中人体运动识别问题,提出了一种基于多路表面肌电（Surface electro myo gram,sEMG）时序特征的人体运动模式识别方法.设计评估类周期sEMG信号波形相似度的方法来对多路sEMG信号进行特征选择;以二维科荷伦自组织竞争网络（Self-organizati on mappingnet,SOM）对多路信息进行编码;最后,建立描述各运动过程多路sEMG时序特征的隐马尔科夫模型（Hidden Markov model,HMM）,基于最大似然估计法对多模型匹配进行综合判决获取识别结果.并在对下肢踏车、椭圆、步行运动模式的识别实验中,相对于经典线性及非线性算法,识别率由72.5%和88.33%提高到91.67%,验证了本文方法的有效性.     

Driving intention recognition and behaviour prediction based on a double-layer hidden Markov model

We propose a model structure with a double-layer hidden Markov model (HMM) to recognise driving intention and predict driving behaviour. The upper-layer multi-dimensional discrete HMM (MDHMM) in the double-layer HMM represents driving intention in a combined working case, constructed according to the driving behaviours in certain single working cases in the lower-layer multi-dimensional Gaussian HMM (MGHMM). The driving behaviours are recognised by manoeuvring the signals of the driver and vehicle state information, and the recognised results are sent to the upper-layer HMM to recognise driving intentions. Also, driving behaviours in the near future are predicted using the likelihood-maximum method. A real-time driving simulator test on the combined working cases showed that the double-layer HMM can recognise driving intention and predict driving behaviour accurately and efficiently. As a result, the model provides the basis for pre-warning and intervention of danger and improving comfort performance.     

Real-time Unsupervised Segmentation of human whole-body motion and its application to humanoid robot acquisition of motion symbols

An interactive loop between motion recognition and motion generation is a fundamental mechanism for humans and humanoid robots. We have been developing an intelligent framework for motion recognition and generation based on symbolizing motion primitives. The motion primitives are encoded into Hidden Markov Models (HMMs), which we call “motion symbols”. However, to determine the motion primitives to use as training data for the HMMs, this framework requires a manual segmentation of human motions. Essentially, a humanoid robot is expected to participate in daily life and must learn many motion symbols to adapt to various situations. For this use, manual segmentation is cumbersome and impractical for humanoid robots. In this study, we propose a novel approach to segmentation, the Real-time Unsupervised Segmentation (RUS) method, which comprises three phases. In the first phase, short human movements are encoded into feature HMMs. Seamless human motion can be converted to a sequence of these feature HMMs. In the second phase, the causality between the feature HMMs is extracted. The causality data make it possible to predict movement from observation. In the third phase, movements having a large prediction uncertainty are designated as the boundaries of motion primitives. In this way, human whole-body motion can be segmented into a sequence of motion primitives. This paper also describes an application of RUS to AUtonomous Symbolization of motion primitives (AUS). Each derived motion primitive is classified into an HMM for a motion symbol, and parameters of the HMMs are optimized by using the motion primitives as training data in competitive learning. The HMMs are gradually optimized in such a way that the HMMs can abstract similar motion primitives. We tested the RUS and AUS frameworks on captured human whole-body motions and demonstrated the validity of the proposed framework.     

Spatio-temporal structure of human motion primitives and its application to motion prediction

This paper proposes a novel approach to structuring behavioral knowledge based on symbolization of human whole body motions, hierarchical classification of the motions, and extraction of the causality among the motions. The motion patterns are encoded into parameters of corresponding Hidden Markov Models (HMMs), where each HMM abstracts the dynamics of motion pattern, and hereafter is referred to as “motion symbol”. The motion symbols allow motion recognition and synthesis. The motion symbols are organized into a hierarchical tree structure representing the property of spatial similarity among the motion patterns, and this tree is referred to as “motion symbol tree”. Seamless motion is segmented into a sequence of motion primitives, each of which is classified as a motion symbol based on the motion symbol tree. The seamless motion results in a sequence of the motion symbols, which is stochastically represented as transitions between the motion symbols by an N-gram model. The motion symbol N-gram model is referred to as “motion symbol graph”. The motion symbol graph extracts the temporal causality among the human behaviors. The integration of the motion symbol tree and the motion symbol graph makes it possible to recognize motion patterns fast and predict human behavior during observation. The experiments on a motion dataset of radio calisthenics and on a large motion dataset provided by CMU motion database validate the proposed framework.