基于Segmental-DTW的无监督行为序列分割

行为序列分割是行为分析与识别中最初始、最基础的一个步骤.提出了一种无监督的行为序列分割算法,主要步骤包括:(1)采用等长有重叠的时间窗口对视频序列进行粗分割;(2)将粗分割的视频段两两作比较,通过Segmental-DTW算法分割出两个视频段中最相似的行为片断;(3)将行为片断的相似性转化为邻接图表示,通过图聚类方法对分割出的行为片断进行聚类.该算法采用了从粗到细的分割思想,能够准确地分割出视频序列中大量出现的行为的片断,并将相同行为的片断聚为一类.分割结果可以直接用于行为建模和识别.实验结果也表明了分割出的行为片断具有较好的代表性和有效性.     

一种基于序列匹配的行为识别方法

为解决智能监控环境下的异常行为识别,提出一种基于序列匹配的人的行为识别算法.对于输入序列采用改进的背景减法获取人体侧影并归一化.获取人体对象的侧影的轮廓线,使用傅立叶描绘子描述人体行为的特征,并在多个数据集上验证了算法的有效性.     

基于复合时空特征的人体行为识别方法

为了有效地表征人体行为中的姿势信息和运动信息,提高行为识别算法的准确率,提出一种融合三维方向梯度直方图特征与光流直方图特征的复合时空特征,并利用其进行人体行为识别.首先采用复合时空特征综合描述三维时空局部区域的像素分布和像素变化;然后构建复合时空特征词典,并根据该特征词典完成对人体行为序列特征集合的描述;最后采用主题模型构建人体行为识别算法,对行为序列中提取的复合时空特征进行分类,实现人体行为的识别.实验结果表明:该方法能有效地提高人体行为识别准确率.     

基于融合模型的人体行为识别方法

可穿戴设备的人体行为识别研究通常是提取传感器数据的特征值,然后结合分类算法识别人体行为动作。针对特征提取与分类器问题,提出一种融合模型的人体行为识别方法(HBRM)。首先将加速度传感器采集的数据转换为二维张量格式,然后结合卷积神经网络(CNN)提取张量的特征,同时考虑到人体行为动作在时间序列上前后具有较强的关联性,提出利用长短期记忆(LSTM)网络进行人体行为动作的识别。由于卷积神经网络在特征提取方面具有较好的性能,且长短期记忆模型擅长处理时间序列问题,因此将这两种模型进行融合理论上具有较好的效果。在WISDM数据集上进行实验,结果表明:该方法对六种人体行为动作的平均识别率达到了96.95%。     

人体行为识别的Markov随机游走半监督学习方法

针对目前人体行为识别方法大都需要大量有标注样本的问题,提出一种基于Markov随机游走的半监督人体行为识别算法.首先提取序列图像各帧人体区域的网格统计特征,再采用基于对手惩罚策略的竞争神经网络对其进行聚类和编码,将图像序列表示的人体行为变换为符号序列;然后根据行为之间的归一化编辑距离建立已标注行为、未标注行为和类别之间的Markov链,并采用Markov随机游走过程来预测未标注行为的类别;最后采用最大后验概率准则对观测到的未知行为进行分类.对Weizmann数据集中人体行为的识别实验结果表明,该方法是一种有效的人体行为识别方法,在标注样本很少的情况下平均识别精度可以超过80％.     

基于多流卷积神经网络的行为识别

人体行为识别与人体姿态有很强的相关性,由于许多公开的行为识别的数据集并未提供相关姿态数据,因此很少有将姿态数据进行训练并与其它模态进行融合的识别方法.针对当今主流基于深度学习的人体行为识别方法采用RGB与光流融合的现状,提出一种融合人体姿态特征的多流卷积神经网络人体行为识别算法.首先,用姿态估计算法从包含人的静态图片生成人体关键点数据,并对关键点连接构建姿态;其次,分别将RGB、光流、姿态数据对多流卷积神经网络进行训练,并进行分数融合;最后,在UCF101与HMDB51数据集进行了大量的消融,识别精度等方面的实验研究.实验结果表明,融合了姿态图像的多流卷积神经网络在UCF101与HMDB51数据集的实验精度分别提高了2.3％和3.1％.实验结果验证了提出算法的有效性.     

基于改进Canny算子和神经网络的人体行为识别模型

为了提高了人体行为识别的正确率,提出了一种基于改进Canny算子和神经网络的人体行为识别模型（ICanny-RBF）。采用改进Canny算子对人体行为图像进行预处理,提取人体行为轮廓,提取7个不变矩特征作为RBF神经网络的输入向量,训练出能够识别人体行为的RBF神经网络模型,并采用取k-means算法确定RBF神经网络聚类中心,采用Weizmann数据集进行仿真实验。仿真结果表明,与传统方法相比,提出的ICanny-RBF模型提高了人体行为的识别正确率。     

一种基于关节点信息的人体行为识别新方法

为了提高机器人服务的主动性与智能性,使用Kinect体感设备获取人体的关节点数据解决人体行为识别问题.首先,利用Kinect采集人体关节点坐标,构造用于表示人体结构的3维空间向量,然后计算结构向量之间的角度和向量模的比值,进行人体姿态描述,同时以一段时间内连续的姿态序列作为行为表示特征量,最后选用动态时间规整(DTW)算法计算测试行为模板与参考行为模板之间的相似度以实现行为识别.实验结果表明,选用的行为表示特征量具有旋转与平移不变性.另外,对人在日常生活中的6种行为进行了识别实验,结果表明本文的行为识别算法可以取得较好的识别效果.     

多聚点子空间下的时空信息融合及其在行为识别中的应用

基于深度序列的人体行为识别,一般通过提取特征图来提高识别精度,但这类特征图通常存在时序信息缺失的问题.针对上述问题,本文提出了一种新的深度图序列表示方式,即深度时空图(Depth space time maps, DSTM). DSTM降低了特征图的冗余度,弥补了时序信息缺失的问题.本文通过融合空间信息占优的深度运动图(Depth motion maps,DMM)与时序信息占优的DSTM,进行高精度的人体行为研究,并提出了多聚点子空间学习(Multi-center subspace learning, MCSL)的多模态数据融合算法.该算法为各类数据构建多个投影聚点,以此增大样本的类间距离,降低了投影目标区域维度.本文在MSR-Action3D数据集和UTD-MHAD数据集上进行人体行为识别.最后实验结果表明,本文方法相较于现有人体行为识别方法有着较高的识别率.     

基于图嵌入线性拓展方法的人体动作识别研究

采用图嵌入降维的方法对人侧影轮廓数据进行降维处理并用来识别人的行为动作.给定一个动作的图像序列,提取序列中每帧中人的侧影轮廓信号并用之表征人体运动,利用两种图嵌入法将提取的时变轮廓信号投影到低维空间,采用Hausdroff距离测量运动的相似性并在最近邻框架下识别人的动作.为验证算法的有效性,采用留一法和统计方法两种测试方法对五类人体常见动作(走、跑、拍手、挥手和拳击)进行测试.实验结果表明,方法不仅有很好的分类性能,而且能有效的降低了计算量.     

Real-time Unsupervised Segmentation of human whole-body motion and its application to humanoid robot acquisition of motion symbols

An interactive loop between motion recognition and motion generation is a fundamental mechanism for humans and humanoid robots. We have been developing an intelligent framework for motion recognition and generation based on symbolizing motion primitives. The motion primitives are encoded into Hidden Markov Models (HMMs), which we call “motion symbols”. However, to determine the motion primitives to use as training data for the HMMs, this framework requires a manual segmentation of human motions. Essentially, a humanoid robot is expected to participate in daily life and must learn many motion symbols to adapt to various situations. For this use, manual segmentation is cumbersome and impractical for humanoid robots. In this study, we propose a novel approach to segmentation, the Real-time Unsupervised Segmentation (RUS) method, which comprises three phases. In the first phase, short human movements are encoded into feature HMMs. Seamless human motion can be converted to a sequence of these feature HMMs. In the second phase, the causality between the feature HMMs is extracted. The causality data make it possible to predict movement from observation. In the third phase, movements having a large prediction uncertainty are designated as the boundaries of motion primitives. In this way, human whole-body motion can be segmented into a sequence of motion primitives. This paper also describes an application of RUS to AUtonomous Symbolization of motion primitives (AUS). Each derived motion primitive is classified into an HMM for a motion symbol, and parameters of the HMMs are optimized by using the motion primitives as training data in competitive learning. The HMMs are gradually optimized in such a way that the HMMs can abstract similar motion primitives. We tested the RUS and AUS frameworks on captured human whole-body motions and demonstrated the validity of the proposed framework.     

一种基于隐马尔科夫模型的步态识别算法

步态是指人体走路时的姿态,步态识别是近年来生物特征识别领域一个备受关注的研究方向。步态阶段的区分是步态识别的重要内容。以隐马尔科夫模型(HMM)为基础,基于安装在膝关节的编码器和大腿部的加速度传感器,在外骨骼辅助行走中识别步态的不同阶段。首先进行数据预处理和特征提取;其次对隐马尔科夫步态识别算法进行设计,包括结构的建立、参数的训练和最终的识别;最后对性能进行评估,总体正确率达到91.06%,说明HMM用于步态阶段识别具有较好的性能。     

Independent shape component-based human activity recognition via Hidden Markov Model

In proactive computing, human activity recognition from image sequences is an active research area. In this paper, a novel human activity recognition method is proposed, which utilizes Independent Component Analysis (ICA) for activity shape information extraction from image sequences and Hidden Markov Model (HMM) for recognition. Various human activities are represented by shape feature vectors from the sequence of activity shape images via ICA. Based on these features, each HMM is trained and activity recognition is achieved by the trained HMMs of different activities. Our recognition performance has been compared to the conventional method where Principal Component Analysis (PCA) is typically used to derive activity shape features. Our results show that superior recognition is achieved with the proposed method especially for activities (e.g., skipping) that cannot be easily recognized by the conventional method. Furthermore, by employing Linear Discriminant Analysis (LDA) on IC features, the recognition results further improved significantly in the recognition performance.     

三维人体运动特征可视化与交互式运动分割

为了从运动序列中提取不同类型的运动片段进行人体动画创作,提出一种三维人体运动数据可视化与交互式分割技术.首先,采用人体各主要骨骼夹角作为对原始运动数据的几何特征表示,并提出一种启发式方法自动检测其潜在分割点,最后将提取的运动特征可视化并使用可交互用户界面对其进行精确分割.实验结果表明。该特征表示及交互式分割方法能够更加方便、准确、高效地对包含多种类型的长序列三维人体运动数据进行分割.     

Multi-body Factorization with Uncertainty: Revisiting Motion Consistency

Dynamic analysis of video sequences often relies on the segmentation of the sequence into regions of consistent motions. Approaching this problem requires a definition of which motions are regarded as consistent. Common approaches to motion segmentation usually group together points or image regions that have the same motion between successive frames (where the same motion can be 2D, 3D, or non-rigid). In this paper we define a new type of motion consistency, which is based on temporal consistency of behaviors across multiple frames in the video sequence. Our definition of consistent “temporal behavior” is expressed in terms of multi-frame linear subspace constraints. This definition applies to 2D, 3D, and some non-rigid motions without requiring prior model selection. We further show that our definition of motion consistency extends to data with directional uncertainty, thus leading to a dense segmentation of the entire image. Such segmentation is obtained by applying the new motion consistency constraints directly to covariance-weighted image brightness measurements. This is done without requiring prior correspondence estimation nor feature tracking.     

A non-linear model of shape and motion for tracking finger spelt American sign language

This work presents a piecewise linear approximation to non-linear Point Distribution Models for modelling the human hand. The work utilises the natural segmentation of shape space, inherent to the technique, to apply temporal constraints, which can be used with CONDENSATION to support multiple hypotheses and discontinuous jumps within shape space. This paper presents a novel method by which the one-state transitions of the English Language are projected into shape space for tracking and model prediction using an HMM like approach. The paper demonstrates that this model of motion provides superior results to that of other tracking approaches.     

基于图像多尺度运动特征的模糊分割算法

从运动分析的角度出发,对所研究的图像序列,提出了一种基于空域－时域连续小波变换（CWT）的模糊分割算法。运用可分离的小波函数作为滤波器来提取不同尺度上的运动参数,由不尺度上的运动特征参数构成特征参数集,然后运用快速模糊聚类算法实现基于运动特征的图象分割。该分割算法不仅对噪声具有较强的适应性,而且也极大地提高了图像分割的速度,同时使得分割结果能够更加准确地反映运动物体的特征。     

基于BTSM和DBN模型的唇读和视素切分研究

为实现文本/语音驱动的说话人头部动画,本文提出基于贝叶斯切线形状模型的口形轮廓特征提取方法和基于动态贝叶斯网络（Dynamic Bayesian Network, DBN）模型的唇读系统。在描述词与它的组成视素关系的基础上,得到视素时间切分序列。为比较性能,音素DBN模型和HMM的音素识别结果被影射成视素序列。在评价准则上,提出绝对视素切分正确性和基于图像与嘴唇几何特征两种相对视素切分正确性的评价标准。实验表明,DBN模型识别性能优于HMM,而基于视素的DBN模型能为说话人头部动画提供最好的口形。     

A selection framework of sensor combination feature subset for human motion phase segmentation

Motion phase plays an important role in the spatial–temporal parameters of human motion analysis. Multi-sensor fusion technology based on inertial sensors frees the monitoring of the human body phase from space constraints and improves the flexibility of the system. However, human phase segmentation methods usually rely on the determination of the positioning of the sensor and the number of sensors, it is difficult to artificially select the number and position of the sensors, especially when human motion phases are diverse. This paper proposes a selection framework for the sensor combination feature subset for motion phase segmentation, which combines feature selection algorithms with the subsequent classifiers, and determine the optimum combination of the sensor and the feature subset according to the performance of the trained model. Through the constraint and the sensor combination feature subset (SCFS), the filter method can select any number of sensors and control the size of the feature subset; the embedded method can select any number of sensors, but the size of the feature subset is determined by the classifier model. Experimental results show that the proposed framework can effectively select a specified number of sensors without human intervention, and the number of sensors has an impact on the recognition rate of the classifier within 1.5%. In addition, the filter method has good adaptability to a variety of classifiers, and the classifier prediction time can be controlled by setting the subset size of the feature; the embedded method can achieve a better phase segmentation effect than the filter method. For the application of motion phase segmentation, the proposed framework can reliably and quickly identify redundant sensors that provide effective support for reducing the complexity of the wearable sensor system and improving user comfort.