单目视频中无标记的人体运动跟踪

提出一种人体运动跟踪算法,从无关节标记的单目视频中获取人体运动,利用一个带外观模板的人体关节模型,通过学习得到的运动模型及基于外观模型的相似性计算,巧妙地利用粒子滤波的概率密度传播策略鲁棒地跟踪普通单目视频中的人体运动,当出现跟踪丢失时,能在后续序列中自动恢复正确跟踪,且能较好地处理遮挡和自遮挡问题,实验表明,该算法鲁棒性好,跟踪结果令人满意。     

双目视觉下三维人体运动跟踪算法*

由于人体运动的复杂性,人体运动轨迹的快速改变和人体自遮挡现象经常发生,这给人体运动跟踪带来了很大的困难。针对此问题提出了一种基于三维Kalman滤波器和人体约束的人体运动跟踪算法。该算法首先利用外极线约束和灰度互相关法对二维标记点进行立体匹配,计算各个标记点的三维位置,从而构建得到三维标记点;然后利用三维Kalman滤波器对三维标记点进行跟踪;最后利用人体约束检验和修正跟踪结果。实验结果表明,该算法能有效地对复杂人体动作进行跟踪并能从跟踪错误中正确恢复。     

基于单幅图像的真实感人体动画合成

由于单幅图像缺失三维信息以及完整的纹理信息,基于单幅图像的真实感三维人体动画合成极具挑战性。针对单幅图像三维信息缺失问题,提出了一种基于SMPL参数模型的三维人体几何重建方法。该方法以单幅图像为输入,先根据输入图像人体轮廓信息变形标准的SMPL参数模型分别生成与目标轮廓一致的正反面的三维几何模型,然后利用基于B样条插值的网格拼接融合算法拼接正反面三维几何,最后为了恢复正确的手部几何,利用基于B样条插值的网格拼接融合算法,将重建后的模型上错误的手部几何用标准SMPL参数模型上正确的手部几何替换。同时,针对单幅图像中纹理缺失的问题,提出了一个称为FBN（front to back network）的对抗生成网络,用于恢复被遮挡的人体背面纹理。实验结果表明,该方法生成的具有完整纹理的人体几何能够由3D运动数据驱动运动,生成具有高度真实感的三维人体动画。     

适用于单目视频的无标记三维人体运动跟踪

在无标记人体运动跟踪过程中,由于被跟踪目标缺乏明显的特征以及背景复杂而使得跟踪到的人体运动姿态与真实值偏差较大,不能进行长序列视频跟踪.针对这一现象,提出一种基于形变外观模板匹配进行单目视频的三维人体运动跟踪算法,其中所用的人体外观模型由三维人体骨骼模型及二维纸板模型组成.首先根据人体骨骼比例约束采用逆运动学计算出关节旋转欧拉角;然后利用正向运动学求得纸板模型中像素在三维空间中的坐标,将这些像素根据摄像机成像模型投影到二维图像中得到形变外观模板;最后采用直方图匹配得到人体运动跟踪结果.实验结果表明,该算法对于一些复杂的长序列人体运动能够得到较为理想的跟踪结果,可应用于人机交互和动画制作等领域.     

一种改进的TLD动态手势跟踪算法

针对目前动态手势跟踪算法TLD(跟踪-学习-检测)算法在手势目标遮挡后易出现跟踪漂移的不足,提出了一种改进的TLD动态手势跟踪算法.在跟踪器跟踪成功后,引入遮挡窗的方法进行手势目标遮挡的判定.若出现部分遮挡,则由TLD学习器处理;若出现严重遮挡,则在TLD的跟踪器中加入卡尔曼滤波器来预测估计当前帧中手势可能存在的区域,缩小跟踪器的搜索范围,提高跟踪器的处理速度;并在TLD检测器中加入基于马尔可夫模型的方向预测器,缩小检测器的检测范围,增强检测器对相似手势轨迹的判别能力.实验结果证明,改进后的TLD算法在不同的实验环境下均有较强的鲁棒性,能够快速准确地进行动态手势运动轨迹的跟踪,并且改善了手势目标遮挡后易出现跟踪漂移的问题.     

基于混合跟踪模型的室内步行人体3D运动估计

针对步行人体3D运动估计过程中的自遮挡问题, 提出了基于混合跟踪模型的粒子滤波算法. 首先, 利用自遮挡状态检测模型, 将步行人体运动划分为四种自遮挡状态; 其次, 根据混合跟踪模型, 针对不同的自遮挡状态, 算法采用不同的跟踪模型; 最后, 为了估计遮挡状态下的人体运动, 算法提出了基于M-估计的在线训练方法 以训练肢体运动相关系数. 经过实验分析, 算法对处 于自遮挡状态下的人体3D运动估计有着良好的效果, 人体3D运动的估计精度得到了提高.     

稀疏编码目标跟踪方法研究

针对稀疏表示用于目标跟踪时存在重构误差表示不够精确、目标模板更新错误等问题,提出一种改进的稀疏编码模型。该模型无需重构误差满足特定的先验概率分布,且加入对编码系数的自适应约束,可以取得更优的编码向量,使得跟踪结果更为准确。在此基础上,将这种改进的编码模型与粒子滤波目标跟踪算法相结合,研究并实现一种新的基于鲁棒稀疏编码模型的目标跟踪方法。该方法对每个粒子的采样区域进行编码,用所得的稀疏编码向量作为当前粒子的观测量,并采用目标模板分级更新策略,使得目标模板更加准确。实验结果表明,方法可以较好地解决目标部分遮挡和光照变化等干扰下的目标跟踪问题。     

基于地基投影的建筑场景立面建模

多视图建筑立面建模通常局限于稀疏点重建,且多方向立面的重建误差较大,对设备要求严格、系统庞大。针对该问题,提出一种基于地基投影的立面建模算法。通过场景预处理,标记视图序列的有序立面。对分组视图计算3D稀疏点,由稀疏点同步求解场景地基线和矫正立面。由地基线生长局部立面模型,经模型融合恢复场景三维立面模型。实验结果表明,算法重建结果具有准确、连续的三维结构与可视化纹理,在一定的稀疏点使用比例内,其立面角度误差小于5°,线比例误差低于1%,可满足室外场景建模工程的需求。     

基于改进压缩感知的运动目标跟踪

针对运动目标在被遮挡和目标纹理变化大时会导致跟踪丢失以及跟踪误差大等问题,提出了一种改进的压缩感知（ CS）算法。算法采用设置Sigmoid函数响应阈值,判定是否存在遮挡,以决定是否更新分类器参数,使得目标在遇到较大遮挡时目标模型不会被错误更新;针对特征单一导致跟踪不稳定问题,提出根据设定融合规则进行灰度特征和纹理特征融合的方法,使得两种特征指导跟踪。实验证明：改进后的算法比传统算法跟踪成功率提高了17.84%,平均误差率降低11.59%。     

结合模型匹配与特征跟踪的人体上半身三维运动姿态恢复方法

人体姿态空间的高维性及单目视频深度信息丢失,导致从单目视频恢复人体三维运动姿态非常困难,为此,利用特征跟踪的快速性及模型匹配的鲁棒性,提出一种无标记人体上半身三维运动跟踪方法.该方法利用匹配SIFT特征,并根据长度不变性约束建立优化目标函数,再采用迭代优化算法得到全局运动位姿;其他关节的姿态先根据逆运动学计算初始估计值,并通过模型匹配验证其可信度,当初始姿态估计错误时,则使用局部搜索获得关节姿态.实验结果表明,文中方法可以准确地恢复单目视频中人体上半身三维运动姿态.     

联合模板先验概率和稀疏表示的目标跟踪

目的 虽然基于稀疏表示的目标跟踪方法表现出了良好的跟踪效果,但仍然无法彻底解决噪声、旋转、遮挡、运动模糊、光照和姿态变化等复杂背景下的目标跟踪问题。针对遮挡、旋转、姿态变化和运动模糊问题,提出一种在粒子滤波框架内,基于稀疏表示和先验概率相结合的目标跟踪方法。方法 通过先验概率衡量目标模板的重要性,并将其引入到正则化模型中,作为模板更新的主要依据,从而获得一种新的候选目标稀疏表示模型。结果 在多个测试视频序列上,与多种流行算法相比,该算法可以达到更好的跟踪性能。在5个经典测试视频下的平均中心误差为6.77像素,平均跟踪成功率为97%,均优于其他算法。结论 实验结果表明,在各种含有遮挡、旋转、姿态变化和运动模糊的视频中,该算法可以稳定可靠地跟踪目标,适用于视频监控复杂场景下的目标跟踪。     

使用计算机视觉的3D模型动作记录器

该文旨在完成一款基于计算机视觉的3D模型动作记录器,即计算机通过摄像头获取人体运动视频并检测跟踪,之后通过处理数据控制3D模型,从而将人体动作进行记录保存。文章主要围绕运动目标检测、运动目标跟踪和3D建模三个方面展开研究。运动目标检测方面使用OpenCV（Open Source Computer Vision Library）提供的背景差分算法对目标进行分析并提取差分元素;运动目标跟踪方面则研究了常用的Camshift跟踪算法,实现对运动目标的连续跟踪以及识别从而保证动作记录器的连贯性;3D建模部分则使用3Dmax进行建立模型以及骨骼动画的制作处理,并使用Ogremax导出模型;而模型的骨骼动画则由OGRE导入测试环境并根据之前的处理结果进行相应的控制,从而实现人体运动的动作记录。     

在线复合模板模型表示的视觉目标跟踪

目的 视觉目标跟踪中,目标往往受到自身或场景中各种复杂干扰因素的影响,这对正确捕捉所感兴趣的目标信息带来极大的挑战。特别是,跟踪器所用的模板数据主要是在线学习获得,数据的可靠性直接影响到候选样本外观模型表示的精度。针对视觉目标跟踪中目标模板学习和候选样本外观模型表示等问题,采用一种较为有效的模板组织策略以及更为精确的模型表示技术,提出一种新颖的视觉目标跟踪算法。方法 跟踪框架中,将候选样本外观模型表示假设为由一组复合模板和最小重构误差组成的线性回归问题,首先利用经典的增量主成分分析法从在线高维数据中学习出一组低维子空间基向量(模板正样本),并根据前一时刻跟踪结果在线实时采样一些特殊的负样本加以扩充目标模板数据,再利用新组织的模板基向量和独立同分布的高斯—拉普拉斯混合噪声来线性拟合候选目标外观模型,最后估计出候选样本和真实目标之间的最大似然度,从而使跟踪器能够准确捕捉每一时刻的真实目标状态信息。结果 在一些公认测试视频序列上的实验结果表明,本文算法在目标模板学习和候选样本外观模型表示等方面比同类方法更能准确有效地反映出视频场景中目标状态的各种复杂变化,能够较好地解决各种不确定干扰因素下的模型退化和跟踪漂移问题,和一些优秀的同类算法相比,可以达到相同甚至更高的跟踪精度。结论 本文算法能够在线学习较为精准的目标模板并定期更新,使得跟踪器良好地适应内在或外在因素(姿态、光照、遮挡、尺度、背景扰乱及运动模糊等)所引起的视觉信息变化,始终保持其最佳的状态,使得候选样本外观模型的表示更加可靠准确,从而展现出更为鲁棒的性能。     

Extending 3D Lucas–Kanade tracking with adaptive templates for head pose estimation

The Lucas–Kanade tracker (LKT) is a commonly used method to track target objects over 2D images. The key principle behind the object tracking of an LKT is to warp the object appearance so as to minimize the difference between the warped object’s appearance and a pre-stored template. Accordingly, the 2D pose of the tracked object in terms of translation, rotation, and scaling can be recovered from the warping. To extend the LKT for 3D pose estimation, a model-based 3D LKT assumes a 3D geometric model for the target object in the 3D space and tries to infer the 3D object motion by minimizing the difference between the projected 2D image of the 3D object and the pre-stored 2D image template. In this paper, we propose an extended model-based 3D LKT for estimating 3D head poses by tracking human heads on video sequences. In contrast to the original model-based 3D LKT, which uses a template with each pixel represented by a single intensity value, the proposed model-based 3D LKT exploits an adaptive template with each template pixel modeled by a continuously updated Gaussian distribution during head tracking. This probabilistic template modeling improves the tracker’s ability to handle temporal fluctuation of pixels caused by continuous environmental changes such as varying illumination and dynamic backgrounds. Due to the new probabilistic template modeling, we reformulate the head pose estimation as a maximum likelihood estimation problem, rather than the original difference minimization procedure. Based on the new formulation, an algorithm to estimate the best head pose is derived. The experimental results show that the proposed extended model-based 3D LKT achieves higher accuracy and reliability than the conventional one does. Particularly, the proposed LKT is very effective in handling varying illumination, which cannot be well handled in the original LKT.     

联合LBS和Snake的3D人体外形和运动跟踪方法

为了解决基于多目视频轮廓信息的3D人体外形和运动跟踪问题,提出一种联合线性混合蒙皮和Snake变形模型的算法框架.首先建立人物对象的蒙皮模型,以每一帧多目同步视频的轮廓作为输入,采用一种基于剪影轮廓的可视外壳重建算法,使得作为3D特征的可视外壳保持了局部细节且更加光滑;并使用关节型迭代最近点算法进行匹配以捕获出每一帧骨架子空间下的人物3D外形及运动;再一次使用当前帧的多目轮廓信息,让Snake内外力共同作用于人物网格模型上的顶点,使之自由地趋近于目标对象.使用带ground-truth的合成数据进行对比实验的结果表明,该方法因同时使用3D误差约束和2D误差约束,提高了跟踪精度.     

Accurate 3D motion tracking by combining image alignment and feature matching

We presents a novel method to improve the accuracy of 3D motion tacking. In contrast to the state-of-the-art tracking approaches, where the 3D structure of target is commonly approximated by a CAD model, the proposed method establishes the target model by an online improved Structure-from-Motion technique. Furthermore, the tracking is implemented by three sequential trackers (feature-based tracker, image-alignment-based tracker and Particle Filter), which continually refine the tracking results. This coarse-to-fine method increases the accuracy of tracking. Moreover, our approach uses keyframe strategy to prevent tracking drift, the new keyframe insertion is determined by a criterion which can ensure a correct update. Thorough evaluations are performed on two public databases, the Biwi Head Pose dataset and the UPNA Head Pose Database. Comparisons illustrate that the proposed method achieves better performance with respect to other state-of-the-art tracking approaches.

     

Human Body Model Acquisition and Tracking Using Voxel Data

We present an integrated system for automatic acquisition of the human body model and motion tracking using input from multiple synchronized video streams. The video frames are segmented and the 3D voxel reconstructions of the human body shape in each frame are computed from the foreground silhouettes. These reconstructions are then used as input to the model acquisition and tracking algorithms.The human body model consists of ellipsoids and cylinders and is described using the twists framework resulting in a non-redundant set of model parameters. Model acquisition starts with a simple body part localization procedure based on template fitting and growing, which uses prior knowledge of average body part shapes and dimensions. The initial model is then refined using a Bayesian network that imposes human body proportions onto the body part size estimates. The tracker is an extended Kalman filter that estimates model parameters based on the measurements made on the labeled voxel data. A voxel labeling procedure that handles large frame-to-frame displacements was designed resulting in very robust tracking performance.Extensive evaluation shows that the system performs very reliably on sequences that include different types of motion such as walking, sitting, dancing, running and jumping and people of very different body sizes, from a nine year old girl to a tall adult male.     

3D Human Motion Tracking with a Coordinated Mixture of Factor Analyzers

A major challenge in applying Bayesian tracking methods for tracking 3D human body pose is the high dimensionality of the pose state space. It has been observed that the 3D human body pose parameters typically can be assumed to lie on a low-dimensional manifold embedded in the high-dimensional space. The goal of this work is to approximate the low-dimensional manifold so that a low-dimensional state vector can be obtained for efficient and effective Bayesian tracking. To achieve this goal, a globally coordinated mixture of factor analyzers is learned from motion capture data. Each factor analyzer in the mixture is a “locally linear dimensionality reducer” that approximates a part of the manifold. The global parametrization of the manifold is obtained by aligning these locally linear pieces in a global coordinate system. To enable automatic and optimal selection of the number of factor analyzers and the dimensionality of the manifold, a variational Bayesian formulation of the globally coordinated mixture of factor analyzers is proposed. The advantages of the proposed model are demonstrated in a multiple hypothesis tracker for tracking 3D human body pose. Quantitative comparisons on benchmark datasets show that the proposed method produces more accurate 3D pose estimates over time than those obtained from two previously proposed Bayesian tracking methods.     

Generative tracking of 3D human motion in latent space by sequential clonal selection algorithm

High dimensional pose state space is the main challenge in articulated human pose tracking which makes pose analysis computationally expensive or even infeasible. In this paper, we propose a novel generative approach in the framework of evolutionary computation, by which we try to widen the bottleneck with effective search strategy embedded in the extracted state subspace. Firstly, we use ISOMAP to learn the low-dimensional latent space of pose state in the aim of both reducing dimensionality and extracting the prior knowledge of human motion simultaneously. Then, we propose a manifold reconstruction method to establish smooth mappings between the latent space and original space, which enables us to perform pose analysis in the latent space. In the search strategy, we adopt a new evolutionary approach, clonal selection algorithm (CSA), for pose optimization. We design a CSA based method to estimate human pose from static image, which can be used for initialization of motion tracking. In order to make CSA suitable for motion tracking, we propose a sequential CSA (S-CSA) algorithm by incorporating the temporal continuity information into the traditional CSA. Actually, in a Bayesian inference view, the sequential CSA algorithm is in essence a multilayer importance sampling based particle filter. Our methods are demonstrated in different motion types and different image sequences. Experimental results show that our CSA based pose estimation method can achieve viewpoint invariant 3D pose reconstruction and the S-CSA based motion tracking method can achieve accurate and stable tracking of 3D human motion.