基于视频的三维人体运动跟踪系统的设计与实现

在优化粒子滤波跟踪框架下,设计并实现了一个结合多种图像特征、在多摄像机环境下跟踪人体运动的三维人体运动跟踪系统.通过定义三维人体模型、摄像机模型以及观测似然模型,得到跟踪所需目标函数,并使用优化粒子滤波算法进行求解.实验结果表明,该系统能够对人体运动进行准确的跟踪和三维重建,可应用于体育运动分析和动画制作等领域.     

运动摄像机下多目标检测与跟踪

针对摄像机运动情况下多目标的检测与跟踪问题,提出一种将Global K均值与模板匹配相结合的方法.利用六参数仿射模型得到摄像机运动参数,对图像进行全局运动补偿,用GlobalK均值算法对前景点进行循环聚类,判断目标数目并进行跟踪,通过对目标区域进行模板匹配使跟踪结果更准确.实验结果表明,该方法能够在运动摄像机下稳定、实时地跟踪多个目标,对发生形变的目标基本也能稳定跟踪.     

基于粒子滤波的多特征融合视频行人跟踪算法

针对车载视频行人跟踪问题,提出一种基于粒子滤波框架下的多特征融合跟踪算法。为克服车载视频中行人运动与摄像机运动产生的非线性和非高斯性,采用基于蒙特卡罗抽样的粒子滤波跟踪算法,使用一阶自回归动态模型预测目标状态,观测模型自适应加权融合的4种互补性特征。实验结果表明,与没有粒子滤波和多特征融合的跟踪算法相比,在相同精确率水平上,该算法的召回率提高20%以上。     

光学头部姿态跟踪的多传感器数据融合研究

精确的头部姿态跟踪是室内增强现实系统实现高精度注册的关键技术之一. 本文介绍了使用传感器数据融合原理实现高精度的光学头部姿态跟踪的新方法. 该方法使用多传感器数据融合中的扩展卡尔曼滤波器和融合滤波器, 将两个互补的单摄像机Inside-out跟踪和双摄像机Outside-in跟踪的头部姿态进行数据融合, 以减小光学跟踪传感器的姿态误差. 设计了一个典型实验装置验证所提出的算法, 实验结果显示, 在静态测试下的姿态输出误差与使用误差协方差传播法则计算得到的结果是一致的; 在动态跟踪条件下, 与单个Inside-out或Outside-in跟踪相比, 所提出的光学头部姿态数据融合算法能够使跟踪器获得精度更高、更稳定的位置和方向信息.     

自然环境视频中基于显著鲁棒轨迹的行为识别

目的人类行为识别是计算机视觉领域的一个重要研究课题。由于背景复杂、摄像机抖动等原因,在自然环境视频中识别人类行为存在困难。针对上述问题,提出一种基于显著鲁棒轨迹的人类行为识别算法。方法该算法使用稠密光流技术在多尺度空间中跟踪显著特征点,并使用梯度直方图(HOG)、光流直方图(HOF)和运动边界直方图(MBH)特征描述显著轨迹。为了有效消除摄像机运动带来的影响,使用基于自适应背景分割的摄像机运动估计技术增强显著轨迹的鲁棒性。然后,对于每一类特征分别使用Fisher Vector模型将一个视频表示为一个Fisher向量,并使用线性支持向量机对视频进行分类。结果在4个公开数据集上,显著轨迹算法比Dense轨迹算法的实验结果平均高1%。增加摄像机运动消除技术后,显著鲁棒轨迹算法比显著轨迹算法的实验结果平均高2%。在4个数据集(即Hollywood2、You Tube、Olympic Sports和UCF50)上,显著鲁棒轨迹算法的实验结果分别是65.8%、91.6%、93.6%和92.1%,比目前最好的实验结果分别高1.5%、2.6%、2.5%和0.9%。结论实验结果表明,该算法能够有效地识别自然环境视频中的人类行为,并且具有较低的时间复杂度。     

适用于单目视频的无标记三维人体运动跟踪

在无标记人体运动跟踪过程中,由于被跟踪目标缺乏明显的特征以及背景复杂而使得跟踪到的人体运动姿态与真实值偏差较大,不能进行长序列视频跟踪.针对这一现象,提出一种基于形变外观模板匹配进行单目视频的三维人体运动跟踪算法,其中所用的人体外观模型由三维人体骨骼模型及二维纸板模型组成.首先根据人体骨骼比例约束采用逆运动学计算出关节旋转欧拉角;然后利用正向运动学求得纸板模型中像素在三维空间中的坐标,将这些像素根据摄像机成像模型投影到二维图像中得到形变外观模板;最后采用直方图匹配得到人体运动跟踪结果.实验结果表明,该算法对于一些复杂的长序列人体运动能够得到较为理想的跟踪结果,可应用于人机交互和动画制作等领域.     

一种基于混沌天牛须搜索算法的摄像机标定方法

为了较好地解决传统智能优化方法在摄像机标定中存在标定精度低、效率和鲁棒性差的问题，提出一种基于混沌天牛须搜索算法的摄像机标定方法。该方法使用MATLAB标定工具箱对摄像机非线性成像模型进行预标定，预标定结果作为混沌天牛须搜索算法的初始值；构造平均重投影误差适应度函数，建立混沌天牛须搜索算法优化模型对标定参数进行优化；与基于传统智能优化方法的摄像机标定方法进行实验对比。实验结果表明，该方法得到的平均重投影误差为0.005 72像素，算法总的运行时间为46.15 s,可以有效提高摄像机标定的精度、鲁棒性与效率。     

一种基于背景模型的运动目标检测与跟踪算法

本文提出了一种静止摄像机条件下的运动目标检测与跟踪算法．它以一种改进的自适应混合 高斯模型为背景更新方法，用连通区检测算法分割出前景目标，以Kalman滤波为运动模型实 现对运动目标的连续跟踪．在目标跟踪时，该算法针对目标遮挡引起的各种可能情况进行了 分析，引入了对运动目标的可靠性度量，增强了目标跟踪的稳定性和可靠性．在对多个室外 视频序列的实验中，该算法显示了良好的性能，说明它对于各种外部因素的影响，如光照变 化、阴影、目标遮挡等，具有很强的适应能力．     

基于序列帧的光学运动捕捉系统及实现

首先,给出了系统的结构图。然后,提出了光学运动捕捉系统中关键技术的解决方法。利用卡尔曼滤波算法进行标记点预测和跟踪,提高了跟踪的准确性。在三维重建之前还需要进行摄像机的三维重建,以便准确地得出标记点的空间位置,提出了标定摄像机的实用方法。由于误差的存在,对三维重建理想状态下的方法进行了扩充,使三维重建更易于实现。实验证明,文中提到的方法能得到理想的数据。     

利用区域变形和背景更新实现运动对象跟踪

从时域统计的角度出发，提出了一种结合自适应混合背景更新模型的区域变形跟踪算法．该算法以模型更新得到的前景／背景二值分割掩膜作为区域特征，将跟踪问题抽象为一个水平集(Level Set)偏微分方程的数值求解问题，并分析了算法的自适应性．为了进一步提高算法的实现效率，引入了窄带跟踪方案．实验表明，该算法可以对视频序列中的指定运动对象进行快速精确的跟踪。     

Markerless tracking of complex human motions from multiple views

We present a method for markerless tracking of complex human motions from multiple camera views. In the absence of markers, the task of recovering the pose of a person during such motions is challenging and requires strong image features and robust tracking. We propose a solution which integrates multiple image cues such as edges, color information and volumetric reconstruction. We show that a combination of multiple image cues helps the tracker to overcome ambiguous situations such as limbs touching or strong occlusions of body parts. Following a model-based approach, we match an articulated body model built from superellipsoids against these image cues. Stochastic Meta Descent (SMD) optimization is used to find the pose which best matches the images. Stochastic sampling makes SMD robust against local minima and lowers the computational costs as a small set of predicted image features is sufficient for optimization. The power of SMD is demonstrated by comparing it to the commonly used Levenberg–Marquardt method. Results are shown for several challenging sequences showing complex motions and full articulation, with tracking of 24 degrees of freedom in ≈1 frame per second.     

Segmentation and tracking of multiple humans in crowded environments

Segmentation and tracking of multiple humans in crowded situations is made difficult by interobject occlusion. We propose a model based approach to interpret the image observations by multiple, partially occluded human hypotheses in a Bayesian framework. We define a joint image likelihood for multiple humans based on the appearance of the humans, the visibility of body obtained by occlusion reasoning, and foreground/background separation. The optimal solution is obtained by using an efficient sampling method, data-driven Markov chain Monte Carlo (DDMCMC), which uses image observations for proposal probabilities. Knowledge of various aspects including human shape, camera model, and image cues are integrated in one theoretically sound framework. We present experimental results and quantitative evaluation, demonstrating that the resulting approach is effective for very challenging data.     

Robust Visual Tracking by Integrating Multiple Cues Based on Co-Inference Learning

Visual tracking can be treated as a parameter estimation problem that infers target states based on image observations from video sequences. A richer target representation may incur better chances of successful tracking in cluttered and dynamic environments, and thus enhance the robustness. Richer representations can be constructed by either specifying a detailed model of a single cue or combining a set of rough models of multiple cues. Both approaches increase the dimensionality of the state space, which results in a dramatic increase of computation. To investigate the integration of rough models from multiple cues and to explore computationally efficient algorithms, this paper formulates the problem of multiple cue integration and tracking in a probabilistic framework based on a factorized graphical model. Structured variational analysis of such a graphical model factorizes different modalities and suggests a co-inference process among these modalities. Based on the importance sampling technique, a sequential Monte Carlo algorithm is proposed to provide an efficient simulation and approximation of the co-inferencing of multiple cues. This algorithm runs in real-time at around 30 Hz. Our extensive experiments show that the proposed algorithm performs robustly in a large variety of tracking scenarios. The approach presented in this paper has the potential to solve other problems including sensor fusion problems.     

自适应融合颜色和深度信息的人体轮廓跟踪

采用活动轮廓对人体目标建模,提出一 种新的水平集框架下自适应融合RGB-D图像的颜色和深度信息的人体轮廓跟踪方法. 设计了一种基于超像素的局部自适应权重计算方法,自动确定深度信息在水平集演化中的重要性. 基于深度信息的活动轮廓驱动外力包括由边缘生成的梯度向量流和由目标/背景深度模型生成的置信图,基于颜色信息的驱动外力由目标/背景颜色模型生成的置信图,这三种外力通过局部自适应权重融合,驱动活动轮廓向目标的边界演化.为了得到更加精确的目标轮廓和防止误差漂移,基于本文观察到的人体表面在深度图像中的两个特性,提出两个简单但有效的算法对水平集方法得到的结果进行精化调整. 最后,通过实验验证了本文算法的优越性.     

视频中多特征融合人体姿态跟踪

目的 目前已有的人体姿态跟踪算法的跟踪精度仍有待提高,特别是对灵活运动的手臂部位的跟踪。为提高人体姿态的跟踪精度,本文首次提出一种将视觉时空信息与深度学习网络相结合的人体姿态跟踪方法。方法 在人体姿态跟踪过程中,利用视频时间信息计算出人体目标区域的运动信息,使用运动信息对人体部位姿态模型在帧间传递;考虑到基于图像空间特征的方法对形态较为固定的人体部位如躯干和头部能够较好地检测,而对手臂的检测效果较差,构造并训练一种轻量级的深度学习网络,用于生成人体手臂部位的附加候选样本;利用深度学习网络生成手臂特征一致性概率图,与视频空间信息结合计算得到最优部位姿态,并将各部位重组为完整人体姿态跟踪结果。结果 使用两个具有挑战性的人体姿态跟踪数据集VideoPose2.0和YouTubePose对本文算法进行验证,得到的手臂关节点平均跟踪精度分别为81.4%和84.5%,与现有方法相比有明显提高;此外,通过在VideoPose2.0数据集上的实验,验证了本文提出的对下臂附加采样的算法和手臂特征一致性计算的算法能够有效提高人体姿态关节点的跟踪精度。结论 提出的结合时空信息与深度学习网络的人体姿态跟踪方法能够有效提高人体姿态跟踪的精度,特别是对灵活运动的人体姿态下臂关节点的跟踪精度有显著提高。     

Visual tracking by fusing multiple cues with context-sensitive reliabilities

Many researchers argue that fusing multiple cues increases the reliability and robustness of visual tracking. However, how the multi-cue integration is realized during tracking is still an open issue. In this work, we present a novel data fusion approach for multi-cue tracking using particle filter. Our method differs from previous approaches in a number of ways. First, we carry out the integration of cues both in making predictions about the target object and in verifying them through observations. Our second and more significant contribution is that both stages of integration directly depend on the dynamically changing reliabilities of visual cues. These two aspects of our method allow the tracker to easily adapt itself to the changes in the context, and accordingly improve the tracking accuracy by resolving the ambiguities.     

A Particle Filtering and DSmT Based Approach for Conflict Resolving in case of Target Tracking with Multiple Cues

In this paper, we propose an efficient and robust method for multiple targets tracking in cluttered scenes using multiple cues. Our approach combines the use of Monte Carlo sequential filtering for tracking and Dezert-Smarandache theory (DSmT) to integrate the information provided by the different cues. The use of DSmT provides the necessary framework to quantify and overcome the conflict that might appear between the cues due to the occlusion. Our tracking approach is tested with color and location cues on a cluttered scene where multiple targets are involved in partial or total occlusion.     

Integrating tracking with fine object segmentation

We present a novel method for on-line, joint object tracking and segmentation in a monocular video captured by a possibly moving camera. Our goal is to integrate tracking and fine segmentation of a single, previously unseen, potentially non-rigid object of unconstrained appearance, given its segmentation in the first frame of an image sequence as the only prior information. To this end, we tightly couple an existing kernel-based object tracking method with Random Walker-based image segmentation. Bayesian inference mediates between tracking and segmentation, enabling effective data fusion of pixel-wise spatial and color visual cues. The fine segmentation of an object at a certain frame provides tracking with reliable initialization for the next frame, closing the loop between the two building blocks of the proposed framework. The effectiveness of the proposed methodology is evaluated experimentally by comparing it to a large collection of state of the art tracking and video-based object segmentation methods on the basis of a data set consisting of several challenging image sequences for which ground truth data is available.     

基于多特征融合的分块采样粒子滤波算法在人体姿态跟踪中的应用

针对单目视频中无标记点的人体姿态跟踪问题,在分块采样粒子滤波算法框架下使用颜色(color)、边缘(edge)和运动(motion)特征相融合构造粒子权值度量函数,并根据肢体间的遮挡情况自适应地选择不同模板和图像特征来进行计算,增加了跟踪过程的鲁棒性,而且成功解决了人体运动中发生的多种形式的自遮挡问题.另外,该方法还提出了一种带约束的2维人体模型,并在此模型基础上使用一种改进的BP算法进行权值的传播,使得在一个关节点上能够同时应用多个人体约束.实验中所用测试视频(室内和室外拍摄)包含复杂背景和运动,实验结果表明该方法具有较强的鲁棒性和较高的跟踪精度.