Robust object tracking via multi-scale patch based sparse coding histogram

There are many visual tracking algorithms that are based on sparse representation appearance model. Most of them are modeled by local patches with fixed patch scale, which make trackers less effective when objects undergone appearance changes such as illumination variation, pose change or partial occlusion. To solve the problem, a novel appearance representation model is proposed via multi-scale patch based sparse coding histogram for robust visual tracking. In this paper, the appearance of an object is modeled by different scale patches, which are represented by sparse coding histogram with different scale dictionaries. Then a similarity measure is applied to the calculation of the distance between the sparse coding histograms of target candidate and target template. Finally, the similarity score of the target candidate is passed to a particle filter to estimate the target state sequentially in the tracking process. Additionally, in order to decrease the visual drift caused by partial occlusion, an occlusion handling strategy is adopted, which takes the spatial information of multi-scale patches and occlusion into account. Based on the experimental results on some benchmarks of video sequences, our tracker outperforms state-of-the-art tracking methods.     

Adaptive visual tracking using the prioritized Q-learning algorithm: MDP-based parameter learning approach

This paper introduces an adaptive visual tracking method that combines the adaptive appearance model and the optimization capability of the Markov decision process. Most tracking algorithms are limited due to variations in object appearance from changes in illumination, viewing angle, object scale, and object shape. This paper is motivated by the fact that tracking performance degradation is caused not only by changes in object appearance but also by the inflexible controls of tracker parameters. To the best of our knowledge, optimization of tracker parameters has not been thoroughly investigated, even though it critically influences tracking performance. The challenge is to equip an adaptive tracking algorithm with an optimization capability for a more flexible and robust appearance model. In this paper, the Markov decision process, which has been applied successfully in many dynamic systems, is employed to optimize an adaptive appearance model-based tracking algorithm. The adaptive visual tracking is formulated as a Markov decision process based dynamic parameter optimization problem with uncertain and incomplete information. The high computation requirements of the Markov decision process formulation are solved by the proposed prioritized Q-learning approach. We carried out extensive experiments using realistic video sets, and achieved very encouraging and competitive results.     

Multi-scale patch-based sparse appearance model for robust object tracking

When objects undergo large pose change, illumination variation or partial occlusion, most existing visual tracking algorithms tend to drift away from targets and even fail to track them. To address the issue, in this paper we propose a multi-scale patch-based appearance model with sparse representation and provide an efficient scheme involving the collaboration between multi-scale patches encoded by sparse coefficients. The key idea of our method is to model the appearance of an object by different scale patches, which are represented by sparse coefficients with different scale dictionaries. The model exploits both partial and spatial information of targets based on multi-scale patches. Afterwards, a similarity score of one candidate target is input into a particle filter framework to estimate the target state sequentially over time in visual tracking. Additionally, to decrease the visual drift caused by frequently updating model, we present a novel two-step object tracking method which exploits both the ground truth information of the target labeled in the first frame and the target obtained online with the multi-scale patch information. Experiments on some publicly available benchmarks of video sequences showed that the similarity involving complementary information can locate targets more accurately and the proposed tracker is more robust and effective than others.     

Tracking rigid objects using integration of model-based and model-free cues

Model-based 3-D object tracking has earned significant importance in areas such as augmented reality, surveillance, visual servoing, robotic object manipulation and grasping. Key problems to robust and precise object tracking are the outliers caused by occlusion, self-occlusion, cluttered background, reflections and complex appearance properties of the object. Two of the most common solutions to the above problems have been the use of robust estimators and the integration of visual cues. The tracking system presented in this paper achieves robustness by integrating model-based and model-free cues together with robust estimators. As a model-based cue, a wireframe edge model is used. As model-free cues, automatically generated surface texture features are used. The particular contribution of this work is the integration framework where not only polyhedral objects are considered. In particular, we deal also with spherical, cylindrical and conical objects for which the complete pose cannot be estimated using only wireframe models. Using the integration with the model-free features, we show how a full pose estimate can be obtained. Experimental evaluation demonstrates robust system performance in realistic settings with highly textured objects and natural backgrounds.     

基于多任务最小软阈值回归方法的目标跟踪*

在视频跟踪中,模型表示是直接影响跟踪效率的核心问题之一.在随时间和空间变化的复杂数据中学习目标外观模型表示所需的有效模板,从而适应内在或外在因素所引起的目标状态变化是非常重要的.文中详细描述较为鲁棒的目标外观模型表示策略,并提出一种新的多任务最小软阈值回归跟踪算法(MLST).该算法框架将候选目标的观测模型假设为多任务线性回归问题,利用目标模板和独立同分布的高斯-拉普拉斯重构误差线性表示候选目标不同状态下的外观模型,从而跟踪器能够很好地适应各种复杂场景并准确预测每一时刻的真实目标状态.大量实验证明,文中在线学习策略能够充分挖掘目标在不同时刻的特殊状态信息以提高模型表示精度,使得跟踪器保持最佳的状态,从而在一定程度上提高跟踪性能.实验结果显示,本文算法体现较好的鲁棒性并优于一些目前较先进的跟踪算法.     

Learning spatially regularized similarity for robust visual tracking

Matching visual appearances of the target object over consecutive frames is a critical step in visual tracking. The accuracy performance of a practical tracking system highly depends on the similarity metric used for visual matching. Recent attempts to integrate discriminative metric learned by sequential visual data (instead of a predefined metric) in visual tracking have demonstrated more robust and accurate results. However, a global similarity metric is often suboptimal for visual matching when the target object experiences large appearance variation or occlusion. To address this issue, we propose in this paper a spatially weighted similarity fusion (SWSF) method for robust visual tracking. In our SWSF, a part-based model is employed as the object representation, and the local similarity metric and spatially regularized weights are jointly learned in a coherent process, such that the total matching accuracy between visual target and candidates can be effectively enhanced. Empirically, we evaluate our proposed tracker on various challenging sequences against several state-of-the-art methods, and the results demonstrate that our method can achieve competitive or better tracking performance in various challenging tracking scenarios.     

Robust object tracking via online dynamic spatial bias appearance models

This paper presents a robust object tracking method via a spatial bias appearance model learned dynamically in video. Motivated by the attention shifting among local regions of a human vision system during object tracking, we propose to partition an object into regions with different confidences and track the object using a dynamic spatial bias appearance model (DSBAM) estimated from region confidences. The confidence of a region is estimated to re ect the discriminative power of the region in a feature space, and the probability of occlusion. We propose a novel hierarchical Monte Carlo (HAMC) algorithm to learn region confidences dynamically in every frame. The algorithm consists of two levels of Monte Carlo processes implemented using two particle filtering procedures at each level and can efficiently extract high confidence regions through video frames by exploiting the temporal consistency of region confidences. A dynamic spatial bias map is then generated from the high confidence regions, and is employed to adapt the appearance model of the object and to guide a tracking algorithm in searching for correspondences in adjacent frames of video images. We demonstrate feasibility of the proposed method in video surveillance applications. The proposed method can be combined with many other existing tracking systems to enhance the robustness of these systems.     

Backtracking: Retrospective multi-target tracking

We introduce a multi-target tracking algorithm that operates on prerecorded video as typically found in post-incident surveillance camera investigation. Apart from being robust to visual challenges such as occlusion and variation in camera view, our algorithm is also robust to temporal challenges, in particular unknown variation in frame rate. The complication with variation in frame rate is that it invalidates motion estimation. As such, tracking algorithms based on motion models will show decreased performance. On the other hand, appearance based detection in individual frames suffers from a plethora of false detections. Our tracking algorithm, albeit relying on appearance based detection, deals robustly with the caveats of both approaches. The solution rests on the fact that for prerecorded video we can make fully informed choices; not only based on preceding, but also based on following frames. We start off from an appearance based object detection algorithm able to detect in each frame all target objects. From this we build a graph structure. The detections form the graph’s nodes and the vertices are formed by connecting each detection in a frame to all detections in the following frame. Thus, each path through the graph shows some particular selection of successive detections. Tracking is then reformulated as a heuristic search for optimal paths, where optimal means to find all detections belonging to a single object and excluding any other detection. We show that this approach, without an explicit motion model, is robust to both the visual and temporal challenges.     

Visual Object Tracking and Servoing Control of a Nano-Scale Quadrotor: System,Algorithms, and Experiments

There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.     

基于改进的MOSSE相关滤波的目标跟踪

视频目标跟踪存在如遮挡、形变、漂移等诸多挑战。虽然研究者提出了大量的算法来解决这一问题, 但大多数不具备普适性和实时性。为了实现目标有效稳定的实时跟踪, 本文在MOSSE相关滤波算法基础上提出了一种多模式的视觉目标跟踪算法, 该算法不仅具有相关算法的实时性, 还适应跟踪目标外观大幅度变化情况。同时, 为了适应跟踪过程中目标外形的复杂变化, 提出了一个控制模式更新率的算法, 利用具有多模式的跟踪算法可以同时处理极小的运动与外形突变。对基准视频数据的仿真实验结果表明, 与对应的单模型跟踪算法相比, 本文提出的算法可以明显改善跟踪精度和稳定性。     

部件级表观模型的目标跟踪方法

因受遮挡、运动模糊、剧烈形变等因素的影响,稳定且准确的目标跟踪是当前计算机视觉研究领域重要挑战之一.首先采用中层视觉线索的超像素描述目标/背景的部件,以部件颜色直方图作为其特征,并通过聚类部件库的特征集构建初始表观模型,部件表达的局部性和灵活性使该模型能够准确描述目标/背景;然后,利用贝叶斯滤波模型计算目标框的初始状态,并提出相似物体干扰的检测和处理算法以避免跟踪漂移,得到更健壮的结果;最后,为了减弱形变、遮挡、模糊对表观模型的影响以更好地保持目标特征,提出一种基于部件库的特征补集的在线表观模型更新算法,根据部件变化实时反映目标/背景的变化情况.在多个具有跟踪挑战的视频序列上的实验结果表明(共12个视频序列):与现有跟踪方法相比,该算法跟踪结果的中心误差更小,成功帧数更多,能够更准确并稳定、有效地跟踪目标物体.     

Robust object tracking under appearance change conditions

We propose a robust visual tracking framework based on particle filter to deal with the object appearance changes due to varying illumination, pose variantions, and occlusions. We mainly improve the observation model and re-sampling process in a particle filter. We use on-line updating appearance model, affine transformation, and M-estimation to construct an adaptive observation model. On-line updating appearance model can adapt to the changes of illumination partially. Affine transformation-based similarity measurement is introduced to tackle pose variantions, and M-estimation is used to handle the occluded object in computing observation likelihood. To take advantage of the most recent observation and produce a suboptimal Gaussian proposal distribution, we incorporate Kalman filter into a particle filter to enhance the performance of the resampling process. To estimate the posterior probability density properly with lower computational complexity, we only employ a single Kalman filter to propagate Gaussian distribution. Experimental results have demonstrated the effectiveness and robustness of the proposed algorithm by tracking visual objects in the recorded video sequences.     

Efficient tracking using a robust motion estimation technique

Camera based supervision is a critical part of event detection and analysis applications. However, visual tracking still remains one of the biggest challenges in the area of computer vision, although it has been extensively discussed during in the previous years. In this paper we propose a robust tracking approach based on object flow, which is a motion model for estimating both the displacement and the direction of an object of interest. In addition, an observation model that utilizes a generative prior is adopted to tackle the pitfalls that derive from the appearance changes of the object under study. The efficiency of our technique is demonstrated using sequences captured in a complex industrial environment. The experimental results show that the proposed algorithm is sound, yielding improved performance in comparison with other tracking approaches.     

Online learning and fusion of orientation appearance models for robust rigid object tracking

We introduce a robust framework for learning and fusing of orientation appearance models based on both texture and depth information for rigid object tracking. Our framework fuses data obtained from a standard visual camera and dense depth maps obtained by low-cost consumer depth cameras such as the Kinect. To combine these two completely different modalities, we propose to use features that do not depend on the data representation: angles. More specifically, our framework combines image gradient orientations as extracted from intensity images with the directions of surface normals computed from dense depth fields. We propose to capture the correlations between the obtained orientation appearance models using a fusion approach motivated by the original Active Appearance Models (AAMs). To incorporate these features in a learning framework, we use a robust kernel based on the Euler representation of angles which does not require off-line training, and can be efficiently implemented online. The robustness of learning from orientation appearance models is presented both theoretically and experimentally in this work. This kernel enables us to cope with gross measurement errors, missing data as well as other typical problems such as illumination changes and occlusions. By combining the proposed models with a particle filter, the proposed framework was used for performing 2D plus 3D rigid object tracking, achieving robust performance in very difficult tracking scenarios including extreme pose variations.     

Visual tracking based on adaptive interacting multiple model particle filter by fusing multiples cues

In visual tracking topic, developing a robust tracking method is very challenging, seen that there are many issues to look at, particularly, fast motion, target appearance changing, background clutter and camera motion. To override these problems, we present a new object tracking method with the fusion of interacting multiple models (IMM) and the particle filter (PF). First, the IMM is applied with a bank of parallel H∞ filter to estimate the global motion, the target motion is efficiently represented using only two parametric single models, and an adaptive strategy is preformed to adjust automatically the parameters of the two sub models at each recursive time step. Second, the particle filter is performed to estimate the local motion, we fuse the color and texture features to describe the appearance of the tracked object, we use the alpha Gaussian mixture model (α-GMM) to model the color feature distribution, the parameter α allows the probability function to possesses a flatter distribution, and the texture feature is represented by the distinctive uniform local binary pattern histogram (DULBP) based on the uniform local binary pattern (ULBP) operator; we fuse then the two features to represent the target’s appearance under the particle filter framework. We conduct quantitative and qualitative experiments on a variety of challenging public sequences; the results show that our method performs robustly and demonstrates strong accuracy.     

A hybrid occlusion free object tracking method using particle filter and modified galaxy based search meta-heuristic algorithm

An important problem in tracking methods is how to manage the changes in object appearance, such as illumination changes, partial/full occlusion, scale, and pose variation during the tracking process. In this paper, we propose an occlusion free object tracking method together with a simple adaptive appearance model. The proposed appearance model which is updated at the end of each time step includes three components: the first component consists of a fixed template of target object, the second component shows rapid changes in object appearance, and the third one maintains slow changes generated along the object path. The proposed tracking method not only can detect occlusion and handle it, but also it is robust against changes in the object appearance model. It is based on particle filter which is a robust technique in tracking and handles non-linear and non-Gaussian problems. We have also employed a meta-heuristic approach that is called Modified Galaxy based Search Algorithm (MGbSA), to reinforce finding the optimum state in the particle filter state space. The proposed method was applied to some benchmark videos and its results were satisfactory and better than results of related works.     

Incremental Tensor Subspace Learning and Its Applications to Foreground Segmentation and Tracking

Appearance modeling is very important for background modeling and object tracking. Subspace learning-based algorithms have been used to model the appearances of objects or scenes. Current vector subspace-based algorithms cannot effectively represent spatial correlations between pixel values. Current tensor subspace-based algorithms construct an offline representation of image ensembles, and current online tensor subspace learning algorithms cannot be applied to background modeling and object tracking. In this paper, we propose an online tensor subspace learning algorithm which models appearance changes by incrementally learning a tensor subspace representation through adaptively updating the sample mean and an eigenbasis for each unfolding matrix of the tensor. The proposed incremental tensor subspace learning algorithm is applied to foreground segmentation and object tracking for grayscale and color image sequences. The new background models capture the intrinsic spatiotemporal characteristics of scenes. The new tracking algorithm captures the appearance characteristics of an object during tracking and uses a particle filter to estimate the optimal object state. Experimental evaluations against state-of-the-art algorithms demonstrate the promise and effectiveness of the proposed incremental tensor subspace learning algorithm, and its applications to foreground segmentation and object tracking.     

Adaptive multi-cue tracking by online appearance learning

This paper proposes a multi-cue based appearance learning algorithm for object tracking. In each frame, the target object is represented by different cues in the image-as-matrix form. This representation can describe the target from different perspectives and can preserve the spatial correlation information inside the target region. Based on these cues, multiple appearance models are learned online by bilinear subspace analysis to account for the target appearance variations over time. Tracking is formulated within the Bayesian inference framework, in which the observation model is constructed by fusing all the learned appearance models. The combination of online appearance modeling and weight update of each appearance model can adapt our tracking algorithm to both the target and background changes. We test our algorithm on a variety of challenging sequences by tracking car, face, pedestrian, and so on. Experimental results and comparisons to several state-of-the-art methods show improved tracking performance.     

Multicue HMM-UKF for real-time contour tracking

We propose an HMM model for contour detection based on multiple visual cues in spatial domain and improve it by joint probabilistic matching to reduce background clutter. It is further integrated with unscented Kalman filter to exploit object dynamics in nonlinear systems for robust contour tracking.     

An online spatio-temporal tensor learning model for visual tracking and its applications to facial expression recognition

Robust visual tracking remains a technical challenge in real-world applications, as an object may involve many appearance variations. In existing tracking frameworks, objects in an image are often represented as vector observations, which discounts the 2-D intrinsic structure of the image. By considering an image in its actual form as a matrix, we construct the 3rd order tensor based object representation to preserve the spatial correlation within the 2-D image and fully exploit the useful temporal information. We perform incremental update of the object template using the N-mode SVD to model the appearance variations, which reduces the influence of template drifting and object occlusions. The proposed scheme efficiently learns a low-dimensional tensor representation through adaptively updating the eigenbasis of the tensor. Tensor based Bayesian inference in the particle filter framework is then utilized to realize tracking. We present the validation of the proposed tracking system by conducting the real-time facial expression recognition with video data and a live camera. Experiment evaluation on challenging benchmark image sequences undergoing appearance variations demonstrates the significance and effectiveness of the proposed algorithm.