Discriminative multi-task objects tracking with active feature selection and drift correction

In this paper, we propose a discriminative multi-task objects tracking method with active feature selection and drift correction. The developed method formulates object tracking in a particle filter framework as multi-Task discriminative tracking. As opposed to generative methods that handle particles separately, the proposed method learns the representation of all the particles jointly and the corresponding coefficients are similar. The tracking algorithm starts from the active feature selection scheme, which adaptively chooses suitable number of discriminative features from the tracked target and background in the dynamic environment. Based on the selected feature space, the discriminative dictionary is constructed and updated dynamically. Only a few of them are used to represent all the particles at each frame. In other words, all the particles share the same dictionary templates and their representations are obtained jointly by discriminative multi-task learning. The particle that has the highest similarity with the dictionary templates is selected as the next tracked target state. This jointly sparsity and discriminative learning can exploit the relationship between particles and improve tracking performance. To alleviate the visual drift problem encountered in object tracking, a two-stage particle filtering algorithm is proposed to complete drift correction and exploit both the ground truth information of the first frame and observations obtained online from the current frame. Experimental evaluations on challenging sequences demonstrate the effectiveness, accuracy and robustness of the proposed tracker in comparison with state-of-the-art algorithms.     

多模态引导的局部特征选择小样本学习方法

深度学习模型取得了令人瞩目的成绩,但其训练依赖于大量的标注样本,在标注样本匮乏的场景下模型表现不尽人意.针对这一问题,近年来以研究如何从少量样本快速学习的小样本学习被提了出来,方法主要采用元学习方式对模型进行训练,取得了不错的学习效果.但现有方法:1)通常仅基于样本的视觉特征来识别新类别,信息源较为单一; 2)元学习的使用使得模型从大量相似的小样本任务中学习通用的、可迁移的知识,不可避免地导致模型特征空间趋于一般化,存在样本特征表达不充分、不准确的问题.为解决上述问题,将预训练技术和多模态学习技术引入小样本学习过程,提出基于多模态引导的局部特征选择小样本学习方法.所提方法首先在包含大量样本的已知类别上进行模型预训练,旨在提升模型的特征表达能力;而后在元学习阶段,方法利用元学习对模型进行进一步优化,旨在提升模型的迁移能力或对小样本环境的适应能力,所提方法同时基于样本的视觉特征和文本特征进行局部特征选择来提升样本特征的表达能力,以避免元学习过程中模型特征表达能力的大幅下降;最后所提方法利用选择后的样本特征进行小样本学习.在MiniImageNet、CIFAR-FS和FC-100这3个基准数...     

Real-time object tracking via compressive feature selection

Recently, compressive tracking (CT) has been widely proposed for its efficiency, accuracy and robustness on many challenging sequences. Its appearance model employs non-adaptive random projections that preserve the structure of the image feature space. A very sparse measurement matrix is used to extract features by multiplying it with the feature vector of the image patch. An adaptive Bayes classifier is trained using both positive samples and negative samples to separate the target from background. On the CT framework, however, some features used for classification have weak discriminative abilities, which reduces the accuracy of the strong classifier. In this paper, we present an online compressive feature selection algorithm(CFS) based on the CT framework. It selects the features which have the largest margin when using them to classify positive samples and negative samples. For features that are not selected, we define a random learning rate to update them slowly. It makes those weak classifiers preserve more target information, which relieves the drift when the appearance of the target changes heavily. Therefore, the classifier trained with those discriminative features couples its score in many challenging sequences, which leads to a more robust tracker. Numerous experiments show that our tracker could achieve superior result beyond many state-of-the-art trackers.     

Visual tracking with multiple Hough detectors

We propose a visual tracking method using multiple Hough detectors to address the problem of long-term robust object tracking in unconstrained environments. The method constructs the detectors based on the feature selection by the mutual information. These detectors serve to learn the partial appearances of target and synchronously evaluate image locations via the voting based detection with the generalized Hough transform. According to the result of detections, the best detector is selected by the minimum entropy criterion and delivers the final hypotheses for target location. The feature selection allows our tracker to be able to obtain and use the most discriminative parts of target and thus more robust to its changes, e.g. occlusion and deformation. The detector selection can correct undesirable model updates and restore the tracker after tracking failure. Meanwhile, the Hough-based detection can reduce the amount of noise introduced during online self-training and thus effectively prevent the tracker from drifting. The method is evaluated on the CVPR2013 Visual Tracker Benchmark and the experimental results demonstrate our method outperforms other tracking algorithms in terms of both success rate and precision.     

视觉语言多模态预训练综述

在多模态机器学习领域,为特定任务而制作的人工标注数据昂贵,且不同任务难以进行迁移,从而需要大量重新训练,导致训练多个任务时效率低下、资源浪费。预训练模型通过以自监督为代表的方式进行大规模数据训练,对数据集中不同模态的信息进行提取和融合,以学习其中蕴涵的通用知识表征,从而服务于广泛的相关下游视觉语言多模态任务,这一方法逐渐成为人工智能各领域的主流方法。依靠互联网所获取的大规模图文对与视频数据,以及以自监督学习为代表的预训练方法的进步,视觉语言多模态预训练模型在很大程度上打破了不同视觉语言任务之间的壁垒,提升了多个任务训练的效率并促进了具体任务的性能表现。本文总结视觉语言多模态预训练领域的进展,首先对常见的预训练数据集和预训练方法进行汇总,然后对目前最新方法以及经典方法进行系统概述,按输入来源分为图像—文本预训练模型和视频—文本多模态模型两大类,阐述了各方法之间的共性和差异,并将各模型在具体下游任务上的实验情况进行汇总。最后,总结了视觉语言预训练面临的挑战和未来发展趋势。     

Visual tracking based on Distribution Fields and online weighted multiple instance learning

This paper presents an improved multiple instance learning (MIL) tracker representing target with Distribution Fields (DFs) and building a weighted-geometric-mean MIL classifier. Firstly, we adopt DF layer as feature instead of traditional Haar-like one to model the target thanks to the DF specificity and the landscape smoothness. Secondly, we integrate sample importance into the weighted-geometric-mean MIL model and derive an online approach to maximize the bag likelihood by AnyBoost gradient framework to select the most discriminative layers. Due to the target model consisting of selected discriminative layers, our tracker is more robust while needing fewer features than the traditional Haar-like one and the original DFs one. The experimental results show higher performances of our tracker than those of five state-of-the-art ones on several challenging video sequences.     

Online unsupervised feature learning for visual tracking

We propose a method for visual tracking-by-detection based on online feature learning. Our learning framework performs feature encoding with respect to an over-complete dictionary, followed by spatial pyramid pooling. We then learn a linear classifier based on the resulting feature encoding. Unlike previous work, we learn the dictionary online and update it to help capture the appearance of the tracked target as well as the background. In more detail, given a test image window, we extract local image patches from it and each local patch is encoded with respect to the dictionary. The encoded features are then pooled over a spatial pyramid to form an aggregated feature vector. Finally, a simple linear classifier is trained on these features.Our experiments show that the proposed powerful—albeit simple—tracker, outperforms all the state-of-the-art tracking methods that we have tested. Moreover, we evaluate the performance of different dictionary learning and feature encoding methods in the proposed tracking framework, and analyze the impact of each component in the tracking scenario. In particular, we show that a small dictionary, learned and updated online is as effective and more efficient than a huge dictionary learned offline. We further demonstrate the flexibility of feature learning by showing how it can be used within a structured learning tracking framework. The outcome is one of the best trackers reported to date, which facilitates the advantages of both feature learning and structured output prediction. We also implement a multi-object tracker, which achieves state-of-the-art performance.     

增强二阶网络调制的目标跟踪

目的 表观模型对视觉目标跟踪的性能起着决定性的作用。基于网络调制的跟踪算法通过构建高效的子网络学习参考帧目标的表观信息,以用于测试帧目标的鲁棒匹配,在多个目标跟踪数据集上表现优异。但是,这类跟踪算法忽视了高阶信息对鲁棒建模物体表观的重要作用,致使在物体表观发生大尺度变化时易产生跟踪漂移。为此本文提出全局上下文信息增强的二阶池化调制子网络,以学习高阶特征提升跟踪器的性能。方法 首先,利用卷积神经网络（convolutional neural networks,CNN）提取参考帧和测试帧的特征;然后,对提取的特征采用不同方向的长短时记忆网络（long shot-term memory networks,LSTM）捕获每个像素的全局上下文信息,再经过二阶池化网络提取高阶信息;最后,通过调制机制引导测试帧学习最优交并比预测。同时,为提升跟踪器的稳定性,在线跟踪通过指数加权平均自适应更新物体表观特征。结果 实验结果表明,在OTB100（object tracking benchmark）数据集上,本文方法的成功率为67.9%,超越跟踪器ATOM （accurate tracking by overlap maximization）1.5%;在VOT （visual object tracking）2018数据集上平均期望重叠率（expected average overlap,EAO）为0.44,超越ATOM 4%。结论 本文通过构建全局上下文信息增强的二阶池化调制子网络来学习高效的表观模型,使跟踪器达到目前领先的性能。     

基于多任务最小软阈值回归方法的目标跟踪*

在视频跟踪中,模型表示是直接影响跟踪效率的核心问题之一.在随时间和空间变化的复杂数据中学习目标外观模型表示所需的有效模板,从而适应内在或外在因素所引起的目标状态变化是非常重要的.文中详细描述较为鲁棒的目标外观模型表示策略,并提出一种新的多任务最小软阈值回归跟踪算法(MLST).该算法框架将候选目标的观测模型假设为多任务线性回归问题,利用目标模板和独立同分布的高斯-拉普拉斯重构误差线性表示候选目标不同状态下的外观模型,从而跟踪器能够很好地适应各种复杂场景并准确预测每一时刻的真实目标状态.大量实验证明,文中在线学习策略能够充分挖掘目标在不同时刻的特殊状态信息以提高模型表示精度,使得跟踪器保持最佳的状态,从而在一定程度上提高跟踪性能.实验结果显示,本文算法体现较好的鲁棒性并优于一些目前较先进的跟踪算法.     

Object tracking via collaborative multi-task learning and appearance model updating

In this paper, we propose a novel visual tracking algorithm using the collaboration of generative and discriminative trackers under the particle filter framework. Each particle denotes a single task, and we encode all the tasks simultaneously in a structured multi-task learning manner. Then, we implement generative and discriminative trackers, respectively. The discriminative tracker considers the overall information of object to represent the object appearance; while the generative tracker takes the local information of object into account for handling partial occlusions. Therefore, two models are complementary during the tracking. Furthermore, we design an effective dictionary updating mechanism. The dictionary is composed of fixed and variational parts. The variational parts are progressively updated using Metropolis–Hastings strategy. Experiments on different challenging video sequences demonstrate that the proposed tracker performs favorably against several state-of-the-art trackers.     

Robust visual tracking using information theoretical learning

This paper presents a novel online object tracking algorithm with sparse representation for learning effective appearance models under a particle filtering framework. Compared with the state-of-the-art ? ₁ sparse tracker, which simply assumes that the image pixels are corrupted by independent Gaussian noise, our proposed method is based on information theoretical Learning and is much less sensitive to corruptions; it achieves this by assigning small weights to occluded pixels and outliers. The most appealing aspect of this approach is that it can yield robust estimations without using the trivial templates adopted by the previous sparse tracker. By using a weighted linear least squares with non-negativity constraints at each iteration, a sparse representation of the target candidate is learned; to further improve the tracking performance, target templates are dynamically updated to capture appearance changes. In our template update mechanism, the similarity between the templates and the target candidates is measured by the earth movers’ distance(EMD). Using the largest open benchmark for visual tracking, we empirically compare two ensemble methods constructed from six state-of-the-art trackers, against the individual trackers. The proposed tracking algorithm runs in real-time, and using challenging sequences performs favorably in terms of efficiency, accuracy and robustness against state-of-the-art algorithms.     

SPiKeS: Superpixel-Keypoints structure for robust visual tracking

In visual tracking, part-based trackers are attractive since they are robust against occlusion and deformation. However, a part represented by a rectangular patch does not account for the shape of the target, while a superpixel does thanks to its boundary evidence. Nevertheless, tracking superpixels is difficult due to their lack of discriminative power. Therefore, to enable superpixels to be tracked discriminatively as object parts, we propose to enhance them with keypoints. By combining properties of these two features, we build a novel element designated as a Superpixel-Keypoints structure (SPiKeS). Being discriminative, these new object parts can be located efficiently by a simple nearest neighbor matching process. Then, in a tracking process, each match votes for the target’s center to give its location. In addition, the interesting properties of our new feature allows the development of an efficient model update for more robust tracking. According to experimental results, our SPiKeS-based tracker proves to be robust in many challenging scenarios by performing favorably against the state of the art.     

Tracking using Numerous Anchor Points

In this paper, an online adaptive model-free tracker is proposed to track single objects in video sequences to deal with real-world tracking challenges like low-resolution, object deformation, occlusion and motion blur. The novelty lies in the construction of a strong appearance model that captures features from the initialized bounding box and then are assembled into anchor point features. These features memorize the global pattern of the object and have an internal star graph-like structure. These features are unique and flexible and help tracking generic and deformable objects with no limitation on specific objects. In addition, the relevance of each feature is evaluated online using short-term consistency and long-term consistency. These parameters are adapted to retain consistent features that vote for the object location and that deal with outliers for long-term tracking scenarios. Additionally, voting in a Gaussian manner helps in tackling inherent noise of the tracking system and in accurate object localization. Furthermore, the proposed tracker uses pairwise distance measure to cope with scale variations and combines pixel-level binary features and global weighted color features for model update. Finally, experimental results on a visual tracking benchmark dataset are presented to demonstrate the effectiveness and competitiveness of the proposed tracker.     

改进的卷积网络目标跟踪算法

在目标跟踪算法中深度网络可以对大量图像进行训练和表示,但是对于特定的跟踪对象,离线训练不仅费时,而且在对大量图像进行学习时,其表示和识别能力效果不佳。基于以上问题提出有模板更新的卷积网络跟踪算法,可以在没有离线训练的大量数据时,也能够利用实现强大的目标跟踪能力。在目标跟踪中,从目标周围区域提取一组归一化的局部小区域块作为新的滤波器,围绕目标定义下一帧中的一组特征映射来提取自适应滤波器周围目标,对随后帧提取的归一化样本进行卷积操作生成一组特征图;利用这些特征图获取每个滤波器和目标的局部强度衍射图样之间的相似性,然后对其局部结构信息进行编码;最后,使用来自全局表示的特征图保存该目标的内部几何设计,再通过软收缩方法去噪抑制噪声值,使其低于自适应阈值,生成目标的稀疏表示。有模板更新改进的CNT算法能稳定地跟踪目标,不会发生严重漂移,具有优于传统CNT的良好跟踪效果。     

Accurate visual representation learning for single object tracking

As a fundamental visual task, single object tracking has witnessed astonishing improvements. However, there still existing many factors should be to addressed for accurately tracking performance. Among them, visual representation is one of important influencers suffer from complex appearance changes. In this work, we propose a rich appearance representation learning strategy for tracking. First, by embedding the saliency feature extractor module, we try to improve the visual representation ability by fusing the saliency information learning from different convolution lays. With leveraging lightweight Convolutional Neural Network VGG-M as the features extractor backbone, we can attain robust appearance model by deep features with fruitful semantic information. Second, as for the classifier has significant complementary guidance for location prediction, we propose to generate diverse feature instances of the target by introducing the adversarial learning strategy. Given the generated diverse instances, many complex situations in the tracking process can be effectively simulated, especially the occlusion that conformed to the long tail distribution. Third, to optimize the bounding boxes refinement, we employ a precise pooling strategy for attaining feature maps with high resolution. Then, our approach can capture the subtle appearance changes effectively over a long time range. Finally, extensive experiments was conducted on several benchmark datasets, the results demonstrate that the proposed approach performs favorably against many state-of-the-art algorithms.

     

变分调整约束下的反向低秩稀疏学习目标跟踪

目的 低秩稀疏学习目标跟踪算法在目标快速运动和严重遮挡等情况下容易出现跟踪漂移现象,为此提出一种变分调整约束下的反向低秩稀疏学习目标跟踪算法。方法 采用核范数凸近似低秩约束描述候选粒子间的时域相关性,去除不相关粒子,适应目标外观变化。通过反向稀疏表示描述目标表观,用候选粒子稀疏表示目标模板,减少在线跟踪中L₁优化问题的数目,提高跟踪效率。在有界变差空间利用变分调整对稀疏系数差分建模,约束目标表观在相邻帧间具有较小变化,但允许连续帧间差异存在跳跃不连续性,以适应目标快速运动。结果 实验利用OTB（object tracking benchmark）数据集中的4组涵盖了严重遮挡、快速运动、光照和尺度变化等挑战因素的标准视频序列进行测试,定性和定量对比了本文算法与5种热点算法的跟踪效果。定性分析基于视频序列的主要挑战因素进行比较,定量分析通过中心点位置误差（central pixel error,CPE）比较跟踪算法的精度。与CNT（convolutional networks training）、SCM（sparse collaborative model）、IST（inverse sparse tracker）、DDL（discriminative dictionary learning）和LLR（locally low-rank representation）算法相比,平均CPE值分别提高了2.80、4.16、13.37、35.94和41.59。实验结果表明,本文算法达到了较高的跟踪精度,对上述挑战因素更具鲁棒性。结论 本文提出的跟踪算法,综合了低秩稀疏学习和变分优化调整的优势,在复杂场景下具有较高的跟踪精度,特别是对严重遮挡和快速运动情况的有效跟踪更具鲁棒性。     

Online multi-view subspace learning via group structure analysis for visual object tracking

In this paper, we focus on incrementally learning a robust multi-view subspace representation for visual object tracking. During the tracking process, due to the dynamic background variation and target appearance changing, it is challenging to learn an informative feature representation of tracking object, distinguished from the dynamic background. To this end, we propose a novel online multi-view subspace learning algorithm (OMEL) via group structure analysis, which consistently learns a low-dimensional representation shared across views with time changing. In particular, both group sparsity and group interval constraints are incorporated to preserve the group structure in the low-dimensional subspace, and our subspace learning model will be incrementally updated to prevent repetitive computation of previous data. We extensively evaluate our proposed OMEL on multiple benchmark video tracking sequences, by comparing with six related tracking algorithms. Experimental results show that OMEL is robust and effective to learn dynamic subspace representation for online object tracking problems. Moreover, several evaluation tests are additionally conducted to validate the efficacy of group structure assumption.     

Learning spatially regularized similarity for robust visual tracking

Matching visual appearances of the target object over consecutive frames is a critical step in visual tracking. The accuracy performance of a practical tracking system highly depends on the similarity metric used for visual matching. Recent attempts to integrate discriminative metric learned by sequential visual data (instead of a predefined metric) in visual tracking have demonstrated more robust and accurate results. However, a global similarity metric is often suboptimal for visual matching when the target object experiences large appearance variation or occlusion. To address this issue, we propose in this paper a spatially weighted similarity fusion (SWSF) method for robust visual tracking. In our SWSF, a part-based model is employed as the object representation, and the local similarity metric and spatially regularized weights are jointly learned in a coherent process, such that the total matching accuracy between visual target and candidates can be effectively enhanced. Empirically, we evaluate our proposed tracker on various challenging sequences against several state-of-the-art methods, and the results demonstrate that our method can achieve competitive or better tracking performance in various challenging tracking scenarios.     

Online learning and fusion of orientation appearance models for robust rigid object tracking

We introduce a robust framework for learning and fusing of orientation appearance models based on both texture and depth information for rigid object tracking. Our framework fuses data obtained from a standard visual camera and dense depth maps obtained by low-cost consumer depth cameras such as the Kinect. To combine these two completely different modalities, we propose to use features that do not depend on the data representation: angles. More specifically, our framework combines image gradient orientations as extracted from intensity images with the directions of surface normals computed from dense depth fields. We propose to capture the correlations between the obtained orientation appearance models using a fusion approach motivated by the original Active Appearance Models (AAMs). To incorporate these features in a learning framework, we use a robust kernel based on the Euler representation of angles which does not require off-line training, and can be efficiently implemented online. The robustness of learning from orientation appearance models is presented both theoretically and experimentally in this work. This kernel enables us to cope with gross measurement errors, missing data as well as other typical problems such as illumination changes and occlusions. By combining the proposed models with a particle filter, the proposed framework was used for performing 2D plus 3D rigid object tracking, achieving robust performance in very difficult tracking scenarios including extreme pose variations.