2-D mesh-based video object segmentation and tracking with occlusion resolution

This paper integrates fully automatic video object segmentation and tracking including detection and assignment of uncovered regions in a 2-D mesh-based framework. Particular contributions of this work are (i) a novel video object segmentation method that is posed as a constrained maximum contrast path search problem along the edges of a 2-D triangular mesh, and (ii) a 2-D mesh-based uncovered region detection method along the object boundary as well as within the object. At the first frame, an optimal number of feature points are selected as nodes of a 2-D content-based mesh. These points are classified as moving (foreground) and stationary nodes based on multi-frame node motion analysis, yielding a coarse estimate of the foreground object boundary. Color differences across triangles near the coarse boundary are employed for a maximum contrast path search along the edges of the 2-D mesh to refine the boundary of the video object. Next, we propagate the refined boundary to the subsequent frame by using motion vectors of the node points to form the coarse boundary at the next frame. We detect occluded regions by using motion-compensated frame differences and range filtered edge maps. The boundaries of detected uncovered regions are then refined by using the search procedure. These regions are either appended to the foreground object or tracked as new objects. The segmentation procedure is re-initialized when unreliable motion vectors exceed a certain number. The proposed scheme is demonstrated on several video sequences.     

Multimodal object tracking by exploiting appearance and class information

In this work, we study the method exploiting natural language network to improve tracking performance. We propose a novel architecture which can combine class and visual information presented in tracking. To this end, we introduce a multimodal feature association network, allowing us to correlate the target class with its appearance during training and aid the localization of the target during inference. Specifically, we first utilize an appearance model to extract the target visual features, from which we obtain appearance cues, for instance shape and color. In order to employ target class information, we design a learned lightweight embedding network to embed the target class into a feature representation. The association network of our architecture contains a multimodal fusion module and a predictor module. The fusion module is used to combine features from class and appearance, yielding multimodal features with more expressive representations for the subsequent module. The predictor module is used to determine the target location in the current frame, from which we associate the class to the appearance. The class embedding module thus can learn appearance cues by exploiting the back-propagation functionality. To verify the abilities of our method, we select the official training and test splits of the LaSOT with annotated images and classes to perform experiments. In particular, we analyze the imbalance in the samples and employ a class validator discriminator to alleviate this problem. Extensive experimental results on LaSOT, UAV20L and UAV123@10fps demonstrate our method achieves competitive results while maintaining a considerable real-time speed.     

Correlation filter via random-projection based CNNs features combination for visual tracking

Object tracking based on the Convolutional Neural Networks (CNNs) with multiple feature correlation filter (CF) has become one of the best object tracking frameworks. In this paper, we propose a novel approach of CNNs based CF, which combines deep features from CNNs into low-dimensional features. To achieve the dimensionality reduction, random-projection is used due to its data-independence and superior computational efficiency over other widely used. In our proposed approach, the spectral graph theory is applied to generate a random projection matrix. This method bypasses the time-consuming Gram–Schmidt orthogonalization, where the dimension of the feature is high. The combined features have very low dimensions, less than one tenth of the dimensions of the original deep features from CNNs, offering an improvement of tracking speed and without loss of performance simultaneously. Extensive experiments are conducted on large-scale benchmark datasets. The results demonstrate that the proposed algorithm outperforms the state-of-the-art methods.     

Multiple instance deep learning for weakly-supervised visual object tracking

Intelligently tracking objects with varied shapes, color, lighting conditions, and backgrounds is an extremely useful application in many HCI applications, such as human body motion capture, hand gesture recognition, and virtual reality (VR) games. However, accurately tracking different objects under uncontrolled environments is a tough challenge due to the possibly dynamic object parts, varied lighting conditions, and sophisticated backgrounds. In this work, we propose a novel semantically-aware object tracking framework, wherein the key is weakly-supervised learning paradigm that optimally transfers the video-level semantic tags into various regions. More specifically, give a set of training video clips, each of which is associated with multiple video-level semantic tags, we first propose a weakly-supervised learning algorithm to transfer the semantic tags into various video regions. The key is a MIL (Zhong et al., 2020) [1]-based manifold embedding algorithm that maps the entire video regions into a semantic space, wherein the video-level semantic tags are well encoded. Afterward, for each video region, we use the semantic feature combined with the appearance feature as its representation. We designed a multi-view learning algorithm to optimally fuse the above two types of features. Based on the fused feature, we learn a probabilistic Gaussian mixture model to predict the target probability of each candidate window, where the window with the maximal probability is output as the tracking result. Comprehensive comparative results on a challenging pedestrian tracking task as well as the human hand gesture recognition have demonstrated the effectiveness of our method. Moreover, visualized tracking results have shown that non-rigid objects with moderate occlusions can be well localized by our method.     

基于TLD框架的上下文目标跟踪算法

提出了一种基于TLD (Tracking-Learning-Detection)框架的上下文目标跟踪算法.在TLD框架中,融入时空上下文跟踪算法,提高跟踪器的鲁棒性和稳定性.引入Kalman滤波来处理目标被严重遮挡时跟踪失效的问题.此外,采用由粗到精的搜索策略进行目标检测,利用帧差法确定运动目标疑似区域,提高检测效率.实验结果表明所提出的算法具有较好的鲁棒性和实时性.     

虚拟场景生成中摄像机机电跟踪的校准

在虚拟场景生成中，虚拟摄像机一般是小孔相机，但是带有机电跟踪系统的真实摄像机与在固定位置的小孔相机的运转是不同的。校准了由于摄像机的机电跟踪系统引起的误差。     

具有遮挡鲁棒性的目标中心距离加权的跟踪算法

本文提出了一种实时跟踪算法,以目标中心距离加权的目标图像直方图作为模板,采用mean-shift迭代方法进行目标定位;当目标被部分遮挡时,用"分块匹配"的方法提高算法鲁棒性;对于超过一定像素的较大目标,本算法进行"降采样",大大减小运算量,从而实现了对大尺度目标的实时跟踪.实时视频流的实际跟踪系统验证了该方法的有效性.     

Online Boosting tracking algorithm combined with occlusion sensing

Online Boosting tracking algorithm combined with occlusion sensing was presented.In this method,occlusion sensor was introduced to check the tracking results,and classifier updating strategy was adjusted depending on the occlusion checking results.By this way,the feature pool of the classifier can be kept pure,which will improve the tracking robustness under occlusion.Experimental results show that compared with traditional Boosting tracking algorithm,improved algorithm can solve the problem of occlusion very well.     

Effective appearance model for probabilistic object tracking

This paper presents a robust object tracking approach via a spatially constrained colour model. Local image patches of the object and spatial relation between these patches are informative and stable during object tracking. So, we propose to partition an object into patches and develop a Spatially Constrained Colour Model (SCCM) by combining the colour distributions and spatial configuration of these patches. The likelihood of the candidate object is given by estimating the confidences of the pixels in the ...     

一种改进的Camshift目标跟踪方法

为了解决单特征在目标跟踪中无法准确描述目标的问题,提出了一种多特征融合的实时目标跟踪方法。该方法将角点特征、轮廓特征融入传统的Camshift算法中,结合原有的颜色特征对目标进行描述。解决了传统算法易受同色物体干扰,抗遮挡性能差等问题。实验结果表明,该方法能够实现对目标的实时跟踪,当目标遮挡的时间较短时能够很好地识别目标,具有较高的鲁棒性。     

Robust object tracking via deformation samples generator

In object tracking applications, it is common for trackers to experience drift problems when the object of interest becomes deformed, which compromises the ability of the tracker to track the object. It is therefore desirable to develop a learning tracker classifier that is robust to deformations. The performance of existing trackers that employ deep classification networks degrades when the amount of training data is limited and does not cover all possible scenarios. While these limitations can be mitigated in part by using larger training datasets, these datasets may still not cover all situations and the positive samples are still monotonous. To overcome this problem, we propose a novel deformation samples generator that generates samples that would normally be difficult for the tracker to classify. In the proposed framework, both the classifier and deformation samples generator learn in a joint manner. Our experiments show that the proposed approach outperforms state-of-the-art methods in both quantitative and qualitative evaluations for the visual object tracking task.     

Adaptive model updating for robust object tracking

In this paper, we exploit features extracted from convolutional neural network (CNN) to be better utilized for visual tracking. It is observed that CNN features in higher levels provide semantic information which is robust to appearance variations. Thus we integrate the hierarchical features in different layers of a deep model to correlation filter tracking framework. More specifically, correlation filters are learned on each layer to encode the object appearance. The peak-to-sidelobe ratio (PSR) is employed to measure the differences between image patches. To leverage the robustness of our model, we develop an adaptive model updating scheme to train the correlation filters according to different response maps. Extensive experimental results on three large scale benchmark datasets show that the proposed algorithm performs favorably against state-of-the-art methods.     

A novel fast moving object contour tracking algorithm

If a somewhat fast moving object exists in a complicated tracking environment, snake’s nodes may fall into the inaccurate local minima. We propose a mean shift snake algorithm to solve this problem. However, if the object goes beyond the limits of mean shift snake module operation in suc- cessive sequences, mean shift snake’s nodes may also fall into the local minima in their moving to the new object position. This paper presents a motion compensation strategy by using particle filter; therefore a new Parti...     

Iterative optimization for frame-by-frame object pose tracking

Joint object tracking and pose estimation is an important issue in Augmented Reality (AR), interactive systems, and robotic systems. Many studies are based on object detection methods that only focus on the reliability of the features. Other methods combine object detection with frame-by-frame tracking using the temporal redundancy in the video. However, in some mixed methods, the interval between consecutive detection frames is usually too short to take the full advantage of the frame-by-frame tracking, or there is no appropriate switching mechanism between detection and tracking. In this paper, an iterative optimization tracking method is proposed to alleviate the deviations of the tracking points and prolong the interval, and thus speed up the pose estimation process. Moreover, an adaptive detection interval algorithm is developed, which can make the switch between detection and frame-by-frame tracking automatically according to the quality of frames so as to improve the accuracy in a tough tracking environment. Experimental results on the benchmark dataset manifest that the proposed algorithms, as an independent part, can be combined with some inter-frame tracking methods for optimization.     

基于局部模板匹配的运动目标跟踪

针对环境中障碍物对被跟踪目标构成不可预知的遮挡问题,提出了一种新的基于局部区域特征匹配的跟踪算法.首先采用一组基本观察片图模拟目标的外观;其次提出了一种将运动轨迹特性与动态模型结合的采样结构,采用马尔可夫链蒙特卡洛(MCMC,Markov chain Monte Carlo)方法独立估计每个基本片图的状态,并使用运动区...     

Multi-frame co-saliency spatio-temporal regularization correlation filters for object tracking

The spatial regularization weight of the correlation filter is not related to the object content and the model degradation in the tracking process. To solve this problem, a new multi-frame co-saliency spatio-temporal regularization correlation filters (MCSRCF) is proposed for visual object tracking. To the best our knowledge, this is the first application of co-saliency regularization to CF-based tracking. In MCSRCF, grayscale features, directional gradient histogram (HOG) features and CNN features are extracted to improve the tracking precision of the tracker. Secondly, the three-dimensional spatial saliency and semantic saliency are introduced to obtain the initial weight of the spatial regularization with object content information. Then, the heterogeneous saliency fusion method is exploited to add a co-saliency spatial regularization term to the objective function to make the spatial penalty weight learn the change of the object region. In additional, the temporal saliency regularization is introduced to learn the information between adjacent frames, which reduces the overfitting effect caused by inaccurate samples. A variety of evaluations are conducted on public benchmarks, and the experimental results show that the proposed tracker achieves good robustness against many state-of-the-art trackers in various complex scenarios.     

Collaborative object tracking model with local sparse representation

There existed many visual tracking methods that are based on sparse representation model, most of them were either generative or discriminative, which made object tracking more difficult when objects have undergone large pose change, illumination variation or partial occlusion. To address this issue, in this paper we propose a collaborative object tracking model with local sparse representation. The key idea of our method is to develop a local sparse representation-based discriminative model (SRDM) and a local sparse representation-based generative model (SRGM). In the SRDM module, the appearance of a target is modeled by local sparse codes that can be formed as training data for a linear classifier to discriminate the target from the background. In the SRGM module, the appearance of the target is represented by sparse coding histogram and a sparse coding-based similarity measure is applied to compute the distance between histograms of a target candidate and the target template. Finally, a collaborative similarity measure is proposed for measuring the difference of the two models, and then the corresponding likelihood of the target candidates is input into a particle filter framework to estimate the target state sequentially over time in visual tracking. Experiments on some publicly available benchmarks of video sequences showed that our proposed tracker is robust and effective.     

一种融合遮挡分割的多目标跟踪算法

复杂环境下的多目标视频跟踪是计算机视觉领域的一个难点,有效处理目标间遮挡是解决多目标跟踪问题的关键。提出了一种融合遮挡分割的多目标跟踪算法,计算每个目标的光流速度概率直方图,反映其运动统计信息;综合使用外观、运动、颜色信息构造新的像素距离表达,借助分阶段分类思想及K均值聚类技术进行遮挡分割,得到准确的运动前景像素;在粒子滤波器跟踪框架下,使用概率外观模型进行多目标跟踪,更好地处理动态遮挡问题。实验表明,所提算法解决了复杂环境下的多目标跟踪问题。     

Video object segmentation and tracking using region-based statistics

Two new region-based methods for video object tracking using active contours are presented. The first method is based on the assumption that the color histogram of the tracked object is nearly stationary from frame to frame. The proposed method is based on minimizing the color histogram difference between the estimated objects at a reference frame and the current frame using a dynamic programming framework. The second method is defined for scenes where there is an out-of-focus blur difference between the object of interest and the background. In such scenes, the proposed “defocus energy” can be utilized for automatic segmentation of the object boundary, and it can be combined with the histogram method to track the object more efficiently. Experiments demonstrate that the proposed methods are successful in difficult scenes with significant background clutter.