Multiscale and local search methods for real time region tracking with particle filters: local search driven by adaptive scale estimation on GPUs

Tracking systems are important in computervision, with applications in surveillance, human computer interaction, etc. Consumer graphics processing units (GPUs) have experienced an extraordinary evolution in both computing performance and programmability, leading to greater use of the GPU for non-rendering applications. In this work we propose a real-time object tracking algorithm, based on the hybridization of particle filtering (PF) and a multi-scale local search (MSLS) algorithm, presented for both CPU and GPU architectures. The developed system provides successful results in precise tracking of single and multiple targets in monocular video, operating in real-time at 70 frames per second for 640 × 480 video resolutions on the GPU, up to 1,100% faster than the CPU version of the algorithm.     

Real-time shape tracking of facial landmarks

Detection of facial landmarks and accurate tracking of their shape are essential in real-time applications such as virtual makeup, where users can see the makeup’s effect by moving their face in diverse directions. Typical face tracking techniques detect facial landmarks and track them using a point tracker such as the Kanade-Lucas-Tomasi (KLT) point tracker. Typically, 5 or 64 points are used for tracking a face. Even though these points are enough to track the approximate locations of facial landmarks, they are not sufficient to track the exact shape of facial landmarks. In this paper, we propose a method that can track the exact shape of facial landmarks in real-time by combining a deep learning technique and a point tracker. We detect facial landmarks accurately using SegNet, which performs semantic segmentation based on deep learning. Edge points of detected landmarks are tracked using the KLT point tracker. In spite of its popularity, the KLT point tracker suffers from the point loss problem. We solve this problem by executing SegNet periodically to recalculate the shape of facial landmarks. That is, by combining the two techniques, we can avoid the computational overhead of SegNet and the point loss problem of the KLT point tracker, which leads to accurate real-time shape tracking. We performed several experiments to evaluate the performance of our method and report some of the results herein.

     

A large margin framework for single camera offline tracking with hybrid cues

We introduce MMTrack (max-margin tracker), a single-target tracker that linearly combines constant and adaptive appearance features. We frame offline single-camera tracking as a structured output prediction task where the goal is to find a sequence of locations of the target given a video. Following recent advances in machine learning, we discriminatively learn tracker parameters by first generating suitable bad trajectories and then employing a margin criterion to learn how to distinguish among ground truth trajectories and all other possibilities. Our framework for tracking is general, and can be used with a variety of features. We demonstrate a system combining a variety of appearance features and a motion model, with the parameters of these features learned jointly in a coherent learning framework. Further, taking advantage of a reliable human detector, we present a natural way of extending our tracker to a robust detection and tracking system. We apply our framework to pedestrian tracking and experimentally demonstrate the effectiveness of our method on two real-world data sets, achieving results comparable to state-of-the-art tracking systems.     

Mean shift tracker combined with online learning-based detector and Kalman filtering for real-time tracking

Color-based visual object tracking is one of the most commonly used tracking methods. Among many tracking methods, the mean shift tracker is used most often because it is simple to implement and consumes less computational time. However, mean shift trackers exhibit several limitations when used for long-term tracking. In challenging conditions that include occlusions, pose variations, scale changes, and illumination changes, the mean shift tracker does not work well. In this paper, an improved tracking algorithm based on a mean shift tracker is proposed to overcome the weaknesses of existing methods based on mean shift tracker. The main contributions of this paper are to integrate mean shift tracker with an online learning-based detector and to newly define the Kalman filter-based validation region for reducing computational burden of the detector. We combine the mean shift tracker with the online learning-based detector, and integrate the Kalman filter to develop a novel tracking algorithm. The proposed algorithm can reinitialize the target when it converges to a local minima and it can cope with scale changes, occlusions and appearance changes by using the online learning-based detector. It updates the target model for the tracker in order to ensure long-term tracking. Moreover, the validation region obtained by using the Kalman filter and the Mahalanobis distance is used in order to operate detector in real-time. Through a comparison against various mean shift tracker-based methods and other state-of-the-art methods on eight challenging video sequences, we demonstrate that the proposed algorithm is efficient and superior in terms of accuracy and speed. Hence, it is expected that the proposed method can be applied to various applications which need to detect and track an object in real-time.     

基于尺度自适应均值偏移优化的TLD跟踪算法

为解决目标在形变、遮挡和快速运动时所导致的跟踪失败,在经典TLD算法的框架下,使用尺度自适应均值偏移算法重新设计跟踪器,提出了MS-TLD算法.通过引入颜色直方图特征和尺度自适应,跟踪器能准确跟踪形变和快速运动的目标.设计跟踪-检测反馈机制,通过跟踪器和检测器相互校正,使新算法在目标被遮挡时具有很好的跟踪鲁棒性.采用TB-50标准测试集进行了实验验证与评测,结果表明所提出算法有效克服了由于目标形变、遮挡和快速运动以及背景干扰所导致的跟踪失败,比TLD等4种经典算法具有更好的跟踪准确性和鲁棒性.     

遮挡判别下的多尺度相关滤波跟踪算法

目的 复杂环境下,运动目标在跟踪过程中受尺度变换以及遮挡因素的影响,跟踪准确率较低。针对这一问题,提出一种遮挡判别下的多尺度相关滤波跟踪方法。方法 首先选取第1帧图像的前景区域,训练目标的位置、尺度滤波器和GMS（grid-based motion statistics）检测器。然后,通过位置滤波器估计目标位置,尺度滤波器计算目标尺度,得到初选目标区域。最后,利用相关滤波响应情况对初选目标区域进行评估,通过相关滤波响应值的峰值和峰值波动情况判断是否满足遮挡和更新条件。若遮挡,启动检测器检测目标位置,检测到目标位置后,更新目标模型;若更新,则更新位置、尺度滤波器和GMS检测器,完成跟踪。结果 本文使用多尺度相关滤波方法作为算法的基本框架,对尺度变化目标跟踪具有较好的适应性。同时,利用目标模型更新机制和GMS检测器检索目标,有效地解决了遮挡情况下的目标丢失问题。在公开数据集上的测试结果表明,本文算法平均中心误差为5.58,平均跟踪准确率为94.2%,跟踪速度平均可达27.5 帧/s,与当前先进的跟踪算法相比,本文算法兼顾了跟踪速度和准确率,表现出更好的跟踪效果。结论 本文提出一种新的遮挡判别下的多尺度相关滤波跟踪算法。实验结果表明,本文算法在不同的尺度变换及遮挡条件下能够快速准确跟踪目标,具有较好的跟踪准确率和鲁棒性。     

基于并行跟踪检测框架与深度学习的目标跟踪算法

在空地协同背景下，地面目标的移动导致其在无人机视角下外观会发生较大变化，传统算法很难满足此类场景的应用要求。针对这一问题，提出基于并行跟踪和检测（PTAD）框架与深度学习的目标检测与跟踪算法。首先，将基于卷积神经网络（CNN）的目标检测算法SSD作为PTAD的检测子处理关键帧获取目标信息并提供给跟踪子；其次，检测子与跟踪子并行处理图像帧并计算检测与跟踪结果框的重叠度及跟踪结果的置信度；最后，根据跟踪子与检测子的跟踪或检测状态来判断是否对跟踪子或检测子进行更新，并对图像帧中的目标进行实时跟踪。在无人机视角下的视频序列上开展实验研究和对比分析，结果表明所提算法的性能高于PTAD框架下最优算法，而且实时性提高了13%，验证了此算法的有效性。     

Tracking using Numerous Anchor Points

In this paper, an online adaptive model-free tracker is proposed to track single objects in video sequences to deal with real-world tracking challenges like low-resolution, object deformation, occlusion and motion blur. The novelty lies in the construction of a strong appearance model that captures features from the initialized bounding box and then are assembled into anchor point features. These features memorize the global pattern of the object and have an internal star graph-like structure. These features are unique and flexible and help tracking generic and deformable objects with no limitation on specific objects. In addition, the relevance of each feature is evaluated online using short-term consistency and long-term consistency. These parameters are adapted to retain consistent features that vote for the object location and that deal with outliers for long-term tracking scenarios. Additionally, voting in a Gaussian manner helps in tackling inherent noise of the tracking system and in accurate object localization. Furthermore, the proposed tracker uses pairwise distance measure to cope with scale variations and combines pixel-level binary features and global weighted color features for model update. Finally, experimental results on a visual tracking benchmark dataset are presented to demonstrate the effectiveness and competitiveness of the proposed tracker.     

使用PSR重检测改进的核相关目标跟踪方法

针对传统的基于核相关滤波器的跟踪方法（KCF）缺少跟踪失败检测的问题,提出了一种改进的KCF目标跟踪方法。改进的KCF跟踪器采用高斯窗口方法在目标位置上截取训练样本,这种采样方法可以获得更有效的目标信噪比并同时减少背景干扰信息的引入,从而使跟踪器可以在复杂场景下具有更强的适应性。在目标跟踪的过程中,通过相关运算的峰值旁瓣比检测目标跟踪是否失败,并在相关匹配值较高的位置学习目标检测器。一旦检测到跟踪失败,便对跟踪器进行纠正,恢复目标跟踪。通过实验验证了改进算法的鲁棒性,相比传统的KCF跟踪器的总体性能提高了13.2%。     

基于区块链技术的高速移动视点视频监控跟踪系统设计

针对传统高速移动视点视频监控跟踪系统视频信号收集能力差,追踪准确率低,课题在区块链技术基础上设计了一种新的视点视频监控跟踪系统。系统硬件设计分为视频监视模块、数据定位模块以及视频监控跟踪模块三个模块进行研究操作,在视频监控跟踪模块中根据硬件元件结构与性质对其进行系统掌控,辅助BDL9830QD监视器强化内部系统监视功能,时刻保持系统中心监控操作,在数据定位模块中,综合数据所处状态,选择数据微型定位器对信号进行追踪定位,标定定位目标,调整数据状态,在视频监控跟踪模块中选用HX-YT01 自动视频追踪器加大对数据信号的跟踪力度,实现精准化监控跟踪,由此完成系统硬件设计。在系统应用程序设计中综合硬件元件特点进行程序改造,构建区块链空间,实现对系统的整体设计。实验结果表明,基于区块链技术的高速移动视点视频监控跟踪系统的追踪能力提高了15.21%,追踪结果准确率提高了22.8%。该设计能够在较高程度上强化系统监控跟踪性能,同时缩减系统操作所需时间,提升整体系统运行效率,能够更好的为使用者提供优质理论研究操作。     

基于PF的窗口自适应Mean-Shift跟踪算法

在视频目标跟踪过程中,Mean-Shift算法存在着核函数带宽固定不变的缺陷,对尺度大小发生变化的目标无法进行有效跟踪。提出一种多尺度理论与粒子滤波器（PF）相结合的改进算法。通过粒子滤波器对多尺度理论统计得到的跟踪窗信息量进行预测修正,据此计算核窗宽大小变化的比例系数,实现跟踪算法的窗口自适应能力。实验结果表明,改进的跟踪算法对尺寸逐渐减小和逐渐增大的目标均能自动选择合适的跟踪窗口大小。     

Trajectory-based vehicle tracking at low frame rates

In smart cities, an intelligent traffic surveillance system plays a crucial role in reducing traffic jams and air pollution, thus improving the quality of life. An intelligent traffic surveillance should be able to detect and track multiple vehicles in real-time using only limited resources. Conventional tracking methods usually run at a high video-sampling rate, assuming that the same vehicles in successive frames are similar and move only slightly. However, in cost effective embedded surveillance systems (e.g., a distributed wireless network of smart cameras), video frame rates are typically low because of limited system resources. Therefore, conventional tracking methods perform poorly in embedded surveillance systems because of discontinuity of the moving vehicles in the captured recordings. In this study, we present a fast and light algorithm that is suitable for an embedded real-time visual surveillance system to detect effectively and track multiple moving vehicles whose appearance and/or position changes abruptly at a low frame rate. For effective tracking at low frame rates, we propose a new matching criterion based on greedy data association using appearance and position similarities between detections and trackers. To manage abrupt appearance changes, manifold learning is used to calculate appearance similarity. To manage abrupt changes in motion, the next probable centroid area of the tracker is predicted using trajectory information. The position similarity is then calculated based on the predicted next position and progress direction of the tracker. The proposed method demonstrates efficient tracking performance during rapid feature changes and is tested on an embedded platform (ARM with DSP-based system).     

A real-time vehicle detection and a novel vehicle tracking systems for estimating and monitoring traffic flow on highways

Real-time highway traffic monitoring systems play a vital role in road traffic management, planning, and preventing frequent traffic jams, traffic rule violations, and fatal road accidents. These systems rely entirely on online traffic flow info estimated from time-dependent vehicle trajectories. Vehicle trajectories are extracted from vehicle detection and tracking data obtained by processing road-side camera images. General-purpose object detectors including Yolo, SSD, EfficientNet have been utilized extensively for real-time object detection task, but, in principle, Yolo is preferred because it provides a high frame per second (FPS) performance and robust object localization functionality. However, this algorithm’s average vehicle classification accuracy is below 57%, which is insufficient for traffic flow monitoring. This study proposes improving the vehicle classification accuracy of Yolo, and developing a novel bounding box (Bbox)-based vehicle tracking algorithm. For this purpose, a new vehicle dataset is prepared by annotating 7216 images with 123831 object patterns collected from highway videos. Nine machine learning-based classifiers and a CNN-based classifier were selected. Next, the classifiers were trained via the dataset. One out of ten classifiers with the highest accuracy was selected to combine to Yolo. This way, the classification accuracy of the Yolo-based vehicle detector was increased from 57% to 95.45%. Vehicle detector 1 (Yolo) and vehicle detector 2 (Yolo + best classifier), and the Kalman filter-based tracking as vehicle tracker 1 and the Bbox-based tracking as vehicle tracker 2 were applied to the categorical/total vehicle counting tasks on 4 highway videos. The vehicle counting results show that the vehicle counting accuracy of the developed approach (vehicle detector 2 + vehicle tracker 2) was improved by 13.25% and this method performed better than the other 3 vehicle counting systems implemented in this study.     

Multi-scale patch-based sparse appearance model for robust object tracking

When objects undergo large pose change, illumination variation or partial occlusion, most existing visual tracking algorithms tend to drift away from targets and even fail to track them. To address the issue, in this paper we propose a multi-scale patch-based appearance model with sparse representation and provide an efficient scheme involving the collaboration between multi-scale patches encoded by sparse coefficients. The key idea of our method is to model the appearance of an object by different scale patches, which are represented by sparse coefficients with different scale dictionaries. The model exploits both partial and spatial information of targets based on multi-scale patches. Afterwards, a similarity score of one candidate target is input into a particle filter framework to estimate the target state sequentially over time in visual tracking. Additionally, to decrease the visual drift caused by frequently updating model, we present a novel two-step object tracking method which exploits both the ground truth information of the target labeled in the first frame and the target obtained online with the multi-scale patch information. Experiments on some publicly available benchmarks of video sequences showed that the similarity involving complementary information can locate targets more accurately and the proposed tracker is more robust and effective than others.     

Markov chain Monte Carlo based video tracking algorithm

The paper considers a problem of multiple person tracking. We present the algorithm to automatic people tracking on surveillance videos recorded by static cameras. Proposed algorithm is an extension of approach based on tracking-by-detection of people heads and data association using Markov chain Monte Carlo (MCMC). Short track fragments (tracklets) are built by local tracking of people heads. Tracklet postprocessing and accurate results interpolation were shown to reduce number of false positives. We use position deviations of tracklets and revised entry/exit points factor to separate pedestrians from false positives. The paper presents a new method to estimate body position, that increases precision of tracker. Finally, we switched HOG-based detector to cascade one. Our evaluation shows proposed modifications significantly increase tracking quality.     

FPGA–DSP co-processing for feature tracking in smart video sensors

Motion estimation in videos is a computationally intensive process. A popular strategy for dealing with such a high processing load is to accelerate algorithms with dedicated hardware such as graphic processor units (GPU), field programmable gate arrays (FPGA), and digital signal processors (DSP). Previous approaches addressed the problem using accelerators together with a general purpose processor, such as acorn RISC machines (ARM). In this work, we present a co-processing architecture using FPGA and DSP. A portable platform for motion estimation based on sparse feature point detection and tracking is developed for real-time embedded systems and smart video sensors applications. A Harris corner detection IP core is designed with a customized fine grain pipeline on a Virtex-4 FPGA. The detected feature points are then tracked using the Lucas–Kanade algorithm in a DSP that acts as a co-processor for the FPGA. The hybrid system offers a throughput of 160 frames per second (fps) for VGA image resolution. We have also tested the benefits of our proposed solution (FPGA + DSP) in comparison with two other traditional architectures and co-processing strategies: hybrid ARM + DSP and DSP only. The proposed FPGA + DSP system offers a speedup of about 20 times and 3 times over ARM + DSP and DSP only configurations, respectively. A comparison of the Harris feature detection algorithm performance between different embedded processors (DSP, ARM, and FPGA) reveals that the DSP offers the best performance when scaling up from QVGA to VGA resolutions.     

Architecture for video streaming application on heterogeneous platform

In this paper, we propose a new architecture, which is an efficient streaming media player application on heterogeneous platform. The streaming library can be used in design for reducing the memory bandwidth on processing RTSP/RTP/RTCP network protocols. And the proposed improvement has to do with the I-frame encoding alone. The architecture can receive higher rate for data transfer and packet loss in embedded system. The framework of the key components is able to adopt Direct Memory Access to reduce the time-consumption resulted from the communication between the dual cores. On the other hand, the approach of the dynamic quality adaption improves the video bitstream by periodically adjusting the values of encoding quality parameters. Through the experiment results, it is evident that the new video streaming architecture greatly enhances the coding efficacy. Our experimental results present that the decoding/encoding speed of the dual-core CPU embedded with Direct Memory Access is enhanced up to 50 %, and its usage of CPU resources and memory bandwidth are 50 % lower than that of the popular JRTPLIB.     

基于关键特征点的改进TLD目标跟踪算法研究

TLD（Tracking-Learning-Detection）算法是一种新颖的单目标长时间视觉跟踪算法,在给定极少的先验知识的情况下,能够迅速地学习目标特征并进行有效的跟踪。TLD算法中跟踪器每次在跟踪目标上均匀地选取特征点进行跟踪,不能保证每个特征点都能够被可靠地跟踪。针对这个问题,提出一种基于关键特征点检测的改进TLD算法,保证所选特征点都能够被正确可靠地跟踪,防止跟踪结果发生漂移,提高了跟踪器的跟踪精度。另一方面,在TLD检测器中引入了基于轨迹连续性的在线位置预测,在保证正确跟踪的前提下,缩小了检测器的检测范围,提高了运算速度。实验结果表明,该算法有较高的跟踪精度和速度。     

一种基于Hausdorff距离的运动物体跟踪算法

针对视频处理中运动物体的检测和跟踪问题,提出了一种基于Hausdorff距离的目标跟踪算法。新算法提出首先采用多尺度分水岭变换获取运动物体模型,消除了传统基于分水岭变换算法存在的缺陷;然后使用部分Hausdorff距离实现后续帧中运动物体模型的匹配;最后再次使用多尺度分水岭算法完成运动物体模型的更新。实验表明,该算法可以有效地跟踪多个刚体或非刚体目标。     

无人车驾驶场景下的多目标车辆与行人跟踪算法

考虑到现有的基于检测的多目标跟踪算法多会出现因目标漏检或数据关联算法冗余而造成的目标ID频繁切换、跟踪轨迹断开等问题,提出了无人车驾驶场景下的多目标车辆与行人跟踪算法.首先,选取CenterNet网络作为目标检测器,并用嵌入了1×1卷积和SE-Net的Res2Net来替代网络原有的残差单元,以提升网络对空间信息和通道信息的提取能力,提高目标检测器性能.接着,用孪生网络来提取目标所在区域的特征,进行关联概率度量,再用匈牙利算法对相邻帧目标进行关联.最后,用区域推荐网络设计的辅助跟踪器对漏检或消失又出现的目标进行持续跟踪,并将可靠的跟踪结果合并到轨迹中.实验结果表明,与已有的方法对比,所提方法在KITTI跟踪基准数据集上对于车辆与行人的跟踪具有竞争力.