Video object segmentation and tracking using region-based statistics

Two new region-based methods for video object tracking using active contours are presented. The first method is based on the assumption that the color histogram of the tracked object is nearly stationary from frame to frame. The proposed method is based on minimizing the color histogram difference between the estimated objects at a reference frame and the current frame using a dynamic programming framework. The second method is defined for scenes where there is an out-of-focus blur difference between the object of interest and the background. In such scenes, the proposed “defocus energy” can be utilized for automatic segmentation of the object boundary, and it can be combined with the histogram method to track the object more efficiently. Experiments demonstrate that the proposed methods are successful in difficult scenes with significant background clutter.     

动态背景下融合运动线索和颜色信息的视频目标分割算法

针对现有动态背景下目标分割算法存在的局限性,提出了一种融合运动线索和颜色信息的视频序列目标分割算法。首先,设计了一种新的运动轨迹分类方法,利用背景运动的低秩特性,结合累积确认的策略,可以获得准确的运动轨迹分类结果;然后,通过过分割算法获取视频序列的超像素集合,并计算超像素之间颜色信息的相似度;最后,以超像素为节点建立马尔可夫随机场模型,将运动轨迹分类信息以及超像素之间颜色信息统一建模在马尔可夫随机场的能量函数中,并通过能量函数最小化获得每个超像素的最优分类。在多组公开发布的视频序列中进行测试与对比,结果表明,本文方法可以准确分割出动态背景下的运动目标,并且较传统方法具有更高的分割准确率。     

Fragmentation handling for visual tracking

Object detection and tracking using background subtraction suffers from the fragmentation problem which means one object fragments into several blobs because of being similar with the reference image in color. In this paper, we build a visual tracking framework using background subtraction for object detection, and we address the association difficulty of blobs with objects caused by the fragmentation problem by two steps. We firstly cluster the blobs according to the boundary distances of them estimated by an approximating method proposed in this paper. Blobs clustered into the same blob-set are considered from the same object. Secondly, we consider blob-sets possibly from the same object if they exhibit coherent motion, since blobs of the same object may be clustered into different blob-sets if the object fragments severely. A background-matching method is proposed to determine whether two blob-sets exhibiting coherent motion are truly from the same object or from different objects. We test the proposed methods on several real-world video sequences. Quantitative and qualitative experimental results show that the proposed methods handle the problems caused by fragmentation effectively.     

A Robust Particle Filter-Based Method for Tracking Single Visual Object Through Complex Scenes Using Dynamical Object Shape and Appearance Similarity

This paper addresses the issue of tracking a single visual object through crowded scenarios, where a target object may be intersected or partially occluded by other objects for a long duration, experience severe deformation and pose changes, and different motion speed in cluttered background. A robust visual object tracking scheme is proposed that exploits the dynamics of object shape and appearance similarity. The method uses a particle filter where a multi-mode anisotropic mean shift is embedded to improve the initial particles. Comparing with the conventional particle filter and mean shift-based tracking (Shan et al. 2004), our method offers the following novelties: We employ a fully tunable rectangular bounding box described by five parameters (2D central location, width, height, and orientation) and full functionaries in the joint tracking scheme; We derive the equations for the multi-mode version of the anisotropic mean shift where the rectangular bounding box is partitioned into concentric areas, allowing better tracking objects with multiple modes. The bounding box parameters are then computed by using eigen-decomposition of mean shift estimates and weighted averaging. This enables a more efficient re-distributions of initial particles towards locations associated with large weights, hence an efficient particle filter tracking using a very small number of particles (N = 15 is used). Experiments have been conducted on video containing a range of complex scenarios, where tracking results are further evaluated by using two objective criteria and compared with two existing tracking methods. Our results have shown that the propose method is robust in terms of tracking drift, tightness and accuracy of tracked bounding boxes, especially in scenarios where the target object contains long-term partial occlusions, intersections, severe deformation, pose changes, or cluttered background with similar color distributions.     

Level set analysis for leukocyte detection and tracking

We propose a cell detection and tracking solution using image-level sets computed via threshold decomposition. In contrast to existing methods where manual initialization is required to track individual cells, the proposed approach can automatically identify and track multiple cells by exploiting the shape and intensity characteristics of the cells. The capture of the cell boundary is considered as an evolution of a closed curve that maximizes image gradient along the curve enclosing a homogeneous region. An energy functional dependent upon the gradient magnitude along the cell boundary, the region homogeneity within the cell boundary and the spatial overlap of the detected cells is minimized using a variational approach. For tracking between frames, this energy functional is modified considering the spatial and shape consistency of a cell as it moves in the video sequence. The integrated energy functional complements shape-based segmentation with a spatial consistency based tracking technique. We demonstrate that an acceptable, expedient solution of the energy functional is possible through a search of the image-level lines: boundaries of connected components within the level sets obtained by threshold decomposition. The level set analysis can also capture multiple cells in a single frame rather than iteratively computing a single active contour for each individual cell. Results of cell detection using the energy functional approach and the level set approach are presented along with the associated processing time. Results of successful tracking of rolling leukocytes from a number of digital video sequences are reported and compared with the results from a correlation tracking scheme.     

Performance measures for video object segmentation and tracking

We propose measures to evaluate quantitatively the performance of video object segmentation and tracking methods without ground-truth (GT) segmentation maps. The proposed measures are based on spatial differences of color and motion along the boundary of the estimated video object plane and temporal differences between the color histogram of the current object plane and its predecessors. They can be used to localize (spatially and/or temporally) regions where segmentation results are good or bad; and/or they can be combined to yield a single numerical measure to indicate the goodness of the boundary segmentation and tracking results over a sequence. The validity of the proposed performance measures without GT have been demonstrated by canonical correlation analysis with another set of measures with GT on a set of sequences (where GT information is available). Experimental results are presented to evaluate the segmentation maps obtained from various sequences using different segmentation approaches.     

基于粒子滤波与多特征融合的视频目标跟踪

提出了一种基于粒子滤波和多特征融合的视频目标跟踪方法.以粒子滤波为跟踪框架,根据颜色跟踪中存在的问题提出将颜色与目标运动信息融合,利用融合后的信息确定粒子的权值.利用重采样策略缓解退化现象对粒子滤波的影响.针对2段不同的视频进行了不同算法的仿真与性能的比较,实验结果表明,本文方法在计算量增加不多的情况下大大改善了跟踪的性能与鲁棒性,尤其当目标与背景颜色相近时仍然能够准确地对目标进行跟踪.     

Multi-view video based multiple objects segmentation using graph cut and spatiotemporal projections

In this paper, we present an automatic algorithm to segment multiple objects from multi-view video. The Initial Interested Objects (IIOs) are automatically extracted in the key view of the initial frame based on the saliency model. Multiple objects segmentation is decomposed into several sub-segmentation problems, and solved by minimizing the energy function using binary label graph cut. In the proposed novel energy function, the color and depth cues are integrated with the data term, which is then modified with background penalty with occlusion reasoning. In the smoothness term, foreground contrast enhancement is developed to strengthen the moving objects boundary, and at the same time attenuates the background contrast. To segment the multi-view video, the coarse predictions of the other views and the successive frame are projected by pixel-based disparity and motion compensation, respectively, which exploits the inherent spatiotemporal consistency. Uncertain band along the object boundary is shaped based on activity measure and refined with graph cut, resulting in a more accurate Interested Objects (IOs) layer across all views of the frames. The experiments are implemented on a couple of multi-view videos with real and complex scenes. Excellent subjective results have shown the robustness and efficiency of the proposed algorithm.     

联合人体模型与块生长的人群目标分割

人群目标的准确分割,是进行多相机联合目标跟踪与识别的关键。该文首先构造包含位置、尺寸、姿态信息的人体粗略姿态模型,并利用贝叶斯模型获得对应的目标模型。然后将前景区域利用分水岭算法划分为颜色分布一致的子块,通过位置和颜色约束解决遮挡目标的种子块选取问题。最后通过块生长获得每个目标区域。对于颜色相似子块,通过比较其产生的边缘能量确定其所属目标。实验结果表明,本文能够实现对人群目标较精确分割,且对背景噪声具有一定的抗干扰能力。     

基于分块的多特征融合变尺度目标跟踪算法

为了增强彩色视频中目标外观描述能力和解决跟踪过程中目标尺度变化的问题,提出一种基于分块的多特征融合变尺度目标跟踪算法.设计了一个能处理不同挑战因素下对目标的精确跟踪算法,首先提取HSV分块的颜色直方图特征和PCA-HOG特征并采用多通道线性核函数对两种特征进行融合构建训练样本,然后求解线性岭回归函数获得位置核相关滤波器模型,并以线性核函数来计算候选区域在7个尺度空间上与跟踪目标的响应值,最后利用尺度自适应模板更新模型参数.实验结果表明,提出的算法在彩色视频中不仅能较好地自适应目标尺度的变化,在复杂场景下也具有较强的鲁棒性.     

增强抗背景干扰能力的均值移动目标跟踪算法

根据目标和背景颜色直方图的特点,针对异色背景干扰和近色背景干扰,提出了一种改进直方图映射和均值移动结合的目标跟踪算法,通过目标主分量提取和干扰分量鉴别,有效地剔除了背景干扰成分,提高了抗背景干扰能力;均值移动算法在生成灰度图中能快速准确定位目标位置.仿真实验结果证明,改进的直方图映射算法能有效地抑制背景干扰,甚至能抑制与目标色调相近的背景干扰,并验证了跟踪算法的实用性和有效性.     

Invariant‐Feature Based Object Tracking Using Discrete Dynamic Swarm Optimization

With the remarkable growth in rich media in recent years, people are increasingly exposed to visual information from the environment. Visual information continues to play a vital role in rich media because people's real interests lie in dynamic information. This paper proposes a novel discrete dynamic swarm optimization (DDSO) algorithm for video object tracking using invariant features. The proposed approach is designed to track objects more robustly than other traditional algorithms in terms of illumination changes, background noise, and occlusions. DDSO is integrated with a matching procedure to eliminate inappropriate feature points geographically. The proposed novel fitness function can aid in excluding the influence of some noisy mismatched feature points. The test results showed that our approach can overcome changes in illumination, background noise, and occlusions more effectively than other traditional methods, including color‐tracking and invariant feature‐tracking methods.     

Model-based temporal object verification using video

An approach to model-based dynamic object verification and identification using video is proposed. From image sequences containing the moving object, we compute its motion trajectory. Then we estimate its three-dimensional (3-D) pose at each time step. Pose estimation is formulated as a search problem, with the search space constrained by the motion trajectory information of the moving object and assumptions about the scene structure. A generalized Hausdorff (1962) metric, which is more robust to noise and allows a confidence interpretation, is suggested for the matching procedure used for pose estimation as well as the identification and verification problem. The pose evolution curves are used to assist in the acceptance or rejection of an object hypothesis. The models are acquired from real image sequences of the objects. Edge maps are extracted and used for matching. Results are presented for both infrared and optical sequences containing moving objects involved in complex motions     

Segmentation and tracking of moving objects for content-based videocoding

To enable content-based functionalities in video coding, a decomposition of the scene into physical objects is required. Such objects are normally not characterised by homogeneous colour, intensity, or optical flow. Therefore, conventional techniques based on these low-level features cannot perform the desired segmentation. The authors address segmentation and tracking of moving objects and present a new video object plane (VOP) segmentation algorithm that extracts semantically meaningful objects. A morphological motion filter detects physical objects by identifying areas that are moving differently from the background. A new filter criterion is introduced that measures the deviation of the estimated local motion from the synthesised global motion. A two-dimensional binary model is derived for the object of interest and tracked throughout the sequence by a Hausdorff object tracker. To accommodate for rotations and changes in shape, the model is updated every frame by a two-stage method that accounts for rigid and non-rigid moving parts of the object. The binary model then guides the actual VOP extraction, whereby a novel boundary post-processor ensures high boundary accuracy. Experimental results demonstrate the performance of the proposed algorithm     

Efficient multiple moving object detection and tracking using combined background subtraction and clustering

Object detection and tracking is a fundamental, challenging task in computer vision because of the difficulties in tracking. Continuous deformation of objects during movement and background clutter leads to poor tracking. In this paper, a method of multiple moving object detection and tracking by combining background subtraction and K-means clustering is proposed. The proposed method can handle objects occlusion, shadows and camera jitter. Background subtraction filters irrelevant information, and K-means clustering is employed to select the moving object from the remaining information, and it is capable of handling merging and splitting of moving objects using spatial information. Experimental results show that the proposed method is robust when compared to other techniques.     

An automatic algorithm for semantic object generation and temporal tracking

Automatic semantic video object extraction is an important step for providing content-based video coding, indexing and retrieval. However, it is very difficult to design a generic semantic video object extraction technique, which can provide variant semantic video objects by using the same function. Since the presence and absence of persons in an image sequence provide important clues about video content, automatic face detection and human being generation are very attractive for content-based video database applications. For this reason, we propose a novel face detection and semantic human object generation algorithm. The homogeneous image regions with accurate boundaries are first obtained by integrating the results of color edge detection and region growing procedures. The human faces are detected from these homogeneous image regions by using skin color segmentation and facial filters. These detected faces are then used as object seed for semantic human object generation. The correspondences of the detected faces and semantic human objects along time axis are further exploited by a contour-based temporal tracking procedure.     

Long transition analysis for digital video sequences

Video indexing is a technique used to extract objects within a video sequence and index them so that they can be used for future retrieval. In many video sequences, special effects such as fade and wipe are incorporated, and in some cases, it is desirable to be able to include objects within such effects for indexing. In this paper, we introduce an automatic process that determines the type of transition and extracts information from it, so that this information can be used in object extraction. Such a process consists of four stages: shot boundary refinement, shot type determination, frame reconstruction for soft transitions, and shot classification for hard transitions. In this paper, we will give the implementation, timing, and performance analysis for each stage. Long transition analysis bridges the gap between shot boundary detection and object tracking and smoothes the process of automatic video indexing for video databases.     

结合多种语义信息的半自动视频对象分割

本文提出了一种半自动视频对象分割方法.该方法结合了多种视频语义信息,来提高分割的效率和分割方法的通用性.在视频的初始帧中用半自动绘制的多边形初始化目标对象区域;然后在后续帧中自动跟踪对象区域,对于一般性对象采用基于后向块匹配的像素对应方法进行跟踪;针对平移运动的刚体对象和变化很缓慢的对象本文提出了专门的跟踪方法;最后利用一种基于同等组的模板修正方法来修正每一帧的分割模板.利用本文的方法获得了精确稳定的实验结果.     

基于HSV空间阴影去除方法研究与应用

在智能视频监控系统中,运动阴影如果被误判为运动目标,将会影响到场景中运动目标的准确提取、跟踪和预测。针对这一问题,设计了一种基于HSV颜色空间的阴影去除方法。方法首先将背景差法和三帧差分法相结合,用于提取运动目标,再将提取的含有阴影的运动目标区域映射到其HSV色彩空间,通过与背景和相邻帧的亮度、饱和度比较,实现对阴影区域的检测和去除,处理过程中无需提前确定特征判别参数。将所设计的方法在标准高速公路视频数据库中进行测试并应用于实时的视频监控系统,验证结果表明该方法能更加有效的消除阴影,从而准确的检测出运动目标,同时方法对光线变化具有一定的鲁棒性。     

Fast perceptual region tracking with coding-depth sensitive access for stream transcoding

Object-based bit allocation can result in significant improvement in the perceptual quality of extremely compressed video. However, real-time video object detection in large format high fidelity video is computationally daunting. Most algorithms begin with extensive use of classical bit analysis, and thus remain computationally heavy. Based on some recent results in human visual perception, in this paper, we present an experimental visual region tracking algorithm particularly designed for perceptual stream transcoding. This exploits the cue order observed in human visual perception to achieve very high computation speed as well as tracking efficiency. Rather than begin processing from pixel level or using any pixel level processing at all, it employs high level motion cue and block shape cue analysis to identify signatures of various relative movements between object of interest, scene background and the camera on the motion vector set, and from there it identifies objects. It then uses predictive filters to track the regions. The result is a fast yet highly effective perceptual region tracking algorithm that can operate in stream rate and track regions of perceptually significant object despite camera movements such as zoom, panning and translation. The technique is not specific to any special class of objects. We have implemented this algorithm in a live ISO-13818/MPEG-2 perceptual transcoder. In this paper, we share the performance of this implementation. This fast object-aware video rate transcoder is particularly suitable for live streaming and can convert a regular stream into a perceptually coded video stream.