Integrating tracking with fine object segmentation

We present a novel method for on-line, joint object tracking and segmentation in a monocular video captured by a possibly moving camera. Our goal is to integrate tracking and fine segmentation of a single, previously unseen, potentially non-rigid object of unconstrained appearance, given its segmentation in the first frame of an image sequence as the only prior information. To this end, we tightly couple an existing kernel-based object tracking method with Random Walker-based image segmentation. Bayesian inference mediates between tracking and segmentation, enabling effective data fusion of pixel-wise spatial and color visual cues. The fine segmentation of an object at a certain frame provides tracking with reliable initialization for the next frame, closing the loop between the two building blocks of the proposed framework. The effectiveness of the proposed methodology is evaluated experimentally by comparing it to a large collection of state of the art tracking and video-based object segmentation methods on the basis of a data set consisting of several challenging image sequences for which ground truth data is available.     

Object tracking in image sequences using point features

This paper presents an object tracking technique based on the Bayesian multiple hypothesis tracking (MHT) approach. Two algorithms, both based on the MHT technique are combined to generate an object tracker. The first MHT algorithm is employed for contour segmentation. The segmentation of contours is based on an edge map. The segmented contours are then merged to form recognisable objects. The second MHT algorithm is used in the temporal tracking of a selected object from the initial frame. An object is represented by key feature points that are extracted from it. The key points (mostly corner points) are detected using information obtained from the edge map. These key points are then tracked through the sequence. To confirm the correctness of the tracked key points, the location of the key points on the trajectory are verified against the segmented object identified in each frame. If an acceptable number of key-points lie on or near the contour of the object in a particular frame (n-th frame), we conclude that the selected object has been tracked (identified) successfully in frame n.     

Tracking deforming objects using particle filtering for geometric active contours

Tracking deforming objects involves estimating the global motion of the object and its local deformations as a function of time. Tracking algorithms using Kalman filters or particle filters have been proposed for finite dimensional representations of shape, but these are dependent on the chosen parametrization and cannot handle changes in curve topology. Geometric active contours provide a framework which is parametrization independent and allow for changes in topology, in the present work, we formulate a particle filtering algorithm in the geometric active contour framework that can be used for tracking moving and deforming objects. To the best of our knowledge, this is the first attempt to implement an approximate particle filtering algorithm for tracking on a (theoretically) infinite dimensional state space.     

Object detection using pulse coupled neural networks

Describes an object detection system based on pulse coupled neural networks. The system is designed and implemented to illustrate the power, flexibility and potential the pulse coupled neural networks have in real-time image processing. In the preprocessing stage, a pulse coupled neural network suppresses noise by smoothing the input image. In the segmentation stage, a second pulse coupled neural-network iteratively segments the input image. During each iteration, with the help of a control module, the segmentation network deletes regions that do not satisfy the retention criteria from further processing and produces an improved segmentation of the retained image. In the final stage each group of connected regions that satisfies the detection criteria is identified as an instance of the object of interest.     

Computing occluding and transparent motions

Computing the motions of several moving objects in image sequences involves simultaneous motion analysis and segmentation. This task can become complicated when image motion changes significantly between frames, as with camera vibrations. Such vibrations make tracking in longer sequences harder, as temporal motion constancy cannot be assumed. The problem becomes even more difficult in the case of transparent motions.A method is presented for detecting and tracking occluding and transparent moving objects, which uses temporal integration without assuming motion constancy. Each new frame in the sequence is compared to a dynamic internal representation image of the tracked object. The internal representation image is constructed by temporally integrating frames after registration based on the motion computation. The temporal integration maintains sharpness of the tracked object, while blurring objects that have other motions. Comparing new frames to the internal representation image causes the motion analysis algorithm to continue tracking the same object in subsequent frames, and to improve the segmentation.     

Efficient planar object tracking and parameter estimation usingcompactly represented cubic B-spline curves

In this paper, we consider the problem of matching 2D planar object curves from a database, and tracking moving object curves through an image sequence. The first part of the paper describes a curve data compression method using B-spline curve approximation. We present a new constrained active B-spline curve model based on the minimum mean square error (MMSE) criterion, and an iterative algorithm for selecting the “best” segment border points for each B-spline curve. The second part of the paper describes a method for simultaneous object tracking and affine parameter estimation using the approximate curves and profiles. We propose a novel B-spline point assignment algorithm which incorporates the significant corners for interpolating corresponding points on the two curves to be compared. A gradient-based algorithm is presented for simultaneously tracking object curves, and estimating the associated translation, rotation and scaling parameters. The performance of each proposed method is evaluated using still images and image sequences containing simple objects     

On multi-view video segmentation for object-based coding

A novel scheme for multi-view segmentation and tracking is proposed aiming to acquire perceptually consistent results for object-based coding. Firstly, a classic image segmentation technique is employed to perform initial segmentation to divide the whole image into spatially homogeneous regions. Secondly, the motion information is extracted based on frame differences and the disparity information is derived by employing a classic disparity estimation technique. Thirdly, a novel scheme is proposed to perform merging of the initial segmentation results based on both motion and disparity information to remove over-segmented regions and extract perceptually consistent semantic objects. Finally, a contour-based tracking algorithm is proposed to implement accurate and robust object tracking along both temporal and view directions. Experiments are conducted and the results demonstrate that the proposed scheme is effective and, compared with the existing technique, it can acquire more perceptually consistent results.     

用于半自动视频对象提取的自适应网格图像分割

随着MPEG－4标准的发展和基于内容的视频处理研究,视频对象平面（VOP）的有效产生成为一个关键问题。本文提出一种基于区域的自适应网格彩色图像分割方法,可用于获得半自动视频对象跟踪和提取所需的初始VOP。该方法利用CIE L＊a*b＊色彩空间的特征量,对视频序列的第一帧进行三角形网格的分裂与合并。对MPEG－4标准测试序列的分割实验取得了较好的结果。     

Dynamical statistical shape priors for level set-based tracking

In recent years, researchers have proposed introducing statistical shape knowledge into level set-based segmentation methods in order to cope with insufficient low-level information. While these priors were shown to drastically improve the segmentation of familiar objects, so far the focus has been on statistical shape priors which are static in time. Yet, in the context of tracking deformable objects, it is clear that certain silhouettes (such as those of a walking person) may become more or less likely over time. In this paper, we tackle the challenge of learning dynamical statistical models for implicitly represented shapes. We show how these can be integrated as dynamical shape priors in a Bayesian framework for level set-based image sequence segmentation. We assess the effect of such shape priors "with memory" on the tracking of familiar deformable objects in the presence of noise and occlusion. We show comparisons between dynamical and static shape priors, between models of pure deformation and joint models of deformation and transformation, and we quantitatively evaluate the segmentation accuracy as a function of the noise level and of the camera frame rate. Our experiments demonstrate that level set-based segmentation and tracking can be strongly improved by exploiting the temporal correlations among consecutive silhouettes which characterize deforming shapes.     

High-resolution SAR images segmentation using NSCT denoising and QIGA based parameters selection of PCNN model

The high-resolution synthetic aperture radar (SAR) images usually contain inhomogeneous coherent speckle noises. For the high-resolution SAR image segmentation with such noises, the conventional methods based on pulse coupled neural networks (PCNN) have to face heavy parameters with a low efficiency. In order to solve the problems, this paper proposes a novel SAR image segmentation algorithm based on non-subsampling Contourlet transform (NSCT) denoising and quantum immune genetic algorithm (QIGA) improved PCNN models. The proposed method first denoising the SAR images for a pre-processing based on NSCT. Then, by using the QIGA to select parameters for the PCNN models, such models self-adaptively select the suitable parameters for segmentation of SAR images with different scenes. This method decreases the number of parameters in the PCNN models and improves the efficiency of PCNN models. At last, by using the optimal threshold to binary the segmented SAR images, the small objects and large scales from the original SAR images will be segmented. To validate the feasibility and effectiveness of the proposed algorithm, four different comparable experiments are applied to validate the proposed algorithm. Experimental results have shown that NSCT pre-processing has a better performance for coherent speckle noises suppression, and QIGA-PCNN model based on denoised SAR images has an obvious segmentation performance improvement on region consistency and region contrast than state-of-the-arts methods. Besides, the segmentation efficiency is also improved than conventional PCNN model, and the level of time complexity meets the state-of-the-arts methods. Our proposed NSCT+QIGA-PCNN model can be used for small object segmentation and large scale segmentation in high-resolution SAR images. The segmented results will be further used for object classification and recognition, regions of interest extraction, and moving object detection and tracking.

     

基于B-样条Snake模型的人体运动跟踪方法研究

提出了一种基于B-样条曲线Snake模型的新的人体运动跟踪方法.Snake算法是通过最小能量来逼近物体的轮廓.采用改进的B-样条曲线Snake模型,每一帧图像中的目标轮廓用三次样条曲线准确地表示,使Snake模型更加稳定和具有较快的收敛速度.计算相邻帧之间的差分图像,通过利用一种基于统计关系双阈值分割方法,有效地检测出图像中运动人体,初步确定目标在每帧图像中的粗略位置.把从上一帧图像中得到的目标轮廓置于该位置,作为B-样条曲线Snake算法中轮廓提取的初始值,经运算后可得到对人体目标的准确分割与跟踪.     

Coupled grouping and matching for sign and gesture recognition

Matching an image sequence to a model is a core problem in gesture or sign recognition. In this paper, we consider such a matching problem, without requiring a perfect segmentation of the scene. Instead of requiring that low- and mid-level processes produce near-perfect segmentation, we take into account that such processes can only produce uncertain information and use an intermediate grouping module to generate multiple candidates. From the set of low-level image primitives, such as constant color region patches found in each image, a ranked set of salient, overlapping, groups of these primitives are formed, based on low-level cues such as region shape, proximity, or color. These groups corresponds to underlying object parts of interest, such as the hands. The sequence of these frame-wise group hypotheses are then matched to a model by casting it into a minimization problem. We show the coupling of these hypotheses with both non-statistical matching (match to sample-based modeling of signs) and statistical matching (match to HMM models) are possible. Our algorithm not only produces a matching score, but also selects the best group in each image frame, i.e. recognition and final segmentation of the scene are coupled. In addition, there is no need for tracking of features across sequences, which is known to be a hard task. We demonstrate our method using data from sign language recognition and gesture recognition, we compare our results with the ground truth hand groups, and achieved less than 5% performance loss for both two models. We also tested our algorithm on a sports video dataset that has moving background.     

Controlling Motion Blur in Synthetic Long Time Exposures

In a photo, motion blur can be used as an artistic style to convey motion and to direct attention. In panning or tracking shots, a moving object of interest is followed by the camera during a relatively long exposure. The goal is to get a blurred background while keeping the object sharp. Unfortunately, it can be difficult to impossible to precisely follow the object. Often, many attempts or specialized physical setups are needed. This paper presents a novel approach to create such images. For capturing, the user is only required to take a casually recorded hand‐held video that roughly follows the object. Our algorithm then produces a single image which simulates a stabilized long time exposure. This is achieved by first warping all frames such that the object of interest is aligned to a reference frame. Then, optical flow based frame interpolation is used to reduce ghosting artifacts from temporal undersampling. Finally, the frames are averaged to create the result. As our method avoids segmentation and requires little to no user interaction, even challenging sequences can be processed successfully. In addition, artistic control is available in a number of ways. The effect can also be applied to create videos with an exaggerated motion blur. Results are compared with previous methods and ground truth simulations. The effectiveness of our method is demonstrated by applying it to hundreds of datasets. The most interesting results are shown in the paper and in the supplemental material.     

视频中运动目标的实时检测和跟踪

基于视频的自动目标检测和跟踪是计算机视觉中一个重要的研究领域，特别是基于视频的智能车辆监控系统中的运动车辆的检测和跟踪。提出了一种自适应的背景相减法来分割运动物体，为了准确地定位运动车辆的区域，采用差分图像投影和边缘投影相结合的方法来定位车体，同时利用双向加权联合图匹配方法对运动车辆区域进行跟踪，即将对运动车辆区域跟踪问题转化为搜索具有最大权的联合图的问题。该算法不仅能实时地定位和跟踪直道上运动的车辆，同时也能实时地定位和跟踪弯道上运动的车辆，从实验结果看，提出的背景更新算法简单，并且运动车辆区域的定位具有很好的鲁棒性，从统计的检测率和运行时间来看，该算法具有很好的检测效果，同时也能满足基于视频的智能交通监控系统的需要。     

Granulation,rough entropy and spatiotemporal moving object detection

A new spatio-temporal segmentation approach for moving object(s) detection and tracking from a video sequence is described. Spatial segmentation is carried out using rough entropy maximization, where we use the quad-tree decomposition, resulting in unequal image granulation which is closer to natural granulation. A three point estimation based on Beta Distribution is formulated for background estimation during temporal segmentation. Reconstruction and tracking of the object in the target frame is performed after combining the two segmentation outputs using its color and shift information. The algorithm is more robust to noise and gradual illumination change, because their presence is less likely to affect both its spatial and temporal segments inside the search window. The proposed methods for spatial and temporal segmentation are seen to be superior to several related methods. The accuracy of reconstruction has been significantly high.     

基于简化Mumford-Shah模型的水平集红外图像分割算法

由于红外图像大多具有目标模糊,对比度低的特点,传统的分割方法容易受到噪声和边界轮廓的影响而导致分割效果不佳,提出了一种基于简化Mumford-Shah模型的水平集红外图像分割算法.该算法能够通过将初始闭合曲线嵌入水平集函数,利用函数的求解从而达到图像分割的目的.仿真实验结果表明,该分割算法与初始轮廓线位置无关,受边界轮廓线和图像噪声的影响较小,具有较强的鲁棒性,在目标与背景灰度级差别较小的红外图像的分割中取得了较好的效果.     

Cell Migration Analysis After In Vitro Wounding Injury with a Multi-Agent Approach

This paper presents a multiagent system for studying in vitro cell motion. A typical application on the wound closure process is presented to illustrate the possibilities of the system, where different image sequences will be treated. The motion issue involves three aspects: image segmentation, object tracking and motion analysis. The current system version focuses mainly on the image segmentation aspect. A general agent model has been designed, which will be further expanded to include tracking and motion analysis behaviors as well. The agents integrate three basic behaviors: perception, interaction and reproduction. The perception evaluates pixels upon static and motion-based criteria. The interaction behavior allows two agents to merge or to negotiate parts of regions. The negotiation can be seen as a segmentation refinement process done by the agents. Finally, the reproduction behavior defines an exploration strategy of the images. Agents can start other agents around them, or they can duplicate themselves in the next frame. The frames are processed in pipeline, where previous information is used to treat the current frame. One unique agent model exist. Agents are specialized on execution time according to their goals. The results, coming from an existing prototype, show different types of cell behavior during cell migration, based on cell nuclei analysis.     

背景估计与运动目标检测跟踪

基于视频的自动目标检测和跟踪是计算机视觉中一个重要的研究领域,特别是基于视频的智能车辆监控系统中的运动车辆的检测和跟踪。提出一种自适应的背景估计方法来实时获得当前背景图像,从而分割出运动物体。为了准确地定位运动车辆的区域,采用差分图像投影和边缘投影相结合的方法来定位车体,同时利用双向加权联合图匹配方法对运动车辆区域进行跟踪,即将对运动车辆区域跟踪问题转化为搜索具有最大权的联合图的问题。该算法不仅能实时地定位和跟踪直道上运动的车辆,同时也能实时地定位和跟踪弯道上运动的车辆,从实验结果看,提出的背景更新算法简单,并且运动车辆区域的定位具有很好的鲁棒性,从统计的检测率和运行时间来看,该算法具有很好的检测效果,同时也能满足基于视频的智能交通监控系统的需要。     

基层层次光流的半自动时空视频分割技术

在新一代MPEG-4视频编码标准中，为了支持面向对象编码和实现基于内容的应用，视频的半自动分割成为关键技术之一，为此提出了一种基于层次光流的半自动时空视频分割算法。该算法由空域分割和时域分割组成。在空域分割中，提出的基于点的图形用户界面（PBGUI），在用户的协助下，能够精确地定义需要分割的视频对象（VO）。时域分割根据空域分割的结果采用层次光流算法对视频对象进行边界和整体跟踪。实验结果表明，利用该算法，能够精确地分割出视频对象。     

基于层次光流的半自动时空视频分割技术

在新一代 MPEG- 4视频编码标准中 ,为了支持面向对象编码和实现基于内容的应用 ,视频的半自动分割成为关键技术之一 ,为此提出了一种基于层次光流的半自动时空视频分割算法 .该算法由空域分割和时域分割组成 .在空域分割中 ,提出的基于点的图形用户界面 (PBGU I) ,在用户的协助下 ,能够精确地定义需要分割的视频对象 (VO) .时域分割根据空域分割的结果采用层次光流算法对视频对象进行边界和整体跟踪 .实验结果表明 ,利用该算法 ,能够较精确地分割出视频对象 .