Spatio-temporal segmentation of image sequences for object-oriented low bit-rate image coding

This paper describes a new method of segmentation of time-varying image sequences whose goal is object-oriented image coding. The segmentation represents a partition of each frame of the sequence into a set of regions which are homogeneous with regard to motion criterion. The region borders correspond to spatial contours of objects in the frame. Each spatio-temporal region is characterized by its temporal component, which is a model-dependent vector of motion parameters, and a structural component representing the polygonal approximation of the spatial contour of the region.

The construction of spatio-temporal segmentation includes two phases: the initialization step and temporal tracking. The initialization step is based on the spatial segmentation of the first frame of the sequence. Then homogeneous spatial regions are merged through motion estimation in accordance with a motion-based criterion. The temporal tracking consists of the projection of the segmentation along the time axis, and its adjustment. Special attention is paid to the processing of occlusions.

A predictive coding scheme is proposed which is based on the temporal coherence of the segmentation. This scheme is promising for a low bit-rate image compression.

The results for teleconference and TV sequences show the high visual quality of reconstructed only by prediction images. Moreover, the bit-rates for motion coding are very low: from 0.002 to 0.007 bit/pixel for teleconference sequence and from 0.004 to 0.021 bit/pixel for complex TV sequence. A scheme for encoding of the structural information is proposed which requires 0.083 – 0.17 bit per pixel depending on the content of the sequence.     

Segmentation and tracking in echocardiographic sequences: activecontours guided by optical flow estimates

This paper presents a method for segmentation and tracking of cardiac structures in ultrasound image sequences. The developed algorithm is based on the active contour framework. This approach requires initial placement of the contour close to the desired position in the image, usually an object outline. Best contour shape and position are then calculated, assuming that at this configuration a global energy function, associated with a contour, attains its minimum. Active contours can be used for tracking by selecting a solution from a previous frame as an initial position in a present frame. Such an approach, however, fails for large displacements of the object of interest. This paper presents a technique that incorporates the information on pixel velocities (optical flow) into the estimate of initial contour to enable tracking of fast-moving objects. The algorithm was tested on several ultrasound image sequences, each covering one complete cardiac cycle. The contour successfully tracked boundaries of mitral valve leaflets, aortic root and endocardial borders of the left ventricle. The algorithm-generated outlines were compared against manual tracings by expert physicians. The automated method resulted in contours that were within the boundaries of intraobserver variability     

一种基于模糊信息粒化的视频时空显著单元提取方法

提出一种基于模糊信息粒化的视频时空显著单元提取方法,为视频分析及检索等高层应用提供一个有效的内容表示模式.本文首先提出了一种类相关的特征粒化方法,粒化后的模糊粒特征简化了分类关系且在一定程度上解决了感知主观性问题,因而通过简单的分类器可以有效地提取空域中具有高视觉感知显著度的区域(简称为显著区域);其次,通过对显著区域的时域一致性分析提取视频序列中时域连续的显著区域集合,定义为时空显著单元.提取的时空显著单元能作为一种较为通用的语义级内容表示模式.实验结果分别从时域和空域两个方面验证了本文方法的有效性.     

基于奇异值分解的特征跟踪方法

在传统的基于模板匹配的跟踪方法中，均是给定一个模板，然后从图像中各个位置取出一个个与模板大小一致的区域进行相似性度量，找出与模板距离最小的一个区域作为当前模板，以便进行下一步的匹配跟踪工作。在景象匹配和相关跟踪过程中，由于所面临的大多数是变化的场景，实时获取的图像与预存模板之间存在比较大的差异，传统相关匹配方法的应用就会受到限制；而且在跟踪过程中，随时更新模板会造成跟踪性能对扰动过分敏感，从而产生漂移。首先拍摄目标不同角度的图像（尽可能包含目标可能出现的所有情况），构成目标图像训练集合，抽取出特征矩阵，对它进行奇异值分解，构成一个关于目标的多维空间。然后再用匹配方法在全局范围搜索，找出目标的大致位置，并利用收敛方法在确定的大致位置内进行搜索，确定目标的仿射变换系数，从而得到一个目标位置的确切描述。     

Active contours for video object tracking using region, boundary and shape information

In this paper, we propose a robust model for tracking in video sequences with non-static backgrounds. The object boundaries are tracked on each frame of the sequence by minimizing an energy functional that combines region, boundary and shape information. The region information is formulated by minimizing the symmetric Kullback–Leibler (KL) distance between the local and global statistics of the objects versus the background. The boundary information is formulated using a color and texture edge map of the video frames. The shape information is calculated adaptively to the dynamic of the moving objects and permits tracking that is robust to background distractions and occlusions. Minimization of the energy functional is implemented using the level set method. We show the effectiveness of the approach for object tracking in color, infrared (IR), and fused color-infrared sequences.     

Integrating Spatial and Temporal Relationship Operators into SQL3 for Historical Data Management

A spatial object changes its states over time. However, existing spatial and temporal database systems cannot fully manage time-varying data with both spatial and nonspatial attributes. To overcome this limitation, we present a framework for spatio-temporal databases that can manage all time-varying historical information and integrate spatial and temporal relationship operators into the select statement in SQL3. For the purpose of our framework, we define three referencing macros and a history aggregate operator and classify the existing spatial and temporal relationship operators into three groups: exclusively spatial relationship operators, exclusively temporal relationship operators, and spatio-temporal common relationship operators. Finally, we believe the integration of spatial and temporal relationship operators into SQL3 will provide a useful framework for the history management of time-varying spatial objects in a uniform manner.     

基于时空二维目标描述的红外小目标跟踪

根据红外弱小目标的时空域特性,提出了一种基于时空二维直方图均值移动的红外弱小目标跟踪方法。本文构建了空域分量和时域分量来表示红外弱小目标,形成时空二维图像。当前帧图像包含目标的空域分布信息,因此将当前帧图像作为空域分量;差分图像包含目标的运动特性,反映了目标的时域特性,因此将前后两帧差分图像作为时域分量。利用时空二维目标表示方法,采用均值移动算法进行目标跟踪,同时根据Bhattacharyya系数来更新目标模板。采用实际拍摄的小目标视频来检测本算法,仿真结果显示本算法能稳健的跟踪红外弱小目标,均值移动算法的跟踪精度得到了较大的提高。     

引入光流法的活动轮廓模型

本文在活动轮廓模型的基础上，引入了“图象统计势能”和光流法，提出了一种新的图象分割方法，该方法改进了活动轮廓模型的一些不足，能准确地检测出物体轮廓边界，且分割结果对初始位置不敏感，同时提高了对运动速度较快的物体轮廓检测的正确性，实验结果证实了该方法的有效性。     

X-ray fluoroscopy spatio-temporal filtering with object detection

One potential way to reduce patient and staff X-ray fluoroscopy dose is to reduce the quantum exposure to the detector and compensate the additional noise with digital filtering. A new filtering method, spatio-temporal filtering with object detection, is described that reduces noise while minimizing motion and spatial blur. As compared to some conventional motion-detection filtering schemes, this object-detection method incorporates additional a priori knowledge of image content; i.e. much of the motion occurs in isolated long thin objects (catheters, guide wires, etc.). We create object-likelihood images and use these to control spatial and recursive temporal filtering such as to reduce blurring the objects of interest. We use automatically computed receiver operating characteristic (ROC) curves to optimize the object-likelihood enhancement method and determine that oriented matched filter kernels with 4 orientations are appropriate. The matched filter kernels are simple projected cylinders. We demonstrate the method on several representative X-ray fluoroscopy sequences to which noise is added to simulate very low dose acquisitions. With processing, we find that noise variance is significantly reduced with slightly less noise reduction near moving objects. We estimate an effective exposure reduction greater than 80%     

一种抗几何攻击的数字水印算法

文中提出了一种基于对象的抗几何攻击数字水印的方法,将水印嵌入到图像中的对象部分而非整幅图像,并利用自参考函数实现抵抗RST变换等各种几何攻击。首先对原始图像进行边缘检测以提取出对象,然后用边界跟踪算法选择嵌入区。然后把要嵌入的水印信号作预处理及周期化,并自适应的嵌入到选择好的嵌入区间中。水印提取时估计出受攻击之后的水印,并用ACF函数判断图像所经历的几何变换,以得到正确的水印信息。实验表明对几何攻击及JPEG压缩都有较好的鲁棒性。     

运动目标的自动分割与跟踪

该文提出了一种对视频序列中的运动目标进行自动分割的算法。该算法分析图像在L U V空间中的局部变化,同时使用运动信息来把目标从背景中分离出来。首先根据图像的局部变化,使用基于图论的方法把图像分割成不同的区域。然后,通过度量合成的全局运动与估计的局部运动之间的偏差来检测出运动的区域,运动的区域通过基于区域的仿射运动模型来跟踪到下一帧。为了提高提取的目标的时空连续性,使用Hausdorff跟踪器对目标的二值模型进行跟踪。对一些典型的MPEG-4测试序列所进行的评估显示了该算法的优良性能。     

An automatic segmentation method for moving objects based on the spatial-temporal information of video

The new MPEG-4 video coding standard enables content-based functions. In order to support the new standard, frames should be decomposed into Video Object Planes （VOP）, each VOP representing a moving object. This paper proposes an image segmentation method to separate moving objects from image sequences. The proposed method utilizes the spatial-temporal information. Spatial segmentation is applied to divide each image into connected areas and to find pre~：ise object boundaries of moving objects. To locate moving objects in image sequences, two consecutive image frames in the temporal direction are examined and a hypothesis testing is performed with Neyman-Pearson criterion. Spatial segmentation produces a spatial segmentation mask, and temporal segmentation yields a change detection mask that indicates moving objects and the background. Then spatial-temporal merging can be used to get the final results. This method has been tested on several images. Experimental results show that this segmentation method is efficient.     

Motion analysis and segmentation through spatio-temporal slices processing

This paper presents new approaches in characterizing and segmenting the content of video. These approaches are developed based upon the pattern analysis of spatio-temporal slices. While traditional approaches to motion sequence analysis tend to formulate computational methodologies on two or three adjacent frames, spatio-temporal slices provide rich visual patterns along a larger temporal scale. We first describe a motion computation method based on a structure tensor formulation. This method encodes visual patterns of spatio-temporal slices in a tensor histogram, on one hand, characterizing the temporal changes of motion over time, on the other hand, describing the motion trajectories of different moving objects. By analyzing the tensor histogram of an image sequence, we can temporally segment the sequence into several motion coherent subunits, in addition, spatially segment the sequence into various motion layers. The temporal segmentation of image sequences expeditiously facilitates the motion annotation and content representation of a video, while the spatial decomposition of image sequences leads to a prominent way of reconstructing background panoramic images and computing foreground objects.     

基于可变形模型的目标跟踪算法

近年来目标跟踪技术的研究已经有了很大的进展,但目标的遮挡和形变仍然是目标跟踪算法面临的重大挑战。针对这些问题提出了一种基于可变形模型的目标跟踪算法。首先,利用可变形模型对跟踪目标进行表达,该模型将目标分为若干子块,目标的特征由局部子块特征和全局特征共同构成。将目标的特征和子块之间的空间关系结合起来,给出了对目标的一个统一的相似度度量函数。然后,在线训练一个结构化输出支持向量机作为分类器,该分类器的输出是可变形模型中目标的结构化描述。利用该分类器可以在视频及图像序列中准确地检测到目标,完成跟踪。通过实验比较,该算法的跟踪性能优于其他主流跟踪算法,尤其在目标发生遮挡和形变的时候仍能准确跟踪。     

A semi-automatic seeded region growing algorithm for video object localization and tracking

This paper describes a semi-automatic method for moving object segmentation and tracking. This method is suitable when a few objects have to be tracked, while the camera moves and fixates on them. The user delineates approximately the initial locations in a selected frame and specifies the depth ordering of the objects to be tracked. First, motion-based segmentation is obtained through an initial application of a region growing algorithm. The partition map is sequentially tracked from frame to frame using motion compensation and location prediction. The segmentation map is obtained by the region growing algorithm. Translational motion is assumed for the moving objects, and local intensity or color average may be used as additional features. A post-processing procedure regularizes the object boundaries over time.     

Improved mean shift target tracking based on self-organizing maps

Thanks to its simplicity and real-time processing possibility, mean shift has been widely used for video tracking. However, it often fails when the background is similar to the intended object or when the object is partially or completely occluded. To address these two problems, in this paper we propose a novel algorithm based on mean shift by exploring simultaneously the temporal and spatial information of the tracked object. A cascade classification method based on nearest neighbor and self-organizing maps is employed as a confirmation step to eliminate spurious objects through the structure information of the object. The forward and backward tracking results are further combined to improve the localization accuracy and tolerate at the same time scale variation. Experiments have shown clearly the superior performance of the proposed system in terms of accuracy, stability and robustness.     

Tracking of migrating cells under phase-contrast video microscopy with combined mean-shift processes

In this paper, we propose a combination of mean-shift-based tracking processes to establish migrating cell trajectories through in vitro phase-contrast video microscopy. After a recapitulation on how the mean-shift algorithm permits efficient object tracking we describe the proposed extension and apply it to the in vitro cell tracking problem. In this application, the cells are unmarked (i.e., no fluorescent probe is used) and are observed under classical phase-contrast microscopy. By introducing an adaptive combination of several kernels, we address several problems such as variations in size and shape of the tracked objects (e.g., those occurring in the case of cell membrane extensions), the presence of incomplete (or noncontrasted) object boundaries, partially overlapping objects and object splitting (in the case of cell divisions or mitoses). Comparing the tracking results automatically obtained to those generated manually by a human expert, we tested the stability of the different algorithm parameters and their effects on the tracking results. We also show how the method is resistant to a decrease in image resolution and accidental defocusing (which may occur during long experiments, e.g., dozens of hours). Finally, we applied our methodology on cancer cell tracking and showed that cytochalasin-D significantly inhibits cell motility.     

建立灰度图像的背景模型方法研究

建立背景模型是在灰度图像序列中进行目标跟踪常用的方法。建立背景模型的目的是对背景象素点的分布及其变化情况作一个描述，以便将前景目标从背景中分离出来。本文对几种建立背景模型的方法做了比较，并通过实验分析了各自的优缺点。     

Region-level motion-based background modeling and subtraction using MRFs.

This paper presents a new approach to automatic segmentation of foreground objects from an image sequence by integrating techniques of background subtraction and motion-based foreground segmentation. First, a region-based motion segmentation algorithm is proposed to obtain a set of motion-coherence regions and the correspondence among regions at different time instants. Next, we formulate the classification problem as a graph labeling over a region adjacency graph based on Markov random fields (MRFs) statistical framework. A background model representing the background scene is built and then is used to model a likelihood energy. Besides the background model, a temporal coherence is also maintained by modeling it as the prior energy. On the other hand, color distributions of two neighboring regions are taken into consideration to impose spatial coherence. Then, the a priori energy of MRFs takes both spatial and temporal coherence into account to maintain the continuity of our segmentation. Finally, a labeling is obtained by maximizing the a posteriori energy of the MRFs. Under such formulation, we integrate two different kinds of techniques in an elegant way to make the foreground detection more accurate. Experimental results for several video sequences are provided to demonstrate the effectiveness of the proposed approach.