An automatic segmentation method for moving objects based on the spatial-temporal information of video

The new MPEG-4 video coding standard enables content-based functions. In order to support the new standard, frames should be decomposed into Video Object Planes （VOP）, each VOP representing a moving object. This paper proposes an image segmentation method to separate moving objects from image sequences. The proposed method utilizes the spatial-temporal information. Spatial segmentation is applied to divide each image into connected areas and to find pre~：ise object boundaries of moving objects. To locate moving objects in image sequences, two consecutive image frames in the temporal direction are examined and a hypothesis testing is performed with Neyman-Pearson criterion. Spatial segmentation produces a spatial segmentation mask, and temporal segmentation yields a change detection mask that indicates moving objects and the background. Then spatial-temporal merging can be used to get the final results. This method has been tested on several images. Experimental results show that this segmentation method is efficient.     

基于非局部随机游走和运动补偿的2D转3D优化方法

D转3D技术可以从2D资源中获取深度信息,以满足3D显示对3D内容的需求。针 对2D转 3D深度估计中的深度优化问题,提出一种基于非局部随机游走(NRW)和运动补偿的 深度优化算 法。本文方法在采用NRW和移动双边滤波(SBF)获得关键帧和非关键帧深度图的基础上,为 了锐化非关 键帧深度序列对象边界,结合纹理信息利用NRW算法优化深度图,同时又考虑相邻帧间的时 域信息,采 用运动补偿的方法对非关键帧深度序列进行优化,获得高质量的深度视频序列。实验结果表 明,本文方法可以得到对象边界更加准确的深度视频估计结果。     

软分割约束边缘保持插值的半自动2D转3D

半自动2D转3D将用户标注的稀疏深度转换成稠密深度,是解决3D片源不足的主要手段之一.针对现有方法利用硬分割增强深度边缘引入误差的问题,提出像素点与超像素深度一致性约束的边缘保持插值方法.首先,建立像素点深度和超像素深度传播的能量模型,通过像素点与所属超像素间深度差异的约束项将二者关联起来;其次,利用矩阵表示形式将两个能量模型的最优化转换成一个稀疏线性方程组的求解问题.通过超像素提供的约束项,可避免深度传播穿过低对比度边缘区域,从而能保持对象边缘.实验结果表明,本文方法对象边缘处深度恢复的准确性优于融合图割的随机游走方法,PSNR改善了1.5dB以上.     

基于模糊聚类的运动对象分割算法研究

为了从视频序列中分割出完整的、一致的运动视频对象,该文使用基于模糊聚类的分割算法获得组成对象边界的像素,从而提取对象。该算法首先使用了当前帧以及之前一些帧的图像信息计算其在小波域中不同子带的运动特征,并根据这些运动特征构造了低分辨率图像的运动特征矢量集;然后,使用模糊C-均值聚类算法分离出图像中发生显著变化的像素,以此代替帧间差图像,并利用传统的变化检测方法获得对象变化检测模型,从而提取对象;同时,使用相继两帧之间的平均绝对差值大小确定计算当前帧运动特征所需帧的数量,保证提取视频对象的精确性。实验结果证明该方法对于分割各种图像序列中的视频对象是有效的。     

Semi-automatic 2D-to-3D video conversion based on background sprite generation

This paper presents a technique for semi-automatic 2D-to-3D stereo video conversion, which is known to provide user intervention in assigning foreground/background depths for key frames and then get depth maps for non-key frames via automatic depth propagation. Our algorithm treats foreground and background separately. For foregrounds, kernel pixels are identified and then used as the seeds for graph-cut segmentation for each non-key frame independently, resulting in results not limited by objects’ motion activity. For backgrounds, all video frames, after foregrounds being removed, are integrated into a common background sprite model (BSM) based on a relay-frame-based image registration algorithm. Users can then draw background depths for BSM in an integrated manner, thus reducing human efforts significantly. Experimental results show that our method is capable of retaining more faithful foreground depth boundaries (by 1.6–2.7 dB) and smoother background depths than prior works. This advantage is helpful for 3D display and 3D perception.     

Depth map misalignment correction and dilation for DIBR view synthesis

The quality of the synthesized views by Depth Image Based Rendering (DIBR) highly depends on the accuracy of the depth map, especially the alignment of object boundaries of texture image. In practice, the misalignment of sharp depth map edges is the major cause of the annoying artifacts at the disoccluded regions of the synthesized views. Conventional smooth filter approach blurs the depth map to reduce the disoccluded regions. The drawbacks are the degradation of 3D perception of the reconstructed 3D videos and the destruction of the texture in background regions. Conventional edge preserving filter utilizes the color image information in order to align the depth edges with color edges. Unfortunately, the characteristics of color edges and depth edges are very different which causes annoying boundaries artifacts in the synthesized virtual views. Recent solution of reliability-based approach uses reliable warping information from other views to fill the holes. However, it is not suitable for the view synthesis in video-plus-depth based DIBR applications. In this paper, a new depth map preprocessing approach is proposed. It utilizes Watershed color segmentation method to correct the depth map misalignment and then the depth map object boundaries are extended to cover the transitional edge regions of color image. This approach can handle the sharp depth map edges lying inside or outside the object boundaries in 2D sense. The quality of the disoccluded regions of the synthesized views can be significantly improved and unknown depth values can also be estimated. Experimental results show that the proposed method achieves superior performance for view synthesis by DIBR especially for generating large baseline virtual views.     

基于Snake活动轮廓模型的视频跟踪分割方法

基于Snake活动轮廓模型,采用时空融合的方式,根据短时间内相邻帧的运动趋势差异相似的前提,首先将视频序列分成若干个小段,每段有k帧视频,选取段内的前两帧为关键帧,通过运动检测的方式自动得到这两帧中运动对象的大致区域;然后进行帧内Snake演变,搜索精确轮廓;最后以关键帧间运动对象形心的运动矢量预测勾勒后续帧的初始轮廓,进行帧内Snake精确轮廓定位,从而实现所有帧的视频对象分割。相比于传统方法,本文方法克服了手动绘制初始轮廓的缺点,在空域对Snake贪婪方法进行了改进而且精确度高,速度快。实验表明,本文方法成功地实现了前后帧图像之间运动对象的对应匹配关系,并通过改进后的Snake贪婪方法得到了精确的分割结果。     

融合颜色信息与深度信息的运动目标检测方法

基于颜色信息的运动目标检测易受光照、阴影等影响,基于深度信息的运动目标检测存在目标边缘噪声大,无法检测距离背景较近的目标等问题。针对上述问题,该文利用CCD相机获取的颜色信息及TOF相机获取的深度信息分别为每个像素建立颜色与深度信息的分类器,根据像素点的深度特征及前一帧的检测结果,自适应地为每个分类器的输出分配不同的权值,实现运动目标的检测。该文采集多组视频序列进行实验,实验结果表明该方法能有效解决单独利用颜色或深度信息进行运动目标检测时出现的问题。     

基于层次聚类的多颜色空间融合分割方法

针对自然图像分割中,由于单一的颜色空间难以表示复杂多变的场景信息以及目标与背景的低对比度等产生的过分割和误分割问题,提出了选择RGB、XYZ和LUV这3个颜色空间的增强图像进行基于层次聚类的融合分割的新方法.对Berkeley分割图像库中的多幅图像进行了多组分割实验,并与Mean-shift等多种经典分割方法进行了定性...     

SRI-Net: Similarity retrieval-based inference network for light field salient object detection

The cutting-edge RGB saliency models are prone to fail for some complex scenes, while RGB-D saliency models are often affected by inaccurate depth maps. Fortunately, light field images can provide a sufficient spatial layout depiction of 3D scenes. Therefore, this paper focuses on salient object detection of light field images, where a Similarity Retrieval-based Inference Network (SRI-Net) is proposed. Due to various focus points, not all focal slices extracted from light field images are beneficial for salient object detection, thus, the key point of our model lies in that we attempt to select the most valuable focal slice, which can contribute more complementary information for the RGB image. Specifically, firstly, we design a focal slice retrieval module (FSRM) to choose an appropriate focal slice by measuring the foreground similarity between the focal slice and RGB image. Secondly, in order to combine the original RGB image and the selected focal slice, we design a U-shaped saliency inference module (SIM), where the two-stream encoder is used to extract multi-level features, and the decoder is employed to aggregate multi-level deep features. Extensive experiments are conducted on two widely used light field datasets, and the results firmly demonstrate the superiority and effectiveness of the proposed SRI-Net.     

Combining static and dynamic features using neural networks and edge fusion for video object extraction

Semantic object representation is an important step for digital multimedia applications such as object-based coding, content-based access and manipulations. The authors propose an image sequence segmentation scheme which provides region information for the semantic object representation of those applications. The objective is to develop a hardware-friendly segmentation algorithm by combining static and dynamic features simultaneously in one scheme. In the initial stage, a multiple feature space is transformed to one-dimensional label space by using self-organising feature map (SOFM) neural networks. The next stage is an edge fusion process in which edge information is incorporated into the neural network outputs to generate more precisely located boundaries of segmentation. The proposed algorithm differs from existing methods as follows: it can segment textured images with low-dimensional features; leads to more meaningful segmentation region boundaries; and is easier to map into hardware than existing methods. Experimental results are compared with an existing segmentation method using evaluation metrics to clarify the advantages of the proposed algorithm objectively.     

Temporal stabilization of Video Object Segmentation for 3D-TV applications

Our aim is to insert depth information into an existing 2D video sequence to provide content for 3D-TV applications, which we try to achieve through segmentation of the objects in the given 2D video sequence. To this effect, we present a method for temporal stabilization of video object segmentation algorithms for 3D-TV applications. First, two quantitative measures to evaluate temporal stability without ground-truth are discussed. Then, a pseudo-3D curve evolution method, which spatio-temporally stabilizes the estimated segmentation of a video object is introduced. Temporal stability is achieved by re-distributing existing object segmentation errors such that they will be less disturbing when the scene is rendered and viewed in 3D. Our starting point is the hypothesis that if making segmentation errors is inevitable, these errors should be made in a temporally consistent way for 3D-TV applications. This hypothesis is supported by the experiments, which show that there is significant improvement in segmentation quality both in terms of the objective quantitative measures and in terms of the viewing comfort in subjective perceptual tests. Therefore, it is possible to increase the perceptual object segmentation quality without increasing the actual segmentation accuracy.     

Space-time segmentation using level set active contours applied to myocardial gated SPECT

This paper presents a new variational method for the segmentation of a moving object against a still background, over a sequence of [two-dimensional or three-dimensional (3-D)] image frames. The method is illustrated in application to myocardial gated single photon emission computed tomography (SPECT) data, and incorporates a level set framework to handle topological changes while providing closed boundaries. The key innovation is the introduction of a geometrical constraint into the derivation of the Euler-Lagrange equations, such that the segmentation of each individual frame can be interpreted as a closed boundary of an object (an isolevel of a set of hyper-surfaces) while integrating information over the entire sequence. This results in the definition of an evolution velocity normal to the object boundary. Applying this method to 3-D myocardial gated SPECT sequences, the left ventricle endocardial and epicardial limits can be computed in each frame. This space-time segmentation method was tested on simulated and clinical 3-D myocardial gated SPECT sequences and the corresponding ejection fractions were computed.     

An Unsupervised Classification for Fully Polarimetric SAR Data Using SPAN/H/ IHSL Transform and The FCM Algorithm

 In this paper, the IHSL transform and the Fuzzy C-Means (FCM) segmentation algorithm are combined together to perform the unsupervised classification for fully polarimetric Synthetic Ap-erture Rader (SAR) data. We apply the IHSL colour transform to space to obtain a new space (RGB colour space) which has a uniform distinguishability among inner parameters and contains the whole polarimetric information in Then the FCM algorithm is applied to this RGB space to finish the classification procedure. The main advantages of this method are that the parameters in the color space have similar interclass distinguishability, thus it can achieve a high performance in the pixel based segmentation algorithm, and since we can treat the parameters in the same way, the segmentation procedure can be simplified. The experiments show that it can provide an improved classification result compared with the method which uses the space di-rectly during the segmentation procedure.     

Automatic Segmentation of Moving Objects in Video Sequences for Indoor and Outdoor Applications

1　IntroductionAutomaticsegmentationofmovingobjectsfromvideosequencesisadifficultandchallengingproblemincomputervisionsystems.Ithasmanyapplicationssuchasvideosurveillance,trafficmonitoring ,peopletrackingandvideocommunication[1～4] .Italsoplaysanimportantroleinsupportingcontent basedimagecoding,especiallyaftertheemergenceofthevideocodingstandardMPEG 4[5～ 1 4 ] .Therearealotofresearchworksonmovingob jectssegmentationandextraction .Thesealgorithmscanberoughlyclassifiedintotwocategories:inter …     

Unsupervised video object segmentation with distractor-aware online adaptation

Unsupervised video object segmentation is a crucial application in video analysis when there is no prior information about the objects. It becomes tremendously challenging when multiple objects occur and interact in a video clip. In this paper, a novel unsupervised video object segmentation approach via distractor-aware online adaptation (DOA) is proposed. DOA models spatiotemporal consistency in video sequences by capturing background dependencies from adjacent frames. Instance proposals are generated by the instance segmentation network for each frame and they are grouped by motion information as positives or hard negatives. To adopt high-quality hard negatives, the block matching algorithm is then applied to preceding frames to track the associated hard negatives. General negatives are also introduced when there are no hard negatives in the sequence. The experimental results demonstrate these two kinds of negatives are complementary. Finally, we conduct DOA using positive, negative, and hard negative masks to update the foreground and background segmentation. The proposed approach achieves state-of-the-art results on two benchmark datasets, the DAVIS 2016 and the Freiburg-Berkeley motion segmentation (FBMS)-59.     

基于目标匹配的视频对象跟踪算法

视频对象分割算法的性能好坏将直接影响MPEG-4编码产品的质量。连续两次差分后自适应处理，对差分图像取交集获得运动对象的边界，形态学处理后获得二值分割掩模进而提取运动目标。基于改进的Hausdorff距离度量法对后续帧中视频对象进行跟踪。实验结果证明，该方法能够从背景不变的图像序列中较好的提取出运动对象，具有较强的鲁棒性。     

AN UNSUPERVISED CLASSIFICATION FOR FULLY POLARIMETRIC SAR DATA USING SPAN/H/α IHSL TRANSFORM AND THE FCM ALGORITHM

In this paper, the IHSL transform and the Fuzzy C-Means (FCM) segmentation algorithm are combined together to perform the unsupervised classification for fully polarimetric Synthetic Ap-erture Rader (SAR) data. We apply the IHSL colour transform to H/α/SPANspace to obtain a new space (RGB colour space) which has a uniform distinguishability among inner parameters and contains the whole polarimetric information in H/α/SPAN.Then the FCM algorithm is applied to this RGB space to finish the classification procedure. The main advantages of this method are that the parameters in the color space have similar interclass distinguishability, thus it can achieve a high performance in the pixel based segmentation algorithm, and since we can treat the parameters in the same way, the segmentation procedure can be simplified. The experiments show that it can provide an improved classification result compared with the method which uses the H/α/SPANspace di-rectly during the segmentation procedure.     

A novel moving object segmentation framework utilizing camera motion recognition for H.264 compressed videos

This paper presents a novel coarse to fine moving object segmentation framework for H.264/AVC compressed videos. The proposed framework integrates the global motion estimation and global motion compensation steps in the segmentation pipeline unlike previous techniques which did not consider such an integration. The integration is based on testing for presence of global motion by classifying the interframe motion vectors into moving camera class and still camera class. The decision boundary separating these two classes is learnt from the training video data. The integration automates the moving object segmentation to be applicable for static, moving and combination of static/moving camera cases which to the best of our knowledge has not been carried out earlier. Further, a novel coarse segmentation technique is proposed by decomposing the inter-frame motion vectors into wavelet sub-bands and utilizing logical operations on LH, HL and HH sub-band wavelet coefficients. The premise is based on the fact that since the LH, HL and HH sub-bands contain the detail information pertaining to horizontal, vertical and diagonal moving blocks respectively, they can be exploited to identify the coarse moving boundaries. The coarse segmentation is fast in comparison to state-of-the-art coarse segmentation methods as demonstrated by our experiments. Finally, these coarse boundaries are modeled in an energy minimization framework and shown that by minimizing the energy using graph cut optimization the segmentation is refined to obtain the fine segmentation. The proposed framework is tested on a number of standard video sequences encoded with H.264/AVC JM encoder and comparison is carried out with state-of-the-art compressed domain moving object segmentation methods as well as with an existing state-of-the-art pixel domain method to establish and validate the proposed moving object segmentation framework.     

基于最大隶属度与SVM的彩色图像分割新算法研究

图像分割是图像处理的主要问题,也是图像处理的难题之一,由于彩色图像提供了比灰度图像更多的信息,因此彩色图像处理,尤其是彩色图像分割受到了越来越多的重视。本论文提出一种基于HSI模型和模糊最大隶属度原则与SVM的彩色图像分割方法,从彩色图像的彩色直方图中获取目标色RGB模型和背景色RGB模型,再将RGB模型转化为HSI模型建立一组色彩模糊集,对HSI模型进行SVM训练识别再计算该图像中的色彩在各模糊集中的隶属度,最后确定色彩的归属,再将HSI模型转化为RGB模型。实现了快速有效的彩色图像分割方法的研究。