Model-based segmentation and tracking of head-and-shoulder video objects for real time multimedia services

A statistical model-based video segmentation algorithm is presented for head-and-shoulder type video. This algorithm uses domain knowledge by abstracting the head-and-shoulder object with a blob-based statistical region model and a shape model. The object segmentation problem is then converted into a model detection and tracking problem. At the system level, a hierarchical structure is designed and spatial and temporal filters are used to improve segmentation quality. This algorithm runs in real time over a QCIF size video, and segments it into background, head and shoulder three video objects on average Pentium PC platforms. Due to its real time feature, this algorithm is appropriate for real time multimedia services such as videophone and web chat. Simulation results are offered to compare MPEG-4 performance with H.263 on segmented video objects with respects to compression efficiency, bit rate adaptation and functionality.     

基于小波变换和ICA的运动目标分割

提出一种时空融合的运动目标分割方法.在时域方面,采用时间轴一维小波变换提取运动对象,然后用独立成分分析法提取独立的运动对象,并基于灰度直方图进一步提取视频对象;在空域方面,提出对轮廓提取后的图像进行分水岭变换的改进方法.与COST211 AM算法比较表明,文中方法能更完整、准确地提取出运动对象.     

Self-organization in vision: stochastic clustering for imagesegmentation, perceptual grouping, and image database organization

We present a stochastic clustering algorithm which uses pairwise similarity of elements and show how it can be used to address various problems in computer vision, including the low-level image segmentation, mid-level perceptual grouping, and high-level image database organization. The clustering problem is viewed as a graph partitioning problem, where nodes represent data elements and the weights of the edges represent pairwise similarities. We generate samples of cuts in this graph, by using Karger's contraction algorithm (1996), and compute an "average" cut which provides the basis for our solution to the clustering problem. The stochastic nature of our method makes it robust against noise, including accidental edges and small spurious clusters. The complexity of our algorithm is very low: O(|E| log² N) for N objects, |E| similarity relations, and a fixed accuracy level. In addition, and without additional computational cost, our algorithm provides a hierarchy of nested partitions. We demonstrate the superiority of our method for image segmentation on a few synthetic and real images, both B&W and color. Our other examples include the concatenation of edges in a cluttered scene (perceptual grouping) and the organization of an image database for the purpose of multiview 3D object recognition     

An integrated approach for scene understanding based on Markov Random Field model

In this paper, we propose a Markov Random Field model-based approach as a unified and systematic way for modeling, encoding and applying scene knowledge to the image understanding problem. In our proposed scheme we formulate the image segmentation and interpretation problem as an integrated scheme and solve it through a general optimization algorithm. More specifically, the image is first segmented into a set of disjoint regions by a conventional region-based segmentation technique which operates on image pixels, and a Region Adjacency Graph (RAG) is then constructed from the resulting segmented regions based on the spatial adjacencies between regions. Our scheme then proceeds on the RAG by defining the region merging and labeling problem based on the MRF models. In the MRF model we specify the a priori knowledge about the optimal segmentation and interpretation in the form of clique functions and those clique functions are incorporated into the energy function to be minimized by a general optimization technique. In the proposed scheme, the image segmentation and interpretation processes cooperate in the simultaneous optimization process such that the erroneous segmentation and misinterpretation due to incomplete knowledge about each problem domain can be compensately recovered by continuous estimation of the single unified energy function. We exploit the proposed scheme to segment and interpret natural outdoor scene images.     

基于超复视域注意模型的视频分割算法

提出一种基于超复视域注意模型的视频分割算法,无需事先针对特定类型的目标进行训练。通过构造超复视域注意帧图像,对超复视域注意帧图像计算相位相关实现运动建模,利用条件随机场对视域注意模型、颜色模型以及邻域关系模型进行约束求解,获得分割结果。采用不同的视频数据对该算法的有效性进行测试,并与其他分割算法的结果进行比较。实验结果表明,该算法的分割错误率较低。     

基于物体间支撑语义关系的室内场景彩色深度图像分割

场景理解是智能自主机器人领域的一个重要研究方向,而图像分割是场景理解的基础.但是,不完备的训练数据集,以及真实环境中的罕见情形,会导致在图像分割时存在先验知识不完备的情况,进而影响图像分割的效果.因此,提出在彩色深度(RGB–D)图像上使用抽象的支撑语义关系来解决多样的物体形态所面对的先验知识不完备问题.在先验知识不完备情况下,针对自底向上的图像分割过程中被过度分割出的物体块,首先对物体块间的支撑语义关系进行建模并计算其支撑概率,然后构造能够度量场景总体稳定性的能量函数,最后通过Swendsen-Wang割(SWC)随机图分割算法最小化该能量函数的值,将物体块间的支撑概率转化为强支撑语义关系并完成物体块合并,实现先验知识不完备情况下的图像分割.实验结果证明,结合支撑语义关系的图像分割能够在先验知识不完备的情况下,将同一物体被过度分割的部分重新合并起来,从而提升了图像分割的准确性.     

Joint segmentation of collectively moving objects using a bag-of-words model and level set evolution

In scenes with collectively moving objects, to disregard the individual objects and take the entire group into consideration for motion characterization is a promising approach with wide application prospects. In contrast to studies on the segmentation of independently moving objects, our purpose is to construct a segmentation of these objects to characterize their motions at a macroscopic level. In general, the collectively moving objects in a group have very similar motion behavior with their neighbors and appear as a kind of global collective motion. This paper presents a joint segmentation approach for these collectively moving objects. In our model, we extract these macroscopic movement patterns based on optical flow field sequences. Specifically, a group of collectively moving objects correspond to a region where the optical flow field has high magnitude and high local direction coherence. As a result, our problem can be addressed by identifying these coherent optical flow field regions. The segmentation is performed through the minimization of a variational energy functional derived from the Bayes classification rule. Specifically, we use a bag-of-words model to generate a codebook as a collection of prototypical optical flow patterns, and the class-conditional probability density functions for different regions are determined based on these patterns. Finally, the minimization of our proposed energy functional results in the gradient descent evolution of segmentation boundaries which are implicitly represented through level sets. The application of our proposed approach is to segment and track multiple groups of collectively moving objects in a large variety of real-world scenes.     

Learning what is where from unlabeled images: joint localization and clustering of foreground objects

“What does it mean, to see? The plain man’s answer would be, to know what is where by looking.” This famous quote by David Marr (Vision: A Computational Investigation into the Human Representation and Processing of Visual Information, Freeman, New York, 1982) sums up the holy grail of vision: discovering what is present in the world, and where it is, from unlabeled images. In this paper we tackle this challenging problem by proposing a generative model of object formation and describe an efficient algorithm to automatically learn the parameters of the model from a collection of unlabeled images. Our algorithm discovers the objects and their spatial extents by clustering together images containing similar foregrounds. Our approach simultaneously solves for the image clusters, the foreground appearance models and the spatial regions containing the objects by optimizing a single likelihood function defined over the entire image collection. We describe two methods for efficient foreground localization: the first method does not require any bottom-up image segmentation and discovers the foreground region as a contiguous rectangular bounding box. The second method expresses the foreground as a collection of super-pixels generated through a bottom-up segmentation of the image. However, unlike previous methods, objects are not assumed to be encapsulated by a single segment. Evaluation on standard benchmarks and comparison with prior methods demonstrate that our approach achieves state-of-the-art results on the problem of unsupervised foreground localization and clustering.     

结合相似性拟合与空间约束的图像分割

目的 图像中的目标一般含有很多子类,仅仅利用某个子类的特征无法完整地分割出目标区域。针对这一问题,提出一种结合相似性拟合与空间约束的图像交互式分割方法。方法 首先,通过手工标记的样本组成各个目标的字典,通过相似度量搜寻测试样本与各个目标的字典中最相似的原子建立拟合项;再结合图像的空间约束项,构建图像分割模型;最后利用连续最大流算法求解,快速实现图像分割的目的。结果 通过对比实验,本文方法的速度比基于稀疏表示的分类方法的速度提高约13倍,而与归一化切割(N-Cut),逻辑回归(logistic regression)等方法相比,本文方法能取得更稳定和准确的分割结果。此外,本文方法无需过完备字典,只需要训练样本能体现各个子类的信息即可得到稳定的图像分割结果。结论 本文交互式图像分割方法,通过结合相似性拟合以及空间约束建立分割模型,并由连续最大流算法求解,实现图像的快速准确的分割。实验结果表明,该方法能够胜任较准确地对自然图像进行分割以及目标提取等任务。     

融合分割先验的多图像目标语义分割

目的 在序列图像或多视角图像的目标分割中，传统的协同分割算法对复杂的多图像分割鲁棒性不强，而现有的深度学习算法在前景和背景存在较大歧义时容易导致目标分割错误和分割不一致。为此，提出一种基于深度特征的融合分割先验的多图像分割算法。方法 首先，为了使模型更好地学习复杂场景下多视角图像的细节特征，通过融合浅层网络高分辨率的细节特征来改进PSPNet-50网络模型，减小随着网络的加深导致空间信息的丢失对分割边缘细节的影响。然后通过交互分割算法获取一至两幅图像的分割先验，将少量分割先验融合到新的模型中，通过网络的再学习来解决前景/背景的分割歧义以及多图像的分割一致性。最后通过构建全连接条件随机场模型，将深度卷积神经网络的识别能力和全连接条件随机场优化的定位精度耦合在一起，更好地处理边界定位问题。结果 本文采用公共数据集的多图像集进行了分割测试。实验结果表明本文算法不但可以更好地分割出经过大量数据预训练过的目标类，而且对于没有预训练过的目标类，也能有效避免歧义的区域分割。本文算法不论是对前景与背景区别明显的较简单图像集，还是对前景与背景颜色相似的较复杂图像集，平均像素准确度（PA）和交并比（IOU）均大于95%。结论 本文算法对各种场景的多图像分割都具有较强的鲁棒性，同时通过融入少量先验，使模型更有效地区分目标与背景，获得了分割目标的一致性。     

基于区域的多类目标识别与分割算法研究

为了实现复杂自然场景中多类目标的识别与分割,利用条件随机场(CRF)对目标特征进行建模,并在此基础上运用过分割算法将图片分为有限个连续区域,提出一种新的基于区域的CRF模型,即R-CRF模型,并采用Joint-boost算法对标注样本进行训练,研究基于主题的R-CRF模型在多类目标识别与分割中的应用。MSRC-21类数据库的实验结果表明,该算法在多类目标识别与分割中取得的结果优于国内外其他算法,尤其对于其他算法中正确率很低的形状多变而样本少的高结构物体的识别和分割取得了很好的结果。     

Bilayer segmentation of webcam videos using tree-based classifiers

This paper presents an automatic segmentation algorithm for video frames captured by a (monocular) webcam that closely approximates depth segmentation from a stereo camera. The frames are segmented into foreground and background layers that comprise a subject (participant) and other objects and individuals. The algorithm produces correct segmentations even in the presence of large background motion with a nearly stationary foreground. This research makes three key contributions: First, we introduce a novel motion representation, referred to as "motons," inspired by research in object recognition. Second, we propose estimating the segmentation likelihood from the spatial context of motion. The estimation is efficiently learned by random forests. Third, we introduce a general taxonomy of tree-based classifiers that facilitates both theoretical and experimental comparisons of several known classification algorithms and generates new ones. In our bilayer segmentation algorithm, diverse visual cues such as motion, motion context, color, contrast, and spatial priors are fused by means of a conditional random field (CRF) model. Segmentation is then achieved by binary min-cut. Experiments on many sequences of our videochat application demonstrate that our algorithm, which requires no initialization, is effective in a variety of scenes, and the segmentation results are comparable to those obtained by stereo systems.     

On using an analogy to heat flow for shape extraction

We introduce a novel evolution-based segmentation algorithm which uses the heat flow analogy to gain practical advantage. The proposed algorithm consists of two parts. In the first part, we represent a particular heat conduction problem in the image domain to roughly segment the region of interest. Then we use geometric heat flow to complete the segmentation, by smoothing extracted boundaries and removing noise inside the prior segmented region. The proposed algorithm is compared with active contour models and is tested on synthetic and medical images. Experimental results indicate that our approach works well in noisy conditions without pre-processing. It can detect multiple objects simultaneously. It is also computationally more efficient and easier to control and implement in comparison with active contour models.     

Multi-object reconstruction from dynamic scenes: An object-centered approach

In this paper, we present a new framework for three-dimensional (3D) reconstruction of multiple rigid objects from dynamic scenes. Conventional 3D reconstruction from multiple views is applicable to static scenes, in which the configuration of objects is fixed while the images are taken. In our framework, we aim to reconstruct the 3D models of multiple objects in a more general setting where the configuration of the objects varies among views. We solve this problem by object-centered decomposition of the dynamic scenes using unsupervised co-recognition approach. Unlike conventional motion segmentation algorithms that require small motion assumption between consecutive views, co-recognition method provides reliable accurate correspondences of a same object among unordered and wide-baseline views. In order to segment each object region, we benefit from the 3D sparse points obtained from the structure-from-motion. These points are reliable and serve as automatic seed points for a seeded-segmentation algorithm. Experiments on various real challenging image sequences demonstrate the effectiveness of our approach, especially in the presence of abrupt independent motions of objects.     

融合深度模型和传统模型的显著性检测

目的 显著性检测是图像和视觉领域一个基础问题,传统模型对于显著性物体的边界保留较好,但是对显著性目标的自信度不够高,召回率低,而深度学习模型对于显著性物体的自信度高,但是其结果边界粗糙,准确率较低。针对这两种模型各自的优缺点,提出一种显著性模型以综合利用两种方法的优点并抑制各自的不足。方法 首先改进最新的密集卷积网络,训练了一个基于该网络的全卷积网络（FCN）显著性模型,同时选取一个现有的基于超像素的显著性回归模型,在得到两种模型的显著性结果图后,提出一种融合算法,融合两种方法的结果以得到最终优化结果,该算法通过显著性结果Hadamard积和像素间显著性值的一对一非线性映射,将FCN结果与传统模型的结果相融合。结果 实验在4个数据集上与最新的10种方法进行了比较,在HKU-IS数据集中,相比于性能第2的模型,F值提高了2.6%;在MSRA数据集中,相比于性能第2的模型,F值提高了2.2%,MAE降低了5.6%;在DUT-OMRON数据集中,相比于性能第2的模型,F值提高了5.6%,MAE降低了17.4%。同时也在MSRA数据集中进行了对比实验以验证融合算法的有效性,对比实验结果表明提出的融合算法改善了显著性检测的效果。结论 本文所提出的显著性模型,综合了传统模型和深度学习模型的优点,使显著性检测结果更加准确。     

Moving object detection using Markov Random Field and Distributed Differential Evolution

In this article, we present an algorithm for detecting moving objects from a given video sequence. Here, spatial and temporal segmentations are combined together to detect moving objects. In spatial segmentation, a multi-layer compound Markov Random Field (MRF) is used which models spatial, temporal, and edge attributes of image frames of a given video. Segmentation is viewed as a pixel labeling problem and is solved using the maximum a posteriori (MAP) probability estimation principle; i.e., segmentation is done by searching a labeled configuration that maximizes this probability. We have proposed using a Differential Evolution (DE) algorithm with neighborhood-based mutation (termed as Distributed Differential Evolution (DDE) algorithm) for estimating the MAP of the MRF model. A window is considered over the entire image lattice for mutation of each target vector of the DDE; thereby enhancing the speed of convergence. In case of temporal segmentation, the Change Detection Mask (CDM) is obtained by thresholding the absolute differences of the two consecutive spatially segmented image frames. The intensity/color values of the original pixels of the considered current frame are superimposed in the changed regions of the modified CDM to extract the Video Object Planes (VOPs). To test the effectiveness of the proposed algorithm, five reference and one real life video sequences are considered. Results of the proposed method are compared with four state of the art techniques and provide better spatial segmentation and better identification of the location of moving objects.     

渐进式前景更新和区域关联的图像协同分割

目的 图像协同分割技术是通过多幅参考图像以实现前景目标与背景区域的分离，并已被广泛应用于图像分类和目标识别等领域中。不过，现有多数的图像协同分割算法只适用于背景变化较大且前景几乎不变的环境。为此，提出一种新的无监督协同分割算法。方法 本文方法是无监督式的，在分级图像分割的基础上通过渐进式优化框架分别实现前景和背景模型的更新估计，同时结合图像内部和不同图像之间的分级区域相似度关联进一步增强上述模型估计的鲁棒性。该无监督的方法不需要进行预先样本学习，能够同时处理两幅或多幅图像且适用于同时存在多个前景目标的情况，并且能够较好地适应前景物体类的变化。结果 通过基于iCoseg和MSRC图像集的实验证明，该算法无需图像间具有显著的前景和背景差异这一约束，与现有的经典方法相比更适用于前景变化剧烈以及同时存在多个前景目标等更为一般化的图像场景中。结论 该方法通过对分级图像分割得到的超像素外观分布分别进行递归式估计来实现前景和背景的有效区分，并同时融合了图像内部以及不同图像区域之间的区域关联性来增加图像前景和背景分布估计的一致性。实验表明当前景变化显著时本文方法相比于现有方法具有更为鲁棒的表现。     

A fast algorithm for automatic segmentation and extraction of a single object by active surfaces

Segmentation is an important problem in various applications. There exist many effective models designed to locate all features and their boundaries in an image. However such global models are not suitable for automatically detecting a single object among many objects of an image, because nearby objects are often selected as well. Several recent works can provide selective segmentation capability but unfortunately when generalized to three dimensions, they are not yet effective or efficient. This paper presents a selective segmentation model which is inherently suited for efficient implementation. With the added solver by a fast nonlinear multigrid method for the inside domain of a zero level set function, the over methodology leads to an effective and efficient algorithm for 3D selective segmentation. Numerical experiments show that our model can produce efficient results in terms of segmentation quality and reliability for a large class of 3D images.     

基于活动轮廓的多分辨率自适应图像分割

本文在活动轮廓模型的基础上,提出了一种自适应图像分割方法,引入了新的图象统计信息、梯度信息有关的加权外部能量,使得分割结果与模型的初始位置无关,不受噪声影响;利用ACD方法使模型自适应地改变其拓扑结构;为了提高图象分的速度和鲁棒性,提出了多分辨率图象分割算法,利用该方法对一些形状、拓扑结构复杂的物体进行了分割实验,结果验证了该方法有效性。     

一种判断交叉物体前后景的方法

在计算机图像处理中,常常会出现两个物体在二维影像中有交叉的现象,当两个物体的灰度比较相似时,通常计算机很难辨别出交叉部分属于哪个物体,从而影响物体图像的分割。文章提出了一种通过检测交叉区域,再用模板相关的方法对交叉区域分割的算法,较好地解决了灰度相近物体图像交叉区域分割问题。