Dynamic learning from multiple examples for semantic object segmentation and search

We present a novel “dynamic learning” approach for an intelligent image database system to automatically improve object segmentation and labeling without user intervention, as new examples become available, for object-based indexing. The proposed approach is an extension of our earlier work on “learning by example,” which addressed labeling of similar objects in a set of database images based on a single example. The proposed dynamic learning procedure utilizes multiple example object templates to improve the accuracy of existing object segmentations and labels. Multiple example templates may be images of the same object from different viewing angles, or images of related objects. This paper also introduces a new shape similarity metric called normalized area of symmetric differences (NASD), which has desired properties for use in the proposed “dynamic learning” scheme, and is more robust against boundary noise that results from automatic image segmentation. Performance of the dynamic learning procedures has been demonstrated by experimental results.     

渐进式前景更新和区域关联的图像协同分割

目的 图像协同分割技术是通过多幅参考图像以实现前景目标与背景区域的分离，并已被广泛应用于图像分类和目标识别等领域中。不过，现有多数的图像协同分割算法只适用于背景变化较大且前景几乎不变的环境。为此，提出一种新的无监督协同分割算法。方法 本文方法是无监督式的，在分级图像分割的基础上通过渐进式优化框架分别实现前景和背景模型的更新估计，同时结合图像内部和不同图像之间的分级区域相似度关联进一步增强上述模型估计的鲁棒性。该无监督的方法不需要进行预先样本学习，能够同时处理两幅或多幅图像且适用于同时存在多个前景目标的情况，并且能够较好地适应前景物体类的变化。结果 通过基于iCoseg和MSRC图像集的实验证明，该算法无需图像间具有显著的前景和背景差异这一约束，与现有的经典方法相比更适用于前景变化剧烈以及同时存在多个前景目标等更为一般化的图像场景中。结论 该方法通过对分级图像分割得到的超像素外观分布分别进行递归式估计来实现前景和背景的有效区分，并同时融合了图像内部以及不同图像区域之间的区域关联性来增加图像前景和背景分布估计的一致性。实验表明当前景变化显著时本文方法相比于现有方法具有更为鲁棒的表现。     

M:N object matching between image and map object data sets by means of latent semantic analysis

In this article, we propose a method to find corresponding object-set pairs between image and map polygon object data sets by means of latent semantic analysis. Latent semantic analysis assigns each polygon object of both data sets to feature vectors in a continuous geometric space in which the similarities between the vectors are proportional to the priorities to constitute a corresponding object-set pair. Thus, object clusters can be obtained by applying an agglomerative hierarchical clustering to the feature vectors. These object clusters are separated into object-set pairs according to the data sets to which the objects belong and are evaluated with a geometric matching criterion to find corresponding object-set pairs. We applied the proposed method to the segmentation result of a composite image with six normalized difference vegetation index (NDVI) images and a forest inventory map. The proposed method was compared to a graph-embedding-based method. The results showed that the proposed method found more corresponding object-set pairs with a similar accuracy in terms of shape similarities and shared information of found pairs.     

图割与非线性统计形状先验的图像分割

提出一种图割与非线性统计形状先验的图像分割方法。首先,在输入空间对输入的形状模板进行配准,得到训练集;其次,采用非线性核函数将目标形状先验映射到特征空间进行主成分分析,获取其投影形状,将此投影形状映射回原输入空间得到目标的平均形状,构成新的能量函数;第三,通过自适应调整形状先验项的权值系数,使能量函数的形状先验项自适应于被分割的图像;最后,用Graph Cuts方法最小化能量函数完成图像分割。实验结果表明,该方法不仅能准确分割与形状先验模板有差别的图像,而且对目标有遮挡或污染的图像也有较好的分割效果,提高了分割效率。     

融合区域和边缘特征的水平集水下图像分割

目的 水平集模型是图像分割中的一种先进方法,在陆地环境图像分割中展现出较好效果。特征融合策略被广泛引入到该模型框架,以拉伸目标-背景对比度,进而提高对高噪声、杂乱纹理等多类复杂图像的处理性能。然而,在水下环境中,由于水体高散射、强衰减等多因素的共同作用,使得现有图像特征及水平集模型难以适用于对水下图像的分割任务,分割结果与目标形态间存在较大差异。鉴于此,提出一种适用于水下图像分割的区域-边缘水平集模型,以提高水下图像目标分割的准确性。方法 综合应用图像的区域特征及边缘特征对水下目标进行辨识。对于区域特征,引入水下图像显著性特征;对于边缘特征,创新性地提出了一种基于深度信息的边缘特征提取方法。所提方法在融合区域级和边缘级特征的基础上,引入距离正则项对水平集函数进行规范,以增强水平集函数演化的稳定性。结果 基于YouTube和Bubblevision的水下数据集的实验结果表明,所提方法不仅对高散射强衰减的低对比度水下图像实现较好的分割效果,同时对处理强背景噪声图像也有较好的鲁棒性,与水平集分割方法（local pre-fitting,LPF）相比,分割精确度至少提高11.5%,与显著性检测方法（hierarchical co-salient detection via color names,HCN）相比,精确度提高6.7%左右。结论 实验表明区域-边缘特征融合以及其基础上的水平集模型能够较好地克服水下图像分割中的部分难点,所提方法能够较好分割水下目标区域并拟合目标轮廓,与现有方法对比获得了较好的分割结果。     

Multiscale Active Contours

We propose a new multiscale image segmentation model, based on the active contour/snake model and the Polyakov action. The concept of scale, general issue in physics and signal processing, is introduced in the active contour model, which is a well-known image segmentation model that consists of evolving a contour in images toward the boundaries of objects. The Polyakov action, introduced in image processing by Sochen-Kimmel-Malladi in Sochen et al. (1998), provides an efficient mathematical framework to define a multiscale segmentation model because it generalizes the concept of harmonic maps embedded in higher-dimensional Riemannian manifolds such as multiscale images. Our multiscale segmentation model, unlike classical multiscale segmentations which work scale by scale to speed up the segmentation process, uses all scales simultaneously, i.e. the whole scale space, to introduce the geometry of multiscale images in the segmentation process. The extracted multiscale structures will be useful to efficiently improve the robustness and the performance of standard shape analysis techniques such as shape recognition and shape registration. Another advantage of our method is to use not only the Gaussian scale space but also many other multiscale spaces such as the Perona-Malik scale space, the curvature scale space or the Beltrami scale space. Finally, this multiscale segmentation technique is coupled with a multiscale edge detecting function based on the gradient vector flow model, which is able to extract convex and concave object boundaries independent of the initial condition. We apply our multiscale segmentation model on a synthetic image and a medical image.     

双层水平集描述眼底图像视杯视盘分割

目的 青光眼是一种可导致视力严重减弱甚至失明的高发眼部疾病。在眼底图像中,视杯和视盘的检测是青光眼临床诊断的重要步骤之一。然而,眼底图像普遍是灰度不均匀的,眼底结构复杂,不同结构之间的灰度重叠较多,受到血管和病变的干扰较为严重。这些都给视盘与视杯的分割带来很大挑战。因此,为了更准确地提取眼底图像中的视杯和视盘区域,提出一种基于双层水平集描述的眼底图像视杯视盘分割方法。方法 通过水平集函数的不同层级分别表示视杯轮廓和视盘轮廓,依据视杯与视盘间的位置关系建立距离约束,应用图像的局部信息驱动活动轮廓演化,克服图像的灰度不均匀性。根据视杯与视盘的几何形状特征,引入视杯与视盘形状的先验信息约束活动轮廓的演化,从而实现视杯与视盘的准确分割。结果 本文使用印度Aravind眼科医院提供的具有视杯和视盘真实轮廓注释的CDRISHTI-GS1数据集对本文方法进行实验验证。该数据集主要用来验证视杯及视盘分割方法的鲁棒性和有效性。本文方法在数据集上对视杯和视盘区域进行分割,取得了67.52%的视杯平均重叠率,81.04%的视盘平均重叠率,0.719的视杯F1分数和0.845的视盘F1分数,结果优于基于COSFIRE（combination of shifted filter responses）滤波模型的视杯视盘分割方法、基于先验形状约束的多相Chan-Vese（C-V）模型和基于聚类融合的水平集方法。结论 实验结果表明,本文方法能够有效克服眼底图像灰度不均匀、血管及病变区域的干扰等影响,更为准确地提取视杯与视盘区域。     

Prior-based Segmentation and Shape Registration in the Presence of Perspective Distortion

Challenging object detection and segmentation tasks can be facilitated by the availability of a reference object. However, accounting for possible transformations between the different object views, as part of the segmentation process, remains difficult. Recent statistical methods address this problem by using comprehensive training data. Other techniques can only accommodate similarity transformations. We suggest a novel variational approach to prior-based segmentation, using a single reference object, that accounts for planar projective transformation. Generalizing the Chan-Vese level set framework, we introduce a novel shape-similarity measure and embed the projective homography between the prior shape and the image to segment within a region-based segmentation functional. The proposed algorithm detects the object of interest, extracts its boundaries, and concurrently carries out the registration to the prior shape. We demonstrate prior-based segmentation on a variety of images and verify the accuracy of the recovered transformation parameters.     

A reinforcement agent for object segmentation in ultrasound images

The principal contribution of this work is to design a general framework for an intelligent system to extract one object of interest from ultrasound images. This system is based on reinforcement learning. The input image is divided into several sub-images, and the proposed system finds the appropriate local values for each of them so that it can extract the object of interest. The agent uses some images and their ground-truth (manually segmented) version to learn from. A reward function is employed to measure the similarities between the output and the manually segmented images, and to provide feedback to the agent. The information obtained can be used as valuable knowledge stored in the Q-matrix. The agent can then use this knowledge for new input images. The experimental results for prostate segmentation in trans-rectal ultrasound images show high potential of this approach in the field of ultrasound image segmentation.     

Automatic segmentation of point clouds from multi-view reconstruction using graph-cut

In multi-view reconstruction systems, the recovered point cloud often consists of numerous unwanted background points. We propose a graph-cut based method for automatically segmenting point clouds from multi-view reconstruction. Based on the observation that the object of interest is likely to be central to the intended multi-view images, our method requires no user interaction except two roughly estimated parameters of objects covering in the central area of images. The proposed segmentation process is carried out in two steps: first, we build a weighted graph whose nodes represent points and edges that connect each point to its k-nearest neighbors. The potentials of each point being object and background are estimated according to distances between its projections in images and the corresponding image centers. The pairwise potentials between each point and its neighbors are computed according to their positions, colors and normals. Graph-cut optimization is then used to find the initial binary segmentation of object and background points. Second, to refine the initial segmentation, Gaussian mixture models (GMMs) are created from the color and density features of points in object and background classes, respectively. The potentials of each point being object and background are re-calculated based on the learned GMMs. The graph is updated and the segmentation of point clouds is improved by graph-cut optimization. The second step is iterated until convergence. Our method requires no manual labeling points and employs available information of point clouds from multi-view systems. We test the approach on real-world data generated by multi-view reconstruction systems.     

改进K-means活动轮廓模型

目的 通过对C-V模型能量泛函的Euler-Lagrange方程进行变形,建立其与K-means方法的等价关系,提出一种新的基于水平集函数的改进K-means活动轮廓模型。方法 该模型包含局部自适应权重矩阵函数,它根据像素点所在邻域的局部统计信息自适应地确定各个像素点的分割阈值,排除灰度非同质对分割目标的影响,进而实现对灰度非同质图像的精确分割。结果 通过分析对合成以及自然图像的分割结果,与传统及最新经典的活动轮廓模型相比,新模型不仅能较准确地分割灰度非同质图像,而且降低了对初始曲线选取的敏感度。结论 提出了包含权重矩阵函数的新活动轮廓模型,根据分割目的和分割图像性质,制定不同的权重函数,该模型具有广泛的适用性。文中给出的一种具有局部统计特性的权重函数,对灰度非同质图像的效果较好,且对初始曲线位置具有稳定性。     

基于水动力学、分数阶微分及Steger算法的线状目标提取

目的 线状目标的检测具有非常广泛的应用领域,如车道线、道路及裂缝的检测等,而裂缝是其中最难检测的线状目标。为避免直接提取线状目标时图像分割难的问题,以裂缝和车道线为例,提出了一种新的跟踪线状目标中线的算法。方法 对图像进行高斯平滑,用一种新的分数阶微分模板增强图像中的模糊及微细线状目标;基于Steger算法提出一种提取线状目标中心线特征点的算法,避免了提取整体目标的困难;根据水动力学思想将裂隙看成溪流,通过最大熵阈值处理后,先进行特征点的连接,再基于线段之间的距离及夹角进行线段之间的连接（溪流之间的融合）。结果 对300幅裂缝图像及4种类别的其他线状目标图像进行试验,并与距离变换、最大熵阈值法+细线化Otsu阈值分割+细线化、谷底边界检测等类似算法进行比较分析,本文算法检测出的线状目标的连续性好、漏检（大间隙少）和误检（毛刺及多余线段少）率均较低。结论 本文算法能够在复杂的线状目标图像中准确快速地提取目标的中心线,一定程度上改善了复杂线状目标图像分割难的问题。     

面向RGBD图像的标记分水岭分割

目的 针对分水岭分割算法中存在的过分割现象及现有基于RGB图像分割方法的局限,提出了一种基于RGB图像和深度图像(RGBD)的标记分水岭分割算法。方法 本文使用物体表面几何信息来辅助进行图像分割,定义了一种深度梯度算子和一种法向量梯度算子来衡量物体表面几何信息的变化。通过生成深度梯度图像和法向量梯度图像,与彩色梯度图像进行融合,实现标记图像的提取。在此基础上,使用极小值标定技术对彩色梯度图像进行修正,然后使用分水岭算法进行图像分割。结果 在纽约大学提供的NYU2数据集上进行实验,本文算法有效抑制了过分割现象,将分割区域从上千个降至数十个,且获得了与人工标定的分割结果更接近的分割效果,分割的准确率也比只使用彩色图像进行分割提高了10%以上。结论 本文算法普遍适用于RGBD图像的分割问题,该算法加入了物体表面几何信息的使用,提高了分割的准确率,且对颜色纹理相似的区域获得了较好的分割结果。     

局部区域拟合的快速水平集图像分割方法

目的 由于灰度不均匀图像在不同目标区域的灰度分布存在严重的重叠,对其进行分割仍然是一个难题;同时,图像中的噪声严重降低了图像分割的准确性。因此,传统水平集方法无法鲁棒、精确、快速地对具有灰度不均匀性和噪声的图像进行分割。针对这一问题,提出一种基于局部区域信息的快速水平集图像分割方法。方法 灰度不均匀图像通常被描述为一个分段常数图像乘以一个缓慢变化的偏移场。首先,通过一个经过微调的多尺度均值滤波器来估计图像的偏移场,并对图像进行预处理以减轻图像的不均匀性;然后,利用基于偏移场校正的方法和基于局部区域信息拟合的方法分别构建能量项,并利用演化曲线轮廓内外图像灰度分布的重叠程度,构建权重函数自适应调整两个能量项之间的权重;最后,引入全方差规则项对水平集进行约束,增强了数值计算的稳定性和对噪声的鲁棒性,并通过加性算子分裂策略实现水平集快速演化。结果 在具有不同灰度不均匀性和噪声图像上的分割结果表明,所提方法不但对初始轮廓的位置、灰度不均匀性和各种噪声具有较强的鲁棒性,而且具有高达94.5%的分割精度和较高的分割效率,与传统水平集方法相比分割精度至少提高了20.6%,分割效率是LIC（local intensity clustering）模型的9倍;结论 本文提出一种基于局部区域信息的快速水平集图像分割方法。实验结果表明,与传统水平集方法相比具有较高的分割精度和分割效率,可以很好地应用于具有灰度不均匀和噪声的医学、红外和自然图像等的分割。     

Shape Based Detection and Top-Down Delineation Using Image Segments

We introduce a segmentation-based detection and top-down figure-ground delineation algorithm. Unlike common methods which use appearance for detection, our method relies primarily on the shape of objects as is reflected by their bottom-up segmentation. Our algorithm receives as input an image, along with its bottom-up hierarchical segmentation. The shape of each segment is then described both by its significant boundary sections and by regional, dense orientation information derived from the segment’s shape using the Poisson equation. Our method then examines multiple, overlapping segmentation hypotheses, using their shape and color, in an attempt to find a “coherent whole,” i.e., a collection of segments that consistently vote for an object at a single location in the image. Once an object is detected, we propose a novel pixel-level top-down figure-ground segmentation by “competitive coverage” process to accurately delineate the boundaries of the object. In this process, given a particular detection hypothesis, we let the voting segments compete for interpreting (covering) each of the semantic parts of an object. Incorporating competition in the process allows us to resolve ambiguities that arise when two different regions are matched to the same object part and to discard nearby false regions that participated in the voting process. We provide quantitative and qualitative experimental results on challenging datasets. These experiments demonstrate that our method can accurately detect and segment objects with complex shapes, obtaining results comparable to those of existing state of the art methods. Moreover, our method allows us to simultaneously detect multiple instances of class objects in images and to cope with challenging types of occlusions such as occlusions by a bar of varying size or by another object of the same class, that are difficult to handle with other existing class-specific top-down segmentation methods.     

Adaptive active contour model driven by image data field for image segmentation with flexible initialization

In this paper, a novel adaptive active contour model based on image data field for image segmentation with robust and flexible initializations is proposed. We firstly construct a new external energy term deduced from the image data field that drives the level set function to move in the opposite direction along the boundaries of object and an adaptive length regularization term based on the image local entropy. The designed external energy and length regularization term are then incorporated into a variationlevel set framework with an additional penalizing energy term. Due to the adaptive sign–changing property of the external energy and the adaptive length regularization term, the proposed model can tackle images with clutter background and noise, the level set function can be initialized as any bounded functions (e.g., constant function), which implies the proposed model is robust to initialization of contours. Experimental results on both synthetic and real images from different modalities confirm the effectiveness and competivive performance of the proposed method compared with other representative models.

     

Self-organizing neural networks for image segmentation based on multiphase active contour

Image segmentation is a process of segregating foreground object from background object in an image. This paper proposes a method to perform image segmentation for the color and textured images with a two-step approach. In the first step, self-organizing neurons based on neural networks are used for clustering the input image, and in the second step, multiphase active contour model is used to get various segments of an image. The contours are initialized in the active contour model with the help of the self-organizing maps obtained as a result of first step. From the results, it is inferred that the proposed method provides better segmentation result for all types of images.

     

On Symmetry, Perspectivity, and Level-Set-Based Segmentation

We introduce a novel variational method for the extraction of objects with either bilateral or rotational symmetry in the presence of perspective distortion. Information on the symmetry axis of the object and the distorting transformation is obtained as a by--product of the segmentation process. The key idea is the use of a flip or a rotation of the image to segment as if it were another view of the object. We call this generated image the symmetrical counterpart image. We show that the symmetrical counterpart image and the source image are related by planar projective homography. This homography is determined by the unknown planar projective transformation that distorts the object symmetry. The proposed segmentation method uses a level-set-based curve evolution technique. The extraction of the object boundaries is based on the symmetry constraint and the image data. The symmetrical counterpart of the evolving level-set function provides a dynamic shape prior. It supports the segmentation by resolving possible ambiguities due to noise, clutter, occlusions, and assimilation with the background. The homography that aligns the symmetrical counterpart to the source level-set is recovered via a registration process carried out concurrently with the segmentation. Promising segmentation results of various images of approximately symmetrical objects are shown.     

Interactive image segmentation by maximal similarity based region merging

Efficient and effective image segmentation is an important task in computer vision and object recognition. Since fully automatic image segmentation is usually very hard for natural images, interactive schemes with a few simple user inputs are good solutions. This paper presents a new region merging based interactive image segmentation method. The users only need to roughly indicate the location and region of the object and background by using strokes, which are called markers. A novel maximal-similarity based region merging mechanism is proposed to guide the merging process with the help of markers. A region R is merged with its adjacent region Q if Q has the highest similarity with Q among all Q's adjacent regions. The proposed method automatically merges the regions that are initially segmented by mean shift segmentation, and then effectively extracts the object contour by labeling all the non-marker regions as either background or object. The region merging process is adaptive to the image content and it does not need to set the similarity threshold in advance. Extensive experiments are performed and the results show that the proposed scheme can reliably extract the object contour from the complex background.