Semiautomatic segmentation with compact shape prior

In recent years, interactive methods for segmentation are increasing in popularity due to their success in different domains such as medical image processing, photo editing, etc. We present an interactive segmentation algorithm that can segment an object of interest from its background with minimum guidance from the user, who just has to select a single seed pixel inside the object of interest. Due to minimal requirements from the user, we call our algorithm semiautomatic. To obtain a reliable and robust segmentation with such low user guidance, we have to make several assumptions. Our main assumption is that the object to be segmented is of compact shape, or can be approximated by several connected roughly collinear compact pieces. We base our work on the powerful graph cut segmentation algorithm of Boykov and Jolly, which allows straightforward incorporation of the compact shape constraint. In order to make the graph cut approach suitable for our semiautomatic framework, we address several well-known issues of graph cut segmentation technique. In particular, we counteract the bias towards shorter segmentation boundaries and develop a method for automatic selection of parameters. We demonstrate the effectiveness of our approach on the challenging industrial application of transistor gate segmentation in images of integrated chips. Our approach produces highly accurate results in real-time.     

图割与非线性统计形状先验的图像分割

提出一种图割与非线性统计形状先验的图像分割方法。首先,在输入空间对输入的形状模板进行配准,得到训练集;其次,采用非线性核函数将目标形状先验映射到特征空间进行主成分分析,获取其投影形状,将此投影形状映射回原输入空间得到目标的平均形状,构成新的能量函数;第三,通过自适应调整形状先验项的权值系数,使能量函数的形状先验项自适应于被分割的图像;最后,用Graph Cuts方法最小化能量函数完成图像分割。实验结果表明,该方法不仅能准确分割与形状先验模板有差别的图像,而且对目标有遮挡或污染的图像也有较好的分割效果,提高了分割效率。     

Self-organizing neural networks for image segmentation based on multiphase active contour

Image segmentation is a process of segregating foreground object from background object in an image. This paper proposes a method to perform image segmentation for the color and textured images with a two-step approach. In the first step, self-organizing neurons based on neural networks are used for clustering the input image, and in the second step, multiphase active contour model is used to get various segments of an image. The contours are initialized in the active contour model with the help of the self-organizing maps obtained as a result of first step. From the results, it is inferred that the proposed method provides better segmentation result for all types of images.

     

基层层次光流的半自动时空视频分割技术

在新一代MPEG-4视频编码标准中，为了支持面向对象编码和实现基于内容的应用，视频的半自动分割成为关键技术之一，为此提出了一种基于层次光流的半自动时空视频分割算法。该算法由空域分割和时域分割组成。在空域分割中，提出的基于点的图形用户界面（PBGUI），在用户的协助下，能够精确地定义需要分割的视频对象（VO）。时域分割根据空域分割的结果采用层次光流算法对视频对象进行边界和整体跟踪。实验结果表明，利用该算法，能够精确地分割出视频对象。     

基于层次光流的半自动时空视频分割技术

在新一代 MPEG- 4视频编码标准中 ,为了支持面向对象编码和实现基于内容的应用 ,视频的半自动分割成为关键技术之一 ,为此提出了一种基于层次光流的半自动时空视频分割算法 .该算法由空域分割和时域分割组成 .在空域分割中 ,提出的基于点的图形用户界面 (PBGU I) ,在用户的协助下 ,能够精确地定义需要分割的视频对象 (VO) .时域分割根据空域分割的结果采用层次光流算法对视频对象进行边界和整体跟踪 .实验结果表明 ,利用该算法 ,能够较精确地分割出视频对象 .     

图注意力网络的场景图到图像生成模型

目的 目前文本到图像的生成模型仅在具有单个对象的图像数据集上表现良好，当一幅图像涉及多个对象和关系时，生成的图像就会变得混乱。已有的解决方案是将文本描述转换为更能表示图像中场景关系的场景图结构，然后利用场景图生成图像，但是现有的场景图到图像的生成模型最终生成的图像不够清晰，对象细节不足。为此，提出一种基于图注意力网络的场景图到图像的生成模型，生成更高质量的图像。方法 模型由提取场景图特征的图注意力网络、合成场景布局的对象布局网络、将场景布局转换为生成图像的级联细化网络以及提高生成图像质量的鉴别器网络组成。图注意力网络将得到的具有更强表达能力的输出对象特征向量传递给改进的对象布局网络，合成更接近真实标签的场景布局。同时，提出使用特征匹配的方式计算图像损失，使得最终生成图像与真实图像在语义上更加相似。结果 通过在包含多个对象的COCO-Stuff图像数据集中训练模型生成64×64像素的图像，本文模型可以生成包含多个对象和关系的复杂场景图像，且生成图像的Inception Score为7.8左右，与原有的场景图到图像生成模型相比提高了0.5。结论 本文提出的基于图注意力网络的场景图到图像生成模型不仅可以生成包含多个对象和关系的复杂场景图像，而且生成图像质量更高，细节更清晰。     

Object segmentation in hyperspectral images using active contours and graph cuts

The interest in object segmentation on hyperspectral images is increasing and many approaches have been proposed to deal with this area. In this project, we developed an algorithm that combines both the active contours and the graph cut approaches for object segmentation in hyperspectral images. The active contours approach has the advantage of producing subregions with continuous boundaries. The graph cut approach has emerged as a technique for minimizing energy functions while avoiding the problems of local minima. Additionally, it guarantees continuity and produces smooth contours, free of self-crossing and uneven spacing problems. The algorithm uses the complete spectral signature of a pixel and also considers spatial neighbourhood for graph construction, thereby combining both spectral and spatial information present in the image. The algorithm is tested using real hyperspectral images taken from a variety of sensors, such as Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) and Hyperspectral Data Imagery Collection Experiment (HYDICE), and also taken by the SOC hyperspectral camera. This approach can segment different objects in an image. This algorithm can be applied in many fields and it should represent an important advance in the field of object segmentation.     

Automatic segmentation of point clouds from multi-view reconstruction using graph-cut

In multi-view reconstruction systems, the recovered point cloud often consists of numerous unwanted background points. We propose a graph-cut based method for automatically segmenting point clouds from multi-view reconstruction. Based on the observation that the object of interest is likely to be central to the intended multi-view images, our method requires no user interaction except two roughly estimated parameters of objects covering in the central area of images. The proposed segmentation process is carried out in two steps: first, we build a weighted graph whose nodes represent points and edges that connect each point to its k-nearest neighbors. The potentials of each point being object and background are estimated according to distances between its projections in images and the corresponding image centers. The pairwise potentials between each point and its neighbors are computed according to their positions, colors and normals. Graph-cut optimization is then used to find the initial binary segmentation of object and background points. Second, to refine the initial segmentation, Gaussian mixture models (GMMs) are created from the color and density features of points in object and background classes, respectively. The potentials of each point being object and background are re-calculated based on the learned GMMs. The graph is updated and the segmentation of point clouds is improved by graph-cut optimization. The second step is iterated until convergence. Our method requires no manual labeling points and employs available information of point clouds from multi-view systems. We test the approach on real-world data generated by multi-view reconstruction systems.     

融合显著物体检测的图像显示适配

目的 图像显著适配旨在自动调节图像尺寸,对图像内容进行非均匀缩放,以便在受限的展示空间内更好地保留显著物体。为了解决显示适配过程中显著物体部分扭曲的问题,提出一种基于显著物体检测的图像显示适配方法。方法 本文方法采用显著物体分割结果来替代显著性图,以改进显示适配结果。首先,采用显著性融合和传播的方法生成显著性图;接着,结合输入图像和显著性图,采用自适应三阈值方法实现显著物体分割;然后,以此为基础,生成输入图像的曲边网格表示;最后,通过对不同网格的非均匀缩放,生成符合目标尺寸的适配结果。结果 在面向图像显示适配的公开数据集RetargetMe上,将本文方法与现有的10种代表性显示适配方法的结果进行了人工评估和比较。本文方法可以有效地减少显著物体出现部分扭曲的现象,能在48.8%的图像上取得无明显缺陷的适配效果,比现有最好的方法提高了5%。结论 基于显著物体检测的图像显示适配方法有助于提高显示适配过程中对显著物体处理的一致性,减少由于显著物体部分扭曲而引起的明显人工处理痕迹,从而达到提升显示适配效果的目的。     

Shape Based Detection and Top-Down Delineation Using Image Segments

We introduce a segmentation-based detection and top-down figure-ground delineation algorithm. Unlike common methods which use appearance for detection, our method relies primarily on the shape of objects as is reflected by their bottom-up segmentation. Our algorithm receives as input an image, along with its bottom-up hierarchical segmentation. The shape of each segment is then described both by its significant boundary sections and by regional, dense orientation information derived from the segment’s shape using the Poisson equation. Our method then examines multiple, overlapping segmentation hypotheses, using their shape and color, in an attempt to find a “coherent whole,” i.e., a collection of segments that consistently vote for an object at a single location in the image. Once an object is detected, we propose a novel pixel-level top-down figure-ground segmentation by “competitive coverage” process to accurately delineate the boundaries of the object. In this process, given a particular detection hypothesis, we let the voting segments compete for interpreting (covering) each of the semantic parts of an object. Incorporating competition in the process allows us to resolve ambiguities that arise when two different regions are matched to the same object part and to discard nearby false regions that participated in the voting process. We provide quantitative and qualitative experimental results on challenging datasets. These experiments demonstrate that our method can accurately detect and segment objects with complex shapes, obtaining results comparable to those of existing state of the art methods. Moreover, our method allows us to simultaneously detect multiple instances of class objects in images and to cope with challenging types of occlusions such as occlusions by a bar of varying size or by another object of the same class, that are difficult to handle with other existing class-specific top-down segmentation methods.     

High-resolution SAR images segmentation using NSCT denoising and QIGA based parameters selection of PCNN model

The high-resolution synthetic aperture radar (SAR) images usually contain inhomogeneous coherent speckle noises. For the high-resolution SAR image segmentation with such noises, the conventional methods based on pulse coupled neural networks (PCNN) have to face heavy parameters with a low efficiency. In order to solve the problems, this paper proposes a novel SAR image segmentation algorithm based on non-subsampling Contourlet transform (NSCT) denoising and quantum immune genetic algorithm (QIGA) improved PCNN models. The proposed method first denoising the SAR images for a pre-processing based on NSCT. Then, by using the QIGA to select parameters for the PCNN models, such models self-adaptively select the suitable parameters for segmentation of SAR images with different scenes. This method decreases the number of parameters in the PCNN models and improves the efficiency of PCNN models. At last, by using the optimal threshold to binary the segmented SAR images, the small objects and large scales from the original SAR images will be segmented. To validate the feasibility and effectiveness of the proposed algorithm, four different comparable experiments are applied to validate the proposed algorithm. Experimental results have shown that NSCT pre-processing has a better performance for coherent speckle noises suppression, and QIGA-PCNN model based on denoised SAR images has an obvious segmentation performance improvement on region consistency and region contrast than state-of-the-arts methods. Besides, the segmentation efficiency is also improved than conventional PCNN model, and the level of time complexity meets the state-of-the-arts methods. Our proposed NSCT+QIGA-PCNN model can be used for small object segmentation and large scale segmentation in high-resolution SAR images. The segmented results will be further used for object classification and recognition, regions of interest extraction, and moving object detection and tracking.

     

Object-based image labeling through learning by example and multi-level segmentation

We propose a method for automatic extraction and labeling of semantically meaningful image objects using “learning by example” and threshold-free multi-level image segmentation. The proposed method scans through images, each of which is pre-segmented into a hierarchical uniformity tree, to seek and label objects that are similar to an example object presented by the user. By representing images with stacks of multi-level segmentation maps, objects can be extracted in the segmentation map level with adequate detail. Experiments have shown that the proposed multi-level image segmentation results in significant reduction in computation complexity for object extraction and labeling (compared to a single fine-level segmentation) by avoiding unnecessary tests of combinations in finer levels. The multi-level segmentation-based approach also achieves better accuracy in detection and labeling of small objects.     

综合边界和纹理信息的合成孔径雷达图像目标分割

目的 针对传统Grab Cut算法需要人工交互操作,无法实现合成孔径雷达（SAR）图像的自动分割,且方式单一（仅利用边界或纹理信息中的一种）的问题,提出一种综合利用边界和纹理信息的改进Grab Cut算法,实现对SAR图像目标的自动分割。方法 首先将其他格式的彩色或灰度SAR图像转化为24 bit的位图,采用图形理论对整幅SAR图像建模,根据最大流算法找到描述图的能量函数最小的割集,从而分割出目标区域;然后采用中值滤波抑制相干噪声;最后通过邻域生长算法滤除图像斑点和小目标的干扰,从而达到目标边界的连接,实现自动对SAR图像中的目标进行分割。结果 在64位Window 7环境下采用MATLAB R2014处理平台,对楼房、车库、大树、汽车群等4幅分辨率不同的SAR图像进行目标分割实验,特征目标被自动分割出来,耗时分别为1.69 s、1.58 s、1.84 s和3.09 s,相比Mean-shift和Otsu算法,平均计算效率分别提升150%和3%,并且图像中的背景杂波、目标阴影和干扰小目标均被有效去除。结论 综合利用边界和纹理信息能够有效抑制相干噪声,去除图像斑点和小目标的干扰,从而达到目标边界的连接,实现对SAR图像目标的自动分割。实验结果表明,本文算法可以满足工程化应用要求,自适应性强,分割精度高,且具有较好的鲁棒性。     

Near-duplicate document image matching: A graphical perspective

A near-duplicate document image matching approach characterized by a graphical perspective is proposed in this paper. Document images are represented by graphs whose nodes correspond to the objects in the images. Consequently, the image matching problem is then converted to graph matching. To deal with the instability of object segmentation, a multi-granularity object tree is constructed for a document image. Each level in the tree corresponds to one possible object segmentation, while different levels are characterized by various object granularities. Some graphs can be generated from the tree and the objects associated with each graph may be of different granularities. Two graphs with the maximum similarity are found from the multi-granularity object trees of the two near-duplicate document images which are to be matched. The encouraging experimental results have demonstrated the effectiveness of the proposed approach.     

Image segmentation by iterative optimization of multiphase multiple piecewise constant model and Four-Color relabeling

In the paper an iteratively unsupervised image segmentation algorithm is developed, which is based on our proposed multiphase multiple piecewise constant (MMPC) model and its graph cuts optimization. The MMPC model use multiple constants to model each phase instead of one single constant used in Chan and Vese (CV) model and cartoon limit so that heterogeneous image object segmentation can be effectively dealt with. We show that the multiphase optimization problem based on our proposed model can be approximately solved by graph cuts methods. Four-Color theorem is used to relabel the regions of image after every iteration, which makes it possible to represent and segment an arbitrary number of regions in image with only four phases. Therefore, the computational cost and memory usage are greatly reduced. The comparison with some typical unsupervised image segmentation methods using a large number of images from the Berkeley Segmentation Dataset demonstrates the proposed algorithm can effectively segment natural images with a good performance and acceptable computational time.     

Interactively multiphase image segmentation based on variational formulation and graph cuts

In this paper the multiple piecewise constant (MPC) active contour model is extended to deal with multiphase case. This proposed multiphase model can be effectively optimized by solving the minimum cuts problem of a specially devised multilayer graph. Based on the proposed energy functional and its graph cuts optimization, an interactively multiphase partition method for image segmentation is presented. The user places some scribbles with different colors on the image according to the practical application demand and each group of scribbles with the same color corresponds to a potential image region. The distribution of each region can be learned from the input scribbles with some particular color. Then the corresponding multilayer graph can be constructed and its minimum cuts can be computed to determine the segmentation result of the image. Numerical experiments show that the proposed interactively multiphase segmentation method can accurately segment the image into different regions according to the input scribbles with different color.     

A novel unsupervised 3D skeleton detection in RGB-D images for video surveillance

In this paper we present a novel moment-based skeleton detection for representing human objects in RGB-D videos with animated 3D skeletons. An object often consists of several parts, where each of them can be concisely represented with a skeleton. However, it remains as a challenge to detect the skeletons of individual objects in an image since it requires an effective part detector and a part merging algorithm to group parts into objects. In this paper, we present a novel fully unsupervised learning framework to detect the skeletons of human objects in a RGB-D video. The skeleton modeling algorithm uses a pipeline architecture which consists of a series of cascaded operations, i.e., symmetry patch detection, linear time search of symmetry patch pairs, part and symmetry detection, symmetry graph partitioning, and object segmentation. The properties of geometric moment-based functions for embedding symmetry features into centers of symmetry patches are also investigated in detail. As compared with the state-of-the-art deep learning approaches for skeleton detection, the proposed approach does not require tedious human labeling work on training images to locate the skeleton pixels and their associated scale information. Although our algorithm can detect parts and objects simultaneously, a pre-learned convolution neural network (CNN) can be used to locate the human object from each frame of the input video RGB-D video in order to achieve the goal of constructing real-time applications. This much reduces the complexity to detect the skeleton structure of individual human objects with our proposed method. Using the segmented human object skeleton model, a video surveillance application is constructed to verify the effectiveness of the approach. Experimental results show that the proposed method gives good performance in terms of detection and recognition using publicly available datasets.

     

Machine vision scheme for stain-release evaluation using Gabor filters with optimized coefficients

This paper presents an efficient and practical approach for automatic, unsupervised object detection and segmentation in two-texture images based on the concept of Gabor filter optimization. The entire process occurs within a hierarchical framework and consists of the steps of detection, coarse segmentation, and fine segmentation. In the object detection step, the image is first processed using a Gabor filter bank. Then, the histograms of the filtered responses are analyzed using the scale-space approach to predict the presence/absence of an object in the target image. If the presence of an object is reported, the proposed approach proceeds to the coarse segmentation stage, wherein the best Gabor filter (among the bank of filters) is automatically chosen, and used to segment the image into two distinct regions. Finally, in the fine segmentation step, the coefficients of the best Gabor filter (output from the previous stage) are iteratively refined in order to further fine-tune and improve the segmentation map produced by the coarse segmentation step. In the validation study, the proposed approach is applied as part of a machine vision scheme with the goal of quantifying the stain-release property of fabrics. To that end, the presented hierarchical scheme is used to detect and segment stains on a sizeable set of digitized fabric images, and the performance evaluation of the detection, coarse segmentation, and fine segmentation steps is conducted using appropriate metrics. The promising nature of these results bears testimony to the efficacy of the proposed approach.     

A graph-based framework for sub-pixel image segmentation

Many image segmentation methods utilize graph structures for representing images, where the flexibility and generality of the abstract structure is beneficial. By using a fuzzy object representation, i.e., allowing partial belongingness of elements to image objects, the unavoidable loss of information when representing continuous structures by finite sets is significantly reduced, enabling feature estimates with sub-pixel precision.This work presents a framework for object representation based on fuzzy segmented graphs. Interpreting the edges as one-dimensional paths between the vertices of a graph, we extend the notion of a graph cut to that of a located cut, i.e., a cut with sub-edge precision. We describe a method for computing a located cut from a fuzzy segmentation of graph vertices. Further, the notion of vertex coverage segmentation is proposed as a graph theoretic equivalent to pixel coverage segmentations and a method for computing such a segmentation from a located cut is given. Utilizing the proposed framework, we demonstrate improved precision of area measurements of synthetic two-dimensional objects. We emphasize that although the experiments presented here are performed on two-dimensional images, the proposed framework is defined for general graphs and thus applicable to images of any dimension.