A graph-based framework for sub-pixel image segmentation

Many image segmentation methods utilize graph structures for representing images, where the flexibility and generality of the abstract structure is beneficial. By using a fuzzy object representation, i.e., allowing partial belongingness of elements to image objects, the unavoidable loss of information when representing continuous structures by finite sets is significantly reduced, enabling feature estimates with sub-pixel precision.This work presents a framework for object representation based on fuzzy segmented graphs. Interpreting the edges as one-dimensional paths between the vertices of a graph, we extend the notion of a graph cut to that of a located cut, i.e., a cut with sub-edge precision. We describe a method for computing a located cut from a fuzzy segmentation of graph vertices. Further, the notion of vertex coverage segmentation is proposed as a graph theoretic equivalent to pixel coverage segmentations and a method for computing such a segmentation from a located cut is given. Utilizing the proposed framework, we demonstrate improved precision of area measurements of synthetic two-dimensional objects. We emphasize that although the experiments presented here are performed on two-dimensional images, the proposed framework is defined for general graphs and thus applicable to images of any dimension.     

Content-based image retrieval based on a fuzzy approach

A typical content-based image retrieval (CBIR) system would need to handle the vagueness in the user queries as well as the inherent uncertainty in image representation, similarity measure, and relevance feedback. We discuss how fuzzy set theory can be effectively used for this purpose and describe an image retrieval system called FIRST (fuzzy image retrieval system) which incorporates many of these ideas. FIRST can handle exemplar-based, graphical-sketch-based, as well as linguistic queries involving region labels, attributes, and spatial relations. FIRST uses fuzzy attributed relational graphs (FARGs) to represent images, where each node in the graph represents an image region and each edge represents a relation between two regions. The given query is converted to a FARG, and a low-complexity fuzzy graph matching algorithm is used to compare the query graph with the FARGs in the database. The use of an indexing scheme based on a leader clustering algorithm avoids an exhaustive search of the FARG database. We quantify the retrieval performance of the system in terms of several standard measures.     

FReBIR: An image retrieval system based on fuzzy region matching

In this paper, we present a method of image indexing and retrieval which takes into account the relative positions of the regions within the image. Indexing is based on a segmentation of the image into fuzzy regions; we propose an algorithm which produces a fuzzy segmentation. The image retrieval is based on inexact graph matching, taking into account both the similarity between regions and the spatial relation between them. We propose, on one hand a solution to reduce the combinatorial complexity of the graph matching, and on the other hand, a measure of similarity between graphs allowing the result images ranking. A relevance feedback process based on region classifiers allows then a good generalization to a large variety of the regions. The method is adapted to partial queries, aiming for example at retrieving images containing a specific type of object. Applications may be of two types, firstly an on-line search from a partial query, with a relevance feedback aiming at interactively leading the search, and secondly an off-line learning of categories from a set of examples of the object. The name of the system is FReBIR for Fuzzy Region-Based Image Retrieval.     

Utilizing similarity relationships among existing data for high accuracy processing of content-based image retrieval

Retrieving similar images based on its visual content is an important yet difficult problem. We propose in this paper a new method to improve the accuracy of content-based image retrieval systems. Typically, given a query image, existing retrieval methods return a ranked list based on the similarity scores between the query and individual images in the database. Our method goes further by relying on an analysis of the underlying connections among individual images in the database to improve this list. Initially, we consider each image in the database as a query and use an existing baseline method to search for its likely similar images. Then, the database is modeled as a graph where images are nodes and connections among possibly similar images are edges. Next, we introduce an algorithm to split this graph into stronger subgraphs, based on our notion of graph’s strength, so that images in each subgraph are expected to be truly similar to each other. We create for each subgraph a structure called integrated image which contains the visual features of all images in the subgraph. At query time, we compute the similarity scores not only between the query and individual database images but also between the query and the integrated images. The final similarity score of a database image is computed based on both its individual score and the score of the integrated image that it belongs to. This leads effectively to a re-ranking of the retrieved images. We evaluate our method on a common image retrieval benchmark and demonstrate a significant improvement over the traditional bag-of-words retrieval model.     

Information Theoretic Deformable Registration Using Local Image Information

We present a deformable registration algorithm for multi-modality images based on information theoretic similarity measures at the scale of individual image voxels. We derive analytical expressions for the mutual information, the joint entropy, and the sum of marginal entropies of two images over a small neighborhood in terms of image gradients. Using these expressions, we formulate image registration algorithms maximizing local similarity over the whole image domain in an energy minimization framework. This strategy produces highly elastic image alignment as the registration is driven by voxel similarities between the images, the algorithms are easily implementable using the closed-form expressions for the derivative of the optimization function with respect to the deformation, and avoid estimation of joint and marginal probability densities governing the image intensities essential to conventional information theoretic image registration methods. This work has been supported in part by NIH grants R01-NS42645 and R01-AG14971.     

Segmentation of color images using multiscale clustering and graph theoretic region synthesis

A multiresolution color image segmentation approach is presented that incorporates the main principles of region-based segmentation and cluster-analysis approaches. The contribution of This work may be divided into two parts. In the first part, a multiscale dissimilarity measure is proposed that makes use of a feature transformation operation to measure the interregion relations with respect to their proximity to the main clusters of the image. As a part of this process, an original approach is also presented to generate a multiscale representation of the image information using nonparametric clustering. In the second part, a graph theoretic algorithm is proposed to synthesize regions and produce the final segmentation results. The latter algorithm emerged from a brief analysis of fuzzy similarity relations in the context of clustering algorithms. This analysis indicates that the segmentation methods in general may be formulated sufficiently and concisely by means of similarity relations theory. The proposed scheme produces satisfying results and its efficiency is indicated by comparing it with: 1) the single scale version of dissimilarity measure and 2) several earlier graph theoretic merging approaches proposed in the literature. Finally, the multiscale processing and region-synthesis properties validate our method for applications, such as object recognition, image retrieval, and emulation of human visual perception.     

A mutual information based face clustering algorithm for movie content analysis

This paper investigates facial image clustering, primarily for movie video content analysis with respect to actor appearance. Our aim is to use novel formulation of the mutual information as a facial image similarity criterion and, by using spectral graph analysis, to cluster a similarity matrix containing the mutual information of facial images. To this end, we use the HSV color space of a facial image (more precisely, only the hue and saturation channels) in order to calculate the mutual information similarity matrix of a set of facial images. We make full use of the similarity matrix symmetries, so as to lower the computational complexity of the new mutual information calculation. We assign each row of this matrix as feature vector describing a facial image for producing a global similarity criterion for face clustering. In order to test our proposed method, we conducted two sets of experiments that have produced clustering accuracy of more than 80%. We also compared our algorithm with other clustering approaches, such as the k-means and fuzzy c-means (FCM) algorithms. Finally, in order to provide a baseline comparison for our approach, we compared the proposed global similarity measure with another one recently reported in the literature.     

Investigations on adaptive connectivity and shape prior based fuzzy graph-cut colour image segmentation

Image segmentation is a challenging problem in computer vision with wide application. It is a process which considers the similarity criterion required to separate an image into different homogenous connected regions. First, an Optimized Adaptive Connectivity and Shape Prior in Modified Graph Cut Segmentation method has been applied to handle the structural irregularities in images. Second, an Optimized Adaptive Connectivity and Shape Prior in Modified Fuzzy Graph Cut Segmentation (Opac-MFGseg) is proposed to partition the images based on feature values. In this method, a fuzzy rule based system is used with optimization algorithm to provide the information on how much a specific feature is involved in image boundaries. The graph obtained from this fuzzy approach is further used in adaptive shape prior in modified graph cuts framework. Moreover, this method supports moving images (videos). In such a situation, a fully dynamic method called Optimized Adaptive Connectivity and Shape Prior in Dynamic Fuzzy Graph Cut Segmentation (Opac-DFGseg) method is proposed for the image segmentation. The effectiveness of the Opac-MFGseg and Opac-DFGseg methods is tested in terms of average sensitivity, precision, area overlap measure, relative error, and accuracy and computation time.     

Fuzzy spectral clustering with robust spatial information for image segmentation

In recent years, spectral clustering has become one of the most popular clustering algorithms in areas of pattern analysis and recognition. This algorithm uses the eigenvalues and eigenvectors of a normalized similarity matrix to partition the data, and is simple to implement. However, when the image is corrupted by noise, spectral clustering cannot obtain satisfying segmentation performance. In order to overcome the noise sensitivity of the standard spectral clustering algorithm, a novel fuzzy spectral clustering algorithm with robust spatial information for image segmentation (FSC_RS) is proposed in this paper. Firstly, a non-local-weighted sum image of the original image is generated by utilizing the pixels with a similar configuration of each pixel. Then a robust gray-based fuzzy similarity measure is defined by using the fuzzy membership values among gray values in the new generated image. Thus, the similarity matrix obtained by this measure is only dependent on the number of the gray-levels and can be easily stored. Finally, the spectral graph partitioning method can be applied to this similarity matrix to group the gray values of the new generated image and then the corresponding pixels in the image are reclassified to obtain the final segmentation result. Some segmentation experiments on synthetic and real images show that the proposed method outperforms traditional spectral clustering methods and spatial fuzzy clustering in efficiency and robustness.     

Leveraging community metadata for multimodal image?ranking

Searching for relevant images given a query term is an important task in nowadays large-scale community databases. The image ranking approach presented in this work represents an image collection as a graph that is built using a multimodal similarity measure based on visual features and user tags. We perform a random walk on this graph to find the most common images. Further we discuss several scalability issues of the proposed approach and show how in this framework queries can be answered fast. Experimental results validate the effectiveness of the presented algorithm.     

A region-based fuzzy feature matching approach to content-basedimage retrieval

This paper proposes a fuzzy logic approach, UFM (unified feature matching), for region-based image retrieval. In our retrieval system, an image is represented by a set of segmented regions, each of which is characterized by a fuzzy feature (fuzzy set) reflecting color, texture, and shape properties. As a result, an image is associated with a family of fuzzy features corresponding to regions. Fuzzy features naturally characterize the gradual transition between regions (blurry boundaries) within an image and incorporate the segmentation-related uncertainties into the retrieval algorithm. The resemblance of two images is then defined as the overall similarity between two families of fuzzy features and quantified by a similarity measure, UFM measure, which integrates properties of all the regions in the images. Compared with similarity measures based on individual regions and on all regions with crisp-valued feature representations, the UFM measure greatly reduces the influence of inaccurate segmentation and provides a very intuitive quantification. The UFM has been implemented as a part of our experimental SIMPLIcity image retrieval system. The performance of the system is illustrated using examples from an image database of about 60,000 general-purpose images     

模糊元图:一种构造模糊知识库的新方法

在分析了现有模糊图论结构的基础上,对模糊超图和模糊有向图进行扩充,提出一种新的图论结构－模糊元图,并将其应用于飞行器邦联诊断中的模糊知识库构造,实际应用表明,基于模糊元图的知识库具有很高的推理效率,并且便于规则的添加和删除。     

No-reference image quality assessment using interval type 2 fuzzy sets

Image quality assessment of distorted or decompressed images without any reference to the original image is challenging from computational point of view. Quality of an image is best judged by human observers without any reference image, and evaluated using subjective measures. The paper aims at designing a generic no-reference image quality assessment (NR-IQA) method by incorporating human visual perception in assigning quality class labels to the images. Using fuzzy logic approach, we consider information theoretic entropies of visually salient regions of images as features and assess quality of the images using linguistic values. The features are transformed into fuzzy feature space by designing an algorithm based on interval type-2 (IT2) fuzzy sets. The algorithm measures uncertainty present in the input–output feature space to predict image quality accurately as close to human observations. We have taken a set of training images belonging to five different pre-assigned quality class labels for calculating foot print of uncertainty (FOU) corresponding to each class. To assess the quality class label of the test images, maximum of T-conorm applied on the lower and upper membership functions of the test images belonging to different classes is calculated. Our proposed image quality metric is compared with other no-reference quality metrics demonstrating more accurate results and compatible with subjective mean opinion score metric.     

Handling of impreciseness in gray level corner detection using fuzzy set theoretic approach

Reliable corner detection is an important task in determining shape of different regions in an image. To detect corners in a gray level image under imprecise information, an algorithm based on fuzzy set theoretic model is proposed. The uncertainties arising due to various types of imaging defects such as blurring, illumination change, noise, etc., usually result in missing of significant curvature junctions (corners). Fuzzy set theory based modeling is well known for efficient handling of impreciseness. In order to handle the incompleteness arising due to imperfection of data, it is reasonable to model image properties in fuzzy frame work for reliable decision making. The robustness of the proposed algorithm is compared with well known conventional detectors. The performance is tested on a number of benchmark test images to illustrate the efficiency of the algorithm.     

Integration of IRS-1A L2 data by fuzzy logic approaches for landuse classification

A methodology has been formulated to integrate images from IRS-1A LISS II of two dates for landuse/landcover classification. The methodology developed includes image classification by fuzzy k-means clustering and fusion of memberships by fuzzy set theoretic operators. The two date images have been geometrically coregistered and classified for the identification of land classes individually. The fuzzy memberships of the classified output images have been integrated by using fuzzy logic operators like algebraic sum and gamma (gamma) operator. The classification accuracy of the resultant land classes in the integrated images was verified with the ground data collected in situ. The resultant images have been evaluated by kappa (kappa) statistic and it was found that output from the image of fuzzy algebraic sum operator scored high in generating the land classes, with an overall accuracy of 95%.     

Graph-based representation for similarity retrieval of symbolic images

Image retrieval from an image database by the image objects and their spatial relationships has emerged as an important research subject in these decades. To retrieve images similar to a given query image, retrieval methods must assess the similarity degree between a database image and the query image by the extracted features with acceptable efficiency and effectiveness. This paper proposes a graph-based model SRG (spatial relation graph) to represent the semantic information of the contained objects and their spatial relationships in an image with no file annotation. In an SRG graph, the image objects are symbolized by the predefined class names as vertices and the spatial relations between object pairs are represented as arcs. The proposed model assesses the similarity degree between two images by calculating the maximum common subgraph of two corresponding SRG’s through intersection, which has quadratic time complexity owing to the characteristics of SRG. Its efficiency remains quadratic regardless of the duplication rate of the object symbols. The extended model SRG_T is also proposed, with the same time complexity, for the applications that need to consider the topological relations among objects. A synthetic symbolic image database and an existing image dataset are used in the conducted experiments to verify the performance of the proposed models. The experimental results show that the proposed models have compatible retrieval quality with remarkable efficiency improvements compared with three well-known methods LCS_Clique, SIM_R, and 2D Be-string, where LCS_Clique utilizes the number of objects in the maximum common subimage as its similarity function, SIM_R uses accumulation-based similarity function of similar object pairs, and 2D Be-string calculates the similarity of 2D patterns by the linear combination of two 1D similarities.     

Normalized cuts and image segmentation

We propose a novel approach for solving the perceptual grouping problem in vision. Rather than focusing on local features and their consistencies in the image data, our approach aims at extracting the global impression of an image. We treat image segmentation as a graph partitioning problem and propose a novel global criterion, the normalized cut, for segmenting the graph. The normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups. We show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion. We applied this approach to segmenting static images, as well as motion sequences, and found the results to be very encouraging     

一种改进的基于结构相似度的图像质量评价方法

基于结构相似度（SSIM）的图像质量评价方法简单高效,准确性较高,评价性能优于峰值信噪比（PNSR）和均方误差（ MSE）,但SSIM模型不能较好地评价严重失真和交叉失真类型的图像。文中提出了一种改进的基于结构相似度的图像质量评价方法（ HSSIM）,该方法将直方图信息作为图像的主要结构信息,根据人眼视觉特性,利用直方图集中度来表示图像模糊度,最终计算得到图像的结构相似度值。实验结果表明,HSSIM比SSIM模型更符合人眼视觉系统特性,能更好地评价失真图像的质量。     

搜索引擎中基于内容的图像重排序

针对基于文本的图像搜索结果的排序不能很好满足用户查询期望的问题,提出两种基于内容的图像搜索结果重排序方法:基于相似性积分的重排序算法(SI算法)和基于Dijkstra算法的重排序算法(D算法)。这两种方法把图像作为节点,利用图像的颜色和形状特征计算图像间的相似性,并将相似性作为边的权重构建相似性图,SI算法根据每个节点图像相似性积分的大小来进行排序,D算法则利用Dijkstra算法从指定图像开始对所有图像节点进行遍历排序。实验结果表明,两种方法都能很好地改善图像搜索排序结果,其中SI算法适合使用在初始查准率在0.5~0.9的情况;而D算法不要求初始查准率,但对图像间相似性值的精确度要求高,可以用于用户指定一张查询相关图像的重排序。     

应用于材料图像分割的Graph-UNet

小样本材料图像分割是图像分割领域的研究难点之一。材料图像的微观结构大多数形状各异、纹理复杂且边界模糊,会导致材料图像的分割不准确。Graph-UNet被提出融合U-Net和图卷积神经网络来解决小样本材料图像自动分割的挑战,它将卷积神经网络的多维特征融合和跳跃连接的思想迁移到图卷积神经网络中实现图卷积和图注意力的有效结合,并且建立了一个通用的模块实现特征图和图结构相互转换。在材料图像数据集上进行了对比和消融实验,证明了Graph-UNet的分割结果优于很多先进方法,准确地识别了多种材料结构,推动了探究材料结构和性能关系的发展。