Constructing a discriminative visual vocabulary with macro and micro sense of visual words

Visual vocabulary representation approach has been successfully applied to many multimedia and vision applications, including visual recognition, image retrieval, and scene modeling/categorization. The idea behind the visual vocabulary representation is that an image can be represented by visual words, a collection of local features of images. In this work, we will develop a new scheme for the construction of visual vocabulary based on the analysis of visual word contents. By considering the content homogeneity of visual words, we design a visual vocabulary which contains macro-sense and micro-sense visual words. The two types of visual words are appropriately further combined to describe an image effectively. We also apply the visual vocabulary to construct image retrieving and categorization systems. The performance evaluation for the two systems indicates that the proposed visual vocabulary achieves promising results.     

Tree-structured image difference for fast histogram and distance between histograms computation

In this paper we present a new method for fast histogram computing and its extension to bin to bin histogram distance computing. The idea consists in using the information of spatial differences between images, or between regions of images (a current one and a reference one), and encoding it into a specific data structure: a tree. The histogram of the current image or of one of its regions is then computed by updating the histogram of the reference one using the temporal data stocked into the tree. With this approach, we never need to store any of the current histograms, except the reference image ones, as a preprocessing step. We compare our approach with the well-known Integral Histogram one, and obtain better results in terms of processing time while reducing the memory footprint. We show theoretically and with experimental results the superiority of our approach in many cases. We also extend our idea to the computation of the Bhattacharyya distance between two histograms, using a similar incremental approach that also avoid current histogram computations: we just need histograms of the reference image, and spatial differences between the reference and the current image to compute this distance using an updating process. Finally, we demonstrate the advantages of our approach on a real visual tracking application using a particle filter framework by improving its correction step computation time.     

An optimized approach to histogram computation on GPU

A histogram is a compact representation of the distribution of data in an image with a full range of applications in diverse fields. Histogram generation is an inherently sequential operation where every pixel votes in a reduced set of bins. This makes finding efficient parallel implementations very desirable but challenging, because on graphics processing units thousands of threads may be atomically updating a short number of histogram bins. Under these circumstances, collisions among threads will be very frequent and such collisions will serialize thread execution, seriously damaging the performance. In this paper we propose a highly optimized approach to histogram calculation, which tackles such performance bottlenecks. It uses histogram replication for eliminating position conflicts, padding to reduce bank conflicts, and an improved access to input data called interleaved read access. Our so-called ${\mathcal{R}}$ -per-block approach to histogram calculation has been successfully compared to the main state-of-the-art works using four histogram-based image processing kernels and two real image databases. Results show that our proposal is between 1.4 and 15.7 faster than every previous implementation for histograms of up to 4,096 bins.     

Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine

In recent years, the rapid growth of multimedia content makes content-based image retrieval (CBIR) a challenging research problem. The content-based attributes of the image are associated with the position of objects and regions within the image. The addition of image content-based attributes to image retrieval enhances its performance. In the last few years, the bag-of-visual-words (BoVW) based image representation model gained attention and significantly improved the efficiency and effectiveness of CBIR. In BoVW-based image representation model, an image is represented as an order-less histogram of visual words by ignoring the spatial attributes. In this paper, we present a novel image representation based on the weighted average of triangular histograms (WATH) of visual words. The proposed approach adds the image spatial contents to the inverted index of the BoVW model, reduces overfitting problem on larger sizes of the dictionary and semantic gap issues between high-level image semantic and low-level image features. The qualitative and quantitative analysis conducted on three image benchmarks demonstrates the effectiveness of the proposed approach based on WATH.     

Multiresolution similarity search in image databases

Typically searching image collections is based on features of the images. In most cases the features are based on the color histogram of the images. Similarity search based on color histograms is very efficient, but the quality of the search results is often rather poor. One of the reasons is that histogram-based systems only support a specific form of global similarity using the whole histogram as one vector. But there is more information in a histogram than the distribution of colors. This paper has two contributions: (1) a new generalized similarity search method based on a wavelet transformation of the color histograms and (2) a new effectiveness measure for image similarity search. Our generalized similarity search method has been developed to allow the user to search for images with similarities on arbitrary detail levels of the color histogram. We show that our new approach is more general and more effective than previous approaches while retaining a competitive performance.     

Image contrast enhancement for intelligent surveillance systems using multi-local histogram transformation

Among all applications to monitor the safety and security of working environments, surveillance systems that use computer vision are the most efficient and intuitive in the manufacturing industry. This paper introduces a new technique of contrast enhancement for surveillance systems using computer vision. The histogram equalization method is a common and widespread image enhancement method which maximizes the contrast of the image. This contrast enhancement method usually improves the quality of images, but it can suffer from visual deterioration caused by excessive histogram modification. To overcome the limitations of conventional contrast enhancement methods, this paper introduces a new multi-local histogram transformation method for surveillance systems. This technique is based on the local histograms, which are separated from the overall histogram of the image, and the contrast of the image can be enhanced through two major processes: range reassignment of local histograms and local histogram equalization. The multi-local histogram transformation in this paper enhances the contrast of images, preventing excessive compression and extension of image histograms. The performance of the suggested contrast enhancement method is verified by the experiments in four different environments.     

基于视觉权值的分块颜色直方图图像检索算法

为了进一步提高基于颜色直方图的图像检索系统工作效率, 本文以人眼视觉感知特性为基础, 结合局部图像相关性, 提出了一种基于视觉权值的分块颜色直方图图像检索新方法. 该方法首先对图像进行分块处理, 并计算出图像子块的颜色直方图; 然后结合反映局部区域变化的像素点颜色复杂度, 计算出图像子块的视觉权值; 最后利用视觉权值对子块颜色直方图进行加权处理, 并根据加权颜色直方图进行图像检索. 仿真实验结果表明, 本文算法能够准确和高效地查找出用户所需内容的彩色图像, 并且具有较好的查准率和查全率.     

Multiresolution histograms and their use for recognition

The histogram of image intensities is used extensively for recognition and for retrieval of images and video from visual databases. A single image histogram, however, suffers from the inability to encode spatial image variation. An obvious way to extend this feature is to compute the histograms of multiple resolutions of an image to form a multiresolution histogram. The multiresolution histogram shares many desirable properties with the plain histogram including that they are both fast to compute, space efficient, invariant to rigid motions, and robust to noise. In addition, the multiresolution histogram directly encodes spatial information. We describe a simple yet novel matching algorithm based on the multiresolution histogram that uses the differences between histograms of consecutive image resolutions. We evaluate it against five widely used image features. We show that with our simple feature we achieve or exceed the performance obtained with more complicated features. Further, we show our algorithm to be the most efficient and robust.     

Automatic image annotation using visual content and folksonomies

Automatic image annotation is an important and challenging task, and becomes increasingly necessary when managing large image collections. This paper describes techniques for automatic image annotation that take advantage of collaboratively annotated image databases, so called visual folksonomies. Our approach applies two techniques based on image analysis: First, classification annotates images with a controlled vocabulary and second tag propagation along visually similar images. The latter propagates user generated, folksonomic annotations and is therefore capable of dealing with an unlimited vocabulary. Experiments with a pool of Flickr images demonstrate the high accuracy and efficiency of the proposed methods in the task of automatic image annotation. Both techniques were applied in the prototypical tag recommender “tagr”.     

A 2-D wavelet decomposition-based bag-of-visual-words model for land-use scene classification

Previous works about spatial information incorporation into a traditional bag-of-visual-words (BOVW) model mainly consider the spatial arrangement of an image, ignoring the rich textural information in land-use remote-sensing images. Hence, this article presents a 2-D wavelet decomposition (WD)-based BOVW model for land-use scene classification, since the 2-D wavelet decomposition method does well not only in textural feature extraction, but also in the multi-resolution representation of an image, which is favourable for the use of both spatial arrangement and textural information in land-use images. The proposed method exploits the textural structures of an image with colour information transformed into greyscale. Moreover, it works first by decomposing the greyscale image into different sub-images using 2-D discrete wavelet transform (DWT) and then by extracting local features of the greyscale image and all the decomposed images with dense regions in which a given image is evenly sampled by a regular grid with a specified grid space. After that, the method generates the corresponding visual vocabularies and computes histograms of visual word occurrences of local features found in each former image. Specifically, the soft-assignment or multi-assignment (MA) technique is employed, accounting for the impact of clustering on visual vocabulary creation that two similar image patches may be clustered into different clusters when increasing the size of visual vocabulary. The proposed method is evaluated on a ground truth image dataset of 21 land-use classes manually extracted from high-resolution remote-sensing images. Experimental results demonstrate that the proposed method significantly outperforms previous methods, such as the traditional BOVW model, the spatial pyramid representation-based BOVW method, the multi-resolution representation-based BOVW method, and so on, and even exceeds the best result obtained from the creator of the land-use dataset. Therefore, the proposed approach is very suitable for land-use scene classification tasks.     

一种基于颜色聚合向量的图像检索方法

颜色直方图被广泛地应用在基于内容的图像检索中，其优点是效率高，对于视点细微的变化不敏感。然而直方图只是图像的一个粗糙特征，对于视觉上完全不相似的图像，其直方图有可能非常相似。该文在颜色直方图的基础上进行了改进，采用一种基于颜色聚合向量的图像检索方法，极大地提高了检索精度。     

Hierarchical Classifiers for Robust Topological Robot Localization

This paper presents a novel appearance-based technique for topological robot localization and place recognition. A vocabulary of visual words is formed automatically, representing local features that frequently occur in the set of training images. Using the vocabulary, a spatial pyramid representation is built for each image by repeatedly subdividing it and computing histograms of visual words at increasingly fine resolutions. An information maximization technique is then applied to build a hierarchical classifier for each class by learning informative features. While top-level features in the hierarchy are selected from the coarsest resolution of the representation, capturing the holistic statistical properties of the images, child features are selected from finer resolutions, encoding more local characteristics, redundant with the information coded by their parents. Exploiting the redundancy in the data enables the localization system to achieve greater reliability against dynamic variations in the environment. Achieving an average classification accuracy of 88.9% on a challenging topological localization database, consisting of twenty seven outdoor places, demonstrates the advantages of our hierarchical framework for dealing with dynamic variations that cannot be learned during training.     

Image indexing and retrieval based on color histograms

While general object recognition is difficult, it is relatively easy to capture various primitive properties such as color distributions, prominent regions and their topological features from an image and use them to narrow down the search space when attempts to retrieving images by contents from an image database are made.In this paper, we present an image database in which images are indexed and retrieved based on color histograms. We first address the problems inherent in color histograms created by the conventional method, and then propose a new method to create histograms which are compact in size and insensitive to minor illumination variations such as highlight, shape, and etc. A powerful indexing scheme where each histogram of an image is encoded into a numerical key, and stored in a two-layered tree structure is introduced. This approach turns the problem of histogram matching, which is computation intensive, into index key search, so as to realize quick data access in a large scale image database. Two types of user interfaces, Query by user provided sample images, and Query by combination of the system provided templates, are provided to meet various user requests. Various experimental evaluations exhibit the effectiveness of the image database system.     

融合颜色词袋特征的视觉词汇树图像检索

针对由图像灰度空间产生的传统词袋模型SIFT特征无法体现图像的颜色信息的问题,提出了一种融合颜色特征的视觉词汇树来对图像进行描述。提取SIFT特征并建立词汇树,获取图像的SIFT表示向量。利用K-means方法对图像库中的所有图像的HSV值进行聚类,获得基于HSV空间的颜色词袋表示向量,避免了传统颜色直方图方法所带来的量化误差。将SIFT特征与颜色词袋特征进行融合,完成了图像的全局特征和局部特征的融合。然后,计算融合特征的相似度,将相似度从高到低排序,完成图像检索。为了验证本方法的有效性,选择Corel图像库对算法性能进行实验分析,从主观评价和客观评价标准分别进行评价,并与传统方法进行了对比。结果表明,特征融合的检索性能与单一特征方法相比有较大提高。特征融合方法的平均检索查准率和查全率-查准率等评价指标,对比传统方法均有不同程度提高。     

Comparing images using joint histograms

Color histograms are widely used for content-based image retrieval due to their efficiency and robustness. However, a color histogram only records an image's overall color composition, so images with very different appearances can have similar color histograms. This problem is especially critical in large image databases, where many images have similar color histograms. In this paper, we propose an alternative to color histograms called a joint histogram, which incorporates additional information without sacrificing the robustness of color histograms. We create a joint histogram by selecting a set of local pixel features and constructing a multidimensional histogram. Each entry in a joint histogram contains the number of pixels in the image that are described by a particular combination of feature values. We describe a number of different joint histograms, and evaluate their performance for image retrieval on a database with over 210,000 images. On our benchmarks, joint histograms outperform color histograms by an order of magnitude.     

图像场景分类中视觉词包模型方法综述

目的关于图像场景分类中视觉词包模型方法的综述性文章在国内外杂志上还少有报导,为了使国内外同行对图像场景分类中的视觉词包模型方法有一个较为全面的了解,对这些研究工作进行了系统总结。方法在参考国内外大量文献的基础上,对现有图像场景分类(主要指针对单一图像场景的分类)中出现的各种视觉词包模型方法从低层特征的选择与局部图像块特征的生成、视觉词典的构建、视觉词包特征的直方图表示、视觉单词优化等多方面加以总结和比较。结果回顾了视觉词包模型的发展历程,对目前存在的多种视觉词包模型进行了归纳,比较常见方法各自的优缺点,总结了视觉词包模型性能评价方法,并对目前常用的标准场景库进行汇总,同时给出了各自所达到的最高精度。结论图像场景分类中视觉词包模型方法的研究作为计算机视觉领域方兴未艾的热点研究领域,在国内外研究中取得了不少进展,在计算机视觉领域的研究也不再局限于直接应用模型描述图像内容,而是更多地考虑图像与文本的差异。虽然视觉词包模型在图像场景分类的应用中还存在很多亟需解决的问题,但是这丝毫不能掩盖其研究的重要意义。