Automatic content based image retrieval using semantic analysis

We present a new text-to-image re-ranking approach for improving the relevancy rate in searches. In particular, we focus on the fundamental semantic gap that exists between the low-level visual features of the image and high-level textual queries by dynamically maintaining a connected hierarchy in the form of a concept database. For each textual query, we take the results from popular search engines as an initial retrieval, followed by a semantic analysis to map the textual query to higher level concepts. In order to do this, we design a two-layer scoring system which can identify the relationship between the query and the concepts automatically. We then calculate the image feature vectors and compare them with the classifier for each related concept. An image is relevant only when it is related to the query both semantically and content-wise. The second feature of this work is that we loosen the requirement for query accuracy from the user, which makes it possible to perform well on users’ queries containing less relevant information. Thirdly, the concept database can be dynamically maintained to satisfy the variations in user queries, which eliminates the need for human labor in building a sophisticated initial concept database. We designed our experiment using complex queries (based on five scenarios) to demonstrate how our retrieval results are a significant improvement over those obtained from current state-of-the-art image search engines.     

A semantic modeling approach for image retrieval by content

We introduce a semantic data model to capture the hierarchical, spatial, temporal, and evolutionary semantics of images in pictorial databases. This model mimics the user's conceptual view of the image content, providing the framework and guidelines for preprocessing to extract image features. Based on the model constructs, a spatial evolutionary query language (SEQL), which provides direct image object manipulation capabilities, is presented. With semantic information captured in the model, spatial evolutionary queries are answered efficiently. Using an object-oriented platform, a prototype medical-image management system was implemented at UCLA to demonstrate the feasibility of the proposed approach.     

Edge based features for content based image retrieval

The common problem in content based image retrieval (CBIR) is selection of features. Image characterization with lesser number of features involving lower computational cost is always desirable. Edge is a strong feature for characterizing an image. This paper presents a robust technique for extracting edge map of an image which is followed by computation of global feature (like fuzzy compactness) using gray level as well as shape information of the edge map. Unlike other existing techniques it does not require pre segmentation for the computation of features. This algorithm is also computationally attractive as it computes different features with limited number of selected pixels.     

基于语义关联技术的信息检索策略

为提高在海量数据中的信息检索效率,分析了元数据目录服务技术的现状及其不足,通过引入语义关联技术,将语义关联的信息检索技术与网格信息服务技术有效结合,提出了一种改进的能有效提高检索效率的检索框架,并结合实例进行了描述.实验结果表明,该优化的语义关联查询方法是有效、可行的,能有效提高海量数据中信息检索的查准率和查全率.     

Sparse semantic metric learning for image retrieval

Typical content-based image retrieval solutions usually cannot achieve satisfactory performance due to the semantic gap challenge. With the popularity of social media applications, large amounts of social images associated with user tagging information are available, which can be leveraged to boost image retrieval. In this paper, we propose a sparse semantic metric learning (SSML) algorithm by discovering knowledge from these social media resources, and apply the learned metric to search relevant images for users. Different from the traditional metric learning approaches that use similar or dissimilar constraints over a homogeneous visual space, the proposed method exploits heterogeneous information from two views of images and formulates the learning problem with the following principles. The semantic structure in the text space is expected to be preserved for the transformed space. To prevent overfitting the noisy, incomplete, or subjective tagging information of images, we expect that the mapping space by the learned metric does not deviate from the original visual space. In addition, the metric is straightforward constrained to be row-wise sparse with the ?_2,1-norm to suppress certain noisy or redundant visual feature dimensions. We present an iterative algorithm with proved convergence to solve the optimization problem. With the learned metric for image retrieval, we conduct extensive experiments on a real-world dataset and validate the effectiveness of our approach compared with other related work.     

An endmember-based distance for content based hyperspectral image retrieval

We propose a specific content-based image retrieval (CBIR) system for hyperspectral images exploiting its rich spectral information. The CBIR image features are the endmember signatures obtained from the image data by endmember induction algorithms (EIAs). Endmembers correspond to the elementary materials in the scene, so that the pixel spectra can be decomposed into a linear combination of endmember signatures. EIA search for points in the high dimensional space of pixel spectra defining a convex polytope, often a simplex, covering the image data. This paper introduces a dissimilarity measure between hyperspectral images computed over the image induced endmembers, proving that it complies with the axioms of a distance. We provide a comparative discussion of dissimilarity functions, and quantitative evaluation of their relative performances on a large collection of synthetic hyperspectral images, and on a dataset extracted from a real hyperspectral image. Alternative dissimilarity functions considered are the Hausdorff distance and robust variations of it. We assess the CBIR performance sensitivity to changes in the distance between endmembers, the EIA employed, and some other conditions. The proposed hyperspectral image distance improves over the alternative dissimilarities in all quantitative performance measures. The visual results of the CBIR on the real image data demonstrate its usefulness for practical applications.     

An effective technique for the content based image retrieval to reduce the semantic gap based on an optimal classifier technique

Content Based Image Retrieval (CBIR) systems use Relevance Feedback (RF) in order to improve the retrieval accuracy. Research focus has been shifted from designing sophisticated low-level feature extraction algorithms to reducing the “semantic gap” between the visual features and the richness of human semantics. In this paper, a novel system is proposed to enhance the gain of long-term relevance feedback. In the proposed system, the general CBIR involves two steps—ABC based training and image retrieval. First, the images other than the query image are pre-processed using median filter and gray scale transformation for removal of noise and resizing. Secondly, the features such as Color, Texture and shape of the image are extracted using Gabor Filter, Gray Level Co-occurrence Matrix and Hu-Moment shape feature techniques and also extract the static features like mean and standard deviation. The extracted features are clustered using k-means algorithm and each cluster are trained using ANN based ABC technique. A method using artificial bee colony (ABC) based artificial neural network (ANN) to update the weights assigned to features by accumulating the knowledge obtained from the user over iterations. Eventually, the comparative analysis performed using the commonly used methods namely precision and recall were clearly shown that the proposed system is suitable for the better CBIR and it can reduce the semantic gap than the conventional systems.     

A scalable approach for content based image retrieval in cloud datacenter

The emergence of cloud datacenters enhances the capability of online data storage. Since massive data is stored in datacenters, it is necessary to effectively locate and access interest data in such a distributed system. However, traditional search techniques only allow users to search images over exact-match keywords through a centralized index. These techniques cannot satisfy the requirements of content based image retrieval (CBIR). In this paper, we propose a scalable image retrieval framework which can efficiently support content similarity search and semantic search in the distributed environment. Its key idea is to integrate image feature vectors into distributed hash tables (DHTs) by exploiting the property of locality sensitive hashing (LSH). Thus, images with similar content are most likely gathered into the same node without the knowledge of any global information. For searching semantically close images, the relevance feedback is adopted in our system to overcome the gap between low-level features and high-level features. We show that our approach yields high recall rate with good load balance and only requires a few number of hops.     

Learning effective color features for content based image retrieval in dermatology

We investigate the extraction of effective color features for a content-based image retrieval (CBIR) application in dermatology. Effectiveness is measured by the rate of correct retrieval of images from four color classes of skin lesions. We employ and compare two different methods to learn favorable feature representations for this special application: limited rank matrix learning vector quantization (LiRaM LVQ) and a Large Margin Nearest Neighbor (LMNN) approach. Both methods use labeled training data and provide a discriminant linear transformation of the original features, potentially to a lower dimensional space. The extracted color features are used to retrieve images from a database by a k-nearest neighbor search. We perform a comparison of retrieval rates achieved with extracted and original features for eight different standard color spaces. We achieved significant improvements in every examined color space. The increase of the mean correct retrieval rate lies between 10% and 27% in the range of k=1-25 retrieved images, and the correct retrieval rate lies between 84% and 64%. We present explicit combinations of RGB and CIE-Lab color features corresponding to healthy and lesion skin. LiRaM LVQ and the computationally more expensive LMNN give comparable results for large values of the method parameter κ of LMNN (κ≥25) while LiRaM LVQ outperforms LMNN for smaller values of κ. We conclude that feature extraction by LiRaM LVQ leads to considerable improvement in color-based retrieval of dermatologic images.     

Hierarchical deep semantic hashing for fast image retrieval

Multimedia Tools and Applications - Content-Based large-scale image retrieval has recently attracted considerable attention because of the explosive increase of online images. Inspired by recent...     

A survey of browsing models for content based image retrieval

The problem of content based image retrieval (CBIR) has traditionally been investigated within a framework that emphasises the explicit formulation of a query: users initiate an automated search for relevant images by submitting an image or draw a sketch that exemplifies their information need. Often, relevance feedback is incorporated as a post-retrieval step for optimising the way evidence from different visual features is combined. While this sustained methodological focus has helped CBIR to mature, it has also brought out its limitations more clearly: There is often little support for exploratory search and scaling to very large collections is problematic. Moreover, the assumption that users are always able to formulate an appropriate query is questionable. An effective, albeit much less studied, method of accessing image collections based on visual content is that of browsing. The aim of this survey paper is to provide a structured overview of the different models that have been explored over the last one to two decades, to highlight the particular challenges of the browsing approach and to focus attention on a few interesting issues that warrant more intense research.

Combining spatial and colour information for content based image retrieval

Colour is one of the most important features in content based image retrieval. However, colour is rarely used as a feature that codes local spatial information, except for colour texture. This paper presents an approach to represent spatial colour distributions using local principal component analysis (PCA). The representation is based on image windows which are selected by two complementary data driven attentive mechanisms: a symmetry based saliency map and an edge and corner detector. The eigenvectors obtained from local PCA of the selected windows form colour patterns that capture both low and high spatial frequencies, so they are well suited for shape as well as texture representation. Projections of the windows selected from the image database to the local PCs serve as a compact representation for the search database. Queries are formulated by specifying windows within query images. System feedback makes both the search process and the results comprehensible for the user.     

A hybrid multi-objective optimization algorithm for content based image retrieval

Relevance feedback methods in CBIR (Content Based Image Retrieval) iteratively use relevance information from the user to search the space for other relevant samples. As several regions of interest may be scattered through the space, an effective search algorithm should balance the exploration of the space to find new potential regions of interest and the exploitation of areas around samples which are known relevant. However, many algorithms concentrate the search on areas which are close to the images that the user has marked as relevant, according to a distance function in the (possibly deformed) multidimensional feature space. This maximizes the number of relevant images retrieved at the first iterations, but limits the discovery of new regions of interest and may leave unexplored a large section of the space. In this paper, we propose a novel hybrid approach that uses a scattered search algorithm based on NSGA II (Non-dominated Sorting Genetic Algorithm) only at the first iteration of the relevance feedback process, and then switches to an exploitation algorithm. The combined approach has been tested on three databases and in combination with several other methods. When the hybrid method does not produce better results from the first iteration, it soon catches up and improves both precision and recall.     

An efficient bi-layer content based image retrieval system

Multimedia Tools and Applications - Large amount of multi-media content, generated by various image capturing devices, is shared and downloaded by millions of users across the globe, every second....     

Encoder-decoder based convolutional neural networks for image forgery detection

Multimedia Tools and Applications - Today, images editing software has greatly evolved, thanks to them that the semantic manipulation of images has become easier. On the other hand, the...     

基于神经网络的图像盲水印技术

将水印嵌入到宿主图像的小波变换域的低频分量;利用BP神经网络的自学习、自适应的特性和一段已知序列训练神经网络,根据确定的神经网络模型可实现水印的盲提取;在神经网络的输入信号计算上提出选择邻域窗口为3＊3方形窗口比十字窗口具有更好的实验效果.仿真实验结果表明该算法对常用的图像处理如JPEG压缩、剪切、加噪和滤波等攻击具有较好的鲁棒性和不可见性.     

A probabilistic semantic model for image annotation and multi-modal image retrieval

This paper addresses automatic image annotation problem and its application to multi-modal image retrieval. The contribution of our work is three-fold. (1) We propose a probabilistic semantic model in which the visual features and the textual words are connected via a hidden layer which constitutes the semantic concepts to be discovered to explicitly exploit the synergy among the modalities. (2) The association of visual features and textual words is determined in a Bayesian framework such that the confidence of the association can be provided. (3) Extensive evaluation on a large-scale, visually and semantically diverse image collection crawled from Web is reported to evaluate the prototype system based on the model. In the proposed probabilistic model, a hidden concept layer which connects the visual feature and the word layer is discovered by fitting a generative model to the training image and annotation words through an Expectation-Maximization (EM) based iterative learning procedure. The evaluation of the prototype system on 17,000 images and 7736 automatically extracted annotation words from crawled Web pages for multi-modal image retrieval has indicated that the proposed semantic model and the developed Bayesian framework are superior to a state-of-the-art peer system in the literature.     

Content-based image retrieval by combining convolutional neural networks and sparse representation

Multimedia Tools and Applications - As stored data and images on memory disks increase, image retrieval has a necessary task on image processing. Although lots of researches have been reported for...