Similarity-based online feature selection in content-based image retrieval.

Content-based image retrieval (CBIR) has been more and more important in the last decade, and the gap between high-level semantic concepts and low-level visual features hinders further performance improvement. The problem of online feature selection is critical to really bridge this gap. In this paper, we investigate online feature selection in the relevance feedback learning process to improve the retrieval performance of the region-based image retrieval system. Our contributions are mainly in three areas. 1) A novel feature selection criterion is proposed, which is based on the psychological similarity between the positive and negative training sets. 2) An effective online feature selection algorithm is implemented in a boosting manner to select the most representative features for the current query concept and combine classifiers constructed over the selected features to retrieve images. 3) To apply the proposed feature selection method in region-based image retrieval systems, we propose a novel region-based representation to describe images in a uniform feature space with real-valued fuzzy features. Our system is suitable for online relevance feedback learning in CBIR by meeting the three requirements: learning with small size training set, the intrinsic asymmetry property of training samples, and the fast response requirement. Extensive experiments, including comparisons with many state-of-the-arts, show the effectiveness of our algorithm in improving the retrieval performance and saving the processing time.     

Research on image feature extraction and retrieval algorithms based on convolutional neural network

With the rapid development of mobile Internet and digital technology, people are more and more keen to share pictures on social networks, and online pictures have exploded. How to retrieve similar images from large-scale images has always been a hot issue in the field of image retrieval, and the selection of image features largely affects the performance of image retrieval. The Convolutional Neural Networks (CNN), which contains more hidden layers, has more complex network structure and stronger ability of feature learning and expression compared with traditional feature extraction methods. By analyzing the disadvantage that global CNN features cannot effectively describe local details when they act on image retrieval tasks, a strategy of aggregating low-level CNN feature maps to generate local features is proposed. The high-level features of CNN model pay more attention to semantic information, but the low-level features pay more attention to local details. Using the increasingly abstract characteristics of CNN model from low to high. This paper presents a probabilistic semantic retrieval algorithm, proposes a probabilistic semantic hash retrieval method based on CNN, and designs a new end-to-end supervised learning framework, which can simultaneously learn semantic features and hash features to achieve fast image retrieval. Using convolution network, the error rate is reduced to 14.41% in this test set. In three open image libraries, namely Oxford, Holidays and ImageNet, the performance of traditional SIFT-based retrieval algorithms and other CNN-based image retrieval algorithms in tasks are compared and analyzed. The experimental results show that the proposed algorithm is superior to other contrast algorithms in terms of comprehensive retrieval effect and retrieval time.     

LSA based multi-instance learning algorithm for image retrieval

Focusing on the problem of natural image retrieval, based on latent semantic analysis (LSA) and support vector machine (SVM), a novel multi-instance learning (MIL) algorithm is proposed, where a bag corresponds to an image and an instance corresponds to the low-level visual features of a segmented region. Firstly, in order to transform every bag into a single sample, a collection of “visual-word” is generated by k-means clustering method to construct a projection space, then a nonlinear mapping is defined using these “visual-word” to embed each bag as a point in the projection space, thereby obtaining every bag's projection feature. Secondly, the matrix consisted of all the projection features of training bags is regarded as a term-document matrix, and LSA method is used to obtain the latent semantic feature of each bag. As a result, the MIL problem is converted into a standard single instance learning (SIL) problem that can be solved directly by SVM method. Experimental results on the COREL data sets show that the proposed method, named LSASVM-MIL, is robust, and its performance is superior to other key existing MIL algorithms.     

基于融合特征的现勘图像检索结果优化算法

刑侦现勘图像数据库是具有保密性高、图像内容罕见等极具行业特色的图像数据库.针对现勘图像内容复杂、目标物体不明确的特点,提出了DCT-DCT波纹理特征,并与HSV颜色直方图特征、GIST特征相融合构成融合特征.与常用的图像特征相比,DCT-DCT波纹理特征能够得到较高的检索效率,而融合特征的平均检索查准率高于构成其本身的三种特征的平均检索查准率.最后,将语义分析技术引入到检索过程中,提出基于检索结果优化的现勘图像检索算法,利用支持向量机（Support Vector Machine,SVM）分类器对查询图像进行语义提取,并对初次检索的结果进行语义分析,根据初检结果中语义类别的占比选择二次检索方案,该算法能在按例查询的基础上进一步提高平均检索查准率.     

基于形状的图像检索技术研究

基于形状的图像检索技术是基于内容的图像检索技术的一个重要组成部分。现有的形状特征检索技术主要集中在形状特征的提取及相似性度量、形状特征与颜色和纹理特征结合、形状特征与高层的语义特征结合的研究。在分析现有的基于形状的图像检索技术的一些关键技术的基础上,对基于小波-傅里叶特征(WFD)的形状检索方法进行了研究,并提出了一些改进算法。结合Matlab和ACCESS实现了一个基于形状的图像检索实验系统,建立了用户界面,选取与设计了4个图像测试集,使用检索性能评价方法对形状特征的检索结果进行了客观的评价。实验结果表明,利用本文所提出改进的形状特征进行检索取得了较好的检索效果。     

Structural image retrieval using automatic image annotation and region based inverted file

Image retrieval has lagged far behind text retrieval despite more than two decades of intensive research effort. Most of the research on image retrieval in the last two decades are on content based image retrieval or image retrieval based on low level features. Recent research in this area focuses on semantic image retrieval using automatic image annotation. Most semantic image retrieval techniques in literature, however, treat an image as a bag of features/words while ignore the structural or spatial information in the image. In this paper, we propose a structural image retrieval method based on automatic image annotation and region based inverted file. In the proposed system, regions in an image are treated the same way as keywords in a structural text document, semantic concepts are learnt from image data to label image regions as keywords and weight is assigned to each keyword according to spatial position and relationship. As the result, images are indexed and retrieved in the same way as structural document retrieval. Specifically, images are broken down to regions which are represented using colour, texture and shape features. Region features are then quantized to create visual dictionaries which are similar to monolingual dictionaries like English or Chinese dictionaries. In the next step, a semantic dictionary similar to a bilingual dictionary like the English–Chinese dictionary is learnt to mapping image regions to semantic concepts. Finally, images are then indexed and retrieved using a novel region based inverted file data structure. Results show the proposed method has significant advantage over the widely used Bayesian annotation models.     

Support Vector Machine Learning for Region‐Based Image Retrieval with Relevance Feedback

We present a relevance feedback approach based on multi‐class support vector machine (SVM) learning and cluster‐merging which can significantly improve the retrieval performance in region‐based image retrieval. Semantically relevant images may exhibit various visual characteristics and may be scattered in several classes in the feature space due to the semantic gap between low‐level features and high‐level semantics in the user's mind. To find the semantic classes through relevance feedback, the proposed method reduces the burden of completely re‐clustering the classes at iterations and classifies multiple classes. Experimental results show that the proposed method is more effective and efficient than the two‐class SVM and multi‐class relevance feedback methods.     

一种基于TSP-KNN的事件相关故事单元检索算法

在传统的基于内容视频检索的方法中,由于视频的领域较宽,视频的低级视觉特征和高级概念之间存在着较大的语义鸿沟,常导致检索效果不佳．本文认为更有现实意义的做法是,以含有比镜头更多语义信息的事件相关故事单元为检索单位,通过提取事件相关媒体中的文本信息并利用机器学习方法自动建立事件类的模型,从而提供概念化的故事单元查询方式．本文提出了组合特征选择方法和一种二阶段修剪KNN：TSP-KNN,组合特征选择方法相对于MI方法更适合事件相关故事单元的检索．二阶段修剪KNN先对训练集进行修剪,然后再用KNN训练得到分类器,该方法解决了样本混叠以及多中心分布问题．实验结果表明所提出的方法是有效的,明显地提高了事件相关故事单元的检索性能．     

基于数据挖掘的面向话题搜索引擎研究

为了解决面向话题的搜索问题,提出一种新的面向话题的检索技术。首先分析了面向话题的搜索技术所面临的问题,然后基于数据挖掘技术提出了解决方案。利用数据挖掘技术抽取文本的多层次语义特征,形成对文本的多精度表示,抽取的特征不仅包括单个词特征也包括多词特征。建立了一个示例检索系统,实验表明利用多层次文本特征能够很好地实现面向话题的文本检索。     

一种改进的综合BTC颜色矩和DT-CWT的图像检索算法

摘 要：特征提取是基于内容的图像检索中的关键技术。针对基于单一特征检索效果不理想的问题,提出一种改进的综合颜色和纹理特征的图像检索算法。该算法在YIQ颜色空间中进行特征提取,首先结合方块编码（BTC）的思想,提取颜色矩作为颜色特征;采用双树复小波变换（DT-CWT）提取纹理特征,融合两种特征并利用相似性度量方式进行图像检索。实验结果表明算法所提取的颜色、纹理特征更利于检索,使用综合特征检索的平均查准率比同类算法更高。     

基于“点到面”策略的图像检索

图像检索是计算机视觉领域的一个重要分支。其主要目的是从图像数据库中找出与查询图像相似的语义图像。传统的图像检索方法是在查询图像和数据库图像之间进行“点到点”检索。但是,单个查询图像包含的类别提示较少,即类别信息较弱,使得检索结果并不理想。为了解决这个问题,本文提出了一种基于“点到面”的类别检索策略来扩展一个图像（点）到一个图像类别（面）,这意味着从单个查询图像到整个图像类别的语义扩展。该方法挖掘了查询图像的类别信息。在两个常用的数据集上对所提出方法的性能进行了评估。实验表明,该方法可以显著提高图像检索的性能。      

深度学习和迭代量化在图像检索中的应用研究

基于内容的图像检索的关键在于对图像进行特征提取和对特征进行多比特量化编码 。近年来,基于内容的图像检索使用低级可视化特征对图像进行描述,存在“语义鸿沟”问题;其次,传统量化编码使用随机生成的投影矩阵,该矩阵与特征数据无关,因此不能保证量化的精确度。针对目前存在的这些问题,本文结合深度学习思想与迭代量化思想,提出基于卷积神经网络VGG16和迭代量化(Iterative Quantization, ITQ)的图像检索方法。使用在公开数据集上预训练VGG16网络模型,提取基于深度学习的图像特征;使用ITQ方法对哈希哈函数进行训练,不断逼近特征与设定比特数的哈希码之间的量化误差最小值,实现量化误差的最小化;最后使用获得的哈希码进行图像检索。本文使用查全率、查准率和平均精度均值作为检索效果的评价指标,在Caltech256图像库上进行测试。实验结果表明,本文提出的算法在检索优于其他主流图像检索算法。      

基于方块编码的图像纹理特征提取及检索算法

针对灰度共生矩阵（GLCM）在提取纹理特征时存在的问题,提出一种基于方块编码（BTC）的图像纹理特征的检索算法。首先将图像分成互不重叠的子图像块,然后利用BTC的思想对这些图像块进行编码,进而定义图像的纹理基元并以此作为对图像的纹理描述,并提出采用一种改进的基于纹理基元的共生矩阵来获取纹理特征。实验结果表明,该方法既有效地利用了图像的纹理信息,又考虑了图像的空间和形状信息,具有较好的检索效果。     

DRM: dynamic region matching for image retrieval using probabilistic fuzzy matching and boosting feature selection

This paper considers the semantic gap in content-based image retrieval from two aspects: (1) irrelevant visual contents (e.g. background) scatter the mapping from image to human perception; (2) unsupervised feature extraction and similarity ranking method can not accurately reveal users’ image perception. This paper proposes a novel region-based retrieval framework—dynamic region matching (DRM) to bridge the semantic gap. (1) To address the first issue, a probabilistic fuzzy region matching algorithm is adopted to retrieve and match images precisely at object level, which copes with the problem of inaccurate segmentation. (2) To address the second issue, a “FeatureBoost” algorithm is proposed to construct an effective “eigen” feature set in relevance feedback (RF) process. And the significance of each region is dynamically updated in RF learning to automatically capture users’ region of interest (ROI). (3) User’s retrieval purpose is predicted using a novel log-learning algorithm, which predicts users’ retrieval target in the feature space using the accumulated user operations. Extensive experiments have been conducted on Corel image database with over 10,000 images. The promising experimental results reveal the effectiveness of our scheme in bridging the semantic gap.     

基于多目标区域的图像检索

针对传统的基于目标区域的图像检索算法中存在的"语义鸿沟"问题,以及基于全局特征的图像检索算法不能很好地处理多目标检索问题,提出了一种基于多目标区域的图像检索模型,并实现了一款高效的检索算法.首先借助于目标检测算法定位出图像中的目标,然后使用卷积神经网络(CNN)提取各个目标的特征,最后采用新提出的多目标区域相似度测量方法计算其与数据库图像的相似度并返回检索结果.实验表明,所提算法与现有的其他检索算法相比,在多目标图像检索任务上性能更佳.     

A Pattern Similarity Scheme for Medical Image Retrieval

In this paper, we propose a novel scheme for efficient content-based medical image retrieval, formalized according to the PAtterns for Next generation DAtabase systems (PANDA) framework for pattern representation and management. The proposed scheme involves block-based low-level feature extraction from images followed by the clustering of the feature space to form higher-level, semantically meaningful patterns. The clustering of the feature space is realized by an expectation–maximization algorithm that uses an iterative approach to automatically determine the number of clusters. Then, the 2-component property of PANDA is exploited: the similarity between two clusters is estimated as a function of the similarity of both their structures and the measure components. Experiments were performed on a large set of reference radiographic images, using different kinds of features to encode the low-level image content. Through this experimentation, it is shown that the proposed scheme can be efficiently and effectively applied for medical image retrieval from large databases, providing unsupervised semantic interpretation of the results, which can be further extended by knowledge representation methodologies.      

基于改进哈希算法的图像检索方法

当前主流图像检索技术所采用的传统视觉特征编码缺少足够的学习能力,影响学习得到的特征表达能力。此外,由于视觉特征维数高,会消耗大量的内存,因此降低了图像检索的性能。文中基于深度卷积神经网络与改进的哈希算法,提出并设计了一种端到端训练方式的图像检索方法。该方法将卷积神经网络提取的高层特征和哈希函数相结合,学习到具有足够表达能力的哈希特征,从而在低维汉明空间中完成对图像数据的大规模检索。在两个常用数据集上的实验结果表明,所提出的哈希图像检索方法的检索性能优于当前的一些主流方法。     

Concept-oriented indexing of video databases: toward semantic sensitive retrieval and browsing

Digital video now plays an important role in medical education, health care, telemedicine and other medical applications. Several content-based video retrieval (CBVR) systems have been proposed in the past, but they still suffer from the following challenging problems: semantic gap, semantic video concept modeling, semantic video classification, and concept-oriented video database indexing and access. In this paper, we propose a novel framework to make some advances toward the final goal to solve these problems. Specifically, the framework includes: 1) a semantic-sensitive video content representation framework by using principal video shots to enhance the quality of features; 2) semantic video concept interpretation by using flexible mixture model to bridge the semantic gap; 3) a novel semantic video-classifier training framework by integrating feature selection, parameter estimation, and model selection seamlessly in a single algorithm; and 4) a concept-oriented video database organization technique through a certain domain-dependent concept hierarchy to enable semantic-sensitive video retrieval and browsing.     

基于图正则化与非负组稀疏的自动图像标注

设计一个稳健的自动图像标注系统的重要环节是提取能够有效描述图像语义的视觉特征。由于颜色、纹理和形状等异构视觉特征在表示特定图像语义时所起作用的重要程度不同且同一类特征之间具有一定的相关性,该文提出了一种图正则化约束下的非负组稀疏(Graph Regularized Non-negative Group Sparsity, GRNGS)模型来实现图像标注,并通过一种非负矩阵分解方法来计算其模型参数。该模型结合了图正则化与l2,1-范数约束,使得标注过程中所选的组群特征能体现一定的视觉相似性和语义相关性。在Corel5K和ESP Game等图像数据集上的实验结果表明:相较于一些最新的图像标注模型,GRNGS模型的鲁棒性更强,标注结果更精确。