Deep multi-query video retrieval

Video retrieval methods have been developed for a single query. Multi-query video retrieval problem has not been investigated yet. In this study, an efficient and fast multi-query video retrieval framework is developed. Query videos are assumed to be related to more than one semantic. The framework supports an arbitrary number of video queries. The method is built upon using binary video hash codes. As a result, it is fast and requires a lower storage space. Database and query hash codes are generated by a deep hashing method that not only generates hash codes but also predicts query labels when they are chosen outside the database. The retrieval is based on the Pareto front multi-objective optimization method. Re-ranking performed on the retrieved videos by using non-binary deep features increases the retrieval accuracy considerably. Simulations carried out on two multi-label video databases show that the proposed method is efficient and fast in terms of retrieval accuracy and time.     

Unsupervised discriminative hashing

Hashing is one of the popular solutions for approximate nearest neighbor search because of its low storage cost and fast retrieval speed, and many machine learning algorithms are adapted to learn effective hash function. As hash codes of the same cluster are similar to each other while the hash codes in different clusters are dissimilar, we propose an unsupervised discriminative hashing learning method (UDH) to improve discrimination among hash codes in different clusters. UDH shares a similar objective function with spectral hashing algorithm, and uses a modified graph Laplacian matrix to exploit local discriminant information. In addition, UDH is designed to enable efficient out-of-sample extension. Experiments on real world image datasets demonstrate the effectiveness of our novel approach for image retrieval.     

基于分层匹配追踪的快速图像检索

为从海量的图像资源中既准确又快速地检索出目 标图像,在传统的 图像检索模型中,图像的特征通常是从固定尺度的图像上提取出的,这将不可避免地降低整 个系统 实际应用能力。为解决这一问题,本文引入分层稀疏编码模型, 提出一种基于分层匹配追踪(HMP)的快速图像检索技术,实现多尺度情况下的图像检索。本 文方法从图像中提取的低层稀疏编码特征传递 到高层,并将提取的高维稀疏编码特征转换为改进后的PCAH特征,利用哈希特征的汉明距 离度量, 实现图像的快速检索。在公共数据集Caltech256和Corel5K上的实验 结果可以看出,本文方法的查 准率和查全率较其他哈希法分别提高了5%和10%以上,而且所用时间 也最短,表明本文方法不仅具有较高的准确率,还能保持较高的时间效率。     

基于深度卷积神经网络和二进制哈希学习的图像检索方法

随着图像数据的迅猛增长,当前主流的图像检索方法采用的视觉特征编码步骤固定,缺少学习能力,导致其图像表达能力不强,而且视觉特征维数较高,严重制约了其图像检索性能。针对这些问题,该文提出一种基于深度卷积神径网络学习二进制哈希编码的方法,用于大规模的图像检索。该文的基本思想是在深度学习框架中增加一个哈希层,同时学习图像特征和哈希函数,且哈希函数满足独立性和量化误差最小的约束。首先,利用卷积神经网络强大的学习能力挖掘训练图像的内在隐含关系,提取图像深层特征,增强图像特征的区分性和表达能力。然后,将图像特征输入到哈希层,学习哈希函数使得哈希层输出的二进制哈希码分类误差和量化误差最小,且满足独立性约束。最后,给定输入图像通过该框架的哈希层得到相应的哈希码,从而可以在低维汉明空间中完成对大规模图像数据的有效检索。在3个常用数据集上的实验结果表明,利用所提方法得到哈希码,其图像检索性能优于当前主流方法。     

Effective and efficient indexing in cross-modal hashing-based datasets

To overcome the barrier of storage and computation, the hashing technique has been widely used for nearest neighbor search in multimedia retrieval applications recently. Particularly, cross-modal retrieval that searches across different modalities becomes an active but challenging problem. Although numerous of cross-modal hashing algorithms are proposed to yield compact binary codes, exhaustive search is impractical for large-scale datasets, and Hamming distance computation suffers inaccurate results. In this paper, we propose a novel search method that utilizes a probability-based index scheme over binary hash codes in cross-modal retrieval. The proposed indexing scheme employs a few binary bits from the hash code as the index code. We construct an inverted index table based on the index codes, and train a neural network for ranking and indexing to improve the retrieval accuracy. Experiments are performed on two benchmark datasets for retrieval across image and text modalities, where hash codes are generated and compared with several state-of-the-art cross-modal hashing methods. Results show the proposed method effectively boosts the performance on search accuracy, computation cost, and memory consumption in these datasets and hashing methods. The source code is available on https://github.com/msarawut/HCI.     

基于图像分割的肺结节CT图像哈希检索

基于相似图像的肺结节CT图像检索辅助诊断对肺结节的发现有着重要的作用。肺结节的诊断难度较大,通常需要充分利用图像的边缘、分叶、毛刺、纹理等各类信息。文中针对目前基于哈希方法的肺结节检索中存在的不能充分利用图像分割信息从而导致部分信息丢失问题做出了改进,提出了一种基于图像分割的肺结节图像哈希检索方法。实验结果表明,在72位哈希码长度时,达到了85.3%的平均准确率。并且,将文中图像分割模块应用于其他哈希检索方法时,平均准确率皆有一定的提升。     

Spline regression hashing for fast image search

Techniques for fast image retrieval over large databases have attracted considerable attention due to the rapid growth of web images. One promising way to accelerate image search is to use hashing technologies, which represent images by compact binary codewords. In this way, the similarity between images can be efficiently measured in terms of the Hamming distance between their corresponding binary codes. Although plenty of methods on generating hash codes have been proposed in recent years, there are still two key points that needed to be improved: 1) how to precisely preserve the similarity structure of the original data and 2) how to obtain the hash codes of the previously unseen data. In this paper, we propose our spline regression hashing method, in which both the local and global data similarity structures are exploited. To better capture the local manifold structure, we introduce splines developed in Sobolev space to find the local data mapping function. Furthermore, our framework simultaneously learns the hash codes of the training data and the hash function for the unseen data, which solves the out-of-sample problem. Extensive experiments conducted on real image datasets consisting of over one million images show that our proposed method outperforms the state-of-the-art techniques.     

Discriminative feature mining hashing for fine-grained image retrieval

With the development of multimedia technology, fine-grained image retrieval has gradually become a new hot topic in computer vision, while its accuracy and speed are limited due to the low discriminative high-dimensional real-valued embedding. To solve this problem, we propose an end-to-end framework named DFMH (Discriminative Feature Mining Hashing), which consists of the DFEM (Discriminative Feature Extracting Module) and SHCM (Semantic Hash Coding Module). Specifically, DFEM explores more discriminative local regions by attention drop and obtains finer local feature expression by attention re-sample. SHCM generates high-quality hash codes by combining the quantization loss and bit balance loss. Validated by extensive experiments and ablation studies, our method consistently outperforms both the state-of-the-art generic retrieval methods as well as fine-grained retrieval methods on three datasets, including CUB Birds, Stanford Dogs and Stanford Cars.     

各向同性的迭代量化哈希算法

准确有效的哈希算法是实现海量高维数据近邻检索的关键.迭代量化哈希（Iterative Quantization,ITQ）和各向同性哈希（Isotropic Hash,IsoHash）是两种知名的编码方法.但是ITQ算法对旋转矩阵施加的约束过于单薄,容易导致过拟合;而IsoHash算法缺乏对哈希编码的更新策略,降低了编码质量.针对上述问题,提出了一种各向同性的迭代量化哈希算法.该方法采用迭代的策略,对编码矩阵和旋转矩阵交替更新,并在正交约束的基础上增加各向同性约束来学习最优旋转矩阵,最小化量化误差.在CIFAR-10、22K LabelMe和ANN_GIST_1M基准库上与多种方法进行对比,实验结果表明本文算法在查准率、查全率以及平均准确率均值等指标上均明显优于对比算法.     

基于稀疏自编码的无监督哈希算法

最近邻搜索在大规模图像检索中变得越来越重要。在最近邻搜索中,许多哈希方法因为快速查询和低内存被提出。然而,现有方法在哈希函数构造过程中对数据稀疏结构研究的不足,本文提出了一种无监督的稀疏自编码的图像哈希方法。基于稀疏自编码的图像哈希方法将稀疏构造过程引入哈希函数的学习过程中,即通过利用稀疏自编码器的KL距离对哈希码进行稀疏约束以增强局部保持映射过程中的判别性,同时利用L2范数来哈希编码的量化误差。实验中用两个公共图像检索数据集CIFAR-10和YouTube Faces验证了本文算法相比其他无监督哈希算法的优越性。     

深度学习和迭代量化在图像检索中的应用研究

基于内容的图像检索的关键在于对图像进行特征提取和对特征进行多比特量化编码 。近年来,基于内容的图像检索使用低级可视化特征对图像进行描述,存在“语义鸿沟”问题;其次,传统量化编码使用随机生成的投影矩阵,该矩阵与特征数据无关,因此不能保证量化的精确度。针对目前存在的这些问题,本文结合深度学习思想与迭代量化思想,提出基于卷积神经网络VGG16和迭代量化(Iterative Quantization, ITQ)的图像检索方法。使用在公开数据集上预训练VGG16网络模型,提取基于深度学习的图像特征;使用ITQ方法对哈希哈函数进行训练,不断逼近特征与设定比特数的哈希码之间的量化误差最小值,实现量化误差的最小化;最后使用获得的哈希码进行图像检索。本文使用查全率、查准率和平均精度均值作为检索效果的评价指标,在Caltech256图像库上进行测试。实验结果表明,本文提出的算法在检索优于其他主流图像检索算法。      

基于非对称监督深度离散哈希的图像检索

哈希广泛应用于图像检索任务。针对现有深度监督哈希方法的局限性,该文提出了一种新的非对称监督深度离散哈希(ASDDH)方法来保持不同类别之间的语义结构,同时生成二进制码。首先利用深度网络提取图像特征,根据图像的语义标签来揭示每对图像之间的相似性。为了增强二进制码之间的相似性,并保证多标签语义保持,该文设计了一种非对称哈希方法,并利用多标签二进制码映射,使哈希码具有多标签语义信息。此外,引入二进制码的位平衡性对每个位进行平衡,鼓励所有训练样本中的–1和+1的数目近似。在两个常用数据集上的实验结果表明,该方法在图像检索方面的性能优于其他方法。     

图像检索研究综述及系统实现

利用文献可视化分析工具对图像检索研究现状进行了综述，同时设计了一种基于位置敏感哈希算法的图像检索系统。通过位置敏感哈希算法将图像的特征向量映射到哈希桶中，从而有效地降低了计算复杂度并提高了图像检索的效率。实验结果表明，文中设计的方法在检索效率以及查全率-查准率两个测度上均获得了较好的性能。     

Research on image feature extraction and retrieval algorithms based on convolutional neural network

With the rapid development of mobile Internet and digital technology, people are more and more keen to share pictures on social networks, and online pictures have exploded. How to retrieve similar images from large-scale images has always been a hot issue in the field of image retrieval, and the selection of image features largely affects the performance of image retrieval. The Convolutional Neural Networks (CNN), which contains more hidden layers, has more complex network structure and stronger ability of feature learning and expression compared with traditional feature extraction methods. By analyzing the disadvantage that global CNN features cannot effectively describe local details when they act on image retrieval tasks, a strategy of aggregating low-level CNN feature maps to generate local features is proposed. The high-level features of CNN model pay more attention to semantic information, but the low-level features pay more attention to local details. Using the increasingly abstract characteristics of CNN model from low to high. This paper presents a probabilistic semantic retrieval algorithm, proposes a probabilistic semantic hash retrieval method based on CNN, and designs a new end-to-end supervised learning framework, which can simultaneously learn semantic features and hash features to achieve fast image retrieval. Using convolution network, the error rate is reduced to 14.41% in this test set. In three open image libraries, namely Oxford, Holidays and ImageNet, the performance of traditional SIFT-based retrieval algorithms and other CNN-based image retrieval algorithms in tasks are compared and analyzed. The experimental results show that the proposed algorithm is superior to other contrast algorithms in terms of comprehensive retrieval effect and retrieval time.     

Image alignment based perceptual image hash for content authentication

Perceptual image hash is an emerging technology that is closely related to many applications such as image content authentication, image forging detection, image similarity detection, and image retrieval. In this work, we propose an image alignment based perceptual image hash method, and a hash-based image forging detection and tampering localization method. In the proposed method, we introduce an image alignment process to provide a framework for image hash method to tolerate a wide range of geometric distortions. The image hash is generated by utilizing hybrid perceptual features that are extracted from global and local Zernike moments combining with DCT-based statistical features of the image. The proposed method can detect various image forging and compromised image regions. Furthermore, it has broad-spectrum robustness, including tolerating content-preserving manipulations and geometric distortion-resilient. Compared with state-of-the-art schemes, the proposed method provides satisfactory comprehensive performances in content-based image forging detection and tampering localization.     

Efficient privacy-preserving image retrieval scheme over outsourced data with multi-user

The traditional privacy-preserving image retrieval schemes not only bring large computational and communication overhead,but also cannot protect the image and query privacy in multi-user scenarios.To solve above problems,an efficient privacy-preserving content-based image retrieval scheme was proposed in multi-user scenarios.The scheme used Euclidean distance comparison technique to rank the pictures according to similarity of picture feature vectors and return top-k returned.Meanwhile,the efficient key conversion protocol designed in proposed image retrieval scheme allowed each search user to generate queries based on his own private key so that he can retrieval encrypted images generated by different data owners.Strict security analysis shows that the user privacy and cloud data security can be well protected during the image retrieval process,and the performance analysis using real-world dataset shows that the proposed image retrieval scheme is efficient and feasible in practical applications.     

An image retrieval method based on spatial distribution of color

Color histogram is now widely used in image retrieval. Color histogram-based image retrieval methods are simple and efficient but without considering the spatial distribution information of the color. To overcome the shortcoming of conventional color histogram-based image retrieval methods, an image retrieval method based on Radon Transform （RT） is proposed. In order to reduce the computational complexity, wavelet decomposition is used to compress image data. Firstly, images are decomposed by Mallat algorithm. The low-frequency components are then projected by RT to generate the spatial color feature. Finally the moment feature matrices which are saved along with original images are obtained. Experimental results show that the RT based retrieval is more accurate and efficient than traditional color histogram-based method in case that there are obvious objects in images. Further more, RT based retrieval runs significantly faster than the traditional color histogram methods.     

Content-Based Image Retrieval Using Moments of Local Ternary Pattern

Due to the availability of large number of digital images, development of an efficient content-based indexing and retrieval method is required. Also, the emergence of smartphones and modern PDAs has further substantiated the need of such systems. This paper proposes a combination of Local Ternary Pattern (LTP) and moments for Content-Based Image Retrieval. Image is divided into blocks of equal size and LTP codes of each block are computed. Geometric moments of LTP codes of each block are computed followed by computation of distance between moments of LTP codes of query and database images. Then, the threshold using distance values is applied to retrieve images similar to the query image. Performance of the proposed method is compared with other state-of-the-art methods on the basis of results obtained on Corel-1,000 database. The comparison shows that the proposed method gives better results in terms of precision and recall as compared to other state-of-the-art image retrieval methods.     

An Indexing and Retrieval Mechanism for Complex Similarity Queries in Image Databases

A content-based image retrieval mechanism to support complex similarity queries is presented. The image content is defined by three kinds of features: quantifiable features describing the visual information, nonquantifiable features describing the semantic information, and keywords describing more abstract semantic information. In correspondence with these feature sets, we construct three types of indexes: visual indexes, semantic indexes, and keyword indexes. Index structures are elaborated to provide effective and efficient retrieval of images based on their contents. The underlying index structure used for all indexes is the HG-tree. In addition to the HG-tree, the signature file and hashing technique are also employed to index keywords and semantic features. The proposed indexing scheme combines and extends the HG-tree, the signature file, and the hashing scheme to support complex similarity queries. We also propose a new evaluation strategy to process the complex similarity queries. Experiments have been carried out on large image collections to demonstrate the effectiveness of the proposed retrieval mechanism.     

Integrated spectral and spatial information mining in remote sensing imagery

Most existing remote sensing image retrieval systems allow only simple queries based on sensor, location, and date of image capture. This approach does not permit the efficient retrieval of useful hidden information from large image databases. This paper presents an integrated approach to retrieving spectral and spatial patterns from remotely sensed imagery using state-of-the-art data mining and advanced database technologies. Land cover information corresponding to spectral characteristics is identified by supervised classification based on support vector machines with automatic model selection, while textural features characterizing spatial information are extracted using Gabor wavelet coefficients. Within identified land cover categories, textural features are clustered to acquire search-efficient space in an object-oriented database with associated images in an image database. Interesting patterns are then retrieved using a query-by-example approach. The evaluation of the study results using coverage and novelty measures validates the effectiveness of the proposed remote sensing image information mining framework, which is potentially useful for applications such as agricultural and environmental monitoring.