基于深度卷积神经网络和二进制哈希学习的图像检索方法

随着图像数据的迅猛增长,当前主流的图像检索方法采用的视觉特征编码步骤固定,缺少学习能力,导致其图像表达能力不强,而且视觉特征维数较高,严重制约了其图像检索性能。针对这些问题,该文提出一种基于深度卷积神径网络学习二进制哈希编码的方法,用于大规模的图像检索。该文的基本思想是在深度学习框架中增加一个哈希层,同时学习图像特征和哈希函数,且哈希函数满足独立性和量化误差最小的约束。首先,利用卷积神经网络强大的学习能力挖掘训练图像的内在隐含关系,提取图像深层特征,增强图像特征的区分性和表达能力。然后,将图像特征输入到哈希层,学习哈希函数使得哈希层输出的二进制哈希码分类误差和量化误差最小,且满足独立性约束。最后,给定输入图像通过该框架的哈希层得到相应的哈希码,从而可以在低维汉明空间中完成对大规模图像数据的有效检索。在3个常用数据集上的实验结果表明,利用所提方法得到哈希码,其图像检索性能优于当前主流方法。     

Deep multi query image retrieval

There exist few studies investigating the multi-query image retrieval problem. Existing methods are not based on hash codes. As a result, they are not efficient and fast. In this study, we develop an efficient and fast multi-query image retrieval method when the queries are related to more than one semantic. Image hash codes are generated by a deep hashing method. Consequently, the method requires lower storage space, and it is faster compared to the existing methods. The retrieval is based on the Pareto front method. Reranking performed on the retrieved images by using non-binary deep-convolutional features increase retrieval accuracy considerably. Unlike previous studies, the method supports an arbitrary number of queries. It outperforms similar multi-query image retrieval studies in terms of retrieval time and retrieval accuracy.     

Effective and efficient indexing in cross-modal hashing-based datasets

To overcome the barrier of storage and computation, the hashing technique has been widely used for nearest neighbor search in multimedia retrieval applications recently. Particularly, cross-modal retrieval that searches across different modalities becomes an active but challenging problem. Although numerous of cross-modal hashing algorithms are proposed to yield compact binary codes, exhaustive search is impractical for large-scale datasets, and Hamming distance computation suffers inaccurate results. In this paper, we propose a novel search method that utilizes a probability-based index scheme over binary hash codes in cross-modal retrieval. The proposed indexing scheme employs a few binary bits from the hash code as the index code. We construct an inverted index table based on the index codes, and train a neural network for ranking and indexing to improve the retrieval accuracy. Experiments are performed on two benchmark datasets for retrieval across image and text modalities, where hash codes are generated and compared with several state-of-the-art cross-modal hashing methods. Results show the proposed method effectively boosts the performance on search accuracy, computation cost, and memory consumption in these datasets and hashing methods. The source code is available on https://github.com/msarawut/HCI.     

基于非对称监督深度离散哈希的图像检索

哈希广泛应用于图像检索任务。针对现有深度监督哈希方法的局限性,该文提出了一种新的非对称监督深度离散哈希(ASDDH)方法来保持不同类别之间的语义结构,同时生成二进制码。首先利用深度网络提取图像特征,根据图像的语义标签来揭示每对图像之间的相似性。为了增强二进制码之间的相似性,并保证多标签语义保持,该文设计了一种非对称哈希方法,并利用多标签二进制码映射,使哈希码具有多标签语义信息。此外,引入二进制码的位平衡性对每个位进行平衡,鼓励所有训练样本中的–1和+1的数目近似。在两个常用数据集上的实验结果表明,该方法在图像检索方面的性能优于其他方法。     

基于深度学习的散列检索技术研究进展

大数据时代，数据呈现维度高、数据量大和增长快等特点。面对大量的复杂数据，如何高效地检索相似近邻数据是近似最近邻查询的研究热点。散列技术通过将数据映射为二进制码的方式，能够显著加快相似性计算，并在检索过程中节省存储和通信开销。近年来深度学习在提取数据特征方面表现出速度快、精度高等优异的性能，使得基于深度学习的散列检索技术得到越来越广泛的运用。总结了深度学习散列的主要方法和前沿进展，并对未来的研究方向展开简要探讨。     

基于稀疏自编码的无监督哈希算法

最近邻搜索在大规模图像检索中变得越来越重要。在最近邻搜索中,许多哈希方法因为快速查询和低内存被提出。然而,现有方法在哈希函数构造过程中对数据稀疏结构研究的不足,本文提出了一种无监督的稀疏自编码的图像哈希方法。基于稀疏自编码的图像哈希方法将稀疏构造过程引入哈希函数的学习过程中,即通过利用稀疏自编码器的KL距离对哈希码进行稀疏约束以增强局部保持映射过程中的判别性,同时利用L2范数来哈希编码的量化误差。实验中用两个公共图像检索数据集CIFAR-10和YouTube Faces验证了本文算法相比其他无监督哈希算法的优越性。     

Spline regression hashing for fast image search

Techniques for fast image retrieval over large databases have attracted considerable attention due to the rapid growth of web images. One promising way to accelerate image search is to use hashing technologies, which represent images by compact binary codewords. In this way, the similarity between images can be efficiently measured in terms of the Hamming distance between their corresponding binary codes. Although plenty of methods on generating hash codes have been proposed in recent years, there are still two key points that needed to be improved: 1) how to precisely preserve the similarity structure of the original data and 2) how to obtain the hash codes of the previously unseen data. In this paper, we propose our spline regression hashing method, in which both the local and global data similarity structures are exploited. To better capture the local manifold structure, we introduce splines developed in Sobolev space to find the local data mapping function. Furthermore, our framework simultaneously learns the hash codes of the training data and the hash function for the unseen data, which solves the out-of-sample problem. Extensive experiments conducted on real image datasets consisting of over one million images show that our proposed method outperforms the state-of-the-art techniques.     

Deep multi-query video retrieval

Video retrieval methods have been developed for a single query. Multi-query video retrieval problem has not been investigated yet. In this study, an efficient and fast multi-query video retrieval framework is developed. Query videos are assumed to be related to more than one semantic. The framework supports an arbitrary number of video queries. The method is built upon using binary video hash codes. As a result, it is fast and requires a lower storage space. Database and query hash codes are generated by a deep hashing method that not only generates hash codes but also predicts query labels when they are chosen outside the database. The retrieval is based on the Pareto front multi-objective optimization method. Re-ranking performed on the retrieved videos by using non-binary deep features increases the retrieval accuracy considerably. Simulations carried out on two multi-label video databases show that the proposed method is efficient and fast in terms of retrieval accuracy and time.     

基于图像分割的肺结节CT图像哈希检索

基于相似图像的肺结节CT图像检索辅助诊断对肺结节的发现有着重要的作用。肺结节的诊断难度较大,通常需要充分利用图像的边缘、分叶、毛刺、纹理等各类信息。文中针对目前基于哈希方法的肺结节检索中存在的不能充分利用图像分割信息从而导致部分信息丢失问题做出了改进,提出了一种基于图像分割的肺结节图像哈希检索方法。实验结果表明,在72位哈希码长度时,达到了85.3%的平均准确率。并且,将文中图像分割模块应用于其他哈希检索方法时,平均准确率皆有一定的提升。     

Discrete hashing with triple supervision learning

In recent years, discrete supervised hashing methods have attracted increasing attention because of their high retrieval efficiency and precision. However, in these methods, some effective semantic information is typically neglected, which means that all the information is not sufficiently utilized. Moreover, these methods often only decompose the first-order features of the original data, ignoring the more fine-grained higher-order features. To address these problems, we propose a supervised hashing learning method called discrete hashing with triple supervision learning (DHTSL). Specifically, we integrate three aspects of semantic information into this method: (1) the bidirectional mapping of semantic labels; (2) pairwise similarity relations; (3) second-order features from the original data. We also design a discrete optimization method to solve the proposed objective function. Moreover, an out-of-sample extension strategy that can better maintain the independence and balance of hash codes is employed to improve retrieval performance. Extensive experiments on three widely used datasets demonstrate its superior performance.     

基于加权自学习散列的高维数据最近邻查询算法

因为查询和存储具有高效性,学习型散列逐渐被应用于解决最近邻查询问题.学习型散列将高维数据转化成二进制编码,并使得原始高维空间中越相似的数据对应二进制编码的汉明距离越小.在实际应用中,每次查询都会返回许多与查询点汉明距离相同而编码互不相同的数据.如何对这些数据进行排序是一个难题.提出了一种基于加权自学习散列的近邻查找算法.实验结果表明,算法能够高效地对具有相同汉明距离的不同编码进行重排序,加权排序后查询的F1值约是原来的2倍并优于同系算法,时间开销可比直接计算原始距离进行排序降低一个数量级.     

Deep supervised hashing using quadratic spherical mutual information for efficient image retrieval

Several deep supervised hashing techniques have been proposed to allow for extracting compact and efficient neural network representations for various tasks. However, many deep supervised hashing techniques ignore several information-theoretic aspects of the process of information retrieval, often leading to sub-optimal results. In this paper, we propose an efficient deep supervised hashing algorithm that optimizes the learned compact codes using an information-theoretic measure, the Quadratic Mutual Information (QMI). The proposed method is adapted to the needs of efficient image hashing and information retrieval leading to a novel information-theoretic measure, the Quadratic Spherical Mutual Information (QSMI). Apart from demonstrating the effectiveness of the proposed method under different scenarios and outperforming existing state-of-the-art image hashing techniques, this paper provides a structured way to model the process of information retrieval and develop novel methods adapted to the needs of different applications.     

基于角点检测和NMF的图像Hash算法

为了有效地实现图像Hash函数在图像认证检索中的应用,提出了结合Harris角点检测和非负矩阵分解(NMF)的图像Hash算法,首先提取图像中的角点,对角点周围图像块信息进行非负矩阵分解得到表征图像局部特征的系数矩阵,进一步量化编码产生图像Hash。实验结果表明,得到的图像Hash对视觉可接受的操作如图像缩放、高斯低通滤波和JPEG压缩具有良好的稳健性,同时能区分出对图像大幅度扰动或修改的操作。     

A robust and secure perceptual hashing system based on a quantization step analysis

Perceptual hashing is conventionally used for content identification and authentication. It has applications in database content search, watermarking and image retrieval. Most countermeasures proposed in the literature generally focus on the feature extraction stage to get robust features to authenticate the image, but few studies address the perceptual hashing security achieved by a cryptographic module. When a cryptographic module is employed [1], additional information must be sent to adjust the quantization step. In the perceptual hashing field, we believe that a perceptual hashing system must be robust, secure and generate a final perceptual hash of fixed length. This kind of system should send only the final perceptual hash to the receiver via a secure channel without sending any additional information that would increase the storage space cost and decrease the security. For all of these reasons, in this paper, we propose a theoretical analysis of full perceptual hashing systems that use a quantization module followed by a crypto-compression module. The proposed theoretical analysis is based on a study of the behavior of the extracted features in response to content-preserving/content-changing manipulations that are modeled by Gaussian noise. We then introduce a proposed perceptual hashing scheme based on this theoretical analysis. Finally, several experiments are conducted to validate our approach, by applying Gaussian noise, JPEG compression and low-pass filtering.     

基于改进哈希算法的图像检索方法

当前主流图像检索技术所采用的传统视觉特征编码缺少足够的学习能力,影响学习得到的特征表达能力。此外,由于视觉特征维数高,会消耗大量的内存,因此降低了图像检索的性能。文中基于深度卷积神经网络与改进的哈希算法,提出并设计了一种端到端训练方式的图像检索方法。该方法将卷积神经网络提取的高层特征和哈希函数相结合,学习到具有足够表达能力的哈希特征,从而在低维汉明空间中完成对图像数据的大规模检索。在两个常用数据集上的实验结果表明,所提出的哈希图像检索方法的检索性能优于当前的一些主流方法。     

各向同性的迭代量化哈希算法

准确有效的哈希算法是实现海量高维数据近邻检索的关键.迭代量化哈希（Iterative Quantization,ITQ）和各向同性哈希（Isotropic Hash,IsoHash）是两种知名的编码方法.但是ITQ算法对旋转矩阵施加的约束过于单薄,容易导致过拟合;而IsoHash算法缺乏对哈希编码的更新策略,降低了编码质量.针对上述问题,提出了一种各向同性的迭代量化哈希算法.该方法采用迭代的策略,对编码矩阵和旋转矩阵交替更新,并在正交约束的基础上增加各向同性约束来学习最优旋转矩阵,最小化量化误差.在CIFAR-10、22K LabelMe和ANN_GIST_1M基准库上与多种方法进行对比,实验结果表明本文算法在查准率、查全率以及平均准确率均值等指标上均明显优于对比算法.     

Research on image feature extraction and retrieval algorithms based on convolutional neural network

With the rapid development of mobile Internet and digital technology, people are more and more keen to share pictures on social networks, and online pictures have exploded. How to retrieve similar images from large-scale images has always been a hot issue in the field of image retrieval, and the selection of image features largely affects the performance of image retrieval. The Convolutional Neural Networks (CNN), which contains more hidden layers, has more complex network structure and stronger ability of feature learning and expression compared with traditional feature extraction methods. By analyzing the disadvantage that global CNN features cannot effectively describe local details when they act on image retrieval tasks, a strategy of aggregating low-level CNN feature maps to generate local features is proposed. The high-level features of CNN model pay more attention to semantic information, but the low-level features pay more attention to local details. Using the increasingly abstract characteristics of CNN model from low to high. This paper presents a probabilistic semantic retrieval algorithm, proposes a probabilistic semantic hash retrieval method based on CNN, and designs a new end-to-end supervised learning framework, which can simultaneously learn semantic features and hash features to achieve fast image retrieval. Using convolution network, the error rate is reduced to 14.41% in this test set. In three open image libraries, namely Oxford, Holidays and ImageNet, the performance of traditional SIFT-based retrieval algorithms and other CNN-based image retrieval algorithms in tasks are compared and analyzed. The experimental results show that the proposed algorithm is superior to other contrast algorithms in terms of comprehensive retrieval effect and retrieval time.     

感知哈希综述

 感知哈希(Perceptual Hashing),是多媒体数据集到感知摘要集的一类单向映射,即将具有相同感知内容的多媒体数字表示唯一地映射为一段数字摘要,并满足感知鲁棒性和安全性.感知哈希为多媒体内容识别、检索、认证等信息服务方式提供安全可靠的技术支撑.本文在人类感知模型(Human Perceptual Model)的基础上,明确了感知哈希的定义、性质和一般性描述.并对目前感知哈希的典型算法、应用模式以及评测基准等进行了综述,指出了感知哈希未来的研究方向.     

Manifold-ranking embedded order preserving hashing for image semantic retrieval

Due to the storage and computational efficiency of hashing technology, it has proven a valuable tool for large scale similarity search. In many cases, the large scale data in real-world lie near some (unknown) low-dimensional and non-linear manifold. Moreover, Manifold Ranking approach can preserve the global topological structure of the data set more effectively than Euclidean Distance-based Ranking approach, which fails to preserve the semantic relevance degree. However, most existing hashing methods ignore the global topological structure of the data set. The key issue is how to incorporate the global topological structure of data set into learning effective hashing function. In this paper, we propose a novel unsupervised hashing approach, namely Manifold-Ranking Embedded Order Preserving Hashing (MREOPH). A manifold ranking loss is introduced to solve the issue of global topological structure preserving. An order preserving loss is introduced to ensure the consistency between manifold ranking and hamming ranking. A hypercubic quantization loss is introduced to learn discrete binary codes. The information theoretic regularization term is taken into consideration for preserving desirable properties of hash codes. Finally, we integrate them in a joint optimization framework for minimizing the information loss in each processing. Experimental results on three datasets for semantic search clearly demonstrate the effectiveness of the proposed method.