基于非对称监督深度离散哈希的图像检索

哈希广泛应用于图像检索任务。针对现有深度监督哈希方法的局限性,该文提出了一种新的非对称监督深度离散哈希(ASDDH)方法来保持不同类别之间的语义结构,同时生成二进制码。首先利用深度网络提取图像特征,根据图像的语义标签来揭示每对图像之间的相似性。为了增强二进制码之间的相似性,并保证多标签语义保持,该文设计了一种非对称哈希方法,并利用多标签二进制码映射,使哈希码具有多标签语义信息。此外,引入二进制码的位平衡性对每个位进行平衡,鼓励所有训练样本中的–1和+1的数目近似。在两个常用数据集上的实验结果表明,该方法在图像检索方面的性能优于其他方法。     

Multi-label semantics preserving based deep cross-modal hashing

Due to the storage and retrieval efficiency of hashing, as well as the highly discriminative feature extraction by deep neural networks, deep cross-modal hashing retrieval has been attracting increasing attention in recent years. However, most of existing deep cross-modal hashing methods simply employ single-label to directly measure the semantic relevance across different modalities, but neglect the potential contributions from multiple category labels. With the aim to improve the accuracy of cross-modal hashing retrieval by fully exploring the semantic relevance based on multiple labels of training data, in this paper, we propose a multi-label semantics preserving based deep cross-modal hashing (MLSPH) method. MLSPH firstly utilizes multi-labels of instances to calculate semantic similarity of the original data. Subsequently, a memory bank mechanism is introduced to preserve the multiple labels semantic similarity constraints and enforce the distinctiveness of learned hash representations over the whole training batch. Extensive experiments on several benchmark datasets reveal that the proposed MLSPH surpasses prominent baselines and reaches the state-of-the-art performance in the field of cross-modal hashing retrieval. Code is available at: https://github.com/SWU-CS-MediaLab/MLSPH.     

Beyond ITQ: Efficient binary multi-view subspace learning for instance retrieval

The existing hashing methods mainly handle either the feature based nearest-neighbor search or the category-level image retrieval, whereas a few efforts are devoted to instance retrieval problem. In this paper, we propose a binary multi-view fusion framework for directly recovering a latent Hamming subspace from the multi-view features for instance retrieval. More specifically, the multi-view subspace reconstruction and the binary quantization are integrated in a unified framework so as to minimize the discrepancy between the original multi-view high-dimensional Euclidean space and the resulting compact Hamming subspace. Besides, our method is essentially an unsupervised learning scheme without any labeled data involved, and thus can be used in the cases when the supervised information is unavailable or insufficient. Experiments on public benchmark and large-scale datasets reveal that our method achieves competitive retrieval performance comparable to the state-of-the-arts and has excellent scalability in large-scale scenario.     

Deep supervised hashing using quadratic spherical mutual information for efficient image retrieval

Several deep supervised hashing techniques have been proposed to allow for extracting compact and efficient neural network representations for various tasks. However, many deep supervised hashing techniques ignore several information-theoretic aspects of the process of information retrieval, often leading to sub-optimal results. In this paper, we propose an efficient deep supervised hashing algorithm that optimizes the learned compact codes using an information-theoretic measure, the Quadratic Mutual Information (QMI). The proposed method is adapted to the needs of efficient image hashing and information retrieval leading to a novel information-theoretic measure, the Quadratic Spherical Mutual Information (QSMI). Apart from demonstrating the effectiveness of the proposed method under different scenarios and outperforming existing state-of-the-art image hashing techniques, this paper provides a structured way to model the process of information retrieval and develop novel methods adapted to the needs of different applications.     

An Indexing and Retrieval Mechanism for Complex Similarity Queries in Image Databases

A content-based image retrieval mechanism to support complex similarity queries is presented. The image content is defined by three kinds of features: quantifiable features describing the visual information, nonquantifiable features describing the semantic information, and keywords describing more abstract semantic information. In correspondence with these feature sets, we construct three types of indexes: visual indexes, semantic indexes, and keyword indexes. Index structures are elaborated to provide effective and efficient retrieval of images based on their contents. The underlying index structure used for all indexes is the HG-tree. In addition to the HG-tree, the signature file and hashing technique are also employed to index keywords and semantic features. The proposed indexing scheme combines and extends the HG-tree, the signature file, and the hashing scheme to support complex similarity queries. We also propose a new evaluation strategy to process the complex similarity queries. Experiments have been carried out on large image collections to demonstrate the effectiveness of the proposed retrieval mechanism.     

Manifold-ranking embedded order preserving hashing for image semantic retrieval

Due to the storage and computational efficiency of hashing technology, it has proven a valuable tool for large scale similarity search. In many cases, the large scale data in real-world lie near some (unknown) low-dimensional and non-linear manifold. Moreover, Manifold Ranking approach can preserve the global topological structure of the data set more effectively than Euclidean Distance-based Ranking approach, which fails to preserve the semantic relevance degree. However, most existing hashing methods ignore the global topological structure of the data set. The key issue is how to incorporate the global topological structure of data set into learning effective hashing function. In this paper, we propose a novel unsupervised hashing approach, namely Manifold-Ranking Embedded Order Preserving Hashing (MREOPH). A manifold ranking loss is introduced to solve the issue of global topological structure preserving. An order preserving loss is introduced to ensure the consistency between manifold ranking and hamming ranking. A hypercubic quantization loss is introduced to learn discrete binary codes. The information theoretic regularization term is taken into consideration for preserving desirable properties of hash codes. Finally, we integrate them in a joint optimization framework for minimizing the information loss in each processing. Experimental results on three datasets for semantic search clearly demonstrate the effectiveness of the proposed method.     

A penalty function semi-continuous thresholding methods for constraints of hashing problems

The Hashing process is an effective tool for handling large-scale data (for example, images, videos, or multi-model data) retrieval problems. To get better retrieval accuracy, hashing models usually are imposed with three rigorous constraints, i.e., discrete binary constraint, uncorrelated condition, and the balanced constraint, which will lead to being ‘NP-hard’. In this study, we divide the whole constraints set into the uncorrelated (orthogonality) constraint and the binary discrete balance constraint and propose a fast and accurate penalty function semi-continuous thresholding (PFSCT) hash coding algorithm based on forward–backward algorithms. In addition, we theoretically analyze the equivalence between the relaxed model and the original problems. Extensive numerical experiments on diverse large-scale benchmark datasets demonstrate comparable performance and effectiveness of the proposed method.     

一种基于加速近端梯度法的视频散列算法研究

散列算法已经被广泛应用于视频数据的索引。然而,当前大多数视频散列方法将视频看成是多个独立帧的简单集合,通过综合帧的索引来对每个视频编制索引,在设计散列函数时往往忽略了视频的结构信息。首先将视频散列问题建模为结构正规化经验损失的最小化问题。然后提出一种有监管算法,通过利用结构学习方法来设计高效的散列函数。其中,结构正规化利用了出现于视频帧(与相同的语义类别存在关联)中的常见局部视觉模式,同时对来自同一视频的后续帧保持时域一致性。证明了通过使用加速近端梯度(APG)法可有效求解最小化目标问题。最后,基于两个大规模基准数据集展开全面实验(150 000个视频片断,1 200万帧),实验结果证明了该方法性能优于当前其他算法。     

Research on image feature extraction and retrieval algorithms based on convolutional neural network

With the rapid development of mobile Internet and digital technology, people are more and more keen to share pictures on social networks, and online pictures have exploded. How to retrieve similar images from large-scale images has always been a hot issue in the field of image retrieval, and the selection of image features largely affects the performance of image retrieval. The Convolutional Neural Networks (CNN), which contains more hidden layers, has more complex network structure and stronger ability of feature learning and expression compared with traditional feature extraction methods. By analyzing the disadvantage that global CNN features cannot effectively describe local details when they act on image retrieval tasks, a strategy of aggregating low-level CNN feature maps to generate local features is proposed. The high-level features of CNN model pay more attention to semantic information, but the low-level features pay more attention to local details. Using the increasingly abstract characteristics of CNN model from low to high. This paper presents a probabilistic semantic retrieval algorithm, proposes a probabilistic semantic hash retrieval method based on CNN, and designs a new end-to-end supervised learning framework, which can simultaneously learn semantic features and hash features to achieve fast image retrieval. Using convolution network, the error rate is reduced to 14.41% in this test set. In three open image libraries, namely Oxford, Holidays and ImageNet, the performance of traditional SIFT-based retrieval algorithms and other CNN-based image retrieval algorithms in tasks are compared and analyzed. The experimental results show that the proposed algorithm is superior to other contrast algorithms in terms of comprehensive retrieval effect and retrieval time.     

A novel unsupervised multiple feature hashing for image retrieval and indexing (MFHIRI)

Recently, techniques that can automatically figure out the incisive information from gigantic visual databases are urging popularity. The existing multi-feature hashing method has achieved good results by fusing multiple features, but in processing these multi-features, fusing multi-features into one feature will cause the feature dimension to be very high, increasing the amount of calculation. On the one hand, it is not easy to discover the internal ties between different features. This paper proposes a novel unsupervised multiple feature hashing for image retrieval and indexing (MFHIRI) method to learn multiple views in a composite manner. The proposed scheme learns the binary codes of various information sources in a composite manner, and our scheme relies on weighted multiple information sources and improved KNN concept. In particular, here we adopt an adaptive weighing scheme to preserve the similarity and consistency among binary codes. Precisely, we follow the graph modeling theory to construct improved KNN concept, which further helps preserve different statistical properties of individual sources. The important aspect of improved KNN scheme is that we can find the neighbors of a data point by searching its neighbors’ neighbors. During optimization, the sub-problems are solved in parallel which efficiently lowers down the computation cost. The proposed approach shows consistent performance over state-of-the-art (three single-view and eight multi-view approaches) on three broadly followed datasets viz. CIFAR-10, NUS-WIDE and Caltech-256.     

Deep multi query image retrieval

There exist few studies investigating the multi-query image retrieval problem. Existing methods are not based on hash codes. As a result, they are not efficient and fast. In this study, we develop an efficient and fast multi-query image retrieval method when the queries are related to more than one semantic. Image hash codes are generated by a deep hashing method. Consequently, the method requires lower storage space, and it is faster compared to the existing methods. The retrieval is based on the Pareto front method. Reranking performed on the retrieved images by using non-binary deep-convolutional features increase retrieval accuracy considerably. Unlike previous studies, the method supports an arbitrary number of queries. It outperforms similar multi-query image retrieval studies in terms of retrieval time and retrieval accuracy.     

Semi-supervised cross-modal representation learning with GAN-based Asymmetric Transfer Network

In this paper, we proposed a semi-supervised common representation learning method with GAN-based Asymmetric Transfer Network (GATN) for cross modality retrieval. GATN utilizes the asymmetric pipeline to guarantee the semantic consistency and adopt (Generative Adversarial Network) GAN to fit the distributions of different modalities. Specifically, the common representation learning across modalities includes two stages: (1) the first stage, GATN trains source mapping network to learn the semantic representation of text modality by supervised method; and (2) the second stage, GAN-based unsupervised modality transfer method is proposed to guide the training of target mapping network, which includes generative network (target mapping network) and discriminative network. Experimental results on three widely-used benchmarks show that GATN have achieved better performance comparing with several existing state-of-the-art methods.     

基于深度哈希算法的极光图像分类与检索方法

面对形态万千、变化复杂的海量极光数据,对其进行分类与检索为进一步研究地球磁场物理机制和空间信息具有重要意义。该文基于卷积神经网络(CNN)对图像特征提取方面的良好表现,以及哈希编码可以满足大规模图像检索对检索时间的要求,提出一种端到端的深度哈希算法用于极光图像分类与检索。首先在CNN中嵌入空间金字塔池化(SPP)和幂均值变换(PMT)来提取图像中多种尺度的区域信息;其次在全连接层之间加入哈希层,将全连接层最能表现图像的高维语义信息映射为紧凑的二值哈希码,并在低维空间使用汉明距离对图像对之间的相似性进行度量;最后引入多任务学习机制,充分利用图像标签信息和图像对之间的相似度信息来设计损失函数,联合分类层和哈希层的损失作为优化目标,使哈希码之间可以保持更好的语义相似性,有效提升了检索性能。在极光数据集和 CIFAR-10 数据集上的实验结果表明,所提出方法检索性能优于其他现有检索方法,同时能够有效用于极光图像分类。     

基于深度卷积神经网络和二进制哈希学习的图像检索方法

随着图像数据的迅猛增长,当前主流的图像检索方法采用的视觉特征编码步骤固定,缺少学习能力,导致其图像表达能力不强,而且视觉特征维数较高,严重制约了其图像检索性能。针对这些问题,该文提出一种基于深度卷积神径网络学习二进制哈希编码的方法,用于大规模的图像检索。该文的基本思想是在深度学习框架中增加一个哈希层,同时学习图像特征和哈希函数,且哈希函数满足独立性和量化误差最小的约束。首先,利用卷积神经网络强大的学习能力挖掘训练图像的内在隐含关系,提取图像深层特征,增强图像特征的区分性和表达能力。然后,将图像特征输入到哈希层,学习哈希函数使得哈希层输出的二进制哈希码分类误差和量化误差最小,且满足独立性约束。最后,给定输入图像通过该框架的哈希层得到相应的哈希码,从而可以在低维汉明空间中完成对大规模图像数据的有效检索。在3个常用数据集上的实验结果表明,利用所提方法得到哈希码,其图像检索性能优于当前主流方法。     

WQ:基于权重求解的哈希算法

由于最近邻查询算法一般需要较高时间和空间代价,往往不能满足大数据查询的需要.哈希技术可以大幅度减少查询时间和存储空间,其主要思想是将原始空间中的高维数据映射成为一组编码,且满足保相似性原则.现有的大部分哈希方法一般认为哈希编码的各维度权重相同.然而在实际情况中,不同的维度往往携带有不同的信息.为此,本文提出了新的算法,为编码的每个维度分配权重,并提出了对应的量化编码方式.理论证明了算法的可行性,在真实数据集下与其他哈希算法对比实验也验证了该算法的有效性.     

Effective and efficient indexing in cross-modal hashing-based datasets

To overcome the barrier of storage and computation, the hashing technique has been widely used for nearest neighbor search in multimedia retrieval applications recently. Particularly, cross-modal retrieval that searches across different modalities becomes an active but challenging problem. Although numerous of cross-modal hashing algorithms are proposed to yield compact binary codes, exhaustive search is impractical for large-scale datasets, and Hamming distance computation suffers inaccurate results. In this paper, we propose a novel search method that utilizes a probability-based index scheme over binary hash codes in cross-modal retrieval. The proposed indexing scheme employs a few binary bits from the hash code as the index code. We construct an inverted index table based on the index codes, and train a neural network for ranking and indexing to improve the retrieval accuracy. Experiments are performed on two benchmark datasets for retrieval across image and text modalities, where hash codes are generated and compared with several state-of-the-art cross-modal hashing methods. Results show the proposed method effectively boosts the performance on search accuracy, computation cost, and memory consumption in these datasets and hashing methods. The source code is available on https://github.com/msarawut/HCI.     

基于随机化视觉词典组和上下文语义信息的目标检索方法

 传统的视觉词典法(Bag of Visual Words,BoVW)具有时间效率低、内存消耗大以及视觉单词同义性和歧义性的问题,且当目标区域所包含的信息不能正确或不足以表达用户检索意图时就得不到理想的检索结果.针对这些问题,本文提出了基于随机化视觉词典组和上下文语义信息的目标检索方法.首先,该方法采用精确欧氏位置敏感哈希(Exact Euclidean Locality Sensitive Hashing,E²LSH)对局部特征点进行聚类,生成一组支持动态扩充的随机化视觉词典组;然后,利用查询目标及其周围的视觉单元构造包含上下文语义信息的目标模型;最后,引入K-L散度(Kullback-Leibler divergence)进行相似性度量完成目标检索.实验结果表明,新方法较好地提高了目标对象的可区分性,有效地提高了检索性能.     

基于改进哈希算法的图像检索方法

当前主流图像检索技术所采用的传统视觉特征编码缺少足够的学习能力,影响学习得到的特征表达能力。此外,由于视觉特征维数高,会消耗大量的内存,因此降低了图像检索的性能。文中基于深度卷积神经网络与改进的哈希算法,提出并设计了一种端到端训练方式的图像检索方法。该方法将卷积神经网络提取的高层特征和哈希函数相结合,学习到具有足够表达能力的哈希特征,从而在低维汉明空间中完成对图像数据的大规模检索。在两个常用数据集上的实验结果表明,所提出的哈希图像检索方法的检索性能优于当前的一些主流方法。     

基于深度学习的散列检索技术研究进展

大数据时代，数据呈现维度高、数据量大和增长快等特点。面对大量的复杂数据，如何高效地检索相似近邻数据是近似最近邻查询的研究热点。散列技术通过将数据映射为二进制码的方式，能够显著加快相似性计算，并在检索过程中节省存储和通信开销。近年来深度学习在提取数据特征方面表现出速度快、精度高等优异的性能，使得基于深度学习的散列检索技术得到越来越广泛的运用。总结了深度学习散列的主要方法和前沿进展，并对未来的研究方向展开简要探讨。