Multi-label semantics preserving based deep cross-modal hashing

Due to the storage and retrieval efficiency of hashing, as well as the highly discriminative feature extraction by deep neural networks, deep cross-modal hashing retrieval has been attracting increasing attention in recent years. However, most of existing deep cross-modal hashing methods simply employ single-label to directly measure the semantic relevance across different modalities, but neglect the potential contributions from multiple category labels. With the aim to improve the accuracy of cross-modal hashing retrieval by fully exploring the semantic relevance based on multiple labels of training data, in this paper, we propose a multi-label semantics preserving based deep cross-modal hashing (MLSPH) method. MLSPH firstly utilizes multi-labels of instances to calculate semantic similarity of the original data. Subsequently, a memory bank mechanism is introduced to preserve the multiple labels semantic similarity constraints and enforce the distinctiveness of learned hash representations over the whole training batch. Extensive experiments on several benchmark datasets reveal that the proposed MLSPH surpasses prominent baselines and reaches the state-of-the-art performance in the field of cross-modal hashing retrieval. Code is available at: https://github.com/SWU-CS-MediaLab/MLSPH.     

Adaptive weight multi-channel center similar deep hashing

To increase the richness of the extracted text modality feature information and deeply explore the semantic similarity between the modalities. In this paper, we propose a novel method, named adaptive weight multi-channel center similar deep hashing (AMCDH). The algorithm first utilizes three channels with different configurations to extract feature information from the text modality; and then adds them according to the learned weight ratio to increase the richness of the information. We also introduce the Jaccard coefficient to measure the semantic similarity level between modalities from 0 to 1, and utilize it as the penalty coefficient of the cross-entropy loss function to increase its role in backpropagation. Besides, we propose a method of constructing center similarity, which makes the hash codes of similar data pairs close to the same center point, and dissimilar data pairs are scattered at different center points to generate high-quality hash codes. Extensive experimental evaluations on four benchmark datasets show that the performance of our proposed model AMCDH is significantly better than other competing baselines. The code can be obtained from https://github.com/DaveLiu6/AMCDH.git.     

基于非对称监督深度离散哈希的图像检索

哈希广泛应用于图像检索任务。针对现有深度监督哈希方法的局限性,该文提出了一种新的非对称监督深度离散哈希(ASDDH)方法来保持不同类别之间的语义结构,同时生成二进制码。首先利用深度网络提取图像特征,根据图像的语义标签来揭示每对图像之间的相似性。为了增强二进制码之间的相似性,并保证多标签语义保持,该文设计了一种非对称哈希方法,并利用多标签二进制码映射,使哈希码具有多标签语义信息。此外,引入二进制码的位平衡性对每个位进行平衡,鼓励所有训练样本中的–1和+1的数目近似。在两个常用数据集上的实验结果表明,该方法在图像检索方面的性能优于其他方法。     

基于稀疏自编码的无监督哈希算法

最近邻搜索在大规模图像检索中变得越来越重要。在最近邻搜索中,许多哈希方法因为快速查询和低内存被提出。然而,现有方法在哈希函数构造过程中对数据稀疏结构研究的不足,本文提出了一种无监督的稀疏自编码的图像哈希方法。基于稀疏自编码的图像哈希方法将稀疏构造过程引入哈希函数的学习过程中,即通过利用稀疏自编码器的KL距离对哈希码进行稀疏约束以增强局部保持映射过程中的判别性,同时利用L2范数来哈希编码的量化误差。实验中用两个公共图像检索数据集CIFAR-10和YouTube Faces验证了本文算法相比其他无监督哈希算法的优越性。     

Spline regression hashing for fast image search

Techniques for fast image retrieval over large databases have attracted considerable attention due to the rapid growth of web images. One promising way to accelerate image search is to use hashing technologies, which represent images by compact binary codewords. In this way, the similarity between images can be efficiently measured in terms of the Hamming distance between their corresponding binary codes. Although plenty of methods on generating hash codes have been proposed in recent years, there are still two key points that needed to be improved: 1) how to precisely preserve the similarity structure of the original data and 2) how to obtain the hash codes of the previously unseen data. In this paper, we propose our spline regression hashing method, in which both the local and global data similarity structures are exploited. To better capture the local manifold structure, we introduce splines developed in Sobolev space to find the local data mapping function. Furthermore, our framework simultaneously learns the hash codes of the training data and the hash function for the unseen data, which solves the out-of-sample problem. Extensive experiments conducted on real image datasets consisting of over one million images show that our proposed method outperforms the state-of-the-art techniques.     

各向同性的迭代量化哈希算法

准确有效的哈希算法是实现海量高维数据近邻检索的关键.迭代量化哈希（Iterative Quantization,ITQ）和各向同性哈希（Isotropic Hash,IsoHash）是两种知名的编码方法.但是ITQ算法对旋转矩阵施加的约束过于单薄,容易导致过拟合;而IsoHash算法缺乏对哈希编码的更新策略,降低了编码质量.针对上述问题,提出了一种各向同性的迭代量化哈希算法.该方法采用迭代的策略,对编码矩阵和旋转矩阵交替更新,并在正交约束的基础上增加各向同性约束来学习最优旋转矩阵,最小化量化误差.在CIFAR-10、22K LabelMe和ANN_GIST_1M基准库上与多种方法进行对比,实验结果表明本文算法在查准率、查全率以及平均准确率均值等指标上均明显优于对比算法.     

基于深度卷积神经网络和二进制哈希学习的图像检索方法

随着图像数据的迅猛增长,当前主流的图像检索方法采用的视觉特征编码步骤固定,缺少学习能力,导致其图像表达能力不强,而且视觉特征维数较高,严重制约了其图像检索性能。针对这些问题,该文提出一种基于深度卷积神径网络学习二进制哈希编码的方法,用于大规模的图像检索。该文的基本思想是在深度学习框架中增加一个哈希层,同时学习图像特征和哈希函数,且哈希函数满足独立性和量化误差最小的约束。首先,利用卷积神经网络强大的学习能力挖掘训练图像的内在隐含关系,提取图像深层特征,增强图像特征的区分性和表达能力。然后,将图像特征输入到哈希层,学习哈希函数使得哈希层输出的二进制哈希码分类误差和量化误差最小,且满足独立性约束。最后,给定输入图像通过该框架的哈希层得到相应的哈希码,从而可以在低维汉明空间中完成对大规模图像数据的有效检索。在3个常用数据集上的实验结果表明,利用所提方法得到哈希码,其图像检索性能优于当前主流方法。     

Unsupervised discriminative hashing

Hashing is one of the popular solutions for approximate nearest neighbor search because of its low storage cost and fast retrieval speed, and many machine learning algorithms are adapted to learn effective hash function. As hash codes of the same cluster are similar to each other while the hash codes in different clusters are dissimilar, we propose an unsupervised discriminative hashing learning method (UDH) to improve discrimination among hash codes in different clusters. UDH shares a similar objective function with spectral hashing algorithm, and uses a modified graph Laplacian matrix to exploit local discriminant information. In addition, UDH is designed to enable efficient out-of-sample extension. Experiments on real world image datasets demonstrate the effectiveness of our novel approach for image retrieval.     

A novel unsupervised multiple feature hashing for image retrieval and indexing (MFHIRI)

Recently, techniques that can automatically figure out the incisive information from gigantic visual databases are urging popularity. The existing multi-feature hashing method has achieved good results by fusing multiple features, but in processing these multi-features, fusing multi-features into one feature will cause the feature dimension to be very high, increasing the amount of calculation. On the one hand, it is not easy to discover the internal ties between different features. This paper proposes a novel unsupervised multiple feature hashing for image retrieval and indexing (MFHIRI) method to learn multiple views in a composite manner. The proposed scheme learns the binary codes of various information sources in a composite manner, and our scheme relies on weighted multiple information sources and improved KNN concept. In particular, here we adopt an adaptive weighing scheme to preserve the similarity and consistency among binary codes. Precisely, we follow the graph modeling theory to construct improved KNN concept, which further helps preserve different statistical properties of individual sources. The important aspect of improved KNN scheme is that we can find the neighbors of a data point by searching its neighbors’ neighbors. During optimization, the sub-problems are solved in parallel which efficiently lowers down the computation cost. The proposed approach shows consistent performance over state-of-the-art (three single-view and eight multi-view approaches) on three broadly followed datasets viz. CIFAR-10, NUS-WIDE and Caltech-256.     

基于加权自学习散列的高维数据最近邻查询算法

因为查询和存储具有高效性,学习型散列逐渐被应用于解决最近邻查询问题.学习型散列将高维数据转化成二进制编码,并使得原始高维空间中越相似的数据对应二进制编码的汉明距离越小.在实际应用中,每次查询都会返回许多与查询点汉明距离相同而编码互不相同的数据.如何对这些数据进行排序是一个难题.提出了一种基于加权自学习散列的近邻查找算法.实验结果表明,算法能够高效地对具有相同汉明距离的不同编码进行重排序,加权排序后查询的F1值约是原来的2倍并优于同系算法,时间开销可比直接计算原始距离进行排序降低一个数量级.     

Deep multi-query video retrieval

Video retrieval methods have been developed for a single query. Multi-query video retrieval problem has not been investigated yet. In this study, an efficient and fast multi-query video retrieval framework is developed. Query videos are assumed to be related to more than one semantic. The framework supports an arbitrary number of video queries. The method is built upon using binary video hash codes. As a result, it is fast and requires a lower storage space. Database and query hash codes are generated by a deep hashing method that not only generates hash codes but also predicts query labels when they are chosen outside the database. The retrieval is based on the Pareto front multi-objective optimization method. Re-ranking performed on the retrieved videos by using non-binary deep features increases the retrieval accuracy considerably. Simulations carried out on two multi-label video databases show that the proposed method is efficient and fast in terms of retrieval accuracy and time.     

Manifold-ranking embedded order preserving hashing for image semantic retrieval

Due to the storage and computational efficiency of hashing technology, it has proven a valuable tool for large scale similarity search. In many cases, the large scale data in real-world lie near some (unknown) low-dimensional and non-linear manifold. Moreover, Manifold Ranking approach can preserve the global topological structure of the data set more effectively than Euclidean Distance-based Ranking approach, which fails to preserve the semantic relevance degree. However, most existing hashing methods ignore the global topological structure of the data set. The key issue is how to incorporate the global topological structure of data set into learning effective hashing function. In this paper, we propose a novel unsupervised hashing approach, namely Manifold-Ranking Embedded Order Preserving Hashing (MREOPH). A manifold ranking loss is introduced to solve the issue of global topological structure preserving. An order preserving loss is introduced to ensure the consistency between manifold ranking and hamming ranking. A hypercubic quantization loss is introduced to learn discrete binary codes. The information theoretic regularization term is taken into consideration for preserving desirable properties of hash codes. Finally, we integrate them in a joint optimization framework for minimizing the information loss in each processing. Experimental results on three datasets for semantic search clearly demonstrate the effectiveness of the proposed method.     

Discrete hashing with triple supervision learning

In recent years, discrete supervised hashing methods have attracted increasing attention because of their high retrieval efficiency and precision. However, in these methods, some effective semantic information is typically neglected, which means that all the information is not sufficiently utilized. Moreover, these methods often only decompose the first-order features of the original data, ignoring the more fine-grained higher-order features. To address these problems, we propose a supervised hashing learning method called discrete hashing with triple supervision learning (DHTSL). Specifically, we integrate three aspects of semantic information into this method: (1) the bidirectional mapping of semantic labels; (2) pairwise similarity relations; (3) second-order features from the original data. We also design a discrete optimization method to solve the proposed objective function. Moreover, an out-of-sample extension strategy that can better maintain the independence and balance of hash codes is employed to improve retrieval performance. Extensive experiments on three widely used datasets demonstrate its superior performance.     

Random Gray code and its performance analysis for image hashing

The discrete-binary conversion stage, which plays the role of converting quantized hash vectors into binary hash strings by encoding, is one of the most important parts of authentication-oriented image hashing. However, very few works have been done on the discrete-binary conversion stage. In this paper, based on Gray code, we propose a key-dependent code called random Gray (RGray) code for image hashing, which, according to our theoretical analysis and experimental results, is likely to increase the security of image hashing to some extent and meanwhile maintains the performance of Gray code in terms of the tradeoff between robustness and fragility. We also apply a measure called distance distortion, which was proposed by Rothlauf (2002) [1] for evolutionary search, to investigate the influence of the discrete-binary conversion stage on the performance of image hashing. Based on distance distortion, we present a theoretical comparison of the encodings applied in the discrete-binary conversion stage of image hashing, including RGray encoding. And our experimental results validate the practical applicability of distance distortion on the performance evaluation of the discrete-binary conversion stage.     

基于深度学习的散列检索技术研究进展

大数据时代，数据呈现维度高、数据量大和增长快等特点。面对大量的复杂数据，如何高效地检索相似近邻数据是近似最近邻查询的研究热点。散列技术通过将数据映射为二进制码的方式，能够显著加快相似性计算，并在检索过程中节省存储和通信开销。近年来深度学习在提取数据特征方面表现出速度快、精度高等优异的性能，使得基于深度学习的散列检索技术得到越来越广泛的运用。总结了深度学习散列的主要方法和前沿进展，并对未来的研究方向展开简要探讨。     

CDGAN: Cyclic Discriminative Generative Adversarial Networks for image-to-image transformation

Generative Adversarial Networks (GANs) have facilitated a new direction to tackle the image-to-image transformation problem. Different GANs use generator and discriminator networks with different losses in the objective function. Still there is a gap to fill in terms of both the quality of the generated images and close to the ground truth images. In this work, we introduce a new Image-to-Image Transformation network named Cyclic Discriminative Generative Adversarial Networks (CDGAN) that fills the above mentioned gaps. The proposed CDGAN generates high quality and more realistic images by incorporating the additional discriminator networks for cycled images in addition to the original architecture of the CycleGAN. The proposed CDGAN is tested over three image-to-image transformation datasets. The quantitative and qualitative results are analyzed and compared with the state-of-the-art methods. The proposed CDGAN method outperforms the state-of-the-art methods when compared over the three baseline Image-to-Image transformation datasets. The code is available at https://github.com/KishanKancharagunta/CDGAN.