基于图卷积网络的无监督跨模态哈希检索算法

针对无监督跨模态检索任务中不能充分利用单个模态内的语义关联信息的问题，提出了一种基于图卷积网络的无监督跨模态哈希检索方法。通过图像和文本编码器分别获得两个模态的特征，输入到图卷积网络中挖掘单个模态的内部语义信息，将结果通过哈希编码层进行二值化操作后，与模态间的深度语义关联相似度矩阵进行对比计算损失，不断重构优化生成的二进制编码，直到生成样本对应的健壮哈希表达。实验结果表明，与经典的浅层方法和深度学习方法对比，该方法在多个数据集上的跨模态检索准确率均有明显提升。证明通过图卷积网络能够进一步挖掘模态内的语义信息，所提模型具有更高的准确性和鲁棒性。

Graph convolutional network based unsupervised cross-modal hashing retrieval

To solve the insufficient mining problem of semantic correlation information within a single modality in the unsupervised cross-modal retrieval task, this paper proposed an unsupervised cross-modal hash retrieval(UCMHR) method based on GCN. It obtained the features of the two modalities through the image and text encoders, respectively, input the features into the GCN to exploit the single intra-modal semantic information. Then it calculated the loss by comparing with the deep semantic correlation similarity matrix, so the generated binary codes were continuously reconstructed and optimized until the robust hashing expression corresponding to the samples was generated. The experimental results show that the cross-modal retrieval accuracy of this method on multiple datasets improves significantly, compared with the classical shallow methods and deep-learning methods. It is proved that the semantic information within the modality can be further mined through the graph convolutional network, the proposed model has higher accuracy and robustness.