面向图文匹配任务的多层次图像特征融合算法

现有主流的利用预训练卷积神经网络提取图像特征的方法存在仅使用单层预训练特征表征图像和预训练任务与实际研究任务不一致的问题，使得现有图文匹配方法无法充分利用图像特征，极易受到噪声特征干扰。针对上述问题，使用了预训练网络中的多层特征，并提出了多层次图像特征融合算法。在图文匹配的学习目标指导下，利用多层感知机（multi-layer perceptron）有监督地融合和降维多层次的预训练图像特征，生成融合图像特征，从而充分利用预训练特征，减少噪声干扰。实验结果表明，提出的融合算法可实现对预训练的图像特征更有效的利用，相比于使用单层次特征的方法能获得更好的图文匹配效果。

Fusion of multi-level image features for image-text matching

The existing mainstream methods use the pre-trained convolutional neural networks to extract image features and usually have the following limitations: a) only using a single layer of pre-trained features to represent image; b) the pre-trained task is inconsistent with the actual research task. These limitations result in that the existing methods of image-text matching cannot make full use of image features and is easily influenced by the noises. To solve the above limitations, this paper used multi-layer features from a pre-trained network and proposed a fusion algorithm of multi-level image features accordingly. Under the guidance of the image-text matching objective function, the proposed algorithm fused the multi-level pre-trained image features and reduced their dimensionality using a multi-layer perceptron to generate fusion features. It was able to make full use of pre-trained features and successfully reduce the influences of noises. The experimental results show that the proposed fusion algorithm makes better use of pre-trained image features and outperforms the methods using single-level features in the image-text matching task.