基于多模态子空间学习的语义标签生成方法

基于已有的视觉空间和文本空间上标签相关性建模方法，提出一种多模态子空间学习的语义标签生成方法。通过建立视觉特征相似图，以非线性方式重构“图像-标签”相关性，进而将图像的视觉模态表示和标签的文本模态表示统一到多模态子空间中，并保证空间变换前后具备结构保持。在该空间中，标签的文本模态与图像的视觉内容模态信息彼此互补，语义相关的图像和标签映射到空间中相近的样本点，进而将语义标签生成问题转换为子空间内图像的近邻标签搜索问题。结果表明，该方法在FLICKR-25K数据集上，性能达到36.88%，在NUS-WIDE数据集上，性能达到44.17%，多模态子空间学习的语义标签生成方法可以大幅度提升标签生成的准确性。

A semantictag generation method based on multi-model subspace learning

A multi-model subspace learning semantic tag generation method was proposed, whic was based on the visual space and label space tag correlation modeling method separately. This method reconstructed the "image-tag" correlation in a non-linear manner by establishing a visual feature similarity map, thereby unifying the visual modal representation of the image and the text modal representation of the tag into a multi-model subspace, and ensuring space structure preservation before and after conversion. In this space, the text modal information of the label and the modal information of the visual content of the image were complementary to each other. The semantically related images and labels were mapped to similar sample points in the space, and the semantic label generation problem was then transformed into the nearest label-neighbors retrieval problem. The results showed that the performance of the proposed method was 36.88% on FLICKR-25K data set, and 44.17% on NUS-WIDE data set, which indicated that the proposed method could greatly improve the accuracy of label generation.