基于多模态表示学习的知识库补全

目前大多数知识图谱表示学习只考虑实体和关系之间的结构知识,性能受存储知识的限制,造成知识库补全能力不稳定,而融入外部信息的知识表示方法大多只针对某一特定的外部模态信息建模,适用范围有限.因此,文中提出带有注意力模块的卷积神经网络模型.首先,考虑文本和图像两种外部模态信息,提出三种融合外部模态信息和实体的方案,获得实体的多模态表示.再通过结合通道注意力模块和空间注意力模块,增强卷积的表现力,提高知识表示的质量,提升模型的补全能力.在多个公开的多模态数据集上进行链路预测和三元组分类实验,结果表明文中模型性能较优.

Knowledge Base Completion Based on Multimodal Representation Learning

In most learning models for knowledge graph representation,only structural knowledge between entities and relations is taken into account.Therefore,the capability of the models is limited by knowledge storage,and the completion performance of knowledge base is unstable.Existing knowledge representation methods incorporating external information mostly model for a specific kind of external modal information,leading to limited application scopes.Thus,a knowledge representation learning model,Conv-AT,is proposed.Firstly,two external modes of information,text and images,are considered,and three schemes fusing external knowledge and entities are introduced to obtain multimodal representation of entities.Secondly,the performance of convolution is enhanced and the quality of knowledge representation as well as the completion ability of the model are improved by combining the channel attention module and spatial attention module.Link prediction and triple classification experiments are conducted on public multimodal datasets,and the results show that the proposed method is superior to other methods.