基于交叉注意力机制的多视图项目文本分类方法

科研项目文本的分类往往需要耗费巨大的人力、物力,因此采用智能方法实现对项目文本分类意义重大。文本分类方法的核心在于文本语义特征的提取,高效的特征提取方法有助于准确构建文本到类别之间的映射。已有的文本分类方法往往基于整个文本或者一部分文本作为分类依据,可能出现信息的冗余或缺失。该文针对结构化的项目文本,在BERT等预训练网络的基础上,创新性地提出基于单交叉注意力机制的两视图项目文本分类学习方法(Two-View Cross Attention, TVCA)和基于双交叉注意力机制的多视图项目文本分类学习方法(Multi-View Cross Attention, MVCA)。MVCA方法基于项目文本的一个主要视图(项目摘要)和两个辅助视图(研究内容、目的和意义),通过两个交叉注意力机制提取包含更丰富语义信息的特征向量,进一步改善分类模型的性能。我们将TVCA和MVCA方法应用于英文论文数据Web of Science Meta-data和南方电网科技项目文本的分类任务中,实验结果验证了TVCA和MVCA方法无论从分类效果还是收敛速度上,都明显优于已有的比较方法。

Multi-view Project Text Classification Based on Cross-attention

Aiming at structured scientific research project text, this paper proposes a novel two-view cross attention (TVCA) and multi-view cross attention text classification method (Multi-View Cross Attention, MVCA) based on pre-trained networks such as BERT. The MVCA method is targeted at one main important chapter (project abstract) and two chapters of the project text (research content, research purpose and meaning), extracting feature vectors containing richer semantic information through a cross-attention mechanism to further improve the performance of the classification model. Applied to the classification tasks of scientific publications and research project texts of China Southern Power Grid, the MVCA method is significantly better than the existing methods in terms of classification effect and convergence speed.