图卷积网络与自注意机制在文本分类任务上的对比分析

图卷积网络近年来受到大量关注，同时自注意机制作为Transformer结构及众多预训练模型的核心之一也得到广泛运用。该文从原理上分析发现，自注意机制可视为图卷积网络的一种泛化形式，其以所有输入样本为节点，构建有向全连接图进行卷积，且节点间连边权重可学。在多个文本分类数据集上的对比实验一致显示，使用自注意机制的模型较使用图卷积网络的对照模型分类效果更佳，甚至超过了目前图卷积网络用于文本分类任务的最先进水平，并且随着数据规模的增大，两者分类效果的差距也随之扩大。这些证据表明，自注意力机制更具表达能力，在文本分类任务上能够相对图卷积网络带来分类效果的提升。

A Comparative Study of Graph Convolutional Networks and Self-Attention Mechanism on Text Classification

Graph Convolutional Networks has drawn much attention recently, and the self-attention mechanism has been widely applied as the core of the Transformer and many pre-trained models. We disclose that the self attention mechanism can be seen as a generalization of Graph Convolutional Networks, in that it takes all input samples as nodes and then constructs a directed fully connected graph with learnable edge weights for convolution. Experiments show that the self attention mechanism achieves better text classification accuracy than many state of the art Graph Convolutional Networks. Meanwhile, the performance gap of classification widens as the data size increases. These show that the self-attention mechanism is more expressive, and may surpass Graph Convolutional Networks with potential performance improvements on the task of text classification.