FMNN:融合多神经网络的文本分类模型

文本分类是自然语言处理中一项基本且重要的任务。基于深度学习的文本分类方法大多只针对单一的模型结构进行深入研究,这种单一的结构缺乏同时捕获并利用全局语义特征与局部语义特征的能力,且网络的加深会损失更多的语义信息。对此,提出了一种融合多神经网络的文本分类模型FMNN(A Text Classification Model Fused with Multiple Neural Network),FMNN在最大限度减小网络深度的同时,融合了BERT,RNN,CNN和Attention等神经网络模型的特性。用BERT作为嵌入层获得文本的矩阵表示,用BiLSTM和Attention联合提取文本的全局语义特征,用CNN提取文本多个粒度下的局部语义特征,将全局语义特征和局部语义特征分别作用于softmax分类器,最后采用算术平均的方式对结果进行融合。在3个公开数据集和1个司法数据集上的实验结果表明,FMNN模型实现了更高的文本分类准确率,其中在司法数据集上的准确率达到了90.31%,证明了该模型具有较好的实用价值。

FMNN:Text Classification Model Fused with Multiple Neural Networks

Text classification is a basic and important task in natural language processing.Most of the text classification methods based on deep learning only focus on a single model structure.The single structure lacks the ability to simultaneously capture and utilize both global and local semantic features.Besides,the deepening of the network will lose more semantic information.In order to overcome the above problems,a text classification model FMNN which is a text classification model fused with multiple neural network is proposed in this paper.The model combines the performances of BERT,RNN,CNN and Attention while minimizing the network depth.BERT is used as the embedding layer to obtain the matrix representation of the text.BiLSTM and Attention are used to jointly extract the global semantic features of the text.CNN is used to extract the local semantic features of the text at multiple granularities.The global semantic features and local semantic features are applied to the softmax classifier respectively.The results are finally fused by arithmetic average.The experimental results on three public data sets and one judicial data set show that the proposed FMNN model achieves higher accuracy rate,and the accuracy rate on the judicial data set reaches 90.31%,which proves that the model has good practical value.