结合BERT和BiSRU-AT的中文文本情感分类

针对传统语言模型的词向量表示无法解决多义词表征的问题，以及现有情感分析模型不能充分捕获长距离语义信息的问题，提出了一种结合BERT和BiSRU-AT的文本情感分类模型BERT- BiSRU-AT。首先用预训练模型BERT获取融合文本语境的词向量表征；然后利用双向简单循环单元（BiSRU）二次提取语义特征和上下文信息；再利用注意力机制对BiSRU层的输出分配权重以突出重点信息;最后使用Softmax激励函数得出句子级别的情感概率分布。实验采用中文版本的推特数据集和酒店评论数据集。实验结果表明，结合BERT和BiSRU-AT的文本情感分析模型能够获得更高的准确率，双向简单循环模型和注意力机制的引入能有效提高模型的整体性能，有较大的实用价值。

Analysis of Chinese text emotions combining BERT and BiSRU-AT

To address the problems that the traditional language model cannot solve the problem of word ambiguity in word vector representation and the existing models of emotion classification cannot capture long distance semantic information, the analysis of text emotions combining BERT (Bidirectional Encoder Representation from Transformers) and BiSRU-AT is proposed. Firstly, BERT is used to obtain the word vector representation that integrates text semantics, and then BiSRU (Bidirectional Simple Recurrent Unit) is used to extract context information again. Secondly, the attention mechanism is utilized to assign corresponding weights to the outputs of BiSRU layer to highlight the key information. Finally, softmax regression is used to obtain the sentence level emotional probability distribution. Experiments are carried out on the Twitter dataset and hotel comment dataset. The results show that the ana- lysis of text emotions model combining BERT and BiSRU-AT can achieve higher accuracy, and the BiSRU model and the adoption of attention mechanism can effectively improve the overall performance of the model which is of great pragmatic value.