一种融合注意力机制的自适应实体识别方法

为了解决命名实体识别任务在面向新兴应用领域时,需要面对烦琐的模型重构过程和语料严重不足的问题,该文提出了一种基于注意力机制的领域自适应命名实体识别方法。首先,在通用领域数据集上构建了基于BERT(bidirectional encoder representations from transformers)预训练语言模型的双向长短时记忆条件随机场(BERT-BiLSTM-CRF)命名实体识别模型;接着,在古代汉语语料集上对原有模型进行微调的同时插入了基于注意力机制的自适应神经网络层;最后,在目标域内应用迁移学习方法训练模型进行对比实验。实验结果表明,自适应迁移学习方法减少了对目标域语料的依赖。该文提出的基于注意力机制的自适应神经网络模型相比通用域BERT-BiLSTM-CRF模型的命名实体识别结果F₁值提高了4.31%,相比古代汉语域BERT-BiLSTM-CRF模型的命名实体识别结果F₁值提高了2.46%,实验表明,该文方法能够提升源域模型迁移学习的效果,并完成跨领域命名实体识别模型的构建。

An Adaptive Entity Recognition Method with Attention Mechanism

To deal with model reconstruction process and the lack of training data for various domains in the task of named entity recognition, a domain adaptive named entity recognition method is proposed based on attention mechanism. Firstly, a bidirectional long-short term memory conditional random field named entity recognition model based on the BERT (BERT-BiLSTM-CRF)is constructed on the general dataset. Then, such-bulit model is fine-tuned using the ancient Chinese corpus, with an adaptive neural network layer based on the attention mechanism inserted. The comparison experiment is set with the model in the target domain and the existing transfer learning method. The experimental results show that the proposed model improves the F₁ value by 4.31% compared with the generic domain BERT-BiLSTM-CRF model, by 2.46% compared with the same model trained only on the ancient Chinese domain corpus.