基于主动学习的命名实体识别算法

命名实体识别的目的是识别文本中的实体指称的边界和类别.在进行命名实体识别模型训练的过程中,通常需要大量的标注样本.本文通过实现有效的选择算法,从大量样本中选择适合模型更新的样本,减少对样本的标注工作.通过5组对比实验,验证使用有效的选择算法能够获得更好的样本集,实现具有针对性的标注样本.通过设计在微博网络数据集上的实验...

Named Entity Recognition Algorithm Based on Active Learning

The purpose of named entity recognition is to identify the boundaries and categories of entities in the text. In the process of training named entity recognition models, a large number of labeled samples are usually required. By implementing effective selection algorithms, this paper reduces the labeling of samples from a large number of samples suitable for model updates. By using five sets of comparison experiments, it is verified that a better set of samples can be obtained by effective selection algorithm, and a targeted sample of annotations is realized. Through experiments designed on microblog network data sets, it is verified that the current-based active learning algorithm can select more appropriate sample sets for a large amount of Internet text data, which can effectively reduce the cost of manual labeling. This paper uses two models to realize the boundary extraction and classification of entities. The sequence labeling model extracts the position of the entity in the sequence, the entity classification model realizes the classification of the labeling results, and uses the active learning method to realize the training on the unlabeled data set. Experiment on two data sets is done by using the training method in this article. Experiments on the Weibo dataset show that the algorithm can learn text features from the unlabeled dataset. The experimental results on the MSRA data set show that when the proportion of the pre-training data set reaches more than 40%, the F1 score of the model on the test data set is stable at about 90%, which is close to the result of using all the data sets, indicating that the model  in unlabeled data sets has certain feature extraction capabilities.