面向问题生成的预训练模型适应性优化方法研究

问题生成的核心任务是"在给定上下文语境的前提下,对目标答案自动生成相应的疑问句".问题生成是自然语言处理领域中富有挑战性的任务之一,其对可靠的语义编码和解码技术有着极高的要求.目前,预训练语言模型已在不同自然语言处理任务中得到广泛应用,并取得了较好的应用效果.该文继承这一趋势,尝试将预训练语言模型UNILM应用于现有"...

Adaptive Optimization Method of Pre-trained Language Model for Question Generation

Automatically question generation (QG for short) is to automatically generate the corresponding interrogative sentence of the target answer under the given context. . In this paper, we take advantage of pre-trained language model and apply the UNILM on encoder-decoder framework of question generation. In particular, in order to solve the problems of "exposure bias" and "mask heterogeneity" in the decoding phase of model, we examine the noise-aware training method and transfer learning on UNILM to raise its adaptability Experiments on SQuAD show that our best model yields state-of-the-art performance in answer-aware QG task with up to 20.31% and 21.95% BLEU score for split1 and split2, respectively, and in answer-agnostic QG task with 17.90% BLEU score for split1.