基于众包标注的语文教材句子难易度评估研究

该文提出了一种基于成对比较的众包标注方法,该方法可以通过非专业人士的简单判断获取标准统一的句子难度标注结果。基于该方法,构建了基于语文教材的由18411个句子组成的汉语句子难度语料库。面向单句绝对难度评估和句对相对难度评估两项基本的句子难易度评估任务,使用机器学习方法训练汉语句子难度评估模型,并进一步探讨了不同层面语言特征对模型性能的影响。实验结果显示,基于机器学习的分类模型可以有效预测句子的绝对难度和相对难度,最高准确率分别为63.37%和67.95%。语言特征可以帮助提升模型的性能,相比于词汇和句法层面的特征,加入汉字层面特征的模型在两项任务上的准确率最高。

Assessing Sentence Difficulty in Chinese Textbooks Based on Crowdsourcing

We propose a crowd-sourcing annotation approach based on pairwise comparison. With this approach, non-experts annotators' comparative judgements would lead to labelled data with a uniform standard. We construct a textbook-based corpus with 18, 411 Chinese sentences and utilize it to train a machine learning model which is capable of predicting the difficulty of sentences and the relative difficulty of sentence-pairs. We also explore the impact of multi-level linguistic features in two difficulty prediction tasks, in which our model achieves 63.37% and 67.95% accuracy respectively. The results show that Chinese character-level features are of greatest prediction among all the features in the two tasks.