基于多粒度特征的文本生成评价方法

近年来,基于预训练语言模型的文本生成评价方法得到了广泛关注,其通过计算两个句子间子词粒度的相似度来评价生成文本的质量.但是对于越南语、泰语等存在大量黏着语素的语言,单个音节或子词不能独立成词表达语义,仅基于子词粒度匹配的方法并不能够完整表征两个句子间的语义相似关系.基于此,该文提出一种基于子词、音节、词组等多粒度特征的...

Evaluation Method of Text Generation Based on Multi-granularity Features

Recently, the evaluation method of text generation based on pre-trained language model has gained attention, which evaluates the quality of generated text by computing the granularity similarity of sub-words of two sentences. However, for languages that contain many adhesive morphemes, such as Vietnamese and Thai, a single syllable or sub-word cannot form the semantic integrity, which means that the sub-word granularity matching method cannot fully represent the semantic relationship between two sentences. Therefore, we propose a text generation evaluation method with multi-granularity features of sub-words, syllables, and phrases. After the representation of text is obtained by MBERT, the semantic similarity of syllables and phrases is introduced to enhance the evaluation model of sub-words. Experimental results on such tasks as cross-language summarization, machine translation, and data screening show that, compared with ROUGE, BLEU based on statistical evaluation and Bertscore based on deep semantic matching, the proposed metric correlates better with human judgments.