基于概念关系对齐的中文抽象语义表示解析评测方法

抽象语义表示(Abstract Meaning Representation,AMR)是一种句子语义表示方法,能够将句子的语义表示为一个单根有向无环图。随着中文AMR语料库规模的扩大,解析系统的研究也相继展开,将句子自动解析为中文AMR。然而,现有的AMR解析评测方法并不能处理中文AMR的重要组成部分——概念对齐和关系对齐信息,尤其是关系对齐中对应到有向弧上的虚词信息。因此,为了弥补中文AMR解析评测在这两个方面上的空缺,该文在Smatch指标的基础上加入了描写概念对齐和关系对齐的三元组,得到用以评测中文AMR的整体性指标Align-Smatch。选取100句人工标注语料与标准语料进行评测对照实验,结果显示,Align-Smatch有效兼容了对齐信息,对有向弧的评测比Smatch更合理。该文还提出了概念对齐指标、关系对齐指标、隐含概念指标共三个分项指标,以进一步评测中文AMR解析器在对齐子任务中的分项性能。

A Novel Evaluation Method for Chinese Abstract Meaning Representation Parsing Based on Alignment of Concept and Relation

Abstract Meaning Representation is a sentence-level meaning representation, which abstracts a sentence’s meaning into a rooted acyclic directed graph. With the continuous expansion of Chinese AMR corpus, more and more scholars have developed parsing systems to automatically parse sentences into Chinese AMR. To make up for the vacancy of Chinese AMR parsing evaluation methods, we have improved the Smatch algorithm of generating triples to make it compatible with concept alignment and relation alignment. We finally complete a new integrity metric Align-Smatch for paring evaluation. Compared on 100 manually annotated AMR and gold AMR, it is revealed that Align-Smatch works well in alignments and more robust in evaluating arcs. We also put forward some fine-grained metric for evaluating concept alignment, relation alignment and implicit concepts, in order to further measure parsers’ performance in subtasks.