Is it worth the effort? Assessing the benefits of partial automatic pre-labeling for frame-semantic annotation |
| |
Authors: | Ines Rehbein Josef Ruppenhofer Caroline Sporleder |
| |
Affiliation: | (1) Saarland University, PO Box 15 11 50, 66041 Saarbr?cken, Germany |
| |
Abstract: | Corpora with high-quality linguistic annotations are an essential component in many NLP applications and a valuable resource
for linguistic research. For obtaining these annotations, a large amount of manual effort is needed, making the creation of
these resources time-consuming and costly. One attempt to speed up the annotation process is to use supervised machine-learning
systems to automatically assign (possibly erroneous) labels to the data and ask human annotators to correct them where necessary.
However, it is not clear to what extent these automatic pre-annotations are successful in reducing human annotation effort,
and what impact they have on the quality of the resulting resource. In this article, we present the results of an experiment
in which we assess the usefulness of partial semi-automatic annotation for frame labeling. We investigate the impact of automatic
pre-annotation of differing quality on annotation time, consistency and accuracy. While we found no conclusive evidence that
it can speed up human annotation, we found that automatic pre-annotation does increase its overall quality. |
| |
Keywords: | |
本文献已被 SpringerLink 等数据库收录! |
|