基于内容相似度的文摘自动评测方法及其有效性分析

为进行快速、可重复地自动文摘评测,提出利用文本余弦相似度评价自动文摘系统的方法,研究了不同项权重选取策略对评测结果的影响.对比了多个英文单文档、多文档自动文摘系统的人工评测和自动评测结果,证明余弦相似度方法能够较有效地区分不同文摘质量的系统.统计相关分析表明该自动评测方法和人工评测存在较高的一致性.评测结果对比表明基于tf-idf词权重信息的余弦相似度方法改善了缺少评测语料的多文档文摘自动评测的质量.

Application and analysis of content-similarity-based automatic evaluation for summarization systems

A fast and reliable content-similarity-based evaluation method is proposed and implemented for automatic summarization system evaluation. Statistical correlation analysis shows that the method is highly consistent with human evaluation for both single- and multiple- English document summarization. A comparison of term weight calculation strategies shows that the ff-idf weighting has improved the cosine similarity based multi-document summarization evaluation performance with less test data.