长度分布约束下的摘要文本无监督分割算法

作为文章内容的浓缩表达,摘要蕴含着文章关键的发现和结论。自动分析挖掘摘要内容,对于充分利用快速增长的科技文献具有重要意义。该文以Medline生物医学文章的摘要为对象,对摘要的文本分割问题进行了研究。针对摘要各论述侧面(内容块)之间在长度分布上倾向于均匀的特点,提出了一种考虑长度分布约束的摘要文本无监督分割算法,该算法以信息熵作为长度分布均匀性的度量指标,将信息熵与块内语义相似度及块间语义相似度相结合作为优化的目标函数,采用动态规划方法搜索最佳分割点。在8 603篇Medline摘要上对算法进行了实验验证,并与文献中最新的无监督分割算法进行了实验对比。结果表明,该文提出的增加了长度分布约束的分割算法更加适用于摘要文本分割,分割的准确率有3%的提高。

A Length Distribution Constrained Text Segmentation for Paper Abstracts

To deal with the text segmentation for academic paper abstracts, an unsupervised text segmentation algorithm is proposed, which incorporates constraint of the length distribution derived from the preference of length uniformity in different discussion aspects (i.e. content blocks) of an abstract. A metric based on information entropy is introduced to the algorithm to measure the length distribution uniformity, and the object function is designed with further combination of semantic similarities of inter-and intra-content blocks. A standard dynamic programming scheme is employed to determine the best segmentation sequence. Experimented on 8603 abstracts from Medline, the results show an improvement of 3% in accuracy compared with baselines.