潜在语义分析聚类算法在文摘句子排序中的应用

针对多文档文摘生成过程中话题容易中断和文摘句子语义出现不连贯这两个研究难点, 分析了潜在语义分析聚类算法在句子排序中的应用, 以期提高文摘的生成质量。先采用潜在语义分析聚类算法将文摘句子聚类, 从而形成话题集, 以达到解决话题中断的目的。通过计算文档的文摘展现力, 挑选出文摘展现力最大的文档作为模板, 然后根据模板对文摘句子进行两趟排序。实验结果表明, 提出的算法是有效的, 该算法能够提高文摘的可读性。     

基于局部主题判定与抽取的多文档文摘技术

提出了一个通过对同一主题的多文档集合内局部主题的判定和抽取生成多文档文摘 的方法.首先在对多文档集合中句子依存分析和语义分析的基础上进行相似度计算,将相似 句子经过聚类形成多文档集合内不同的局部主题,然后进行每个局部主题中质心句的抽取和 排序,生成多文档文摘.该方法实现了文摘长度随文档内容自动确定,从而保证了文摘中包 含的信息的全面和简洁.最后文中还给出了多文档文摘的评价方法和实验结果,文摘的平均 精确率和平均压缩率分别为71.4%和25.2%.     

一种基于云模型的文摘单元选取方法研究

该文提出了一种基于云模型的文摘单元选取方法,利用云模型,全面考虑文摘单元的随机性和模糊性,提高面向查询的多文档自动文摘系统的性能。首先计算文摘单元和查询条件的相关性,将文摘单元和各个查询词的相关度看成云滴,通过对云的不确定性的计算,找出与查询条件真正意义相关的文摘单元;随后利用文档集合重要度对查询相关的结果进行修正,将文摘句和其他各文摘句的相似度看成云滴,利用云的数字特征计算句子重要度,找出能够概括尽可能多的文档集合内容的句子,避免片面地只从某一个方面回答查询问题。为了证明文摘单元选取方法的有效性,在英文大规模公开语料上进行了实验,并参加了国际自动文摘公开评测,取得了较好的成绩。
     

基于词汇链的多文档自动文摘研究

提出了一种基于词汇链抽取,文法分析的抽取文本代表词条的多文档摘要生成的方法。通过计算词义相似度构建词汇链,结合词频与位置特征进行文本代表词条成员的选择,将含有词条权值高的句子经过聚类形成多文档文摘句集合,然后进行质心句的抽取和排序,生成多文档文摘。该方法不仅考虑了词汇之间的语义信息,还考虑了词条对文本的代表成度,能够改善文摘句抽取的性能。实验结果表明,与单纯的由关键词确定文摘的方法相比,召回率和准确率都有不少的提高。     

基于基本要素向量空间的英文多文档自动摘要

在基于基本要素(BE)向量空间的英文多文档自动文摘中,句子不再用术语向量或词向量来表达,而是用基本要素向量来表示。在用k-均值聚类算法时,采用一种自动探测k值的技术。实验表明,基于基本要素的多文档自动文摘MSBEC比基于词更优越。     

基于潜在语义索引和句子聚类的中文自动文摘

自动文摘是自然语言处理领域的一项重要的研究课题.提出一种基于潜在语义索引和句子聚类的中文自动文摘方法.该方法的特色在于：使用潜在语义索引计算句子的相似度,并将层次聚类算法和K-中心聚类算法相结合进行句子聚类,这样提高了句子相似度计算和主题划分的准确性,有利于生成的文摘在全面覆盖文档主题的同时减少自身的冗余.实验结果验证了该文提出的方法的有效性,对比传统的基于聚类的自动文摘方法,该方法生成的文摘质量获得了显著的提高.     

主题信息的中文多文档自动文摘系统

多文档自动文摘能够帮助人们自动、快速地获取信息,使用主题模型构建多文档自动文摘系统是一种新的尝试,其中主题模型采用浅层狄利赫雷分配（LDA）。该模型是一个多层的产生式概率模型,能够检测文档中的主题分布。使用LDA为多文档集合建模,通过计算句子在不同主题上的概率分布之间的相似度作为句子的重要度,并根据句子重要度进行文摘句的抽取。实验结果表明,该方法所得到的文摘性能优于传统的文摘方法。     

基于增量图聚类的动态多文档摘要算法

目前的动态文摘方法几乎都是基于文档批处理机制的,无法适应实际应用中文档数据是以不稳定的数据流形式到来,需要实时更新摘要的需求。针对上述问题,提出一种利用K近邻思想对句子进行建模,再增量聚类句子实现子主题划分的动态文本摘要方法。该方法根据K近邻基本思想形成两层句子图模型,用增量图聚类方法对句子进行处理,同时考虑结合时间因素提高句子新颖度来抽取动态文摘。该方法能基于文档数据流增量式地抽取动态文摘,实现文摘内容的实时更新。通过在TAC2008和TAC2009的Update Summarization数据集上的测试,证明本文方法在动态文摘抽取上的有效性。     

一种主题句发现的中文自动文摘研究

提出了一种基于主题句发现的中文自动文摘方法。该方法使用术语代替传统的词语作为最小语义单位,采用术语长度术语频率方法进行术语权重计算,获得特征词。利用一种改进的k-means聚类算法进行句子聚类,根据聚类结果进行主题句发现。实验表明,该算法所得到的文摘,在各项指标上优于传统的文摘。     

一种文本分割技术的多文档文摘方法研究

提出一种基于文本分割技术的多文档自动文摘方法。该方法使用HowNet作为概念获取工具,通过建立句子概念向量空间模型和利用改进的DotPlotting模型来进行文本分割。利用建立的句子概念向量空间模型计算句子重要度,并根据句子重要度、文本分割结果和文摘句相似度等因素产生文本摘要。使用ROUGE-N评测方法和F_Score作为评测指标对系统产生的文摘进行评测,结果显示使用文本分割技术进行多文档摘要是有效的。     

基于DSC的多文本自动摘要

多文本摘要的目标是对给定的查询和多篇文本（文本集）,创建一个简洁明了的摘要,要求该摘要能够表达这些文本的关键内容,同时和给定的查询相关。一个给定的文本集通常包含一些主题,而且每个主题由一类句子来表示,一个优秀的摘要应该要包含那些最重要的主题。如今大部分的方法是建立一个模型来计算句子得分,然后选择得分最高的部分句子来生成摘要。不同于这些方法,我们更加关注文本的主题而不是句子,把如何生成摘要的问题看成一个主题的发现,排序和表示的问题。我们首次引入dominant sets cluster（DSC）来发现主题,然后建立一个模型来对主题的重要性进行评估,最后兼顾代表性和无重复性来从各个主题中选择句子组成摘要。我们在DUC2005、2006、2007三年的标准数据集上进行了实验,最后的实验结果证明了该方法的有效性。     

融入事件实体知识的汉越跨语言新闻事件检索

现有汉越跨语言新闻事件检索方法较少使用新闻领域内的事件实体知识,在候选文档中存在多个事件的情况下,与查询句无关的事件会干扰查询句与候选文档间的匹配精度,影响检索性能。提出一种融入事件实体知识的汉越跨语言新闻事件检索模型。通过查询翻译方法将汉语事件查询句翻译为越南语事件查询句,把跨语言新闻事件检索问题转化为单语新闻事件检索问题。考虑到查询句中只有单个事件,候选文档中多个事件共存会影响查询句和文档的精准匹配,利用事件触发词划分候选文档事件范围,减小文档中与查询无关事件的干扰。在此基础上,利用知识图谱和事件触发词得到事件实体丰富的知识表示,通过查询句与文档事件范围间的交互,提取到事件实体知识表示与词以及事件实体知识表示之间的排序特征。在汉越双语新闻数据集上的实验结果表明,与BM25、Conv-KNRM、ATER等基线模型相比,该模型能够取得较好的跨语言新闻事件检索效果,NDCG和MAP指标最高可提升0.712 2和0.587 2。     

Extractive single-document summarization based on genetic operators and guided local search

Due to the exponential growth of textual information available on the Web, end users need to be able to access information in summary form – and without losing the most important information in the document when generating the summaries. Automatic generation of extractive summaries from a single document has traditionally been given the task of extracting the most relevant sentences from the original document. The methods employed generally allocate a score to each sentence in the document, taking into account certain features. The most relevant sentences are then selected, according to the score obtained for each sentence. These features include the position of the sentence in the document, its similarity to the title, the sentence length, and the frequency of the terms in the sentence. However, it has still not been possible to achieve a quality of summary that matches that performed by humans and therefore methods continue to be brought forward that aim to improve on the results. This paper addresses the generation of extractive summaries from a single document as a binary optimization problem where the quality (fitness) of the solutions is based on the weighting of individual statistical features of each sentence – such as position, sentence length and the relationship of the summary to the title, combined with group features of similarity between candidate sentences in the summary and the original document, and among the candidate sentences of the summary. This paper proposes a method of extractive single-document summarization based on genetic operators and guided local search, called MA-SingleDocSum. A memetic algorithm is used to integrate the own-population-based search of evolutionary algorithms with a guided local search strategy. The proposed method was compared with the state of the art methods UnifiedRank, DE, FEOM, NetSum, CRF, QCS, SVM, and Manifold Ranking, using ROUGE measures on the datasets DUC2001 and DUC2002. The results showed that MA-SingleDocSum outperforms the state of the art methods.     

Summarization of Imaged Documents without OCR

A system is presented for creating a summary indicating the contents of an imaged document. The summary is composed from selected regions extracted from the imaged document. The regions may include sentences, key phrases, headings, and figures. The extracts are identified without the use of optical character recognition. The imaged document is first processed to identify the word-bounding boxes, the reading order of words, and the location of sentence and paragraph boundaries in the text. The word-bounding boxes are grouped into equivalence classes to mimic the terms in a text document. Equivalence classes representing content words are identified, and key phrases are identified from the set of content words. Summary sentences are selected using a statistically based classifier applied to a set of discrete sentence features. Evaluation of sentence selection against a set of abstracts created by a professional abstracting company is given.     

Document structure model for survey generation using neural network

Survey generation aims to generate a summary from a scientific topic based on related papers. The structure of papers deeply influences the generative process of survey, especially the relationships between sentence and sentence, paragraph and paragraph. In principle, the structure of paper can influence the quality of the summary. Therefore, we employ the structure of paper to leverage contextual information among sentences in paragraphs to generate a survey for documents. In particular, we present a neural document structure model for survey generation.We take paragraphs as units, and model sentences in paragraphs, we then employ a hierarchical model to learn structure among sentences, which can be used to select important and informative sentences to generate survey. We evaluate our model on scientific document data set. The experimental results show that our model is effective, and the generated survey is informative and readable.     

SUBTOPIC‐BASED MULTIMODALITY RANKING FOR TOPIC‐FOCUSED MULTIDOCUMENT SUMMARIZATION

Topic‐focused multidocument summarization has been a challenging task because the created summary is required to be biased to the given topic or query. Existing methods consider the given topic as a single coarse unit, and then directly incorporate the relevance between each sentence and the single topic into the sentence evaluation process. However, the given topic is usually not well defined and it consists of a few explicit or implicit subtopics. In this study, the related subtopics are discovered from the topic's narrative text or document set through topic analysis techniques. Then, the sentence relationships against each subtopic are considered as an individual modality and the multimodality manifold‐ranking method is proposed to evaluate and rank sentences by fusing the multiple modalities. Experimental results on the DUC benchmark data sets show the promising results of our proposed methods.     

iSpreadRank: Ranking sentences for extraction-based summarization using feature weight propagation in the sentence similarity network

Sentence extraction is a widely adopted text summarization technique where the most important sentences are extracted from document(s) and presented as a summary. The first step towards sentence extraction is to rank sentences in order of importance as in the summary. This paper proposes a novel graph-based ranking method, iSpreadRank, to perform this task. iSpreadRank models a set of topic-related documents into a sentence similarity network. Based on such a network model, iSpreadRank exploits the spreading activation theory to formulate a general concept from social network analysis: the importance of a node in a network (i.e., a sentence in this paper) is determined not only by the number of nodes to which it connects, but also by the importance of its connected nodes. The algorithm recursively re-weights the importance of sentences by spreading their sentence-specific feature scores throughout the network to adjust the importance of other sentences. Consequently, a ranking of sentences indicating the relative importance of sentences is reasoned. This paper also develops an approach to produce a generic extractive summary according to the inferred sentence ranking. The proposed summarization method is evaluated using the DUC 2004 data set, and found to perform well. Experimental results show that the proposed method obtains a ROUGE-1 score of 0.38068, which represents a slight difference of 0.00156, when compared with the best participant in the DUC 2004 evaluation.     

Recent advances in document summarization

The task of automatic document summarization aims at generating short summaries for originally long documents. A good summary should cover the most important information of the original document or a cluster of documents, while being coherent, non-redundant and grammatically readable. Numerous approaches for automatic summarization have been developed to date. In this paper we give a self-contained, broad overview of recent progress made for document summarization within the last 5 years. Specifically, we emphasize on significant contributions made in recent years that represent the state-of-the-art of document summarization, including progress on modern sentence extraction approaches that improve concept coverage, information diversity and content coherence, as well as attempts from summarization frameworks that integrate sentence compression, and more abstractive systems that are able to produce completely new sentences. In addition, we review progress made for document summarization in domains, genres and applications that are different from traditional settings. We also point out some of the latest trends and highlight a few possible future directions.     

.基于用户查询扩展的自动摘要技术*

提出了一种新的文档自动摘要方法,利用非负矩阵分解算法将原始文档表示为若干语义特征向量的线性组合,通过相似性计算来确定与用户查询高度相关的语义特征向量,抽取在该向量上具有较大投影系数的句子作为摘要,在此过程中,多次采用相关反馈技术对用户查询进行扩展优化。实验表明,该方法所得摘要在突出文档主题的同时,体现了用户的需求和兴趣,有效改善了信息检索的效率。