基于潜在语义索引和句子聚类的中文自动文摘

自动文摘是自然语言处理领域的一项重要的研究课题.提出一种基于潜在语义索引和句子聚类的中文自动文摘方法.该方法的特色在于：使用潜在语义索引计算句子的相似度,并将层次聚类算法和K-中心聚类算法相结合进行句子聚类,这样提高了句子相似度计算和主题划分的准确性,有利于生成的文摘在全面覆盖文档主题的同时减少自身的冗余.实验结果验证了该文提出的方法的有效性,对比传统的基于聚类的自动文摘方法,该方法生成的文摘质量获得了显著的提高.     

一种主题句发现的中文自动文摘研究

提出了一种基于主题句发现的中文自动文摘方法。该方法使用术语代替传统的词语作为最小语义单位，采用术语长度术语频率方法进行术语权重计算，获得特征词。利用一种改进的k-means聚类算法进行句子聚类，根据聚类结果进行主题句发现。实验表明，该算法所得到的文摘，在各项指标上优于传统的文摘。     

基于规则和统计的中文自动文摘系统

自动文摘是自然语言处理领域里一个重要课题,本文在传统方法基础上提出了一种中文自动文摘的方法。在篇章结构分析里,我们提出了基于连续段落相似度的主题划分算法,使生成的文摘更具内容全面性与结构平衡性。同时结合了若干规则对生成的文摘初稿进行可读性加工处理,使最终生成的文摘更具可读性。最后提出了一种新的文摘评价方法(F-new-measure)对系统进行测试。系统测试表明该方法在不同文摘压缩率时,评价值均较为稳定。     

基于局部主题关键句抽取的自动文摘方法

自动文摘是语言信息处理中的重要环节。该文提出一种基于局部主题关键句抽取的中文自动文摘方法。通过层次分割的方法对文档进行主题分割,从各个局部主题单元中抽取一定数量的句子作为文章的文摘句。通过事先对文档进行语义分析,有效地避免了数据冗余和容易忽略分布较小的主题等问题。实验结果表明了该方法的有效性。     

文本聚类在自动文摘中的应用研究

针对当前自动文摘方法的不足，提出了基于文本聚类的自动文摘实现方法。将文本聚类引入自动文摘中，能实现多文档的自动文摘。实现了面向“塑料”行业的基于文本聚类的自动文摘系统TCAAS，其单文档自动文摘的正确率和召回率在80％以上，多文档自动文摘的正确率和召回率在75％以上。实验表明该方法可行，对自动文摘系统的设计具有借鉴意义和深入研究的价值。     

基于局部话题句群的事件相关多文档摘要研究

多文档自动文摘研究的目的是给用户提供简洁全面的文档信息并提高用户获取信息的效率。在进行局部话题确定时，通常是利用聚类分析的方法把相似的文本单元聚成一个局部话题。该文提出了一种针对新闻事件的多文档摘要生成方法，其特色在于：在提取基本新闻要素和扩展新闻要素的基础上分别形成了基本局部话题句群（BPTSG）和扩展局部话题句群（EPTSG），这样可以在尽可能全面地覆盖多个话题的同时缩减自身的冗余。此外，文中还提出了一种基于事件时间和句子位置信息的文摘句排序方法。实验结果验证了该文所提的方法是有效的，与基于聚类的自动文摘系统相比较，该系统生成的摘要质量有显著提高。     

自动文摘的方法研究

文中总结了自动文摘的主要研究方法和策略并把方法分成了三大类:自动摘录、基于信息抽取的自动文摘和基于理解的自动文摘.自动摘录方法是从文章中抽取重要句子来形成文摘;基于信息抽取的文摘方法是用从文章中抽取的信息填充已经编好的框架,然后用模板将内容输出;基于理解的文摘方法是利用自然语言处理技术生成文摘.文中重点总结了单主题文章和多主题文章的自动摘录方法,在多种算法进行优缺点比较后提出了一种新的多主题划分方法.     

使用多元语义特征的评论文本主题聚类

特征是一切观点挖掘和情感分析任务的关键所在。对于无监督的文本聚类任务,文本特征的优劣直接影响聚类效果。考察三种语义特征(名词、名词短语、语义角色)对主题聚类的作用以及不同特征之间的相容关系,提出一种消除冗余特征的方法。该方法能有效地去除冗余特征,提高聚类精度。同时还提出一种基于语义角色标注的直接定位有效词特征的聚类方法,实验表明该方法是直接的和有效的,并为特征选择方法提供了新思路。     

基于文本聚类的自动文摘系统的研究与实现

针对当前自动文摘方法的不足，提出了基于文本聚类和自然语言理解的自动文摘实现方法。可以克服常规自动文摘方法的不足，使文摘的质量和效果得到大大的提高。将文本聚类引入自动文摘中，不但使单文档的文摘质景得到提高，而且能够实现多文档的自动文摘，这是现有的自动文摘技术所没有涉及的。实现了面向“塑料”行业的基于文本聚类和自然语言理解的自动文摘系统TCAAS。     

基于文本聚类的多文档自动文摘研究

针对当前自动文摘方法的不足,提出了基于文本聚类的自动文摘实现方法.可以克服常规自动文摘方法的不足,使文摘的质量和效果得到大大的提高.将文本聚类引入自动文摘中,不但使单文档的文摘质量得到提高,而且能够实现多文档的自动文摘,这是现有的自动文摘技术所没有涉及的.实现了面向"塑料"行业的基于文本聚类的自动文摘系统TCAAS.实验表明该方法可行, 对自动文摘系统的设计具有借鉴意义和深入研究的价值.     

COSUM: Text summarization based on clustering and optimization

Text summarization is a process of extracting salient information from a source text and presenting that information to the user in a condensed form while preserving its main content. In the text summarization, most of the difficult problems are providing wide topic coverage and diversity in a summary. Research based on clustering, optimization, and evolutionary algorithm for text summarization has recently shown good results, making this a promising area. In this paper, for a text summarization, a two‐stage sentences selection model based on clustering and optimization techniques, called COSUM, is proposed. At the first stage, to discover all topics in a text, the sentences set is clustered by using k‐means method. At the second stage, for selection of salient sentences from clusters, an optimization model is proposed. This model optimizes an objective function that expressed as a harmonic mean of the objective functions enforcing the coverage and diversity of the selected sentences in the summary. To provide readability of a summary, this model also controls length of sentences selected in the candidate summary. For solving the optimization problem, an adaptive differential evolution algorithm with novel mutation strategy is developed. The method COSUM was compared with the 14 state‐of‐the‐art methods: DPSO‐EDASum; LexRank; CollabSum; UnifiedRank; 0–1 non‐linear; query, cluster, summarize; support vector machine; fuzzy evolutionary optimization model; conditional random fields; MA‐SingleDocSum; NetSum; manifold ranking; ESDS‐GHS‐GLO; and differential evolution, using ROUGE tool kit on the DUC2001 and DUC2002 data sets. Experimental results demonstrated that COSUM outperforms the state‐of‐the‐art methods in terms of ROUGE‐1 and ROUGE‐2 measures.     

融合文本分类的多任务学习摘要模型

文本摘要应包含源文本中所有重要信息,传统基于编码器-解码器架构的摘要模型生成的摘要准确性较低。根据文本分类和文本摘要的相关性,提出一种多任务学习摘要模型。从文本分类辅助任务中学习抽象信息改善摘要生成质量,使用K-means聚类算法构建Cluster-2、Cluster-10和Cluster-20文本分类数据集训练分类器,并研究不同分类数据集参与训练对摘要模型的性能影响,同时利用基于统计分布的判别法全面评价摘要准确性。在CNNDM测试集上的实验结果表明,该模型在ROUGE-1、ROUGE-2和ROUGE-L指标上相比强基线模型分别提高了0.23、0.17和0.31个百分点,生成摘要的准确性更高。     

A novel approach to update summarization using evolutionary manifold-ranking and spectral clustering

Update summarization is a new challenge in automatic text summarization. Different from the traditional static summarization, it deals with the dynamically evolving document collections of a single topic changing over time, which aims to incrementally deliver salient and novel information to a user who has already read the previous documents. How to have a content selection and linguistic quality control in a temporal context are the two new challenges brought by update summarization. In this paper, we address a novel content selection framework based on evolutionary manifold-ranking and normalized spectral clustering. The proposed evolutionary manifold-ranking aims to capture the temporal characteristics and relay propagation of information in dynamic data stream and user need. This approach tries to keep the summary content to be important, novel and relevant to the topic. Incorporation with normalized spectral clustering is to make summary content have a high coverage for each sub-topic. Ordering sub-topics and selecting sentences are dependent on the rank score from evolutionary manifold-ranking and the proposed redundancy removal strategy with exponent decay. The evaluation results on the update summarization task of Text Analysis Conference (TAC) 2008 demonstrate that our proposed approach is competitive. In the 71 run systems, we receive three top 1 under PYRAMID metrics, ranking 13th in ROUGE-2, 15th in ROUGE-SU4 and 21st in BE.     

A multiagent,dynamic rank-driven multi-deme architecture for real-valued multiobjective optimization

As information is available in abundance for every topic on internet, condensing the important information in the form of summary would benefit a number of users. Hence, there is growing interest among the research community for developing new approaches to automatically summarize the text. Automatic text summarization system generates a summary, i.e. short length text that includes all the important information of the document. Since the advent of text summarization in 1950s, researchers have been trying to improve techniques for generating summaries so that machine generated summary matches with the human made summary. Summary can be generated through extractive as well as abstractive methods. Abstractive methods are highly complex as they need extensive natural language processing. Therefore, research community is focusing more on extractive summaries, trying to achieve more coherent and meaningful summaries. During a decade, several extractive approaches have been developed for automatic summary generation that implements a number of machine learning and optimization techniques. This paper presents a comprehensive survey of recent text summarization extractive approaches developed in the last decade. Their needs are identified and their advantages and disadvantages are listed in a comparative manner. A few abstractive and multilingual text summarization approaches are also covered. Summary evaluation is another challenging issue in this research field. Therefore, intrinsic as well as extrinsic both the methods of summary evaluation are described in detail along with text summarization evaluation conferences and workshops. Furthermore, evaluation results of extractive summarization approaches are presented on some shared DUC datasets. Finally this paper concludes with the discussion of useful future directions that can help researchers to identify areas where further research is needed.     

基于PageRank的中文多文档文本情感摘要

文本情感摘要任务旨在对带有情感的文本数据进行浓缩、提炼进而产生文本所表达的关于情感意见的摘要。该文主要研究基于多文档的文本情感摘要问题, 重点针对网络上存在同一个产品的多个评论产生相应的摘要。首先,为了进行关于文本情感摘要的研究,该文收集并标注了一个基于产品评论的中文多文档文本情感摘要语料库。其次,该文提出了一种基于情感信息的PageRank算法框架用于实现多文档文本情感摘要,该算法同时考虑了情感和主题相关两方面的信息。实验结果表明,该文采用的方法和已有的方法相比在ROUGE值上有显著提高。     

基于评论质量的多文档文本情感摘要

文本情感摘要任务旨在对带有情感的文本数据进行浓缩、提炼进而产生文本所表达的关于情感意见的摘要,用以帮助用户更好地阅读、理解情感文本的内容。该文主要研究多文档的文本情感摘要问题,重点针对网络上存在的同一个产品的多个评论进行摘要抽取。在情感文本中,情感相关性是一个重要的特点,该文将充分考虑情感信息对文本情感摘要的重要影响。同时,对于评论语料,质量高的评论或者说可信度高的评论可以帮助用户更好的了解评论中所评价的对象。因此,该文将充分考虑评论质量对文本情感摘要的影响。并且为了进行关于文本情感摘要的研究,该文收集并标注了一个基于产品评论的英文多文档文本情感摘要语料库。实验证明,情感信息和评论质量能够帮助多文档文本情感摘要,提高摘要效果。     

基于用户兴趣的文本摘要方法研究

提出的摘要方法,以句子为基本抽取单位,以兴趣主题词为句子的加权特征。对句子基于潜语义聚类,提出语义结构,这种结构对摘要质量的提高有重要作用，并且提出了较为客观和有效的摘要评价方法。实验表明,本文方法是行之有效的。     

How to Improve Text Summarization and Classification by Mutual Cooperation on an Integrated Framework

Text summarization and classification are core techniques to analyze a huge amount of text data in the big data environment. Moreover, as the need to read texts on smart phones, tablets and television as well as personal computers continues to grow, text summarization and classification techniques become more important and both of them do essential processes for text analysis in many applications.Traditional text summarization and classification techniques have individually been considered as different research fields in this literature. However, we find out that they can help each other as text summarization makes use of category information from text classification and text classification does summary information from text summarization. Therefore, we propose an effective integrated learning framework using both of summary and category information in this paper. In this framework, the feature-weighting method for text summarization utilizes a language model to combine feature distributions in each category and text, and one for text classification does the sentence importance scores estimated from the text summarization.In the experiments, the performances of the integrated framework are better than ones of individual text summarization and classification. In addition, the framework has some advantages of easy implementation and language independence because it is based on only simple statistical approaches and POS tagger.     

基于词性软模板注意力机制的短文本自动摘要方法

在短文本摘要任务中,带有直观主谓宾结构的摘要句语义完整性较强,但词性组合对该结构具有约束作用.为此文中提出基于词性软模板注意力机制的短文本自动摘要方法.首先对文本进行词性标记,将标记的词性序列视为文本的词性软模板,指导方法构造摘要句的结构规范,在编码端实现词性软模板的表征.再引入词性软模板注意力机制,增强对文中核心词性(如名词、动词等)的关注.最后在解码端联合词性软模板注意力与传统注意力,产生摘要句.在短文本摘要数据集上的实验验证文中方法的有效性     

结合主题信息聚类编码的文本摘要模型

结合注意力机制的序列到序列模型在生成式文本摘要的研究中已取得了广泛应用,但基于该模型的摘要生成技术依然存在信息编码不充分、生成的摘要偏离主题的问题,对此提出了一种结合主题信息聚类编码的文本摘要生成模型TICTS(theme information clustering coding text summarization)。将传统的抽取式文本摘要方法与基于深度学习的生成式文本摘要方法相结合,使用基于词向量的聚类算法进行主题信息提取,利用余弦相似度计算输入文本与所提取关键信息的主题相关性,将其作为主题编码的权重以修正注意力机制,在序列到序列模型的基础上结合主题信息与注意力机制生成摘要。模型在LCSTS数据集上进行实验,以ROUGE为评价标准,实验结果相对于基线模型在ROUGE-1的得分上提高了1.1,ROUGE-2提高了1.3,ROUGE-L提高了1.1。实验证明结合主题信息聚类编码的摘要模型生成的摘要更切合主题,摘要质量有所提高。