卷积自注意力编码过滤的强化自动摘要模型

技术是一种能从海量文本中获取重要信息的方法,它可以缓解大数据时代信息过载的问题.传统基于编码-解码自动摘要模型生成的摘要易出现句内重复、语义无关等现象,不利于读者理解文本的核心思想.受人工摘要书写方式的启发,即先理解文本局部信息,再从全局层面归纳信息、书写摘要,提出一种基于卷积自注意力编码过滤的自动摘要模型(CSAG).模型由编码器、卷积自注意力门控单元、解码器组成,结合卷积神经网络可以提取局部特征,多端自注意力机制可以学习长期依赖关系,模型可以根据上下文的局部和全局特征,从不同角度和不同层面提取文本潜在信息,确保模型生成正确流畅的摘要.然后通过策略梯度强化学习可直接利用不可微的度量指标ROUGE对模型进行优化,避免推理过程中出现曝光偏差问题.在Gigaword数据集上的多组对比实验结果表明,该文提出的模型在自动摘要任务上具有一定的优势.

Convolutional Self-attention Encoding for Reinforced Automatic Summarization Model

Automatic text summarization technology is a method to obtain important information from massive texts,which can alleviate the problem of information overload in the era of big data.The summarization generated by the traditional automatic summarization model based on encoder-decoder is prone to be repeated in sentences and semantic independence,which is not conducive to readers’understanding of the core ideas of the text.Inspired by the method of manual summarization,the reader first understands the local information of the text,then induces the information and writing summary from the global level.We propose an automatic summarization model based on convolutional self-attention encoding filtering(CSAG).CSAG consists of encoder,gated unit based on convolutional self-attention,decoder.In combination with the convolutional neural network can extracted local features,and the Multi-head self-attention mechanism can learn long-term dependence.The model can extract important feature information from different perspectives and levels according to the local and global features of the context,so as to ensure that the model can generate correct and smooth summarization.Then through policy gradient algorithm for reinforcement learning,the non-differentiable metric ROUGE can be directly used to optimize the model and avoid exposure bias in the testing process.Experimental results on Gigaword dataset show that the performance of the proposed model is better than the existing methods.