网络文本主题词的提取与组织研究

网络信息的指数爆炸给人们获取与掌控信息带来了困扰,为了挖掘海量信息中的关键因子并以恰当的方式进行组织,本文设计了网络文本主题词提取和组织算法。该算法基于多级滤噪的切分词拼接,利用特定的噪音库与滤噪策略严格控制拼接过程,在合理收录策略的挑选下,算法提取出了能够准确反映海量网络数据中关键因子的主题词串。为清晰地组织主题词,建立主题词与网络事件的有机联系,设计了新的词聚类策略对主题词提取结果进行处理,使表达同一热点的主题词合理地组织在一起,共同描述同一事件。在以实际网络文本为语料的实验中,算法表现出令人满意的性能。

Research on the Extraction and Organization of Key Phrases in Web Texts

The exponential explosion of Internet information complicates human access to those useful information.To detect the most important aspects of the vast information and manage the it accordingly,a key phrase extraction algorithm is proposed on the basis of a multi-level segmented word concatenation.Supported by a customized noise library and filtering strategies,the algorithm is capable of extracting key phrases in large amount of internet data.Further,a carefully designed clustering algorithms is applied so that the key phrases describing the same event are correctly grouped together.Experiment in real internet data proves the efficiency of our algorithms.