一种垂直页面分割与信息提取方法的研究

在综合分析不同页面分割算法和适用条件的基础上,研究针对垂直型网站的页面分割和信息提取算法。以DOM(document object model)树为基础,提出页面内容聚集度的概念,统计获取页面分割标签和样式层叠表映射,对页面进行分割;采用正文识别和前缀匹配的方法,完成从页面分块信息提取。结合实际的网络视频项目需求,实现面向垂直型网站页面的分割和信息提取器。实验结果表明,该网页分割和信息提取方法对垂直页面信息提取具有良好的性能,满足实际项目需求。

Approach to Webpage segmentation andinformation extraction for vertical Websites

Analyzing existed Webpage segmentation algorithms along with their corresponding usage conditions, this paper investigated a vertical Webpage segmentation and information extraction method. Based on DOM tree, this paper proposed the notion of content crowding level, segmented the Webpage by using segment tag which obtained by statistical method and the mapping of cascading style sheets, and then extracted information from each segment by using text recognition and prefix matching. Given actual project requirements, a page segment and information extractor for vertical Webpage was designed and implemented. The experimental results show that the proposed method has achieved good performance and meets its needs.