Web页面信息块的自动分割

随着Internet的发展,Web页面数量的急剧增加,如何快速有效地获取信息变得越来越重要。一类Web页面往往包含着多个信息单元,它们在展现上排列紧凑、风格相似,在HTML语法上具有类似的模式,例如一个BBS页面上多个发言,每个信息被称为一个信息块。对于信息抽取、信息过滤等应用,需要首先将原始页面中分割为若干合适的信息块以便于后续的处理。本文提出了一种自动将Web页面分割为信息块的方法:首先通过创建Web页面结构化的HMTL分析树,然后根据包含有效文本量等确定包含信息块的子树,最后根据子树深度信息利用2-rank PAT算法进行分割。通过对BBS页面的信息块抽取实验,证明了该方法的有效性。

Automatic Segmentation of Web information block

With the development of the Internet the number of the Web pages increases dramatically, efficient information extraction from Web pages becomes more and more important. Some Web pages often contain multiple information units, which are arranged orderly and compactly with same presentation style and similar HTML syntax, for example, a BBS page that contains multiple posts. For information extraction, information filtering and suchlike Web application, we need segment this kind of original Web page into several appropriate information blocks as the preprocessing. This paper proposed a new automatic approach to segment the Web page into information blocks. First, we construct a structural HTML parsing tree for the Web page, and then locate the sub tree that contains all information blocks. Finally, 2 rank PAT algorithm is applied to segment the sub tree according to the depth of the sub tree and the information of node under the sub tree. Our experiments on BBS pages show this approach is fairly effective.