基于文本块密度和标签路径覆盖率的网页正文抽取

针对大多数网页除了正文信息外，还包括导航、广告和免责声明等噪声信息的问题。为了提高网页正文抽取的准确性，提出了一种基于文本块密度和标签路径覆盖率的抽取方法（CETD-TPC），结合网页文本块密度特征和标签路径特征的优点，设计了融合两种特征的新特征，利用新特征抽取网页中的最佳文本块，最后，抽取该文本块中的正文内容。该方法有效地解决了网页正文中噪声块信息过滤和短文本难以抽取的问题，且无需训练和人工处理。在CleanEval数据集和从知名网站上随机选取的新闻网页数据集上的实验结果表明，CETD-TPC方法在不同数据源上均具有很好的适用性，抽取性能优于CETR、CETD和CEPR等算法。

Webpage Content Extraction via Text Block Density and Tag Tath Coverage

Most webpages contains the content information, as well as noisy information such as navigation, advertisements and disclaimer notices. To address this problem and improve the accuracy of web page extraction, this paper proposes a webpage content extraction method via text block density and tag path coverage (CETD-TPC). Combining the advantages of webpage text block density feature and tag path feature, we design a new feature, TDTPC, which mixes the two features together. Then we extract the best text block from a webpage by using the TDTPC feature. Finally, contents are extracted from the content block. Without the manual processing and training, CETD-TPC is an effective solution to deal with the problems of noise block information filtering and short text extraction. Experimental results on CleanEval datasets and web news pages randomly selected from well-known websites show that the CETD-TPC method has good applicability on different data sets and performs better than CETR, CETD and CEPR.