基于启发式规则优化的网页元素提取方法

网页信息提取方法中的启发式规则,是识别网页标签信息、利用网页节点分析结果、针对网页不同内容、完成信息提取的重要手段。本研究在对现有启发式规则进行研究分析的基础上,提出了几种优化的启发式规则,实现对网页标题、发布时间、来源以及正文内容等元素信息的精准提取。本研究进一步提出了运用编辑距离算法实现正文内容提取准确率的判定,并提出阙值优化方法,克服了正文提取中噪声节点多、内容识别不完全的缺陷,大大提高了提取的准确度。

Heuristic-Optimizing Based Webpage Elements Extraction

The heuristic rules of web information extraction method, is an important means for identifying tag＇s information, analyzing the nodes and different body of pages, and finally extracting the information in the Webpage. Our study based on the existed research of heuristic rules, presents some kinds of optimized heuristic rules, by which the accurate extraction of title, release time, source and body cloud be achieved. Furthermore, this study proposes the use of Levenshtein Distance Method to determination the accuracy of the extraction of body, and proposes the threshold optimization method to overcome the shortage in body extraction resulted from the large number of noise nodes and the incompletely identification of body, which method can be utilized to improve the accuracy of the extracted information greatly.