基于本体的主题爬虫技术研究

最好优先搜索算法在主题页面附近搜索时能够表现出良好的性能,但算法只关注能"立即回报"的链接而容易遗失那些有远期价值的链接,导致当搜索位置距离主题团较远时容易迷失搜索方向。在最好优先搜索算法基础上,引入了本体帮助主题爬虫识别那些"未来回报"的链接,对抛弃的主题无关链接再基于领域本体进行一次其他主题相关度的判断实验。实验结果表明,基于本体的主题爬虫能够抓取大量主题相关度高的网页,提高了主题资源覆盖率,有效解决了传统算法无法穿过隧道的问题。

Focused Crawler Technology Based on Ontology

Best first search algorithm exhibits good performance when searching pages in the vicinity of the subject pages,however,it only pays close attention to "immediate payback" links while easily discarding the forward value links,so always leads to lose search direction when the search direction is far away from the subject group.On the basis of best first search algorithm,this paper introduces ontology to help focused crawler distinguish "the future payback" links,and uses domain ontology once again to judge w...