领域相关的Web网站抓取方法

本文提出了一种抓取领域相关的Web站点的方法，可以在较小的代价下准确地收集用户所关心领域内的网站。这种方法主要改进了传统的聚焦爬虫（Focused Crawler）技术，首先利用Meta-Search技术来改进传统Crawler的通过链接分析来抓取网页的方法，而后利用启发式搜索大大降低了搜索代价，通过引入一种评价领域相关性的打分方法，迭到了较好的准确率。本文详细地描述了上述算法并通过详细的实验验证了算法的效率和效果。

Website Crawling for Specific Topics

In this paper, we propose a new approach to discover the Websites for special topic in WWW with high precision and low cost. This approach improves traditional Focused Crawler techniques, different from the common Web crawler which accesses the Web graph composed by HTML pages and hyperlinks, our crawler uses Meta-Seareh to get the URLs of relevant page, then uses heuristic search method to reduce the search cost, and uses topic relevant rules to increase the precision. The experimental results show the presented approach is both effective and efficient.