基于朴素贝叶斯算法的主题爬虫的研究

主题爬虫是实现主题搜索引擎的关键部分。提出了利用朴素贝叶斯算法进行主题识别的方法,介绍了主题爬虫实现过程中所涉及到的关键部分,包括种子URL集合的生成、页面分析及特征提取、主题识别等。将基于朴素贝叶斯算法的主题爬虫,与基于链接分析的主题爬虫和基于主题词表的主题爬虫进行比较,实验表明基于朴素贝叶斯算法的主题爬虫准确性较好,论证了方法的可行性,为主题信息的采集奠定了良好的基础。

Research on Focused Crawler Based on Naive Bayes Algorithm

Focused crawler is a key part of the focused search engine.This paper proposed a method of using Naive Bayes algorithm to identify topics,introduced the core part of the focused crawler,including the generation of seed URL collection,the page analysis and feature extraction and the topic identify.Compared the focused crawler based on Naive Bayes algorithm with the focused crawler base on links analysis and thesaurus,the experiment result proved that the focused crawler based on Naive Bayes algorithm has better accuracy and the method is feasible.It laid good foundation for the topic information collection.