| 基于Heritrix的商品信息搜索的网络爬虫系统的设计 |
| |
| 引用本文： | 袁小洁. 基于Heritrix的商品信息搜索的网络爬虫系统的设计[J]. 电脑编程技巧与维护, 2012, 0(22): 74+81-74,81 |
| |
| 作者姓名： | 袁小洁 |
| |
| 作者单位： | 贵州大学计算机科学与信息学院,贵阳550025;贵州省贸易经济学校,贵阳550018 |
| |
| 摘 要： | 探讨以开源软件Heritrix体系构建的获取商品信息爬虫系统,针对Heritrix开源爬虫项目存在的问题和商品采集的特点,项目设计了定向抓取包含某一特定内容的网页的类,从而改进Heritrix,并引入ELFHash算法进行URL散列中,以提高抓取效率,为面向商品的搜索系统以及数据挖掘提供可靠的数据源。
|
| 关 键 词： | Heritrix 网络爬虫 爬虫 商品信息采集 |
Commodity Information Search Web Crawler System Design Based on Heritrix |
| |
| YUAN Xiao-jie. Commodity Information Search Web Crawler System Design Based on Heritrix[J]. Computer Programming Skills & Maintenance, 2012, 0(22): 74+81-74,81 |
| |
| Authors: | YUAN Xiao-jie |
| |
| Affiliation: | YUAN Xiao-jie 1,2(1.College of Computer Science & Information,Guizhou University,Guiyang 550025,China; 2.Guizhou Trade & economic school,Guiyang 550018,China) |
| |
| Abstract: | This paper discusses on the construction of open source software Heritrix system for commodity information crawler system,in view of the Heritrix open source reptiles project problems and the characteristics of the goods collection, this project designed the directional grab contains a particular content page classes,so as to improve Heritrix,and introducing ELFHash in URL hash algorithm in order to improve the efficiency of grab,to face commodity search system and data mining provides reliable datasource. |
| |
| Keywords: | Heritrix Web crawler Crawler merchandise information collection |
| 本文献已被 CNKI 维普 万方数据 等数据库收录！ |
|
