主题网络爬虫研究综述

网络信息资源呈指数级增长,面对用户越来越个性化的需求,主题网络爬虫应运而生。主题网络爬虫是一种下载特定主题网页的程序。利用在采集页面过程获得的特定信息,主题网络爬虫抓取的页面都是与主题相关的。基于主题网络爬虫的搜索引擎以及基于主题网络爬虫构建领域语料库等应用已经得到广泛运用。首先介绍了主题爬虫的定义、工作原理;然后介绍了近年来国内外关于主题爬虫的研究状况,并比较了各种爬行策略及相关算法的优缺点;最后提出了主题网络爬虫未来的研究方向。关键词：     

主题爬虫技术研究综述

随着移动互联网的普及以及网络信息指数的增长,如何有效地提取和利用这些信息面临巨大挑战。介绍了主题爬虫的工作原理、分类;回顾了近年来国内外关于主题爬虫的研究状况,分析了各种主题相似度的方法以及搜索策略,得出相比于普通的爬虫系统,基于网页内容和基于链接分析的爬虫系统的查准率、查全率都大幅度地提升;最后分析比较了主题网络爬虫两种动态搜索策略并指出了未来研究方向。     

分布式主题爬虫的研究与设计

本文通过对分布式技术和主题网络爬虫的研究,设计了一个能处理海量数据的分布式主题爬虫。设计内容主要包括分布式主题网络爬虫的各个功能模块及其实现方法。如页面的主题相关度判定方法、URL去重过滤方法等。主要使用了Hadoop技术和向量空间模型。该分布式主题爬虫的研究与设计为后面分布式主题爬虫的实现奠定了基础。     

深入解析Web主题爬虫的关键性原理

随着互联网的快速发展,搜索引擎的应用越来越重要,作为搜索引擎的首要组成部分网络爬虫一直备受人们的关注。主题爬虫作为网络爬虫的重要种类使用越来越广泛,深入分析的网络主题爬虫关键性原理有助于根据需求设计出科学合理的爬虫。     

基于朴素贝叶斯算法的主题爬虫的研究

主题爬虫是实现主题搜索引擎的关键部分。提出了利用朴素贝叶斯算法进行主题识别的方法,介绍了主题爬虫实现过程中所涉及到的关键部分,包括种子URL集合的生成、页面分析及特征提取、主题识别等。将基于朴素贝叶斯算法的主题爬虫,与基于链接分析的主题爬虫和基于主题词表的主题爬虫进行比较,实验表明基于朴素贝叶斯算法的主题爬虫准确性较好,论证了方法的可行性,为主题信息的采集奠定了良好的基础。     

分布式多主题网络爬虫系统的研究与实现

提出一种基于数据抽取器的分布式爬虫架构。该架构采用基于分类标注的多主题策略,解决同一爬虫系统内多主题自适应兼容的问题。介绍二级加权任务分割算法,解决基于目标导向、负载均衡的URL分配问题,增强系统可扩展性。给出基于Trie树的URL存储策略的改进方法,可以高效地支持URL查询、插入和重复性检测。     

化学主题网络爬虫的设计和实现

由于通用搜索引擎检索返回的结果过多、主题相关性不强以及随着人们对提供的各项信息服务的要求越来越高,基于整个Web的信息采集越来越力不从心。同时它无法及时地采集到足够的最新的Web信息,也不能满足人们日益增长的个性化需求。本文通过把Internet化学资源导航系统所积累的化学知识与搜索引擎的自动采集技术相结合展开了对化学主题网络爬虫开发的研究。结果表明,基于Widrow-Hoff分类器的化学主题网络爬虫能有效的采集化学相关的网页。     

用户定制主题聚焦爬虫的设计与实现

传统的聚焦爬虫在主题未知或者缺少相应训练集的情况下无法完成主题爬行。为让聚焦爬虫具有更好的主题适应性,提出基于聚类算法的自适应主题模型,指导聚焦爬虫在只有少量相同主题(主题未知)初始url的情况下完成主题爬行。通过对初始页面聚类得到主题中心向量,寻找相关网页更新主题中心位置;基于best-first策略实现url排序;基于该模型实现用户定制主题聚焦爬虫。通过对比实验验证了使用该模型的爬虫具有较高的收获比(havest rate)。     

基于遗传算法的主题爬虫

针对目前主题网络爬虫搜索策略难以在全局范围内找到最优解,通过对遗传算法的分析与研究,文中设计了一个基于遗传算法的主题爬虫方案.引入了结合文本内容的 PageRank 算法;采用向量空间模型算法计算网页主题相关度;采取网页链接结构与主题相关度来评判网页的重要性;依据网页重要性选择爬行中的遗传因子;设置适应度函数筛选与主题相关的网页.与普通的主题爬虫比较,该策略能够获取大量主题相关度高的网页信息,能够提高获取的网页的重要性,能够满足用户对所需主题网页的检索需求,并在一定程度上解决了上述问题     

基于贝叶斯分类器的主题爬虫研究*

主题爬虫是实现定题搜索引擎的核心技术。提出了基于贝叶斯分类器实现主题爬虫的方法,介绍了基于贝叶斯分类器的主题爬虫的系统结构以及系统关键部分的实现,包括URL队列、爬行历史、页面下载以及页面分析,并重点介绍了基于贝叶斯分类器的主题相关度算法。爬虫使用改进的TF-IDF算法来提取网页内容的特征,并采用贝叶斯分类器计算其主题相关度。实验结果表明,在搜索大量网络资源的情况下,贝叶斯分类器比PageRank算法更适合用于实现主题爬虫。     

Call for Papers The 6^th International Conference on Grid and Cooperative Computing

Recent years have seen rapid advances in various grid-related technologies, middleware, and applications. The GCC conference has become one of the largest scientific events worldwide in grid and cooperative computing. The 6th international conference on grid and cooperative computing (GCC2007) Sponsored by China Computer Federation (CCF),Institute of Computing Technology, Chinese Academy of Sciences (ICT) and Xinjiang University ,and in Cooperation with IEEE Computer Soceity ,is to be held from August 16 to 18, 2007 in Urumchi, Xinjiang, China.     

煤矿井下安全监控分站的设计及其仿真实现

为了设计一种具有低成本、低功耗、易操作、功能强且可靠性高的煤矿井下安全分站,针对煤矿安全生产实际,文章提出了采用MCS-51系列单片机为核心、具有CAN总线通信接口的煤矿井下安全监控分站的设计方案;首先给出煤矿井下安全监控分站的整体构架设计,然后着重阐述模拟量输入信号处理系统的设计过程,最后说明单片机最小系统及其键盘、显示、报警、通信等各个组成部分的设计;为验证设计方案的可行性与有效性,使用Proteus软件对设计内容进行仿真验证,设计的煤矿井下安全监控分站具有瓦斯、温度等模拟量参数超标报警功能和电机开停、风门开闭等开关量指示功能;仿真结果表明:设计的煤矿井下安全监控分站具有一定的实际应用价值.     

法律信息库系统

本文分析了法律数据库的结构和特点，介绍了采用面向对象设计方法和超文本数据库技术开发和实现法律信息库系统将作为重要网络资源之一为不同用户进行法律咨询服务。     

View-based model-driven architecture for enhancing maintainability of data access services

In modern service-oriented architectures, database access is done by a special type of services, the so-called data access services (DAS). Though, particularly in data-intensive applications, using and developing DAS are very common today, the link between the DAS and their implementation, e.g. a layer of data access objects (DAOs) encapsulating the database queries, still is not sufficiently elaborated, yet. As a result, as the number of DAS grows, finding the desired DAS for reuse and/or associated documentation can become an impossible task. In this paper we focus on bridging this gap between the DAS and their implementation by presenting a view-based, model-driven data access architecture (VMDA) managing models of the DAS, DAOs and database queries in a queryable manner. Our models support tailored views of different stakeholders and are scalable with all types of DAS implementations. In this paper we show that our view-based and model driven architecture approach can enhance software development productivity and maintainability by improving DAS documentation. Moreover, our VMDA opens a wide range of applications such as evaluating DAS usage for DAS performance optimization. Furthermore, we provide tool support and illustrate the applicability of our VMDA in a large-scale case study. Finally, we quantitatively prove that our approach performs with acceptable response times.     

International Journal of Automation and Computing Table of Contents

正~~     

Information for authors

正SCIENCE CHINA Information Sciences(Sci China Inf Sci),cosponsored by the Chinese Academy of Sciences and the National Natural Science Foundation of China,and published by Science China Press,is committed to publishing highquality,original results of both basic and applied research in all areas of information sciences,including computer science and technology;systems science,control science and engineering(published in Issues with odd numbers);information and communication engineering;electronic science and technology(published in Issues with even numbers).Sci China Inf Sci is published monthly in both print and electronic forms.It is indexed by Academic OneFile,Astrophysics Data System(ADS),CSA,Cabells,Current Contents/Engineering,Computing and Technology,DBLP,Digital Mathematics Registry,Earthquake Engineering Abstracts,Engineering Index,Engineered Materials Abstracts,Gale,Google,INSPEC,Journal Citation Reports/Science Edition,Mathematical Reviews,OCLC,ProQuest,SCOPUS,Science Citation Index Expanded,Summon by Serial Solutions,VINITI,Zentralblatt MATH.     

Erratum to: Fast global kernel fuzzy c-means clustering algorithm for consonant/vowel segmentation of speech signal

正Erratum to:J Zhejiang Univ-Sci C(ComputElectron)2014 15(7):551-563doi:10.1631/jzus.C1300320The original version of this article unfortunately contained mistakes.Algorithm 6 should be as follows:Algorithm 6 FGKFCM-F clustering Input:(1)X={x_1,x_2,…,x_N},,x_iR~d,i=1,2,…,N,the dataset;(2)C,1C≤N,the number of clusters;(3)ε0,the stopping criterion;     

Journal of Zhejiang University-SCIENCE C (Computers & Electronics) Total contents

正~~     

A pragmatic multi-method investigation of discrepant technological events: Coping,attributions, and ‘accidental’ learning

Discrepant technological events or situations that entail a problem, a misunderstanding or a difficulty with the Information Technology (IT) being employed, are common in the workplace, and can lead to frustration and avoidance behaviors. Little is known, however, about how individuals cope with these events. This paper examines these events by using a multi-method pragmatic approach informed by coping theory. The results of two studies – a critical incident study and an experiment – serve to build and test, respectively, a theoretical model that posits that individuals use a variety of strategies when dealing with these events: they experience negative emotions, make external attributions, and adopt engagement coping strategies directed at solving the event, eventually switching to a disengagement coping strategy when they feel they have no control over the situation. Furthermore, users’ efforts may result in ‘accidental’ learning as they try to overcome the discrepant IT events through engagement coping. The paper ends with a discussion of the results in light of existing literature, future opportunities for research, and implications for practice.