网络信息采集中链接与主题相关性的判定研究

面向主题的Web信息采集需判断提取的URL链接主题相关性。基于主题链接上下文提取,主题型语义块采用提取链接周围一定长度的文本,目录型和图片型语义块利用DOM树层次结构,对链接数据进行URL相关性判定;利用知网基于语义相似度的链接判定,给出一种综合内容和链接结构分析的URL主题相关性判定NPR算法,比较PageRank算法能提供更精确的主题页面。其成果对我国信息机构进行学科网络信息资源的深度建设有实用价值。

RESEARCH ON LINK AND SUBJECT RELEVENCE DETERMINATION IN NETWORK INFORMATION COLLECTION

Subject-oriented Web information collection requires determination of the relevence of extracted URL link subject.Context extraction is based on subject links.The subject type of semantic blocks are extracted from a certain length of texts around a link.The catalog and image types of semantic blocks use the DOM tree hierarchy to determine URL relevence for link data.Hownet is utilized for link determination based on semantic similiarity to provide a URL subject relevence determination NPR algorithm that comprises content and link structure analysis,which offers more accurite subject pages than PageRank algorithm.The achievements are of practical value for the depth construction of subject network information resources for domestic information organizations.