基于在线社会网络的网络爬虫的研究和设计

在线社会网络是一个由亿万级用户及联接关系构成的大规模集合,其中以新浪微博为典型代表,目前已经成为人们日常交流的重要方式.如何在新浪微博中分析其用户的特征和网络拓扑结构成为研究在线社会网络的基础,本文设计一个网络爬虫系统,通过新浪微博开放平台提供的应用程序编辑接口（API）采集数据,实验证明,该方法切实可行.

Research and design of crawler based on online social network

Online social network is large collection,which include millions of users and connection.Sina microblog as typical representative becomes an important part of our lives.The goal of this paper is only to study how do users connect with each other and form the network using standard complex network techniques.Topic detection and hot topic prediction are very important research content in online social network.However,they are beyond the scope of this paper.In this paper,we design a crawler and using API try to gather data on the Sina microblog platform.The proposed method is proved in practice to be of real feasibility.