一种面向大规模微博数据的话题挖掘方法

随着微博的日趋流行,新浪微博已成为公众获取和传播信息的重要平台之一,针对微博数据的话题挖掘也成为当前的研究热点。提出一个面向大规模微博数据的话题挖掘方法。首先对大规模微博数据进行分析,基于Bloom Filter算法对数据进行去重处理,针对微博的特有结构,对文本进行预处理,提出改进的LDA主题模型Social Network LDA（SNLDA）,采用吉布斯采样法进行模型推导,挖掘出微博话题。实验结果表明,方法能有效地从大规模微博数据中挖掘出话题信息。     

大规模社交网络中高效的关键用户选取方法

针对大规模社交网络及其用户发布消息的历史数据,如何快速有效地选取具有较强信息传播能力的关键用户,提出了一种关键用户选取方法。首先,利用社交网络的结构信息,构建以用户为节点的有向图,利用用户发布消息的历史数据,基于Spark计算框架,定量计算由用户活跃度、转发交互度和信息量占比刻画的权重,从而构建社交网络的有向带权图模型;然后,借鉴PageRank算法,建立用户信息传播能力的度量机制,给出基于Spark的大规模社交网络中用户信息传播能力的计算方法;进而,给出基于Spark的d-距选取算法,通过多次迭代,使得所选取的不同关键用户的信息传播范围尽量少地重叠。建立在新浪微博数据上的实验结果表明,所提方法具有高效性、可行性和可扩展性,对于控制不良突发信息传播、社交网络舆情监控具有一定的支撑作用。     

基于PageRank的微博用户影响力算法研究

随着Web技术的发展,微博已经成为最受欢迎的社交平台之一了。在中国,微博用户规模已经达到了2.42亿。微博用户影响力计算对社会信息在微博里面有效传播,正确传播,健康传播有着非常重要的意义。本文以新浪微博数据为实验的对象,通过改进传统的PageRank模型,提出了的新型用户影响力排名算法---MBUI-Rank(Micro-Blog User Influence Rank)算法。在考虑传统PageRank方法的用户链接关系的同时,MBUI-Rank算法还考虑到微博用户自身行为活动,构建用户对微博的影响的动态挖掘模型。实验结果表明,MBUI-Rank算法与传统的PageRank算法相比,可以更加真实有效地反映微博用户的实际影响力。     

面向微博话题的用户影响力分析算法

微博用户影响力分析作为社交网络分析的重要组成部分，一直受到研究人员的关注。针对现有研究工作分析用户行为时间性的不足和忽略用户与参与话题之间关联性等问题，提出了一种面向微博话题的用户影响力分析算法——基于话题和传播能力的用户排序（TSRank）算法。首先，基于微博话题分析用户转发行为时间性，进一步构建用户转发和用户博文转发两种话题转发关系网络，预测用户话题信息传播能力；然后，分析用户个人历史微博和背景话题微博文本内容，挖掘用户与背景话题之间的关联性；最后，综合考虑用户话题信息传播能力以及用户与背景话题间关联性计算微博用户影响力。爬取新浪微博真实话题数据进行实验，实验结果表明，话题关联度更高用户的话题转发量明显大于关联度很低的用户，引入用户转发行为时间性相比无转发时间性，TSRank算法的捕获率（CR）提高了18.7%，进一步与典型影响力分析算法WBRank、TwitterRank和PageRank相比，TSRank算法在准确率和召回率上分别提高了5.9%、8.7%、13.1%和6.7%、9.1%、14.2%，验证了TSRank算法的有效性。该研究成果对社交网络的社会属性、话题传播等理论研究以及好友推荐、舆情监控等应用研究具有支撑作用。     

SocialRobot: a big data-driven humanoid intelligent system in social media services

The blooming of social media services makes them the attractive resources for publishing and seeking information (posting, reposting and searching tweets) as well as socializing and interacting with other users (following and messaging other users) in social media services. With the increasing number of users and the interacting frequency between users, tremendous user-generated contents are bursting out every day. Hence, users may face a overload of information. To address the above problem, in this paper, we present SocialRobot, a humanoid intelligent system that has been deployed in Sina Weibo. For its socialization characteristic, we naturally implement it as a virtual user in Sina Weibo. By following and interacting with SocialRobot, high-quality and user-interested content can be recommended to the users from the big social media data. And the need for searching specific information can be satisfied by chatting with the SocialRobot. A crowdsourcing evaluation shows the effective performance of the SocialRobot on resolving the information overload.     

基于Python的新浪微博数据爬虫

目前很多的社交网络研究都是采用国外的平台数据,而国内的新浪微博没有很好的接口方便研究人员采集数据进行分析。为了快速地获取到微博中的数据,开发了一款支持并行的微博数据抓取工具。该工具可以实时抓取微博中指定用户的粉丝信息、微博正文等内容;该工具利用关键字匹配技术,匹配符合规定条件的微博,并抓取相关内容;该工具支持并行抓取,可以同时抓取多个用户的信息。最后将串行微博爬虫工具和其并行版本进行对比,并使用该工具对部分微博数据作了一个关于流感问题的分析。实验结果显示:并行爬虫拥有较好的加速比,可以快速地获取数据,并且这些数据具有实时性和准确性。     

Analysis of and defense against crowd-retweeting based spam in social networks

Social networking websites with microblogging functionality, such as Twitter or Sina Weibo, have emerged as popular platforms for discovering real-time information on the Web. Like most Internet services, these websites have become the targets of spam campaigns, which contaminate Web contents and damage user experiences. Spam campaigns have become a great threat to social network services. In this paper, we investigate crowd-retweeting spam in Sina Weibo, the counterpart of Twitter in China. We carefully analyze the characteristics of crowd-retweeting spammers in terms of their profile features, social relationships and retweeting behaviors. We find that although these spammers are likely to connect more closely than legitimate users, the underlying social connections of crowd-retweeting campaigns are different from those of other existing spam campaigns because of the unique features of retweets that are spread in a cascade. Based on these findings, we propose retweeting-aware link-based ranking algorithms to infer more suspicious accounts by using identified spammers as seeds. Our evaluation results show that our algorithms are more effective than other link-based strategies.

     

Inferring urban air quality based on social media

Outdoor air pollution is a serious environmental problem in many developing countries; obtaining timely and accurate information about urban air quality is a first step toward air pollution control. Many developing countries however, do not have any monitoring stations and therefore the means to measure air quality. We address this problem by using social media to collect urban air quality information and propose a method for inferring urban air quality in Chinese cities based on China's largest social media platform, Sina Weibo combined with other meteorological data. Our method includes a data crawler to locate and acquire air-quality associated historical Weibo data, a procedure for extracting indicators from these Weibo and factors from meteorological data, a model to infer air quality index (AQI) of a city based on the extracted Weibo indicators supported by meteorological factors. We implemented the proposed method in case studies at Beijing, Shanghai, and Wuhan, China. The results show that based the Weibo indicators and meteorological factors we extracted, this method can infer the air quality conditions of a city within narrow margins of error. The method presented in this article can aid air quality assessment in cities with few or even no air quality monitoring stations.     

基于节点属性和信息价值的V-UKIR传播模型研究及仿真

为更有效地在网络中进行广告信息推广和谣言控制,针对真实社交网络信息传播的特点,对信息传播机制进行了研究。首先,在传统SIR模型的基础上,将节点状态分为未知、阅读、感染、免疫状态。然后,考虑节点自身属性的差别和不同信息在不同社区中的价值,定义了自适应节点状态转移概率函数,建立了更加符合社交网络真实传播特点的V-UKIR模型。最终,通过不同信息传播过程对比,验证了该模型的有效性。同时,通过在不同模型下的实际传播对比,证明了该模型在新浪微博和Twitter上有较好的信息推广和谣言控制效果。     

User interest mining via tags and bidirectional interactions on Sina Weibo

Sina Weibo, one of the biggest social services in China, provides users with opportunities to share information and express their personal views, leading an explosive growth of information. How to recommend the right information to the proper person among massive data has received considerable critical attention in recent years. One of the main obstacles is the access to user topic interests. In this paper, we proposed an algorithm based on tags and bidirectional interactions to mine user topic interests on Sina Weibo. The algorithm, formulated by user interaction graph, fully takes advantage of the discordance between user interactions. Forward spread and back spread are thus utilized to update tag spread weights. We also quantify the impact of these two spread by tuning parameters on three sub data sets. In order to prove the superiority of the algorithm, we compare our algorithm with famous methods on Sina Weibo. The result demonstrates that our new algorithm outperforms other methods both in precision rate and recall rate, with the ability of mining user interest effectively with respect to tags and bidirectional interactions.     

融合用户历史传播信息的微博谣言检测

随着互联网技术的发展,以微博为主的社交媒体平台上网络谣言逐渐泛滥,研究微博谣言的自动检测对维护社会稳定具有重要意义。现今主流的基于深度学习的谣言检测方法普遍存在没有充分考虑微博文本语义信息的问题,同时,过分依赖传播信息的谣言检测方法使得检测时间滞后,不能满足谣言检测的现实需求。针对以上问题,本文提出一种融合用户历史交互信息的微博谣言检测模型,不使用待检测微博的传播信息,构建并训练AbaNet(ALBERT-BiGRU-Attention)深度学习网络模型,充分考虑待检测微博和用户历史传播信息文本的文本特征和语义信息进行谣言检测。实验结果显示,本文模型具有准确率高、稳定性强的特点,并且能够在获得较高检测精度的情况下大大缩短谣言检测的时间。     

基于关联关系的微博用户可信度分析方法

随着微博研究的深入,对微博用户可信度的评价成为一个研究热点。针对微博用户可信度评价的问题,提出了一种基于关联关系的用户可信度分析方法。以新浪微博为研究对象,首先从用户的资料信息、交互信息和行为信息三个方面出发,分析了用户的7个相关特征,利用层次分析法（AHP）,进而得到用户自评价可信度;然后以用户自评价作为基点,以用户关系网络作为载体,结合用户之间潜在的用户互评关系,通过改进PageRank算法,提出了用户可信度评价模型User-Rank,进而,利用关系网络中其他用户对待分析用户的可信度进行综合评价。大规模的微博真实数据的实验表明,所提方法能够取得良好的用户可信度评价效果。     

基于人工免疫危险理论的微博水军用户检测研究

将人工免疫危险理论引入到用户行为特征的分析中,以有效地识别微博水军用户。以新浪微博为例,分析了新浪微博水军的行为特征,选取微博总数、微博等级、是否认证、阳光信用、粉丝数等特征属性,将属性分析结果作为区别水军与正常用户的特征信号,并基于树突状细胞算法(Dendritic Cells Algorithm,DCA)实现新浪微博水军的识别。使用新浪微博用户的真实数据对算法的有效性进行了验证和对比实验,结果表明该方法能够有效检测出新浪微博中的水军用户,具有较高的检测准确率。     

The Influence of Network Structural Properties on Information Dissemination Power in Microblogging Systems

The potential of microblogging networks to spread information, ideas, and influence via social links has been recognized. Individuals at central or critical positions in the microblogging networks are expected to play an important role in spreading information, but this belief must be tested with empirical investigations in actual information dissemination. Furthermore, it is possible that individuals with different types of structural importance influence the information dissemination in different ways. This article examines the impact of five structural properties (degree, betweenness, closeness, tie strength, and k-value) on information dissemination capabilities by tracking the dissemination of 150 messages in Sina Weibo. The results highlight the importance of betweenness centrality, which best explains the influence on information dissemination distance and coverage size. It also best predicts the information dissemination efficiency. Degree centrality is found to be a strong predictor of one’s influence among immediate followers. Tie strength negatively correlates with influence on the information dissemination distance and coverage size. These structural properties address different aspects of the information dissemination power and should be used in a complementary way when planning information dissemination strategies in microblogging systems.     

新浪微博谣言检测研究

社会网络信息的可信度问题近年来受到了相当大的关注。谣言的散播可能造成社会恐慌,引发信任危机。在国内,新浪微博用户量的快速增长,使其成为了谣言传播的温床。及时清理在新浪微博中传播的谣言,对于社会的和谐发展有着现实的意义。该文以新浪微博为背景,将谣言检测任务作为分类问题,首次提出将微博评论的情感倾向作为谣言检测分类器的一项特征。实验结果表明,引入评论的评论情感倾向特征后,使得谣言检测的分类结果得到了可观的提升。     

面向网络社交媒体的少样本新冠谣言检测

在社交媒体上发布和传播有关新冠的谣言对民生、经济、社会等都产生了严重影响,因此通过机器学习和人工智能技术开展新冠谣言检测具有重要的研究价值和社会意义.现有谣言检测研究,一般假定进行建模和预测的事件已有充足的有标签数据,但对于新冠这类突发事件,由于可训练样本较少,所以此类模型存在局限性.该文聚焦少样本谣言检测问题,旨在使...     

基于领域的影响力最大化与病毒营销

近年来随着新浪微博、人人网等社交网络新媒体的涌现,线上影响力传播得到了越来越多企业和研究机构的关注。如何在给定资源的约束下实现最大的传播范围(影响力最大化问题),对病毒营销等市场战略的有效开展有着重要意义。如果能充分利用社交网络上的异质性信息来更准确地定位用户所属的领域,进而基于领域实现影响力最大化,将对从整体角度出发的传统研究和片面的结构或内容角度的研究形成很好的补充。该文同时利用新浪微博上用户之间的社交关系和微博内容的话题两个维度的信息将用户划分为不同的领域;进而提出了一种基于贪心和动态规划混合的改良算法实现基于领域的影响力最大化。实验表明该文的领域影响力模型较好优化了传统影响力最大化的时间消耗,同时拥有相近的精度。     

Topic Evolution Analysis in Social Networking Services: Taking Sina Weibo as an Example

Event-related topics in social networking services are always the epitome of heated society issues, therefore determining the significance of analyzing its evolution patterns. In this paper, we present a comprehensive survey on the tweets about "ransomware" in Sina Weibo, a famous social networking service similar to twitter in China. The keyword corresponds to a global ransomware attack in May 2017, on which our example event-related topics are based. We collect text data from sina Weibo and vectorize each tweets, before using a dynamic topic model to discover the event-related topics. The results of the topic model are explainable enough and help us to understand the evolution of those topics more thoroughly.     

基于多特征信息传播模型的微博意见领袖挖掘

在线社交网络中的意见领袖通常是指在社交网络的信息传播中具有较大社会影响力的个体。针对当前意见领袖挖掘方法中只考虑社交网络的拓扑结构和节点的个体属性,缺乏信息传播中交互特征的问题,该文提出了基于扩展独立级联模型,并融入网络结构特征、个体属性和行为特征的意见领袖挖掘模型(extended independent cascade, EIC)。该模型以个体属性、个体在信息传播过程中的交互行为建立加权的传播网络,利用改进的CELF(cost effective lazy forward)算法,挖掘网络中影响力较大的个体。通过实验验证,在意见领袖的扩展核心率指标上,该算法优于拓扑结构类算法,且具有较好的稳定性,同时并未降低意见领袖的传播范围。     

基于弱监督预训练深度模型的微博情感分析

微博情感分析对于商业事务和政治选举等应用非常重要。传统的做法主要基于浅层机器学习模型,对人工提取的特征有较大的依赖,而微博情感特征往往难以提取。深度学习可以自动学习层次化的特征,并被用于解决情感分析问题。随着新的深度学习技术的提出,人们发现只要提供足够多的监督数据,就能训练出好的深度模型。然而,在微博情感分析中,通常监督数据都非常少。微博中广泛存在着弱监督数据。该文提出基于弱监督数据的“预训练—微调整”训练框架(distant pretrain-finetune),使用弱监督数据对深度模型进行预训练,然后使用监督数据进行微调整。这种做法的好处是可以利用弱监督数据学习到一个初始的模型,然后利用监督数据来进一步改善模型并克服弱监督数据存在的一些问题。我们在新浪微博数据上进行的实验表明,这种做法可以在监督数据较少的情况下使用深度学习,并取得比浅层模型更好的效果。