Identifying urban crowds using geo-located Social media data: a Twitter experiment in New York City

The massive growth of GPS equipped smartphones coupled with the increasing importance of Social Media has led to the emergence of new services over LBSNs (Location-based Social Networks) where both, opinions and location, are shared. This proactive attitude allow us to consider citizens as sensors in motion whose information supports our approach: monitoring multitudes or crowds all around the city. More specifically, our proposal is mining geotagged data from LBSNs in order to analyze crowds according to different parameters as size, duration, composition, motivation, cohesion and proximity. This analysis is gathered under a methodology for crowd detection in cities that combines social data mining, density-based clustering and outlier detection into a solution that can operate on-the-fly. This methodology enables foreseeing crowds in short term based on the prior analysis of time and previous behavior of individuals in the geographical area under study. Our approach was validated using Twitter, as public social network par excellence, to analyze geotagged data in New York City on a normal day (reference day) and on New Year’s Eve, as the study day, when public crowds are expected.     

Bird's‐Eye ‐ Large‐Scale Visual Analytics of City Dynamics using Social Location Data

The analysis of behavioral city dynamics, such as temporal patterns of visited places and citizens' mobility routines, is an essential task for urban and transportation planning. Social media applications such as Foursquare and Twitter provide access to large‐scale and up‐to‐date dynamic movement data that not only help to understand the social life and pulse of a city but also to maintain and improve urban infrastructure. However, the fast growth rate of this data poses challenges for conventional methods to provide up‐to‐date, flexible analysis. Therefore, planning authorities barely consider it. We present a system and design study to leverage social media data that assist urban and transportation planners to achieve better monitoring and analysis of city dynamics such as visited places and mobility patterns in large metropolitan areas. We conducted a goal‐and‐task analysis with urban planning experts. To address these goals, we designed a system with a scalable data monitoring back‐end and an interactive visual analytics interface. The monitoring component uses intelligent pre‐aggregation to allow dynamic queries in near real‐time. The visual analytics interface leverages unsupervised learning to reveal clusters, routines, and unusual behavior in massive data, allowing to understand patterns in time and space. We evaluated our approach based on a qualitative user study with urban planning experts which demonstrates that intuitive integration of advanced analytical tools with visual interfaces is pivotal in making behavioral city dynamics accessible to practitioners. Our interviews also revealed areas for future research.     

From Mobile Phone Sensing to Human Geo‐Social Behavior Understanding

The study of geo‐social behaviors has long been a scientific problem. In contrast to traditional social science, which suffers from the problems such as high data collection cost and imported user subjectivity, a new approach is presented to study social behaviors based on mobile phone sensing data. Different from other similar studies on mobile social sensing, three different types of geo‐social behaviors, including online interaction, offline interaction, and mobility patterns, are characterized based on a newly released Nokia mobile phone data set. We further discuss the impact factors to these behaviors as well as the correlation among them. The findings in this article are crucial for many different fields, ranging from urban planning, location‐based services, to social recommendation.     

DynamiCITY: Revealing city dynamics from citizens social media broadcasts

Social media and mobile devices have revolutionized the way people communicate and share information in various contexts, such as in cities. In today’s “smart” cities, massive amounts of multiple forms of geolocated content is generated daily in social media, out of which knowledge for social interactions and urban dynamics can be derived. This work addresses the problem of detecting urban social activity patterns and interactions, by modeling cities into “dynamic areas”, i.e., coherent geographic areas shaped through social activities. Social media users provide the information on such social activities and interactions in cases when they are on the move around the city neighborhoods. The proposed approach models city places as feature vectors which represent users visiting patterns (social activity), the time of observed visits (temporal activity), and the context of functionality of visited places category. To uncover the dynamics of city areas, a clustering approach is proposed which considers the derived feature vectors to group people’s activities with respect to location, time, and context. The proposed methodology has been implemented on the DynamiCITY platform which demonstrates neighborhood analytics via a Web interface that allows end-users to explore neighborhoods dynamics and gain insights for city cross-neighborhood patterns and inter-relationships.     

基于ACP的动态网民群体运动组织建模与计算实验研究

由互联网促成的社会运动组织一经出现, 就受到了广大社会学者以及计算机领域专家的广泛关注. 一方面, 互联网特别是移动互联网在整合信息、引发共振、实时分享及高度互动等方面的特性, 为网民行为的大规模快速聚集提供了直通渠道, 使得多角度超视距观察并研究在线人群复杂行为及其组织特性成为可能; 另一方面, 这一研究在社会化媒体营销、共享经济、非军事组织行动中的应用意义愈加显著. 本文引入群体行为动力学和社会运动组织理论的研究, 提出基于ACP的动态网民群体运动组织(Cyber movement organizations, CMOs)研究方法. 本文工作首先使用多智能体建模方法构造双层结构的人工社区模型, 以此为基础对动态网民的个体以及群体动态组织行为展开计算实验探讨, 重点阐释了社区用户的交互行为机制及群体组织活动的建模机制, 为揭示微观个体简单行为对于宏观群体复杂涌现现象的影响奠定基础.     

基于区域语义的城市移动模式可视分析

针对城市区域语义及移动模式难以提取的问题,提出一种基于区域语义的城市移动模式可视分析方法用于直观地分析人群出行情况.通过提取用户通话特征,使用高斯混合模型区分基站通话模式来发现城市区域的功能性信息;进一步使用层次聚类算法对用户行为进行语义发现,分析区域用户行为规律;区域语义与用户语义结合分析,挖掘人群在区域间的移动模式.案例分析表明,该方法能有效地发现区域功能特征,结合数据能帮助分析人员发现城市间移动模式以及探索用户移动意图,得到用户移动模式和功能区域之间的联系.     

一种大数据环境下的在线社交媒体位置推断方法

随着在线社交媒体的快速发展和可定位设备的大量普及,地理位置作为社交媒体大数据中一种质量极高的信息资源,开始在疾病控制、人口流动性分析和广告精准投放等方面得到广泛应用.但是,由于大量用户没有指定或者不能准确指定位置,社交媒体上的地理位置数据十分稀疏.针对此数据稀疏性问题,提出一种基于用户生成内容的位置推断方法UGC-LI(user generate content driven location inference method),实现对社交媒体用户和生成文本位置的推断,为基于位置的个性化信息服务提供数据支撑.通过抽取用户生成文本中的本地词语,构建一个基于词汇地理分布差异和用户社交图谱的概率模型,在多层次的地理范围内推断用户位置.同时,提出一个基于位置的参数化语言模型,计算用户生成文本发出的城市.在真实数据集上进行的评估实验表明:UGC-LI方法能够在15km偏移距离准确定位64.2%的用户,对用户所在城市的推断准确率达到81.3%;同时,可正确定位32.7%的用户生成文本发出的城市,与现有方法相比有明显的提高.     

Urban area characterization based on crowd behavioral lifelogs over Twitter

Recent location-based social networking sites are attractively providing us with a novel capability of monitoring massive crowd lifelogs in the real-world space. In particular, they make it easier to collect publicly shared crowd lifelogs in a large scale of geographic area reflecting the crowd’s daily lives and even more characterizing urban space through what they have in minds and how they behave in the space. In this paper, we challenge to analyze urban characteristics in terms of crowd behavior by utilizing crowd lifelogs in urban area over the social networking sites. In order to collect crowd behavioral data, we exploit the most famous microblogging site, Twitter, where a great deal of geo-tagged micro lifelogs emitted by massive crowds can be easily acquired. We first present a model to deal with crowds’ behavioral logs on the social network sites as a representing feature of urban space’s characteristics, which will be used to conduct crowd-based urban characterization. Based on this crowd behavioral feature, we will extract significant crowd behavioral patterns in a period of time. In the experiment, we conducted the urban characterization by extracting the crowd behavioral patterns and examined the relation between the regions of common crowd activity patterns and the major categories of local facilities.     

Discovering socially important locations of social media users

Socially important locations are places that are frequently visited by social media users in their social media life. Discovering socially interesting, popular or important locations from a location based social network has recently become important for recommender systems, targeted advertisement applications, and urban planning, etc. However, discovering socially important locations from a social network is challenging due to the data size and variety, spatial and temporal dimensions of the datasets, the need for developing computationally efficient approaches, and the difficulty of modeling human behavior. In the literature, several studies are conducted for discovering socially important locations. However, majority of these studies focused on discovering locations without considering historical data of social media users. They focused on analysis of data of social groups without considering each user’s preferences in these groups. In this study, we proposed a method and interest measures to discover socially important locations that consider historical user data and each user’s (individual’s) preferences. The proposed algorithm was compared with a naïve alternative using real-life Twitter dataset. The results showed that the proposed algorithm outperforms the naïve alternative.     

Informationsunschärfe in Big Data

While the classic definition of Big Data included the dimensions volume, velocity, and variety, a fourth dimension, veracity, has recently come to the attention of researchers and practitioners. The increasing amount of user-generated data associated with the rise of social media emphasizes the need for methods to deal with the uncertainty inherent to these data sources. In this paper we address one aspect of uncertainty by developing a new methodology to establish the reliability of user-generated data based upon causal links with recurring patterns. We associate a large data set of geo-tagged Twitter messages in San Francisco with points of interest, such as bars, restaurants, or museums, within the city. This model is validated by causal relationships between a point of interest and the amount of messages in its vicinity. We subsequently analyze the behavior of these messages over time using a jackknifing procedure to identify categories of points of interest that exhibit consistent patterns over time. Ultimately, we condense this analysis into an indicator that gives evidence on the certainty of a data set based on these causal relationships and recurring patterns in temporal and spatial dimensions.     

Quantify city-level dynamic functions across China using social media and POIs data

Location-aware big data from social media have been widely used to study functions of different zones in a city but not across a city as a whole. In this study, a novel framework is proposed to quantify city-level dynamic functions of 200 cities in China from a perspective of collective human activities. The random forest model was used to determine the temporal variations in the proportions of different urban functions by examining the relationship between Points-of-Interest (POIs) and Tencent Location Request (TLR) data. We then hierarchically clustered and analyzed the structures and distribution patterns of the dynamic urban functions of 200 Chinese cities at different temporal scales. In the end, we calculated an urban functional equilibrium index based on the urban functional proportion and then mapped spatial distribution patterns of the indexes across mainland China. Results show that on a daily scale when the cities were grouped into two clusters, they are either dominated by the work/education and commerce or residence functions. The cities in the former cluster are mainly the provincial capitals and located within major urban agglomerations. When the cities were grouped into four clusters, the clusters are dominated their commerce, work, residence, and balanced multiple functions, respectively. For each of the 200 cities, its urban functions change dynamically from the daybreak to the evening in terms of human activities. Besides, the equilibrium indexes show a power-law relationship with their rankings. Our research shows that city-level dynamic function can be quantified from the perspective of variations in human activities by using social media big data that otherwise could not be achieved in the conventional urban functions’ studies.     

Mobile Geotagging: Reexamining Our Interactions with Urban Space

Mobile geotagging services offer people new ways to interact with and through urban space. In this paper, we focus on a mobile geotagging service called Socialight and the social practices associated with it. In‐depth interviews and participant observation were conducted in order to explore how Socialight's virtual “sticky notes” were used in everyday life. Findings indicated how users communicate about place to help build social familiarity with urban places and communicate through place to allow users to create place‐based narratives and engage in identity management. Such findings deepen our understanding of the social production of space and have implications for future location‐based mobile services.     

A paradigm for controlling virtual humans in urban environment simulations

This paper presents a new architecture for simulating virtual humans in complex urban environments. The approach is based on the integration of six modules. Four key modules are used in order to manage environmental data, simulate human crowds, control interactions between virtual humans and objects, and generate tasks based on a rule-based behavioral model. The communication between these modules is made through a client/server system. Finally, all low-level virtual human actions are delegated to a single motion and behavioral control module. Our architecture combines various human and object simulation aspects, based on the coherent extraction and classification of information froma virtual city database. This architecture is discussed in this paper, together with a detailed case study example.     

Uncovering inconspicuous places using social media check-ins and street view images

There is a Chinese proverb, “if your wine tastes really good, you do not need to worry about the location of your bar (酒香不怕巷子深)”, which implies that the popular places for local residents are sometimes hidden behind an unassuming door or on unexpected streets. Discovering these unassuming places (e.g. restaurants) of a city will benefit the understanding of local culture and help to build livable neighborhoods. Previous work has been limited by the lack of appropriate data sources and efficient tools to evaluate the popularity, ambiance and physical surroundings of places in large-scale urban areas. In addition, how to characterize places with respect to different groups of people remains unclear. In this work, we propose a data-driven approach using social media check-ins and street-level images to compare the different activity patterns of visitors and locals, and uncover inconspicuous but interesting places for them in a city. We use check-in records as a proxy of the popularity of a particular type of place, and differentiate visitors and locals based on their travel and social media behaviors. In addition, we employ street-level images to represent the physical environments of places. As a result, we discovered a number of inconspicuous yet popular restaurants in Beijing. These restaurants are located mostly in deep alleys of Old Beijing neighborhoods, where the physical environments are not particularly appealing; however, these places are frequently visited by locals for social engagements. We also discovered beautiful but unpopular outdoor places in Beijing. These places are potential recreational areas for all groups of people and could be improved regarding urban design and planning to make these public infrastructures more attractive. This work demonstrates how multi-source big geo-data can be combined to build comprehensive place-based representations for different groups of people.     

Monitoring social-distance in wide areas during pandemics: a density map and segmentation approach

With the relaxation of the containment measurements around the globe, monitoring the social distancing in crowded public spaces is of great importance to prevent a new massive wave of COVID-19 infections. Recent works in that matter have limited themselves by assessing social distancing in corridors up to small crowds by detecting each person individually, considering the full body in the image. In this work, we propose a new framework for monitoring the social-distance using end-to-end Deep Learning, to detect crowds violating social-distancing in wide areas, where important occlusions may be present. Our framework consists in the creation of new ground truth social distance labels, based on the ground truth density maps, and the proposal of two different solutions, a density-map-based and a segmentation-based, to detect crowds violating social-distancing constraints. We assess the results of both approaches by using the generated ground truth from the PET2009 and CityStreet datasets. We show that our framework performs well at providing the zones where people are not following the social-distance, even when heavily occluded or far away from the camera, compared to current detection and tracking approaches.

     

Inferring urban air quality based on social media

Outdoor air pollution is a serious environmental problem in many developing countries; obtaining timely and accurate information about urban air quality is a first step toward air pollution control. Many developing countries however, do not have any monitoring stations and therefore the means to measure air quality. We address this problem by using social media to collect urban air quality information and propose a method for inferring urban air quality in Chinese cities based on China's largest social media platform, Sina Weibo combined with other meteorological data. Our method includes a data crawler to locate and acquire air-quality associated historical Weibo data, a procedure for extracting indicators from these Weibo and factors from meteorological data, a model to infer air quality index (AQI) of a city based on the extracted Weibo indicators supported by meteorological factors. We implemented the proposed method in case studies at Beijing, Shanghai, and Wuhan, China. The results show that based the Weibo indicators and meteorological factors we extracted, this method can infer the air quality conditions of a city within narrow margins of error. The method presented in this article can aid air quality assessment in cities with few or even no air quality monitoring stations.     

Portraying the spatial dynamics of urban vibrancy using multisource urban big data

Understanding urban vibrancy aids policy-making to foster urban space and therefore has long been a goal of urban studies. Recently, the emerging urban big data and urban analytic methods have enabled us to portray citywide vibrancy. From the social sensing perspective, this study presents a comprehensive and comparative framework to cross-validate urban vibrancy and uncover associated spatial effects. Spatial patterns of urban vibrancy indicated by multisource urban sensing data (points-of-interest, social media check-ins, and mobile phone records) were investigated. A comprehensive urban vibrancy metric was formed by adaptively weighting these metrics. The association between urban vibrancy and demographic, economic, and built environmental factors was revealed with global regression models and local regression models. An empirical experiment was conducted in Shenzhen. The results demonstrate that four urban vibrancy metrics are all higher in the special economic zone (SEZ) and lower in non-SEZs but with different degrees of spatial aggregation. The influences of employment and road density on all vibrancy metrics are significant and positive. However, the effects of metro stations, land use mix, building footprints, and distance to district center depend on the vibrancy indicator and location. These findings unravel the commonalities and differences in urban vibrancy metrics derived from multisource urban big data and the hidden spatial dynamics of the influences of associated factors. They further suggest that urban policies should be proposed to foster vibrancy in Shenzhen therefore benefit social wellbeing and urban development in the long term. They also provide valuable insights into the reliability of urban big data-driven urban studies.     

Discovering public sentiment in social media for predicting stock movement of publicly listed companies

The popularity of many social media sites has prompted both academic and practical research on the possibility of mining social media data for the analysis of public sentiment. Studies have suggested that public emotions shown through Twitter could be well correlated with the Dow Jones Industrial Average. However, it remains unclear how public sentiment, as reflected on social media, can be used to predict stock price movement of a particular publicly-listed company. In this study, we attempt to fill this research void by proposing a technique, called SMeDA-SA, to mine Twitter data for sentiment analysis and then predict the stock movement of specific listed companies. For the purpose of experimentation, we collected 200 million tweets that mentioned one or more of 30 companies that were listed in NASDAQ or the New York Stock Exchange. SMeDA-SA performs its task by first extracting ambiguous textual messages from these tweets to create a list of words that reflects public sentiment. SMeDA-SA then made use of a data mining algorithm to expand the word list by adding emotional phrases so as to better classify sentiments in the tweets. With SMeDA-SA, we discover that the stock movement of many companies can be predicted rather accurately with an average accuracy over 70%. This paper describes how SMeDA-SA can be used to mine social media date for sentiments. It also presents the key implications of our study.     

A large-scale study of cultural differences using urban data about eating and drinking preferences

Traditional ways to study urban social behavior, e.g. surveys, are costly and do not scale. Recently, some studies have been showing new ways of obtaining data through location-based social networks (LBSNs), such as Foursquare, which could revolutionize the study of urban social behavior. We use Foursquare check-ins to represent user preferences regarding eating and drinking habits. Considering datasets differing in terms of volume of data and observation window size, our results indicate that spatio-temporal eating and drinking habits of users voluntarily expressed in LBSNs has the potential to explain cultural habits of the users. From this, we propose a methodology to identify cultural boundaries and similarities across populations at different scales, e.g., countries, cities, or neighborhoods. This methodology is extensively evaluated in several aspects. For instance, by proposing some variations of it disregarding some of the considered dimensions, as well as analyzing the results using datasets from different periods and window of observation. The results indicate that our proposed methodology is a promising approach for automatic cultural habits separation, which could enable new urban services.