基于云技术的智慧城市中大规模房屋数据处理

在智慧城市中传感器和设备收集各种数据,实现增值服务.主要关注来自智慧城市中的智能房屋数据,提出用于存储和处理大规模房屋数据的Scallop4SC平台,实现智慧城市服务.房屋数据分为房屋日志数据和房屋配置数据,分别使用HBase和MYSQL管理.通过HBase键值以schemaless方式存储管理异构日志数据.MySQL管理配置数据,完成房屋数据查询.多节点Hadoop/Map Reduce集群处理巨量房屋日志数据.最后通过实际一年的房屋日志数据,计算智能设备能耗并完成实验性评估.根据实验结果,探讨Scallop4SC处理智慧城市的数据规模.

Using Cloud Technologies for Large-Scale House Data in Smart City

In the smart city environment, a wide variety of data are collected from sensors and devices to achieve value-added services. In this paper, we especially focus on data taken from smart houses in the smart city, and propose a platform, called Scallop4SC, that stores and processes the large-scale house data. The house data is classified into log data or configuration data. Since the amount of the log is extremely large, we introduce the Hadoop/MapReduce with a multi-node cluster. On top of this, we use HBase key-value store to manage heterogeneous log data in a schemaless manner. On the other hand, to manage the configuration data, we choose MySQL to process various queries to the house data efficiently. We propose practical data models of the log data and the configuration data on HBase and MySQL, respectively. Then we show how Scallop4SC works as an efficient data platform for smart city services. We conduct an experimental evaluation to calculate device-wise energy consumption, using actual house log recorded for one year in our smart house. Based on the result, we discuss the applicability of Scallop4SC to city-scale data processing.