| Sensor data compression based on MapReduce简 |
| |
| 作者姓名： | YU Yu GUO Zhong-wen |
| |
| 基金项目： | supported by the National Natural Science Foundation of China（60933011,61170258） |
| |
| 摘 要： | A compression algorithm is proposed in this paper for reducing the size of sensor data. By using a dictionary-based lossless compression algorithm, sensor data can be compressed efficiently and interpreted without decompressing. The correlation between redundancy of sensor data and compression ratio is explored. Further, a parallel compression algorithm based on MapReduce 1] is proposed. Meanwhile, data partitioner which plays an important role in performance of MapReduce application is discussed along with performance evaluation criteria proposed in this paper. Experiments demonstrate that random sampler is suitable for highly redundant sensor data and the proposed compression algorithms can compress those highly redundant sensor data efficiently.
|
| 关 键 词： | 数据压缩 传感器 无损压缩算法 冗余度 评价标准 应用程序 数据分区 解压缩 |
| 收稿时间： | 8 August 2013 |
Sensor data compression based on MapReduce |
| |
| YU Yu,GUO Zhong-wen.Sensor data compression based on MapReduce[J].The Journal of China Universities of Posts and Telecommunications,2014,21(1):60-66. |
| |
| Authors: | YU Yu GUO Zhong-wen |
| |
| Affiliation: | Department of Computer Science and Engineer, Ocean University of China, Qingdao 266100, China |
| |
| Abstract: | A compression algorithm is proposed in this paper for reducing the size of sensor data. By using a dictionary-based lossless compression algorithm, sensor data can be compressed efficiently and interpreted without decompressing. The correlation between redundancy of sensor data and compression ratio is explored. Further, a parallel compression algorithm based on MapReduce 1] is proposed. Meanwhile, data partitioner which plays an important role in performance of MapReduce application is discussed along with performance evaluation criteria proposed in this paper. Experiments demonstrate that random sampler is suitable for highly redundant sensor data and the proposed compression algorithms can compress those highly redundant sensor data efficiently. |
| |
| Keywords: | data compression sensor data MapReduce surveillance application measurement system |
| 本文献已被 CNKI 维普 ScienceDirect 等数据库收录！ |
