|
|||||
|
|
| 大数据技术在环境信息中的应用 | |
| 引用本文： | 李安增,王宁,王常权,邱燕. 大数据技术在环境信息中的应用[J]. 计算机系统应用, 2015, 24(1): 60-64 |
| 作者姓名： | 李安增 王宁 王常权 邱燕 |
| 作者单位： | 1. 中国科学院大学,北京100049;中国科学院沈阳计算技术研究所,沈阳110168 2. 中国科学院沈阳计算技术研究所,沈阳,110168 3. 中国核示范电站有限责任公司,荣成,264312 4. 德阳广播电视大学,德阳,618000 |
| 基金项目： | 国家水体污染控制与治理科技重大专项(2012ZX07505003) |
| 摘 要： | 在“辽河流域水环境管理技术综合示范”项目中,随着时间的累积,环境监测数据处理系统采集到的数据量越来越大.然而目前辽宁省环境监测数据处理系统无法有效处理日益增长的海量数据.研究运用大数据技术,改进环境监测数据处理系统中的数据中心.利用HDFS强大的数据存储、管理功能,以应对数据量的增长,利用MapReduce及Hadoop其他相关工具,快速处理海量数据,降低数据规模,最后将数据存储到数据库中. |
| 关 键 词： | 大数据 MapReduce Hadoop 并行计算 分布式 |
| 收稿时间： | 2014-04-21 |
| 修稿时间： | 2014-05-19 |
Application of Big Data Technology to Environment Information |
|
| LI An-Zeng,WANG Ning,WANG Chang-Quan and QIU Yan. Application of Big Data Technology to Environment Information[J]. Computer Systems& Applications, 2015, 24(1): 60-64 | |
| Authors: | LI An-Zeng WANG Ning WANG Chang-Quan QIU Yan |
| Affiliation: | University of Chinese Academy of Sciences, Beijing 100049, China;Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang 110168, China;Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang 110168, China;China Nuclear Demonstration Power Plant Limited Liability Company, Rongcheng 264312, China;Deyang Radio and Television University, Deyang 618000, China |
| Abstract: | With the accumulation of time, data collected by environmental monitoring data processing system is more in the project of "Comprehensive Demonstration of Water Environment Management about Liao He River". However, Liaoning province environmental monitoring data processing system cannot effectively deal with massive data growing. In this paper, we study and use big data technology. After that, we improve the data center of the environmental monitoring data processing system. We can use HDFS powerful function of data storage and management to cope with the growth of the massive data and take advantage of MapReduce and other Hadoop related tools, rapidly processing massive data and reducing the data size. Finally, the data will be stored in the database. |
| Keywords: | big data MapReduce Hadoop parallel computing distributed |
| 本文献已被 万方数据 等数据库收录！ | |
| 点击此处可从《计算机系统应用》浏览原始摘要信息 | |
| 点击此处可从《计算机系统应用》下载全文 | |