基于Hadoop的气象信息数据仓库建立与测试

气象相关的数据随着气象事业现代化水平的不断提高而与日俱增,使得气象部门对于气象数据存储、管理和读取的要求越来越高;通过分析开源云平台Hadoop的分布式文件系统HDFS、数据仓库工具Hive等架构,研究了Hadoop气象云平台的构建过程,最终实现了气象信息数据仓库的建立与测试;该气象信息数据仓库实现了海量气象数据文件的分布式存储、元数据管理以及气象数据的查询;应用表明,使用气象数据仓库进行大型气象数据文件存储和操作时,可以大大提升数据吞吐率和数据读写操作效率。

Establishment and Test of Meteorological Data Warehouse Based on Hadoop

With the development of the meteorological modernization level,the related data is steadily on increase.It is very important for meteorological department to improve the data storage,management,and read/written rate.The establishment and testing of the meteorological information data warehouse are finally implemented by analyzing the infrastructure of the distributed file system HDFS and data warehouse tools Hive in the open source platform Hadoop.Many functions are realized in this warehouse,such as distributed storage of massive meteorological data file,the management of metadata and data query.The experiment finally shows that it can enhance data throughput and data reading/written efficiency when storing and operating large-scale meteorological data file by using this warehouse.