基于Hadoop集群的海量数据处理系统

近年来,随着各个领域中大规模、海量数据存储和处理需求的不断增加,集群作为一种廉价的可以提供强大计算能力的并行计算技术得到越来越广泛的应用,其具有大型机的超级计算能力和较低成本投入.从而成为各种高性能计算的主流方向,如科学计算与其他需要大规模并行计算的应用服务等.本文在分析现有分布式储存和计算等关键技术基础上,结合对Hadoop的集群技术的研究以及自身的业务需求和实际软硬件实力,提出了一种基于Hadoop的海量数据处理模型.

Massive Data Processing System Based On Hadoop Cluster

In recent years, with the continuous increase in various fields of massive data storage and processing requirements, Cluster comes out to be a cheap solution of parallel computation which can provide powerful computing capability.With it ＂ s large computer computing ability and low cost, In order to become the mainstream direction calculation of various high performance.For example, scientific computing and other large-scale parallel application service computing etc. Based on the analysis of the key technology of existing distributed storage and computing, combined with the research on cluster technology as well as the business needs and the actual hardware condition, this page puts forward a kind of Massive Data Processing System Based On Hadoop Cluster.