| Hadoop云平台的一种新的任务调度和监控机制 |
| |
| 引用本文： | 许 丞,刘 洪,谭 良. Hadoop云平台的一种新的任务调度和监控机制[J]. 计算机科学, 2013, 40(1): 112-117 |
| |
| 作者姓名： | 许 丞 刘 洪 谭 良 |
| |
| 作者单位： | (四川师范大学计算机科学学院 成都610068)；(中国科学院计算技术研究所 北京100190) |
| |
| 基金项目： | 国家自然科学基金(60970113);国家自然科学基金青年基金(60903073);四川省教育厅青年基金项目(08zb02);四川师范大学校级项目(11KYL03)资助 |
| |
| 摘 要： | 云平台任务监控与资源调度机制是云平台的核心功能之一。Hadoop云平台中任务监控和资源管理的任务是由JobTracker负责处理,并通过slave节点向其发送心跳消息来实现。这种方式导致JobTracker的负载过重,降低了Hadoop云平台的工作效率,限制了Hadoop云平台的规模。提出了一种新的任务监控方案,该方案将JobTracker的任务监控和资源管理功能分离,任务监控功能仍由JobTracker节点完成,资源管理功能由新增的资源管理节点完成,JobTracker通过增量更新的算法将任务调度所需的对象信息动态同步到资源管理节点上,资源管理节点根据心跳消息进行任务分配,并将分配结果返回给JobTracker节点。实验结果表明,本方案不仅通过监控节点实现了任务的监控,增加了监控的灵活性和鲁棒性,而且降低了Jobtracker节点的负担,可有效提高Hadoop云平台的工作效率和规模。
|
| 关 键 词： | 云计算 Hadoop 任务监控 任务调度 资源管理 增量更新算法 |
New Mechanism of Monitoring on Hadoop Cloud Platform |
| |
| XU Cheng,LIU Hong,TAN Liang. New Mechanism of Monitoring on Hadoop Cloud Platform[J]. Computer Science, 2013, 40(1): 112-117 |
| |
| Authors: | XU Cheng LIU Hong TAN Liang |
| |
| Affiliation: | 1,2(College of Computer,Sichuan Normal University,Chengdu 610068,China)1(Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China)2 |
| |
| Abstract: | Job monitoring and resource managing mechanism in cloud computing platform arc the core functions. The tasks of job monitoring and resource managing in Hadoop are assigned to JobTracker which is implemented on receiving heartbeat message sending by slave node, so JobTracker is the bottleneck of Hadoop. A new mechanism of job monitoring and resource managing scheme was presented. In this scheme,job monitoring and resource managing arc separated from original JobTracker, but job monitoring function is still assigned to JobTracker node, and the resource managingfunction is accomplish by newly added resource monitoring node. JobTracker sends the necessary information of Objects to resource managing node by using a new method of delta updates algorithm. Resource managing node schedules tasks with heartbeat message, and returns the result to Job Tracker node. Experimental results show that the scheme implemenu monitoring node monitor job completely,makes JobTracker node more flexible and robust, reduces the loading of Job Tracker node, improves the efficiency of Hadoop platform. |
| |
| Keywords: | Cloud computing Hadoop Task scheduling Resource managing Delta updating algorithm |
| 本文献已被 CNKI 等数据库收录！ |
|
点击此处可从《计算机科学》下载全文 |
|
