| 同构Hadoop环境作业执行时间计算方法 |
| |
| 引用本文： | 张霄宏,海林鹏,贾宗璞,沈记全,赵文涛. 同构Hadoop环境作业执行时间计算方法[J]. 计算机工程与应用, 2014, 50(10): 249-252 |
| |
| 作者姓名： | 张霄宏 海林鹏 贾宗璞 沈记全 赵文涛 |
| |
| 作者单位： | 1.中国科学院 深圳先进技术研究院，广东 深圳 5180552.河南理工大学 计算机科学与技术学院，河南 焦作 4540033.河南理工大学 现代教育中心，河南 焦作 454003 |
| |
| 基金项目： | 国家自然科学基金面上项目（No.51274088）;河南省教育厅项目（No.ITE12103）;河南理工大学矿山信息化省级重点实验室项目（No.KY2012-05）;河南理工大学博士基金项目（No.B2012-099）;河南省基础与前沿技术研究计划项目（No.122300410415）. |
| |
| 摘 要： | 执行时间是作业调度的重要参考因素之一。通过分析Hadoop MapReduce环境作业的执行特征，提出了以map任务和reduce任务执行时间为输入，估算作业执行时间的方法。该方法在一定假设条件下，借助作业预执行来获取map任务和reduce任务的执行时间。实验结果表明，该方法估算作业执行时间的误差率小于7%。
|
| 关 键 词： | HadoopMapReduce 作业执行时间 调度 |
Method for computing execution time of jobs in homogeneous hadoop environments |
| |
| ZHANG Xiaohong,HAI Linpeng,JIA Zongpu,SHEN Jiquan,ZHAO Wentao. Method for computing execution time of jobs in homogeneous hadoop environments[J]. Computer Engineering and Applications, 2014, 50(10): 249-252 |
| |
| Authors: | ZHANG Xiaohong HAI Linpeng JIA Zongpu SHEN Jiquan ZHAO Wentao |
| |
| Affiliation: | 1.Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, Guangdong 518055, China2.School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, Henan 454003, China3.Center of Modern Education, Henan Polytechnic University, Jiaozuo, Henan 454003, China |
| |
| Abstract: | Execution time is very important for job scheduling. In this paper, the execution characters of Hadoop MapReduce jobs are analyzed, and then a new method is proposed to compute the execution times of these jobs. The method takes the execution times of map task and reduce task as input data. It captures these execution times by pre-executing under an assumption. The method has been evaluated in a Linux cluster, the experiment results show that the method computed the execution times of jobs with the error rate no more than 7%. |
| |
| Keywords: | Hadoop MapReduce execution time scheduling |
| 本文献已被 CNKI 维普 等数据库收录！ |
| 点击此处可从《计算机工程与应用》浏览原始摘要信息 |
|
点击此处可从《计算机工程与应用》下载全文 |
|
