| 基于共享的MapReduce多查询优化技术 |
| |
| 引用本文： | 赵保学,李战怀,陈 群,潘 巍,姜 涛,金 健.基于共享的MapReduce多查询优化技术[J].计算机应用研究,2013,30(5):1405-1409. |
| |
| 作者姓名： | 赵保学 李战怀 陈 群 潘 巍 姜 涛 金 健 |
| |
| 作者单位： | 西北工业大学 计算机学院, 西安 710072 |
| |
| 基金项目： | 国家自然科学基金资助项目（61033007）; 国家“973”重点基础发展规划基金资助项目（2012CB316203）; 国家“863”高技术研究发展基金资助项目（2012AA011004）; 西北工业大学研究生创业种子基金资助项目（Z2012128） |
| |
| 摘 要： | 为解决MapReduce处理多个查询时效率低下的问题, 提出了一种基于查询共享的MapReduce查询优化方法——ShareOpt优化。通过分析所有查询的操作模式, 找出其中共享的子查询部分, 并根据子查询的执行顺序构造执行计划有向图（DAG）, 最终确定一组查询的整体执行计划。通过与Hive和Pig的对比, 验证了该方法能够在保证准确性的情况下有效地减少执行步数, 提高查询执行的效率。
|
| 关 键 词： | 查询优化 MapReduce 查询共享 执行计划 |
Sharing-based multi-query optimization approach using MapReduce |
| |
| ZHAO Bao-xue,LI Zhan-huai,CHEN Qun,PAN Wei,JIANG Tao,JIN Jian.Sharing-based multi-query optimization approach using MapReduce[J].Application Research of Computers,2013,30(5):1405-1409. |
| |
| Authors: | ZHAO Bao-xue LI Zhan-huai CHEN Qun PAN Wei JIANG Tao JIN Jian |
| |
| Affiliation: | School of Computer Science, Northwestern Polytechnical University, Xi'an 710072, China |
| |
| Abstract: | To improve the multi-query processing efficiency for MapReduce, this paper proposed a multi-query optimization approach based on sub-query sharing and merging. Firstly, it analyzed the patterns of all the queries and identified those sub-queries which had sharing opportunities and could be merged. Next it constructed a directed acyclic graph (DAG) according to the sub-queries execution sequences. It added the non-sharing sub-queries to the DAG graph and finally got the overall execution plan for the queries. Experiment results show that the approach can effectively eliminate unnecessary re-computation and save query execution time. |
| |
| Keywords: | query optimization MapReduce query sharing execution plan |
|
| 点击此处可从《计算机应用研究》浏览原始摘要信息 |
|
点击此处可从《计算机应用研究》下载全文 |
|
