基于任务分类的延迟调度算法

MapReduce已经成为主流的海量数据处理模式，任务调度作为其关键环节已受到业界广泛关注。针对已有的延迟调度算法存在的问题，即建立在任务都是短任务的理论假设有一定限制，当节点处理不同长度的任务时算法性能严重下降和基于静态的等待时间阈值不能适应不同用户的作业需求，提出了一种基于任务分类的延迟调度算法。该算法通过给不同长度的任务设置不同的等待时间阈值，以适应不同作业的响应需求。通过分析各动态参数，根据所建任务模型调整任务的等待时间阈值。仿真验证该算法在响应时间及负载均衡性方面优于已有的延迟调度算法。

Delay Scheduling Algorithm Based on Task Classification

MapReduce has become a mainstream mass data processing mode, as its crucial part, the scheduler has received extensive concerns of the industry. There are two deficiencies in the current delay scheduling algorithms. Firstly, a limitation of these policies is that all the tasks to be processed should be short as assumed, the performance of the algorithms declined serious when servers handle the tasks of different lengths. Secondly, delay scheduling algorithms based on static waiting time threshold, cannot adapt to the different user needs. To address this issue, this paper proposed a delay scheduling algorithm based on task classification. It adjusted tasks waiting time threshold dynamically according to the information of the different lengths. It shows that this algorithm outperforms previous delay scheduling algorithms in term of the job response time and load balance of the node.