Storm环境下基于拓扑结构的任务调度策略

针对Storm流式计算平台中默认轮询调度策略存在通信开销大、负载不均衡的问题，提出基于拓扑结构的任务调度策略（TS²）。首先，选取CPU资源充足且可用的工作节点并各分配一个进程，消除节点内进程间通信开销，优化进程部署；然后，分析拓扑结构，找出拓扑中度最大的组件，优先分配该组件的线程；最后，在满足节点可承载最大线程数的条件下，尽可能将关联任务部署到同一个节点来减少节点间通信开销，改善集群负载均衡，优化线程部署。实验结果表明：在系统延迟方面，与Storm默认调度策略和离线调度策略相比，TS²的平均优化率分别为16.91%和5.69%，有效提高了系统的实时性；在节点间通信开销方面，TS²相比于Storm默认调度策略平均降低了15.75%；在平均吞吐量方面，TS²相比于Storm默认调度策略平均提升了14.21%。

Task scheduling strategy based on topology structure in Storm

In order to solve the problems of large communication cost and unbalanced load in the default round-robin scheduling strategy of Storm stream computing platform, a Task Scheduling Strategy based on Topology Structure (TS²) in Storm was proposed. Firstly, the work nodes with sufficient and available Central Processing Unit (CPU) resources were selected and only a process was allocated to each work node to eliminate the communication cost between processes within the nodes and optimize the process deployment. Then, the topology structure was analyzed, the component with the biggest degree in the topology was found and the thread of the component was assigned with the highest priority. Finally, under the condition of the maximum number of threads that a node could carry, the associated tasks were deployed to the same node as far as possible to reduce the communication cost between nodes, improve the load balance of cluster and optimize the thread deployment. The experimental results show that, in terms of system latency, the average optimization rate of TS² is 16.91% and 5.69% respectively compared with Storm default scheduling strategy and offline scheduling strategy, which effectively improves the real-time performance of system. Additionally, compared with the Storm default scheduling strategy, the communication cost between nodes of TS² is reduced by 15.75% and its average throughput is improved by 14.21%.