面向大数据流式计算的任务管理技术综述

流式计算是大数据的一种重要计算模式,大数据流式计算已成为研究热点。任务管理是大数据流式计算的核心功能之一,负责对流式计算的任务进行资源调度及全生命周期管理。目前对于大数据流式计算的技术调研工作主要集中于流式计算应用需求、体系结构及整体技术,缺乏对大数据流式计算任务管理技术的精细化调研分析。首先给出流式计算任务管理的抽象功能模型,其次基于该模型对任务管理的关键技术进行了分类和综述,最后对既有主流的大数据流式计算系统对上述关键技术的应用、集成和优化进行了调研分析。

A survey of task management techniques for big data stream computing

Stream computing is an important part of big data computing, which has become a hot topic in big data research. Task management is one of the essential features of stream computing, and is responsible for resource scheduling and lifecycle management of stream computing tasks. Current researches focus on application requirements, architecture and overall technology of stream computing, and they are lack of dedicated investigation and analysis of task management techniques. Firstly, we present a general abstract function model of task management for stream computing systems. Secondly, we classify and analyze the key techniques for task management based on this model. Finally, we investigate their applications in current stream processing systems, and the integration and optimization of above techniques.