基于多Agent与工作流的分布式ETL引擎的研究

针对传统ETL工具集中式执行方式的不足，提出了一种基于多Agent与工作流相结合的分布式ETL引擎的体系结构。该体系结构由一个主控引擎和多个执行引擎组成，执行引擎可自主地向主控引擎注册执行服务，并利用分布式计算和多线程并行计算技术，实现由多个执行引擎协同执行ETL工作流，从而提高整个系统的灵活性和吞吐率。实验结果表明，该引擎具有较好的可扩展性和负载平衡性能，并提高了执行效率。

Research of distributed ETL engine based on multi-agent and workflow

Aiming at the deficiency of centralized execution mode of traditional Extraction-Transformation-Loading (ETL) tools, this paper put forward the architecture of distributed ETL engine based on the combination of multi-agent and workflow technique, which consists of one main engine and many executive engines. The executive engines can register execution services to the main engine independently, and an ETL Workflow can be executed cooperatively on several executive engines by using distributed and concurrent computing technique, which improves the system's flexibility and throughput rate. Finally, results prove that it has better expansibility and can achieve load balancing quite well and raise the performance efficiency.