电网量测数据海量终端的前置中间件技术研究

针对目前智能电网状态监测与分析中面临的WAMS、SCADA、AMI等量测系统海量、多源、高速数据处理问题,提出一种海量终端的数据前置处理中间件技术,着重解决海量数据中目标信息高效挖掘与处理器负载均衡问题。在前置数据处理中间件架构中设计了基于采样的目标信息数据并行挖掘算法,同时通过基于Map-Reduce并行计算模型及轮转算法思想均衡负载,以采样挖掘方式聚合数据内联关系,设计出单机多核并行数据挖掘策略。通过广域电网中海量PMU数据进行对比测试,结果表明文中提出的中间件技术可以有效的提高挖掘速度和多处理器负载均衡度,同时极大地减轻海量数据挖掘中的内存负担。

Research on Pre-middleware Technology for Massive Terminal of Power Grid Measurement Data

Aiming at the massive, multi-source and high-speed data processing problems of WAMS, SCADA, AMI and other measurement systems currently facing monitoring and analysis of smart grid, this paper proposes a data pre-processing middleware technology for mass terminals, which focuses on the efficient mining of target information and processor load balancing in massive data. In the pre-data processing middleware architecture, there designs a parallel mining algorithm based on sampling target information data, and the load is balanced by the idea of parallel computing based on Map-Reduce and the idea of rotation algorithm. With aggregating data inline relationships by sampling mining, this paper designs a single-machine multi-core parallel data mining strategy. Finally, through the comparative test of massive PMU data in wide-area power grid, the results show that the middleware technology can effectively improve the mining speed and multi-processor load balance, and greatly reduce the memory burden in massive data mining.