面向新型电力系统的电力大数据副本管理算法

随着新型电力系统建设步伐的加快,电网信息化程度不断加深,上万台虚拟机构成的庞大数据中心,仅仅依靠传统任务调度型副本管理策略,已无法满足未来精准负荷控制等新型电力业务对大数据处理时延的需求.对此,文章在充分考虑网络流量和数据中心位置分布的基础上,构建一种基于随机配置网络(SCN)的电力大数据自适应副本管理系统.同时,提出...

Research on Replica Management Strategy for Adaptive Storage of Electric Power Big Data

With the continuous acceleration of the construction of the power Internet of Things, the scale of power grid informatization continues to grow, and the power information system has developed from hundreds of servers to a huge data center with tens of thousands of virtual machines. Statistics and management are not only time-consuming and labor-intensive, but it is also difficult to find the root cause of the alarm information to trace the source of the fault. How to achieve efficient big data storage and meet the needs of low-latency processing applications is a very challenging problem. This paper proposes an adaptive power storage replica management system based on random configuration network (SCN), which takes into account the network traffic and the geographical distribution of data centers, and improves the real-time performance of data. First of all, the SCN model is used as a fast learning model with a small amount of calculation and good predictive performance to estimate the flow status of the power data network. Then, a series of data copy management algorithms are proposed to reduce the impact of limited bandwidth and fixed underlying infrastructure. Finally, the model is implemented using Data Parallel Computing Frameworks (DCFs) in the power industry. Pilot verification was carried out in the corresponding provincial company, and the program can effectively handle the large data storage of electric power, and the completion time of operations across distributed DCs was reduced by 12.19% on average.