基于深度强化学习的城市配电网多级动态重构优化运行方法

随着配电网分布式电源的大量接入以及城市区域负荷的快速发展，使得配电网运行环境愈发复杂。同时由于配电网重构涉及大量的开关状态二进制零散变量，现有优化方法很难求解大规模城市配电网重构问题。基于此，提出一种基于深度强化学习的城市配电网多级动态重构方法。首先，建立基于深度学习的配电网多级重构快速判断模型，通过该模型实现对重构级别在线决策，并对智能体动作空间进行降维。其次，使用含参数冻结和经验回放机制的深度Q网络对预测负荷、光伏能源输出功率等环境信息进行学习。以运行成本、电压偏移度以及负荷均衡度最优为目标，通过习得的策略集对配电网进行动态重构与运行优化。建立多智能体强化学习模型，对各个时段的不同重构主体进行联合优化。最后，通过算例分析验证了所提方法的有效性。

Multi-level dynamic reconfiguration and operation optimization method for an urban distribution network based on deep reinforcement learning

With the large amount of distributed generation in the distribution network and the rapid development of urban regional loads, the operating environment of a distribution network has become increasingly complicated. At the same time, because the distribution network reconfiguration involves a large number of binary discrete variables of switch states, it is difficult for existing optimization methods to solve the large-scale urban distribution network reconfiguration problem. Thus a multi-level dynamic reconstruction method for an urban distribution network, one based on deep reinforcement learning, is proposed. First, a fast judgment model for multi-level reconstruction of the network based on deep learning is established, through which the online decision-making of the reconstruction level is realized, and the dimensionality of the action space of the agent is reduced. Second, a deep Q-network with parameter freezing and experience playback mechanisms is used to learn environmental information such as predicted load and photovoltaic energy output power. Then, with the objective of optimal operation cost, voltage offset and load balance degrees, the distribution network is dynamically reconfigured and operationally optimized via a learned strategy set. A multi-agent reinforcement learning model is established to jointly optimize different reconstruction subjects in each period. Finally, the effectiveness of the proposed method is verified by an example analysis. This work is supported by the National Natural Science Foundation of China (No. 52077146).