基于柔性行动器–评判器深度强化学习的电–气综合能源系统优化调度

多能流协同优化调度是实现综合能源系统高效经济运行的核心技术之一。面向电–气综合能源系统运行优化问题,提出一种基于柔性行动器-评判器框架的深度强化学习方法,通过智能体与能源系统的交互,自适应学习控制策略。该方法可实现多能流系统的连续动作控制,且能够灵活处理风电、光伏、多能负荷等源荷不确定性问题,实现多场景下的电-气综合能源优化调度决策。首先,构建面向电-气综合能源系统调度的强化学习基本框架,介绍柔性行动器-评判器强化学习的基本原理;然后,构建与智能体交互的电-气综合能源系统环境模型,设计深度强化学习的动作与状态空间、奖励机制、神经网络结构、学习流程等关键环节;最后,针对2个电-气综合能源系统算例进行强化学习优化调度结果分析。

Optimal Dispatch of Integrated Electricity-gas System With Soft Actor-critic Deep Reinforcement Learning

Optimal dispatching of multi-energy flow is one of the core technologies to realize the efficient operation of integrated energy system. In this paper, a reinforcement learning based on the framework of soft actor-critic was proposed for optimizing the operation of integrated electricity-gas energy system. The agent adaptively learned the control strategies through its interaction with the power system. This method is able to take continuous control actions of the multi-energy flow system and flexibly deal with the complicated stochastic problem with uncertain wind power, photovoltaic power and loads. Thus the stochastic dispatching of integrated electricity-gas energy system can be implemented. First, the framework of the reinforcement learning for optimal dispatching was built and the methodology of the soft actor-critic was introduced. Then the interactive environment for the agent was built. The action and state space, reward approach, neural network structure and training process were designed. Finally, the results calculated by the proposed method were analyzed in two different integrated electricity-gas energy systems.