不确定性环境下基于深度强化学习的综合能源系统动态调度

随着综合能源系统中间歇性能源和负荷不确定性的逐步增强，传统的调度方法局限于固定物理模型及参数设定，难以较好地动态响应源荷的随机波动。针对这一问题，提出了一种基于深度强化学习的综合能源系统动态调度方法。首先，以数据驱动方式构建面向综合能源系统的深度强化学习模型，通过智能体与综合能源系统的持续交互，自适应学习调度策略，降低对物理模型的依赖程度。其次，通过添加随机扰动的方式表征源荷不确定性变化特征，针对不确定性变化特征改进深度强化学习模型的状态空间、动作空间、奖励机制以及训练流程等关键环节，并经由近端策略优化算法优化求解，实现了综合能源系统的动态调度决策。最后，通过算例仿真验证了所提方法在不同时间尺度以及不确定性环境下的可行性和有效性。

Dynamic dispatch of an integrated energy system based on deep reinforcement learning in an uncertain environment

As the uncertainties of intermittent energy and load in the integrated energy system gradually increase, traditional dispatch methods are limited to fixed physical models and parameter settings that can hardly respond to the random fluctuations in the dynamic system with source-load. In this paper, a deep reinforcement learning-based dynamic dispatch method for the integrated energy system is proposed to address this problem. First, a data-driven deep reinforcement learning model is constructed for the integrated energy system. Through the continuous interaction between agent and integrated energy system, the dispatch strategies are learned adaptively to reduce dependence on the physical models. Secondly, the variations of source-load uncertainties are characterized by adding random disturbances. Pivotal aspects such as state spaces, action spaces, reward mechanisms and the training process of the deep reinforcement learning model are improved according to the characteristics of uncertainties. Then a proximal policy optimization algorithm is used to solve the problem, and the dynamic dispatch decisions of the integrated energy system are realized. Finally, simulation results verify the feasibility and effectiveness of the proposed method over different time scales and in uncertain environments.This work is supported by the National Natural Science Foundation of China (No. 51977005).