基于多智能体Nash-Q强化学习的综合能源市场交易优化决策

目前求解综合能源市场多参与主体竞价博弈问题普遍采用数学推导法与启发式算法,但两类方法均须以完全信息环境为前提假设,同时前者忽略市场参与者非凸非线性属性,后者易陷入局部最优解.为此,引入多智能体Nash-Q强化学习算法,将市场参与主体构建成智能体,经由智能体在动态市场环境中反复探索与试错寻找博弈均衡点.首先,构建竞价决策...

Optimal Trading Decision-making for Integrated Energy Market Based on Multi-agent Nash-Q Reinforcement Learning

Currently, mathematical derivation methods and heuristic algorithms are commonly used to resolve the multi-participant bidding game issue in the integrated energy market. However, both methods require the assumption of a complete information environment. In addition, the former ignores the non-convex and nonlinear properties of market participants, and the latter is easy to be trapped in local optimal solutions. To this end, this paper introduces the multi-agent Nash-Q reinforced learning algorithm which structures market participants into agents and uses agents to find the equilibrium point of the game in a dynamic market environment through the repeated exploration and by trial and error. Firstly, a two-layer iterative trading framework for electricity-gas integrated energy market with bidding decision-market clearing is constructed. Secondly, the interest relationship model among market participants is constructed in the bidding decision-making layer through the game theory, and the multi-agent Nash-Q reinforced learning algorithm is used to optimize the bidding strategies of participants. Then, the game bidding strategy is jointly used in the market clearing layer to obtain the Nash equilibrium solution of the trade. Finally, the effectiveness and accuracy of the proposed method are verified by the example simulation.