具有先验知识的 Q学习算法在AGC中的应用

传统的自动发电控制（AGC）系统通常基于经典的线性控制理论，并且大部分二次调频采用比例积分（PI）控制器，但系统固有的非线性以及结构多变使得积分增益系数不易确定，容易造成超调或调节不足的问题，从而影响系统频率稳定。文中采用强化学习控制器代替传统的PI调节器，将考虑了死区、出力约束、机组爬坡率和时延等非线性环节的AGC系统离散化成Markov链，直接将区域控制误差作为系统状态量，并充分利用AGC环境中的已有信息，结合模糊综合决策方法，获得能够改善 Q学习效率的先验知识，采用Q学习算法对其进行学习得出离散的AGC策略。数值仿真的结果验证在非线性AGC系统中应用具有先验知识的 Q学习方法可以加快收敛速度，提高学习效率，并通过控制性能评价标准（CPS）进一步检验了该方法的可行性。

Application of Q -learning Approach with Prior Knowledge to Non-linear AGC System

Conventional AGC systems are based on classical linear control theory.Most load frequency controls are primarily composed of an integral controller.Because of the inherent non-linearity and variable structure of the system,integrator gain is very difficult to set.This often results in generator overshoot or insufficient regulation.In this paper,the effect of non-linearity,such as dead-band of speed governor,transmission delay,generation rate control(GRC) and unit ramp control(URC) for AGC is considered comprehensively.AGC system is discretized as a Markov chain,and Q-learning algorithm as controller replaces PI controller.Area control error as state variable of the system is learned by Q-learning,one of reinforcement learning method.Furthermore,to improve study efficiency,the information of AGC environment using fuzzy integrated decision-making is translated into prior knowledge of Q-learning.Simulation result of this AGC system shows the improved Q-learning method has a higher learning efficiency and convergence speed.In order to test feasibility of this method,the results of simulation are calculated to test if it meets the CPS standard.The result is satisfactory in this regard.