基于深度强化学习的电网前瞻调度智能决策架构及关键技术初探

随着新型电力系统建设的快速推进,电网运行方式不确定性增加,调度对象类型/数量指数级增长,当前基于物理模型的电网调度计划存在优化决策计算速度慢、耗时长以及应对多重不确定场景适应性不够等问题,特别是日内阶段仍经常依赖调度员人工调控。为此,该文结合电网前瞻调度的时序滚动优化、多元对象决策、调度多目标构建等实际特点,提出基于深度强化学习的电网前瞻调度智能决策功能架构,分析离线训练模块、在线决策模块和效果评估模块3部分的具体实现;并在适用于电网前瞻调度的深度强化学习算法、学习样本效率提升、调度多目标奖励函数设计、拓扑改变情形下的迁移学习和前瞻调度效果评估等关键技术方面进行了初步探索,基于IEEE30节点算例验证了所提算法和技术的有效性。最后,探讨了电网前瞻调度智能决策需进一步研究的问题。     

基于深度强化学习算法的“电网脑”及其示范工程应用

可再生能源、电力电子设备渗透率持续增大以及大功率交直流混联,电网的动态性、随机性和不确定性显著增强,给电力系统安全稳定运行带来新的挑战.为更有效解决电网中出现的电压、潮流快速波动而导致的安全问题,提出一种基于最大熵深度强化学习算法的智能电网调控辅助决策方法,同时考虑多种控制目标,对电网运行方式进行在线优化控制.该方法将电网调度控制决策建模为马尔科夫决策过程,训练多线程智能体,并采用周期性在线训练机制对智能体的控制性能进行不断提升.基于该方法所研发的辅助决策原型软件部署在国网江苏电力调度控制中心,可与电网调度控制系统环境直接交互,自主学习且不断提升智能体调控决策能力.训练好的智能体可针对电压越限、联络线潮流越限、网损等综合控制目标在毫秒级时间内给出有效控制策略.     

一种基于深度强化学习的电网潮流特征提取方法

为了解决现有特征提取方法存在特征辨识度低的问题,基于深度强化学习设计电网潮流特征提取方法,为了提升潮流特征的辨识度,利用点估计法计算电网潮流,以此为基础,通过模拟退火算法生成电网潮流图,并灰度处理电网潮流图,以灰度处理后的电网潮流图为依据,利用深度强化学习方法提取电网潮流特征,实现了电网潮流特征的提取。实验结果表明：与现有的电网潮流特征提取方法相比,文中电网潮流特征提取方法极大地提升了特征辨识度,证明了基于深度强化学习的电网潮流特征提取方法具备更好的特征提取性能。     

基于深度强化学习的电动汽车充电调度算法研究进展

对电动汽车的充电过程进行优化调度有利于电网安全稳定运行,提升道路通行效率,提高可再生能源利用率,减少用户充电时间和充电费用。深度强化学习可以有效解决电动汽车充电优化调度面临的随机性和不确定性因素的影响。首先,概述了深度强化学习的工作原理,对比分析了不同种类强化学习的特点和应用场合。然后,从静态充电调度和动态充电调度两方面综述了基于深度强化学习的电动汽车充电调度算法研究成果,分析了现有研究的不足。最后,展望了该领域未来的研究方向。     

电网仿真分析与决策的人工智能方法

电网仿真是电网运行规划的支撑型技术,被广泛应用于电网各种运行方式的分析与决策。然而,目前基于电网仿真的分析与决策仍然需要人的参与,难以满足日益复杂的大电网需求,亟需寻找一条新的道路。人工智能的发展为电网仿真分析与决策提供了新的思路与方法。已有学者对人工智能在电网仿真分析与决策的应用做了广泛而深入的探索。该文旨在提供一个该领域最新进展的综合而清晰的蓝图。首先,介绍电网仿真分析与决策的概念和人工智能基本方法。然后,对人工智能在安全评估、稳定评估、潮流调整、安全控制、稳定控制、优化运行中的应用进行综述,系统地总结其思路、方法与优劣。尽管现有研究已经取得了不错的效果,但是人工智能应用过程中在知识建模、数据需求、特征提取以及迁移能力等方面还面临着不少挑战。该文在讨论这些挑战的同时,给出未来值得探索的研究方向。     

基于深度强化学习的暂态稳定紧急控制决策方法

随着广域测量系统在暂态稳定控制中的应用,广域信息的随机性时滞造成了系统受控时状态的不确定性,并且切机和切负荷控制的离散决策变量维度极高,电网在线紧急控制决策面临着挑战。为此,将暂态稳定紧急控制问题建模为马尔可夫决策问题,提出一种深度Q网络(DQN)强化学习与暂态能量函数相结合的紧急控制决策方法,多步序贯决策过程中可应对紧急控制的时滞不确定性影响。奖励函数以考虑控制目标和约束条件的短期奖励函数和考虑稳定性的长期奖励函数构成,并在奖励函数中引入暂态能量函数的势能指数来提高学习效率。以最大化累计奖励为目标,通过DQN算法在离散化动作空间中学习得到最优紧急控制策略,解决暂态稳定紧急控制问题。所提方法通过IEEE 39节点系统验证了模型在紧急控制决策中的有效性。     

基于深度强化学习的微电网在线优化

针对微电网的随机优化调度问题,提出了一种基于深度强化学习的微电网在线优化算法。利用深度神经网络近似状态-动作值函数,把蓄电池的动作离散化作为神经网络输出,然后利用非线性规划求解剩余决策变量并计算立即回报,通过Q学习算法,获取最优策略。为使得神经网络适应风光负荷的随机性,根据风电、光伏和负荷功率预测曲线及其预测误差,利用蒙特卡洛抽样生成多组训练曲线来训练神经网络;训练完成后,保存权重,根据微电网实时输入状态,神经网络能实时输出蓄电池的动作,实现微电网的在线优化调度。在风电、光伏和负荷功率发生波动的情况下与日前优化结果进行对比,验证了该算法相比于日前优化在微电网在线优化中的有效性和优越性。     

基于深度学习的电网安防视频分析技术

以变电站监控视频为出发点,利用帧间差分法发现视频序列中存在目标的帧,然后在分析电网监控场景的基础上,基于Faster R CNN深度学习框架训练目标检测模型来检测视频中的目标,实时记录目标检测结果并将结果存入数据库中。从试验结果可看出,深度学习具有较高的检测准确率,可加强智能电网的电力安防建设。     

基于深度学习的微电网优化调度辅助决策方法

随着可再生能源以及电力电子设备的高比例接入,微电网控制决策优化以及调度方式将面临极大的问题和挑战,南方电网公司正在向“能源价值链整合商”转型,将有可能运维数以万计的微电网,传统的模型驱动、预案式控制、人工值守的调度模式将难以满足需求。面向人工智能在微电网自动运行调控领域的需求,提出了基于深度学习的微电网优化调度辅助决策方法。首先介绍了微电网优化运行的典型数学规划模型,分析了模型驱动的建模求解方法的难点和局限性,接着提出了基于深度双向长短期记忆网络的微电网日前优化调度深度学习模型和方法,给出了模型输出结果的修正与处理原则,最后通过算例分析验证了本文模型和算法的有效性。     

基于风险的电力系统安全预警的预防性控制决策分析

针对事故发生的可能性和严重性进行电力系统安全分析的风险评估,将基于风险的安全性评估指标应用到电力系统安全预警中定量分析电力系统的运行状态转换风险,并进行电力系统预防性控制研究.根据系统低电压风险指标、系统过负荷风险指标和系统电压失稳风险指标3种风险指标综合表征的系统安全等级,在事故发生前进行预防性控制决策分析,以灵敏度确定参与决策的关键节点,用遗传算法确定参与决策的节点的最优有功和无功的注入量,从而有效地降低电网的运行风险,提高电网的安全预警等级.以上海电网84节点系统的仿真表明算法的可行性和有效性.     

强化学习理论在电力系统中的应用及展望

强化学习理论是人工智能领域中机器学习方法的一个重要分支,也是马尔可夫决策过程的一类重要方法.所谓强化学习就是智能系统从环境到行为映射的学习,以使奖励信号(强化信号)函数值最大.强化学习理论及其应用研究近年来日益受到国际机器学习和智能控制学术界的重视.系统地介绍了强化学习的基本思想和算法,综述了目前强化学习在安全稳定控制、自动发电控制、电压无功控制及电力市场等方面应用研究的主要成果与方法,并探讨了该课题在电力系统运行控制中的巨大潜力,以及与经典控制、神经网络、模糊理论和多Agent系统等智能控制技术的相互结合问题,最后对强化学习在电力科学领域的应用前景作出了展望.     

强化学习理论在电力系统中的应用及展望

强化学习理论是人工智能领域中机器学习方法的一个重要分支,也是马尔可夫决策过程的一类重要方法。所谓强化学习就是智能系统从环境到行为映射的学习,以使奖励信号（强化信号）函数值最大。强化学习理论及其应用研究近年来日益受到国际机器学习和智能控制学术界的重视。系统地介绍了强化学习的基本思想和算法,综述了目前强化学习在安全稳定控制、自动发电控制、电压无功控制及电力市场等方面应用研究的主要成果与方法,并探讨了该课题在电力系统运行控制中的巨大潜力,以及与经典控制、神经网络、模糊理论和多Agent系统等智能控制技术的相互结合问题,最后对强化学习在电力科学领域的应用前景作出了展望。     

Artificial intelligence for operation and control: The case of microgrids

Research on artificial intelligence (AI) has advanced significantly in recent years. A variety of AI algorithms have shown great promise in a large number of applications for power system operation and control. This article examines the potential of applying AI in microgrids (MGs). Specifically, as MGs commonly employ onsite generation including an increasing penetration of non-dispatchable distributed energy resources (DERs) and require seamless transition between operation modes (e.g., grid-connected and island) for different operation scenarios, the energy management within an MG is particularly complicated. Many factors including lack of inertia needed for system stability, generation uncertainty from DERs, and complex MG network topology composition (e.g., AC, DC, and hybrid AC/DC MGs) contribute to the difficulty of microgrid energy management. AI techniques such as deep learning (DL) and deep reinforcement learning (DRL) have recently demonstrated their excellence in tackling problems pertinent to decision making, providing a possible solution to overcome the above-mentioned challenges. This article discusses the applications of AI to MG operation and control, with an emphasis on DL and DRL. We survey the available DL and DRL technologies and their applications to power grids. We also investigate the unique issues associated with MGs including their layered control architecture, single vs. networked structure, and topology optimization. Perspectives on the ongoing challenges and viable AI solutions to MG operation and control are presented.     

Development of a WAMS based testplatform for power system real timetransient stability detection and control

The real-time transient stability detection and emergency control technology based on wide area response has become a hot research area in power system stability studies. Several different technologies have been proposed, but lots of problems remain to be solved before they can be applied in practice. A wide area measurement system (WAMS) based test platform is developed for transient stability detection and control. The design as well as main function modules of the platform are introduced. In addition, three generator power angle prediction methods and six response based transient instability detection technologies are given. Results of engineering application demonstrate that the developed test platform can provide a real-time operation environment, which can effectively compare and analyze the validity and practicability of these transient stability detection technologies. Based on the measured perturbed trajectories from actual power systems or the Real-Time Digital Simulators (RTDS), the platform can realize the assessment and visual result presentation of various responses from different transient instability detection technologies. The test platform can be applied to different power systems and it is convenient to embed new transient instability detection modules. Meanwhile some deficiencies and shortcomings in engineering application are pointed out and corresponding suggestions are given. In conclusion, the hardware and software structure, function modulus and engineering applications are presented. The application in actual power systems shows that it has a good application perspective.     

Hierarchical economic dispatch for multi-area power systems

In a multi-area power system, power exchange through tie lines such that the overall cost of the system operation is a minimum is a major economic dispatch problem. In this paper, techniques and methods are presented for solving the economic dispatch problem of radially interconnected power systems. The proposed method, based on a multi-area approach, uses an hierarchical control concept to improve the computation efficiency and accuracy; it has certain advantages over the conventional single-area approach. Theoretical formulations are derived and discussed from a simple power transfer concept. An efficient algorithm is organized. Numerical examples have been tested for a fictitious three-area system. The simulation results strongly support the proposed method with real-time application capability.     

Development of a WAMS based test platform for power system real time transient stability detection and control

The real-time transient stability detection and emergency control technology based on wide area response has become a hot research area in power system stability studies. Several different technologies have been proposed, but lots of problems remain to be solved before they can be applied in practice. A wide area measurement system (WAMS) based test platform is developed for transient stability detection and control. This platform can provide a real-time operation environment, which can effectively compare and analyze the validity and practicability of these transient stability detection technologies. According to the measured perturbed trajectories from the actual power system or the Real-Time Digital Simulators (RTDS), the platform can realize the assessment and visual result presentation of various responses from different transient instability detection technologies. The test platform can be applied to different power systems and it is convenient to embed new transient instability detection modules. The hardware and software structure, function modulus and engineering applications are presented. The application in actual power system shows that it has a good application perspective.     

基于深度Q值网络的自动小车控制方法

随着计算机技术和人工智能的飞速发展,无人驾驶车辆成为了一个新的热点。提出了一种自动小车的验证模型来模拟无人车,并验证了深度Q值网络（deep Q network, DQN）算法对自动小车的控制。该算法使用了强化学习和神经网络技术,能够在缺乏先验知识的情况下,根据获取的传感器信息训练神经网络,然后做出正确的决策,实现对车辆的控制,达到躲避障碍物的效果。此外,通过在模拟环境下的实验验证了DQN算法对自动小车的控制效果。实验结果表明,经过一定时间的训练,DQN算法可以有效的控制自动小车。     

全局安时无功优化调度的MAS方法

电力系统无功优化调度问题在数学上属于一种具有多目标、多不确定因素、多约束、多极值、非线性的组合最优化问题。通常的寻优方法遇到了许多困难，因此目前无功优化调度技术的应用还停留在计算规模较小的地区局部电网中。为实现电网全局实时的无功优化调度，提出一种基于分布式人工智能中多Agent系统(MAS)的无功优化调度模型。将全网无功调度按区域分解为若干个相互关联的调度子系统进行分布式求解，全局调度系统采用网络控制结构，各调度子系统则采用分层控制结构。在各子网局部实现无功优化的基础上，采用多Agent智能协调技术实现全网的全局无功优化。     

The application of artificial intelligence techniques for intelligent control of dynamical physical systems

There is an increasing concern among the scientific community and industrialists over the safe and reliable operation of control systems in industries. Although several adaptive control techniques have been introduced, they are not robust enough for many real-world problem domains where the degree of uncertainty is high and therefore classical methods of mathematical modelling and control fail. Computer technology has reached a point where machine intelligence can be incorporated in many of the systems that we use daily. Changes in environments, unmeasurable disturbances, changing reference models and performance criteria and component failures are some of the characteristics which necessitate intelligent control. the developments in the field of artificial intelligence have reached a stage which will help to reduce these control complexities by incorporating intelligence into the control systems. In this paper we explain various artificial intelligence techniques that can be used to control dynamical physical systems.