A multi-agent deep reinforcement learning based energy management for behind-the-meter resources

The future communities are becoming more and more electrically connected via increased penetrations of behind-the-meter (BTM) resources, specifically, electric vehicles (EVs), smart buildings (SBs), and distributed renewables. The electricity infrastructure is thus seeing increased challenges in its reliable, secure, and economic operation and control with increased and hard to predict demands (due to EV charging and demand management of SBs), fluctuating generation from renewables, as well as their plug-N-play dynamics. Reinforcement learning has been extensively used to enable network entities to obtain optimal policies. The recent development of deep learning has enabled deep reinforcement learning (DRL) to drive optimal policies for sophisticated and capable agents, which can outperform conventional rule-based operation policies in applications such as games, natural language processing, and biology. Furthermore, DRL has shown promising results in many resource management tasks. Numerous studies have been conducted on the application of single-agent DRL to energy management. In this paper, a fully distributed energy management framework based on multi-agent deep reinforcement learning (MADRL) is proposed to optimize the BTM resource operations and improve essential service delivery to community residents.     

基于深度强化学习的电网自主控制与决策技术

高比例可再生能源和电力电子设备渗透率的不断增加给电力系统运行与调控带来诸多挑战。本文基于深度强化学习技术（深度确定策略梯度, DDPG）提出了具有在线学习功能的电网自主优化控制与决策框架，即“电网脑”系统；通过不断的学习和经验累积，AI智能体可以在亚秒级时间内根据实时量测数据给出调控指令及预期效果。该系统近期可用于辅助调度员决策，远期可为自动调度提供技术手段。本文以电网电压和联络线潮流控制为例，从多方面详细介绍了自主调控的方法，包括问题描述、控制目标和样本设定、奖惩机制定义、状态空间和控制动作集定义、算法实现流程等。大量的数值仿真实验验证了所提方法强大的学习能力以及应用于电力系统自主控制与决策的可行性。     

Comparative framework for AC-microgrid protection schemes: challenges, solutions, real applications, and future trends

With the rapid development of electrical power systems in recent years, microgrids (MGs) have become increasingly prevalent. MGs improve network efficiency and reduce operating costs and emissions because of the integration of distributed renewable energy sources (RESs), energy storage, and source-load management systems. Despite these advances, the decentralized architecture of MGs impacts the functioning patterns of the entire system, including control strategy, energy management philosophy, and protection scheme. In this context, developing a convenient protection strategy for MGs is challenging because of various obstacles, such as the significant variance in short-circuit values under different operating modes, two-way power flow, asynchronous reclosing, protection blinding, sympathetic tripping, and loss of coordination. In light of these challenges, this paper reviews prior research on proposed protection schemes for AC-MGs to thoroughly evaluate network protection''s potential issues. The paper also provides a comprehensive overview of the MG structure and the associated protection challenges, solutions, real applications, and future trends.     

Jointly improving energy efficiency and smoothing power oscillations of integrated offshore wind and photovoltaic power: a deep reinforcement learning approach

This paper proposes a novel deep reinforcement learning (DRL) control strategy for an integrated ofshore wind and photovoltaic (PV) power system for improving power generation efciency while simultaneously damping oscillations. A variable-speed ofshore wind turbine (OWT) with electrical torque control is used in the integrated ofshore power system whose dynamic models are detailed. By considering the control system as a partially-observable Markov decision process, an actor-critic architecture model-free DRL algorithm, namely, deep deterministic policy gradient, is adopted and implemented to explore and learn the optimal multi-objective control policy. The potential and efectiveness of the integrated power system are evaluated. The results imply that an OWT can respond quickly to sudden changes of the infow wind conditions to maximize total power generation. Signifcant oscillations in the overall power output can also be well suppressed by regulating the generator torque, which further indicates that complementary operation of ofshore wind and PV power can be achieved.     

考虑预测不确定性的微电网实时控制策略研究

风能、光伏等可再生能源的高比例并网成为了缓解全球能源危机的一项重要措施。然而,可再生能源实时出力中的间歇性和波动性给系统的安全性带来了一定的挑战。为了在提高可再生能源利用率的同时保证系统安全性,提出了一种基于深度强化学习(DRL)算法的运行优化实时调度模型。首先,构建了负荷预测模型实现负荷预测和高斯混合模型拟合预测误差;其次,考虑系统各节点的约束条件,以系统运行成本和安全运行作为优化目标,建立相应优化模型;然后,将优化问题转化为马尔可夫决策过程,并采用双延迟深度确定性策略梯度算法求解;最后,利用DRL算法的环境交互机制和策略自由探索,获得联合调度策略的最优结果。实验结果表明,所提方法具有良好的适应性,并且可以进行在线实时调度。     

含分布式能源直流配电网的优化调度

直流配电网对促进分布式可再生能源消纳、解决传统交流配电网发展瓶颈等方面具有巨大优势。基于直流配电网现有讨论和研究,针对含多种分布式能源（distributed energy resource,DER）的直流配电网,讨论其组网方式与系统结构,重点研究适用于接有多种分布式能源的直流配电网的优化调度方法,设计各并网单元运行策略及直流配电网多时段优化调度策略,并建立了综合考虑运行成本、环境效益以及系统损耗的多目标优化调度模型。通过算例的计算和分析,表明所提优化调度方法在促进接入多种分布式能源后直流配电网的优化运行以及接入分布式能源前后配电网优化目标的改善方面具有良好作用,验证了该方法的有效性。     

分布式直流微电网分级控制技术综述

近年来,作为保证直流微电网稳定高效运行的一种有效技术方法,分级控制在国内外受到广泛关注。分级控制的实现通常建立在下垂控制之上,因此该文对基于下垂控制的直流微电网分级控制技术进行综述研究。首先,通过对已有下垂控制方法进行分析比较,系统地评述传统下垂控制方法的局限性。其次,针对传统下垂控制的缺陷,主要介绍了改进的主级控制、次级控制和第三级控制的分级控制方法,同时根据通信方式的不同,将次级控制分为三种协调控制方法。此外,该文着重讨论基于一致性算法的分布式次级控制在直流微电网应用中的优越性和协调控制所面临的通信问题及其解决方法。最后,对直流微电网下垂分级控制的现有研究技术进行对比总结,指出其目前存在的问题和未来发展趋势。     

一种中压直流短路故障下不间断运行的MMC-SST拓扑及控制

多端口固态变压器是多电压形态多电压等级的交直流混合电网的核心设备,模块化多电平(modular multilevel converter,MMC)型固态变压器(solid state transformer,SST)具有中压直流端口,可接入中压直流配电网,构成多区域交流配电网的柔性互联,提升区域网络间功率灵活调节能力。而采用传统的MMC-SST拓扑及控制,中压直流线路短路故障会引起低压端口供电中断。文中提出一种混合型MMC-SST的拓扑及控制,其具备中压交流、中压直流和低压交直流端口,通过控制使其具有中压直流短路故障耐受能力,同时故障期间保持中压交流和低压端口的不间断功率交互,从而提升低压用户供电可靠性。分析MMC-SST在正常运行和中压直流故障不间断运行控制下内部能量平衡机理,提出中压直流短路故障下电容电压平衡及不间断运行控制策略,实现MMC-SST中压直流短路故障时不间断稳定运行。通过理论分析,仿真与物理动模实验,验证了所提拓扑及控制的可行性及有效性。     

一种VVVF电源的模糊PID控制

变压变频电源由于开关的非线性和对电路参数的依赖性,即使采用多种优化方式,也仍然难以提高其动态性能,因此限制了其在更宽负载范围内的运用.在此,依据模糊控制理论,用一个模糊PID开关切换控制,对现有的控制系统进行了改进.仿真结果表明,该方法在没有提高电路复杂度和主开关工作频率的情况下,提高了系统动态性能,拓宽了负载工作范围.     

Reliability evaluation of multi-microgrids considering optimal operation of small scale energy zones under load-generation uncertainties

A Microgrid (MG) is a scaled down version of the centralized power system that generates, distributes, and regulates the flow of electricity and efficiently serves the local demand. It can operate either grid connected or islanded and, if required, can switch between two modes. In this paper the optimal operation of interconnected MGs considering market operation and network reliability based on MGs is discussed. In supposed structure of MGs, each MG is considered as small scale energy zone (SSEZ) and energy sources within MGs play a role of small scale energy resources (SSER). Because of unpredictable behavior of some SSERs and consumption power of MGs, the probabilistic analysis is impossible to ignore. So, the reliability evaluation is done under uncertain behavior of MGs components. Some recently introduced reliability indices in the literature for MGs are used to evaluate the MGs reliability. The reliability in the proposed paper is evaluated in two different structures of MGs. In first structure, the MGs are interconnected and in second structure, all MGs in islanded mode. The imperialist competitive algorithm (ICA) is applied to optimal power dispatch problem and the obtained results are compared by Monte Carlo Simulation (MCS) method.     

Three-phase two-leg buck-boost DC-AC inverter with differential power processor unit

Renewable energy sources (RESs) need power-electronics-based converters to deliver the acquired power to the grid. Those converters should provide voltage-bucking/boosting capabilities to accommodate various grid modes specially for three-phase distorted commonly unbalanced distribution utility networks. Several power electronic-based converters have been elaborated to fulfill this high-demand market. Single-stage converters are the most dominant in the market where the current source inverters (CSIs), impedance source inverters (ZSIs), and boost inverters are the high-end candidates. The aspects of cost, footprint, and minimal numbers of active switches, in addition to simplified controllability, build the main challenges that face the evolution of robust renewable energy-associated grid-tied converters. This paper presents a novel three-phase differential-mode buck-boost inverter based on two bidirectional buck-boost DC/DC converters and one differential power processor (DPP) unit. The proposed topology is a single-stage DC/AC converter offering bucking/boosting capability, with reduced hardware requirements. The proposed topology features a simplified control methodology in addition to reduced size and cost of the hardware setup which makes it more suitable for grid-tied renewable energy applications. The operation principles, small-signal model, and control strategy of the proposed topology are also illustrated. Simulation and experimental results are presented in details to verify the topology-enhanced performance under various operating conditions. The deducted results elucidate the viability of the proposed configuration alongside with the claimed merits.     

Research on power electronic transformer applied in AC/DC hybrid distribution networks

The AC/DC hybrid distribution network is one of the trends in distribution network development, which poses great challenges to the traditional distribution transformer. In this paper, a new topology suitable for AC/DC hybrid distribution network is put forward according to the demands of power grid, with advantages of accepting DG and DC loads, while clearing DC fault by blocking the clamping double sub-module(CDSM) of input stage. Then, this paper shows the typical structure of AC/DC distribution network that is hand in hand. Based on the new topology, this paper designs the control and modulation strategies of each stage, where the outer loop controller of input stage is emphasized for its twocontrol mode. At last, the rationality of new topology and the validity of control strategies are verified by the steady and dynamic state simulation. At the same time, the simulation results highlight the role of PET in energy regulation.     

一种基于CSC-VSC的DC/DC换流器及其控制策略

在能源互联网建设中,DC/DC换流器是实现不同直流电压等级的可再生能源并网的必要环节,引起了广泛的研究。DC/DC换流器的体积、成本、动稳态性能和故障性能是其广泛应用的限制因素。为此,提出了一种并网端采用有源电流源换流器、分布式能源端采用电压源换流器的front-to-front混合DC/DC换流器,其中CSC和VSC交流侧采用高频率正弦波调制。所提出的基于CSC-VSC的混合DC/DC换流器具有体积小、成本低、可穿越直流短路故障等优点。通过对混合DC/DC换流器数学模型的分析,提出了换流器参数设计方法,给出了换流器运行范围约束。再提出了一种适用于混合DC/DC换流器的综合控制策略,其中CSC采用有功无功闭环控制、VSC采用交流电压幅值闭环控制。利用RTLAB半实物仿真平台分别对混合DC/DC换流器额定运行工况和直流短路工况进行了实验研究,结果验证了所提混合DC/DC换流器拓扑和控制策略的合理性和有效性。     

基于虚拟同步发电机控制技术的能量路由器交直流协调控制

作为能源互联网的关键设备,能量路由器的电路拓扑和控制策略与可再生能源接纳、灵活电力变换息息相关。基于虚拟电机理论,提出一种能量路由器拓扑结构,对此结构的交、直流端口均采用虚拟同步发电机控制,以实现对交、直流接口能量的协调控制。分别建立了虚拟直流电机和虚拟交流电机控制数学模型,以实现交、直流侧协调控制;并针对直流侧突增负荷、电网频率突增和突减三种工况,验证了所提出的能量路由器在拓扑结构、能量流动、维持母线电压稳定的正确性。并且在MATLAB中与传统的下垂控制进行了比较研究,对比研究结果显示下垂控制具有调节速度较快的优点,但超调大,变化较为突兀的缺点;而虚拟电机控制方法则由于具备阻尼和惯性特性的优越性,所以超调小,调节则较为平缓有利于系统的稳定运行。所提出的控制策略为能量路由器的稳定运行提出了一种新的解决方案。     

一种适用于远海风电直流汇集送出换流阀的拓扑及其技术经济性分析

高压直流送出技术是大规模、远距离海上风电开发的重要手段,送出换流阀是其核心装备。基于模块化多电平换流器(MMC)的交流汇集送出换流阀存在成本高、体积和重量大等问题,海上平台工程建设成本高,而基于直流变压器(DCT)的直流汇集送出换流阀有望解决这一问题。首先,通过分析直流汇集送出换流阀的技术特征和可行的拓扑结构,明确了采用容性能量转移型直流变压器(CET-DCT)构成直流汇集送出换流阀,具有成本低、体积和重量小、效率高等特点。然后,进一步将CET-DCT拓扑改进为适用于工程应用的双极拓扑,介绍了所提拓扑的基本工作原理,并搭建了仿真模型和实验平台,仿真结果和实验结果充分验证了拓扑的可行性。最后,与基于MMC的交流汇集送出换流阀拓扑综合对比表明,基于CET-DCT的直流汇集送出换流阀技术经济性更好。     

基于动态随机模型的微电网群能量管理方法

微电网可以协调调度分布式电源、储能装置和负荷,最大限度地利用可再生能源。文中以互联微电网群为研究对象,研究微电网间能量交易方法。针对微电网的动态随机因素,采用多时间尺度方法进行处理,包括日前调度和日内优化,提出微电网能量管理框架。然后建立微电网能量管理模型,包括日前调度模型和日内优化模型,确定其目标函数和约束条件,利用粒子群算法求解实现日经济成本最低,提高微电网群的经济性。选取3个微电网构成的微电网群系统进行算例分析,对所提动态随机模型、能量交易方法和能量管理模型进行验证和分析,仿真结果表明,文中所提微电网群能量交易方法能够有效提高微电网群的运行经济性。     

高比例新能源交直流混合配电网优化运行与安全分析研究综述

高比例间歇式清洁能源的接入实现了能源利用率的提高和碳排放的减少,但其出力的不确定性为交直流混合配电网的安全运行带来了挑战.同时,高效调度泛在灵活性资源可以促进风光资源的消纳,实现交直流混合配电网的安全经济运行.鉴于此,对高比例新能源交直流混合配电网优化运行与安全分析研究进行了综述.首先介绍了交直流混合配电网潮流模型及运行约束,包括非线性模型、线性化模型和凸松弛模型;其次阐述了交直流混合配电网优化调度研究,包括随机优化方法、两阶段/多阶段随机优化模型和灵活性运行;接着对交直流混合配电网进行了N-1安全分析和可靠性评估,构建了安全域模型;最后展望了未来交直流混合配电网安全分析与优化调度可研究的方向.     

Controlling the renewable microgrid using semidefinite programming technique

Given the significant concerns regarding carbon emissions from fossil fuels, global warming and energy crisis, renewable distributed energy resources (DERs) are going to be integrated in smart grids, which will make the energy supply more reliable and decrease the costs and transmission losses. Unfortunately, one of the key technical challenges in power system planning, control and operation with DERs is the voltage regulation at the distribution level. This problem stimulates the deployment of smart sensors and actuators in smart grids so that the voltage can be stabilized. The observation from the microgrid incorporating DERs is transmitted to the control center via wireless communication systems. In other words, the proposed communication infrastructure provides an opportunity to address the voltage regulation challenge by offering the two-way communication links for microgrid state information collection, estimation and stabilization. Based on the communication infrastructure, we propose a least square based Kalman filter algorithm for state estimation and an optimal feedback control framework for stabilizing the microgrid states. Specifically, we propose to optimize the performance index by using semidefinite programming techniques in the context of smart grid applications. At the end, the efficacy of the developed approaches is demonstrated using a microgrid incorporating multiple DERs.     

基于储能稳压的交直流混合电能路由器协调控制策略

将交/直/交级联变换器、直流变换器、储能及其变换器通过公共直流母线组合,构成含两个交流端口、三个直流端口的电能路由器拓扑结构。分析典型运行模式并提出储能稳压的交直流混合电能路由器虚拟同步机协调控制策略:在交流单/双端并网模式下,通过储能稳定直流电压,两端交/直、直/交变换器通过虚拟同步机功率外环控制功率流向及大小;在交流双端离网模式下,通过储能稳定直流电压的同时,配合分布式电源为交直流负荷供电。所提策略无需模式切换,降低了控制复杂性,可实现电能路由器各模式下直流电压稳定、就地消纳分布式发电,保证交直流负荷持续稳定供电,还可实现双端并网时电网馈线间的柔性互联、电网故障时潮流转供以及双端离网下的自稳定运行,有效提高了低压配电网的供电可靠性。最后,通过仿真和实验验证了所提协调控制策略的正确性和有效性。