融合无监督和有监督学习的虚假数据注入攻击检测

虚假数据注入攻击（False Data Injection Attack,FDIA）是智能电网安全与稳定运行面临的严重威胁。本文针对FDIA检测中存在有标签数据稀少、正常和攻击样本极不平衡的问题,提出了融合无监督和有监督学习的虚假数据注入攻击检测算法。首先引入对比学习捕获少量攻击数据特征,生成新的攻击样本实现数据扩充,然后利用多种无监督检测算法对海量的无标签样本进行特征自学习,解决有标签样本稀缺的问题,最后将无监督算法提取的特征与历史特征集进行融合,在新的特征空间上构建有监督XGBoost分类器进行识别,输出正常或异常的检测结果。在IEEE 30 节点系统上的算例分析表明,与其它FDIA检测的算法相比,本文方法增强了 FDIA 检测模型在有标签样本稀少和不平衡数据下的稳定性,提升了FDIA的识别精度并降低误报率。     

基于Tri-training-SSAE半监督学习算法的电力系统暂态稳定评估

基于机器学习的暂态稳定评估方法主要采用监督学习方法,为了解决监督学习方法所需的有标签样本难以获取的问题,提出基于三体训练-稀疏堆叠自动编码器(Tri-training-SSAE)半监督学习算法的电力系统暂态稳定评估方法。构建基于堆叠稀疏自动编码器的暂态稳定评估模型;在传统的三体训练过程中加入伪标签样本置信度判断,以减小噪声数据对模型训练的影响;以堆叠稀疏自动编码器为基分类器构建三体训练-稀疏堆叠自动编码器模型,利用大量的无标签样本提高模型的泛化能力。通过IEEE 39节点系统与华东某省级电网进行分析验证,结果表明,所提方法在有标签样本数较少时具有更高的评估准确度。     

基于深度强化学习的两阶段显著性目标检测

为提高复杂场景下的显著性目标检测速度和精度,提出了一种基于深度强化学习的两阶段显著性目标检测方法.该算法由显著性区域定位网络(salient region localization network,SRLN)和显著性目标分割网络(salient object segmentation network,SOSN)组成,分...     

Adaptive predictive control of a differential drive robot tuned with reinforcement learning

One of the most important steps in designing a model predictive control strategy is selecting appropriate parameters for the relative weights of the objective function. Typically, these are selected through trial and error to meet the desired performance. In this paper, a reinforcement learning technique called learning automata is used to select appropriate parameters for the controller of a differential drive robot through a simulation process. Results of the simulation show that the parameters always converge, although to different values. A controller chosen by the learning process is then ported to a real platform. The selected controller is shown to control the robot better than a standard model predictive control.     

基于自然语言处理的弱监督知识获取系统的实现

知识获取多年来一直被认为是阻碍智能系统开发的瓶颈问题,尤其是互联网时代,大量的信息都以非结构化的文本形式存在。本文运用分布式计算思想设计了一个基于互联网大规模语料库的知识自动获取系统。采用弱监督条件下机器学习的方法对信息自动挖掘和获取,实现机器对知识的自动学习和挖掘、新词词典发现、实体关系模板提取、命名实体识别等功能。利用该系统分别对未登录新词发现和地名识别两种应用进行了实验,运用N gram和互信息(PMI)方法分别取得了72.1%和87.28%的准确率。     

基于监督学习的非侵入式负荷监测算法比较

非侵入式负荷监测(NILM)能够在不干扰用户正常用电的情况下,低成本地实现用户用电设备类型的识别和用电负荷的分解,因此非常适用于家庭用户用电监测。大量智能电表在家庭用户中的安装为居民NILM提供了数据支撑,也使得居民NILM研究成为热点。基于家庭负荷稳态电流样本,采用负荷电流谐波系数作为负荷分类特征,建立了基于多层感知器(MLP)神经网络、k-近邻算法、逻辑回归、支持向量机的4种NILM分类模型,利用BLUED数据库对4种分类器进行训练和测试,对比分析其在识别精度、训练时间、识别速度和抗噪能力方面的表现,并对其在家庭负荷识别中的应用效果进行对比研究。结果表明,4种分类器中MLP神经网络具有总体最优的分类效果和计算性能,更适用于家庭用户负荷监测。     

Evolutionary reinforcement learning system with time‐varying parameters

In this paper, an evolutionary reinforcement learning system with time‐varying parameters that can learn appropriate policy in dynamical POMDPs is proposed. The proposed system has time‐varying parameters that can be adjusted by using reinforcement learning. Hence, the system can adapt to the time variation of the dynamical environment even if its variation cannot be observed. In addition, the state space of the environment is divided evolutionarily. Thus, one need not divide the state space in advance. The efficacy of the proposed system is shown by mobile robot control simulation under the environment belonging to dynamical POMDPs. The environment is the passage that has gates iterate opening and closing. © 2006 Wiley Periodicals, Inc. Electr Eng Jpn, 156(1): 54–60, 2006; Published online in Wiley InterScience ( www.interscience.wiley.com ). DOI 10.1002/eej.20170     

Feature extraction by supervised independent component analysis based on category information

Independent component analysis (ICA) is a technique of transforming observation signals into their unknown independent components; hence, ICA has often been applied to blind signal separation problems. In this application, it is expected that the obtained independent components extract essential information of independent signal sources from input data in an unsupervised fashion. Based on such characteristics, ICA is currently utilized as a feature extraction method for images and sounds for recognition purposes. However, since ICA is an unsupervised learning, the obtained independent components are not always useful in recognition. To overcome this problem, we propose a supervised approach to ICA using category information. The proposed method is implemented in a conventional three‐layered neural network, but its objective function to be minimized is defined not only for the output layer but also for the hidden layer. The objective function consists of the following two terms: one evaluates the kurtosis of hidden unit outputs and the other evaluates the error between output signals and their teacher signals. The experiments are performed using several standard datasets to evaluate performance of the proposed algorithm. It is confirmed that a higher recognition accuracy is attained by the proposed method as compared with a conventional ICA algorithm. © 2007 Wiley Periodicals, Inc. Electr Eng Jpn, 161(2): 25–32, 2007; Published online in Wiley InterScience ( www.interscience.wiley.com ). DOI 10.1002/eej.20522     

基于二次聚类弱监督学习的图像语义分割

针对在给定大量用户标注图像基础上,将图像分割成一系列具有单一语义的完整区域,并同时对各区域实现语义标注的问题,提出了一种基于弱监督学习的二次聚类的图像语义分割算法。将谱聚类和判别式聚类相结合,用谱聚类学到的类标指示函数来指导判别式聚类,学习特征的潜在数据结构,利用弱监督信息对聚类分配标签。该方法能够充分利用区域上下文信息,为每个类别选择判别式特征,并且输出鲁棒的多类分类器,对外来没有标签的图像也可以进行有效的区域标注。通过在公用数据集上的充分实验,证明了本方法的有效性。     

融合图神经网络模型与强化学习的综合能源系统优化调度

随着人工智能技术特别是强化学习在能源优化调度领域的深入研究,将系统状态表示为向量用于学习的模式,其训练效率与信息利用率较低。针对这一问题,提出了一种融合图神经网络模型与强化学习的综合能源系统优化调度方法。首先,将电-热-气综合能源系统建模为图结构数据,充分利用系统的拓扑信息。其次,提出了基于图神经网络架构的强化学习模型,使其可以充分利用图结构信息实现更快的训练速度,获得更大的探索空间。最后,将表示系统状态的图结构信息送入该模型进行训练,算例仿真验证了该方法的训练效率与探索能力。     

基于深度强化学习的微电网优化运行策略

风电、光伏、负荷的不确定性给含有高比例可再生能源的微电网制定运行策略带来了挑战,人工智能技术的发展为解决微电网运行优化问题提供了新思路。基于强化学习框架,将微电网运行问题转化为马尔可夫决策过程,以最大化微电网经济利益和居民满意度为目标,提出一种基于深度强化学习的微电网在线调度方法。为了在深度强化学习训练的过程中高效利用经验,设计一种优先经验存储的深度确定性策略梯度(PES-DDPG)算法,学习各类环境下不同时段的微电网最优调度策略。算例结果表明,PES-DDPG算法能够为微电网提供有效的调度策略,并实现微电网的实时优化。     

基于actor-critic框架的在线积分强化学习算法研究

针对轮式移动机器人动力学系统难以实现无模型的最优跟踪控制问题,提出了一种基于actor-critic框架的在线积分强化学习控制算法。首先,构建RBF评价神经网络并基于近似贝尔曼误差设计该网络的权值更新律,以拟合二次型跟踪控制性能指标函数。其次,构建RBF行为神经网络并以最小化性能指标函数为目标设计权值更新律,补偿动力学系统中的未知项。最后,通过Lyapunov理论证明了所提出的积分强化学习控制算法可以使得价值函数,行为神经网络权值误差与评价神经网络权值误差一致最终有界。仿真和实验结果表明,该算法不仅可以实现对恒定速度以及时变速度的跟踪,还可以在嵌入式平台上进行实现。     

基于无监督学习的MMC子模块开路故障诊断方法

模块化多电平换流器(MMC)子模块发生故障时,快速准确地检测并定位故障是提高换流器运行可靠性的关键.目前,机器学习在MMC故障诊断领域得到了一定的应用,但大多需要采集每种故障情况下的样本,而故障样本采集难度较大.针对此问题,提出一种无需采集故障样本,仅根据正常样本训练分类模型即可实现故障检测和定位的无监督故障诊断方法....     

A multi-agent deep reinforcement learning based energy management for behind-the-meter resources

The future communities are becoming more and more electrically connected via increased penetrations of behind-the-meter (BTM) resources, specifically, electric vehicles (EVs), smart buildings (SBs), and distributed renewables. The electricity infrastructure is thus seeing increased challenges in its reliable, secure, and economic operation and control with increased and hard to predict demands (due to EV charging and demand management of SBs), fluctuating generation from renewables, as well as their plug-N-play dynamics. Reinforcement learning has been extensively used to enable network entities to obtain optimal policies. The recent development of deep learning has enabled deep reinforcement learning (DRL) to drive optimal policies for sophisticated and capable agents, which can outperform conventional rule-based operation policies in applications such as games, natural language processing, and biology. Furthermore, DRL has shown promising results in many resource management tasks. Numerous studies have been conducted on the application of single-agent DRL to energy management. In this paper, a fully distributed energy management framework based on multi-agent deep reinforcement learning (MADRL) is proposed to optimize the BTM resource operations and improve essential service delivery to community residents.     

Assured learning-enabled autonomy: A metacognitive reinforcement learning framework

Reinforcement learning (RL) agents with pre-specified reward functions cannot provide guaranteed safety across variety of circumstances that an uncertain system might encounter. To guarantee performance while assuring satisfaction of safety constraints across variety of circumstances, an assured autonomous control framework is presented in this article by empowering RL algorithms with metacognitive learning capabilities. More specifically, adapting the reward function parameters of the RL agent is performed in a metacognitive decision-making layer to assure the feasibility of RL agent. That is, to assure that the learned policy by the RL agent satisfies safety constraints specified by signal temporal logic while achieving as much performance as possible. The metacognitive layer monitors any possible future safety violation under the actions of the RL agent and employs a higher-layer Bayesian RL algorithm to proactively adapt the reward function for the lower-layer RL agent. To minimize the higher-layer Bayesian RL intervention, a fitness function is leveraged by the metacognitive layer as a metric to evaluate success of the lower-layer RL agent in satisfaction of safety and liveness specifications, and the higher-layer Bayesian RL intervenes only if there is a risk of lower-layer RL failure. Finally, a simulation example is provided to validate the effectiveness of the proposed approach.     

A neural-net based short term load forecasting using moving window procedure

An improved neural-net approach based on a combined unsupervised/supervised learning concept is proposed. A ‘moving window’ procedure is applied to the most recent load and weather information for creating training set data base. A forecasting lead time that varies from 16 hours to 88 hours is introduced to produce the short term electric load forecasting that meets requirements of real electric utility operating practice. The unsupervised learning (UL) is used to identify days with similar daily load patterns. A feed forward three-layer neural net is designed to predict 24-hour loads within the supervised learning (SL) phase. The effectiveness of proposed methods is demonstrated by comparison of forecasted hourly loads in every single day during 1991 with data realized in the same period in the Electric Power Utility of Serbia (EPS). A better choice of input features and more appropriate training set selection procedure allow significant improvement in forecasting results comparing with our previous UL/SL concept characterized by a fixed neural-net structure and absence of re-training procedure. The improvement is illustrated by reduction of average error in daily energy forecasting for 0.83% and reduction of 90th percentile of 2.04%.     

基于分层强化学习的电动汽车充电引导方法

为了有效解决电动汽车充电目的地优化和充电路径规划问题,以及充电引导的在线实时决策问题,建立了考虑多种不确定因素的电动汽车充电引导双层优化模型,提出了一种基于分层增强深度Q网络强化学习(HEDQN)的电动汽车充电引导方法。所提HEDQN算法采用基于Huber损失函数的双竞争型深度Q网络算法,并包含2层增强深度Q网络(eDQN)算法。上层eDQN用于对电动汽车充电目的地的优化;在此基础上,下层eDQN用于对电动汽车充电路径的实时优化。最后,在某城市交通网络中对所提HEDQN算法进行仿真验证,仿真结果表明相比基于Dijkstra最短路径的就近推荐算法、单层深度Q网络强化学习算法和传统的分层深度Q网络强化学习算法,所提HEDQN算法能够有效降低电动汽车充电费用,实现电动汽车在线实时的充电引导。此外还验证了所提HEDQN算法在仿真环境变化后的适应性。     

基于强化学习的异常用电判决方法

目前异常用电检测问题有许多基于分类的方法,但大多都是基于短期用电行为的判决来判断长期用电行为,判决阈值与比例难以确定,且在实际应用中,不同区域、时段的用户用电数据分布差异较大,比例与阈值也会有较大的不同,难以以固定的比例通用于所有的用户数据。针对此问题,文中提出一种基于强化学习的异常用电判决方法,创新地利用强化学习模型来动态生成阈值,以适应差异较大的不同数据集。首先获取分类器输出的数个用户短期行为的异常概率,然后输入到强化学习模型深度递归Q网络（DRQN）中,学习得到动态阈值即判决阈值与判决比例。试验结果表明,相比于人工调参的传统投票法,文中方法在评估指标上有明显提升,面对数据分布差异较大的数据集时也有较好的表现,说明文中方法具有较强的泛化能力,在数据类型复杂的现实环境中也有较好的应用场景。     

基于模糊神经网络再励学习控制器设计及其在倒车模型中的应用

考虑到神经网络学习算法的特点，给出了一种基于再励学习的自组织模糊CPN。它结合了模糊自组织CPN和再励算法的优点，在控制过程中在线调整网络结构以及对网络参数学习，学习效率高，控制结构简单。可以不要求受控对象的学习模型，实现在线控制，应用在倒车模型中仿真结果展示了所设计系统的良好控制性能。     

基于注意力机制的信息预处理多智能体强化学习算法

多智能体强化学习在群体控制领域具有广泛应用,然而传统的强化学习方法(如 Q-Learning 或策略梯度)在多智能体 环境中表现不佳。在训练过程中,每个智能体的策略不断变化。当一个智能体基于环境信息做出决策时,其他智能体的决策 可能已经影响了环境信息,导致智能体感知的转移概率分布和奖赏函数发生变化,使得环境变得非平稳,训练无法有效进行。 为了缓解这一问题,研究了一种基于多头自注意力的多智能体强化学习算法。该方法考虑了其他智能体的行动策略,利用多 头自注意力算法使智能体能够学习对决策影响最大的因素,成功地学习了复杂的多智能体协调策略。在实验结果中平均回 报达值到了0.82,远高于传统算法的表现。实验结果表明,所提出的基于多头自注意力的多智能体强化学习算法能够有效解 决环境不平稳导致的多智能体学习困难问题,提高了多智能体强化学习算法的收敛速度和平稳性。