基于Struts框架的Web应用重构

重构是在不改变软件可观察行为的前提下,通过对软件内部结构的改变,提高软件的可理解性并降低变化成本。Struts是目前Web应用表现层MVC模式的一个流行实现框架。以一个典型的Web应用为例,通过分析发生在此应用的两个迭代生命周期之间的重构过程（基于Struts框架）,以及实施重构之后给系统带来的积极影响,验证了重构在平衡满足用户需求与保证开发过程健康有序进行之间的作用。     

基于DRL的综合能源系统优化运行研究

发展以电网为核心,电、热、气多能互补、协同供能的综合能源系统是落实“双碳”的重要手段,但是电-热-气联合运行的综合能源系统存在的经济性问题和稳定性问题有待解决。本文致力于采用机器学习算法在兼顾运行稳定性的情况下解决电-热-气联合运行系统的经济性问题。首先,本文对包含储能和电转气装置的综合能源系统进行建模,结合优化运行问题优化目标-约束条件的一般框架,在约束条件中考虑功率平衡、各机组出力限制、爬坡率限制和容量限制因素;然后,本文设计了基于DRL的电-热-气联合系统优化运行问题求解策略,算法结合了强化学习策略选择的优势和深度学习环境模拟的优势,在算法设计中详细考虑动作空间、回报函数、状态空间、DRL算法、DRL网络五大模块;最后,本文设计了4个算例,结合电-热-气联合系统典型日运行条件,验证了采用电-热-气联合运行供能模式可以有效实现多能互补降低用能成本,并且本文设计的DRL方法可以有效求解电-热-气联合系统的优化运行问题。     

深度强化学习复原多目标航迹的TOC奖励函数

针对航迹探测领域中探测器获得的目标地理位置通常是同一帧下无法区分的多目标场景,需要利用目标位置信息还原各航迹并区分各目标的问题进行研究,提出采用深度强化学习复原目标航迹的方法。依据目标航迹的物理特点,提取数学模型,结合目标航迹的方向、曲率等提出轨迹曲率圆（TOC）奖励函数,使深度强化学习能够有效复原多目标航迹并区分各目标。首先描述多目标航迹复原问题,并将问题建模成深度强化学习能够处理的模型;结合TOC奖励函数对多目标航迹复原问题进行实验;最后给出该奖励函数的数学推导和物理解释。实验结果表明,TOC奖励函数驱动下的深度强化网络能够有效还原目标的航迹,在航向和航速方面切合实际目标航迹。     

Routing using reinforcement learning in vehicular ad hoc networks

In vehicular ad hoc networks (VANETs), the frequent change in vehicle mobility creates dynamic changes in communication link and topology of the network. Hence, the key challenge is to address and resolve longer transmission delays and reduced transmission stability. During the establishment of routing path, the focus of entire research is on traffic detection and road selection with high traffic density for increased packet transmission. This reduces the transmission delays and avoids carry-and-forward scenarios; however, these techniques fail in obtaining accurate traffic density in real-time scenario due to rapid change in traffic density. Thus, it is necessary to create a model that efficiently monitors the traffic density and assist VANETs in route selection in an automated way with increased accuracy. In this article, a novel machine learning architecture using deep reinforcement learning (DRL) model is proposed to monitor and estimate the data essential for the routing protocol. In this model, the roadside unit maintains the traffic information on roads using DRL. The DRL predicts the movement of the vehicle and makes a suitable routing path for transmitting the packets with improved transmission capacity. It further uses predicted transmission delays and the destination location to choose the forwarding directions between two road safety units (RSUs). The application of DRL over VANETs yields increased network performance, which provides on-demand routing information. The simulation results show that the DRL-based routing is effective in routing the data packets between the source and destination vehicles than other existing method.     

Parametric Matter

The assimilation of digital techniques in production and manufacturing has enabled innovative explorations into nonstandard material organisation. Yusuke Obuchi, Theodore Spyropoulos and Tom Verebes of the Design Research Lab (DRL) at the Architectural Association (AA) describe how an interest in a textiles approach has developed out of a preoccupation with the interrelationship between material and computational design methods. They describe three projects that aspire to embed forms of material intelligence into architectural space. Copyright © 2006 John Wiley & Sons, Ltd.     

Collective Intelligence in Design

In their introduction to this issue, Christopher Hight and Chris Perry define the idea of collective intelligence in its relationship to design practice and to broader technological and social formations. First they suggest a reformulation of practices around networked communication infrastructures as conduits for the new orchestrations of power that Antonio Negri and Michael Hardt detailed in their books Empire and Multitude. They then describe how such practices are often involved in the development of responsive sensing environments as new sites for manifesting the social organisations and communities made possible via telecommunications and the Internet. Lastly, they address how traditional boundaries of design disciplines and knowledge, from architecture to programming, are opening into complex co-minglings of their respective isolated ‘intelligences’ into collectives capable of engaging these new sites, new briefs and new sorts of projects. Copyright © 2006 John Wiley & Sons, Ltd.     

基于深度强化学习的电力物资配送多目标路径优化

针对现有电力物资车辆路径问题（EVRP）优化时考虑目标函数较为单一、约束不够全面,并且传统求解算法效率不高的问题,提出一种基于深度强化学习（DRL）的电力物资配送多目标路径优化模型和求解算法。首先,充分考虑了电力物资配送区域的加油站分布情况、物资运输车辆的油耗等约束,建立了以电力物资配送路径总长度最短、成本最低、物资需求点满意度最高为目标的多目标电力物资配送模型;其次,设计了一种基于DRL的电力物资配送路径优化算法DRL-EVRP求解所提模型。DRL-EVRP使用改进的指针网络（Ptr-Net）和Q-学习（Q-learning）算法结合的深度Q-网络（DQN）来将累积增量路径长度的负值与满意度之和作为奖励函数。所提算法在进行训练学习后,可直接用于电力物资配送路径规划。仿真实验结果表明,DRL-EVRP求解得到的电力物资配送路径总长度相较于扩展C-W(ECW)节约算法、模拟退火（SA）算法更短,且运算时间在可接受范围内,因此所提算法能更加高效、快速地进行电力物资配送路径优化。     

基于深度强化学习的多阶段信息物理协同拓扑攻击方法

随着智能电网的发展及通信设备不断引入到信息物理系统(cyber physical system,CPS)中,CPS正面临一种破坏性更强的新型攻击方式——信息物理协同攻击(coordinated cyber physical attack,CCPA),其隐蔽性与威胁性易导致系统出现级联故障。首先,基于攻击者的视角,提出一种多阶段信息物理协同拓扑攻击模型,单阶段的物理攻击使线路中断,双阶段的网络攻击分别用来掩盖物理攻击的断开线路和制造一条新的虚假断开线路。其次,结合深度强化学习(deep reinforcement learning,DRL)理论,提出一种基于深度Q网络(deep Q-network,DQN)的最小攻击资源确定方法。然后,给出攻击者考虑上层最大化物理攻击效果和下层最小化攻击代价的具体模型及求解方法。最后,以IEEE 30节点系统为例,验证了所提多阶段攻击模型的有效性。仿真结果表明,多阶段信息物理协同拓扑攻击较单一攻击更加隐蔽且有效,对电网的破坏程度更大,为防御此类攻击提供了参考。     

一种基于图神经网络的改进邻域搜索算法

近年来图神经网络与深度强化学习的发展为组合优化问题的求解提供了新的方法。当前此类方法大多未考虑到算法参数学习问题,为解决该问题,基于图注意力网络设计了一种智能优化模型。该模型对大量问题数据进行学习,自动构建邻域搜索算子与序列破坏终止符,并使用强化学习训练模型参数。在标准算例集上测试模型并进行三组不同实验。实验结果表明,该模型学习出的邻域搜索算子具备较强的寻优能力和收敛性,同时显著降低了训练占用显存。该模型能够在较短时间内求解包含数百节点的CVRP问题,并具有一定的扩展潜力。     

强化学习求解组合最优化问题的研究综述

组合最优化问题(COP)的求解方法已经渗透到人工智能、运筹学等众多领域.随着数据规模的不断增大、问题更新速度的变快,运用传统方法求解COP问题在速度、精度、泛化能力等方面受到很大冲击.近年来,强化学习(RL)在无人驾驶、工业自动化等领域的广泛应用,显示出强大的决策力和学习能力,故而诸多研究者尝试使用RL求解COP问题,...