基于多Agent强化学习的多站点CSPS系统的协作Look-ahead控制

研究多站点传送带给料生产加工站(Conveyor-serviced production station, CSPS)系统的最优控制问题, 其优化目标是通过合理选择每个CSPS的Look-ahead控制策略, 实现整个系统的工件处理率最大.本文首先根据多Agent系统的反应扩散思想, 对每个Agent的原始性能函数进行改进, 引入了具有扩散功能的局域信息交互项(原始项看作具有反应功能); 并运用性能势理论, 构建一种适用于平均和折扣两种性能准则的Wolf-PHC多Agent学习算法, 以求解决策时刻不同步的多站点的协作Look-ahead控制策略. 最后,论文通过仿真实验验证了该算法的有效性,学习结果表明, 通过性能函数的改进,各工作站的负载平衡性得到改善, 整个系统的工件处理率也明显提高.

Coordinated Look-ahead Control of Multiple CSPS System by Multi-agent Reinforcement Learning

The optimal control problem of a multiple conveyor-serviced production station (CSPS) system is concerned. The objective is to maximize the part-processing rate of the entire system by choosing a suitable look-ahead control strategy for each CSPS. According to the reaction-diffusion mechanism of multi-agent systems, the original performance function of each agent is first modified by introducing an item with a diffusion function that denotes the interaction of local information (The original item is assumed to have a reaction function). Then, combined with the concept of performance potentials, a multi-agent algorithm, i.e., Wolf-PHC algorithm, is proposed to derive the coordinated look-ahead control strategy for systems with either discounted or average performance criteria, where the decision epoch of each agent is asynchronous. Finally, a simulation example is used to illustrate the effectiveness of the algorithm, and the simulation results show that due to the modification of the performance functions, the contributions of all the stations are well balanced, and the part-processing rate of the entire system is increased significantly.