基于改进近端策略优化算法的草酸钴合成过程优化

金属钴被广泛用于电池和金属复合材料,草酸钴合成过程是影响产品质量的关键工序.针对草酸钴平均粒径的优化问题,提出一种基于改进的近端策略优化(PPO)算法的草酸钴合成过程优化方法.首先,根据草酸钴合成过程的优化目标及约束条件设计相应的奖励函数,通过建立过程的马尔科夫决策模型,将优化问题纳入强化学习框架;其次,针对策略网络在训练过程中出现的梯度消失问题,提出将残差网络作为PPO算法的策略网络;最后,针对过程连续状态空间导致PPO算法陷入局部最优策略问题,利用交错模仿学习对初始策略进行改进.将所提出的方法与传统PPO算法进行比较,改进的PPO算法在满足约束条件的同时,具有更好的优化效果和收敛性.

Optimization of cobalt oxalate synthesis process based on modified proximal policy optimization algorithm

Cobalt is widely used in batteries and metal composite materials, and the cobalt oxalate synthesis process is a key process that affects product quality. To optimize the average particle size of cobalt oxalate, this work presents an optimization method for the cobalt oxalate synthesis process based on the modified proximal policy optimization (MPPO) algorithm. First, the reward function is designed according to the optimization objectives and constraints of the cobalt oxalate synthesis process, and the optimization problem is incorporated into the reinforcement learning framework by establishing the Markov decision model of the process. Secondly, to deal with the problem of gradient disappearance in the training process of the policy network, the residual network is proposed as the policy network of the PPO algorithm. Finally, to avoid the PPO algorithm falling into the local optimal strategy due to the continuous state space of the process, the initial strategy is improved by interleaved imitation learning. The proposed method is compared with the traditional PPO algorithm, and the modified PPO algorithm has a better optimization effect and convergence while satisfying the constraints.