基于改进强化学习的模块化自重构机器人编队

针对传统强化学习算法在训练初期缺乏对周围环境的先验知识，模块化自重构机器人会随机选择动作，导致迭代次数浪费和算法收敛速度缓慢的问题，提出一种两阶段强化学习算法。在第一阶段，利用基于群体和知识共享的Q-learning训练机器人前往网格地图的中心点，以获得一个最优共享Q表。在这个阶段中，为了减少迭代次数，提高算法的收敛速...

Formation of Modular Self-reconfigurable Robots Based on Improved Reinforcement Learning

Based on the traditional reinforcement learning algorithm, due to a lack of prior knowledge of the surrounding environment, the modular self-reconfigurable robot will randomly select actions, resulting in a waste of iterations and slow convergence. A two-stage reinforcement learning algorithm is proposed. In the first stage, based on knowledge sharing among robots, the improved Q-learning algorithm is proposed to speed up the training process and obtain the optimal Q table. In this stage, to reduce the number of iterations and improve the convergence speed of the algorithm, Manhattan distance is introduced as the reward value to guide the robot to move in the direction favorable to the center point and reduce the influence of sparse reward. In the second stage, according to the resulting Q table and the current position, each robot finds the optimal path to the specified target point and forms the specified formation. The experimental results show that in a 50×50 grid map, compared with the comparison algorithm, the algorithm successfully trains the robots to reach the specified target points, reducing the total number of exploration steps by nearly 50%. In addition, when the robots perform formation switching, the formation runtime is reduced by nearly five times.