Hierarchical multi-robot navigation and formation in unknown environments via deep reinforcement learning and distributed optimization |
| |
| Affiliation: | 1. School of Control Science and Engineering, Shandong University, 17923 Jingshi Road, Jinan 250061, PR. China;2. Engineering Research Center of Intelligent Unmanned System, Ministry of Education, Jinan 250061, PR. China;1. Key Laboratory of Road Construction Technology and Equipment of MOE, Chang''an University, Xi''an 710064, China;2. School of Construction Machinery, Chang''an University, Xi''an, 710064, China;3. Department of Mechanical Engineering, University of Alberta, Edmonton, AB T6G 2E1, Canada;4. Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, 518055, China;1. School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, 200444, China;2. Department of Mechanical and Mechatronics Engineering, The University of Auckland, Auckland, 1010, New Zealand;3. Department of Aerospace Engineering, Toronto Metropolitan University, Toronto, ON M5B 2K3, Canada;4. Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, Shanghai, 200072, China |
| |
| Abstract: | Compared with a single robot, Multi-robot Systems (MRSs) can undertake more challenging tasks in complex scenarios benefiting from the increased transportation capacity and fault tolerance. This paper presents a hierarchical framework for multi-robot navigation and formation in unknown environments with static and dynamic obstacles, where the robots compute and maintain the optimized formation while making progress to the target together. In the proposed framework, each single robot is capable of navigating to the global target in unknown environments based on its local perception, and only limited communication among robots is required to obtain the optimal formation. Accordingly, three modules are included in this framework. Firstly, we design a learning network based on Deep Deterministic Policy Gradient (DDPG) to address the global navigation task for single robot, which derives end-to-end policies that map the robot’s local perception into its velocity commands. To handle complex obstacle distributions (e.g. narrow/zigzag passage and local minimum) and stabilize the training process, strategies of Curriculum Learning (CL) and Reward Shaping (RS) are combined. Secondly, for an expected formation, its real-time configuration is optimized by a distributed optimization. This configuration considers surrounding obstacles and current formation status, and provides each robot with its formation target. Finally, a velocity adjustment method considering the robot kinematics is designed which adjusts the navigation velocity of each robot according to its formation target, making all the robots navigate to their targets while maintaining the expected formation. This framework allows for formation online reconfiguration and is scalable with the number of robots. Extensive simulations and 3-D evaluations verify that our method can navigate the MRS in unknown environments while maintaining the optimal formation. |
| |
| Keywords: | Multi-robot systems (MRSs) Deep reinforcement learning Mobile robot navigation Collision avoidance Formation control Distributed optimization |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
