Path planning of modular robots on various terrains using Q-learning versus optimization algorithms

Self-reconfigurable modular robots (SRMRs) have recently attracted considerable attention because of their numerous potential applications in the real world. In this paper, we draw a comprehensive comparison among five different algorithms in path planning of a novel SRMR system called ACMoD through an environment comprised of various terrains in a static condition. The contribution of this work is that the reconfiguration ability of ACMoD has been taken into account. This consideration, though raises new algorithmic challenges, equips the robot with new capability to pass difficult terrains rather than bypassing them, and consequently the robot can achieve better performance in terms of traversal time and energy consumption. In this work, four different optimization algorithms, including Adaptive Genetic Algorithm, Elitist Ant System, Dijkstra and Dynamic Weighting A*, along with a well-known reinforcement learning algorithm called Q-Learning, are proposed to solve this path planning problem. The outputs of these algorithms are the optimal path through the environment and the associated configuration on each segment of the path. The challenges involved in mapping the path planning problem to each algorithm are discussed in full details. Eventually, all algorithms are compared in terms of the quality of their solutions and convergence rate.