Visual Avoidance of Collision with Randomly Moving Obstacles through Approximate Reinforcement Learning

In this research work, a hierarchical controller has been designed for an autonomous navigation robot to avoid unexpected moving obstacles where the state and action spaces are continuous. The proposed scheme consists of two parts: 1) a controller with a high-level approximate reinforcement learning (ARL) technique for choosing an optimal trajectory in autonomous navigation; and 2) a low-level, appearance-based visual servoing (ABVS) controller which controls and execute the motion of the robot. A novel approach for path planning and visual servoing has been proposed by the combined system framework. The characteristics of the on-board camera which is equipped on the robot is naturally suitable for conducting the reinforcement learning algorithm. Regarding the ARL controller, the computational overhead is quite low thanks to the fact that a knowledge of obstacle motion is not necessary. The developed scheme has been implemented and validated in a simulation system of obstacle avoidance. It is noted that findings of the proposed method are successfully verified by obtaining an optimal robotic plan motion strategy.