Reinforcement Learning Transfer Based on Subgoal Discovery and Subtask Similarity |
| |
Authors: | Hao Wang Shunguo Fan Jinhua Song Yang Gao Xingguo Chen |
| |
Affiliation: | 1. Department of Computer Science and Technology, Nanjing University, China;2. School of Computer Science of Technology, School of Software, Nanjing University of Posts and Telecommunications, China |
| |
Abstract: | This paper studies the problem of transfer learning in the context of reinforcement learning. We propose a novel transfer learning method that can speed up reinforcement learning with the aid of previously learnt tasks. Before performing extensive learning episodes, our method attempts to analyze the learning task via some exploration in the environment, and then attempts to reuse previous learning experience whenever it is possible and appropriate. In particular, our proposed method consists of four stages: 1) subgoal discovery, 2) option construction, 3) similarity searching, and 4) option reusing. Especially, in order to fulfill the task of identifying similar options, we propose a novel similarity measure between options, which is built upon the intuition that similar options have similar stateaction probabilities. We examine our algorithm using extensive experiments, comparing it with existing methods. The results show that our method outperforms conventional non-transfer reinforcement learning algorithms, as well as existing transfer learning methods, by a wide margin. |
| |
Keywords: | Reinforcement learning transfer learning subtask options source task library |
|
| 点击此处可从《IEEE/CAA Journal of Automatica Sinica》浏览原始摘要信息 |
|
点击此处可从《IEEE/CAA Journal of Automatica Sinica》下载免费的PDF全文 |
|