Graph kernels and Gaussian processes for relational reinforcement learning |
| |
Authors: | Kurt Driessens Jan Ramon Thomas Gärtner |
| |
Affiliation: | (1) Department of Computer Science, University of Waikato, Private Bag 3105, Hamilton, New Zealand;(2) Department of Computer Science, Katholieke Universiteit Leuven, Celestijnenlaan 200A, B-3001 Heverlee, Belgium;(3) Knowledge Discovery, Fraunhofer Institut Autonome Intelligente Systeme, Schloss Birlinghoven, 53754 Sankt Augustin, Germany |
| |
Abstract: | RRL is a relational reinforcement learning system based on Q-learning in relational state-action spaces. It aims to enable
agents to learn how to act in an environment that has no natural representation as a tuple of constants. For relational reinforcement
learning, the learning algorithm used to approximate the mapping between state-action pairs and their so called Q(uality)-value
has to be very reliable, and it has to be able to handle the relational representation of state-action pairs. In this paper
we investigate the use of Gaussian processes to approximate the Q-values of state-action pairs. In order to employ Gaussian
processes in a relational setting we propose graph kernels as a covariance function between state-action pairs. The standard
prediction mechanism for Gaussian processes requires a matrix inversion which can become unstable when the kernel matrix has
low rank. These instabilities can be avoided by employing QR-factorization. This leads to better and more stable performance
of the algorithm and a more efficient incremental update mechanism. Experiments conducted in the blocks world and with the
Tetris game show that Gaussian processes with graph kernels can compete with, and often improve on, regression trees and instance
based regression as a generalization algorithm for RRL.
Editors: David Page and Akihiro Yamamoto |
| |
Keywords: | Reinforcement learning Relational learning Graph kernels Gaussian processes |
本文献已被 SpringerLink 等数据库收录! |
|