An Evolutionary Dynamical Analysis of Multi-Agent Learning in Iterated Games |
| |
Authors: | Karl Tuyls Pieter Jan ’T Hoen Bram Vanschoenwinkel |
| |
Affiliation: | (1) Theoretical Computer Science Group, Hasselt University, Belgium;(2) Centre for Mathematics and Computer Science, Amsterdam, The Netherlands;(3) Computational Modeling Lab, Vrije Universiteit Brussel, Belgium |
| |
Abstract: | In this paper, we investigate Reinforcement learning (RL) in multi-agent systems (MAS) from an evolutionary dynamical perspective.
Typical for a MAS is that the environment is not stationary and the Markov property is not valid. This requires agents to
be adaptive. RL is a natural approach to model the learning of individual agents. These Learning algorithms are however known
to be sensitive to the correct choice of parameter settings for single agent systems. This issue is more prevalent in the
MAS case due to the changing interactions amongst the agents. It is largely an open question for a developer of MAS of how
to design the individual agents such that, through learning, the agents as a collective arrive at good solutions. We will
show that modeling RL in MAS, by taking an evolutionary game theoretic point of view, is a new and potentially successful
way to guide learning agents to the most suitable solution for their task at hand. We show how evolutionary dynamics (ED)
from Evolutionary Game Theory can help the developer of a MAS in good choices of parameter settings of the used RL algorithms.
The ED essentially predict the equilibriums outcomes of the MAS where the agents use individual RL algorithms. More specifically,
we show how the ED predict the learning trajectories of Q-Learners for iterated games. Moreover, we apply our results to (an
extension of) the COllective INtelligence framework (COIN). COIN is a proved engineering approach for learning of cooperative
tasks in MASs. The utilities of the agents are re-engineered to contribute to the global utility. We show how the improved
results for MAS RL in COIN, and a developed extension, are predicted by the ED.
Author funded by a doctoral grant of the institute for advancement of scientific technological research in Flanders (IWT). |
| |
Keywords: | multi-agent systems iterated games reinforcement learning Evolutionary Game Theory COllective INtelligence |
本文献已被 SpringerLink 等数据库收录! |
|