基于Markov对策的多Agent强化学习模型及算法研究

在ＭＤＰ，单Ａｇｅｎｔ可以通过强化学习来寻找问题的最优解。但在多Ａｇｅｎｔ系统中，ＭＤＰ模型不再适用。同样极小极大Ｑ算法只能解决采用零和对策模型的ＭＡＳ学习问题。文中采用非零和Ｍａｒｋｏｖ对策作为多Ａｇｅｎｔ系统学习框架，并提出元对策强化学习的学习模型和元对策Ｑ算法。理论证明元对策Ｑ算法收敛在非零和Ｍａｒｋｏｖ对策的元对策最优解。

RESEARCH ON MARKOV GAME-BASED MULTIAGENT REIN-FORCEMENT LEARNING MODEL AND ALGORITHMS

In Markov decision process, a single agent could find the optimal policy of the problem by reinforcement learning. But the model of the MDP doesn't adapt to the multi-agent system. And the minmax-Q learning algorithm could only solve the problem of zero-sum Markov games. In this paper, the non-zero-sum Markov games are adopted as a framework for multi-agent reinforcement learning, and the learning model and learning algorithms of the metagame reinforcement learning are brought forward. It is proved that this metagame-Q algorithms must converge at the most optimal value of the non-zero-game Markov games.