一种基于独立任务的POMDP问题的解决方法*

部分可观测马尔可夫决策过程(POMDP)是马尔可夫决策过程(MDP)的扩展。通常利用POMDPs来模拟在部分可观测的随机环境中决策的Agents。针对完整POMDP的求解方法扩展能力弱的问题,提出把一个多元的POMDP分解成一组受限制的POMDPs,然后分别独立地求解每个这样的模型,获得一个值函数并将这些受限制的POMDPs的值函数结合起来以便获得一个完整POMDP的策略。该方法主要阐述了识别与独立任务相关的状态变量的过程,以及如何构造一个被限制在一个单独任务上的模型。将该方法应用到两个不同规模的岩石采样问题中,实验结果表明,该方法能够获得很好的策略。

A solution based on Independent-Tasks POMDP problems

A partially observable Markov decision process (POMDP) is an extension of a Markov decision process (MDP). POMDPs are widely used to model agents acting in a stochastic environment under partial observability. Because the complete POMDP solvers have poor ability to scale up, We propose to decompose a factored POMDP into a set of restricted POMDPs and solve each such model independently, acquiring a value function. And then, the combination of the value functions of the restricted POMDPs is used to form a policy for the complete POMDP. In this paper, We mainly explain the process of identifying state variables that correspond to independent tasks, and how to create a model restricted to a single task. Using this method on RockSample domain with two different size, experiment results show that this method can gain a good policy.