| Abstract: | Solution techniques for Markov decision problems rely on exact knowledge of the transition rates, which may be difficult or impossible to obtain. In this paper, we consider Markov decision problems with uncertain transition rates represented as compact sets. We first consider the problem of sensitivity analysis where the aim is to quantify the range of uncertainty of the average per‐unit‐time reward given the range of uncertainty of the transition rates. We then develop solution techniques for the problem of obtaining the max‐min optimal policy, which maximizes the worst‐case average per‐unit‐time reward. In each of these problems, we distinguish between systems that can have their transition rates chosen independently and those where the transition rates depend on each other. Our solution techniques are applicable to Markov decision processes with fixed but unknown transition rates and to those with time‐varying transition rates. |
