Multi-objective discounted Markov decision processes with expectation and variance criteria

A multi-objective discounted Markov decision process (MDP) with expectation and variance criteria is discussed. First, difficulties in variance minimization are discussed and it is shown that variance minimization is much more difficult than the expectation optimization. Then, the multi-objective MDP with expectation and variance criteria is formulated as a multi-objective non-linear programming problem. An algorithm for finding a stationary satisfactory Pareto policy is proposed by applying the satisficing trade-off method of Nakayama. In the proposed algorithm, a decision-maker need not have a high degree of judgment and it is easy to take the balance of expectation and variance criteria and furthermore, the number of auxiliary optimization problems to be solved is quite small. Numerical examples show the efficiency of the proposed algorithm.