Finite state stochastic games: Existence theorems and computational procedures

Let{X_{n}}be a Markov process with finite state space and transition probabilitiesp_{ij}(u_{i}, v_{i})depending on uiandv_{i}.State 0 is the capture state (where the game ends;p_{oi} equiv delta_{oi});u = {u_{i}}andv = {v_{i}}are the pursuer and evader strategies, respectively, and are to be chosen so that capture is advanced or delayed and the costC_{i^{u,v}} = ESum_{0}^{infty} k (u(X_{n}), v(X_{n}), X_{n}) | X_{0} = i]is minimaxed (or maximined), wherek(alpha, beta, 0) equiv 0. The existence of a saddle point and optimal strategy pair or e-optimal strategy pair is considered under several conditions. Recursive schemes for computing the optimal or ε-optimal pairs are given.