Optimal Stopping and Gittins' Indices for Piecewise Deterministic Evolution Processes

Westudy the optimal stopping problem for a class of continuoustime random evolutions described by stochastic differential equationswith alternating renewal processes as noise sources. The exactsolution of this stopping problem provides, in explicit form,an expression for the Gittins' indices needed to derive the optimalscheduling of a class of multi-armed bandit problems in continuoustime. The underlying random processes to which the bandits' armsobey are random velocity models. Such processes are commonlyused to describe, in the fluid limit, the random production flowsdelivered by failure prone machines.