Performance optimization algorithms based on potentials for semi-Markov control processes |
| |
Authors: | G.-P. Dai B.-Q. Yin Y.-J. Li H.-S. Xi |
| |
Affiliation: | G.-P. Dai,B.-Q. Yin *,Y.-J. Li,H.-S. Xi |
| |
Abstract: | Optimization algorithms are studied for a class of semi-Markov control processes (SMCPs) with compact action set. Both the long-run average and discounted cost problems are considered. Formulas of performance potentials and optimality equations for SMCPs are derived. A policy iteration algorithm and a value iteration algorithm are proposed, which can lead to an optimal or suboptimal stationary policy in a finite number of iterations. The convergence of these algorithms is established, without the assumption of the corresponding iteration operator being a span-contraction. In addition, an online policy iteration optimization algorithm based on a single sample path is provided to escape ‘curse of dimensionality’. In the end, a numerical example is provided to illustrate the application of the algorithms. |
| |
Keywords: | |
|
|