Application of Monte Carlo method to optimal control for linear systems under measurement noise with Markov dependent statistical property

This paper presents an optimal control algorithm for linear systems with measurement noise which has a Markov dependent statistical property. Ordinarily, the optimal control for this problem involves a very large number of sequences, and the usual calculation method becomes impractical. In the algorithm proposed here, the optimal control is calculated with a relatively small number of sequences, sampled at random from the set of all the sequences.

Evidently, the algorithm for a control problem should be obtained directly from the performance criterion. Unlike the state estimation problem, the problem considered here has a difficulty that there exists an interaction between the algorithm and the state of the system. Because of this, a special consideration is required for the design of the algorithm. In this paper control-free measurement data are introduced to establish the convergence of the algorithm and to find a desirable way of sampling the sequences. Then, certain approximations are made to design a practical and efficient algorithm. A few digital simulation results appear to show the effectiveness of the proposed method.