Optimal control of unknown parameter systems

The problem is discussed of finding a cost functional for which an adaptive control law is optimal. The system under consideration is a partially observed linear stochastic system with unknown parameters. It is well known that an optimal finite-dimensional filter for this problem can be derived when the parameters belong to a finite set. Since the optimal filter involves the evaluation of a finite set of a posteriori probabilities for each of the parameter values given the observations, a natural adaptive control scheme is: (i) develop the optimal linear feedback law given each parameter; (ii) use the a posteriori probabilities to form the weighted average (convex combination) of the individual control policies; and (iii) use the weighted average as the control law. A quadratic cost functional is devised for which this strategy is optimal, in a general case, and it is shown that the probing effect identified with dual control problems is inherent in the standard linear-quadratic-Gaussian problem with parameter uncertainty