A NEW APPROACH TO FILTERING AND ADAPTIVE CONTROL: OPTIMALITY RESULTS

Abstract

In a previous paper (Tesfatsion, in press) a new method was proposed for adaptive control. The key distinguishing feature is the direct consistent estimation and updating of the criterion (expected returns) function without recourse to prior state space specification, updated state probabilities, and Bayes's rule. The stability of a simple linear criterion function filtering scheme designed for control-dependent states was investigated in detail. In particular, it was shown that control variable sequences selected in accordance with the directly updated criterion function estimates converge under plausible restrictions to a local maximum of the true criterion function.

The principal purpose of the present paper is to establish sufficient conditions for control variables selected in accordance with the linear scheme to converge to a global maximum of the true criterion function. As will be clarified, when states are nontrivially dependent on control variable selection, the decision maker determines the trade-off between rate of convergence and asymptotic global optimality of control variable selections by his choice of greatest lower bound for the prior (initial period) criterion function. For trivial state-control dependence the asymptotic global optimality of control variable selections holds under weak restrictions.