A learning theory approach to system identification and stochastic adaptive control

In this paper, we present an approach to system identification based on viewing identification as a problem in statistical learning theory. Apparently, this approach was first mooted in [E. Weyer, R.C. Williamson, I. Mareels, Sample complexity of least squares identification of FIR models, in: Proceedings of the 13th World Congress of IFAC, San Francisco, CA, July 1996, pp. 239–244]. The main motivation for initiating such a program is that traditionally system identification theory provide asymptotic results. In contrast, statistical learning theory is devoted to the derivation of finite-time estimates. If system identification is to be combined with robust control theory to develop a sound theory of indirect adaptive control, it is essential to have finite-time estimates of the sort provided by statistical learning theory. As an illustration of the approach, a result is derived showing that in the case of systems with fading memory, it is possible to combine standard results in statistical learning theory (suitably modified to the present situation) with some fading memory arguments to obtain finite-time estimates of the desired kind. It is also shown that the time series generated by a large class of BIBO stable nonlinear systems has a property known as β-mixing. As a result, earlier results of [E. Weyer, Finite sample properties of system identification of ARX models under mixing conditions, Automatica, 36 (9) (2000) 1291–1299] can be applied to many more situations than shown in that paper.