Asymptotically optimal smoothing of averaged LMS estimates for regression parameter tracking

The sequence of estimates formed by the LMS algorithm for a standard linear regression estimation problem is considered. It is known since earlier that smoothing these estimates by simple averaging will lead to, asymptotically, the recursive least-squares algorithm. In this paper, it is first shown that smoothing the LMS estimates using a matrix updating will lead to smoothed estimates with optimal tracking properties, also in case the true parameters are slowly changing as a random walk. The choice of smoothing matrix should be tailored to the properties of the random walk. Second, it is shown that the same accuracy can be obtained also for a modified algorithm, SLAMS, which is based on averages and requires much less computations.