State inverse and decorrelated state stochastic approximation

Stochastic approximation algorithms for parameter identification are derived by a sequential optimization and weighted averaging procedure with an instructive geometric interpretation. Known algorithms including standard least squares and suboptimal versions requiring less computational effort are thereby derived. More significantly, novel schemes emerge from the theory which, in the cases studied to date and reported here, converge much more rapidly than their nearest rivals amongst the class of known simple schemes. The novel algorithms are distinguished from the known ones by either a different step size selection, and/or by working with a transformed state variable with components relatively less correlated, and/or by replacing the state vector in a crucial part of the calculations by its componentwise pseudoinverse.The convergence rate of the novel schemes in our simulations is significantly closer to that of the more sophisticated optimal least square recursions than other stochastic approximations schemes in the literature. For the case of extended least squares and recursive maximum likelihood schemes, the novel stochastic recursion performs, in loose terms within a factor of 10 (rms error), of the more sophisticated schemes in the literature. An asymptotic convergence analysis for the algorithms is a minor extension of known theory.