Convergence of adaptive control schemes using least-squaresparameter estimates

The stability, convergence, asymptotic optimality, and self-tuning properties of stochastic adaptive control schemes based on least-squares estimates of the unknown parameters are examined. It is assumed that the additive noise is i.i.d. and Gaussian, and that the true system is of minimum phase. The Bayesian embedding technique is used to show that the recursive least-squares parameter estimates converge in general. The normal equations of least squares are used to establish that all stable control law designs used in a certainty-equivalent (i.e. indirect) procedure generally yield a stable adaptive control system. Four results are given to characterize the limiting behavior precisely. A certainty-equivalent self-tuning regulator is shown to yield strongly consistent parameter estimates when the delay is strictly greater than one, even without any excitation in the reference trajectory     

Performance analysis of stochastic gradient algorithms under weak conditions

By using the stochastic martingale theory, convergence properties of stochastic gradient （SG） identification algorithms are studied under weak conditions. The analysis indicates that the parameter estimates by the SG algorithms consistently converge to the true parameters, as long as the information vector is persistently exciting （i.e., the data product moment matrix has a bounded condition number） and that the process noises are zero mean and uncorrelated. These results remove the strict assumptions, made in existing references, that the noise variances and high-order moments exist, and the processes are stationary and ergodic and the strong persis- tent excitation condition holds. This contribution greatly relaxes the convergence conditions of stochastic gradient algorithms. The simulation results with bounded and unbounded noise variances confirm the convergence conclusions proposed.     

Adaptive control with the stochastic approximation algorithm: Geometry and convergence

New geometric properties possessed by the sequence of parameter estimates are exhibited, which yield valuable insight into the behavior of the stochastic approximation based algorithm as it is used in minimum variance adaptive control. In particular, these geometric properties, together with certain probabilistic arguments, prove that if the system does not have a reduced-order minimum variance controller, then the parameter estimates converge to a random multiple of the true parameter. An explicit expression for the limiting parameter estimate is also available. With strictly positive probability, the limiting parameter estimate is not the true parameter, and in some cases differs from the true parameter with probability one. If the system possesses reduced-order minimum variance controllers, then convergence to a minimum variance controller in a Cesaro sense is shown. The geometry of the limit set is described. Sufficient conditions are also given for some of these results to hold for parameter estimation schemes other than stochastic approximation.     

Invariant manifolds and asymptotic properties of adaptive nonlinearstabilizers

A classical question in adaptive control is that of convergence of the parameter estimates to constant values in the absence of persistent excitation. The author provides an affirmative answer for a class of adaptive stabilizers for nonlinear systems. Then the author studies their asymptotic behavior by considering the problem of whether the parameter estimates converge to stabilizing values-the values which would guarantee stabilization if used in a nonadaptive controller. The author approaches this problem by studying invariant manifolds and shows that except for a set of initial conditions of Lebesgue measure zero, the parameter estimates do converge to stabilizing values. Finally, the author determines a (sufficiently large) time instant after which the adaptation can be disconnected at any time without destroying the closed-loop system stability     

Stability, convergence, and performance of an adaptive controlalgorithm applied to a randomly varying system

The stability and performance of a stochastic adaptive control algorithm applied to a randomly varying linear system are investigated. The authors demonstrate that: loss functions on the input-output process converge to their expectation with respect to an invariant probability at a geometric rate, and hence, a form of stochastic exponential asymptotic stability is established; and when the parameter variation and measurement noise are small, it is shown that the performance is nearly optimal, and if an excitation signal is added in the control law, near consistency of the parameter estimates is obtained. Further results include central limit theorems and the law of large numbers of the input-output and parameter processes     

A control structure using an adaptive observer

The state and parameter estimates produced by a recently reported adaptive observer [3] are used to form an adaptive control scheme. For a large class of inputs, the adaptive process will converge in the sense that asymptotically the system transfer function will exhibit a set of desired poles. The rate of parameter convergence can be made arbitrarily fast by appropriate choice of the constant gain parameters in the adaptive laws. However, a limitation of the apparent closed-loop control structure is exposed.     

Almost sure convergence of extremum seeking algorithm using stochastic perturbation

This paper presents an extremum seeking (ES) algorithm where the perturbation signal is a martingale difference sequence (m.d.s.) with a vanishing variance. The measurement noise at the plant output is modeled by a superposition of deterministic component, and a non-stationary colored noise signal. The optimizing set point of the uncertain reference-to-output equilibrium map is estimated by a stochastic approximation (SA)-type algorithm. The algorithm has a vanishing gain sequence dependent on the set point estimates. By utilizing powerful tools of the martingale convergence theory it is proved that with probability one the set point estimates converge to the optimizing equilibrium point, in spite of the presence of a measurement noise. This result is derived without requiring boundedness or any prior condition on the set point estimates.     

Continuous-time adaptive filtering

This note establishes necessary and sufficient conditions for convergence of Bayesian parameter estimates in continuous-time adaptive Kalman filter with a denumerable or finite set of parameter values.     

A method for adaptive estimation of ARMA processes

When the noise process in adaptive identification of linear stochastic systems is correlated, and can be represented by a moving average model, extended least squares algorithms are commonly used, and converge under a strictly positive real (SPR) condition on the noise model. In this paper, we present an adaptive algorithm for the estimation of autoregressive moving average (ARMA) processes, and show that it is convergent without any SPR condition, and has a convergence rate of O({loglog t)/t}^1/2).     

Preconditioned Bayesian Regression for Stochastic Chemical Kinetics

We develop a preconditioned Bayesian regression method that enables sparse polynomial chaos representations of noisy outputs for stochastic chemical systems with uncertain reaction rates. The approach is based on the definition of an appropriate multiscale transformation of the state variables coupled with a Bayesian regression formalism. This enables efficient and robust recovery of both the transient dynamics and the corresponding noise levels. Implementation of the present approach is illustrated through applications to a stochastic Michaelis–Menten dynamics and a higher dimensional example involving a genetic positive feedback loop. In all cases, a stochastic simulation algorithm (SSA) is used to compute the system dynamics. Numerical experiments show that Bayesian preconditioning algorithms can simultaneously accommodate large noise levels and large variability with uncertain parameters, and that robust estimates can be obtained with a small number of SSA realizations.     

Adaptive control for systems with bounded disturbances

The authors present an identification algorithm that can be used to design globally stable indirect adaptive controllers for minimum- and nonminimum-phase systems subject to bounded disturbances. The parameter estimation scheme is a least-squares algorithm with dead zone. The dead zone is such that the estimates converge and make it possible to define projected estimates having the same convergence properties as the original estimates. In the minimum-phase case, the projection facility can be used to ensure that the leading coefficient projected estimate is greater than or equal to the true leading coefficient in absolute value. The projection procedure can also be used to avoid pole-zero cancellations in an adaptive pole-placement algorithm     

Extended LQG self-tuning controller in a mixed background

We consider the problem of constructing a LQG-based adaptive controller so that the control system can track a deterministic command signal and simultaneously regulate stochastic disturbances. A new measure for mixed signal in the feedback loop is first introduced. Then a generalized cost function is formulated and the optimal ‘extended LQG controller’ can be constructed by solving a diophantine equation. Based on the extended LQG design, a self-tuning controller can be constructed. System tracking performance under ergodic process noise is analysed in detail and better results are obtained. Finally, the optimal tracking performance is attained if the parameter estimates converge to their true values.     

基于数据滤波的随机梯度辨识方法

针对有色噪声干扰下的随机系统,利用数据滤波技术,对输入输出数据进行滤波,将具有滑动平均噪声的原始系统转换为白噪声干扰下的系统,提出有限脉冲响应滑动平均系统的滤波增广随机梯度算法,并对该算法进行收敛性分析.此外,为了提高参数估计的精度和加快算法的收敛速度,使用多新息辨识理论提出滤波多新息增广随机梯度算法,并分析其收敛性.与增广随机梯度算法相比,所提出的滤波增广随机梯度算法和滤波多新息增广随机梯度算法可以得到更高精度的参数估计.最后,通过仿真实例表明了所提出算法的有效性.     

一种盲自适应随机共振数字基带信号处理方法

随机共振能够明显提高输出信噪比,在信号处理领域得到了广泛关注。与传统噪声调节随机共振相比,参数调节随机共振增强了随机共振的鲁棒性,但面临如何选取最佳系统参数的问题。以峭度和负熵来度量非线性系统输出信号的概率分布情况,找出了输出信号的概率分布特性与最佳系统参数之间的对应关系。在此基础上,提出一种盲自适应随机共振方法,并将其应用于数字基带二进制信号处理之中。该方法利用输出信号的峭度或负熵数值引导非线性系统参数迭代,使之自适应达到随机共振状态。该方法可解决最佳系统参数的选取问题,能够增强随机共振的灵活性及鲁棒性。利用MATLAB软件搭建随机共振仿真平台对提出方法进行了实验验证,仿真结果表明,该方法能够迅速收敛到最佳系统参数值,进而明显提高输出信号的信噪比。     

On robustness to noise of least squares based adaptive control

Current engineering practice for adaptive control schemes is to base the design on globally convergent schemes for simple plant models. An important class of such schemes uses least squares estimation of assumed simple input-output models and constructs the controller using the parameter estimates. This paper studies the robustness of such schemes to the presence of unmodelled plant coloured noise. Such noise is sometimes an adequate model for unmodelled plant dynamics.The theory of the paper makes a connection between the least squares parameter error equations and those associated with extended least squares using a posteriori noise estimates for which there are known global convergence results. For the case of adaptive minimum variance control of minimum phase plants, this connection permits stronger convergence results than those hitherto derived from the theory of extended least squares based on a priori noise estimates.     

时变系统遗忘因子最小二乘法的有界收敛性

利用随机过程理论研究了遗忘因子最小二乘法 (FFLS)的有界收敛性, 给出了参数估计误差的上界. 分析表明: i)对于时不变确定性系统, FFLS算法产生的参数估计以指数速度收敛于真参数; ii)对于时不变随机系统, FFLS算法给出有界均方估计误差; iii)对于时变随机系统, FFLS算法可以跟踪时变参数, 且跟踪误差有界.     

Recursive identification of Hammerstein systems with dead-zone nonlinearity in the presence of bounded noise

The existing identification algorithms for Hammerstein systems with dead-zone nonlinearity are restricted by the noise-free condition or the stochastic noise assumption. Inspired by the practical bounded noise assumption, an improved recursive identification algorithm for Hammerstein systems with dead-zone nonlinearity is proposed. Based on the system parametric model, the algorithm is derived by minimising the feasible parameter membership set. The convergence conditions are analysed, and the adaptive weighting factor and the adaptive covariance matrix are introduced to improve the convergence. The validity of this algorithm is demonstrated by two numerical examples, including a practical DC motor case.     

Adaptive and robust controller design for uncertain nonlinear systems via fuzzy modeling approach

The issue for designing robust adaptive stabilizing controllers for nonlinear systems in Takagi-Sugeno fuzzy model with both parameter uncertainties and external disturbances is studied in this paper. It is assumed that the parameter uncertainties are norm-bounded and may be of some structure properties and that the external disturbances satisfy matching conditions and, besides, are also norm-bounded, but the bounds of the external disturbances are not necessarily known. Two adaptive controllers are developed based on linear matrix inequality technique and it is shown that the controllers can guarantee the state variables of the closed loop system to converge, globally, uniformly and exponentially, to a ball in the state space with any pre-specified convergence rate. Furthermore, the radius of the ball can also be designed to be as small as desired by tuning the controller parameters. The effectiveness of our approach is verified by its application in the control of a continuous stirred tank reactor.     

一类虚拟控制系数未知的随机非线性时滞大系统的适应镇定控制

研究了一类具有未知虚拟控制系数和未知噪声协方差的随机非线性时滞大系统的适应镇定问题. 首先, 针对系统的未知虚拟控制系数和未知噪声协方差, 选取了相应的估计参数; 然后, 针对时变时滞对闭环系统稳定性的影响, 构造了适当形式的Lyapunov-Krasovskii泛函, 采用积分反推方法给出了无记忆状态反馈控制律的系统设计过程. 在一定条件下, 证明了闭环系统平衡点依概率全局稳定, 且除参数估计以外的所有闭环信号几乎均收敛到零点. 仿真算例验证了所给方法的有效性.     

Robust recursive identification of multidimensional linear regression models

Stochastic adaptive estimation and control algorithms involving recursive prediction estimates have guaranteed convergence rates when the noise is not ‘too’ coloured, as when a positive-real condition on the noise mode is satisfied. Moreover, the whiter the noise environment the more robust are the algorithms. This paper shows that for linear regression signal models, the suitable introduction of while noise into the estimation algorithm can make it more robust without compromising on convergence rates. Indeed, there are guaranteed attractive convergence rates independent of the process noise colour. No positive-real condition is imposed on the noise model.