Self-optimizing generalized adaptive notch filters — comparison of three optimization strategies

The paper provides comparison of three different approaches to on-line tuning of generalized adaptive notch filters (GANFs) — the algorithms used for identification/tracking of quasi-periodically varying dynamic systems. Tuning is needed to adjust adaptation gains, which control tracking performance of GANF algorithms, to the unknown and/or time time-varying rate of system nonstationarity. Two out of three compared approaches are classical solutions — the first one incorporates sequential optimization of adaptation gains while the second one is based on the concept of parallel estimation. The main contribution of the paper is that it suggests the third way — it shows that the best results can be achieved when both approaches mentioned above are combined in a judicious way. Such joint sequential/parallel optimization preserves advantages of both treatments: adaptiveness (sequential approach) and robustness to abrupt changes (parallel approach). Additionally the paper shows how, using the concept of surrogate outputs, one can extend the proposed single-frequency algorithm to the multiple frequencies case, without falling into the complexity trap known as the “curse of dimensionality”.     

Estimation and tracking of complex-valued quasi-periodically varying systems

The problem of identification/tracking of quasi-periodically varying complex systems is considered. This problem is a generalization, to the system case, of a classical signal processing task of either elimination or extraction of nonstationary sinusoidal signals buried in noise. The proposed solution is based on the exponentially weighted basis function (EWBF) approach. First, the basic EWBF algorithm is derived. Then its frequency-decoupled, parallel-form and cascade-form variants, with highly modular structure and reduced computational requirements, are described. Finally, the frequency-adaptive versions of all schemes are obtained using the recursive prediction error method.     

Optimal and suboptimal smoothing algorithms for identification of time-varying systems with randomly drifting parameters

Noncausal estimation algorithms, which involve smoothing, can be used for off-line identification of nonstationary systems. Since smoothing is based on both past and future data, it offers increased accuracy compared to causal (tracking) estimation schemes, incorporating past data only. It is shown that efficient smoothing variants of the popular exponentially weighted least squares and Kalman filter-based parameter trackers can be obtained by means of backward-time filtering of the estimates yielded by both algorithms. When system parameters drift according to the random walk model and the adaptation gain is sufficiently small, the properly tuned two-stage Kalman filtering/smoothing algorithm, derived in the paper, achieves the Cramér-Rao type lower smoothing bound, i.e. it is the optimal noncausal estimation scheme. Under the same circumstances performance of the modified exponentially weighted least-squares algorithm is often only slightly inferior to that of the Kalman filter-based smoother.     

On the lower smoothing bound in identification of time-varying systems

In certain applications of nonstationary system identification the model-based decisions can be postponed, i.e. executed with a delay. This allows one to incorporate in the identification process not only the currently available information, but also a number of “future” data points. The resulting estimation schemes, which involve smoothing, are not causal. Assuming that the infinite observation history is available, the paper establishes the lower steady-state estimation bound for any noncausal estimator applied to a linear system with randomly drifting coefficients (under Gaussian assumptions). This lower bound complements the currently available one, which is restricted to causal estimators.     

Identification of linear systems using polynomial kernels in the frequency domain

In prior work we presented an identification algorithm using polynomials in the time domain. In this article, we extend this algorithm to include polynomials in the frequency domain. A polynomial is used to represent the imaginary part of the Fourier transform of the impulse response. The Hilbert transform relationship is used to compute the real part of the frequency response and hence the complete process model. The polynomial parameters are computed based on the computationally efficient linear least square method. The order of the polynomial is estimated based on residue decrement. Simulated and experimental results show the effectiveness of this method, particularly for short input/output data sequence with high signal to noise ratio. The frequency domain polynomial model complements the time domain methods since it can provide a good estimate of the time to steady state for time domain FIR (finite impulse response) models. Confidence limits in time or frequency domain can be computed using this approach. Noise rejection properties of the algorithm are illustrated using data from both simulated and real processes.     

An adaptive method for consistent estimation of real-valued non-minimum phase zeros in stable LTI systems

An adaptive algorithm, consisting of a recursive estimator for a finite impulse response model having two non-zero lags only, and an adaptive input are presented. The model is parametrized in terms of the first impulse response coefficient and the model zero. For linear time-invariant single-input single-output systems with real rational transfer functions possessing at least one real-valued non-minimum phase zero of multiplicity one, it is shown that the model zero converges to such a zero of the true system. In the case of multiple non-minimum phase zeros, the algorithm can be tailored to converge to a particular zero. The result is shown to hold for systems and noise spectra of arbitrary degree. The algorithm requires prior knowledge of the sign of the high frequency gain of the system as well as an interval to which the non-minimum phase zero of interest belongs.     

On “cheap smoothing” opportunities in identification of time-varying systems

In certain applications of nonstationary system identification the model-based decisions can be postponed, i.e. executed with a delay. This allows one to incorporate into the identification process not only the currently available information, but also a number of “future” data points. The resulting estimation schemes, which involve smoothing, are not causal. Despite the possible performance improvements, the existing smoothing algorithms are seldom used in practice, mainly because of their high computational requirements. We show that the computationally attractive smoothing procedures can be obtained by means of compensating estimation delays that arise in the standard exponentially weighted least squares, least mean squares and Kalman filter-based parameter trackers.     

Set-based adaptive estimation for a class of nonlinear systems with time-varying parameters

In this paper, an adaptive estimation technique is proposed for the estimation of time-varying parameters for a class of continuous-time nonlinear system. A set-based adaptive estimation is used to estimate the time-varying parameters along with an uncertainty set. The proposed method is such that the uncertainty set update is guaranteed to contain the true value of the parameters. Unlike existing techniques that rely on the use of polynomial approximations of the time-varying behaviour of the parameters, the proposed technique does require a functional representation of the time-varying behaviour of the parameter estimates. A simulation example and a building systems estimation example are considered to illustrate the developed procedure and ascertain the theoretical results.     

On the equivalence of time and frequency domain maximum likelihood estimation

Maximum likelihood estimation has a rich history. It has been successfully applied to many problems including dynamical system identification. Different approaches have been proposed in the time and frequency domains. In this paper we discuss the relationship between these approaches and we establish conditions under which the different formulations are equivalent for finite length data. A key point in this context is how initial (and final) conditions are considered and how they are introduced in the likelihood function.     

Stability analysis of periodic orbit of an adaptive notch filter for frequency estimation of a periodic signal

Online frequency estimation of a sinusoidal signal is a classical problem and has many practical applications. Recently an adaptive notch filter (ANF) with global convergence property has been developed for frequency estimation of a pure sinusoidal signal. This paper addresses a modified ANF structure that can estimate the fundamental frequency of any periodic signal including pure sinusoidal signals. To prove the stability of the modified ANF, the paper introduces a new theorem that shows for any periodic signal, there exists a locally asymptotically stable periodic orbit of this ANF by which the frequency estimation becomes feasible. This alternative stability proof is simple and uses widely known mathematical tools, and therefore alleviates the problem complexity even when the input signal is a pure sinusoidal signal. A further contribution of this paper is obtaining a necessary and sufficient condition in terms of design parameters for local asymptotical stability of the modified ANF. This condition, obtained from the numerical study of Floquet multipliers of a linear time-varying periodic system, provides a strict stability region in the modified ANF design parameters space.     

On noncausal weighted least squares identification of nonstationary stochastic systems

In this paper, we consider the problem of noncausal identification of nonstationary, linear stochastic systems, i.e., identification based on prerecorded input/output data. We show how several competing weighted (windowed) least squares parameter smoothers, differing in memory settings, can be combined together to yield a better and more reliable smoothing algorithm. The resulting parallel estimation scheme automatically adjusts its smoothing bandwidth to the unknown, and possibly time-varying, rate of nonstationarity of the identified system. We optimize the window shape for a certain class of parameter variations and we derive computationally attractive recursive smoothing algorithms for such an optimized case.     

Errors-in-variables identification of dynamic systems excited by arbitrary non-white input

This work deals with the identification of dynamic systems from noisy input–output observations, where the noise-free input is not parameterized. The basic assumptions made are (1) the dynamic system can be modeled by a (discrete- or continuous-time) rational transfer function model, (2) the temporal input–output disturbances are mutually independent, identically distributed noises, and (3) the input power spectrum is non-white (not necessarily rational) and is modeled nonparametrically. The system identifiability is guaranteed by exploiting the non-white spectrum property of the noise-free input. A frequency domain identification strategy is developed to estimate consistently the plant model parameters and the input–output noise variances. The uncertainty bound of the estimates is calculated and compared to the Cramér–Rao lower bound. The efficiency of the proposed algorithm is illustrated on numerical examples.     

On the frequency domain accuracy of closed-loop estimates

It has been argued that the frequency domain accuracy of high model-order estimates obtained on the basis of closed-loop data is largely invariant to whether direct or indirect approaches are used. The analysis underlying this conclusion has employed variance expressions that are asymptotic both in the data length and the model order, and hence are approximations when either of these are finite. However, recent work has provided variance expressions that are exact for finite (possibly low) model order, and hence can potentially deliver more accurate quantification of estimation accuracy. This paper, and a companion one, revisits the study of identification from closed-loop data in light of these new quantifications and establishes that, under certain assumptions, there can be significant differences in the accuracy of frequency response estimates. These discrepencies are established here and in the companion paper to be dependent on what type of direct, indirect or joint input-output identification strategy is pursued.     

一类线性时变系统组合自适应迭代学习辨识

针对一类在有限时间区间上可重复运行的既含时变参数又含时不变参数的高阶线性时变系统,提出了一种模型参考组合自适应迭代学习参数辨识算法.应用Lyapunov方法,给出了时不变参数的时域自适应学习律和时变参数的迭代域自适应学习律,分析了参数估计和模型状态跟踪误差的有界性与收敛性.该算法适于时变和时不变参数并存的线性系统的参数辨识,可加快参数估计的收敛速度.仿真例子验证了所提出的辨识算法的有效性.     

Robust maximum-likelihood estimation of multivariable dynamic systems

This paper examines the problem of estimating linear time-invariant state-space system models. In particular, it addresses the parametrization and numerical robustness concerns that arise in the multivariable case. These difficulties are well recognised in the literature, resulting (for example) in extensive study of subspace-based techniques, as well as recent interest in ‘data driven’ local co-ordinate approaches to gradient search solutions. The paper here proposes a different strategy that employs the expectation-maximisation (EM) technique. The consequence is an algorithm that is iterative, with associated likelihood values that are locally convergent to stationary points of the (Gaussian) likelihood function. Furthermore, theoretical and empirical evidence presented here establishes additional attractive properties such as numerical robustness, avoidance of difficult parametrization choices, the ability to naturally and easily estimate non-zero initial conditions, and moderate computational cost. Moreover, since the methods here are maximum-likelihood based, they have associated known and asymptotically optimal statistical properties.     

A frequency response function for linear,time-varying systems

It is well known that the steady-state response of a linear, time-invariant, finite-dimensional, exponentially stable system to a periodic input signal results, after a phase shift, in a periodic output signal of the same period with amplitude equal to the rescaling of the input amplitude by the modulus of the value of the transfer function at the given frequency. Moreover, the phase shift of the output signal is equal to the phase of the value of the transfer function at the given frequency. For this reason the transfer function is also referred to as the frequency response function. We present an analogue of this idea for linear, finite-dimensional, time-varying systems, in both the continuous- and discrete-time settings. The problem of constructing a time-varying system which associates a given output signal to each complex exponential input signal in a prescribed set can be posed as a modeling question. This leads to a new modeling interpretation for some of the time-varying interpolation problems which have recently been studied in the literature and a new motivation for the study of point evaluation for triangular operators recently introduced by Alpay, Dewilde, and Dym and by the authors for the continuous-time case.The first author was partially supported by NSF Grant 9500912.     

Direct pole placement adaptive tracking of non-minimum phase systems

In this paper, we propose a direct pole placement adaptive tracking scheme for non-minimum-phase, open-loop stable, linear plants with time delays. This controller utilizes the internal model principle to eliminate steady-state tracking error for signals with known distinct frequencies. The controller order depends only on the number of frequencies in the reference input, but not on the order of the plant. It is shown that with sufficiently small loop gain, the controller can guarantee stable closed loop, and asymptotic tracking.     

Adaptive parameter identification of linear SISO systems with unknown time-delay

An adaptive online parameter identification is proposed for linear single-input-single-output (SISO) time-delay systems to simultaneously estimate the unknown time-delay and other parameters. After representing the system as a parameterized form, a novel adaptive law is developed, which is driven by appropriate parameter estimation error information. Consequently, the identification error convergence can be proved under the conventional persistent excitation (PE) condition, which can be online tested in this paper. A finite-time (FT) identification scheme is further studied by incorporating the sliding mode scheme into the adaptation to achieve FT error convergence. The previously imposed constraint on the system relative degree is removed and the derivatives of the input and output are not required. Comparative simulation examples are provided to demonstrate the validity and efficacy of the proposed algorithms.     

基于观测器的非线性时变时滞系统自适应重复控制

针对一类未知时变时滞非线性系统,提出一种基于观测器的重复控制方案．采用线性矩阵不等式设计非线性观测器,所设计的控制律含有ＰＩＤ 反馈项,常值参数自适应律是微分差分型的,时变参数学习律是差分型的．在假设未知时变时滞、时变参数和参考输出的周期有已知的最小公倍数下,通过构造一个Ｌｙaｐｕｎｏｖ-Ｋｒａｓｏｖｓｋｉｉ型复合能量函数,证明了所有闭环信号有界且输出跟踪误差收敛．仿真实例表明了算法的有效性．     

Non-stationary stochastic embedding for transfer function estimation

This paper presents a consistent framework for the quantification of noise and undermodelling errors in transfer function model estimation. We use the, so-called, “stochastic embedding” approach, in which both noise and undermodelling errors are treated as stochastic processes. In contrast to previous applications of stochastic embedding, in this paper we represent the undermodelling as a multiplicative error characterised by random walk processes in the frequency domain. The benefit of the present formulation is that it significantly simplifies the estimation of the parameters of the embedded process yielding a closed-form expression for the model error quantification. Simulation and experimental examples illustrate how the random walks effectively capture typical cases of undermodelling found in practice, including underdamped modes. The examples also show how to use the method as a tool in the determination of model order and pole location in fixed denominator model structures.