A robust recursive least squares algorithm

A new algorithm is developed, which guarantees the normalized bias in the weight vector due to persistent and bounded data perturbations to be bounded. Robustness analysis for this algorithm has been presented. An approximate recursive implementation is also proposed. It is termed as the robust recursive least squares (RRLS) algorithm since it resembles the RLS algorithm in its structure and is robust with respect to persistent bounded data perturbation. Simulation results are presented to illustrate the efficacy of the RRLS algorithm     

A linear systolic array for recursive least squares

Classical systolic design procedures rely on linear or affine space-time transformations because of the well-understood properties of linear operations. In order to increase the efficiency of the final processor, various ad hoc manipulations applied to transformations that appeared to be nonlinear at the physical array level have been proposed. Folding is one of these possible transformations. The authors show that folding can actually be considered to be an overall linear procedure by artificially increasing the dimensionality of the dependence graph of the algorithm. A 1-D array for recursive least squares is also derived as an application of a systematic linear design procedure including folding     

一种快速递归全局最小二乘算法

针对FIR系统输入和输出信号均被噪声干扰的情况,提出一种快速递归全局最小二乘(XS-RTLS)算法用于迭代计算全局最小二乘解,算法沿着输入数据的符号方向并采用著名的快速增益矢量,搜索约束瑞利商(c-RQ)的最小值得到系统参数估计。算法关于方向更新矢量的内积运算可通过加减运算实现,有效降低了计算复杂度;另外XS-RTLS算法没有进行相关矩阵求逆递归运算,因而具有长期稳定性,算法的全局收敛性通过Laslle不变性原理得到证明。最后通过仿真比较了XS-RTLS算法和递归最小二乘(RLS)算法在非时变系统和时变系统中的性能,验证了XS-RTLS算法的长期稳定性。     

A novel approach for stabilizing recursive least squares filters

A novel approach for stabilizing recursive least squares (RLS) filters is presented. The approach relies on a detailed fixed point analysis, which provides two important benefits. The analysis reveals a bias in the error propagation mechanism, providing an analytical basis for instability problems. The analysis then indicates which specific roundoff errors are causing instability. These roundoff errors are then biased in such a way that the overall filter is biased towards stable performance. Experimental results indicate that stability can be achieved with negligible loss in least squares performance     

A two-dimensional fast lattice recursive least squares algorithm

This paper is mainly devoted to the derivation of a new two-dimensional fast lattice recursive least squares (2D FLRLS) algorithm. This algorithm updates the filter coefficients in growing-order form with linear computational complexity. After appropriately defining the “order” of 2D data and exploiting the relation with 1D multichannel, “order” recursion relations and shift invariance property are derived. The geometrical approaches of the vector space and the orthogonal projection then can be used for solving this 2D prediction problem. We examine the performances of this new algorithm in comparison with other fast algorithms     

A novel fault tolerance technique for recursive least squares minimization

Existing fault tolerance schemes have often been ignored by systolic array designers because they are too costly and unwieldy to implement. With this in mind, we have developed a new technique specially tailored for recursive least squares minimization that emphasizes simplicity. We propose a new decoding scheme that allows for error detection while wasting no precious processor cycles and preserving the basic structure of the systolic array. We will show that errors can be detected by examining a single scalar. The technique can be implemented with negligible algorithmic modification and little additional hardware. The simplicity of our method invites its use in future systolic arrays.     

基于相关函数的递推最小二乘算法及其在回波消除中的应用

本文给出一种新的类似于RLS（recursive least squares)算法的递推最小二乘算法，该算法直接对输入信号的相关函数进行处理而不是对输入信号本身进行处理，理论分析表明了该算法的收敛性。该算法应用于回波消除问题中，克服了常规自适应滤波算法在出现双方对讲的情况下需停止调节自适应滤波器系数这一不足。计算机模拟仿真表明该算法在双方对讲的情况下有良好的收敛性能。     

Regularized fast recursive least squares algorithms for adaptivefiltering

Fast recursive least squares (FRLS) algorithms are developed by using factorization techniques which represent an alternative way to the geometrical projections approach or the matrix-partitioning-based derivations. The estimation problem is formulated within a regularization approach, and priors are used to achieve a regularized solution which presents better numerical stability properties than the conventional least squares one. The numerical complexity of the presented algorithms is explicitly related to the displacement rank of the a priori covariance matrix of the solution. It then varies between O(5m) and that of the slow RLS algorithms to update the Kalman gain vector, m being the order of the solution. An important advantage of the algorithms is that they admit a unified formulation such that the same equations may equally treat the prewindowed and the covariance cases independently from the used priors. The difference lies only in the involved numerical complexity, which is modified through a change of the dimensions of the intervening variables. Simulation results are given to illustrate the performances of these algorithms     

A fast recursive total least squares algorithm for adaptive IIR filtering

This work develops a new fast recursive total least squares (N-RTLS) algorithm to recursively compute the total least squares (TLS) solution for adaptive infinite-impulse-response (IIR) filtering. The new algorithm is based on the minimization of the constraint Rayleigh quotient in which the first entry of the parameter vector is fixed to the negative one. The highly computational efficiency of the proposed algorithm depends on the efficient computation of the gain vector and the adaptation of the Rayleigh quotient. Using the shift structure of the input data vectors, a fast algorithm for computing the gain vector is established, which is referred to as the fast gain vector (FGV) algorithm. The computational load of the FGV algorithm is smaller than that of the fast Kalman algorithm. Moreover, the new algorithm is numerically stable since it does not use the well-known matrix inversion lemma. The computational complexity of the new algorithm per iteration is also O(L). The global convergence of the new algorithm is studied. The performances of the relevant algorithms are compared via simulations.     

A square root covariance algorithm for constrained recursive least squares estimation

In this contribution, a covariance counterpart is described of the information matrix approach to constrained recursive least squares estimation. Unlike information-type algorithms, covariance algorithms are amenable to parallel implementation, e.g., on processor arrays, and this is also demonstrated. As compared to previously described combined covariance-information algorithms/arrays, the present implementation avoids a doubling of the hardware requirement, and therefore constitutes a significant improvement over these combined implementations as well.     

用于波束形成的最小二乘广义模值算法研究

对用于波束形成的最小二乘广义模值算法(LSGMA)在多种信号环境下的收敛性能进行了分析;在此基础上提出一种新的多用户盲波束形成算法--迭代最小二乘广义模投影(ILSP-GMA)算法,克服LSGMA算法当恒模干扰信号强于所需信号时会错误收敛的缺陷.仿真结果表明该算法可有效适用于多用户情况,并可获得较原迭代最小二乘投影算法(ILSP)更快的收敛速度.     

Regularized fast recursive least squares algorithms for finitememory filtering

Novel fast recursive least squares algorithms are developed for finite memory filtering, by using a sliding data window. These algorithms allow the use of statistical priors about the solution, and they maintain a balance between a priori and data information. They are well suited for computing a regularized solution, which has better numerical stability properties than the conventional least squares solution. The algorithms have a general matrix formulation, such that the same equations are suitable for the prewindowed as well as the covariance case, regardless of the a priori information used. Only the initialization step and the numerical complexity change through the dimensions of the intervening matrix variables. The lower bound of O (16m) is achieved in the prewindowed case when the estimated coefficients are assumed to be uncorrelated, m being the order of the estimated model. It is shown that a saving of 2m multiplications per recursion can always be obtained. The lower bound of the resulting numerical complexity becomes O(14m ), but then the general matrix formulation is lost     

An efficient recursive total least squares algorithm for FIRadaptive filtering

An algorithm for recursively computing the total least squares (TLS) solution to the adaptive filtering problem is described. This algorithm requires O(N) multiplications per iteration to effectively track the N-dimensional eigenvector associated with the minimum eigenvalue of an augmented sample covariance matrix. It is shown that the recursive least squares (RLS) algorithm generates biased adaptive filter coefficients when the filter input vector contains additive noise. The TLS solution on the other hand, is seen to produce unbiased solutions. Examples of standard adaptive filtering applications that result in noise being added to the adaptive filter input vector are cited. Computer simulations comparing the relative performance of RLS and recursive TLS are described     

FAST RECURSIVE LEAST SQUARES LEARNING ALGORITHM FOR PRINCIPAL COMPONENT ANALYSIS

Based on the least-square minimization a computationally efficient learning algorithm for the Principal Component Analysis(PCA) is derived. The dual learning rate parameters are adaptively introduced to make the proposed algorithm providing the capability of the fast convergence and high accuracy for extracting all the principal components. It is shown that all the information needed for PCA can be completely represented by the unnormalized weight vector which is updated based only on the corresponding neuron input-output product. The convergence performance of the proposed algorithm is briefly analyzed.The relation between Oja's rule and the least squares learning rule is also established. Finally, a simulation example is given to illustrate the effectiveness of this algorithm for PCA.     

A new recursive pseudo least squares algorithm for ARMA filteringand modeling. I

This study is based on the observation that if the bootstrapping is combined with different parameterizations of the ARMA (autoregressive moving average) process, then different linearized problems are obtained for the underlying nonlinear ARMA modeling problem. In this part, a specific parameterization termed the predictor space representation for an ARMA process, which decouples the estimation for the AR and the MA parameters, is used. A vector space formalism for the given data case is then defined, and the least-squares ARMA filtering problem is interpreted in terms of projection operations on some linear spaces. A new projection operator update formula, which is particularly suited for the underlying problem, is then used in conjunction with the vector space formalism to develop a computationally efficient pseudo-least-squares algorithm for ARMA filtering. It is noted that these recursions can be put in the form of a filter structure     

A new recursive pseudo least squares algorithm for ARMA filteringand modeling. II

For pt.I see ibid., vol.40, no.11, p.2766-74 (Nov. 1992). A recursive algorithm for ARMA (autoregressive moving average) filtering has been developed in a companion paper. These recursions are seen to have a lattice-like filter structure. The ARMA parameters, however, are not directly available from the coefficients of this filter. The problem of identification of the ARMA model from the coefficients of this filter is addressed here. Two new update relations for certain pseudoinverses are derived and used to obtain a recursive least squares algorithm for AR parameter estimation. Two methods for the estimation of the MA parameters are also presented. Numerical results demonstrate the usefulness of the proposed algorithms     

基于递推最小二乘算法的小信号检测

受到强干扰影响的小信号通常难于有效检测。在分析递推最小二乘算法(RLS)原理及其几种改进形式的基础上,采用自适应方法将已检测出的大信号与原混叠信号对消,降低大信号对小信号的遮蔽作用,再进行小信号的检测。最后通过仿真证明,该方法能够在较小失真的情况下,有效检测出被大调幅信号干扰下的小调频信号;同时分别比较了各种算法的优劣,得出基于可变遗忘因子的RLS(VFF-RLS)算法不仅具有较快的收敛速度,而且收敛之后具有很好的平稳性能。     

Selfperturbing recursive least squares algorithm with fast tracking capability

A novel recursive least squares (RLS) type algorithm with a selfperturbing action is devised. The algorithm possesses a fast tracking capability in itself because its adaptation gain is automatically revitalised through perturbation of the covariance update dynamics by the filter output error square when it encounters sudden parameter changes. Furthermore, the algorithm converges to the true parameter values in stationary environments.<>     

Multichannel recursive least squares adaptive filtering without adesired signal

The author presents a pair of adaptive QR decomposition-based algorithms for the adaptive mixed filter in which no desired signal is available, but the signal-to-data cross-correlation vector is known. The algorithms are derived by formulating the recursive mixed filter as a least-squares problem and then applying orthogonal QR-based techniques in its solution. This leads to algorithms with the performance, numerical, and structural advantages of the RLS/ QR algorithm, but without the requirement of a desired signal. Both Givens and square-root-free Givens rotations are used in implementing the recursive QR decomposition. Because of their structural regularity, the algorithms are easily implemented by triangular systolic array structures. Simulations show that these algorithms require fewer computations and less precision than recursive sample matrix inversion approaches     

Weighted least squares implementation of Cohen-Posch time-frequencydistributions

We present an improvement of the least-squares method of Sang et al. (see Proc. IEEE-SP Int. Symp. Time-Freq./Time-Scale Anal., p.165-8, 1996) for constructing nonnegative joint time-frequency distributions (TFDs) satisfying the time and frequency marginals (i.e., Cohen-Posch (1985) distributions). The proposed technique is a positivity constrained iterative weighted least-squares (WLS) algorithm used to modify an initial TFD (e.g., any bilinear TFD) to obtain a Cohen-Posch TFD. The new algorithm solves the “leakage” problem of the least-squares approach and is computationaly faster. Examples illustrating the performance of the new algorithm are presented. The results for the WLS method compare favorably with the minimum cross-entropy method previously developed by Loughlin et al. (1992)