Regularized total least squares approach for nonconvolutionallinear inverse problems

In this correspondence, a solution is developed for the regularized total least squares (RTLS) estimate in linear inverse problems where the linear operator is nonconvolutional. Our approach is based on a Rayleigh quotient (RQ) formulation of the TLS problem, and we accomplish regularization by modifying the RQ function to enforce a smooth solution. A conjugate gradient algorithm is used to minimize the modified RQ function. As an example, the proposed approach has been applied to the perturbation equation encountered in optical tomography. Simulation results show that this method provides more stable and accurate solutions than the regularized least squares and a previously reported total least squares approach, also based on the RQ formulation     

Iterative least squares estimators in nonlinear image restoration

The concept of iterative least squares estimation as applied to nonlinear image restoration is considered. Regarding the convergence analysis of nonlinear iterative algorithms, the potential of the global convergence theorem (GCT) is explored. The theoretical analysis is performed on a general class of nonlinear algorithms, which defines a signal-dependent linear mapping of the residual. The descent properties of two normed functions are considered. Furthermore, a procedure for the selection of the iteration parameter is introduced. The steepest descent (SD) iterative approach for the solution of the least squares optimization problem is introduced. The convergence properties of the particular algorithm are readily derived on the basis of the generalized analysis and the GCT. The factors that affect the convergence rate of the SD algorithm are thoroughly studied. In the case of the SD algorithm, structural modifications are proposed, and two hybrid-SD algorithms attain convergence in a more uniform fashion with respect to their entries. In general, the algorithms achieve larger convergence rates than the conventional SD technique     

Total least mean squares algorithm

Widrow (1971) proposed the least mean squares (LMS) algorithm, which has been extensively applied in adaptive signal processing and adaptive control. The LMS algorithm is based on the minimum mean squares error. On the basis of the total least mean squares error or the minimum Raleigh quotient, we propose the total least mean squares (TLMS) algorithm. The paper gives the statistical analysis for this algorithm, studies the global asymptotic convergence of this algorithm by an equivalent energy function, and evaluates the performances of this algorithm via computer simulations     

A weighted least squares algorithm for quasi-equiripple FIR and IIRdigital filter design

It has been demonstrated by several authors that if a suitable frequency response weighting function is used in the design of a finite impulse response (FIR) filter, the weighted least squares solution is equiripple. The crux of the problem lies in the determination of the necessary least squares frequency response weighting function. A novel iterative algorithm for deriving the least squares frequency response weighting function which will produce a quasi-equiripple design is presented. The algorithm converges very rapidly. It typically produces a design which is only about 1 dB away from the minimax optimum solution in two iterations and converges to within 0.1 dB in six iterations. Convergence speed is independent of the order of the filter. It can be used to design filters with arbitrarily prescribed phase and amplitude response     

Adaptive blind channel estimation by least squares smoothing

A least squares smoothing (LSS) approach is presented for the blind estimation of single-input multiple-output (SIMO) finite impulse response systems. By exploiting the isomorphic relation between the input and output subspaces, this geometrical approach identifies the channel from a specially formed least squares smoothing error of the channel output. LSS has the finite sample convergence property, i.e., in the absence of noise, the channel is estimated perfectly with only a finite number of data samples. Referred to as the adaptive least squares smoothing (A-LSS) algorithm, the adaptive implementation has a high convergence rate and low computation cost with no matrix operations. A-LSS is order recursive and is implemented in part using a lattice filter. It has the advantage that when the channel order varies, channel estimates can be obtained without structural change of the implementation. For uncorrelated input sequence, the proposed algorithm performs direct deconvolution as a by-product     

用于波束形成的最小二乘广义模值算法研究

对用于波束形成的最小二乘广义模值算法(LSGMA)在多种信号环境下的收敛性能进行了分析;在此基础上提出一种新的多用户盲波束形成算法--迭代最小二乘广义模投影(ILSP-GMA)算法,克服LSGMA算法当恒模干扰信号强于所需信号时会错误收敛的缺陷.仿真结果表明该算法可有效适用于多用户情况,并可获得较原迭代最小二乘投影算法(ILSP)更快的收敛速度.     

基于最小二乘支持向量机的移动机器人导航

针对未知环境下的移动机器人导航,提出将最小二乘支持向量机与强化学习相结合的导航方法。首先以移动机器人CASIA-I和它的工作环境为实验平台,确定出强化学习的回报函数;然后利用基于滚动窗的最小支持向量机解决强化学习中的泛化问题。最后对所提方法进行了实验,实验结果表明所提方法能够避免导航陷入局部极小,并对未知环境具有较强的适应性。     

Blind digital signal separation using successive interferencecancellation iterative least squares

Blind separation of instantaneous linear mixtures of digital signals is a basic problem in communications. When little or nothing can be assumed about the mixing matrix, signal separation may be achieved by exploiting structural properties of the transmitted signals, e.g., finite alphabet or coding constraints. We propose a monotonically convergent and computationally efficient iterative least squares (ILS) blind separation algorithm based on an optimal scaling lemma. The signal estimation step of the proposed algorithm is reminiscent of successive interference cancellation (SIC) ideas. For well-conditioned data and moderate SNR, the proposed SIC-ILS algorithm provides a better performance/complexity tradeoff than competing ILS algorithms. Coupled with blind algebraic digital signal separation methods, SIC-ILS offers a computationally inexpensive true least squares refinement option. We also point out that a widely used ILS finite alphabet blind separation algorithm can exhibit limit cycle behavior     

Recursive total least squares algorithm for image reconstruction from noisy,undersampled frames

It is shown how the efficient recursive total least squares algorithm recently developed by C.E. Davila [3] for real data can be applied to image reconstruction from noisy, undersampled multiframes when the displacement of each frame relative to a reference frame is not accurately known. To do this, the complex-valued image data in the wavenumber domain is transformed into an equivalent real data problem to which Davila's algorithm is successfully applied. Two detailed illustrative examples are provided in support of the procedure. Similar reconstruction in the presence of blur as well as noise is currently under investigation.     

Recursive total least squares algorithm for single-user blindchannel equalisation

The problem of blind channel identification/equalisation using second-order statistics or equivalent deterministic properties of the oversampled channel output has attracted considerable attention. Deterministic blind subspace algorithms are particularly attractive because of their finite sample convergence property and because their solution can be obtained in closed form. Most subspace algorithms developed up until now, however, are based on block processing and have high computational and memory requirements. In the paper, adaptive techniques are used to lower the computational burden. A single-user direct symbol estimation algorithm is presented. The first step in the algorithm consists of an adaptive matrix singular value decomposition for a (virtual) channel identification-type operation. A recursive total least squares algorithm is then used to recover the input symbols. The algorithm is able to track time-varying channels     

A recursive least squares implementation for LCMP beamforming underquadratic constraint

Quadratic constraints on the weight vector of an adaptive linearly constrained minimum power (LCMP) beamformer can improve robustness to pointing errors and to random perturbations in sensor parameters. We propose a technique for implementing a quadratic inequality constraint with recursive least squares (RLS) updating. A variable diagonal loading term is added at each step, where the amount of loading has a closed-form solution. Simulations under different scenarios demonstrate that this algorithm has better interference suppression than both the RLS beamformer with no quadratic constraint and the RLS beamformer using the scaled projection technique, as well as faster convergence than LMS beamformers     

A constrained weighted least squares approach for time-frequencydistribution kernel design

In most applications of time-frequency (t-f) distributions, the t-f kernel is of finite extent and applied to discrete time signals. This paper introduces a matrix-based approach for t-f distribution kernel design. In this new approach, the optimum kernel is obtained as the solution of a linearly constrained weighted least squares minimization problem in which the kernel is vectorial and the constraints form a linear subspace. Similar to FIR temporal and spatial constrained least squares (LS) design methods, the passband, stopband, and transition band of an ideal kernel are first specified. The optimum kernel that best approximates the ideal kernel in the LS error sense, and simultaneously satisfies the multiple linear constraints, is then obtained using closed-form expressions. This proposed design method embodies a well-structured procedure for obtaining fixed and data-dependent kernels that are difficult to obtain using other design approaches     

Consistent normalized least mean square filtering with noisy data matrix

When the ordinary least squares method is applied to the parameter estimation problem with noisy data matrix, it is well-known that the estimates turn out to be biased. While this bias term can be somewhat reduced by the use of models of higher order, or by requiring a high signal-to-noise ratio (SNR), it can never be completely removed. Consistent estimates can be obtained by means of the instrumental variable method (IVM),or the total/data least squares method (TLS/DLS). In the adaptive setting for the such problem, a variety of least-mean-squares (LMS)-type algorithms have been researched rather than their recursive versions of IVM or TLS/DLS that cost considerable computations. Motivated by these observations, we propose a consistent LMS-type algorithm for the data least square estimation problem. This novel approach is based on the geometry of the mean squared error (MSE) function, rendering the step-size normalization and the heuristic filtered estimation of the noise variance, respectively, for fast convergence and robustness to stochastic noise. Monte Carlo simulations of a zero-forcing adaptive finite-impulse-response (FIR) channel equalizer demonstrate the efficacy of our algorithm.     

基于小波变换的加权最小二乘相位解缠算法

提出了一种求解加权最小二乘相位解缠问题的新算法,采用相位导数方差相关图定义了相位数据的初始权重,给出了利用多分辨率小波变换进行相位解缠的具体实现方法.利用离散小波变换,将原线性系统转化成具有较好收敛条件的等价新系统,通过独立求解新系统中的低频部分,加快了系统的收敛速度.仿真实验表明:该算法具有收敛速度快、解缠效果好等特点.     

基于相关函数的递推最小二乘算法及其在回波消除中的应用

本文给出一种新的类似于RLS（recursive least squares)算法的递推最小二乘算法，该算法直接对输入信号的相关函数进行处理而不是对输入信号本身进行处理，理论分析表明了该算法的收敛性。该算法应用于回波消除问题中，克服了常规自适应滤波算法在出现双方对讲的情况下需停止调节自适应滤波器系数这一不足。计算机模拟仿真表明该算法在双方对讲的情况下有良好的收敛性能。     

A two-dimensional fast lattice recursive least squares algorithm

This paper is mainly devoted to the derivation of a new two-dimensional fast lattice recursive least squares (2D FLRLS) algorithm. This algorithm updates the filter coefficients in growing-order form with linear computational complexity. After appropriately defining the “order” of 2D data and exploiting the relation with 1D multichannel, “order” recursion relations and shift invariance property are derived. The geometrical approaches of the vector space and the orthogonal projection then can be used for solving this 2D prediction problem. We examine the performances of this new algorithm in comparison with other fast algorithms     

Large sample properties of separable nonlinear least squares estimators

In this paper, the large sample properties of the separable nonlinear least squares algorithm are investigated. Unlike the previous results in the literature, the data are assumed to be complex valued, and the whiteness assumption on the measurement noise sequence has been relaxed. Convergence properties of the parameter estimates are established. Asymptotic accuracy analysis has been carried out, in which the assumptions used are relatively weaker than the assumptions in the previous related works. It is shown under quite general conditions that the parameter estimates are asymptotically circular. Conditions for asymptotic complex normality are also established. Next, a bound on the deviation of the asymptotic covariance matrix from the Crame/spl acute/r-Rao bound (CRB) is derived. Finally, a sufficient condition for the nonlinear least squares estimate to achieve the Crame/spl acute/r-Rao lower bound is established. The results presented in this paper are general and can be applied to any specific application where separable nonlinear least squares is employed.     

Design of impedance transformers by the method of least squares

The method of least squares is applied to the theory of small reflections of transmission lines to develop numerical algorithms for the design of stepline and tapered line impedance transformers to match two impedances over a frequency band. The transformer characteristic impedance function is expanded by polynomials, pulse functions, approximate operators, and piecewise linear functions to construct an error function for the input reflection coefficient which, after minimization, gives the line impedance and length. The computer programs could be used to design a transformer under the specified conditions and then to optimize the design under the constraints of a problem     

Givens rotation based least squares lattice and related algorithms

The author presents a general and systematic approach for deriving new LS (least squares) estimation algorithms that are based solely on Givens rotations. In particular, this approach is used to derive efficient Givens-rotation-based LS lattice algorithms-the Givens-lattice algorithms. By exploiting the relationship between the Givens algorithms and the recursive modified Gram-Schmidt algorithm, it is shown that the time and order update of any order-recursive LS estimation algorithm can be realized by employing only Givens rotations. Applying this general conclusion to LS estimation of time-series signals results in the Givens-lattice algorithms. Two Givens-lattice algorithms, one with square roots and the other without, are presented. It is shown that the Givens-lattice algorithms are computationally more efficient than the fast QR algorithm of Cioffi (1987). The derivation of other Givens rotation-based LS estimation algorithms and their systolic array implementations are discussed     

A weighted least squares approach to the design of FIR filters synthesized using the modified frequency response masking structure

This paper presents an application of the weighted least squares (WLS) method to the design of sharp linear phase finite-impulse response (FIR) digital filters synthesized using a modified frequency-response masking (FRM) structure. In our approach, the original minimax design problem is converted into a WLS problem. The WLS problem is highly nonlinear with respect to the coefficients of the filter. However, it can be decomposed into four linear least squares (LS) problems, each of which can be solved analytically. The design problem is then solved iteratively by using an alternating variable approach. The effectiveness of the method is demonstrated through solving a low-pass linear phase sharp FIR digital filter example.