Iterative and recursive least squares estimation algorithms for moving average systems

An iterative least squares algorithm and a recursive least squares algorithms are developed for estimating the parameters of moving average systems. The key is use the least squares principle and to replace the unmeasurable noise terms in the information vector. The steps and flowcharts of computing the parameter estimates are given. The simulation results validate that the proposed algorithms can work well.     

Simplified neural networks for solving linear least squares andtotal least squares problems in real time

In this paper a new class of simplified low-cost analog artificial neural networks with on chip adaptive learning algorithms are proposed for solving linear systems of algebraic equations in real time. The proposed learning algorithms for linear least squares (LS), total least squares (TLS) and data least squares (DLS) problems can be considered as modifications and extensions of well known algorithms: the row-action projection-Kaczmarz algorithm and/or the LMS (Adaline) Widrow-Hoff algorithms. The algorithms can be applied to any problem which can be formulated as a linear regression problem. The correctness and high performance of the proposed neural networks are illustrated by extensive computer simulation results.     

Accelerated training algorithm for feedforward neural networks based on least squares method

A least squares based training algorithm for feedforward neural networks is presented. By decomposing each neuron of the network into a linear part and a nonlinear part, the learning error can then be minimized on each neuron by applying the least squares method to solve the linear part of the neuron. In all the problems investigated, the proposed algorithm is capable of achieving the required error level in one training iteration. Comparing to the conventional backpropagation algorithm and other fast training algorithms, the proposed training algorithm provides a major breakthrough in speeding up the training process.     

Two regularizers for recursive least squared algorithms infeedforward multilayered neural networks

Recursive least squares (RLS)-based algorithms are a class of fast online training algorithms for feedforward multilayered neural networks (FMNNs). Though the standard RLS algorithm has an implicit weight decay term in its energy function, the weight decay effect decreases linearly as the number of learning epochs increases, thus rendering a diminishing weight decay effect as training progresses. In this paper, we derive two modified RLS algorithms to tackle this problem. In the first algorithm, namely, the true weight decay RLS (TWDRLS) algorithm, we consider a modified energy function whereby the weight decay effect remains constant, irrespective of the number of learning epochs. The second version, the input perturbation RLS (IPRLS) algorithm, is derived by requiring robustness in its prediction performance to input perturbations. Simulation results show that both algorithms improve the generalization capability of the trained network.     

On least squares algorithms for system parameter identification

A new least squares solution for obtaining asymptotically unbiased and consistent estimates of unknown parameters in noisy linear systems is presented. The proposed algorithms are in many ways more advantageous than generalized least squares algorithm. Extensions to on-line and multivariable problems can be easily implemented. Examples are given to illustrate the performance of these new algorithms.     

Pure time updates for least squares lattice algorithms

This note presents some new time update formulas for certain types of lattice algorithms used in autoregressive modeling of stationary time series. The new formulas enable the computation of the autoregressive coefficients in a number of operations per time step proportional to the model order.     

Sequential weighted least squares algorithm for PET image reconstruction

The weighted least squares (WLS) algorithm has proven useful for modern positron emission tomography (PET) scanners to approach reconstructions with non-Poisson precorrected measurement data. In this paper, we propose a new time recursive sequential WLS algorithm whose derivation uses the time-varying property of data acquisition of PET scanning. It ties close relationship with the time-varying Kalman filtering and can be extended appropriately to an iteration fashion as the absence of proper a priori initializations. The performance of sequential WLS is evaluated experimentally. The results show its fast convergence over both the multiplicative and coordinate-based iterative WLS methods. It also produces relative uniform estimate variances that makes it more suitable for routine applications.     

Determining initial weights of feedforward neural networks based on least squares method

An algorithm for determining the optimal initial weights of feedforward neural networks based on linear algebraic methods is presented. With the optimal initial weights, the initial network error is enormous smaller. In one of the examples presented in this letter, the achieved accuracy is sufficient for direct application. If further smaller network error is required, the networks can be trained using backpropagation algorithm.     

带有稳定学习的递归神经网络动态偏最小二乘建模

针对目前非线性动态偏最小二乘(PLS)建模方法在拟合较强非线性化工过程时存在的问题, 提出一种基于稳定学习的递归神经网络动态PLS建模方法. 该算法将递归神经网络与Hammerstein模型相结合, 对外部PLS提取的特征向量进行内部建模, 具有逼近较强非线性化工过程的能力, 改善了模型的适用范围. 此外, 采用带有稳定学习的参数更新算法对模型参数进行在线修正, 改善了模型的预测精度和自适应能力. 将此方法应用于氧化铝生产过程铝酸钠溶液组分浓度建模实验, 仿真结果表明, 本方法是可行有效的.     

Edge-preserving image decomposition via joint weighted least squares

Computational Visual Media - Recent years have witnessed the emergence of image decomposition techniques which effectively separate an image into a piecewise smooth base layer and several residual...     

Data filtering based least squares algorithms for multivariable CARAR-like systems

This paper focuses on the identification problem of multivariable controlled autoregressive autoregressive (CARAR-like) systems. The corresponding identification model contains a parameter vector and a parameter matrix, and thus the conventional least squares methods cannot be applied to directly estimate the parameters of the systems. By using the hierarchical identification principle, this paper presents a hierarchical generalized least squares algorithm and a filtering based hierarchical least squares algorithm for the multivariable CARAR-like systems. The simulation results show that the two hierarchical least squares algorithms are effective.     

Computational experience with algorithms for separable nonlinear least squares problems

Nonlinear least squares problems frequently arise in which the fitting function can be written as a linear combination of functions involving further parameters in a nonlinear manner. This paper outlines an efficient implementation of an iterative procedure originally developed by Golub and Pereyra and successively modified by various authors, which takes advantage of the linear-nonlinear structure, and investigates its performances on various test problems as compared with the standard Gauss-Newton and Gauss-Newton-Marquardt schemes. A preliminary version of this note has been presented at the CNR-GNIM meeting held in Florence, september 1976.     

Algebraic algorithms for least squares problem in quaternionic quantum theory

Quaternionic least squares (QLS) problem is one method of solving overdetermined sets of quaternion linear equations AX≈B that is appropriate when there is error in the matrix B. In this paper, by means of complex representation of a quaternion matrix, we introduce a concept of norm of quaternion matrices, discuss singular values and generalized inverses of a quaternion matrix, study the QLS problem and derive two algebraic methods for finding solutions of the QLS problem in quaternionic quantum theory.     

On strong consistency of least squares identification algorithms

In this paper almost sure convergence results are derived for least squares identification algorithms. The convergence conditions expressed in terms of the measurable signal model states derived for asymptotically stable signal models and possibly nonstationary processes are in essence the same as those previously given, but are derived more directly. Strong consistency results are derived for the case of signal models with unstable modes and exponential rates of convergence to the unstable modes are demonstrated. These latter convergence results are stronger than those earlier ones in which weak consistency conditions are given and there is also less restriction on the noise disturbances than in earlier theories. The derivations in the paper appeal to martingale convergence theorems and the Toeplitz lemma.     

Orthogonal least squares learning algorithm for radial basisfunction networks

The radial basis function network offers a viable alternative to the two-layer neural network in many applications of signal processing. A common learning algorithm for radial basis function networks is based on first choosing randomly some data points as radial basis function centers and then using singular-value decomposition to solve for the weights of the network. Such a procedure has several drawbacks, and, in particular, an arbitrary selection of centers is clearly unsatisfactory. The authors propose an alternative learning procedure based on the orthogonal least-squares method. The procedure chooses radial basis function centers one by one in a rational way until an adequate network has been constructed. In the algorithm, each selected center maximizes the increment to the explained variance or energy of the desired output and does not suffer numerical ill-conditioning problems. The orthogonal least-squares learning strategy provides a simple and efficient means for fitting radial basis function networks. This is illustrated using examples taken from two different signal processing applications.     

Minimax classifiers based on neural networks

The problem of designing a classifier when prior probabilities are not known or are not representative of the underlying data distribution is discussed in this paper. Traditional learning approaches based on the assumption that class priors are stationary lead to sub-optimal solutions if there is a mismatch between training and future (real) priors. To protect against this uncertainty, a minimax approach may be desirable. We address the problem of designing a neural-based minimax classifier and propose two different algorithms: a learning rate scaling algorithm and a gradient-based algorithm. Experimental results show that both succeed in finding the minimax solution and it is also pointed out the differences between common approaches to cope with this uncertainty in priors and the minimax classifier.     

Use of weighted least squares for geo-positioning using dual-satellite image pairs

This study investigates the use of weighted least squares (WLSs) estimation for geo-positioning using dual-satellite image pairs. Although many believe that the WLS method may be the optimal method for handling such pairs composed of different-resolution images, this study reveals that it has not been thoroughly validated using real satellite data and has obvious limitations for the space intersection of dual-satellite images, despite its potential. In addition, this article addresses the fact that the positioning accuracy may depend on the geometric conditions, as well as the resolution. This study visually and quantitatively checked the effect of the WLS method on the positioning accuracy using all the dual-satellite pairs available from two KOMPSAT-2, IKONOS, and QuickBird images. The results reveal that the WLS method is very effective for horizontal mapping. This observation was also valid in principle even for three rays, with a higher-resolution single image integrated into an existing stereo pair. In particular, it tends to cause a larger improvement when the pairs form a weaker geometry, which increases the spatial uncertainty near the intersection point. However, the WLS method did not improve the vertical mapping. It yielded a lower accuracy than the conventional approach, particularly in the case of three rays, owing to the weak convergence geometry created by integrating different-satellite images. This indicates that the WLS method must be conditionally accepted for mapping using dual-satellite images. This article demonstrates the potential and limitations of WLS estimation for finding the intersection of dual-satellite image pairs theoretically and experimentally, as well as its effects associated with the geometric conditions of imaging, visually and quantitatively.     

基于最小二乘优化的加权DV-Hop改进算法

针对传统DV-Hop算法定位精度差的问题,加权DV-Hop算法优化了待计算节点的平均单跳距离。在存在GPS定位误差的情况下,对加权DV-Hop算法进行了改进,利用最小二乘法优化全网信标节点的平均单跳距离,利用二次曲线算法代替三边测量法。随机单次仿真的平均定位误差较传统算法降低13.01%,较加权DV-Hop算法降低8.94%,重复实验仿真结果同样表明算法精度、稳定性有显著提高。     

损失数据线性参数系统的递推最小二乘辨识方法

针对损失数据线性参数系统的参数辨识问题, 借助辅助模型辨识思想推导出其变递推间隔辅助模型递 推最小二乘算法.为了提高该算法的计算效率, 利用分解技术得到变递推间隔分解递推最小二乘算法 估计系统参数.此外, 在变递推间隔分解递推最小二乘算法中引入遗忘因子, 从而提高参数估计精度和收敛速度.仿真结果表明, 所提出的算法能有效估计系统参数.