Using a spectral scaling structured BFGS method for constrained nonlinear least squares

In this paper, we use a spectral scaled structured BFGS formula for approximating projected Hessian matrices in an exact penalty approach for solving constrained nonlinear least-squares problems. We show this spectral scaling formula has a good self-correcting property. The reported numerical results show that the use of the spectral scaling structured BFGS method outperforms the standard structured BFGS method.     

An iterative algorithm for least squares problem in quaternionic quantum theory

Quaternionic least squares (QLS) problem is one method of solving overdetermined sets of quaternion linear equations AXB=E that is appropriate when there is error in the matrix E. In this paper, by means of real representation of a quaternion matrix, we introduce a concept of norm of quaternion matrices, which is different from that in [T. Jiang, L. Chen, Algebraic algorithms for least squares problem in quaternionic quantum theory, Comput. Phys. Comm. 176 (2007) 481-485; T. Jiang, M. Wei, Equality constrained least squares problem over quaternion field, Appl. Math. Lett. 16 (2003) 883-888], and derive an iterative method for finding the minimum-norm solution of the QLS problem in quaternionic quantum theory.     

The regularized least squares algorithm and the problem of learning halfspaces

We provide sample complexity of the problem of learning halfspaces with monotonic noise, using the regularized least squares algorithm in the reproducing kernel Hilbert spaces (RKHS) framework.     

Matrix LSQR algorithm for structured solutions to quaternionic least squares problem

In this paper, we employ matrix LSQR algorithm to deal with quaternionic least squares problem in order to find the minimum norm solutions with kinds of special structures, and propose a strategy to accelerate convergence rate of the algorithm via right–left preconditioning of the coefficient matrices. We mainly focus on analyzing the minimum norm $\eta$-Hermitian solution and the minimum norm $\eta$-biHermitian solution to the quaternionic least squares problem, $\eta \in \{\mathbf{i},\mathbf{j},\mathbf{k}\}$. Other structured solutions also can be obtained using the proposed technique. A number of numerical experiments are performed to show the efficiency of the preconditioned matrix LSQR algorithm.     

Efficient algorithm for a class of least squares estimationproblems

A computationally efficient algorithm for solving least squares estimation problems is proposed. It is well suited for problems with the normal equation matrix factorizable in terms of Kronecker's products. Three classes of identification problems factorizable in this sense are pointed out. Computational complexity of the algorithm and its robustness against round off errors is also discussed     

A least squares algorithm for a mixture model for compositional data

The estimation of a model for compositional data is studied where the data are approximated by a mixture of latent compositions. This model is variously known as “endmember analysis” or “latent budget analysis”. Two estimation procedures are available. The first uses a procedure which is incorrect in the sense that, although it claims to be a least squares procedure, it does not always minimize a least squares criterion. The second uses a maximum likelihood procedure starting from assumptions that are often violated for compositional data. In this paper we propose a constrained (weighted) least squares procedure for the estimation of the model.     

Implementation of regularization for separable nonlinear least squares problems

This paper considers a class of separable nonlinear least squares problems in which a model can be represented as a linear combination of nonlinear functions. A regularized nonlinear parameter optimization approach is presented for coping with the potential ill-conditioned problem of parameter divergence. Together with a regularization parameter detection technique, Tikhonov regularization and truncated singular value decomposition are utilized in the estimation of the linear parameters if the nonlinear parameters are changed during the parameter optimization process, which centers on a nonlinear parameter search using the Levenberg-Marquardt algorithm. Benefiting from the regularization in parameter optimization, the potential ill-conditioned issue can be avoided, and the multi-step-ahead forecasting accuracy of the estimated model may be largely improved. The usefulness of this approach is illustrated by means of a chaotic time-series prediction and nonlinear industrial process modeling.     

Algebraic algorithms for least squares problem in quaternionic quantum theory

Quaternionic least squares (QLS) problem is one method of solving overdetermined sets of quaternion linear equations AX≈B that is appropriate when there is error in the matrix B. In this paper, by means of complex representation of a quaternion matrix, we introduce a concept of norm of quaternion matrices, discuss singular values and generalized inverses of a quaternion matrix, study the QLS problem and derive two algebraic methods for finding solutions of the QLS problem in quaternionic quantum theory.     

An iterative algorithm for a least squares solution of a matrix equation

In this article, we propose an iterative algorithm to compute the minimum norm least-squares solution of AXB+CYD=E, based on a matrix form of the algorithm LSQR for solving the least squares problem. We then apply this algorithm to compute the minimum norm least-squares centrosymmetric solution of min _X ‖AXB?E‖ _F . Numerical results are provided to verify the efficiency of the proposed method.     

Stochastic adaptive control using a modified least squares algorithm

Recent papers on stochastic adaptive control have established global convergence for algorithms using a stochastic approximation iteration. However, to date, global convergence has not been established for algorithms incorporating a least squares iteration. This paper establishes global convergence for a slightly modified least squares stochastic adaptive control algorithm. It is shown that, with probability one, the algorithm will ensure that the system inputs and outputs are sample mean square bounded and the mean square output tracking error achieves its global minimum possible value for linear feedback control.     

Rotational fluid flow using a least squares collocation technique

A potential theory approach for incompressible viscous flow which leads to the biharmonic equation is first developed. A numerical least squares collocation technique using fundamental singular solutions of the biharmonic equation is then applied to a rotational flow problem with moving boundaries that produce discontinuous boundary conditions associated with the biharmonic. It is shown that the least squares technique smoothes out local disturbances in boundary data of the type which are likely to present difficulties to the more commonly used boundary element method. A compact computer program for the method and the results for the problem of a rectangular channel with one moving boundary are included along with an experimental verification of the results using the thin plate bending analogy.     

Robust recursive least squares learning algorithm for principalcomponent analysis

A learning algorithm for the principal component analysis (PCA) is developed based on the least-square minimization. The dual learning rate parameters are adjusted adaptively to make the proposed algorithm capable of fast convergence and high accuracy for extracting all principal components. The proposed algorithm is robust to the error accumulation existing in the sequential PCA algorithm. We show that all information needed for PCA can he completely represented by the unnormalized weight vector which is updated based only on the corresponding neuron input-output product. The updating of the normalized weight vector can be referred to as a leaky Hebb's rule. The convergence of the proposed algorithm is briefly analyzed. We also establish the relation between Oja's rule and the least squares learning rule. Finally, the simulation results are given to illustrate the effectiveness of this algorithm for PCA and tracking time-varying directions-of-arrival.     

Sequential weighted least squares algorithm for PET image reconstruction

The weighted least squares (WLS) algorithm has proven useful for modern positron emission tomography (PET) scanners to approach reconstructions with non-Poisson precorrected measurement data. In this paper, we propose a new time recursive sequential WLS algorithm whose derivation uses the time-varying property of data acquisition of PET scanning. It ties close relationship with the time-varying Kalman filtering and can be extended appropriately to an iteration fashion as the absence of proper a priori initializations. The performance of sequential WLS is evaluated experimentally. The results show its fast convergence over both the multiplicative and coordinate-based iterative WLS methods. It also produces relative uniform estimate variances that makes it more suitable for routine applications.     

Computational experience with algorithms for separable nonlinear least squares problems

Nonlinear least squares problems frequently arise in which the fitting function can be written as a linear combination of functions involving further parameters in a nonlinear manner. This paper outlines an efficient implementation of an iterative procedure originally developed by Golub and Pereyra and successively modified by various authors, which takes advantage of the linear-nonlinear structure, and investigates its performances on various test problems as compared with the standard Gauss-Newton and Gauss-Newton-Marquardt schemes. A preliminary version of this note has been presented at the CNR-GNIM meeting held in Florence, september 1976.     

Orthogonal least squares learning algorithm for radial basisfunction networks

The radial basis function network offers a viable alternative to the two-layer neural network in many applications of signal processing. A common learning algorithm for radial basis function networks is based on first choosing randomly some data points as radial basis function centers and then using singular-value decomposition to solve for the weights of the network. Such a procedure has several drawbacks, and, in particular, an arbitrary selection of centers is clearly unsatisfactory. The authors propose an alternative learning procedure based on the orthogonal least-squares method. The procedure chooses radial basis function centers one by one in a rational way until an adequate network has been constructed. In the algorithm, each selected center maximizes the increment to the explained variance or energy of the desired output and does not suffer numerical ill-conditioning problems. The orthogonal least-squares learning strategy provides a simple and efficient means for fitting radial basis function networks. This is illustrated using examples taken from two different signal processing applications.     

Limits to resolution in nonlinear least squares model fitting

This note discusses properties of least squares estimates of the parameters of weighted sum models. These are weighted sums of functions of the same nonlinearly parametric family as, for example, multiexponentials. The unknown parameters are the weights and the non-linear parameters. It is shown that errors in the observations may preclude the resolving of distinct but close nonlinear parameters. The results presented generalize earlier work in which the weights were assumed known.     

Algorithm Q-LSQR for the least squares problem in quaternionic quantum theory

In this paper, an iterative algorithm for the standard quaternionic least squares problem is proposed without using the real (complex) representation. Our algorithm is implemented in the quaternion field and by means of direct quaternion arithmetic and is a natural generalization of the LSQR algorithm for the real least squares problem.     

Design of decentralized constant-volume controllers for open-channels by solving a least squares problem

In this paper we consider a mathematical model for open-channels that expresses the dynamic relationships, in terms of transcendental functions, between the gate opening sections and the corresponding stored water volume variations in the different canal reaches with respect to an initial reference configuration of uniform flow. Series expansion around s = 0 gives a state variable linear and time invariant model as well as the corresponding rational transfer matrix. Both a proportional and a proportional integral decentralized constant-volume control law are designed by solving a linear least squares problem. Such a procedure enables us to impose the desired structure on the feedback gain matrix by means of the optimization of the controller parameters. It makes the closed loop transfer function approach a target function as closely as possible over a specified frequency range. In this paper, the target closed loop functions are designed by means of an LQR technique applied to the linear time-invariant model obtained by means of the Taylor series expansion.