An evolutionary algorithm for robust regression

A drawback of robust statistical techniques is the increased computational effort often needed as compared to non-robust methods. Particularly, robust estimators possessing the exact fit property are NP-hard to compute. This means that—under the widely believed assumption that the computational complexity classes NP and P are not equal—there is no hope to compute exact solutions for large high dimensional data sets. To tackle this problem, search heuristics are used to compute NP-hard estimators in high dimensions. A new evolutionary algorithm that is applicable to different robust estimators is presented. Further, variants of this evolutionary algorithm for selected estimators—most prominently least trimmed squares and least median of squares—are introduced and shown to outperform existing popular search heuristics in difficult data situations. The results increase the applicability of robust methods and underline the usefulness of evolutionary algorithms for computational statistics.     

不受约束的全局最优加权观测融合估计

利用矩阵满秩分解方法,基于加权最小二乘理论提出了一种不受各传感器观测阵是否相同、观测噪声是否相关约束限制的加权观测融合估计算法。证明了其估计结果每时刻恒同于集中式融合Kalman估计结果,因而具有全局最优性,且可明显减小计算负担,便于实时应用。通过对GPS目标跟踪系统的两种方案进行仿真说明了它的功能等价性、快速性以及最优性。     

Weighted and robust archetypal analysis

Archetypal analysis represents observations in a multivariate data set as convex combinations of a few extremal points lying on the boundary of the convex hull. Data points which vary from the majority have great influence on the solution; in fact one outlier can break down the archetype solution. The original algorithm is adapted to be a robust M-estimator and an iteratively reweighted least squares fitting algorithm is presented. As a required first step, the weighted archetypal problem is formulated and solved. The algorithm is demonstrated using an artificial example, a real world example and a detailed simulation study.     

Some edge correction methods for marked spatio-temporal point process models

Three edge correction methods for (marked) spatio-temporal point processes are proposed. They are all based on the idea of placing an approximated expected behaviour of the process at hand (simulated realisations) outside the study region which interacts with the data during the estimation. These methods are applied to the so-called growth-interaction model. The specific choices of growth function and interaction function made are purely motivated by the forestry applications considered. The parameters of the growth and interaction functions, i.e. the parameters related to the development of the marks, are estimated using the least-squares approach together with the proposed edge corrections. Finally, the edge corrected estimation methods are applied to a data set of Swedish Scots pine.     

On a fast, robust estimator of the mode: Comparisons to other robust estimators with applications

Advances in computing power enable more widespread use of the mode, which is a natural measure of central tendency since it is not influenced by the tails in the distribution. The properties of the half-sample mode, which is a simple and fast estimator of the mode of a continuous distribution, are studied. The half-sample mode is less sensitive to outliers than most other estimators of location, including many other low-bias estimators of the mode. Its breakdown point is one half, equal to that of the median. However, because of its finite rejection point, the half-sample mode is much less sensitive to outliers that are all either greater or less than the other values of the sample. This is confirmed by applying the mode estimator and the median to samples drawn from normal, lognormal, and Pareto distributions contaminated by outliers. It is also shown that the half-sample mode, in combination with a robust scale estimator, is a highly robust starting point for iterative robust location estimators such as Huber's M-estimator. The half-sample mode can easily be generalized to modal intervals containing more or less than half of the sample. An application of such an estimator to the finding of collision points in high-energy proton–proton interactions is presented.     

Extended Kalman filtering and weighted least squares dynamic identification of robot

This paper presents an experimental comparison between the weighted least squares (WLS) estimation and the extended Kalman filtering (EKF) methods for robot dynamic identification. Comparative results and discussion are presented for a SCARA robot, depending on a priori knowledge and data filtering.     

Asymptotic properties of least squares estimation with fuzzy observations

This paper deals with the asymptotic properties of the least squares estimators for fuzzy linear regression models with fuzzy triangular input-output and random error terms. The asymptotic normality and strong consistency of the fuzzy least squares estimator (FLSE) are investigated; a confidence region based on a class of FLSEs is proposed; the asymptotic relative efficiency of FLSEs with respect to the crisp least squares estimators is also provided and a numerical example is given. Some simulation results are also presented to illustrate the behavior of FLSEs.     

Dynamic consensus estimation of weighted average on directed graphs

Recent applications call for distributed weighted average estimation over sensor networks, where sensor measurement accuracy or environmental conditions need to be taken into consideration in the final consensused group decision. In this paper, we propose new dynamic consensus filter design to distributed estimate weighted average of sensors’ inputs on directed graphs. Based on recent advances in the filed, we modify the existing proportional-integral consensus filter protocol to remove the requirement of bi-directional gain exchange between neighbouring sensors, so that the algorithm works for directed graphs where bi-directional communications are not possible. To compensate for the asymmetric structure of the system introduced by such a removal, sufficient gain conditions are obtained for the filter protocols to guarantee the convergence. It is rigorously proved that the proposed filter protocol converges to the weighted average of constant inputs asymptotically, and to the weighted average of time-varying inputs with a bounded error. Simulations verify the effectiveness of the proposed protocols.     

AREION: Software effort estimation based on multiple regressions with adaptive recursive data partitioning

ContextAlong with expert judgment, analogy-based estimation, and algorithmic methods (such as Function point analysis and COCOMO), Least Squares Regression (LSR) has been one of the most commonly studied software effort estimation methods. However, an effort estimation model using LSR, a single LSR model, is highly affected by the data distribution. Specifically, if the data set is scattered and the data do not sit closely on the single LSR model line (do not closely map to a linear structure) then the model usually shows poor performance. In order to overcome this drawback of the LSR model, a data partitioning-based approach can be considered as one of the solutions to alleviate the effect of data distribution. Even though clustering-based approaches have been introduced, they still have potential problems to provide accurate and stable effort estimates.ObjectiveIn this paper, we propose a new data partitioning-based approach to achieve more accurate and stable effort estimates via LSR. This approach also provides an effort prediction interval that is useful to describe the uncertainty of the estimates.MethodEmpirical experiments are performed to evaluate the performance of the proposed approach by comparing with the basic LSR approach and clustering-based approaches, based on industrial data sets (two subsets of the ISBSG (Release 9) data set and one industrial data set collected from a banking institution).ResultsThe experimental results show that the proposed approach not only improves the accuracy of effort estimation more significantly than that of other approaches, but it also achieves robust and stable results according to the degree of data partitioning.ConclusionCompared with the other considered approaches, the proposed approach shows a superior performance by alleviating the effect of data distribution that is a major practical issue in software effort estimation.     

Multivariate discount weighted regression and local level models

The technique of multivariate discount weighted regression is used for forecasting multivariate time series. In particular, the discount regression model is modified to cater for the popular local level model for predicting vector time series. The proposed methodology is illustrated with London metal exchange data consisting of aluminium spot and future contract closing prices. The estimate of the measurement noise covariance matrix suggests that these data exhibit high cross-correlation, which is discussed in some detail. The performance of the proposed model is evaluated via an error analysis based on the mean of squared forecast errors, the mean of absolute forecast errors and the mean of absolute percentage forecast errors. A sensitivity analysis shows that a low discount factor should be used and practical guidelines are given for general future use.     

Local polynomial estimation in partial linear regression models under dependence

A regression model whose regression function is the sum of a linear and a nonparametric component is presented. The design is random and the response and explanatory variables satisfy mixing conditions. A new local polynomial type estimator for the nonparametric component of the model is proposed and its asymptotic normality is obtained. Specifically, this estimator works on a prewhitening transformation of the dependent variable, and the results show that it is asymptotically more efficient than the conventional estimator (which works on the original dependent variable) when the errors of the model are autocorrelated. A simulation study and an application to a real data set give promising results.     

A geometric approach for adaptive estimation of unknown growth kinetics in bioreactors

This paper proposes a new approach for the estimation of unknown and time-varying specific growth rate in fed-batch bioprocess. A novel adaptive estimation technique based on the concept of invariant manifold is proposed as an effective approach to estimate growth kinetic parameters. An asymptotic nonlinear observer is used to provide simultaneous on-line estimation of biomass concentration and growth kinetic. The method is easy to implement and requires only one tuning parameter. The effectiveness of the proposed algorithm is illustrated with representative bioreactor simulation examples.     

标量及对角阵加权的序贯估计融合算法研究

针对多传感器分布式估计融合系统,在最小化估计误差的协方差矩阵迹的准则下,采用标量加权及对角阵加权融合方法,提出了估计误差相关条件下的序贯处理式最优估计融合Kalman滤波器。该融合滤波器以两传感器估计融合算法为基础,对传感器采集信息依次进行融合计算,得到多传感器融合结果。比较两种算法与局部滤波器的估计精度,并进行了仿真。仿真结果表明了基于加权估计融合的序贯处理算法的可行性和有效性。     

Matrix strategies for computing the least trimmed squares estimation of the general linear and SUR models

An algorithm for computing the exact least trimmed squares (LTS) estimator of the standard regression model has recently been proposed. The LTS algorithm is adapted to the general linear and seemingly unrelated regressions models with possible singular dispersion matrices. It searches through a regression tree to find the optimal estimates and has combinatorial complexity. The model is formulated as a generalized linear least squares problem. Efficient matrix techniques are employed to update the generalized residual sum of squares of a subset model. Specifically, the new algorithm utilizes previous computations to update a generalized QR decomposition by a single row. The sparse structure of the model is exploited. Theoretical measures of computational complexity are provided. Experimental results confirm the ability of the new algorithms to identify outlying observations.     

A novel robust adaptive control algorithm with finite-time online parameter estimation of a humanoid robot arm

A novel robust adaptive control algorithm is proposed and implemented in real-time on two degrees-of-freedom (DOF) of the humanoid Bristol-Elumotion-Robotic-Torso II (BERT II) arm in joint-space. In addition to having a significant robustness property for the tracking, the algorithm also features a sliding-mode term based adaptive law that captures directly the parameter estimation error. An auxiliary filtered regression vector and filtered computed torque is introduced. This allows the definition of another auxiliary matrix, a filtered regression matrix, which facilitates the introduction of a sliding mode term into the adaptation law. Parameter error convergence to zero can be guaranteed within finite-time with a Persistent-Excitation (PE) condition or Sufficient Richness condition for the demand. The proposed scheme also exhibits robustness both in the tracking and parameter estimation errors to any bounded additive disturbance. This theoretical result is then exemplified for the BERT II robot arm in simulation and for experiments.     

The theoretic framework of local weighted approximation for microarray missing value estimation

Microarray data are used in many biomedical experiments. They often contain missing values which significantly affect statistical algorithms. Although a number of imputation algorithms have been proposed, they have various limitations to exploit local and global information effectively for estimation. It is necessary to develop more effective techniques to solve the data imputation problem. In this paper, we propose a theoretic framework of local weighted approximation for missing value estimation, based on the Taylor series approximation. Besides revealing that k-nearest neighbor imputation (KNNimpute) is a special case of the framework, we focus on the study of its linear case—local weighted linear approximation imputation (LWLAimpute) from theory to experiment. Experimental results show that LWLAimpute and its iterative version can achieve better performance than some existing imputation methods, the superiority becomes more significant with increasing level of missing values.     

基于广义加权的AMDF时延估计方法研究

时延估计是声源定位常用的方法。多途效应严重影响声源信号时延估计性能,传统方法难以克服。提出一种基于广义加权的平均幅度差函数（Average Magnitude Difference Function,AMDF）时延估计方法,利用改进的AMDF方法提高对多途效应的抑制,通过广义加权方法降低算法的噪声敏感性。仿真及实验表明,对于窄带信号,该方法能够获得比传统广义互相关方法更高的时延估计性能,估计结果的误差减小,稳定性能提高。     

Spreadsheet template approach for nonlinear regression estimation

Industrial Engineers may encounter variables which are nonlinearly related such that the relationship cannot be transformed to one which is linear in the unknown parameters. One instance where this can occur is a learning curve which approaches a non-zero asymptote. This paper presents a spreadsheet template for implementing the Gauss-Newton Method for solving such nonlinear regression estimation problems.     

A continuous-time framework for least squares parameter estimation

This paper proposes a continuous-time framework for the least-squares parameter estimation method through evolution equations. Nonlinear systems in the standard state space representation that are linear in the unknown, constant parameters are investigated. Two estimators are studied. The first one consists of a linear evolution equation while the second one consists of an impulsive linear evolution equation. The paper discusses some theoretical aspects related to the proposed estimators: uniqueness of a solution and an attractive equilibrium point which solves for the unknown parameters. A deterministic framework for the estimation under noisy measurements is proposed using a Sobolev space with negative index to model the noise. The noise can be of large magnitude. Concrete signals issued from an electronic device are used to discuss numerical aspects.     

Constrained parameter estimation with applications to blending operations

The classical least squares approach to parameter estimation for dynamic models ignores a priori information about the feasible values of the estimated parameters. But in many practical problems, such information is available in the form of upper and lower limits. In this paper, two alternative techniques are evaluated for this important class of constrained parameter estimation problems for dynamic systems. Simulation results for two blending problems illustrate that more accurate parameter estimates and better predictions can be obtained by using a quadratic programming approach.