Robust regression for face recognition

In this paper we address the problem of robust face recognition by formulating the pattern recognition task as a problem of robust estimation. Using a fundamental concept that in general, patterns from a single object class lie on a linear subspace (Barsi and Jacobs, 2003 [1]), we develop a linear model representing a probe image as a linear combination of class specific galleries. In the presence of noise, the well-conditioned inverse problem is solved using the robust Huber estimation and the decision is ruled in favor of the class with the minimum reconstruction error. The proposed Robust Linear Regression Classification (RLRC) algorithm is extensively evaluated for two important cases of robustness i.e. illumination variations and random pixel corruption. Illumination invariant face recognition is demonstrated on three standard databases under exemplary evaluation protocols reported in the literature. Comprehensive comparative analysis with the state-of-art illumination tolerant approaches indicates a comparable performance index for the proposed RLRC algorithm. The efficiency of the proposed approach in the presence of severe random noise is validated under several exemplary noise models such as dead-pixel problem, salt and pepper noise, speckle noise and Additive White Gaussian Noise (AWGN). The RLRC algorithm is found to be favorable compared with the benchmark generative approaches.     

Robust semi-supervised mixture probabilistic principal component regression model development and application to soft sensors

Traditional data-based soft sensors are constructed with equal numbers of input and output data samples, meanwhile, these collected process data are assumed to be clean enough and no outliers are mixed. However, such assumptions are too strict in practice. On one hand, those easily collected input variables are sometimes corrupted with outliers. On the other hand, output variables, which also called quality variables, are usually difficult to obtain. These two problems make traditional soft sensors cumbersome. To deal with both issues, in this paper, the Student's t distributions are used during mixture probabilistic principal component regression modeling to tolerate outliers with regulated heavy tails. Furthermore, a semi-supervised mechanism is incorporated into traditional probabilistic regression so as to deal with the unbalanced modeling issue. For simulation, two case studies are provided to demonstrate robustness and reliability of the new method.     

Robust diagnostics for the heteroscedastic regression model

The assumption of equal variance in the normal regression model is not always appropriate. Cook and Weisberg (1983) provide a score test to detect heteroscedasticity, while Patterson and Thompson (1971) propose the residual maximum likelihood (REML) estimation to estimate variance components in the context of an unbalanced incomplete-block design. REML is often preferred to the maximum likelihood estimation as a method of estimating covariance parameters in a linear model. However, outliers may have some effect on the estimate of the variance function. This paper incorporates the maximum trimming likelihood estimation ( [Hadi and Luce?o, 1997] and [Vandev and Neykov, 1998]) in REML to obtain a robust estimation of modelling variance heterogeneity. Both the forward search algorithm of Atkinson (1994) and the fast algorithm of Neykov et al. (2007) are employed to find the resulting estimator. Simulation and real data examples are used to illustrate the performance of the proposed approach.     

Robust regression for high throughput drug screening

Effective analysis of high throughput screening (HTS) data requires automation of dose-response curve fitting for large numbers of datasets. Datasets with outliers are not handled well by standard non-linear least squares methods, and manual outlier removal after visual inspection is tedious and potentially biased. We propose robust non-linear regression via M-estimation as a statistical technique for automated implementation. The approach of finding M-estimates by Iteratively Reweighted Least Squares (IRLS) and the resulting optimization problem are described. Initial parameter estimates for iterative methods are important, so self-starting methods for our model are presented. We outline the software implementation, done in Matlab and deployed as an Excel application via the Matlab Excel Builder Toolkit. Results of M-estimation are compared with least squares estimates before and after manual editing.     

鲁棒双子支持向量回归

针对传统支持向量回归机缺乏鲁棒性而鲁棒支持向量回归机稀疏性不理想,提出了新的支持向量回归方法（鲁棒双子支持向量回归）。为了求解的方便,该方法的损失函数由两个可微的凸函数构成,并且采用CCCP技术对其进行求解。该方法在获得良好稀疏性的同时有效地抑制了过失误差的影响。通过人工数据和现实真实数据对该方法的测试,验证了新方法的有效性。     

Robust fuzzy regression analysis

In this paper we propose a robust fuzzy linear regression model based on the Least Median Squares-Weighted Least Squares (LMS-WLS) estimation procedure. The proposed model is general enough to deal with data contaminated by outliers due to measurement errors or extracted from highly skewed or heavy tailed distributions. We also define suitable goodness of fit indices useful to evaluate the performances of the proposed model. The effectiveness of our model in reducing the outliers influence is shown by using applicative examples, based both on simulated and real data, and by a simulation study.     

Competitive EM algorithm for finite mixture models

In this paper, we present a novel competitive EM (CEM) algorithm for finite mixture models to overcome the two main drawbacks of the EM algorithm: often getting trapped at local maxima and sometimes converging to the boundary of the parameter space. The proposed algorithm is capable of automatically choosing the clustering number and selecting the “split” or “merge” operations efficiently based on the new competitive mechanism we propose. It is insensitive to the initial configuration of the mixture component number and model parameters.Experiments on synthetic data show that our algorithm has very promising performance for the parameter estimation of mixture models. The algorithm is also applied to the structure analysis of complicated Chinese characters. The results show that the proposed algorithm performs much better than previous methods with slightly heavier computation burden.     

Robust estimation of mixture complexity for count data

For finite mixtures, consistent estimation of unknown number of components, called mixture complexity, is considered based on a random sample of counts, when the exact form of component probability mass functions are unknown but are postulated to belong to some parametric family. Following a recent approach of Woo and Sriram [2006. Robust estimation of mixture complexity. J. Amer. Statist. Assoc., to appear.], we develop an estimator of mixture complexity as a by-product of minimizing a Hellinger information criterion, when all the parameters associated with the mixture model are unknown. The estimator is shown to be consistent. Monte Carlo simulations illustrate the ability of our estimator to correctly determine the mixture complexity when the postulated Poisson mixture model is correct. When the postulated model is a Poisson mixture but the data comes from a negative binomial mixture with moderate to more extreme overdispersion in one of its components, simulation results show that our estimator continues to perform well. These confirm the efficiency of the estimator when the model is correctly specified and the robustness when the model is incorrectly specified. A count dataset with overdispersion and possible zero inflation is analyzed to further illustrate the ability of our estimator to determine the number of components.     

Robust regression using biased objectives

For the regression task in a non-parametric setting, designing the objective function to be minimized by the learner is a critical task. In this paper we propose a principled method for constructing and minimizing robust losses, which are resilient to errant observations even under small samples. Existing proposals typically utilize very strong estimates of the true risk, but in doing so require a priori information that is not available in practice. As we abandon direct approximation of the risk, this lets us enjoy substantial gains in stability at a tolerable price in terms of bias, all while circumventing the computational issues of existing procedures. We analyze existence and convergence conditions, provide practical computational routines, and also show empirically that the proposed method realizes superior robustness over wide data classes with no prior knowledge assumptions.     

Robust accelerated failure time regression

Robust estimators for accelerated failure time models with asymmetric (or symmetric) error distribution and censored observations are proposed. It is assumed that the error model belongs to a log-location-scale family of distributions and that the mean response is the parameter of interest. Since scale is a main component of mean, scale is not treated as a nuisance parameter. A three steps procedure is proposed. In the first step, an initial high breakdown point S estimate is computed. In the second step, observations that are unlikely under the estimated model are rejected or down weighted. Finally, a weighted maximum likelihood estimate is computed. To define the estimates, functions of censored residuals are replaced by their estimated conditional expectation given that the response is larger than the observed censored value. The rejection rule in the second step is based on an adaptive cut-off that, asymptotically, does not reject any observation when the data are generated according to the model. Therefore, the final estimate attains full efficiency at the model, with respect to the maximum likelihood estimate, while maintaining the breakdown point of the initial estimator. Asymptotic results are provided. The new procedure is evaluated with the help of Monte Carlo simulations. Two examples with real data are discussed.     

Robust FETI-DP methods for heterogeneous three dimensional elasticity problems

Iterative substructuring methods with Lagrange multipliers are considered for heterogeneous linear elasticity problems with large discontinuities in the material stiffnesses. In particular, results for algorithms belonging to the family of dual-primal FETI methods are presented. The core issue of these algorithms is the construction of an appropriate global problem, in order to obtain a robust method which converges independently of the material discontinuities. In this article, several necessary and sufficient conditions arising from the theory are numerically tested and confirmed. Furthermore, results of numerical experiments are presented for situations which are not covered by the theory, such as curved edges and material discontinuities not aligned with the interface, and an attempt is made to develop rules for these cases.     

Robust object matching for persistent tracking with heterogeneous features

This paper addresses the problem of matching vehicles across multiple sightings under variations in illumination and camera poses. Since multiple observations of a vehicle are separated in large temporal and/or spatial gaps, thus prohibiting the use of standard frame-to-frame data association, we employ features extracted over a sequence during one time interval as a vehicle fingerprint that is used to compute the likelihood that two or more sequence observations are from the same or different vehicles. Furthermore, since our domain is aerial video tracking, in order to deal with poor image quality and large resolution and quality variations, our approach employs robust alignment and match measures for different stages of vehicle matching. Most notably, we employ a heterogeneous collection of features such as lines, points, and regions in an integrated matching framework. Heterogeneous features are shown to be important. Line and point features provide accurate localization and are employed for robust alignment across disparate views. The challenges of change in pose, aspect, and appearances across two disparate observations are handled by combining a novel feature-based quasi-rigid alignment with flexible matching between two or more sequences. However, since lines and points are relatively sparse, they are not adequate to delineate the object and provide a comprehensive matching set that covers the complete object. Region features provide a high degree of coverage and are employed for continuous frames to provide a delineation of the vehicle region for subsequent generation of a match measure. Our approach reliably delineates objects by representing regions as robust blob features and matching multiple regions to multiple regions using Earth Mover's Distance (EMD). Extensive experimentation under a variety of real-world scenarios and over hundreds of thousands of Confirmatory Identification (CID) trails has demonstrated about 95 percent accuracy in vehicle reacquisition with both visible and Infrared (IR) imaging cameras.     

TRUST-TECH-based expectation maximization for learning finite mixture models

In spite of the initialization problem, the Expectation-Maximization (EM) algorithm is widely used for estimating the parameters of finite mixture models. Most popular model-based clustering techniques might yield poor clusters if the parameters are not initialized properly. To reduce the sensitivity of initial points, a novel algorithm for learning mixture models from multivariate data is introduced in this paper. The proposed algorithm takes advantage of TRUST-TECH (TRansformation Under STability-reTaining Equilibra CHaracterization) to compute neighborhood local maxima on likelihood surface using stability regions. Basically, our method coalesces the advantages of the traditional EM with that of the dynamic and geometric characteristics of the stability regions of the corresponding nonlinear dynamical system of the log-likelihood function. Two phases namely, the EM phase and the stability region phase, are repeated alternatively in the parameter space to achieve improvements in the maximum likelihood. The EM phase obtains the local maximum of the likelihood function and the stability region phase helps to escape out of the local maximum by moving towards the neighboring stability regions. The algorithm has been tested on both synthetic and real datasets and the improvements in the performance compared to other approaches are demonstrated. The robustness with respect to initialization is also illustrated experimentally.     

On finite Newton method for support vector regression

In this paper, we propose a Newton iterative method of solution for solving an ε-insensitive support vector regression formulated as an unconstrained optimization problem. The proposed method has the advantage that the solution is obtained by solving a system of linear equations at a finite number of times rather than solving a quadratic optimization problem. For the case of linear or kernel support vector regression, the finite termination of the Newton method has been proved. Experiments were performed on IBM, Google, Citigroup and Sunspot time series. The proposed method converges in at most six iterations. The results are compared with that of the standard, least squares and smooth support vector regression methods and of the exact solutions clearly demonstrate the effectiveness of the proposed method.     

空间可变有限混合模型

目的 有限混合模型是一种无监督学习方法,它被广泛的应用到数据分类任务中。然而,在图像分割过程中,由于有限混合模型没有引入邻域像素间的空间关系,导致了图像分割结果对噪声非常敏感。为了增强有限混合模型的抗噪性,提出一种新的空间可变有限混合模型。方法 该模型通过在像素的先验分布中引入一种新的空间关系来降低噪声对图像分割结果的干扰。在构建空间关系的过程中,利用形态学膨胀原理将空间邻域内特征值出现的概率而不是特征值本身进行膨胀操作,然后通过根据具有最大概率的分类标记在高斯混合模型迭代地计算过程中进行局部像素空间平滑,从而起到抑制噪声干扰的作用。结果 本文实验包含了人工合成图像和医学CT图像的图像分割实验。在人工合成图像分割实验中,对人工合成图像添加了不同程度的噪声来测试本文模型和对比模型对噪声抑制能力的高低;对医学CT图像进行图像分割实验,以是比较本文模型与对比模型之间在实际图像分割中的效果。结论 实验数据显示,本文提出的模型在噪声抑制能力上,图像分割精度和计算效率上均有更优的性能。     

Robust regression methods for computer vision: A review

Regression analysis (fitting a model to noisy data) is a basic technique in computer vision, Robust regression methods that remain reliable in the presence of various types of noise are therefore of considerable importance. We review several robust estimation techniques and describe in detail the least-median-of-squares (LMedS) method. The method yields the correct result even when half of the data is severely corrupted. Its efficiency in the presence of Gaussian noise can be improved by complementing it with a weighted least-squares-based procedure. The high time-complexity of the LMedS algorithm can be reduced by a Monte Carlo type speed-up technique. We discuss the relationship of LMedS with the RANSAC paradigm and its limitations in the presence of noise corrupting all the data, and we compare its performance with the class of robust M-estimators. References to published applications of robust techniques in computer vision are also given.     

Adaptive finite element heterogeneous multiscale method for homogenization problems

In this paper we present an a posteriori error analysis for elliptic homogenization problems discretized by the finite element heterogeneous multiscale method. Unlike standard finite element methods, our discretization scheme relies on macro- and microfinite elements. The desired macroscopic solution is obtained by a suitable averaging procedure based on microscopic data. As the macroscopic data (such as the macroscopic diffusion tensor) are not available beforehand, appropriate error indicators have to be defined for designing adaptive methods. We show that such indicators based only on the available macro- and microsolutions (used to compute the actual macrosolution) can be defined, allowing for a macroscopic mesh refinement strategy which is both reliable and efficient. The corresponding a posteriori estimates for the upper and lower bound are derived in the energy norm. In the case of a uniformly oscillating tensor, we recover the standard residual-based a posteriori error estimate for the finite element method applied to the homogenized problem. Numerical experiments confirm the efficiency and reliability of the adaptive multiscale method.     

Robust clusterwise linear regression through trimming

The presence of clusters in a data set is sometimes due to the existence of certain relations among the measured variables which vary depending on some hidden factors. In these cases, observations could be grouped in a natural way around linear and nonlinear structures and, thus, the problem of doing robust clustering around linear affine subspaces has recently been tackled through the minimization of a trimmed sum of orthogonal residuals. This “orthogonal approach” implies that there is no privileged variable playing the role of response variable or output. However, there are problems where clearly one variable is wanted to be explained in terms of the other ones and the use of vertical residuals from classical linear regression seems to be more advisable. The so-called TCLUST methodology is extended to perform robust clusterwise linear regression and a feasible algorithm for the practical implementation is proposed. The algorithm includes a “second trimming” step aimed to diminishing the effect of leverage points.     

Robust linear and support vector regression

The robust Huber M-estimator, a differentiable cost function that is quadratic for small errors and linear otherwise, is modeled exactly, in the original primal space of the problem, by an easily solvable simple convex quadratic program for both linear and nonlinear support vector estimators. Previous models were significantly more complex or formulated in the dual space and most involved specialized numerical algorithms for solving the robust Huber linear estimator. Numerical test comparisons with these algorithms indicate the computational effectiveness of the new quadratic programming model for both linear and nonlinear support vector problems. Results are shown on problems with as many as 20000 data points, with considerably faster running times on larger problems