Robust pose invariant face recognition using coupled latent space discriminant analysis

We propose a novel pose-invariant face recognition approach which we call Discriminant Multiple Coupled Latent Subspace framework. It finds the sets of projection directions for different poses such that the projected images of the same subject in different poses are maximally correlated in the latent space. Discriminant analysis with artificially simulated pose errors in the latent space makes it robust to small pose errors caused due to a subject’s incorrect pose estimation. We do a comparative analysis of three popular latent space learning approaches: Partial Least Squares (PLSs), Bilinear Model (BLM) and Canonical Correlational Analysis (CCA) in the proposed coupled latent subspace framework. We experimentally demonstrate that using more than two poses simultaneously with CCA results in better performance. We report state-of-the-art results for pose-invariant face recognition on CMU PIE and FERET and comparable results on MultiPIE when using only four fiducial points for alignment and intensity features.     

Joint gender,ethnicity and age estimation from 3D faces: An experimental illustration of their correlations

Humans present clear demographic traits which allow their peers to recognize their gender and ethnic groups as well as estimate their age. Abundant literature has investigated the problem of automated gender, ethnicity and age recognition from facial images. However, despite the co-existence of these traits, most of the studies have addressed them separately, very little attention has been given to their correlations. In this work, we address the problem of joint demographic estimation and investigate the correlation through the morphological differences in 3D facial shapes. To this end, a set of facial features are extracted to capture the 3D shape differences among the demographic groups. Then, a correlation-based feature selection is applied to highlight salient features and remove redundancy. These features are later fed to Random Forest for gender and ethnicity classification, and age estimation. Extensive experiments conducted on FRGCv2 dataset, under Expression-Dependent and Expression-Independent settings, demonstrate the effectiveness of the proposed approaches for the three traits, and also show the accuracy improvement when considering their correlations. To the best of our knowledge, this is the first study exploring the correlations of these facial soft-biometric traits using 3D faces. This is also the first work which studies the problem of age estimation from 3D Faces.¹     

The regularized least squares algorithm and the problem of learning halfspaces

We provide sample complexity of the problem of learning halfspaces with monotonic noise, using the regularized least squares algorithm in the reproducing kernel Hilbert spaces (RKHS) framework.     

Robust least squares methods under bounded data uncertainties

We study the problem of estimating an unknown deterministic signal that is observed through an unknown deterministic data matrix under additive noise. In particular, we present a minimax optimization framework to the least squares problems, where the estimator has imperfect data matrix and output vector information. We define the performance of an estimator relative to the performance of the optimal least squares (LS) estimator tuned to the underlying unknown data matrix and output vector, which is defined as the regret of the estimator. We then introduce an efficient robust LS estimation approach that minimizes this regret for the worst possible data matrix and output vector, where we refrain from any structural assumptions on the data. We demonstrate that minimizing this worst-case regret can be cast as a semi-definite programming (SDP) problem. We then consider the regularized and structured LS problems and present novel robust estimation methods by demonstrating that these problems can also be cast as SDP problems. We illustrate the merits of the proposed algorithms with respect to the well-known alternatives in the literature through our simulations.     

Kernel continuum regression

The continuum regression technique provides an appealing regression framework connecting ordinary least squares, partial least squares and principal component regression in one family. It offers some insight on the underlying regression model for a given application. Moreover, it helps to provide deep understanding of various regression techniques. Despite the useful framework, however, the current development on continuum regression is only for linear regression. In many applications, nonlinear regression is necessary. The extension of continuum regression from linear models to nonlinear models using kernel learning is considered. The proposed kernel continuum regression technique is quite general and can handle very flexible regression model estimation. An efficient algorithm is developed for fast implementation. Numerical examples have demonstrated the usefulness of the proposed technique.     

Geometric properties of partial least squares for process monitoring

Projection to latent structures or partial least squares (PLS) produces output-supervised decomposition on input X, while principal component analysis (PCA) produces unsupervised decomposition of input X. In this paper, the effect of output Y on the X-space decomposition in PLS is analyzed and geometric properties of the PLS structure are revealed. Several PLS algorithms are compared in a geometric way for the purpose of process monitoring. A numerical example and a case study are given to illustrate the analysis results.     

Iterative constrained weighted least squares estimator for TDOA and FDOA positioning of multiple disjoint sources in the presence of sensor position and velocity uncertainties

Sensor position and velocity uncertainties are known to be able to degrade the source localization accuracy significantly. This paper focuses on the problem of locating multiple disjoint sources using time differences of arrival (TDOAs) and frequency differences of arrival (FDOAs) in the presence of sensor position and velocity errors. First, the explicit Cramér–Rao bound (CRB) expression for joint estimation of source and sensor positions and velocities is derived under the Gaussian noise assumption. Subsequently, we compare the localization accuracy when multiple-source positions and velocities are determined jointly and individually based on the obtained CRB results. The performance gain resulted from multiple-target cooperative positioning is also quantified using the orthogonal projection matrix. Next, the paper proposes a new estimator that formulates the localization problem as a quadratic programming with some indefinite quadratic equality constraints. Due to the non-convex nature of the optimization problem, an iterative constrained weighted least squares (ICWLS) method is developed based on matrix QR decomposition, which can be achieved through some simple and efficient numerical algorithms. The newly proposed iterative method uses a set of linear equality constraints instead of the quadratic constraints to produce a closed-form solution in each iteration. Theoretical analysis demonstrates that the proposed method, if converges, can provide the optimal solution of the formulated non-convex minimization problem. Moreover, its estimation mean-square-error (MSE) is able to reach the corresponding CRB under moderate noise level. Simulations are included to corroborate and support the theoretical development in this paper.     

3种空隙率测量建模方法的比较研究

介绍了测量空隙率的电容层析成像（ECT）系统,分析了ECT测量空隙率的基本原理。简述了3种空隙率测量建模方法,即分别基于遗传算法-偏最小二乘（GA-PLS）算法、蚁群算法（ASA）和最小二乘支持向量机（LS—SVM）的建模方法。建模数据均来自Ecr传感器获取的电容值。在此基础上,提出了以ECT传感器为信息获得手段的空隙率测量建模的统一框架。通过实验数据对3种方法进行了评价,对它们从泛化误差、最大误差、平均误差和测量时间几个角度进行了比较。结果表明：璐-SVM方法的精度最高,泛化能力最强。     

Frequency estimation of undamped exponential signals using genetic algorithms

In this paper, we consider the problem of frequency estimation of undamped superimposed exponential signals model. We propose two iterative techniques of frequency estimation using genetic algorithms. The proposed methods use an elitism based generational genetic algorithm for obtaining the least squares and the approximate least squares estimates. In the simulation studies, it is observed that the proposed methods give nearly efficient estimates, having mean square error almost attaining the corresponding Cramér-Rao lower bound. The proposed methods significantly do not depend on the initial guess values otherwise required for other iterative methods of frequency estimation. It is also observed that the proposed methods have fairly high breakdown point with respect to different types of outliers present in the data. Outlier robustness and accuracy of the proposed methods are compared with the classical approaches for this problem.     

Functional PLS logit regression model

Functional logistic regression has been developed to forecast a binary response variable from a functional predictor. In order to fit this model, it is usual to assume that the functional observations and the parameter function of the model belong to a same finite space generated by a basis of functions. This consideration turns the functional model into a multiple logit model whose design matrix is the product of the matrix of sample paths basic coefficients and the matrix of the inner products between basic functions. The likelihood estimation of the parameter function of this model is very inaccurate due to the high dependence structure of the so obtained design matrix (multicollinearity). In order to solve this drawback several approaches have been proposed. These employ standard multivariate data analysis methods on the design matrix. This is the case of the functional principal component logistic regression model. As an alternative a functional partial least squares logit regression model is proposed, that has as covariates a set of partial least squares components of the design matrix of the multiple logit model associated to the functional one.     

On the effect of measurement errors in regression-adjusted monitoring of multistage manufacturing processes

A critical challenge in multistage process monitoring is the complex relationships between quality characteristics at different stages. A popular method to deal with this problem is regression adjustment in which each quality characteristic is regressed on its preceding quality characteristics and the resulting residual is monitored to detect changes in local variations. However, the performance of this method depends on the accuracy of the regression coefficient estimation. One source of the estimation errors is measurement errors which commonly exist in practice. To provide guidance on the use of regression-adjusted monitoring methods, this study investigates the effect of measurement errors on the bias of regression estimation theoretically and numerically. Two estimators, the ordinary least squares (OLS) estimator and the total least squares (TLS) estimator, are compared, and insights regarding their performance are obtained.     

A theorem on the generalized canonical projective vectors

This paper proposes a kind of generalized canonical projective vectors (GCPV), based on the framework of canonical correlation analysis (CCA) applying image recognition. Apart from canonical projective vectors (CPV), the process of obtaining GCPV contains the class information of samples, such that the combined features extracted according to the basis of GCPV can give a better classification performance. The experimental result based on the Concordia University CENPARMI handwritten Arabian numeral database has proved that our method is superior to the method based on CPV.     

A revisit to block and recursive least squares for parameter estimation

In this paper, the classical least squares (LS) and recursive least squares (RLS) for parameter estimation have been re-examined in the light of the present day computing capabilities. It has been demonstrated that for linear time-invariant systems, the performance of blockwise least squares (BLS) is always superior to that of RLS. In the context of parameter estimation for dynamic systems, the current computational capability of personal computers are more than adequate for BLS. However, for time-varying systems with abrupt parameter changes, standard blockwise LS may no longer be suitable due to its inefficiency in discarding “old” data. To deal with this limitation, a novel sliding window blockwise least squares approach with automatically adjustable window length triggered by a change detection scheme is proposed. Two types of sliding windows, rectangular and exponential, have been investigated. The performance of the proposed algorithm has been illustrated by comparing with the standard RLS and an exponentially weighted RLS (EWRLS) using two examples. The simulation results have conclusively shown that: (1) BLS has better performance than RLS; (2) the proposed variable-length sliding window blockwise least squares (VLSWBLS) algorithm can outperform RLS with forgetting factors; (3) the scheme has both good tracking ability for abrupt parameter changes and can ensure the high accuracy of parameter estimate at the steady-state; and (4) the computational burden of VLSWBLS is completely manageable with the current computer technology. Even though the idea presented here is straightforward, it has significant implications to virtually all areas of application where RLS schemes are used.     

基于非线性最小二乘的图像自适应SR重建以及运动估计

图像超分辨率（SR）重建是利用数字信号处理技术由一系列低分辨率观测图像得到高分辨率图像。为了扩展SR技术的应用范围,提出了一种同时进行图像超分辨率重建和全局运动估计的方法。该方法首先基于最大后验概率(MAP)给出了图像SR重建和运动估计框架,该框架不仅考虑了前后两次迭代所得的HR图像差值对最终重建图像的影响,而且引入了不同LR图像对重建图像的重要性权值,使得算法具有自适应性;然后将总体框架转换为图像SR重建模型和运动估计模型;最后基于非线性最小二乘法对模型进行优化求解,得出了SR重建图像及其全局运动域。实验表明,该方法不仅图像重建效果良好,并有着良好的收敛性。     

质量相关的带钢热连轧过程监控

带钢热连轧过程控制是钢铁制造过程极其复杂的过程,近年来随着市场对带钢产品质量要求的日益提高,提高热连轧带钢质量具有广泛的经济和社会效益。为了确保热连轧过程安全运行,同时提高产品质量,有必要对热连轧过程的异常状况或故障进行检测、诊断和消除。以多元统计过程监控技术(MSPM)为理论指导,以主元分析(PCA)和偏最小二乘方法(PLS)为依托,研究和分析了PCA和PLS以及二者与核函数结合构成的核主元分析方法(KP-CA)和核偏最小二乘方法(KPLS)在热连轧机质量相关的故障分析与检测,通过现场数据及实验验证,在厚度质量相关的故障检测与诊断中取得较好的效果。     

An integrated advanced process control framework using run-to-run control,virtual metrology and fault detection

This paper develops a new advanced process control (APC) system for the multiple-input multiple-output (MIMO) semiconductor processes using the partial least squares (PLS) technique to provide the run-to-run control with the virtual metrology data, via the gradual mode or the rapid mode depending on the current system status, in order to deal with metrology delays and compensate for different types of system disturbances. First, we present a controller called the PLS-MIMO double exponentially weighted moving average (PLS-MIMO DEWMA) controller. It employs the PLS method as the model building/estimation technique to help the DEWMA controller generate more consistent and robust control outputs than purely using the conventional DEWMA controller. To cope with metrology delays, the proposed APC system uses the pre-processing metrology data to build up the virtual metrology (VM) system that can provide the estimated process outputs for the PLS-MIMO DEWMA controller. Lastly, the Fault Detection (FD) system is added based upon the principal components of the PLS modeling outcomes, which supplies the process status for the VM mechanism and the PLS-MIMO DEWMA controller as to how the process faults are responded. Two scenarios of the simulation study are conducted to illustrate the APC system proposed in this paper.     

Estimation and inference on central mean subspace for multivariate response data

In this paper, we introduce the notion of the central mean subspace when the response is multivariate, and propose a profile least squares approach to perform estimation and inference. Unlike existing methods in the sufficient dimension reduction literature, the profile least squares method does not require any distributional assumptions on the covariates, and facilitates statistical inference on the central mean subspace. We demonstrate theoretically and empirically that the properly weighted profile least squares approach is more efficient than its unweighted counterpart. We further confirm the promising finite-sample performance of our proposal through comprehensive simulations and an application to an etiologic study on essential hypertension conducted in P. R. China.     

Canonical correlation analysis using within-class coupling

Fisher’s linear discriminant analysis (LDA) is one of the most popular supervised linear dimensionality reduction methods. Unfortunately, LDA is not suitable for problems where the class labels are not available and only the spatial or temporal association of data samples is implicitly indicative of class membership. In this study, a new strategy for reducing LDA to Hotelling’s canonical correlation analysis (CCA) is proposed. CCA seeks prominently correlated projections between two views of data and it has been long known to be equivalent to LDA when the data features are used in one view and the class labels are used in the other view. The basic idea of the new equivalence between LDA and CCA, which we call within-class coupling CCA (WCCCA), is to apply CCA to pairs of data samples that are most likely to belong to the same class. We prove the equivalence between LDA and such an application of CCA. With such an implicit representation of the class labels, WCCCA is applicable both to regular LDA problems and to problems in which only spatial and/or temporal continuity provides clues to the class labels.     

核诱导距离度量的鲁棒典型相关分析

典型相关分析(canonical correlation analysis,CCA)是寻找同一对象两组变量间线性相关性的一种常用的多元统计分析方法,其采用的欧氏距离度量方式导致了算法的非鲁棒性。核诱导的距离度量不仅在理论上被证明是鲁棒的,而且在(聚类)应用上获得了有效验证。将其进一步应用于CCA,发展出了核诱导距离度量的鲁棒CCA(CCA based on kernel-induced measure,KI-CCA)。该算法不仅克服了CCA非鲁棒的不足,而且使现有基于最大相关熵的鲁棒主成分分析(half-quadratic principal component analysis,HQ-PCA)成为特例,且具有非线性相关分析的能力。一方面,核的多样性使得KI-CCA也具有多样性,从而使其成为一般性的分析算法。另一方面,与CCA刻画上的相似性,使其求解可归结为广义特征值问题。在人工数据、多特征手写体数据库(multiple feature database,MFD)和人脸数据集(Yale、AR、ORL)上的实验验证了该算法的有效性。     

Recursive Least Squares Identification With Variable-Direction Forgetting via Oblique Projection Decomposition

In this paper, a new recursive least squares(RLS)identification algorithm with variable-direction forgetting(VDF)is proposed for multi-output systems. The objective is to enhance parameter estimation performance under non-persistent excitation. The proposed algorithm performs oblique projection decomposition of the information matrix, such that forgetting is applied only to directions where new information is received.Theoretical proofs show that even without persistent excitation,the informatio...