Parametric image alignment using enhanced correlation coefficient maximization

In this work we propose the use of a modified version of the correlation coefficient as a performance criterion for the image alignment problem. The proposed modification has the desirable characteristic of being invariant with respect to photometric distortions. Since the resulting similarity measure is a nonlinear function of the warp parameters, we develop two iterative schemes for its maximization, one based on the forward additive approach and the second on the inverse compositional method. As it is customary in iterative optimization, in each iteration the nonlinear objective function is approximated by an alternative expression for which the corresponding optimization is simple. In our case we propose an efficient approximation that leads to a closed form solution (per iteration) which is of low computational complexity, the latter property being particularly strong in our inverse version. The proposed schemes are tested against the Forward Additive Lucas-Kanade and the Simultaneous Inverse Compositional algorithm through simulations. Under noisy conditions and photometric distortions our forward version achieves more accurate alignments and exhibits faster convergence whereas our inverse version has similar performance as the Simultaneous Inverse Compositional algorithm but at a lower computational complexity.     

基于邻域内相关系数与平均梯度的图像融合方法

通过分析研究图像融合的客观评价指标,提出了一种新的基于邻域的融合策略,即首先对源图像进行双树复小波(DT-CWT)分解,得到高频和低频图像.针对低频图像采用相关系数为阚值,以标准差加权平均进行邻域融合,高频图像以平均梯度为测度参数进行邻域融合,最后进行逆变换得到融合图像.采用均值、方差、熵和平均梯度4种客观评价指标来评价融合图像效果.实验结果表明,该方法能够较好地适应相似度差异较大的多传感器图像融合,有效地增强融合图像的细节信息和清晰度.     

Unsupervised manifold alignment using soft-assign technique

In this paper, we propose a robust unsupervised algorithm for automatic alignment of two manifolds in different datasets with possibly different dimensionalities. The significant contribution is that the proposed alignment algorithm is performed automatically without any assumptions on the correspondences between the two manifolds. For such purpose, we first automatically extract local feature histograms at each point of the manifolds and establish an initial similarity between the two datasets by matching their histogram-based features. Based on such similarity, an embedding space is estimated where the distance between the two manifolds is minimized while maximally retaining the original structure of the manifolds. The elegance of this idea is that such complicated problem is formulated as a generalized eigenvalue problem, which can be easily solved. The alignment process is achieved by iteratively increasing the sparsity of correspondence matrix until the two manifolds are correctly aligned and consequently one can reveal their joint structure. We demonstrate the effectiveness of our algorithm on different datasets by aligning protein structures, 3D face models and facial images of different subjects under pose and lighting variations. Finally, we also compare with a state-of-the-art algorithm and the results show the superiority of the proposed manifold alignment in terms of vision effect and numerical accuracy.     

Unsupervised image matching based on manifold alignment

This paper challenges the issue of automatic matching between two image sets with similar intrinsic structures and different appearances, especially when there is no prior correspondence. An unsupervised manifold alignment framework is proposed to establish correspondence between data sets by a mapping function in the mutual embedding space. We introduce a local similarity metric based on parameterized distance curves to represent the connection of one point with the rest of the manifold. A small set of valid feature pairs can be found without manual interactions by matching the distance curve of one manifold with the curve cluster of the other manifold. To avoid potential confusions in image matching, we propose an extended affine transformation to solve the nonrigid alignment in the embedding space. The comparatively tight alignments and the structure preservation can be obtained simultaneously. The point pairs with the minimum distance after alignment are viewed as the matchings. We apply manifold alignment to image set matching problems. The correspondence between image sets of different poses, illuminations, and identities can be established effectively by our approach.     

Unsupervised face recognition by associative chaining

We propose a novel method for unsupervised face recognition from time-varying sequences of face images obtained in real-world environments. The method utilizes the higher level of sensory variation contained in the input image sequences to autonomously organize the data in an incrementally built graph structure, without relying on category-specific information provided in advance. This is achieved by “chaining” together similar views across the spatio-temporal representations of the face sequences in image space by two types of connecting edges depending on local measures of similarity. Experiments with real-world data gathered over a period of several months and including both frontal and side-view faces from 17 different subjects were used to test the method, achieving correct self-organization rate of 88.6%. The proposed method can be used in video surveillance systems or for content-based information retrieval.     

Unsupervised group matching with application to cross-lingual topic matching without alignment information

We propose a method for unsupervised group matching, which is the task of finding correspondence between groups across different domains without cross-domain similarity measurements or paired data. For example, the proposed method can find matching of topic categories in different languages without alignment information. The proposed method interprets a group as a probability distribution, which enables us to handle uncertainty in a limited amount of data, and to incorporate the high order information on groups. Groups are matched by maximizing the dependence between distributions, in which we use the Hilbert Schmidt independence criterion for measuring the dependence. By using kernel embedding which maps distributions into a reproducing kernel Hilbert space, we can calculate the dependence between distributions without density estimation. In the experiments, we demonstrate the effectiveness of the proposed method using synthetic and real data sets including an application to cross-lingual topic matching.     

最大散度差无监督鉴别特征抽取与人脸识别

最大散度差准则是对Fisher准则的改进,消除了小样本问题,但是该方法是基于整体特征的人脸识另q方?法,没有考虑到样本的局部特性.无监督的鉴别投影(UDP)技术,用于对高维数据进行维数缩减,它同时考虑到样本的局部特征和非局部特征,但是在人脸等高维图像识别的应用中,不可避免地会出现小样本问题.提出一种基于散度差的无监督鉴别特征抽取,避免了局部协方差奇异所产生的问题.在ORL人脸库和AR人脸库上的实验结果验证了该算法的有效性.     

用于工作面液压支架对齐的激光对位传感器设计

为了满足煤矿井下工作面跟机自动化连续采煤需求,设计了一种用于工作面液压支架对齐的激光对位传感器,详细介绍了基于激光对位传感器的移架对齐原理及传感器的软硬件设计。该传感器采用小型化设计,具有安装方便、价格低廉等优点。井下工业性实验结果表明,将该传感器应用在跟机自动化采煤中,可使工作面走向的相邻液压支架间的平均对齐精度达到30mm。     

Unsupervised multi-manifold linear differential projection(UMLDP) for face recognition

Multimedia Tools and Applications - A novel efficient algorithm called unsupervised multi-manifold linear differential projection(UMLDP) is proposed to overcome the drawbacks of existing...     

An intuitive general rank-based correlation coefficient

Correlation analysis is an effective mechanism for studying patterns in data and making predictions. Many interesting discoveries have been made by formulating correlations in seemingly unrelated data. We propose an algorithm to quantify the theory of correlations and to give an intuitive, more accurate correlation coefficient. We propose a predictive metric to calculate correlations between paired values, known as the general rank-based correlation coefficient. It fulfills the five basic criteria of a predictive metric: independence from sample size, value between ?1 and 1, measuring the degree of monotonicity, insensitivity to outliers, and intuitive demonstration. Furthermore, the metric has been validated by performing experiments using a real-time dataset and random number simulations. Mathematical derivations of the proposed equations have also been provided. We have compared it to Spearman’s rank correlation coefficient. The comparison results show that the proposed metric fares better than the existing metric on all the predictive metric criteria.     

Generative face alignment through 2.5D active appearance models

This work addresses the matching of a 3D deformable face model to 2D images through a 2.5D Active Appearance Models (AAM). We propose a 2.5D AAM that combines a 3D metric Point Distribution Model (PDM) and a 2D appearance model whose control points are defined by a full perspective projection of the PDM. The advantage is that, assuming a calibrated camera, 3D metric shapes can be retrieved from single view images. Two model fitting algorithms and their computational efficient approximations are proposed: the Simultaneous Forwards Additive (SFA) and the Normalization Forwards Additive (NFA), both based on the Lucas–Kanade framework. The SFA algorithm searches for shape and appearance parameters simultaneously whereas the NFA projects out the appearance from the error image and searches only for the shape parameters. SFA is therefore more accurate. Robust solutions for the SFA and NFA are also proposed in order to take into account the self-occlusion or partial occlusion of the face. Several performance evaluations for the SFA, NFA and theirs efficient approximations were performed. The experiments include evaluating the frequency of converge, the fitting performance in unseen data and the tracking performance in the FGNET Talking Face sequence. All results show that the 2.5D AAM can outperform both the 2D + 3D combined models and the 2D standard methods. The robust extensions to occlusion were tested on a synthetic sequence showing that the model can deal efficiently with large head rotation.     

改进级联回归模型的人脸特征点定位

针对级联回归模型依赖形状初始化且结构复杂使其在人脸特征点定位中速度慢、精度低的问题,提出了改进的级联回归人脸特征点定位算法.采用仿射变换参数回归初始化人脸形状,使变换后的初始形状更接近真实人脸以提高模型的收敛速度和精度;在各特征点局部区域构造随机蕨局部学习器,并学习得到易于计算且高度稀疏的二值化特征应用提高模型的速度;对二值化特征使用全局线性回归求得形状增量,实现特征点定位.仿真实验结果表明:相比于原算法,所提算法在LFPW,HELEN,AFW库上定位误差平均降低了11%,定位时间平均减少了14%.     

基于深度可分离沙漏网络的快速人脸对齐

针对沙漏网络应用于人脸对齐中存在网络结构复杂、时间开销大的问题,提出一种带有深度可分离的轻量级沙漏网络.通过知识蒸馏的思想构造轻量级沙漏网络,解决网络结构复杂的问题;在叠层沙漏网络中使用深度可分离卷积,通过深度卷积和逐点卷积共同作用简化复杂网络,解决时间开销大的问题.实验结果表明,在300w数据集和WFLW数据集上,该...     

Efficient constrained local model fitting for non-rigid face alignment

Active appearance models (AAMs) have demonstrated great utility when being employed for non-rigid face alignment/tracking. The “simultaneous” algorithm for fitting an AAM achieves good non-rigid face registration performance, but has poor real time performance (2–3 fps). The “project-out” algorithm for fitting an AAM achieves faster than real time performance (>200 fps) but suffers from poor generic alignment performance. In this paper we introduce an extension to a discriminative method for non-rigid face registration/tracking referred to as a constrained local model (CLM). Our proposed method is able to achieve superior performance to the “simultaneous” AAM algorithm along with real time fitting speeds (35 fps). We improve upon the canonical CLM formulation, to gain this performance, in a number of ways by employing: (i) linear SVMs as patch-experts, (ii) a simplified optimization criteria, and (iii) a composite rather than additive warp update step. Most notably, our simplified optimization criteria for fitting the CLM divides the problem of finding a single complex registration/warp displacement into that of finding N simple warp displacements. From these N simple warp displacements, a single complex warp displacement is estimated using a weighted least-squares constraint. Another major advantage of this simplified optimization lends from its ability to be parallelized, a step which we also theoretically explore in this paper. We refer to our approach for fitting the CLM as the “exhaustive local search” (ELS) algorithm. Experiments were conducted on the CMU MultiPIE database.     

Joint face alignment and segmentation via deep multi-task learning

Multimedia Tools and Applications - Face alignment and segmentation are challenging problems which have been extensively studied in the field of multimedia. These two tasks are closely related and...     

DC-EDN: densely connected encoder-decoder network with reinforced depthwise convolution for face alignment

High accuracy and fast face alignment algorithms play an important role in many face-related applications. Generally, the model speed is inversely related to the number of parameters. We construct our network based on densely connected encoder-decoders, which is an efficient method to balance the parameter number and localization results. In each encoder-decoder, we introduce stacking depthwise convolution and depthwise feature fusion within the same channel, which greatly improves the performance of depthwise convolution and reduces the number of model parameters. In addition, we enhance the mean square loss function by assigning different penalty weights to each coordinate according to the distance to the position corresponding to the maximum value in the label heatmap. Experiments show that the model with the improved loss function obtains better localization results. In the experiment, we compare our method to state-of-the-art methods based on 300W and WFLW. The localization error is 2.76% with the common subset of 300W and the model size (0.7M) is small and even utilizes approximately 1% of the number of parameters of the other models. The dataset and model based on WFLW are publicly available at https://github.com/iam-zhanghongliang/DC-EDN.

     

Unsupervised feature selection via joint local learning and group sparse regression

Feature selection has attracted a great deal of interest over the past decades. By selecting meaningful feature subsets, the performance of learning algorithms can be effectively improved. Because label information is expensive to obtain, unsupervised feature selection methods are more widely used than the supervised ones. The key to unsupervised feature selection is to find features that effectively reflect the underlying data distribution. However, due to the inevitable redundancies and noise in a dataset, the intrinsic data distribution is not best revealed when using all features. To address this issue, we propose a novel unsupervised feature selection algorithm via joint local learning and group sparse regression (JLLGSR). JLLGSR incorporates local learning based clustering with group sparsity regularized regression in a single formulation, and seeks features that respect both the manifold structure and group sparse structure in the data space. An iterative optimization method is developed in which the weights finally converge on the important features and the selected features are able to improve the clustering results. Experiments on multiple real-world datasets (images, voices, and web pages) demonstrate the effectiveness of JLLGSR.

     

基于相关系数的适用于视频拼接的帧对齐方法

为解决视频拼接中源视频在时间上不同步的问题,提出了一种视频帧对齐方法.利用图像矩阵间的相关系数特性,根据视频拼接中源视频的特点,运用运动量检测方法检测视频中不同时间点的运动量,在低运动量点运用相位相关法确定交叠区域大致范围,在高运动量点计算相关系数确定同一时间点上的对应帧图像.在对应帧的搜索过程中,使用变步长搜索算法提高搜索效率,减少搜索时间.实验中使用无标定的双摄像机采集多组视频数据,结果表明了该方法的准确性和有效性.     

Data driven image models through continuous joint alignment

This paper presents a family of techniques that we call congealing for modeling image classes from data. The idea is to start with a set of images and make them appear as similar as possible by removing variability along the known axes of variation. This technique can be used to eliminate "nuisance" variables such as affine deformations from handwritten digits or unwanted bias fields from magnetic resonance images. In addition to separating and modeling the latent images - i.e., the images without the nuisance variables - we can model the nuisance variables themselves, leading to factorized generative image models. When nuisance variable distributions are shared between classes, one can share the knowledge learned in one task with another task, leading to efficient learning. We demonstrate this process by building a handwritten digit classifier from just a single example of each class. In addition to applications in handwritten character recognition, we describe in detail the application of bias removal from magnetic resonance images. Unlike previous methods, we use a separate, nonparametric model for the intensity values at each pixel. This allows us to leverage the data from the MR images of different patients to remove bias from each other. Only very weak assumptions are made about the distributions of intensity values in the images. In addition to the digit and MR applications, we discuss a number of other uses of congealing and describe experiments about the robustness and consistency of the method.     

Computer programs for the concordance correlation coefficient

The CCC macro is presented for computation of the concordance correlation coefficient (CCC), a common measure of reproducibility. The macro has been produced in both SAS and R, and a detailed presentation of the macro input and output for the SAS program is included. The macro provides estimation of three versions of the CCC, as presented by Lin [L.I.-K. Lin, A concordance correlation coefficient to evaluate reproducibility, Biometrics 45 (1989) 255-268], Barnhart et al. [H.X. Barnhart, J.L. Haber, J.L. Song, Overall concordance correlation coefficient for evaluating agreement among multiple observers, Biometrics 58 (2002) 1020-1027], and Williamson et al. [J.M. Williamson, S.B. Crawford, H.M. Lin, Resampling dependent concordance correlation coefficients, J. Biopharm. Stat. 17 (2007) 685-696]. It also provides bootstrap confidence intervals for the CCC, as well as for the difference in CCCs for both independent and dependent samples. The macro is designed for balanced data only. Detailed explanation of the involved computations and macro variable definitions are provided in the text. Two biomedical examples are included to illustrate that the macro can be easily implemented.