Unsupervised manifold alignment using soft-assign technique

In this paper, we propose a robust unsupervised algorithm for automatic alignment of two manifolds in different datasets with possibly different dimensionalities. The significant contribution is that the proposed alignment algorithm is performed automatically without any assumptions on the correspondences between the two manifolds. For such purpose, we first automatically extract local feature histograms at each point of the manifolds and establish an initial similarity between the two datasets by matching their histogram-based features. Based on such similarity, an embedding space is estimated where the distance between the two manifolds is minimized while maximally retaining the original structure of the manifolds. The elegance of this idea is that such complicated problem is formulated as a generalized eigenvalue problem, which can be easily solved. The alignment process is achieved by iteratively increasing the sparsity of correspondence matrix until the two manifolds are correctly aligned and consequently one can reveal their joint structure. We demonstrate the effectiveness of our algorithm on different datasets by aligning protein structures, 3D face models and facial images of different subjects under pose and lighting variations. Finally, we also compare with a state-of-the-art algorithm and the results show the superiority of the proposed manifold alignment in terms of vision effect and numerical accuracy.