Fast topology estimation for image mosaicing using adaptive information thresholding

Over the past decade, several image mosaicing methods have been proposed in robotic mapping and remote sensing applications. Owing to rapid developments in obtaining optical data from areas beyond human reach, there is a high demand from different science fields for creating large-area image mosaics, often using images as the only source of information. One of the most important steps in the mosaicing process is motion estimation between overlapping images to obtain the topology, i.e., the spatial relationships between images.In this paper, we propose a generic framework for feature-based image mosaicing capable of obtaining the topology with a reduced number of matching attempts and of getting the best possible trajectory estimation. Innovative aspects include the use of a fast image similarity criterion combined with a Minimum Spanning Tree (MST) solution, to obtain a tentative topology and information theory principles to decide when to update trajectory estimation. Unlike previous approaches for large-area mosaicing, our framework is able to naturally deal with the cases where time-consecutive images cannot be matched successfully, such as completely unordered sets. This characteristic also makes our approach robust to sensor failure. The performance of the method is illustrated with experimental results obtained from different challenging underwater image sequences.     

基于流行排序的多示例图像检索方法

为了提高图像检索的性能,提出了一种基于流行排序的多示例图像检索方法,将分割后的图像表示为多示例的形式,通过给出适合图像在包空间的度量方式,有效结合流行排序和多示例学习的方法来进行图像检索.实验结果表明,采用所提出的方法的检索结果与传统的检索方法相比,检索率得到了明显的提高,检索结果更符合人的视觉习惯.     

Manifold Ranking using Hessian Energy

In recent years, learning on manifolds has attracted much attention in the academia community. The idea that the distribution of real-life data forms a low dimensional manifold embedded in the ambient space works quite well in practice, with applications such as ranking, dimensionality reduction, semi-supervised learning and clustering. This paper focuses on ranking on manifolds. Traditional manifold ranking methods try to learn a ranking function that varies smoothly along the data manifold by using a Laplacian regularizer. However, the Laplacian regularization suffers from the issue that the solution is biased towards constant functions. In this work, we propose using second-order Hessian energy as regularization for manifold ranking. Hessian energy overcomes the above issue by only penalizing accelerated variation of the ranking function along the geodesics of the data manifold. We also develop a manifold ranking framework for general graphs/hypergraphs for which we do not have an original feature space (i.e. the ambient space). We evaluate our ranking method on the COREL image dataset and a rich media dataset crawled from Last.fm. The experimental results indicate that our manifold ranking method is effective and outperforms traditional graph Laplacian based ranking method.     

一种基于流行排序的区域图像检索方法*

提出了一种结合多示例学习和流行排序的图像检索方法,将图像检索作为多示例学习框架下的流行排序,通过给出适合图像在包空间的有效度量方式,将流行排序的方法和多示例学习有效结合起来,从而获得更准确的检索结果。实验结果表明,运用流行排序的区域图像检索方法是可行的,同时,检索结果与传统的区域图像检索方法相比,检索率得到了明显的提高。     

Mosaicing on adaptive manifolds

Image mosaicing is commonly used to increase the visual field of view by pasting together many images or video frames. Existing mosaicing methods are based on projecting all images onto a predetermined single manifold: A plane is commonly used for a camera translating sideways, a cylinder is used for a panning camera, and a sphere is used for a camera which is both panning and tilting. While different mosaicing methods should therefore be used for different types of camera motion, more general types of camera motion, such as forward motion, are practically impossible for traditional mosaicing. A new methodology to allow image mosaicing in more general cases of camera motion is presented. Mosaicing is performed by projecting thin strips from the images onto manifolds which are adapted to the camera motion. While the limitations of existing mosaicing techniques are a result of using predetermined manifolds, the use of more general manifolds overcomes these limitations.     

Manifold Based Low-Rank Regularization for Image Restoration and Semi-Supervised Learning

Low-rank structures play important roles in recent advances of many problems in image science and data science. As a natural extension of low-rank structures for data with nonlinear structures, the concept of the low-dimensional manifold structure has been considered in many data processing problems. Inspired by this concept, we consider a manifold based low-rank regularization as a linear approximation of manifold dimension. This regularization is less restricted than the global low-rank regularization, and thus enjoy more flexibility to handle data with nonlinear structures. As applications, we demonstrate the proposed regularization to classical inverse problems in image sciences and data sciences including image inpainting, image super-resolution, X-ray computer tomography image reconstruction and semi-supervised learning. We conduct intensive numerical experiments in several image restoration problems and a semi-supervised learning problem of classifying handwritten digits using the MINST data. Our numerical tests demonstrate the effectiveness of the proposed methods and illustrate that the new regularization methods produce outstanding results by comparing with many existing methods.     

Rectified Surface Mosaics

We approach mosaicing as a camera tracking problem within a known parameterized surface. From a video of a camera moving within a surface, we compute a mosaic representing the texture of that surface, flattened onto a planar image. Our approach works by defining a warp between images as a function of surface geometry and camera pose. Globally optimizing this warp to maximize alignment across all frames determines the camera trajectory, and the corresponding flattened mosaic image. In contrast to previous mosaicing methods which assume planar or distant scenes, or controlled camera motion, our approach enables mosaicing in cases where the camera moves unpredictably through proximal surfaces, such as in medical endoscopy applications.     

Manifold splines with a single extraordinary point

This paper develops a novel computational technique to define and construct manifold splines with only one singular point by employing the rigorous mathematical theory of Ricci flow. The central idea and new computational paradigm of manifold splines are to systematically extend the algorithmic pipeline of spline surface construction from any planar domain to an arbitrary topology. As a result, manifold splines can unify planar spline representations as their special cases. Despite its earlier success, the existing manifold spline framework is plagued by the topology-dependent, large number of singular points (i.e., |2g−2| for any genus-g surface), where the analysis of surface behaviors such as continuity remains extremely difficult. The unique theoretical contribution of this paper is that we devise new mathematical tools so that manifold splines can now be constructed with only one singular point, reaching their theoretic lower bound of singularity for real-world applications. Our new algorithm is founded upon the concept of discrete Ricci flow and associated techniques. First, Ricci flow is employed to compute a special metric of any manifold domain (serving as a parametric domain for manifold splines), such that the metric becomes flat everywhere except at one point. Then, the metric naturally induces an affine atlas covering the entire manifold except this singular point. Finally, manifold splines are defined over this affine atlas. The Ricci flow method is theoretically sound, and practically simple and efficient. We conduct various shape experiments and our new theoretical and algorithmic results alleviate the modeling difficulty of manifold splines, and hence, promote the widespread use of manifold splines in surface and solid modeling, geometric design, and reverse engineering.     

结合流形排序和区域匹配的图像检索

给出一种基于数据流形排序(Manifold Ranking)和分割区域匹配的图像检索方法.在Manifold Ranking方法的基础上,提出区域匹配图(Region Matching Graph,RMG)的方法,通过计算图像的区域匹配权值,进行第二次相似性匹配,提高了匹配准确性.在Corel图像数据库对该方法进行了检索仿真,结果表明该方法能有效提高检索的准确性.     

面向旋转机械故障诊断的深度流形迁移学习

深度学习因强大的特征提取能力已逐渐成为旋转机械故障诊断的主要方法。但深层模型缺乏领域适应能力,工况变化时性能衰退严重。迁移学习为解决变工况诊断问题提供新的途径。然而现有深度迁移学习方法大多仅对齐不同领域分布的均值中心,未考虑特征分布的流形结构,其适配性能仍难以应对不同工况复杂的机械故障信号。针对该问题,提出一种深度流形迁移学习方法,以堆叠自编码器为框架,在无监督预训练阶段同时利用源域和目标域样本训练,充分挖掘数据本质特征;针对模型微调,提出流行迁移框架,在适配分布差异同时还保持领域间特征分布结构的一致性。将新方法与现有迁移学习方法在旋转机械故障诊断案例进行充分的比较实验,结果表明,新方法优于现有方法,能显著提高变工况故障诊断精度。通过有效性分析在机理上进一步证明了融合目标域数据的无监督预训练策略和流形迁移微调策略对提高变工况故障诊断的有效性。     

基于对应流形的错误匹配剔除研究

可靠对应点的建立是计算机视觉中的一个基本问题, 它是许多已有算法的前题假设. 本文提出了对应流形的概念, 并给出了通过学习对应流形的视图剔除错误匹配点的学习方法. 该方法不依赖于被估计的参数模型, 克服了传统方法计算效率随着错误匹配比例的增大或被估参数个数的增长而急剧下降的缺点; 同时弥补了错误匹配的剔除与被估模型的选择及其参数估计相耦合的不足. 实验结果表明该方法能有效地剔除错误匹配点, 验证了所给模型的适用性与合理性. 为可靠对应关系的建立提供一种新的高效的方法．     

基于相关反馈和流形结构重构的图像检索方法

当检索样例位于数据库之外时,传统基于线性流形学习的图像检索方法在反馈迭代后的检索精度提高较小。为此,提出一种基于相关反馈和流形结构重构的图像检索方法。反馈时计算被检索样例的最邻近点,将被检索样例重构入需要保留的结构图中,从而满足映射时需保证相似图像和被检索样例距离尽可能近的要求。实验结果表明,该方法在额外耗时较少的情况下,能有效提高检索精度。     

基于黎曼流形稀疏编码的图像检索算法

针对视觉词袋（Bag-of-visual-words,BOVW）模型直方图量化误差大的缺点,提出基于稀疏编码的图像检索算法.由于大多数图像特征属于非线性流形结构,传统稀疏编码使用向量空间对其度量必然导致不准确的稀疏表示.考虑到图像特征空间的流形结构,选择对称正定矩阵作为特征描述子,构建黎曼流形空间.利用核技术将黎曼流形结构映射到再生核希尔伯特空间,非线性流形转换为线性稀疏编码,获得图像更准确的稀疏表示.实验在Corel1000和Caltech101两个数据集上进行,与已有的图像检索算法对比,提出的图像检索算法不仅提高了检索准确率,而且获得了更好的检索性能.     

Essential Matrix Estimation Using Gauss-Newton Iterations on a Manifold

A novel approach for essential matrix estimation is presented, this being a key task in stereo vision processing. We estimate the essential matrix from point correspondences between a stereo image pair, assuming that the internal camera parameters are known. The set of essential matrices forms a smooth manifold, and a suitable cost function can be defined on this manifold such that its minimum is the desired essential matrix. We seek a computationally efficient optimization scheme towards meeting the demands of on-line processing of video images. Our work extends and improves the earlier research by Ma et al., who proposed an intrinsic Riemannian Newton method for essential matrix computations. In contrast to Ma et al., we propose three Gauss-Newton type algorithms that have improved convergence properties and reduced computational cost. The first one is based on a novel intrinsic Newton method, using the normal Riemannian metric on the manifold consisting of all essential matrices. The other two methods are Newton-like methods, that are more efficient from a numerical point of view. Local quadratic convergence of the algorithms is shown, based on a careful analysis of the underlying geometry of the problem.     

Manifold label prediction for low dimensional palmprint recognition

Minimum Squared Error Classification (MSEC) is a learning method for predicting the class labels of samples in real time. However, as a regression algorithm, MSEC tries its best to map the training samples into their class labels using a linear projection without considering the manifold structure of the data. In this paper, we introduce a supervised label learning framework using an effective manifold learning strategy. This method which is referred to as Manifold Supervised Label Prediction (MSLP) generalizes MSEC objective function to incorporate intra-class relationships of data. Thus, in addition to relying on the relationship between a training sample and its label, we propose to also learn the relationship between the training samples while transforming them. As a testbed for MSLP, we apply it to an image identification venue in which image samples with a very low spatial resolution (16 × 16) are used. These images have been dramatically influenced by a down-sampling process in order to reduce their size and hence, improving over computation time. We also show that the blurring process for reducing the artifacts introduced by down-sampling serendipitously results in better identification accuracies. Finally, unlike MSEC that classifies a query sample based on the deviation between the predicted and the true class labels, we compare both the training and the query samples in the label prediction space. A set of comprehensive experiments on benchmark palmprint databases including Multispectral PolyU, PolyU 2D/3D, and PolyU Contact-free I shows meaningful improvements over existing state-of-the-art algorithms.     

Human Age Estimation With Regression on Discriminative Aging Manifold

Recently, extensive studies on human faces in the human-computer interaction (HCI) field reveal significant potentials for designing automatic age estimation systems via face image analysis. The success of such research may bring in many innovative HCI tools used for the applications of human-centered multimedia communication. Due to the temporal property of age progression, face images with aging features may display some sequential patterns with low-dimensional distributions. In this paper, we demonstrate that such aging patterns can be effectively extracted from a discriminant subspace learning algorithm and visualized as distinct manifold structures. Through the manifold method of analysis on face images, the dimensionality redundancy of the original image space can be significantly reduced with subspace learning. A multiple linear regression procedure, especially with a quadratic model function, can be facilitated by the low dimensionality to represent the manifold space embodying the discriminative property. Such a processing has been evaluated by extensive simulations and compared with the state-of-the-art methods. Experimental results on a large size aging database demonstrate the effectiveness and robustness of our proposed framework.     

一种基于QPSO优化的流形学习的视频人脸识别算法

视频场景复杂多变,视频采集设备不一致等原因,导致无约束视频中充斥着大量的遮挡和人脸旋转,视频人脸识别方法的准确率不高且性能不稳定.为解决上述问题,本文提出了一种基于QPSO优化的流形学习的视频人脸识别算法.该算法将视频人脸识别视为图像集相似度度量问题,首先帧图像对齐后提取纹理特征并进行融合,再利用带有QPSO优化的黎曼流形大幅度简约维度以获得视频人脸的内在表示,相似度则由凸包距离表示,最后利用SVM分类器获得分类结果.通过在Youtube Face数据库和Honda/UCSD数据库上与当前主流算法进行的对比实验,验证了本文算法的有效性,所提算法识别精度较高,误差较低,并且对光照和表情变化具有较强的鲁棒性.     

融合模糊推理和流形正则化的特征迁移学习

迁移学习利用源域中丰富的数据来为目标域构建精确的模型提供辅助和支持。特征迁移学习是迁移学习中被广泛研究的一类技术,但是现有的特征迁移方法面临着如下的问题:一些已有的方法仅能实现线性的特征迁移学习,因此这些方法迁移学习的能力有限。另一类方法虽然能实现非线性特征迁移学习,但往往需要引进核技巧等策略,这使得特征迁移的过程难以理解。针对此,引入模糊推理技术,提出基于不确定推理规则的特征迁移方法。该方法基于模糊推理系统来实现特征迁移,并利用流形正则化技术来避免特征迁移过程中的信息损失。由于模糊系统具有很好的非线性建模能力以及基于规则的良好的解释性,因此提出的方法具有良好的非线性特征迁移能力,并易于对新特征进行理解。大量实验证明,该算法在跨域图像分类问题上可以明显优于已有的多种方法。