Systems and Experiment Paper: Construction of Panoramic Image Mosaics with Global and Local Alignment

This paper presents a complete system for constructing panoramic image mosaics from sequences of images. Our mosaic representation associates a transformation matrix with each input image, rather than explicitly projecting all of the images onto a common surface (e.g., a cylinder). In particular, to construct a full view panorama, we introduce a rotational mosaic representation that associates a rotation matrix (and optionally a focal length) with each input image. A patch-based alignment algorithm is developed to quickly align two images given motion models. Techniques for estimating and refining camera focal lengths are also presented.In order to reduce accumulated registration errors, we apply global alignment (block adjustment) to the whole sequence of images, which results in an optimally registered image mosaic. To compensate for small amounts of motion parallax introduced by translations of the camera and other unmodeled distortions, we use a local alignment (deghosting) technique which warps each image based on the results of pairwise local image registrations. By combining both global and local alignment, we significantly improve the quality of our image mosaics, thereby enabling the creation of full view panoramic mosaics with hand-held cameras.We also present an inverse texture mapping algorithm for efficiently extracting environment maps from our panoramic image mosaics. By mapping the mosaic onto an arbitrary texture-mapped polyhedron surrounding the origin, we can explore the virtual environment using standard 3D graphics viewers and hardware without requiring special-purpose players.An erratum to this article can be found at     

一种用于动态场景的全景表示方法

针对全景图无法表示动态场景这一问题，提出一种用于动态场景的全景图表示方法，将视频纹理和全景图结合起来，构造动态全景图。系统首先将一系列定点拍摄的图像拼接成全景图，然后用摄像机拍摄场景中周期或随机运动的物体，提取视频纹理，最后视频纹理与全景图对准并融合，生成动态全景图。动态全景图既保持静态全景图全视角漫游的优点，又使得场景具有动态的特征，极大地增强漫游的真实感。     

基于Hu向量相似度的图像序列自动排序算法

为了实现全景图拼接中图像序列的自动排序,提出了一种利用图像Hu矩向量相似度的图像序列全自动排序算法。首先利用图像的七个Hu矩构造一个矩向量,然后计算两个矩向量的欧氏距离,用以表示两幅图像的相似度,最后根据图像序列中各图像之间的相似度实现全自动排序。实验结果表明,该算法能有效地实现顺序混乱的图像序列的自动排序;同时,对图像序列中存在旋转及尺寸大小不同的特殊情况也能很好地处理。     

一种基于特征块的图像拼接算法

提出一种基于基本特征块的图像拼接算法。该方法先定义边缘特征量,逐点计算各像素点邻域的边缘信息量大小,并采用局部非极大值抑制及阈值法进行排除,最终确定基本特征块;先利用基本特征块的位置信息初步估计模型参数,再提取特征点对模型参数进行修正;最后,对图像进行拼接获得更大视野更高分辨率的拼接图。实验表明,该算法运算量较小,但可保证一定的精确度,具有一定的可行性和适用性。     

Adaptive Compositing and Navigation of Variable Resolution Images

We present a new, high‐quality compositing pipeline and navigation approach for variable resolution imagery. The motivation of this work is to explore the use of variable resolution images as a quick and accessible alternative to traditional gigapixel mosaics. Instead of the common tedious acquisition of many images using specialized hardware, variable resolution images can achieve similarly deep zooms as large mosaics, but with only a handful of images. For this approach to be a viable alternative, the state‐of‐the‐art in variable resolution compositing needs to be improved to match the high‐quality approaches commonly used in mosaic compositing. To this end, we provide a novel, variable resolution mosaic seam calculation and gradient domain color correction. This approach includes a new priority order graph cuts computation along with a practical data structure to keep memory overhead low. In addition, navigating variable resolution images is challenging, especially at the zoom factors targeted in this work. To address this challenge, we introduce a new image interaction for variable resolution imagery: a pan that automatically, and smoothly, hugs available resolution. Finally, we provide several real‐world examples of our approach producing high‐quality variable resolution mosaics with deep zooms typically associated with gigapixel photography.     

一种基于多尺度小波分解的图象镶嵌技术

为了获得一种具有艺术视觉效果的镶嵌图象,提出了一种基于多尺度小波分解的图象镶嵌技术,该技术首先对原始图象的各子块区域和图象库中的每一幅图象进行多尺度小波分解;然后逐层计算各图象小波分解系数的标准方差和它们之间的相似距离,并据此从图象库中选取与原始图象中的各子块区域最佳匹配的贴图,再将其镶嵌到原始图象中的对应区域;最后对贴图进行逐像素的颜色校正,使贴图的颜色与原始图象尽可能一致.同时,根据人眼观察某个区域时往往通过取整或将细节取平均来得到一个总体强度效果这一视觉特性,使最终的镶嵌图象具有在近处看到的是各个贴图的内容,而在远处观看则是原始图象的总体轮廓的视觉效果.利用计算机来自动地生成这种镶嵌图象的实验结果表明,该方法是有效的.     

不同季相SPOT5影像镶嵌前色调处理方法研究

只有少数影像可供选择时,如何镶嵌季相差异大的影像,以取得较好的镶嵌效果,是一个很常见的技术问题。该问题的难点主要在于如何减小镶嵌影像间色调的差异。以厦门市两景不同年份、不同季相、色调差异很大的SPOT5卫星影像为例,采用数值调整,栅格编辑填充和特征信息提取、分类等图像处理方法,对影像上色调差异很大的斑块状地物和不规则零碎地物进行处理,达到了减小两景影像中同一地物特征色调差异的目的。文中提出的均色方法可以改善影像镶嵌效果,具有较强的实用性。     

无人机航拍图像拼接重影消除技术研究

为了解决拼接图像中的模糊、畸形,运动物体图像拼接后会出现重影错位等问题,首先在图像配准阶段通过感知哈希值相似度量准则选取变换效果最佳的权重ωi,j来计算局部投影变换矩阵;然后利用帧间差分法和区域生长算法对差异图像中的运动物体进行分割,对差异性区域进行单采样,其他区域采用线性加权融合;最后,构建了一个用于图像拼接的航空影像数据集并进行实验。实验结果证明,该方法运行耗时增加量较小,不仅有效抑制了重叠区域出现的拼接模糊、严重畸形现象,还能消除包含运动物体图像出现的重影错位,显著提高无人机航拍图像的拼接质量。     

图像拼接的改进算法

图像拼接在制作全景图的过程中具有重要作用，但是现有的方法不能很好地处理鬼影和曝光差异，针对存在的问题，提出了一种实现图像拼接的算法．首先利用动态规划的方法在要进行拼接的图像中找到一条最佳的缝合线，然后利用多分辨率技术进行拼接，解决了鬼影和曝光差异。     

基于改进相位相关与特征点配准的多图拼接算法

针对拼接过程易受图像采集时曝光、尺度变化、旋转、环境噪声、光照等因素的影响,以及多图手动排序出错率高、耗时长等问题,提出了一种基于改进相位相关与特征点配准的多图拼接算法。首先,基于对数极坐标变换的改进相位相关算法来计算缩放、旋转和平移参数,根据冲激函数峰值实现多图自动排序;接着,在重叠位置提取Harris角点,改进的Ransac算法精确提纯匹配点对,优化变换矩阵以完成拼接;最后,通过利用NSCT变换算法多尺度分解低频、高频子带来制定融合策略,从而解决接缝明显的问题。实验结果表明,新算法 建立的模型参数准确且高效,拼接融合效果过渡自然,能较好地解决复杂环境及乱序图像的拼接问题。     

多摄像机图像拼接自动色彩均衡算法

针对多摄像机视频图像,分析了图像拼接技术中存在的问题,讨论了目前图像处理中常用的图像色彩均衡算法,提出了一种基于图像像素均值统计的亮度和色彩均衡处理算法。首先提取相邻两摄像机同步帧图像的重叠区域并对重叠区图像进行通道分离（RGB）,把其中一幅作为参考图像,另一幅作为目标图像,分别统计两幅图像各颜色通道像素均值差,用差值强制修正整幅目标图像。然后对修正后的图像和参考图像（整幅图像）进行颜色空间转换（RGB到HSV）,再次统计两幅图像亮度通道（V通道）均值差,用差值强制修正整幅目标图像亮度。实验结果证明,该算法能有效校正相邻摄像机图像的亮度和色差,对后期的拼接融合处理起到了很好的改善效果。     

A fast mosaic method for airborne images: the new Template-Convolution Speed-Up Robust Features (TSURF) algorithm

In this paper, we present a new feature extraction algorithm termed Template-Convolution Speed-Up Robust Features (TSURF), which uses template convolution to extract points of interest based on the Speed-Up Robust Features (SURF) algorithm. Feature extraction is applied to extract corresponding characteristics in overlapping fields from adjacent images. The characteristics include area, line, and point features. As the point feature is easy to compute and is invariant to image scale, rotation, intensity, and so on, we use it to register and mosaic images widely. By filtering redundant points of interest, TSURF can greatly reduce the running time and keep the mosaic quality good during real-time mosaicking. SURF and TSURF are applied to airborne images to compare the efficiency of each algorithm in constructing a mosaic. We calculate the average coordinate error and angle error for evaluating mosaic precision. We also use the average gradient, standard deviation, and forecast root mean-square error to compare the mosaic quality of SURF with that of TSURF. The first mosaic experiment consisting of 20 groups of airborne images showed that the mosaic speed of TSURF is three times faster than that of SURF and maintains comparative mosaic accuracy. In the second experiment, a series of continuous thumbnail images were mosaicked using the SURF and TSURF algorithms fully automatically. The TSURF speed was 63.40% faster than that of SURF and the accuracy remained consistent.     

基于最佳缝合线的序列遥感图像拼接融合方法

在图像融合过程中常采用重叠区域像素加权融合方法,这会存在鬼影现象。为解决这一问题,提出了一种改进的最佳缝合线生成算法和沿最佳缝合线的融合方法,并将其应用于序列遥感图像的拼接融合。首先在重叠图像中,对重叠区域的边赋予权值,然后进行最大流最小割,最终获取最佳缝合线。在计算边权值时引入了图像梯度信息,使缝合线更准确。在图像融合时,沿最佳缝合线生成一个条带形融合区域,采用渐入渐出法对缝合线两侧的条带形融合区域图像进行过渡处理,使拼接后的图像更为真实。在序列遥感图像拼接上,采用捆绑调整算法调整拼接图像的参数来实现全局误差最小化。实验表明,该方法能够有效消除鬼影并且能获得准确的拼接图像与融合,对于序列遥感图像的拼接融合能够获得很好的效果。     

Optimal seamline detection in dynamic scenes via graph cuts for image mosaicking

In this paper, we present a novel method for creating a seamless mosaic from a set of geometrically aligned images captured from the scene with dynamic objects at different times. The artifacts caused by dynamic objects and geometric misalignments can be effectively concealed in our proposed seamline detection algorithm. In addition, we simultaneously compensate the image regions of dynamic objects based on the optimal seamline detection in the graph cuts energy minimization framework and create the mosaic with a relatively clean background. To ensure the high quality of the optimal seamline, the energy functions adopted in graph cuts combine the pixel-level similarities of image characteristics, including intensity and gradient, and the texture complexity. To successfully compensate the image regions covered by dynamic objects for creating a mosaic with a relatively clean background, we initially detect them in overlap regions between images based on pixel-level and region-level similarities, then refine them based on segments, and determine their image source in probability based on contour matching. We finally integrate all of these into the energy minimization framework to detect optimal seamlines. Experimental results on different dynamic scenes demonstrate that our proposed method is capable of generating high-quality mosaics with relatively clean backgrounds based on the detected optimal seamlines.     

一种微型零件的拼接方法

针对微型零件的特点,提出了一种结合图像灰度及轮廓信息的新拼接方法.该方法首先利用Canny算子得到图像的轮廓曲线,然后利用曲率尺度空间(CCS)检测图像的角点,然后利用相关法配准角点,得到图像的初步拼接,最后利用动态规划对图像轮廓进行精细拼接.本方法具有拼接精度高、自动化程度高、可靠性好的特点,适合高精度图像的拼接.     

多视点平行拼图

提出了一种新的多视点拼图,称之为平行拼图,平行拼图不仅可以应用到基于图像的绘制技术中,而且可以广泛地运用于基于图像的三维重建。该文设计了从真实照片中得到高分辨率平行拼图的算法,建立了从平行拼图得到高精度绘制输出的数学模型,讨论了用照片建立平行拼图的精度上限,并且给出了良好的实验结果。     

Optimized design of automatic image mosaic

Virtual reality technology has been widely used in the fields of aerospace, robotics remote operation and biology medicine and so on. Panoramic image mosaic is one of the very important parts. Since photographs taken by the ordinary camera may appear distorted, overlapping and tilting, we propose a wide mosaic algorithm used in the projection transformation in this paper. The algorithm first uses the Harris operator to extract corners, adopting the improved corner response function for avoiding the randomness of k value. Then fast RANSAC method is used to match the images approximately, and the cross-correlation method of gray window as the center of feature points is used to the redundant feature points for further exact match. And then it need solve the model transformation parameters between two images according to these corners information and obtain the projection transformation matrix. Finally, the application of image morphing technique is for reconstructing the image having spatial transform, the result of which are carried on stitching seamlessly with another source image. Experimental results show that the algorithm is effective to achieve a good mosaic.     

灰度投影在块匹配算法中的应用

运动参数估计在图像配准,电子稳像以及图像拼接中起着很重要的作用,其中块匹配算法是经常使用的一种方法。当平移较大的时候,块匹配算法仍然存在不足,因此本文提出了一种新的算法。首先利用灰度投影法得到相邻两幅图像的平移量,在此基础上再利用块匹配算法进行分块处理。实验说明了此方法的有效性,能够满足一般要求。     

针对大视差图像拼接的显性子平面配准

目的 针对图像拼接中大视差图像难以配准的问题,提出一种显性子平面自动配准算法。方法 假设大视差图像包含多个显性子平面且每个平面内所含特征点密集分布。对该假设进行了验证性实验。所提算法以特征点分布为依据,通过聚类算法实现子平面分割,进而对子平面进行局部配准。首先,使用层次聚类算法对已匹配的特征点聚类,通过一种本文设计的拼接误差确定分组数目,并以各组特征点的聚类中心为新的聚类中心对重叠区域再聚类,分割出目标图像的显性子平面。然后,求解每个显性子平面的投影参数,并采用就近原则分配非重叠区域的单应性矩阵。结果 采用公共数据集对本文算法进行测试,并与Auto-Stitching、微软Image Composite Editor两种软件及全局投影拼接方法（Baseline）、尽可能投影算法（APAP）进行对比,采用均方根误差作为配准精度的客观评判标准。实验结果表明,该算法在拼接大视差图像时,能有效地配准局部区域,解决软件和传统方法由误配准引起的鬼影、错位等问题。其均方根误差比Baseline方法平均减小55%左右。与APAP算法相比,均方根误差平均相差10%左右,但可视化配准效果相同且无需调节复杂参数,可实现自动配准。结论 提出的显性子平面自动配准算法,通过分割图像所含子平面进而实现局部配准。该方法具有较高的配准精度,在大视差图像配准方面,优于部分软件及算法,可应用于图像拼接中大视差图像的自动配准。     

Test of a new lacunarity estimation method for image texture analysis

Based on a differential box counting method and a gliding-box algorithm, a new method for estimating the lacunarity of grey scale digital image surfaces is introduced, and directionality of lacunarity defined. To test the performance of the new lacunarity measure, a Brodatz texture image mosaic is employed and several other texture analysis approaches are also applied to the texture mosaic. Quantitative comparison shows that the new lacunarity estimation method for grey-scale images can provide more accurate texture measurements than some existing lacunarity measures, the grey level co-occurrence matrix based texture measures, the Min-Max operator, and the fractal dimension.