Image mosaicing using sequential bundle adjustment

We describe the construction of accurate panoramic mosaics from multiple images taken with a rotating camera, or alternatively of a planar scene. The novelty of the approach lies in (i) the transfer of photogrammetric bundle adjustment techniques to mosaicing; (ii) a new representation of image line measurements enabling the use of lines in camera self-calibration, including computation of the radial and other non-linear distortion; and (iii) the application of the variable state dimension filter to obtain efficient sequential updates of the mosaic as each image is added.We demonstrate that our method achieves better results than the alternative approach of optimising over pairs of images.     

Generalized mosaicing: polarization panorama

We present an approach to image the polarization state of object points in a wide field of view, while enhancing the radiometric dynamic range of imaging systems by generalizing image mosaicing. The approach is biologically inspired, as it emulates spatially varying polarization sensitivity of some animals. In our method, a spatially varying polarization and attenuation filter is rigidly attached to a camera. As the system moves, it senses each scene point multiple times, each time filtering it through a different filter polarizing angle, polarizance, and transmittance. Polarization is an additional dimension of the generalized mosaicing paradigm, which has recently yielded high dynamic range images and multispectral images in a wide field of view using other kinds of filters. The image acquisition is as easy as in traditional image mosaics. The computational algorithm can easily handle nonideal polarization filters (partial polarizers), variable exposures, and saturation in a single framework. The resulting mosaic represents the polarization state at each scene point. Using data acquired by this method, we demonstrate attenuation and enhancement of specular reflections and semi reflection separation in an image mosaic.     

Generalized Mosaicing: High Dynamic Range in a Wide Field of View

We present an approach that significantly enhances the capabilities of traditional image mosaicking. The key observation is that as a camera moves, it senses each scene point multiple times. We rigidly attach to the camera an optical filter with spatially varying properties, so that multiple measurements are obtained for each scene point under different optical settings. Fusing the data captured in the multiple images yields an image mosaic that includes additional information about the scene. We refer to this approach as generalized mosaicing. In this paper we show that this approach can significantly extend the optical dynamic range of any given imaging system by exploiting vignetting effects. We derive the optimal vignetting configuration and implement it using an external filter with spatially varying transmittance. We also derive efficient scene sampling conditions as well as ways to self calibrate the vignetting effects. Maximum likelihood is used for image registration and fusion. In an experiment we mounted such a filter on a standard 8-bit video camera, to obtain an image panorama with dynamic range comparable to imaging with a 16-bit camera.     

Generalized mosaicing: wide field of view multispectral imaging

We present an approach to significantly enhance the spectral resolution of imaging systems by generalizing image mosaicing. A filter transmitting spatially varying spectral bands is rigidly attached to a camera. As the system moves, it senses each scene point multiple times, each time in a different spectral band. This is an additional dimension of the generalized mosaic paradigm, which has demonstrated yielding high radiometric dynamic range images in a wide field of view, using a spatially varying density filter. The resulting mosaic represents the spectrum at each scene point. The image acquisition is as easy as in traditional image mosaics. We derive an efficient scene sampling rate, and use a registration method that accommodates the spatially varying properties of the filter. Using the data acquired by this method, we demonstrate scene rendering under different simulated illumination spectra. We are also able to infer information about the scene illumination. The approach was tested using a standard 8-bit black/white video camera and a fixed spatially varying spectral (interference) filter.     

Manifold Modeling and Its Application to Tubular Scene Manifold Mosaicing Algorithm

Manifold mosaicing is arguably the most important class of image mosaicing methods. The existing manifold mosaicing methods work reasonably well only for scenes with simple plane structure and for images taken in a usual way such that the camera??s motion direction is perpendicular to its optical axis. A novel manifold modeling theoretical framework is presented to unify the current image mosaicing methods. Based on this framework, an effective mosaicing algorithm is also proposed to mosaic tubular scenes which have failed most existing methods. We adopt a two-step shaping strategy to parallelize the optical flows and change the topological genus of the image strips by M?bius mappings and circular ring extending. We performed computational experiments via image sequences acquired from tubular scenes and obtained excellent panoramas. The theoretical framework and algorithms in this paper have immediate applications to many practical tubular scene mosaicing problems in medical imaging, industrial inspection, gaming, virtual reality and robotics.     

A new approach to real-time mosaicing of aerial images

We present a new image mosaicing technique that uses sequential aerial images captured from a camera and is capable of creating consistent large scale mosaics in real-time. To find the alignment of every new image, we use all the available images in the mosaic that have intersection with the new image instead of using only the previous one. To detect image intersections in an efficient manner, we utilize ‘Separating Axis Theorem’, a geometric tool from computer graphics which is used for collision detection. Moreover, after a certain number of images are added to the mosaic, a novel affine refinement procedure is carried out to increase global consistency. Finally, gain compensation and multi-band blending are optionally used as offline steps to compensate for photometric defects and seams caused by misregistrations. Proposed approach is tested on some public datasets and it is compared with two state-of-the-art algorithms. Results are promising and show the potential of our algorithm in various practical scenarios.     

Mosaicing on adaptive manifolds

Image mosaicing is commonly used to increase the visual field of view by pasting together many images or video frames. Existing mosaicing methods are based on projecting all images onto a predetermined single manifold: A plane is commonly used for a camera translating sideways, a cylinder is used for a panning camera, and a sphere is used for a camera which is both panning and tilting. While different mosaicing methods should therefore be used for different types of camera motion, more general types of camera motion, such as forward motion, are practically impossible for traditional mosaicing. A new methodology to allow image mosaicing in more general cases of camera motion is presented. Mosaicing is performed by projecting thin strips from the images onto manifolds which are adapted to the camera motion. While the limitations of existing mosaicing techniques are a result of using predetermined manifolds, the use of more general manifolds overcomes these limitations.     

Drift-Free Real-Time Sequential Mosaicing

We present a sequential mosaicing algorithm for a calibrated rotating camera which can for the first time build drift-free, consistent spherical mosaics in real-time, automatically and seamlessly even when previously viewed parts of the scene are re-visited. Our mosaic is composed of elastic triangular tiles attached to a backbone map of feature directions over the unit sphere built using a sequential EKF SLAM (Extend Kalman Filter Simultaneous Localization And Mapping) approach. This method represents a significant advance on previous mosaicing techniques which either require off-line optimization or which work in real-time but use local alignment of nearby images and ultimately drift. We demonstrate the system’s real-time performance with real-time mosaicing results from sequences with 360 degrees pan. The system shows good global mosaicing ability despite the challenging conditions: hand-held simple low-resolution webcam, varying natural outdoor illumination, and people moving in the scene.     

Image mosaicking for polyhedral scene and in particular singly visible surfaces

Image mosaic construction is about stitching together a number of images about the same scene to construct a single image with a larger field of view. The majority of the previous work was rooted at the use of a single image-to-image mapping termed planar homography for representing the imaged scene. However, the mapping is applicable only to cases where the imaged scene is either a single planar surface, or very distant from the cameras, or imaged under a pure rotation of the camera, and that greatly limits the range of applications of the mosaicking methods. This paper presents a novel mosaicking solution for scenes that are polyhedral (thus consisting of multiple surfaces) and that are pictured possibly in closed range of the camera. The solution has two major advantages. First, it requires only a few correspondences over the entire scene, not correspondences over every surface patch in it to work. Second, it conquers a seemingly impossible task—warping image data of surfaces that are visible in only one of the input images, which we refer to as the singly visible surfaces, to another viewpoint to constitute the mosaic there. We also provide a detail analysis of what determines whether a singly visible surface could be mosaicked or not. Experimental results on real image data are presented to illustrate the performance of the method.     

Robust Image Mosaicing of Soccer Videos using Self-Calibration and Line Tracking

In this paper we propose an accurate and robust image mosaicing method of soccer video taken from a rotating and zooming camera using line tracking and self-calibration. The mosaicing of soccer videos is not easy, because their playing fields are low textured and moving players are included in the fields. Our approach is to track line features on the playing fields. The line features are detected and tracked using a self-calibration technique for a rotating and zooming camera. To track line features efficiently, we propose a new line tracking algorithm, called camera parameter guided line tracking, which works even when the camera motion undergoes sudden changes. Since we do not need to know any model for scenes beforehand, the proposed algorithm can be easily extended to other video sources, as well as other sports videos. Experimental results show the accuracy and robustness of the algorithm. An application of mosaicing is also presented.     

Minimal Aspect Distortion (MAD) Mosaicing of Long Scenes

Long scenes can be imaged by mosaicing multiple images from cameras scanning the scene. We address the case of a video camera scanning a scene while moving in a long path, e.g. scanning a city street from a driving car, or scanning a terrain from a low flying aircraft. A robust approach to this task is presented, which is applied successfully to sequences having thousands of frames even when using a hand-held camera. Examples are given on a few challenging sequences. The proposed system consists of two components: (i) Motion and depth computation. (ii) Mosaic rendering. In the first part a “direct” method is presented for computing motion and dense depth. Robustness of motion computation has been increased by limiting the motion model for the scanning camera. An iterative graph-cuts approach, with planar labels and a flexible similarity measure, allows the computation of a dense depth for the entire sequence. In the second part a new minimal aspect distortion (MAD) mosaicing uses depth to minimize the geometrical distortions of long panoramic images. In addition to MAD mosaicing, interactive visualization using X-Slits is also demonstrated. This research was supported by the Israel Science Foundation. Video examples and high resolution images can be viewed in .     

Appearance-Guided Monocular Omnidirectional Visual Odometry for Outdoor Ground Vehicles

In this paper, we describe a real-time algorithm for computing the ego-motion of a vehicle relative to the road. The algorithm uses as input only those images provided by a single omnidirectional camera mounted on the roof of the vehicle. The front ends of the system are two different trackers. The first one is a homography-based tracker that detects and matches robust scale-invariant features that most likely belong to the ground plane. The second one uses an appearance-based approach and gives high-resolution estimates of the rotation of the vehicle. This planar pose estimation method has been successfully applied to videos from an automotive platform. We give an example of camera trajectory estimated purely from omnidirectional images over a distance of 400 m. For performance evaluation, the estimated path is superimposed onto a satellite image. In the end, we use image mosaicing to obtain a textured 2-D reconstruction of the estimated path.      

基于全局拼接的航拍图像拼接算法研究

研究航拍图像的拼接问题,提高图像拼接的准确度。由于当通过航拍获取的图像中部分图像相互存在重叠区域的比例不大时,造成拼接不准确。传统的区域的图像拼接算法无法将具有较小重叠区域的图像准确拼接。为了提高航拍图像拼接的准确率,提出一种全局的图像拼接算法,通过使用SIFT算法提取图像的SIFT特征点,根据位置误差最小的原则完成两幅图像的SIFT特征点匹配,最后利用整体最优化方法对拼接结果进行优化,采用全局特征点的拼接方法,可避免传统方法只利用重叠区域灰度特征而不能准确拼接低重叠度图像的问题。实验证明,改进方法利用图像的全局信息,准确地实现图像的拼接,取得了满意的结果。     

基于复值小波分解的图象拼合

图象拼合是一种基于图象的场景编码方法，它被很多基于图象的绘制IBR（image based rendering）系统采用，用来建立复杂的虚拟场景表示（例如，360°球面和柱面全景图、环境映照及高分辨率图象等）.基于复值小波多分辨率分解(Complex Wavelet Multiresolution Decomposition)提出了一种有效的图象拼合方法，它能同时地、逐步求精地对图象进行匹配和整合.首先，采用复值小波变换不仅可以保证全局优化的结果，还能够满足图象整合的规模不变和平移不变性.其次，基于多分辨     

一种基于运动特征的快速有效的视频镜头全拼图方法

由于全拼图能比关键帧提供更多的视觉信息,因此它已经成为视觉计算中一个重要的分析工具。为了提高全拼图的质量和拼图速度,提出了一种基于运动特征的快速有效的全拼图生成方法。该方法首先给定一个视频镜头,并基于运动相位熵的分析方法决定该镜头内容是否适合生成全拼图;然后,对于适合生成全拼图的镜头,通过构造全局运动路径的方法,仅需要挑选全部视频帧的一个子集用来生成高质量的全拼图。实验结果表明,与传统的全拼图方法相比,该新方法在提高全拼图的视觉质量的同时,显著地降低了计算时间。     

一种基于多尺度小波分解的图象镶嵌技术

为了获得一种具有艺术视觉效果的镶嵌图象,提出了一种基于多尺度小波分解的图象镶嵌技术,该技术首先对原始图象的各子块区域和图象库中的每一幅图象进行多尺度小波分解;然后逐层计算各图象小波分解系数的标准方差和它们之间的相似距离,并据此从图象库中选取与原始图象中的各子块区域最佳匹配的贴图,再将其镶嵌到原始图象中的对应区域;最后对贴图进行逐像素的颜色校正,使贴图的颜色与原始图象尽可能一致.同时,根据人眼观察某个区域时往往通过取整或将细节取平均来得到一个总体强度效果这一视觉特性,使最终的镶嵌图象具有在近处看到的是各个贴图的内容,而在远处观看则是原始图象的总体轮廓的视觉效果.利用计算机来自动地生成这种镶嵌图象的实验结果表明,该方法是有效的.     

病理切片图像拼接修正技术

图像拼接技术具有广泛的应用性,是当前图像处理领域的研究热点.由于技术原因,在拼接时会造成误配.目前一般利用全局优化的方法进行图像拼接修正,理论上较为合理,但是速度和效果方面都欠佳.针对图像在拼接过程中,位置之间的相关性,提出一种基于单元拼接理论的修正方法.提出拼接参数的相关公式,分析并证明公式的正确性,并给出修正简化规...     

边缘检测法在晶圆缺陷检测中的应用

边缘检测技术是一种重要的图像预处理技术,是图像拼接前必不可少的环节。文章提出了一种新方法-水平垂直边缘检测法,其目的是为Hough直线检测提供准备,最终是为了确定图像拼接的基准线,该方法对医学显微图像的拼接,印刷行业、电路板设计等工序的缺陷检测具有一定的现实意义。     

基于匹配与平差的景象镶嵌方法

在概括介绍消附相邻景象辐射差异常用算法的基础上,为有效消除相邻景象间几何位置的差异,提出了一种将影象匹配与平差理论相结合的影象镶嵌新算法,该方法是在镶嵌影象的重叠部分进行基于特征的影象匹配,即首先获取一序列同名点,且这些同名点的匹配精度在一个象元以内;然后用正形多项式模拟这些同名点间的几何位置差异,再按最小二乘法原理对右边影象进行平差改正,以实现两影象空间位置上的无缝连接。实验结果表明,庐镶嵌方法具有良好的效果。     

Structure and event mining in sports video with efficient mosaic

Video is an information-intensive media with much redundancy. Therefore, it is desirable to be able to mine structure or semantics of video data for efficient browsing, summarization and highlight extraction. In this paper, we propose a mosaic based approach to key-event as well as structure mining, which is regarded as a complementary view for sports video analysis. Mosaic is generated for each shot by a novel efficient mosaicing scheme, which constructs a global motion path and selects a best subset of frames for mosaicing. These improved mosaics are then used as the representative image of shot content. Based on mosaic, the structure and event in sports video are mined by the methods with prior knowledge and without prior knowledge. Without prior knowledge, our system is able to locate global view shots taken by dominant camera. If prior knowledge is available, the events in these global view shots are detected using robust features extracted from mosaics. For global view mining, the experiments compared with key-frame-based scheme have demonstrated that this mosaic-based scheme presents better results in several kinds of sports videos; for events mining, the detection of key-plays and key-events in the specific-domain of soccer videos have proved its effectiveness.