Generalized mosaicing: wide field of view multispectral imaging

We present an approach to significantly enhance the spectral resolution of imaging systems by generalizing image mosaicing. A filter transmitting spatially varying spectral bands is rigidly attached to a camera. As the system moves, it senses each scene point multiple times, each time in a different spectral band. This is an additional dimension of the generalized mosaic paradigm, which has demonstrated yielding high radiometric dynamic range images in a wide field of view, using a spatially varying density filter. The resulting mosaic represents the spectrum at each scene point. The image acquisition is as easy as in traditional image mosaics. We derive an efficient scene sampling rate, and use a registration method that accommodates the spatially varying properties of the filter. Using the data acquired by this method, we demonstrate scene rendering under different simulated illumination spectra. We are also able to infer information about the scene illumination. The approach was tested using a standard 8-bit black/white video camera and a fixed spatially varying spectral (interference) filter.     

Generalized Mosaicing: High Dynamic Range in a Wide Field of View

We present an approach that significantly enhances the capabilities of traditional image mosaicking. The key observation is that as a camera moves, it senses each scene point multiple times. We rigidly attach to the camera an optical filter with spatially varying properties, so that multiple measurements are obtained for each scene point under different optical settings. Fusing the data captured in the multiple images yields an image mosaic that includes additional information about the scene. We refer to this approach as generalized mosaicing. In this paper we show that this approach can significantly extend the optical dynamic range of any given imaging system by exploiting vignetting effects. We derive the optimal vignetting configuration and implement it using an external filter with spatially varying transmittance. We also derive efficient scene sampling conditions as well as ways to self calibrate the vignetting effects. Maximum likelihood is used for image registration and fusion. In an experiment we mounted such a filter on a standard 8-bit video camera, to obtain an image panorama with dynamic range comparable to imaging with a 16-bit camera.     

Image mosaicing using sequential bundle adjustment

We describe the construction of accurate panoramic mosaics from multiple images taken with a rotating camera, or alternatively of a planar scene. The novelty of the approach lies in (i) the transfer of photogrammetric bundle adjustment techniques to mosaicing; (ii) a new representation of image line measurements enabling the use of lines in camera self-calibration, including computation of the radial and other non-linear distortion; and (iii) the application of the variable state dimension filter to obtain efficient sequential updates of the mosaic as each image is added.We demonstrate that our method achieves better results than the alternative approach of optimising over pairs of images.     

基于对数极坐标映射的图像拼接方法

图像拼接在基于图像的绘制、视频检索以及景物匹配等领域有着广泛的应用，为了获取大画面宽视场的场景表示，针对存在旋转及缩放变化的图像，提出了一种基于对数极坐标映射的图像拼接方法，该方法先将图像从笛卡儿坐标空间转换到对数极坐标空间，使得笛卡儿坐标空间中图像的旋转和缩放转换为对数极坐标空间中图像的二维平移，这样可直接利用相位相关法来估算出图像间的旋转角度和缩放因子，然后以此作为初值，再采用非线性最小化优化算法进一步求精图像间的运动参数来实现图像的配准，最后通过图像融合来实现图像的拼接。实验结果表明，该方法是有效的。     

大场景的视频全景图拼接

为了解决大场景全景图拼接过程中的误差累积问题,提高图像拼接的一致性,提出了一个新的全景图拼接算法。把全局图像配准看作是一个统计估计问题,运用增广的卡尔曼滤波对图像的全局变换参数进行递归地增广和估计。实验结果表明,得到的变换参数是全局一致的,生成的全景图不存在拼接裂缝。拼接算法考虑了各种不确定性,适用于噪声大场景的全景图拼接。     

一种基于多尺度小波分解的图象镶嵌技术

为了获得一种具有艺术视觉效果的镶嵌图象,提出了一种基于多尺度小波分解的图象镶嵌技术,该技术首先对原始图象的各子块区域和图象库中的每一幅图象进行多尺度小波分解;然后逐层计算各图象小波分解系数的标准方差和它们之间的相似距离,并据此从图象库中选取与原始图象中的各子块区域最佳匹配的贴图,再将其镶嵌到原始图象中的对应区域;最后对贴图进行逐像素的颜色校正,使贴图的颜色与原始图象尽可能一致.同时,根据人眼观察某个区域时往往通过取整或将细节取平均来得到一个总体强度效果这一视觉特性,使最终的镶嵌图象具有在近处看到的是各个贴图的内容,而在远处观看则是原始图象的总体轮廓的视觉效果.利用计算机来自动地生成这种镶嵌图象的实验结果表明,该方法是有效的.     

基于块匹配的鲁棒视频图像镶嵌算法

为了获取宽视野的场景表示,提出了一种基于块匹配的视频图像镶嵌算法,该算法首先采用基于相位相关的块匹配方法估计出视频图像间的运动矢量场,并剔除其中由于图像噪声或运动物体的遮挡而导致外点运动矢量,然后根据图像的运动矢量场确定出图像子块之间的对应点对,进而利用得到的对应点对迭代求解图像间的变换模型参数以实现视频图像的自动镶嵌.针对真实场景的视频图像序列进行实验,获得了较好的镶嵌结果,表明了该算法的有效性.     

Drift-Free Real-Time Sequential Mosaicing

We present a sequential mosaicing algorithm for a calibrated rotating camera which can for the first time build drift-free, consistent spherical mosaics in real-time, automatically and seamlessly even when previously viewed parts of the scene are re-visited. Our mosaic is composed of elastic triangular tiles attached to a backbone map of feature directions over the unit sphere built using a sequential EKF SLAM (Extend Kalman Filter Simultaneous Localization And Mapping) approach. This method represents a significant advance on previous mosaicing techniques which either require off-line optimization or which work in real-time but use local alignment of nearby images and ultimately drift. We demonstrate the system’s real-time performance with real-time mosaicing results from sequences with 360 degrees pan. The system shows good global mosaicing ability despite the challenging conditions: hand-held simple low-resolution webcam, varying natural outdoor illumination, and people moving in the scene.     

基于WiFi的分布式监视场景拼接系统

随着数字化信息技术的不断发展,监视系统已经成为各行业领域中的重要应用.随着WiFi等无线技术的发展,实现了视频数据的无线传输,因其特有的便捷性将传统的有线监视系统取而代之.图像拼接是目前图像处理的一项热门技术,将两幅或者多幅具有重叠区域的图像拼接成一幅无缝隙高分辨率的图像.本论文结合WiFi无线技术和图像拼接技术的优势,提出了基于WiFi的分布式监视场景拼接系统,多个客户端采集的图像通过WiFi无线网络传输到服务器,服务器对传输过来的图像采用SIFT算法和RANSAC算法完成拼接,最终实现了宽场景监视.实验结果表明该系统可以有效地实现监视场景拼接.     

Rectified Surface Mosaics

We approach mosaicing as a camera tracking problem within a known parameterized surface. From a video of a camera moving within a surface, we compute a mosaic representing the texture of that surface, flattened onto a planar image. Our approach works by defining a warp between images as a function of surface geometry and camera pose. Globally optimizing this warp to maximize alignment across all frames determines the camera trajectory, and the corresponding flattened mosaic image. In contrast to previous mosaicing methods which assume planar or distant scenes, or controlled camera motion, our approach enables mosaicing in cases where the camera moves unpredictably through proximal surfaces, such as in medical endoscopy applications.     

Manifold Modeling and Its Application to Tubular Scene Manifold Mosaicing Algorithm

Manifold mosaicing is arguably the most important class of image mosaicing methods. The existing manifold mosaicing methods work reasonably well only for scenes with simple plane structure and for images taken in a usual way such that the camera??s motion direction is perpendicular to its optical axis. A novel manifold modeling theoretical framework is presented to unify the current image mosaicing methods. Based on this framework, an effective mosaicing algorithm is also proposed to mosaic tubular scenes which have failed most existing methods. We adopt a two-step shaping strategy to parallelize the optical flows and change the topological genus of the image strips by M?bius mappings and circular ring extending. We performed computational experiments via image sequences acquired from tubular scenes and obtained excellent panoramas. The theoretical framework and algorithms in this paper have immediate applications to many practical tubular scene mosaicing problems in medical imaging, industrial inspection, gaming, virtual reality and robotics.     

Spatially varying image based lighting by light probe sequences

We present a novel technique for capturing spatially or temporally resolved light probe sequences, and using them for image based lighting. For this purpose we have designed and built a real-time light probe, a catadioptric imaging system that can capture the full dynamic range of the lighting incident at each point in space at video frame rates, while being moved through a scene. The real-time light probe uses a digital imaging system which we have programmed to capture high quality, photometrically accurate color images of 512×512 pixels with a dynamic range of 10000000:1 at 25 frames per second. By tracking the position and orientation of the light probe, it is possible to transform each light probe into a common frame of reference in world coordinates, and map each point and direction in space along the path of motion to a particular frame and pixel in the light probe sequence. We demonstrate our technique by rendering synthetic objects illuminated by complex real world lighting, first by using traditional image based lighting methods and temporally varying light probe illumination, and second an extension to handle spatially varying lighting conditions across large objects and object motion along an extended path.     

Enhancing resolution along multiple imaging dimensions using assorted pixels

Multisampled imaging is a general framework for using pixels on an image detector to simultaneously sample multiple dimensions of imaging (space, time, spectrum, brightness, polarization, etc.). The mosaic of red, green, and blue spectral filters found in most solid-state color cameras is one example of multisampled imaging. We briefly describe how multisampling can be used to explore other dimensions of imaging. Once such an image is captured, smooth reconstructions along the individual dimensions can be obtained using standard interpolation algorithms. Typically, this results in a substantial reduction of resolution (and, hence, image quality). One can extract significantly greater resolution in each dimension by noting that the light fields associated with real scenes have enormous redundancies within them, causing different dimensions to be highly correlated. Hence, multisampled images can be better interpolated using local structural models that are learned offline from a diverse set of training images. The specific type of structural models we use are based on polynomial functions of measured image intensities. They are very effective as well as computationally efficient. We demonstrate the benefits of structural interpolation using three specific applications. These are 1) traditional color imaging with a mosaic of color filters, 2) high dynamic range monochrome imaging using a mosaic of exposure filters, and 3) high dynamic range color imaging using a mosaic of overlapping color and exposure filters.     

一种用于动态场景的全景表示方法

针对全景图无法表示动态场景这一问题，提出一种用于动态场景的全景图表示方法，将视频纹理和全景图结合起来，构造动态全景图。系统首先将一系列定点拍摄的图像拼接成全景图，然后用摄像机拍摄场景中周期或随机运动的物体，提取视频纹理，最后视频纹理与全景图对准并融合，生成动态全景图。动态全景图既保持静态全景图全视角漫游的优点，又使得场景具有动态的特征，极大地增强漫游的真实感。     

New research and explorations into multiuser immersive display systems

The basic design challenge is to create a system that can display multiple images in a common area, occluding all but the appropriate pair of these images for each user. Thus, the system would deliver a unique image for each eyeball viewing the scene. Solutions fall into four general categories: Spatial barriers use the display's physical configuration and user placement to block users from seeing each other's view. Optical filtering involves systems that filter viewpoints using light's electromagnetic properties, such as polarization or wavelength. Optical routing uses the angle-sensitive optical characteristics of certain materials to direct or occlude images based on the user's position. Time multiplexing solutions use time-sequenced light and shutters to determine which user sees an image at a given point in time. Systems also can mix solutions from these categories. Time multiplexing, for example, could serve to create stereoscopic images, with spatial barriers employed to ensure that each user sees only the correct image.     

基于全局拼接的航拍图像拼接算法研究

研究航拍图像的拼接问题,提高图像拼接的准确度。由于当通过航拍获取的图像中部分图像相互存在重叠区域的比例不大时,造成拼接不准确。传统的区域的图像拼接算法无法将具有较小重叠区域的图像准确拼接。为了提高航拍图像拼接的准确率,提出一种全局的图像拼接算法,通过使用SIFT算法提取图像的SIFT特征点,根据位置误差最小的原则完成两幅图像的SIFT特征点匹配,最后利用整体最优化方法对拼接结果进行优化,采用全局特征点的拼接方法,可避免传统方法只利用重叠区域灰度特征而不能准确拼接低重叠度图像的问题。实验证明,改进方法利用图像的全局信息,准确地实现图像的拼接,取得了满意的结果。     

基于匹配与平差的景象镶嵌方法

在概括介绍消附相邻景象辐射差异常用算法的基础上,为有效消除相邻景象间几何位置的差异,提出了一种将影象匹配与平差理论相结合的影象镶嵌新算法,该方法是在镶嵌影象的重叠部分进行基于特征的影象匹配,即首先获取一序列同名点,且这些同名点的匹配精度在一个象元以内;然后用正形多项式模拟这些同名点间的几何位置差异,再按最小二乘法原理对右边影象进行平差改正,以实现两影象空间位置上的无缝连接。实验结果表明,庐镶嵌方法具有良好的效果。     

宽基线视差图像的拼接算法

针对两幅视差图像的拼接问题,提出了一种新算法,即利用Hessian仿射不变检测算子检测出特征区域,利用SIFT特征描述算子提取特征区域特征矢量,根据特征矢量的欧几里德距离来建立图像间的稀疏对应关系;由这些对应点稀疏地确定场景中的一些点,以这些点为顶点建立场景的三角面片近似,再据此将重叠区域重投影生成推扫式成像的中间部分图像。将中间部分推扫式成像图像和原左图像的左半部分以及原右图像的右半部分一起拼接生成大图像。利用实际图像进行的拼接实验表明该算法是一个有效的视差图像拼接算法。     

Learning Explicit Smoothing Kernels for Joint Image Filtering

Smoothing noises while preserving strong edges in images is an important problem in image processing. Image smoothing filters can be either explicit (based on local weighted average) or implicit (based on global optimization). Implicit methods are usually time‐consuming and cannot be applied to joint image filtering tasks, i.e., leveraging the structural information of a guidance image to filter a target image. Previous deep learning based image smoothing filters are all implicit and unavailable for joint filtering. In this paper, we propose to learn explicit guidance feature maps as well as offset maps from the guidance image and smoothing parameter that can be utilized to smooth the input itself or to filter images in other target domains. We design a deep convolutional neural network consisting of a fully‐convolution block for guidance and offset maps extraction together with a stacked spatially varying deformable convolution block for joint image filtering. Our models can approximate several representative image smoothing filters with high accuracy comparable to state‐of‐the‐art methods, and serve as general tools for other joint image filtering tasks, such as color interpolation, depth map upsampling, saliency map upsampling, flash/non‐flash image denoising and RGB/NIR image denoising.     

Collusion,MPEG4 compression and frame dropping resistant video watermarking

We present in this article a new video watermarking which resists collusion, MPEG4 compression and frame dropping attacks. This scheme is based on video mosaicing. For that, we are going to start by describing the mosaicing technique in order to illustrate the contribution of this technique in video watermarking. In fact, mosaicing allows to select an interesting area where the mark should be embedded. The idea is to insert the same mark into the same pixels which represent the same physical point. This is exactly the information which can be provided by a mosaic image at least for the located points in the scene background. Next, we present extensive experimental simulations which prove the watermark imperceptibility and robustness against several video attacks.