Visual Localization in Urban Area Using Orthogonal Building Boundaries and a GIS Database

A framework is presented for robustly estimating the location of a mobile robot in urban areas based on images extracted from a monocular onboard camera, given a 2D map with building outlines with neither 3D geometric information nor appearance data. The proposed method firstly reconstructs a set of vertical planes by sampling and clustering vertical lines from the image with random sample consensus (RANSAC), using the derived 1D homographies to inform the planar model. Then, an optimal autonomous localization algorithm based on the 2D building boundary map is proposed. The physical experiments are carried out to validate the robustness and accuracy of our localization approach.     

Estimating fisheye camera parameters from homography

This paper presents a method to linearly estimate fisheye camera model parameters from the homography induced by the space plane between two fisheye images.The homography is firstly calculated by using four feature points,instead of three points on the same line,in the fisheye image.And the constraint on the model parameters of fisheye camera can be derived from the homography under the assumption that fisheye camera model is a polynomial model.Then the model parameters for different order polynomials are computed.The proposed technique requires only multiple fisheye images to include a planar scene and need not a priori knowledge of 3D coordinate of the planar scene.Experimental results with synthetic data and real fisheye images demonstrate the validity of our method.The method can also be extended to fisheye image of other planar scene excluding the planar calibration object.     

Downsampling sparse representation and discriminant information aided occluded face recognition

In this paper, a strategy is proposed to deal with a challenging research topic, occluded face recog- nition. Our approach relies on sparse representation on downsampled input image to first locate unoccluded face parts, and then exploits the linear discriminant ability of those pixels to identify the input subject. The advantages and novelties of our method include, 1） since the sparse representation based occlusion detection is conducted on dowsampled image, our algorithm is much faster than classic SRC; 2） the discriminant informa- tion learned from training samples is combined with sparse representation to recognize occluded face for the first time. The verification experiments are conducted on both sinmlated block occlusion images and genuine occluded images.     

Halftoning based algorithms for image hiding

Digital halftoning is an important process to convert a continuous-tone image into a binary image with pure black and white pixels. This process is necessary when printing a monochrome or color image by a printer with limited number of ink colors. The main contribution of this paper is to present a halftoning method that conceals a binary image into two binary images. More specifically, three distinct gray scale images are given, such that one of them should be hidden in the other two gray scale images. Our halftoning method generates three binary images that reproduce the tone of the corresponding original three gray scale images. Quite surprisingly, the secret binary image can be seen by overlapping the other two binary images. In other words, the secret binary image is hidden in two public binary images. Also, it is very hard to guess the secret images using only one of the two public images, and both of these two public images are necessary to get the secret image. Another contribution of this paper is to extend our halftoning method to hide one image and more than one image into more than two images. The resulting images show that our halftoning method hides and recovers the original images. Hence, our halftoning technique can be used for watermarking as well as amusement purpose.     

Contact-free and pose-invariant hand-biometric-based personal identification system using RGB and depth data

Hand-biometric-based personal identification is considered to be an effective method for automatic recognition. However, existing systems require strict constraints during data acquisition, such as costly devices,specified postures, simple background, and stable illumination. In this paper, a contactless personal identification system is proposed based on matching hand geometry features and color features. An inexpensive Kinect sensor is used to acquire depth and color images of the hand. During image acquisition, no pegs or surfaces are used to constrain hand position or posture. We segment the hand from the background through depth images through a process which is insensitive to illumination and background. Then finger orientations and landmark points, like finger tips or finger valleys, are obtained by geodesic hand contour analysis. Geometric features are extracted from depth images and palmprint features from intensity images. In previous systems, hand features like finger length and width are normalized, which results in the loss of the original geometric features. In our system, we transform 2D image points into real world coordinates, so that the geometric features remain invariant to distance and perspective effects. Extensive experiments demonstrate that the proposed hand-biometric-based personal identification system is effective and robust in various practical situations.     

Affective Image Colorization

Colorization of gray-scale images has attracted many attentions for a long time.An important role of image color is the conveyer of emotions(through color themes).The colorization with an undesired color theme is less useful,even it is semantically correct.However this has been rarely considered.Automatic colorization respecting both the semantics and the emotions is undoubtedly a challenge.In this paper,we propose a complete system for affective image colorization.We only need the user to assist object segmentation along with text labels and an affective word.First,the text labels along with other object characters are jointly used to filter the internet images to give each object a set of semantically correct reference images.Second,we select a set of color themes according to the affective word based on art theories.With these themes,a generic algorithm is used to select the best reference for each object,balancing various requirements.Finally,we propose a hybrid texture synthesis approach for colorization.To the best of our knowledge,it is the first system which is able to efficiently colorize a gray-scale image semantically by an emotionally controllable fashion.Our experiments show the effectiveness of our system,especially the benefit compared with the previous Markov random field(MRF) based method.     

Automatic Image-Based Pencil Sketch Rendering

This paper presents an automatic image-based approach for converting greyscale images to pencil sketches,in which strokes follow the image features.The algorithm first extracts a dense direction field automatically using Logical/Linear operators which embody the drawing mechanism.Next,a reconstruction approach based on a sampling-and-interpolation scheme is introduced to generate stroke paths from the direction field.Finally,pencil strokes are rendered along the specified paths with consideration of image tone and artificial illumingation.As an important application,the technique is applied to render portraits from images with little user interaction.The experimental results demonstrate that the approach can automatically achieve copmelling pencil sketches from reference images.     

Text extraction and enhancement of binary images using cellular automata

Text characters embedded in images represent a rich source of information for content-based indexing and retrieval applications. However, these text characters are difficult to be detected and recognized due to their various sizes, grayscale values, and complex backgrounds. Existing methods cannot handle well those texts with different contrast or embedded in a complex image background. In this paper, a set of sequential algorithms for text extraction and enhancement of image using cellular automata are proposed. The image enhancement includes gray level, contrast manipulation, edge detection, and filtering. First, it applies edge detection and uses a threshold to filter out for low-contrast text and simplify complex background of high-contrast text from binary image. The proposed algorithm is simple and easy to use and requires only a sample texture binary image as an input. It generates textures with perceived quality, better than those proposed by earlier published techniques. The performance of our method is demonstrated by presenting experimental results for a set of text based binary images. The quality of thresholding is assessed using the precision and recall analysis of the resultant text in the binary image.     

Image geometry and radio metrics correction and optimization in image mosaic

In image guided navigation, due to radiometric and geometric distortions of images from photography, it can mislead the content of the image, image series are finally converted into the same coordinate system. Based on the analysis of correction algorithm, a much better one is presented, which is proved to be effective in the correction of radiometric image distortion. A theory using image local maximum modules, got by wavelet transform, as image feature, is promoted. Using image mosaic technique based on multi-resolution analysis, redundancies between images are eliminated. The whole image produced at last is a much bigger one without seam and distortion.     

一种鲁棒的基于两个特征点的深度估计方法

This paper presents a novel depth estimation method based on feature points. Two points are selected arbitrarily from an object and their distance in the space is assumed to be known.The proposed technique can estimate simultaneously their depths according to two images taken before and after a camera moves and the motion parameters of the camera may be unknown. In addition, this paper analyzes the ways to enhance the precision of the estimated depths and presents a feature point image coordinates search algorithm to increase the robustness of the proposed method.The search algorithm can find automatically more accurate image coordinates of the feature points based on their detected image coordinates. Experimental results demonstrate the efficiency of the presented method.     

基于平面的Warping技术

提出了一种基于平面的逆向Warping算法,用于根据多幅参考图像生成任意视点下的新图像.首先通过参考图像的深度信息来重建三维平面,然后寻找这些重建平面间的对应关系,并比较它们对空间平面同一部分的采样密度,以获得最好的采样结果.在生成新视点图像时,首先对那些采样密度最好的重建平面进行可见性判断,然后将其投影到新视点下,在此基础上求得目标图像上各点的深度,最后将目标图像上的点逆向Warping到相应的参考图像中,以获取它们的颜色值.同时,对于参考图像中不能被重建成平面的像素点,用正向Warping的方法对其进     

生成当前视点目标图像的快速逆映射算法

利用极线的属性及深度图像隐含的边界信息,提出了一种可以处理非深度连续图像的快速逆映射算法,以从多幅参考图像精确合成当前视点目标图像.该算法分为3步:首先由参考图像的深度信息得到图像中的边界;然后从参考图像中选出一幅作为主参考图像,利用极线的整体匹配特性和对应点在极线上分布的单调性,逐个处理目标极线,以生成目标图像;最后根据其他参考图像填补目标图像中的空洞.由于在第2步中只需处理参考图像的边界点对即可得到对应目标极线上所有点的深度信息及其在参考极线上的对应点,因此,新算法很好地提高了速度.在填补空洞时,利用参考图像的边界信息及所隐含的遮挡关系,还提出了一些加速的方法.     

基于图像的实时漫游

提出了一个点和多边形模型混合的场景表达方式,从而实现了对复杂场景的视点不受限制的实时漫游.它从多幅带深度的参考图像出发,在预处理阶段区别对待场景中平面物体和曲面物体在参考图像中的对应像素.对于在参考图像所占区域较大的空间平面,用传统的多边形模型方式对其进行表达,试图恢复出其平面方程,然后通过采样密度比较和重采样过程,将该空间平面在所有参考图像中的出现合并成一个均匀采样的纹理图像;而对于空间曲面或在参考图像中所占面积较小的平面,使用点的形式对其进行表达,通过采样密度的比较去掉冗余的点,将保留下来的点按其空间位置进行聚类.同时,对于场景中那些不被所有参考图像所拍摄到而在漫游过程中可能形成空洞的部分,提出一个空洞预填补技术,在预处理阶段即对这类空洞进行填补,从而大大减少了漫游阶段出现空洞的几率.在漫游阶段则使用纹理映射和点Warping的方式进行绘制,以充分利用图形硬件的加速功能.     

CSA--一种加速的图像逆映射算法

采用方程求解代替逐像素搜索，提出一种加速的图像逆映射CSA算法．首先根据深度信息标记出参考域中的边界像素，并在映射时保存在像素链表中；然后将链表中的像素逐个映射到其他参考域建立线段方程，并与对应的极线方程联立求解，若有解则求出匹配的参考像素，否则标记为空洞；最后利用插值或其他参考信息填补空洞．由于仅处理边界像素，降低了计算复杂度，经实验证实，该算法有较好的加速作用．     

用于图像逆映射的搜索转移算法

图像逆映射是基于图像绘制技术的基础算法，但由于需要进行繁琐的参考域搜索过程，传统图像逆映射算法计算复杂度大、效率低。本文提出一种快速图像逆映射算法——搜索转移算法。该算法从场景的极线几何性质出发，将大量匹配搜索过程从参考域转移到目的域中进行，由于在目的域只需进行简单的判断操作，因此可以大大降低整体的匹配复杂度；同时，算法充分利用场景的空间连贯性，通过为目的域制定自适应的绘制顺序，有效地减小了实际需要搜索的步长。实验表明，搜索转移算法简单有效，能极大地降低搜索复杂度95％以上，提高计算效率10倍左右。     

基于几何与图像混合绘制中的快速WARP变换算法研究

基于几何与图像的混合绘制中，3D Warp算法以严格的数学变换为基础，从而能够保证准确的投影关系，但该算法在实时绘制阶段需进行大量的数学运算，故其时间复杂度较高，该文提出了一种新的快速Warp变换算法，算法以3D Warp算法为基础，采用了崭新的投影过程，从而使时间复杂度较3D Warp算法有较大幅度的下降（降低约7．51倍）。同时，该算法是一种流水结构，能够有效利用现有的加速硬件，而无需改变图形硬件的体系结构。     

基于DIBR和图像融合的任意视点绘制

虚拟视点生成是3维视频会议等应用领域中的关键技术,为了快速高质量地进行任意视点绘制,提出了一种基于深度图像绘制(DIBR)和图像融合的新视点生成方法,该方法首先对参考图像进行预处理,包括深度图像的边缘滤波和参考图像规正,以减少目标图像中产生的较大空洞和虚假边缘;然后利用3维图像变换生成新视点图像,并用遮挡兼容算法对遮挡进行快速处理;接着再对两幅目标图像进行融合得到新视点图像;最后用插值法填充剩余的较小空洞。实验证明,该新方法能获得令人满意的绘制效果。     

Composition of novel views through an efficient image warping

This paper presents an efficient inverse warping algorithm for generating novel views from multiple reference images taken from different viewpoints. The method proceeds in three steps: preprocess for edge-pixel extraction, inverse warping from the primary image, and hole filling from remaining reference images. In using advantages of epipolar-line features and the depth discontinuity in the reference images, the inverse warping can be efficiently applied in segments, accelerating the rendering process substantially. Two acceleration techniques are proposed at the last stage for hole filling. By using the algorithm proposed, we may navigate a virtual environment at an interactive rate.     

一种基于视觉约束的当前视点画面生成方法

由于视点变化中画面可见性的变化以及在当前视点物体表面的扩张,仅由一幅带深度的源参考图像的三维重投影所得到的目标视点画面存在空洞问题.为此,人们采取多源参考图像的合成方法来解决此问题.如何从具有大量冗余信息的多幅源参考图像中快速提取当前目标视点所需信息,成为基于图像的虚拟环境建模和绘制的关键技术.该文给出一种从多幅源参考图像(目前仅两幅)合成当前视点画面的算法.算法结合正向映射及逆向映射技术完成对当前视点画面的合成,并且利用视觉约束-极线约束这一性质从其他源参考图像中寻找填补空洞的像素,从而避免了整幅参考图像的重投影过程及深度比较.同时,算法利用极线对上像素的排序特性和已有目标图像的信息加速填补空洞像素的搜索.算法在不增加原有图像映射算法复杂度的基础上,大大降低了原有多幅图像合成算法的计算量.     

虚平面映射:深度图像内部视点的绘制技术

首先推导与归纳了图像三维变换中像素深度场的变换规律,同时提出了基于深度场和极线原则的像素可见性别方法,根据上述理论和方法,提出一种基于深度图像的建模与绘制(image-based modeling and rendering,简称IBMR)技术,称为虚平面映射.该技术可以基于图像空间内任意视点对场景进行绘制.绘制时,先在场景中根据视线建立若干虚拟平面,将源深度图像中的像素转换到虚平面上,然后通过对虚平面上像素的中间变换,将虚平面转换成平面纹理,再利用虚平面的相互拼接,将视点的成像以平面纹理映射的方式完成.新方法还能在深度图像内侧,基于当前视点快速获得该视点的全景图,从而实现视点的实时漫游.新方法视点运动空间大、存储需求小,且可以发挥图形硬件的纹理映射功能,并能表现物体表面的三维凹凸细节和成像视差效果,克服了此前类似算法的局限和不足.