Shape from Sharp and Motion-Blurred Image Pair

Motion blur due to camera shake is a common occurrence. During image capture, the apparent motion of a scene point in the image plane varies according to both camera motion and scene structure. Our objective is to infer the camera motion and the depth map of static scenes using motion blur as a cue. To this end, we use an unblurred–blurred image pair. Initially, we develop a technique to estimate the transformation spread function (TSF) which symbolizes the camera shake. This technique uses blur kernels estimated at different points across the image. Based on the estimated TSF, we recover the complete depth map of the scene within a regularization framework.     

Robust and efficient detection of salient convex groups

This paper describes an algorithm that robustly locates salient convex collections of line segments in an image. The algorithm is guaranteed to find all convex sets of line segments in which the length of the gaps between segments is smaller than some fixed proportion of the total length of the lines. This enables the algorithm to find convex groups whose contours are partially occluded or missing due to noise. We give an expected case analysis of the algorithm performance. This demonstrates that salient convexity is unlikely to occur at random, and hence is a strong clue that grouped line segments reflect underlying structure in the scene. We also show that our algorithm run time is O(n ²log(n)+nm), when we wish to find the m most salient groups in an image with n line segments. We support this analysis with experiments on real data, and demonstrate the grouping system as part of a complete recognition system     

Tracking 3-D motion from straight lines with trifocal tensors

We present a novel approach to track the position and orientation of a stereo camera using line features in the images. The method combines the strengths of trifocal tensors and Bayesian filtering. The trifocal tensor provides a geometric constraint to lock line features among every three frames. It eliminates the explicit reconstruction of the scene even if the 3-D scene structure is not known. Such a trifocal constraint thus makes the algorithm fast and robust. The twist motion model is applied to further improve its computation efficiency. Another major contribution is that our approach can obtain the 3-D camera motion using as little as 2 line correspondences instead of 13 in the traditional approaches. This makes the approach attractive for realistic applications. The performance of the proposed method has been evaluated using both synthetic and real data with encouraging results. Our algorithm is able to estimate 3-D camera motion in real scenarios accurately having little drifting from an image sequence longer than a 1,000 frames.     

模糊影响感知的交互设计

图像的模糊变化会影响人类视觉对图像的感知。在特定条件下,它能提升人类感知和识别图像的能力。在摄像过程中,图像的清晰度或模糊度是由拍摄相机的参数决定的,需要较为专业的摄像器材,成本太高。论文构造一个图像模糊区域生成交互系统,以便于普通用户生成模糊区域达到特殊效果。具体表现为利用模糊算法,构建相机成像模型,通过操纵成像参数,达到对模糊有效操控,实现交互目的。交互平台内容主要包括,在图像上交互地生成一个与场景中一些平行线方向一致的网格,通过预设参数求出图像特定区域中的每个像素成像后的模糊量,最后合成模糊图像。在交互平台进行了实验,结果验证了算法和交互平台的有效性。     

多分辨率线段提取方法及线段语义分析

线段是一种组成几何体的基本元素,蕴含着非常丰富的几何信息。从图像中提取完整、连续且具有语义信息的线段对恢复场景的几何结构具有重要意义。该文提出了一种多分辨率线段提取方法,并对线段进行语义分析以区分轮廓线段和纹理线段。该方法首先运用多分辨率思想进行线段提取,然后结合深度神经网络技术对线段进行语义分析,最后对线段进行聚类合并得到最终结果。在线段连续性和完整性方面,该文提出的方法与当前常用的线段提取方法相比具有明显优势;在语义分析准确性方面,该文提出的方法在测试集上的像素精度高达 97.82%。     

Road obstacle detection and tracking by an active and intelligent sensing strategy

In this paper, we address the problem of road obstacle deletion. We propose a method based on an active and intelligent sensing strategy. A sensor composed of a range finder coupled with a (charge-coupled-device) CCD camera is used. This sensor is mounted in front of a vehicle. The basic idea is first to determine 2D visual targets in intensity images of the camera. Then the range finder will be used not only to confirm or reject the existence of the detected visual targets, but also to acquire 3D information of the confirmed visual targets. The central problem of this strategy is how to detect 2D visual targets from intensity images of a road scene. In our method, we consider line segments as significant features. We use the concept ofline segment of interest and the concept ofdominant line segment. With the help of the identified dominant line segments in an image, we can effectively ascertain 2D visual targets. Finally, we use the range finder to confirm or reject a 2D visual target. A confirmed visual target is temporally tracked with the help of the range finder.     

基于点线结合特征的单目视觉里程计

SLAM(即时定位与地图构建)系统是近年来计算机视觉领域的一大重要课题,其中特征法的SLAM凭借稳定性好、计算效率高的优点成为SLAM算法的主流。目前特征法SLAM主要基于点特征进行。针对基于点特征的视觉里程计依赖于数据质量,相机运动过快时容易跟丢,且生成的特征地图不包含场景结构信息等缺点,提出了一种基于点线结合特征的优化算法。相较于传统基于线段端点的六参数表达方式,算法采用一种四参数的方式表示空间直线,并使用点线特征进行联合图优化估计相机位姿。使用公开数据集和自采集鱼眼影像数据分别进行实验的结果表明,与仅使用点特征的方法相比,该方法可有效改善因相机运动过快产生的跟丢问题,增加轨迹长度,提升位姿估计精度,且生成的稀疏特征地图更能反映场景结构特征。     

Segment Based Camera Calibration

The basic idea of calibrating a camera system in previous approaches is to determine camera parmeters by using a set of known 3D points as calibration reference.In this paper,we present a method of camera calibration in whih camera parameters are determined by a set of 3D lines.A set of constraints is derived on camea parameters in terms of perspective line mapping.Form these constraints,the same perspective transformation matrix as that for point mapping can be computed linearly.The minimum number of calibration lines is 6.This result generalizes that of Liu,Huang and Faugeras^[12] for camera location determination in which at least 8 line correspondences are required for linear computation of camera location.Since line segments in an image can be located easily and more accurately than points,the use of lines as calibration reference tends to ease the computation in inage preprocessing and to improve calibration accuracy.Experimental results on the calibration along with stereo reconstruction are reported.     

Plenoptic Image Editing

This paper presents a new class of interactive image editing operations designed to maintain consistency between multiple images of a physical 3D scene. The distinguishing feature of these operations is that edits to any one image propagate automatically to all other images as if the (unknown) 3D scene had itself been modified. The modified scene can then be viewed interactively from any other camera viewpoint and under different scene illuminations. The approach is useful first as a power-assist that enables a user to quickly modify many images by editing just a few, and second as a means for constructing and editing image-based scene representations by manipulating a set of photographs. The approach works by extending operations like image painting, scissoring, and morphing so that they alter a scene's plenoptic function in a physically-consistent way, thereby affecting scene appearance from all viewpoints simultaneously. A key element inrealizing these operations is a new volumetric decomposition technique for reconstructing an scene's plenoptic function from an incomplete set of camera viewpoints.     

A local method for contour matching and its parallel implementation

This paper presents a local approach for matching contour segments in an image sequence. This study has been primarily motivated by work concerned with the recovery of 3D structure using active vision. The method to recover the 3D structure of the scene requires to track in real-time contour segments in an image sequence. Here, we propose an original and robust approach that is ideally suited for this problem. It is also of more general interest and can be used in any context requiring matching of line boundaries over time. This method only involves local modeling and computation of moving edges dealing “virtually” with a contour segment primitive representation. Such an approach brings robustness to contour segmentation instability and to occlusion, and easiness for implementation. Parallelism has also been investigated using an SIMD-based real-time image-processing system. This method has been validated with experiments on several real-image sequences. Our results show quite satisfactory performance and the algorithm runs in a few milliseconds. Received: 11 December 1996 / Accepted: 8 August 1997     

Robust and efficient map-to-image registration with line segments

This work investigates map-to-image registration for planar scenes in the context of robust parameter estimation. Registration is posed as the problem of estimating a projective transformation which optimally aligns transformed model line segments from a map with data line segments extracted from an image. Matching and parameter estimation is solved simultaneously by optimizing an objective function which is based on M-estimators, and depends on overlap and the weighted orthogonal distance between transformed model segments and data segments. An extensive series of registration experiments was conducted to test the performance of the proposed parameter estimation algorithm. More than 200 000 registration experiments were run with different objective functions for 12 aerial images and randomly corrupted maps distorted by randomly selected projective transformations. Received: 10 August 2000 / Accepted: 29 January 2001     

Modeling the space of camera response functions

Many vision applications require precise measurement of scene radiance. The function relating scene radiance to image intensity of an imaging system is called the camera response. We analyze the properties that all camera responses share. This allows us to find the constraints that any response function must satisfy. These constraints determine the theoretical space of all possible camera responses. We have collected a diverse database of real-world camera response functions (DoRF). Using this database, we show that real-world responses occupy a small part of the theoretical space of all possible responses. We combine the constraints from our theoretical space with the data from DoRF to create a low-parameter empirical model of response (EMoR). This response model allows us to accurately interpolate the complete response function of a camera from a small number of measurements obtained using a standard chart. We also show that the model can be used to accurately estimate the camera response from images of an arbitrary scene taken using different exposures. The DoRF database and the EMoR model can be downloaded at http://www.cs.columbia.edu/CAVE.     

利用运动线索的单目深度测量

目的 传统的单目视觉深度测量方法具有设备简单、价格低廉、运算速度快等优点,但需要对相机进行复杂标定,并且只在特定的场景条件下适用。为此,提出基于运动视差线索的物体深度测量方法,从图像中提取特征点,利用特征点与图像深度的关系得到测量结果。方法 对两幅图像进行分割,获取被测量物体所在区域;然后采用本文提出的改进的尺度不变特征变换SIFT（scale-invariant feature transtorm）算法对两幅图像进行匹配,结合图像匹配和图像分割的结果获取被测量物体的匹配结果;用Graham扫描法求得匹配后特征点的凸包,获取凸包上最长线段的长度;最后利用相机成像的基本原理和三角几何知识求出图像深度。结果 实验结果表明,本文方法在测量精度和实时性两方面都有所提升。当图像中的物体不被遮挡时,实际距离与测量距离之间的误差为2.60%,测量距离的时间消耗为1.577 s;当图像中的物体存在部分遮挡时,该方法也获得了较好的测量结果,实际距离与测量距离之间的误差为3.19%,测量距离所需时间为1.689 s。结论 利用两幅图像上的特征点来估计图像深度,对图像中物体存在部分遮挡情况具有良好的鲁棒性,同时避免了复杂的摄像机标定过程,具有实际应用价值。     

Camera self-calibration with varying intrinsic parameters by an unknown three-dimensional scene

This work proposes a method of camera self-calibration having varying intrinsic parameters from a sequence of images of an unknown 3D object. The projection of two points of the 3D scene in the image planes is used with fundamental matrices to determine the projection matrices. The present approach is based on the formulation of a nonlinear cost function from the determination of a relationship between two points of the scene and their projections in the image planes. The resolution of this function enables us to estimate the intrinsic parameters of different cameras. The strong point of the present approach is clearly seen in the minimization of the three constraints of a self-calibration system (a pair of images, 3D scene, any camera): The use of a single pair of images provides fewer equations, which minimizes the execution time of the program, the use of a 3D scene reduces the planarity constraints, and the use of any camera eliminates the constraints of cameras having constant parameters. The experiment results on synthetic and real data are presented to demonstrate the performance of the present approach in terms of accuracy, simplicity, stability, and convergence.     

高速公路云台相机的自动标定

目的 云台相机因监控视野广、灵活度高,在高速公路监控系统中发挥出重要的作用,但因云台相机焦距与角度不定时地随监控需求变化,对利用云台相机的图像信息获取真实世界准确的物理信息造成一定困难,因此进行云台相机非现场自动标定方法的研究对高速公路监控系统的应用具有重要价值。方法 本文提出了一种基于消失点约束与车道线模型约束的云台相机自动标定方法,以建立高速公路监控系统的图像信息与真实世界物理信息之间准确描述关系。首先,利用车辆目标运动轨迹的级联霍夫变换投票实现纵向消失点的准确估计,其次以车道线模型物理度量为约束,并采用枚举策略获取横向消失点的准确估计,最终在已知相机高度的条件下实现高速公路云台相机标定参数的准确计算。结果 将本文方法在不同的场景下进行实验,得到在不同的距离下的平均误差分别为4.63%、4.74%、4.81%、4.65%,均小于5%。结论 对多组高速公路监控场景的测试实验结果表明,本文提出的云台相机自动标定方法对高速公路监控场景的物理测量误差能够满足应用需求,与参考方法相比较而言具有较大的优势和一定的应用价值,得到的相机内外参数可用于计算车辆速度与空间位置等。     

A hierarchy of cameras for 3D photography

The view-independent visualization of 3D scenes is most often based on rendering accurate 3D models or utilizes image-based rendering techniques. To compute the 3D structure of a scene from a moving vision sensor or to use image-based rendering approaches, we need to be able to estimate the motion of the sensor from the recorded image information with high accuracy, a problem that has been well-studied. In this work, we investigate the relationship between camera design and our ability to perform accurate 3D photography, by examining the influence of camera design on the estimation of the motion and structure of a scene from video data. By relating the differential structure of the time varying plenoptic function to different known and new camera designs, we can establish a hierarchy of cameras based upon the stability and complexity of the computations necessary to estimate structure and motion. At the low end of this hierarchy is the standard planar pinhole camera for which the structure from motion problem is non-linear and ill-posed. At the high end is a camera, which we call the full field of view polydioptric camera, for which the motion estimation problem can be solved independently of the depth of the scene which leads to fast and robust algorithms for 3D Photography. In between are multiple view cameras with a large field of view which we have built, as well as omni-directional sensors.     

Direct recovery of planar-parallax from multiple frames

We present an algorithm that estimates dense planar-parallax motion from multiple uncalibrated views of a 3D scene. This generalizes the "plane+parallax" recovery methods to more than two frames. The parallax motion of pixels across multiple frames (relative to a planar surface) is related to the 3D scene structure and the camera epipoles. The parallax field, the epipoles, and the 3D scene structure are estimated directly from image brightness variations across multiple frames, without precomputing correspondences.     

铁路视频序列的FOE的估计

在结构化场景的轨道交通中,车载视频观测因相机平移运动而呈现出图像内容以某点为中心向四周扩散的现象,该点被称为FOE (Focus of Expansion)。 当前计算FOE的算法对噪声敏感且计算量大,不能准确地计算铁路场景中的FOE。鉴于此,文中提出一种铁路视频序列的FOE估计方法。该方法首先利用金字塔光流法对检测的Harris角点进行跟踪和粗匹配,并在此基础上利用RANSAC算法进行精确的匹配,求得基础矩阵,然后提取图像中的极线束并计算FOE。实验结果表明,所提算法比Hough 直线求得的FOE误差小,适于实时应用。     

Generalized Mosaicing: High Dynamic Range in a Wide Field of View

We present an approach that significantly enhances the capabilities of traditional image mosaicking. The key observation is that as a camera moves, it senses each scene point multiple times. We rigidly attach to the camera an optical filter with spatially varying properties, so that multiple measurements are obtained for each scene point under different optical settings. Fusing the data captured in the multiple images yields an image mosaic that includes additional information about the scene. We refer to this approach as generalized mosaicing. In this paper we show that this approach can significantly extend the optical dynamic range of any given imaging system by exploiting vignetting effects. We derive the optimal vignetting configuration and implement it using an external filter with spatially varying transmittance. We also derive efficient scene sampling conditions as well as ways to self calibrate the vignetting effects. Maximum likelihood is used for image registration and fusion. In an experiment we mounted such a filter on a standard 8-bit video camera, to obtain an image panorama with dynamic range comparable to imaging with a 16-bit camera.     

Accurate sensing of scene geo-context via mobile visual localization

Image geo-tagging has drawn a great deal of attention in recent years. The geographic information associated with images can be used to promote potential applications such as location recognition or virtual navigation. In this paper, we propose a novel approach for accurate mobile image geo-tagging in urban areas. The approach is able to provide a comprehensive set of geo-context information based on the current image, including the real location of the camera and the viewing angle, as well as the location of the captured scene. Moreover, the parsed building facades and their geometric structures can also be estimated. First, for the image to be geo-tagged, we perform partial duplicate image retrieval to filter crowd-sourced images capturing the same scene. We then employ the structure-from-motion technique to reconstruct a sparse 3D point cloud of the scene. Meanwhile, the geometric structure of the query image is analyzed to extract building facades. Finally, by combining the reconstructed 3D scene model and the extracted structure information, we can register the camera location and viewing direction to a real-world map. The captured building location and facade orientation are also aligned. The effectiveness of the proposed system is demonstrated by experiment results.