Prior-based Segmentation and Shape Registration in the Presence of Perspective Distortion

Challenging object detection and segmentation tasks can be facilitated by the availability of a reference object. However, accounting for possible transformations between the different object views, as part of the segmentation process, remains difficult. Recent statistical methods address this problem by using comprehensive training data. Other techniques can only accommodate similarity transformations. We suggest a novel variational approach to prior-based segmentation, using a single reference object, that accounts for planar projective transformation. Generalizing the Chan-Vese level set framework, we introduce a novel shape-similarity measure and embed the projective homography between the prior shape and the image to segment within a region-based segmentation functional. The proposed algorithm detects the object of interest, extracts its boundaries, and concurrently carries out the registration to the prior shape. We demonstrate prior-based segmentation on a variety of images and verify the accuracy of the recovered transformation parameters.     

Shape-Based Mutual Segmentation

We present a novel variational approach for simultaneous segmentation of two images of the same object taken from different viewpoints. Due to noise, clutter and occlusions, neither of the images contains sufficient information for correct object-background partitioning. The evolving object contour in each image provides a dynamic prior for the segmentation of the other object view. We call this process mutual segmentation. The foundation of the proposed method is a unified level-set framework for region and edge based segmentation, associated with a shape similarity term. The suggested shape term incorporates the semantic knowledge gained in the segmentation process of the image pair, accounting for excess or deficient parts in the estimated object shape. Transformations, including planar projectivities, between the object views are accommodated by a registration process held concurrently with the segmentation. The proposed segmentation algorithm is demonstrated on a variety of image pairs. The homography between each of the image pairs is estimated and its accuracy is evaluated.     

Registration and interactive planar segmentation for stereo images of polyhedral scenes

We introduce a two-step iterative segmentation and registration method to find coplanar surfaces among stereo images of a polyhedral environment. The novelties of this paper are: (i) to propose a user-defined initialization easing the image matching and segmentation, (ii) to incorporate color appearance and planar projection information into a Bayesian segmentation scheme, and (iii) to add consistency to the projective transformations related to the polyhedral structure of the scenes. The method utilizes an assisted Bayesian color segmentation scheme. The initial user-assisted segmentation is used to define search regions for planar homography image registration. The two reliable methods cooperate to obtain probabilities for coplanar regions with similar color information that are used to get a new segmentation by means of quadratic Markov measure fields (QMMF). We search for the best regions by iterating both steps: registration and segmentation.     

Prior Knowledge,Level Set Representations &; Visual Grouping

In this paper, we propose a level set method for shape-driven object extraction. We introduce a voxel-wise probabilistic level set formulation to account for prior knowledge. To this end, objects are represented in an implicit form. Constraints on the segmentation process are imposed by seeking a projection to the image plane of the prior model modulo a similarity transformation. The optimization of a statistical metric between the evolving contour and the model leads to motion equations that evolve the contour toward the desired image properties while recovering the pose of the object in the new image. Upon convergence, a solution that is similarity invariant with respect to the model and the corresponding transformation are recovered. Promising experimental results demonstrate the potential of such an approach. Electronic Supplementary Material The online version of this article () contains supplementary material, which is available to authorized users.     

Object-based image labeling through learning by example and multi-level segmentation

We propose a method for automatic extraction and labeling of semantically meaningful image objects using “learning by example” and threshold-free multi-level image segmentation. The proposed method scans through images, each of which is pre-segmented into a hierarchical uniformity tree, to seek and label objects that are similar to an example object presented by the user. By representing images with stacks of multi-level segmentation maps, objects can be extracted in the segmentation map level with adequate detail. Experiments have shown that the proposed multi-level image segmentation results in significant reduction in computation complexity for object extraction and labeling (compared to a single fine-level segmentation) by avoiding unnecessary tests of combinations in finer levels. The multi-level segmentation-based approach also achieves better accuracy in detection and labeling of small objects.     

Augmented reality camera tracking with homographies

To realistically integrate 3D graphics into an unprepared environment, camera position must be estimated by tracking natural image features. We apply our technique to cases where feature positions in adjacent frames of an image sequence are related by a homography, or projective transformation. We describe this transformation's computation and demonstrate several applications. First, we use an augmented notice board to explain how a homography, between two images of a planar scene, completely determines the relative camera positions. Second, we show that the homography can also recover pure camera rotations, and we use this to develop an outdoor AR tracking system. Third, we use the system to measure head rotation and form a simple low-cost virtual reality (VR) tracking solution.     

一种平面物体射影不变性识别方法

提出一种基于方向可变滤波器的平面物体射影不变性识别方法。该方法首先利用方向可变滤波器检测出平面物体的边缘方向特征,从单幅图像中提取平面物体在射影变化下的不变特征,建立经典框架,然后用填充经典框架图像的矩识别物体。该方法是图像局部识别方法,允许景物中有部分的遮挡物存在。     

Automated extraction of street-scene objects from mobile lidar point clouds

Mobile laser scanning or lidar is a new and rapid system to capture high-density three-dimensional (3-D) point clouds. Automatic data segmentation and feature extraction are the key steps for accurate identification and 3-D reconstruction of street-scene objects (e.g. buildings and trees). This article presents a novel method for automated extraction of street-scene objects from mobile lidar point clouds. The proposed method first uses planar division to sort points into different grids, then calculates the weights of points in each grid according to the spatial distribution of mobile lidar points and generates the geo-referenced feature image of the point clouds using the inverse-distance-weighted interpolation method. Finally, the proposed method transforms the extraction of street-scene objects from 3-D mobile lidar point clouds into the extraction of geometric features from two-dimensional (2-D) imagery space, thus simplifying the automated object extraction process. Experimental results show that the proposed method provides a promising solution for automatically extracting street-scene objects from mobile lidar point clouds.     

Point matching under large image deformations and illumination changes

To solve the general point correspondence problem in which the underlying transformation between image patches is represented by a homography, a solution based on extensive use of first order differential techniques is proposed. We integrate in a single robust M-estimation framework the traditional optical flow method and matching of local color distributions. These distributions are computed with spatially oriented kernels in the 5D joint spatial/color space. The estimation process is initiated at the third level of a Gaussian pyramid, uses only local information, and the illumination changes between the two images are also taken into account. Subpixel matching accuracy is achieved under large projective distortions significantly exceeding the performance of any of the two components alone. As an application, the correspondence algorithm is employed in oriented tracking of objects.     

Geometric techniques for 3D tracking of ultrasound sensor,tumor segmentation in ultrasound images,and 3D reconstruction

This paper presents different methods, some based on geometric algebra, for ultrasound probe tracking in endoscopic images, 3D allocation of the ultrasound probe, ultrasound image segmentation (to extract objects like tumors), and 3D reconstruction of the surface defined by a set of points. The tracking of the ultrasound probe in endoscopic images is done with a particle filter and an auxiliary method based on thresholding in the HSV space. The 3D pose of the ultrasound probe is calculated using conformal geometric algebra (to locate each slide in 3D space). Each slide (ultrasound image) is segmented using two methods: the level-set method and the morphological operators approach in order to obtain the object we are interested in. The points on the object of interest are obtained from the segmented ultrasound images, and then a 3D object is obtained by refining the convex hull. To do that, a peeling process with an adaptive radius is applied, all of this in the geometric algebra framework. Results for points from ultrasound images, as well as for points from objects from the AimatShape Project, are presented (A.I.M.A.T.S.H.A.P.E. – Advanced an Innovative Models And Tools for the development of Semantic-based systems for Handling, Acquiring, and Processing knowledge Embedded in multidimensional digital objects).     

Preliminary coarse image registration by using straight lines found on them for constructing super resolution mosaics and 3D scene recovery

An algorithm of coarse image registration of a 3D scene taken from different camera perspectives is proposed. The algorithm uses information on geometrical parameters of straight lines found on the images and on distribution of color and/or brightness around these lines. Colors are taken into account by using the fuzzy logic technique. The result of the algorithm operation is a planar projective transformation (planar homography) matching approximately the images. In order to use the technique in algorithms of 3D scene reconstruction, an estimate of size of the window used for searching correspondent points after the coarse image registration is obtained.     

Segmentation of Planar Objects and Their Shadows in Motion Sequences

We investigate segmentation of planar objects and their cast shadows in image sequences. Given two moving image regions in an image sequence we present an algorithm for determining if the two moving regions can be interpreted as a planar object and its cast shadow. Projective geometry and motion properties are employed to directly recover a homology that constrains point correspondences of the outlines of the image regions and determine if they obey an object/shadow relationship. This homology is derived directly from the motions of the regions and therefore is easier to accomplish than determining point-to-point correspondences between candidate object-shadow pairs. Several experiments under approximate point light source illumination are presented. First online version published in February, 2006     

Planar shape representation and matching under projective transformation

This paper introduces a new representation for planar objects which is invariant to projective transformation. Proposed representation relies on a new shape basis which we refer to as the conic basis. The conic basis takes conic-section coefficients as its dimensions and represents the object as a convex combination of conic-sections. Pairs of conic-sections in this new basis and their projective invariants provides the proposed view invariant representation. We hypothesize that two projectively transformed versions of an object result in the same representation. We show that our hypothesis provides promising recognition performance when we use the nearest neighbor rule to match projectively deformed objects.     

基于水动力学、分数阶微分及Steger算法的线状目标提取

目的 线状目标的检测具有非常广泛的应用领域,如车道线、道路及裂缝的检测等,而裂缝是其中最难检测的线状目标。为避免直接提取线状目标时图像分割难的问题,以裂缝和车道线为例,提出了一种新的跟踪线状目标中线的算法。方法 对图像进行高斯平滑,用一种新的分数阶微分模板增强图像中的模糊及微细线状目标;基于Steger算法提出一种提取线状目标中心线特征点的算法,避免了提取整体目标的困难;根据水动力学思想将裂隙看成溪流,通过最大熵阈值处理后,先进行特征点的连接,再基于线段之间的距离及夹角进行线段之间的连接（溪流之间的融合）。结果 对300幅裂缝图像及4种类别的其他线状目标图像进行试验,并与距离变换、最大熵阈值法+细线化Otsu阈值分割+细线化、谷底边界检测等类似算法进行比较分析,本文算法检测出的线状目标的连续性好、漏检（大间隙少）和误检（毛刺及多余线段少）率均较低。结论 本文算法能够在复杂的线状目标图像中准确快速地提取目标的中心线,一定程度上改善了复杂线状目标图像分割难的问题。     

地物提取的多尺度特征遥感应用分析

通过空间尺度效应分析,阐述不同属性景观地物在同一分辨率或同一尺度影像中提取的不合理性。为获得精确的地表信息,提出多尺度遥感影像分析方法,解决不同地物在不同空间尺度影像数据中提取的难题。通过多种分辨率影像的多尺度影像信息提取的应用实践,分析地物提取中的多尺度特性、尺度与分辨率关系等。     

射影不变性在旋转对称形状描述中的一种应用

旋转对称性是计算机视觉形状分析中的一类重要结构属性。自动检测该类目标的旋转特性以及提取目标的旋转中心（轴）,是描述和分析旋转对称目标的关键步骤。旋转对称的形状经过透视成像后,其成像特点不再具有对称性甚至扭对称性,但对称的结构单元之间仍然存在内在的对应不变特性。该文以简单连通的平面多边形目标为研究模型,利用射影几何中的交比不变量定义了目标轮廓上关键点的结构参数和结构参数的相似性度量,用于自动检验目标上的旋转对称点对;并利用正多边形顶点与形心的关系约束形成射影不变性,用来自动检测目标的旋转中心。实验结果验证了方法的有效性。     

Contour extraction of moving objects in complex outdoor scenes

This paper presents a new approach to the extraction of the contour of a moving object. The method is based on the fusion of a motion segmentation technique using image subtraction and a color segmentation technique based on the split-and-merge paradigm and edge information obtained from using the Canny edge detector. The advantages of this method are the following: it can detect large moving objects, the background can be arbitrarily complicated and contain many nonmoving objects, and it requires only three image frames that need not be consecutive provided that the moving object is entirely contained in the three frames. It is assumed that there is only one moving object in the image and the objects are not blurred by their motion so that the edges in the image are sharp. The method was applied to road images containing a moving vehicle, and the results show that the contour was correctly extracted in 18 of the 20 cases. We show that this contour extraction method gives good results for other types of moving objects as well. We also describe how the extracted contour can be used to classify a given vehicle into five generic categories. In this study, 19 out of the 20 vehicles were correctly classified. These results demonstrate that integration of multiple cues obtained from relatively simple image analysis techniques leads to a robust extraction of the object of interest in complex outdoor scenes.Research supported by a grant from the U.S. Department of Transportation through the Great Lakes Center for Truck Transportation Research and by a grant from the National Science Foundation (CDA-8806599).     

On the consistency of line-drawings, obtained by projections of piecewise planar objects

This paper deals with line-drawings, obtained from images of piecewise planar objects after edge detection. Such images are used, e.g., for navigation and recognition. In order to be a possible image of a three dimensional piecewise planar object, it has to obey some projective conditions. Criteria for a line-drawing to be correct is given in this paper, along with methods to find possible interpretations. In real life situations, due to digitization errors and noise, a line-drawing in general does not obey the geometric conditions imposed by the projective imaging process. Under various optimality conditions, algorithms are presented for the correction of such distorted line-drawings.This work has been done within the ESPRIT-BRA project VIVA.     

Multiview constraints on homographies

The image motion of a planar surface between two camera views is captured by a homography (a 2D projective transformation). The homography depends on the intrinsic and extrinsic camera parameters, as well as on the 3D plane parameters. While camera parameters vary across different views, the plane geometry remains the same. Based on this fact, we derive linear subspace constraints on the relative homographies of multiple (⩾ 2) planes across multiple views. The paper has three main contributions: 1) We show that the collection of all relative homographies (homologies) of a pair of planes across multiple views, spans a 4-dimensional linear subspace. 2) We show how this constraint can be extended to the case of multiple planes across multiple views. 3) We show that, for some restricted cases of camera motion, linear subspace constraints apply also to the set of homographies of a single plane across multiple views. All the results derived are true for uncalibrated cameras. The possible utility of these multiview constraints for improving homography estimation and for detecting nonrigid motions are also discussed