Theory and Practice of Projective Rectification

This paper gives a new method for image rectification, the process of resampling pairs of stereo images taken from widely differing viewpoints in order to produce a pair of matched epipolar projections. These are projections in which the epipolar lines run parallel with the x-axis and consequently, disparities between the images are in the x-direction only. The method is based on an examination of the fundamental matrix of Longuet-Higgins which describes the epipolar geometry of the image pair. The approach taken is consistent with that advocated by Faugeras (1992) of avoiding camera calibration. The paper uses methods of projective geometry to determine a pair of 2D projective transformations to be applied to the two images in order to match the epipolar lines. The advantages include the simplicity of the 2D projective transformation which allows very fast resampling as well as subsequent simplification in the identification of matched points and scene reconstruction.     

Robot Homing by Exploiting Panoramic Vision

We propose a novel, vision-based method for robot homing, the problem of computing a route so that a robot can return to its initial home position after the execution of an arbitrary prior path. The method assumes that the robot tracks visual features in panoramic views of the environment that it acquires as it moves. By exploiting only angular information regarding the tracked features, a local control strategy moves the robot between two positions, provided that there are at least three features that can be matched in the panoramas acquired at these positions. The strategy is successful when certain geometric constraints on the configuration of the two positions relative to the features are fulfilled. In order to achieve long-range homing, the features trajectories are organized in a visual memory during the execution of the prior path. When homing is initiated, the robot selects Milestone Positions (MPs) on the prior path by exploiting information in its visual memory. The MP selection process aims at picking positions that guarantee the success of the local control strategy between two consecutive MPs. The sequential visit of successive MPs successfully guides the robot even if the visual context in the home position is radically different from the visual context at the position where homing was initiated. Experimental results from a prototype implementation of the method demonstrate that homing can be achieved with high accuracy, independent of the distance traveled by the robot. The contribution of this work is that it shows how a complex navigational task such as homing can be accomplished efficiently, robustly and in real-time by exploiting primitive visual cues. Such cues carry implicit information regarding the 3D structure of the environment. Thus, the computation of explicit range information and the existence of a geometric map are not required.     

Image-based homing navigation with landmark arrangement matching

Many insects and animals exploit their own navigation systems to navigate in space. Biologically-inspired methods have been introduced for landmark-based navigation algorithms of a mobile robot. The methods determine the movement direction based on a home snapshot image and another snapshot from the current position. In this paper, we suggest a new landmark-based matching method for robotic homing navigation that first computes the distance to each landmark based on ego-motion and estimates the landmark arrangement in the snapshot image. Then, landmark vectors are used to localize the robotic agent in the environment and to choose the appropriate direction to return home. As a result, this method has a higher success rate for returning home from an arbitrary position than do the conventional image-matching algorithms.     

基于全景视觉的机器人回航方法

提出一种使用全景视觉系统引导机器人回航的方法.利用全景视觉装置采集出发位置(Home位置)的全景图像,使用SURF(Speeded-Up Robust Feature)算法提取全景图像中的特征点作为自然路标点.机器人回航过程中,将当前位置获得的全景图像与Home位置的全景图像进行特征匹配,确定白然路标点之间的对应关系....     

Moment feature-based three-dimensional tracking using two high-speed vision systems

When considering real-world applications of robot control with visual servoing, both three-dimensional (3-D) information and a high feedback rate are required. We have developed a 3-D target-tracking system with a 1-ms feedback rate using two high-speed vision systems called Column Parallel Vision (CPV) systems. To obtain 3-D information, such as position, orientation and shape parameters of the target object, a feature-based algorithm has been introduced using moment feature values extracted from vision systems for a spheroidal object model. Also, we propose a new 3-D self-windowing method to extract the target in 3-D space using epipolar geometry, which is an extension of the conventional self-windowing method in 2-D images.     

Invariants of six points and projective reconstruction from threeuncalibrated images

There are three projective invariants of a set of six points in general position in space. It is well known that these invariants cannot be recovered from one image, however an invariant relationship does exist between space invariants and image invariants. This invariant relationship is first derived for a single image. Then this invariant relationship is used to derive the space invariants, when multiple images are available. This paper establishes that the minimum number of images for computing these invariants is three, and the computation of invariants of six points from three images can have as many as three solutions. Algorithms are presented for computing these invariants in closed form. The accuracy and stability with respect to image noise, selection of the triplets of images and distance between viewing positions are studied both through real and simulated images. Applications of these invariants are also presented. Both the results of Faugeras (1992) and Hartley et al. (1992) for projective reconstruction and Sturm's method (1869) for epipolar geometry determination from two uncalibrated images with at least seven points are extended to the case of three uncalibrated images with only six points     

Cleaning robot navigation using panoramic views and particle clouds as landmarks

The paper describes a visual method for the navigation of autonomous floor-cleaning robots. The method constructs a topological map with metrical information where place nodes are characterized by panoramic images and by particle clouds representing position estimates. Current image and position estimate of the robot are interrelated to landmark images and position estimates stored in the map nodes through a holistic visual homing method which provides bearing and orientation estimates. Based on these estimates, a position estimate of the robot is updated by a particle filter. The robot’s position estimates are used to guide the robot along parallel, meandering lanes and are also assigned to newly created map nodes which later serve as landmarks. Computer simulations and robot experiments confirm that the robot position estimate obtained by this method is sufficiently accurate to keep the robot on parallel lanes, even in the presence of large random and systematic odometry errors. This ensures an efficient cleaning behavior with almost complete coverage of a rectangular area and only small repeated coverage. Furthermore, the topological-metrical map can be used to completely cover rooms or apartments by multiple meander parts.     

Panoramic Depth Imaging: Single Standard Camera Approach

In this paper we present a panoramic depth imaging system. The system is mosaic-based which means that we use a single rotating camera and assemble the captured images in a mosaic. Due to a setoff of the camera's optical center from the rotational center of the system we are able to capture the motion parallax effect which enables stereo reconstruction. The camera is rotating on a circular path with a step defined by the angle, equivalent to one pixel column of the captured image. The equation for depth estimation can be easily extracted from the system geometry. To find the corresponding points on a stereo pair of panoramic images the epipolar geometry needs to be determined. It can be shown that the epipolar geometry is very simple if we are doing the reconstruction based on a symmetric pair of stereo panoramic images. We get a symmetric pair of stereo panoramic images when we take symmetric pixel columns on the left and on the right side from the captured image center column. Epipolar lines of the symmetrical pair of panoramic images are image rows. The search space on the epipolar line can be additionaly constrained. The focus of the paper is mainly on the system analysis. Results of the stereo reconstruction procedure and quality evaluation of generated depth images are quite promissing. The system performs well for reconstruction of small indoor spaces. Our finall goal is to develop a system for automatic navigation of a mobile robot in a room.     

Arbitrary viewpoint video synthesis from multiple uncalibrated cameras.

We propose a method for arbitrary view synthesis from uncalibrated multiple camera system, targeting large spaces such as soccer stadiums. In Projective Grid Space (PGS), which is a three-dimensional space defined by epipolar geometry between two basis cameras in the camera system, we reconstruct three-dimensional shape models from silhouette images. Using the three-dimensional shape models reconstructed in the PGS, we obtain a dense map of the point correspondence between reference images. The obtained correspondence can synthesize the image of arbitrary view between the reference images. We also propose a method for merging the synthesized images with the virtual background scene in the PGS. We apply the proposed methods to image sequences taken by a multiple camera system, which installed in a large concert hall. The synthesized image sequences of virtual camera have enough quality to demonstrate effectiveness of the proposed method.     

Recovery of ego-motion using region alignment

A method for computing the 3D camera motion (the ego-motion) in a static scene is described, where initially a detected 2D motion between two frames is used to align corresponding image regions. We prove that such a 2D registration removes all effects of camera rotation, even for those image regions that remain misaligned. The resulting residual parallax displacement field between the two region-aligned images is an epipolar field centered at the FOE (Focus-of-Expansion). The 3D camera translation is recovered from the epipolar field. The 3D camera rotation is recovered from the computed 3D translation and the detected 2D motion. The decomposition of image motion into a 2D parametric motion and residual epipolar parallax displacements avoids many of the inherent ambiguities and instabilities associated with decomposing the image motion into its rotational and translational components, and hence makes the computation of ego-motion or 3D structure estimation more robust     

A fast robot homing approach using sparse image waypoints

This paper proposes a fast image sequence-based navigation approach for a flat route represented in sparse waypoints. Instead of purely optimizing the length of the path, this paper aims to speed up the navigation by lengthening the distance between consecutive waypoints. When local visual homing in a variable velocity is applied for robot navigation between two waypoints, the robot's speed changes according to the distance between waypoints. Because long distance implies large scale difference between the robot's view and the waypoint image, log-polar transform is introduced to find a correspondence between images and infer a less accurate motion vector. In order to maintain the navigation accuracy, our prior work on local visual homing with SIFT feature matching is adopted when the robot is relatively close to the waypoint. Experiments support the proposed navigation approach in a multiple-waypoint route. Compared to other prior work on visual homing with SIFT feature matching, the proposed navigation approach requires fewer waypoints and the navigation speed is improved without compromising the accuracy in navigation.     

Simultaneous two-view epipolar geometry estimation and motion segmentation by 4D tensor voting

We address the problem of simultaneous two-view epipolar geometry estimation and motion segmentation from nonstatic scenes. Given a set of noisy image pairs containing matches of n objects, we propose an unconventional, efficient, and robust method, 4D tensor voting, for estimating the unknown n epipolar geometries, and segmenting the static and motion matching pairs into n, independent motions. By considering the 4D isotropic and orthogonal joint image space, only two tensor voting passes are needed, and a very high noise to signal ratio (up to five) can be tolerated. Epipolar geometries corresponding to multiple, rigid motions are extracted in succession. Only two uncalibrated frames are needed, and no simplifying assumption (such as affine camera model or homographic model between images) other than the pin-hole camera model is made. Our novel approach consists of propagating a local geometric smoothness constraint in the 4D joint image space, followed by global consistency enforcement for extracting the fundamental matrices corresponding to independent motions. We have performed extensive experiments to compare our method with some representative algorithms to show that better performance on nonstatic scenes are achieved. Results on challenging data sets are presented.     

Visual correction for mobile robot homing

We present a method to send a mobile robot to locations specified by images previously taken from these positions, which sometimes has been referred as homing. Classically this has been carried out using the fundamental matrix, but the fundamental matrix is ill conditioned with planar scenes, which are quite usual in man made environments. Many times in robot homing, small baseline images with high disparity due to rotation are compared, where the fundamental matrix also gives bad results. We use a monocular vision system and we compute motion through an homography obtained from automatically matched lines. In this work we compare the use of the homography and the fundamental matrix and we propose the correction of motion directly from the parameters of the 2D homography, which only needs one calibration parameter. It is shown that it is robust, sufficiently accurate and simple.     

Evaluation of texture registration by epipolar geometry

In the process of digitizing the geometry and appearance of 3D objects, texture registration is a necessary step that solves the 2D–3D mapping between the 2D texture images and the 3D geometric model. For evaluation of texture registration with ground truth, accurate datasets can be obtained with a complex setup consisting of calibrated geometry and texture capture devices. We do not have any knowledge of such evaluation performed; current evaluations reflect, at their best, the precision achieved by the algorithms, but fail to identify a possible bias. We propose a new evaluation measure based on the epipolar geometry of texture image pairs, with the advantage that the ground truth can be extracted solely from the texture images, independent of the 3D acquisition. We developed a noise model suitable to our purpose and analysed three distance measures based on epipolar geometry, well known in the computer vision community, to find new theoretical and experimental results. Finally, using the proposed framework, we evaluated a texture registration algorithm based on mutual information and found that its accuracy was under half-pixel.     

Stereo reconstruction from multiperspective panoramas

A new approach to computing a panoramic (360 degrees) depth map is presented in this paper. Our approach uses a large collection of images taken by a camera whose motion has been constrained to planar concentric circles. We resample regular perspective images to produce a set of multiperspective panoramas and then compute depth maps directly from these resampled panoramas. Our panoramas sample uniformly in three dimensions: rotation angle, inverse radial distance, and vertical elevation. The use of multiperspective panoramas eliminates the limited overlap present in the original input images and, thus, problems as in conventional multibaseline stereo can be avoided. Our approach differs from stereo matching of single-perspective panoramic images taken from different locations, where the epipolar constraints are sine curves. For our multiperspective panoramas, the epipolar geometry, to the first order approximation, consists of horizontal lines. Therefore, any traditional stereo algorithm can be applied to multiperspective panoramas with little modification. In this paper, we describe two reconstruction algorithms. The first is a cylinder sweep algorithm that uses a small number of resampled multiperspective panoramas to obtain dense 3D reconstruction. The second algorithm, in contrast, uses a large number of multiperspective panoramas and takes advantage of the approximate horizontal epipolar geometry inherent in multiperspective panoramas. It comprises a novel and efficient 1D multibaseline matching technique, followed by tensor voting to extract the depth surface. Experiments show that our algorithms are capable of producing comparable high quality depth maps which can be used for applications such as view interpolation.     

Collision Detection and Safe Reaction Algorithm for Non-backdrivable Manipulator with Single Force/Torque Sensor

The paper presents a robust control law for homing of an autonomous robot. The proposed work aims to solve this problem for practical conditions such as random errors in commanded velocities and unknown distance sensor characteristics. The proposed steering control aligns the robot’s orientation with homing vector using arbitrary real valued distance function providing the capability to work in changing environment conditions. Finite time convergence to the equilibrium using proposed control law is achieved in the presence of bounded random velocity errors regardless of the initial position and orientation. Just the sign information as feedback supports applicability of proposed control law with any distance function. A matching parameter between panoramic images obtained at home and current positions is a function of distance between home and current positions. However, explicit relation between distance and image matching parameter is unknown. This work demonstrates the application of proposed method for visual homing based on image distance function rendering the benefit of minimal image processing. Various simulation and experimental results are presented for visual homing to support the theory presented in this paper. Advantage of proposed visual homing is also explored in changing environment conditions.     

Angle-based homing from a reference image set using the 1D trifocal tensor

This paper presents a visual homing method for a robot moving on the ground plane. The approach employs a set of omnidirectional images acquired previously at different locations (including the goal position) in the environment, and the current image taken by the robot. We present as contribution a method to obtain the relative angles between all these locations, using the computation of the 1D trifocal tensor between views and an indirect angle estimation procedure. The tensor is particularly well suited for planar motion and provides important robustness properties to our technique. Another contribution of our paper is a new control law that uses the available angles, with no range information involved, to drive the robot to the goal. Therefore, our method takes advantage of the strengths of omnidirectional vision, which provides a wide field of view and very precise angular information. We present a formal proof of the stability of the proposed control law. The performance of our approach is illustrated through simulations and different sets of experiments with real images.     

一种骨外科手术机器人的图像导航方法

精确目标定位和手术路径规划是将机器人用于长骨骨折内固定手术中的关键技术.文中使用了局部法对X光图像进行畸变校正;并提出了一种基于几何模型的对准方法,得到一条能够穿过髓内钉远端孔的路径,从而引导串联机器人锁定髓内钉.实验表明,该导航方法仅需一张X光图像,有足够的定位精度和稳定性,可大大减少手术时间,降低医生所受辐射剂量.     

架空输电线路巡线机器人的视觉导航

巡线机器人沿相线行走时必须探测识别和定位各种障碍物,并根据障碍类型规划越障行为。针对220 kV架空输电线路的结构特点,利用视觉传感器,设计了基于结构约束的障碍识别算法,完成了障碍识别和分类。根据障碍物的结构特点,设计了一种自适应多窗口区域立体匹配算法,实现了障碍物的双目视觉定位。模拟线路实验结果表明,算法能可靠地从复杂背景中识别并定位出防振锤、悬垂线夹和耐张线夹等障碍物,满足了巡线机器人导航要求。