栅格法视觉传感集成及机器人实时避障

研究移动机器人在室内环境下集成双目视觉和激光测距仪信息进行障碍物实时检测。由双目视觉系统检测环境获取视差信息,通过直接对视差信息进行地平面拟合的方法快速检测障碍物;拟合过程中采用了随机采样一致性估计算法去除干扰点的影响,提高了障碍物检测的鲁棒性。用栅格地图表示基于机器人坐标系的地平面障碍物信息并对栅格信息进行提取,最后把双目视觉与激光测距得到的栅格信息进行集成。实验表明,通过传感信息集成,移动机器人既得到了充分的障碍物信息,又保证了检测的实时性、准确性。     

3-D shape reconstruction in an active stereo vision system using genetic algorithms

The recovery of 3-D shape information (depth) using stereo vision analysis is one of the major areas in computer vision and has given rise to a great deal of literature in the recent past. The widely known stereo vision methods are the passive stereo vision approaches that use two cameras. Obtaining 3-D information involves the identification of the corresponding 2-D points between left and right images. Most existing methods tackle this matching task from singular points, i.e. finding points in both image planes with more or less the same neighborhood characteristics. One key problem we have to solve is that we are on the first instance unable to know a priori whether a point in the first image has a correspondence or not due to surface occlusion or simply because it has been projected out of the scope of the second camera. This makes the matching process very difficult and imposes a need of an a posteriori stage to remove false matching.In this paper we are concerned with the active stereo vision systems which offer an alternative to the passive stereo vision systems. In our system, a light projector that illuminates objects to be analyzed by a pyramid-shaped laser beam replaces one of the two cameras. The projections of laser rays on the objects are detected as spots in the image. In this particular case, only one image needs to be treated, and the stereo matching problem boils down to associating the laser rays and their corresponding real spots in the 2-D image. We have expressed this problem as a minimization of a global function that we propose to perform using Genetic Algorithms (GAs). We have implemented two different algorithms: in the first, GAs are performed after a deterministic search. In the second, data is partitioned into clusters and GAs are independently applied in each cluster. In our second contribution in this paper, we have described an efficient system calibration method. Experimental results are presented to illustrate the feasibility of our approach. The proposed method yields high accuracy 3-D reconstruction even for complex objects. We conclude that GAs can effectively be applied to this matching problem.     

Development of a laser range finder for three-dimensional map building in rubble

In order to search and rescue victims in rubble effectively, a three-dimensional (3D) map of the rubble is required. As a part of the national project on rescue robot systems, we are investigating a method for constructing a 3D map of rubble by teleoperated mobile robots. In this paper, we developed a laser range finder for 3D map building in rubble. The developed range finder consists of a ring laser beam module and an omnivison camera. The ring laser beam is generated by using a conical mirror and it is radiated toward the interior wall of the rubble around a mobile robot on which the laser range finder is mounted. The ominivison camera with a hyperbolic mirror can capture the reflected image of the ring laser on the rubble. Based on the triangulation principle, cross-section range data is obtained. Continuing this measurement as the mobile robot moves inside the rubble, a 3D map is obtained. We constructed a geometric model of the laser range finder for error analysis and obtained an optimal dimension of the laser range finder. Based on this analysis, we actually prototyped a range finder. Experimental results show that the actual measurement errors are well matched to the theoretical values. Using the prototyped laser range finder, a 3D map of rubble was actually built with reasonable accuracy.     

移动机器人基于激光测距和单目视觉的室内同时定位和地图构建

该文研究了部分结构化室内环境中自主移动机器人同时定位和地图构建问题.基于激光和视觉传感器模型的不同,加权最小二乘拟合方法和非局部最大抑制算法被分别用于提取二维水平环境特征和垂直物体边缘.为完成移动机器人在缺少先验地图支持的室内环境中的自主导航任务,该文提出了同时进行扩展卡尔曼滤波定位和构建具有不确定性描述的二维几何地图的具体方法.通过对于SmartROB-2移动机器人平台所获得的实验结果和数据的分析讨论,论证了所提出方法的有效性和实用性.     

A Stochastic Map Building Method for Mobile Robot using 2-D Laser Range Finder

This paper presents a stochastic map building method for mobile robot using a 2-D laser range finder. Unlike other methods that are based on a set of geometric primitives, the presented method builds a map with a set of obstacle regions. In building a map of the environment, the presented algorithm represents the obstacles with a number of stochastic obstacle regions, each of which is characterized by its own stochastic parameters such as mean and covariance. Whereas the geometric primitives based map sometimes does not fit well to sensor data, the presented method reliably represents various types of obstacles including those of irregular walls and sets of tiny objects. Their shapes and features are easily extracted from the stochastic parameters of their obstacle regions, and are used to develop reliable navigation and obstacle avoidance algorithms. The algorithm updates the world map in real time by detecting the changes of each obstacle region. Consequently, it is adequate for modeling the quasi-static environment, which includes occasional changes in positions of the obstacles rather than constant dynamic moves of the obstacles. The presented map building method has successfully been implemented and tested on the ARES-II mobile robot system equipped with a LADAR 2D-laser range finder.     

On-line road boundary modeling with multiple sensory features, flexible road model, and particle filter

This paper describes a method of robustly modeling road boundaries on-line for autonomous navigation. Since sensory evidence for road boundaries might change from place to place, we cannot depend on a single cue but have to use multiple sensory features. It is also necessary to cope with various road shapes and road type changes. These requirements are naturally met in the proposed particle filter-based method, which makes use of multiple features with the corresponding likelihood functions and keeps multiple road hypotheses as particles. The proposed method has been successfully applied to various road scenes with cameras and a laser range finder. To show that the proposed method is applicable to other sensors, preliminary results of using stereo instead of the laser range finder are also described.     

Extrinsic calibration of a camera and laser range finder using a new calibration structure of a plane with a triangular hole

Sensor fusion of a camera and laser range finder is important for the autonomous navigation of mobile robots. Finding the transformation between the camera and laser range finder is the first necessary step for the fusion of information. Many algorithms have been proposed, but these tend to require many different steps in order to achieve reliable and accurate results. A calibration structure that has triangular hole on its plane is proposed for the extrinsic calibration of a camera and laser range finder. Locations of laser scan data that are invisible on the calibration plane can be determined using property on the proposed calibration structure. First, we classify the laser scan data into two groups where one is on the plane and the other is off the plane. Then, we determine the absolute location of the laser scan data on the plane through a search of the parameters of the line. Finally, we can establish 3D-3D correspondences between the camera and laser range finder. Extrinsic calibration between a camera and laser range finder is found using a conventional 3D-3D transformation computing algorithm. Keywords: Calibration k]camera k]extrinsic calibration k]laser range finder     

基于激光测距仪的障碍物检测的仿真研究

为了进行准确和有效的导航,提出了利用机器人的车载激光测距仪来检测环境中的障碍物.利用激光测距仪对机器人的局部环境建立栅格地图,在该模型下对障碍物进行检测.针对连续变化时刻(t-1)和(t)下的两帧“图像”,在极坐标系中进行障碍物分割,根据分割结果建立障碍物链Object_ List (t-1)和Object_ List (t),计算出各特征参数,并作进一步的匹配和分类分析.如果存在动态障碍物,估计出相应的运动参数,更新得到t时刻的动态障碍物集Dob (N).为了验证算法的有效性,利用visual C++进行仿真,仿真实验结果表明,该方法进行环境中的动静态障碍物的检测是可行的和有效的.     

一种用于立体图像匹配的改进稀疏匹配算法

立体匹配有着广泛的应用前景,是计算机视觉领域的研究热点。立体匹配是立体视觉中最为关键和困难的一步,它的目标是计算标识匹配像素位置的视差图。文中提出的立体匹配算法基于置信传播（Belief Propagation,BP）。左图像首先经过非均匀采样,得到一个内容自适应的网格近似表示。算法的关键是使用基于置信传播的立体匹配算法,匹配稀疏的左图像和右图像得到稀疏视差图。通过左图像得到网格,稀疏视差图可以经过简单的插值得到稠密视差图。实验结果表明,该方法与现有稀疏立体匹配技术相比在视差图质量上平均有40％的提高。     

High-resolution terrain map from multiple sensor data

The authors present 3-D vision techniques for incrementally building an accurate 3-D representation of rugged terrain using multiple sensors. They have developed the locus method to model the rugged terrain. The locus method exploits sensor geometry to efficiently build a terrain representation from multiple sensor data. The locus method is used to estimate the vehicle position in the digital elevation map (DEM) by matching a sequence of range images with the DEM. Experimental results from large-scale real and synthetic terrains demonstrate the feasibility and power of the 3-D mapping techniques for rugged terrain. In real world experiments, a composite terrain map was built by merging 125 real range images. Using synthetic range images, a composite map of 150 m was produced from 159 images. With the proposed system, mobile robots operating in rugged environments can build accurate terrain models from multiple sensor data     

A real-time fuzzy hardware structure for disparity map computation

Stereo images acquired by a stereo camera setup provide depth estimation of a scene. Numerous machine vision applications deal with retrieval of 3D information. Disparity map recovery from a stereo image pair involves computationally complex algorithms. Previous methods of disparity map computation are mainly restricted to software-based techniques on general-purpose architectures, presenting relatively high execution time. In this paper, a new hardware-implemented real-time disparity map computation module is realized. This enables a hardware-based fuzzy inference system parallel-pipelined design, for the overall module, implemented on a single FPGA device with a typical operating frequency of 138 MHz. This provides accurate disparity map computation at a rate of nearly 440 frames per second, given a stereo image pair with a disparity range of 80 pixels and 640 × 480 pixels spatial resolution. The proposed method allows a fast disparity map computational module to be built, enabling a suitable module for real-time stereo vision applications.     

基于立体视觉的水下三维测量系统研究

随着我国海洋战略的提出,对于海洋观测技术和装备的需求日趋迫切。针对现有水下成像系统无法实现精确三维测量这一难题,该文提出了一种基于双目立体视觉原理的水下三维测量系统研究方法,并对其可行性进行了验证。针对水下成像过程存在的水体界面折射问题,该文提出了相应的相机成像模型及系统参数标定方法,建立了防水深度达 30 m 的双目水下测量及照明装置,并在水池、近海条件下进行了实地测试。实验结果显示,在水体条件较好的情况下,系统观测距离可达 8 m 以上,有效测量距离为 0.5～4.5 m,在 0.5 m 和 4.5 m 距离处的测量误差分别为 2 mm 和 20 mm。实验验证了 水下双目成像模型、立体标定、测量模型等方法的有效性和精确性,可为水下检修作业等海洋工程行业提供一种有效的三维测量技术手段。     

Occlusions as a guide for planning the next view

A strategy for acquiring 3-D data of an unknown scene, using range images obtained by a light stripe range finder is addressed. The foci of attention are occluded regions, i.e., only the scene at the borders of the occlusions is modeled to compute the next move. Since the system has knowledge of the sensor geometry, it can resolve the appearance of occlusions by analyzing them. The problem of 3-D data acquisition is divided into two subproblems due to two types of occlusions. An occlusion arises either when the reflected laser light does not reach the camera or when the directed laser light does not reach the scene surface. After taking the range image of a scene, the regions of no data due to the first kind of occlusion are extracted. The missing data are acquired by rotating the sensor system in the scanning plane, which is defined by the first scan. After a complete image of the surface illuminated from the first scanning plane has been built, the regions of missing data due to the second kind of occlusions are located. Then, the directions of the next scanning planes for further 3-D data acquisition are computed     

基于单相机的全向深度图获取

为了搜寻移动机器人周围最大的可通行区域,采用全向立体视觉系统,提出获取可靠的致密三维深度图方法。视觉系统由1个普通相机和2个双曲面镜组成。当系统标定后,空间点的三维坐标可以通过匹配上下镜面的成像点计算得出。匹配方法分3步：最大FX匹配,特征匹配和歧义去除。定义合适的能量函数通过动态规划来实现剩余点的匹配。实验表明该系统精度高、具有实用价值。     

Iterative point matching for registration of free-form curves and surfaces

A heuristic method has been developed for registering two sets of 3-D curves obtained by using an edge-based stereo system, or two dense 3-D maps obtained by using a correlation-based stereo system. Geometric matching in general is a difficult unsolved problem in computer vision. Fortunately, in many practical applications, some a priori knowledge exists which considerably simplifies the problem. In visual navigation, for example, the motion between successive positions is usually approximately known. From this initial estimate, our algorithm computes observer motion with very good precision, which is required for environment modeling (e.g., building a Digital Elevation Map). Objects are represented by a set of 3-D points, which are considered as the samples of a surface. No constraint is imposed on the form of the objects. The proposed algorithm is based on iteratively matching points in one set to the closest points in the other. A statistical method based on the distance distribution is used to deal with outliers, occlusion, appearance and disappearance, which allows us to do subset-subset matching. A least-squares technique is used to estimate 3-D motion from the point correspondences, which reduces the average distance between points in the two sets. Both synthetic and real data have been used to test the algorithm, and the results show that it is efficient and robust, and yields an accurate motion estimate.     

Traversability Analysis and Path Planning for a Planetary Rover

A method of analyzing three-dimensional data such as might be produced by stereo vision or a laser range finder in order to plan a path for a vehicle such as a Mars rover is described. In order to produce robust results from data that is sparse and of varying accuracy, the method takes into account the accuracy of each data point, as represented by its covariance matrix. It computes estimates of smoothed and interpolated height, slope, and roughness at equally spaced horizontal intervals, as well as accuracy estimates of these quantities. From this data, a cost function is computed that takes into account both the distance traveled and the probability that each region is traversable. A parallel search algorithm that finds the path of minimum cost also is described. Examples using real data are presented.     

FPGA based disparity map computation with vergence control

Depth estimation in a scene using image pairs acquired by a stereo camera setup, is one of the important tasks of stereo vision systems. The disparity between the stereo images allows for 3D information acquisition which is indispensable in many machine vision applications. Practical stereo vision systems involve wide ranges of disparity levels. Considering that disparity map extraction of an image is a computationally demanding task, practical real-time FPGA based algorithms require increased device utilization resource usage, depending on the disparity levels operational range, which leads to significant power consumption. In this paper a new hardware-efficient real-time disparity map computation module is developed. The module constantly estimates the precisely required range of disparity levels upon a given stereo image set, maintaining this range as low as possible by verging the stereo setup cameras axes. This enables a parallel-pipelined design, for the overall module, realized on a single FPGA device of the Altera Stratix IV family. Accurate disparity maps are computed at a rate of more than 320 frames per second, for a stereo image pair of 640 × 480 pixels spatial resolution with a disparity range of 80 pixels. The presented technique provides very good processing speed at the expense of accuracy, with very good scalability in terms of disparity levels. The proposed method enables a suitable module delivering high performance in real-time stereo vision applications, where space and power are significant concerns.     

Inertial Sensed Ego‐motion for 3D Vision

Inertial sensors attached to a camera can provide valuable data about camera pose and movement. In biological vision systems, inertial cues provided by the vestibular system are fused with vision at an early processing stage. In this article we set a framework for the combination of these two sensing modalities. Cameras can be seen as ray direction measuring devices, and in the case of stereo vision, depth along the ray can also be computed. The ego‐motion can be sensed by the inertial sensors, but there are limitations determined by the sensor noise level. Keeping track of the vertical direction is required, so that gravity acceleration can be compensated for, and provides a valuable spatial reference. Results are shown of stereo depth map alignment using the vertical reference. The depth map points are mapped to a vertically aligned world frame of reference. In order to detect the ground plane, a histogram is performed for the different heights. Taking the ground plane as a reference plane for the acquired maps, the fusion of multiple maps reduces to a 2D translation and rotation problem. The dynamic inertial cues can be used as a first approximation for this transformation, allowing a fast depth map registration method. They also provide an image independent location of the image focus of expansion and center of rotation useful during visual based navigation tasks. © 2004 Wiley Periodicals, Inc.     

Autonomous Driving with Concurrent Goals and Multiple Vehicles: Experiments and Mobility Components

In this paper, we describe a complete system for mission planning and execution for multiple robots in natural terrain. We report on experiments with a system for autonomously driving two vehicles based on complex mission specifications. We show that the system is able to plan local paths in obstacle fields based on sensor data, to plan and update global paths to goals based on frequent obstacle map updates, and to modify mission execution, e.g., the assignment and ordering of the goals, based on the updated paths to the goals.Two recently developed sensors are used for obstacle detection: a high-speed laser range finder, and a video-rate stereo system. An updated version of a dynamic path planner, D*, is used for on-line computation of routes. A new mission planning and execution-monitoring tool, GRAMMPS, is used for managing the allocation and ordering of goals between vehicles.We report on experiments conducted in an outdoor test site with two HMMWVs. Implementation details and performance analysis, including failure modes, are described based on a series of twelve experiments, each over 1/2 km distance with up to nine goals.The work reported here includes a number of results not previously published, including the use of a real-time stereo machine and a high-performance laser range finder, and the use of the GRAMMPS planning system.     

双目视觉的新颖实时局部2维地图构建方法

根据轮式机器人移动的特点,提出一种采用双目视觉的新颖实时局部2维栅格地图构建算法。首先,提出虚拟高度线投影成像原理,将场景均匀栅格化,在栅格中引入虚拟高度线,并将其投影到立体视觉系统的立体图对中产生投影线,将求解场景点高度值的问题转化为在投影线上和水平视差搜索范围内寻找具有最大相似测度的对应点问题;然后,提出一种新颖的局部2维地图构建方法,该方法以机器人所能越过的最大高度为阈值,对高于阈值部分的虚拟高度投影线上的场景点由其在图像对中的相似测度确定其是否为障碍物区域。实验结果表明,该方法满足机器人导航所要求的有效性和实时性,并可应用到构建3维地图。