Outdoor Localization Using Stereo Vision Under Various Illumination Conditions

We present a mobile robot localization method using only a stereo camera. Vision-based localization in outdoor environments is a challenging issue because of extreme changes in illumination. To cope with varying illumination conditions, we use two-dimensional occupancy grid maps generated from three-dimensional point clouds obtained by a stereo camera. Furthermore, we incorporate salient line segments extracted from the ground into the grid maps. The grid maps are not significantly affected by illumination conditions because occupancy information and salient line segments can be robustly obtained. On the grid maps, a robot's poses are estimated using a particle filter that combines visual odometry and map matching. We use edge-point-based stereo simultaneous localization and mapping to obtain simultaneously occupancy information and robot ego-motion estimation. We tested our method under various illumination and weather conditions, including sunny and rainy days. The experimental results showed the effectiveness and robustness of the proposed method. Our method enables localization under extremely poor illumination conditions, which are challenging for even existing state-of-the-art methods.     

Online learning and fusion of orientation appearance models for robust rigid object tracking

We introduce a robust framework for learning and fusing of orientation appearance models based on both texture and depth information for rigid object tracking. Our framework fuses data obtained from a standard visual camera and dense depth maps obtained by low-cost consumer depth cameras such as the Kinect. To combine these two completely different modalities, we propose to use features that do not depend on the data representation: angles. More specifically, our framework combines image gradient orientations as extracted from intensity images with the directions of surface normals computed from dense depth fields. We propose to capture the correlations between the obtained orientation appearance models using a fusion approach motivated by the original Active Appearance Models (AAMs). To incorporate these features in a learning framework, we use a robust kernel based on the Euler representation of angles which does not require off-line training, and can be efficiently implemented online. The robustness of learning from orientation appearance models is presented both theoretically and experimentally in this work. This kernel enables us to cope with gross measurement errors, missing data as well as other typical problems such as illumination changes and occlusions. By combining the proposed models with a particle filter, the proposed framework was used for performing 2D plus 3D rigid object tracking, achieving robust performance in very difficult tracking scenarios including extreme pose variations.     

一种新型的并联机器人位姿立体视觉检测系统

建立了一种并联机器人位姿立体视觉测量系统框架,主要包括图像采集与传输、摄像机标定、尺度不变量特征变换（SIFT）匹配、空间点重建和位姿测量五个部分。该系统基于SIFT,能够很好地处理图像在大视角有遮挡、平移、旋转、亮度和尺度变化时的特征点匹配,有较高的匹配精度,特别适用于对并联机器人多自由度和空间复杂运动的检测。最后使用该方法对并联机器人位姿检测做了仿真实验。     

Depth incorporating with color improves salient object detection

Detecting salient objects in challenging images attracts increasing attention as many applications require more robust method to deal with complex images from the Internet. Prior methods produce poor saliency maps in challenging cases mainly due to the complex patterns in the background and internal color edges in the foreground. The former problem may introduce noises into saliency maps and the later forms the difficulty in determining object boundaries. Observing that depth map can supply layering information and more reliable boundary, we improve salient object detection by integrating two features: color information and depth information which are calculated from stereo images. The two features collaborate in a two-stage framework. In the object location stage, depth mainly helps to produce a noise-filtered salient patch, which indicates the location of the object. In the object boundary inference stage, boundary information is encoded in a graph using both depth and color information, and then we employ the random walk to infer more reliable boundaries and obtain the final saliency map. We also build a data set containing 100+ stereo pairs to test the effectiveness of our method. Experiments show that our depth-plus-color based method significantly improves salient object detection compared with previous color-based methods.     

星球漫游车超广角实时立体视觉系统

给出了一种用于星球漫游车障碍检测和定位的超大视场立体视觉系统的实现方法. 该系统采用具有超广角镜头(对角视场角约160度)的双目或三目摄像机获取场景立体图像 对,利用摄像机标定参数对大变形图像进行修正等预处理,然后在外极线、连续性等约束条 件下,基于查找表和Intel MMx指令集,使用SAD算法快速进行对应点匹配计算.实验表 明,该系统在图像分辨率为320×120像素、视差为64级时,利用普通工控机恢复稠密深度 图的速度为10帧/秒,并能使机器人以1米/秒的速度行走.     

Appearance-based mapping and localization for mobile robots using a feature stability histogram

The strength of appearance-based mapping models for mobile robots lies in their ability to represent the environment through high-level image features and to provide human-readable information. However, developing a mapping and a localization method using these kinds of models is very challenging, especially if robots must deal with long-term mapping, localization, navigation, occlusions, and dynamic environments. In other words, the mobile robot has to deal with environmental appearance change, which modifies its representation of the environment. This paper proposes an indoor appearance-based mapping and a localization method for mobile robots based on the human memory model, which was used to build a Feature Stability Histogram (FSH) at each node in the robot topological map. This FSH registers local feature stability over time through a voting scheme, and the most stable features were considered for mapping, for Bayesian localization and for incrementally updating the current appearance reference view in the topological map. The experimental results are presented using an omnidirectional images dataset acquired over the long-term and considering: illumination changes (time of day, different seasons), occlusions, random removal of features, and perceptual aliasing. The results include a comparison with the approach proposed by Dayoub and Duckett (2008) [19] and the popular Bag-of-Words (Bazeille and Filliat, 2010) [35] approach. The obtained results confirm the viability of our method and indicate that it can adapt the internal map representation over time to localize the robot both globally and locally.     

Wide-baseline stereo vision for terrain mapping

Terrain mapping is important for mobile robots to perform localization and navigation. Stereo vision has been used extensively for this purpose in outdoor mapping tasks. However, conventional stereo does not scale well to distant terrain. This paper examines the use of wide-baseline stereo vision in the context of a mobile robot for terrain mapping, and we are particularly interested in the application of this technique to terrain mapping for Mars exploration. In wide-baseline stereo, the images are not captured simultaneously by two cameras, but by a single camera at different positions. The larger baseline allows more accurate depth estimation of distant terrain, but the robot motion between camera positions introduces two new problems. One issue is that the robot estimates the relative positions of the camera at the two locations imprecisely, unlike the precise calibration that is performed in conventional stereo. Furthermore, the wide-baseline results in a larger change in viewpoint than in conventional stereo. Thus, the images are less similar and this makes the stereo matching process more difficult. Our methodology addresses these issues using robust motion estimation and feature matching. We give results using real images of terrain on Earth and Mars and discuss the successes and failures of the technique.     

A stereo matching approach based on particle filters and scattered control landmarks

In robot localization, particle filtering can estimate the position of a robot in a known environment with the help of sensor data. In this paper, we present an approach based on particle filtering, for accurate stereo matching. The proposed method consists of three parts. First, we utilize multiple disparity maps in order to acquire a very distinctive set of features called landmarks, and then we use segmentation as a grouping technique. Secondly, we apply scan line particle filtering using the corresponding landmarks as a virtual sensor data to estimate the best disparity value. Lastly, we reduce the computational redundancy of particle filtering in our stereo correspondence with a Markov chain model, given the previous scan line values. More precisely, we assist particle filtering convergence by adding a proportional weight in the predicted disparity value estimated by Markov chains. In addition to this, we optimize our results by applying a plane fitting algorithm along with a histogram technique to refine any outliers. This work provides new insights into stereo matching methodologies by taking advantage of global geometrical and spatial information from distinctive landmarks. Experimental results show that our approach is capable of providing high-quality disparity maps comparable to other well-known contemporary techniques.     

A wavelet-based multiresolution approach to solve the stereo correspondence problem using mutual information.

In this correspondence, we propose a wavelet-based hierarchical approach using mutual information (MI) to solve the correspondence problem in stereo vision. The correspondence problem involves identifying corresponding pixels between images of a given stereo pair. This results in a disparity map, which is required to extract depth information of the relevant scene. Until recently, mostly correlation-based methods have been used to solve the correspondence problem. However, the performance of correlation-based methods degrades significantly when there is a change in illumination between the two images of the stereo pair. Recent studies indicate MI to be a more robust stereo matching metric for images affected by such radiometric distortions. In this short correspondence paper, we compare the performances of MI and correlation-based metrics for different types of illumination changes between stereo images. MI, as a statistical metric, is computationally more expensive. We propose a wavelet-based hierarchical technique to counter the increase in computational cost and show its effectiveness in stereo matching.     

Combining Shape from Shading and Stereo: A Joint Variational Method for Estimating Depth,Illumination and Albedo

Shape from shading (SfS) and stereo are two fundamentally different strategies for image-based 3-D reconstruction. While approaches for SfS infer the depth solely from pixel intensities, methods for stereo are based on a matching process that establishes correspondences across images. This difference in approaching the reconstruction problem yields complementary advantages that are worthwhile being combined. So far, however, most “joint” approaches are based on an initial stereo mesh that is subsequently refined using shading information. In this paper we follow a completely different approach. We propose a joint variational method that combines both cues within a single minimisation framework. To this end, we fuse a Lambertian SfS approach with a robust stereo model and supplement the resulting energy functional with a detail-preserving anisotropic second-order smoothness term. Moreover, we extend the resulting model in such a way that it jointly estimates depth, albedo and illumination. This in turn makes the approach applicable to objects with non-uniform albedo as well as to scenes with unknown illumination. Experiments for synthetic and real-world images demonstrate the benefits of our combined approach: They not only show that our method is capable of generating very detailed reconstructions, but also that joint approaches are feasible in practice.