An active trinocular vision system of sensing indoor navigation environment for mobile robots

Intelligent autonomous mobile robots must be able to sense and recognize 3D indoor space where they live or work. However, robots are frequently situated in cluttered environments with various objects hard to be robustly perceived. Although the monocular and binocular vision sensors have been widely used for mobile robots, they suffer from image intensity variations, insufficient feature information and correspondence problems. In this paper, we propose a new 3D sensing system, in which the laser-structured-lighting method is basically utilized because of the robustness on the nature of the navigation environment and the easy extraction of feature information of interest. The proposed active trinocular vision system is composed of the flexible multi-stripe laser projector and two cameras arranged with a triangular shape. By modeling the laser projector as a virtual camera and using the trinocular epipolar constraints, the matching pairs of line features observed into two real camera images are established, and 3D information from one-shot image can be extracted on the patterned scene. For robust feature matching, here we propose a new correspondence matching technique based on line grouping and probabilistic voting. Finally, a series of experimental tests is performed to show the simplicity, efficiency, and accuracy of this proposed sensor system for 3D environment sensing and recognition.     

基于自适应聚合的立体视觉合作算法

提出了一种恢复高质量稠密视差图的立体视觉合作算法.该算法采用基于形态学相似性的自适应加权方法,迭代地进行局部邻域的自适应聚合和抑制放大,实现高效率和高质量稠密视差图计算.将该算法推广到三目摄像机立体匹配系统中,通过重建摄像机坐标系实现图像校正,并根据连续性假设和唯一性假设,建立视差空间中的支持关系和三目摄像机之间的抑制关系.实验结果表明,三目立体合作算法能够得到精确的场景视差映射,并可以实现多基线方向的遮挡检测.该算法特别适用于由多个廉价摄像机组成的立体视觉系统,在几乎不增加软件和硬件资源的情况下,就可以得到高质量的稠密视差图.     

A loop-consistency measure for dense correspondences in multi-view video

Many applications in computer vision and computer graphics require dense correspondences between images of multi-view video streams. Most state-of-the-art algorithms estimate correspondences by considering pairs of images. However, in multi-view videos, several images capture nearly the same scene. In this article we show that this redundancy can be exploited to estimate more robust and consistent correspondence fields. We use the multi-video data structure to establish a confidence measure based on the consistency of the correspondences in a loop of three images. This confidence measure can be applied after flow estimation is terminated to find the pixels for which the estimate is reliable. However, including the measure directly into the estimation process yields dense and highly accurate correspondence fields. Additionally, application of the loop consistency confidence measure allows us to include sparse feature matches directly into the dense optical flow estimation. With the confidence measure, spurious matches can be successfully suppressed during optical flow estimation while correct matches contribute to increase the accuracy of the flow.     

Trinocular vision for AGV guidance: Path localization and obstacle detection

This paper presents an integrated approach to the problem of AGV guidance using trinocular vision. Although this method uses three cameras, we will not do any processings related to stereo matching nor 3-D reconstruction. Instead, our trinocular vision is divided into two subsystems: monocular vision for path localization and binocular vision for obstacle detection. First, we demonstrate the geometric principle of 2-D vision. Then, we show its application for the task of AGV's path localization as well as the task of AGV's obstacle detection in the framework of trinocular vision. Experimental results with real images prove the usefulness of our method.     

Three-Dimensional Reconstruction of Points and Lines with Unknown Correspondence across Images

Three-dimensional reconstruction from a set of images is an important and difficult problem in computer vision. In this paper, we address the problem of determining image feature correspondences while simultaneously reconstructing the corresponding 3D features, given the camera poses of disparate monocular views. First, two new affinity measures are presented that capture the degree to which candidate features from different images consistently represent the projection of the same 3D point or 3D line. An affinity measure for point features in two different views is defined with respect to their distance from a hypothetical projected 3D pseudo-intersection point. Similarly, an affinity measure for 2D image line segments across three views is defined with respect to a 3D pseudo-intersection line. These affinity measures provide a foundation for determining unknown correspondences using weighted bipartite graphs representing candidate point and line matches across different images. As a result of this graph representation, a standard graph-theoretic algorithm can provide an optimal, simultaneous matching and triangulation of points across two views, and lines across three views. Experimental results on synthetic and real data demonstrate the effectiveness of the approach.An erratum to this article can be found at     

A New Approach to Pose Detection using a Trinocular Stereovision System

Object location and tracking is a major issue in computer vision. This problem is normally solved through the extraction of representative features of the object, and the two-dimensional coordinates of these image features are used to compute the position of the object. When more than one camera is used, a certain similarity measure between the image features extracted from both stereoscopic images helps to match the correspondences. In this way, three-dimensional measurements can be recovered from the 2D coordinates of the features extracted from different cameras. In this paper the use of a trinocular system is considered to estimate both the position and velocity of known objects by using their apparent area, and with no use of the image-plane coordinates of the object 's features. A high precision low-level image processor has been developed for performing object labeling and noise filtering of the images at video rate. Then, a position measurement tool uses the apparent area captured by every camera to locate the object. This enables us to estimate the position of the object. Finally, a prediction tool refines the estimation in locating the object. We show the performance of the trinocular system with a real implementation. This system has been designed to process the images provided by any conventional of high-speed cameras at video rate.     

图像三维重建中的特征点匹配方法研究

针对传统三维重建中点特征匹配算法的局限性,提出了基于相关系数的图像点特征匹配方法,实现了对图像点特征匹配与三维重建同步进行的目的。基于同一物体的两幅图像,给出了其特征匹配和三维重建的算法,并通过实验数据分析了此算法的有效性。     

基于多视角的高精度三维人脸重建

提出了一种多阶段优化的方法来解决基于多视角图片在未知姿态、表情以及光照条件下的高精度三维人脸重建问题.首先,通过重新渲染合成的方法将参数化模型拟合到输入的多视角图片,然后在纹理域上求解一个光流问题来获取不同视角之间的对应关系.通过对应关系可以恢复出人脸的点云,并利用基于明暗恢复几何的方法来恢复人脸细节.在真实数据以及合成数据下的实验结果表明,文中方法能够恢复出带有几何细节的高精度的三维人脸模型,并且提高了现有方法的重建精度.     

基于单幅光模板的高分辨率可靠3维形貌重建

如今在计算机视觉领域,基于结构光从2维图像重建物体的3维轮廓已成为研究热点。然而低分辨率、纹理受限、对光照敏感是此项技术实际应用的主要制约。由于结构光视觉的关键技术是编码解码,因此为了进行高分辨率可靠的3维形貌重建,提出了一种新的彩色编码结构光技术,用来对单幅图像进行3维重建。该技术首先设计一个基于De Bruijn序列的等距白色光条间隔的彩色光模板,同时采用K均值聚类的方法来匹配投影仪和摄像机平面之间对应的光条颜色。然后引入聚类的统计特征量,以自适应不同场景、纹理,使之能够在无控光照条件下实现光条边界局部的精确定位、模板正确匹配和解码。另外,针对静态场景,同一光模板相移方法还可以获取高分辨率的3维形貌,可适用于高密度重建任务。实验结果表明,该方法可在日常光照下重建高分辨率可靠3维模型,并可提高3维形貌获取的适用条件。     

Automatic 3d free form shape matching using the graduated assignment algorithm

Three-dimensional free form shape matching is a fundamental problem in both the machine vision and pattern recognition literatures. However, the automatic approach to 3D free form shape matching still remains open. In this paper, we propose using k closest points in the second view for the automatic 3D free form shape matching. For the sake of computational efficiency, the optimised k-D tree is employed for the search of the k closest points. Since occlusion and appearance and disappearance of points almost always occur, slack variables have to be employed, explicitly modelling outliers in the process of matching. Then the relative quality of each possible point match is estimated using the graduated assignment algorithm, leading the camera motion parameters to be estimated by the quaternion method in the weighted least-squares sense. The experimental results based on both synthetic data and real images without any pre-processing show the effectiveness and efficiency of the proposed algorithm for the automatic matching of overlapping 3D free form shapes with either sparse or dense points.     

A 2-point algorithm for 3D reconstruction of horizontal lines from a single omni-directional image

Reconstruction of 3D scenes with abundant straight line features has many applications in computer vision and robot navigation. Most approaches to this problem involve stereo techniques, in which a solution to the correspondence problem between at least two different images is required. In contrast, 3D reconstruction of straight horizontal lines from a single 2D omni-directional image is studied in this paper. The authors show that, for symmetric non-central catadioptric systems, a 3D horizontal line can be estimated using only two points extracted from a single image of the line. One of the two points is the symmetry point of the image curve of horizontal line, and the other is a generic point on the image curve. This paper improves on several prior works, including horizontal line detection in omni-directional image and line reconstruction from four viewing rays, but is simpler than those methods while being more robust. We evaluate how the precision of feature point extraction can affect line reconstruction accuracy, and discuss preliminary experimental results.     

Shape Reconstruction of 3D Bilaterally Symmetric Surfaces

The paper presents a new approach for shape recovery based on integrating geometric and photometric information. We consider 3D bilaterally symmetric objects, that is, objects which are symmetric with respect to a plane (e.g., faces), and their reconstruction from a single image. Both the viewpoint and the illumination are not necessarily frontal. Furthermore, no correspondence between symmetric points is required.The basic idea is that an image taken from a general, non frontal viewpoint, under non-frontal illumination can be regarded as a pair of images. Each image of the pair is one half of the object, taken from different viewing positions and with different lighting directions. Thus, one-image-variants of geometric stereo and of photometric stereo can be used. Unlike the separate invocation of these approaches, which require point correspondence between the two images, we show that integrating the photometric and geometric information suffice to yield a dense correspondence between pairs of symmetric points, and as a result, a dense shape recovery of the object. Furthermore, the unknown lighting and viewing parameters, are also recovered in this process.Unknown distant point light source, Lambertian surfaces, unknown constant albedo, and weak perspective projection are assumed. The method has been implemented and tested experimentally on simulated and real data.     

基于单相机的全向深度图获取

为了搜寻移动机器人周围最大的可通行区域,采用全向立体视觉系统,提出获取可靠的致密三维深度图方法。视觉系统由1个普通相机和2个双曲面镜组成。当系统标定后,空间点的三维坐标可以通过匹配上下镜面的成像点计算得出。匹配方法分3步：最大FX匹配,特征匹配和歧义去除。定义合适的能量函数通过动态规划来实现剩余点的匹配。实验表明该系统精度高、具有实用价值。     

视皮层中的视差计算

对应问题是从图像恢复物体三维几何形状的核心问题,也是计算机视觉领域长久以来没有很好解决的问题.尽管文献中 有大量对应点匹配算法报道,但这些主体基于数学和工程的方法,在鲁棒性方面离人类视觉还有很大的差距.所以,建立基于生理机理的 立体感知模型,可望有效提高匹配的鲁棒性.然而,要建立这种基于生理机制的计算模型, 需要对生物立体感知的机理有所了解.基于此,本文对近年来 关于生物立体感知的进展和主要研究成果进行了总结,包括初级区域V1, V2, V3区,以及 背部通道的MT, MST, IPS和腹部通道的V4, IT区.除了对这些区域 关于神经生理的成果进行介绍外,我们还简单介绍了著名的视差能量模型及其推广模型.本文对研究计算机视觉的人员有一定的参考价值.     

Skeleton-guided 3D shape distance field metamorphosis

We introduce an automatic 3D shape morphing method without the need of manually placing anchor correspondence points. Given a source and a target shape, we extract their skeletons and automatically estimate the meaningful anchor points based on their skeleton node correspondences. Based on the anchors, dense correspondences between the interior of source and target shape can be established using earth mover’s distance (EMD) optimization. Skeleton node correspondence, estimated with a voting-based method, leads to part correspondence which can be used to confine the dense correspondence within matched part pairs. This produces smooth and plausible morphing sequence based on distance field interpolation (DFI). We demonstrate the effectiveness of our algorithm over shapes with large geometric variation and structural difference.     

Towards reliable matching of images containing repetitive patterns

This paper aims to solve the problem of matching images containing repetitive patterns. Although repetitive patterns widely exist in real world images, these images are difficult to be matched due to local ambiguities even if the viewpoint changes are not very large. It is still an open and challenging problem. To solve the problem, this paper proposes to match pairs of interest points and then obtain point correspondences from the matched pairs of interest points based on the low distortion constraint, which is meant that the distortions of point groups should be small across images. By matching pairs of interest points, local ambiguities induced by repetitive patterns can be reduced to some extent since information in a much larger region is used. Meanwhile, owing to our newly defined compatibility measure between one correspondence and a set of point correspondences, the obtained point correspondences are very reliable. Experiments have demonstrated the effectiveness of our method and its superiority to the existing methods.     

宽基线主动视觉中感兴趣目标的对应技术

在主动视觉系统中,通常需要多个代理对同一场景中的感兴趣目标进行协同处理,以提高系统智能分析感兴趣目标的能力。其中,基于多视几何关系解决感兴趣目标的对应问题是协同处理的基础。一方面,主动视觉系统一般工作在宽基线条件下,这增加了对应问题描述的复杂性;另一方面,主动视觉系统以最佳视角观察目标,因此摄像头需做实时的姿态调整,由此导致的视间几何关系变化进一步加深了对应问题的解决难度。本文基于仿射不变的几何特征,建立宽基线条件下的多视几何关系,并针对频繁使用几何特征不能满足主动视觉系统实时要求的问题,提出一种快速更新多视几何关系的方法,并在多视几何约束下实现对应感兴趣目标的鲁棒标识。实验结果表明,该方法能解决宽基线主动视觉系统中感兴趣目标的复杂对应问题,并能达到实时要求。     

中国-中南半岛经济走廊重要节点城市发展状况 遥感分析

针对发展中国家城市经济与环境发展不均衡的问题,以中国-中南半岛经济走廊沿线的7个重要节点城市为例,采用面向对象的随机森林分类方法提取2000—2015年间的土地利用结果。将不透水层空间扩展指数、景观格局指数、区域NO2柱浓度和夜间灯光指数相结合,从城镇化、经济发展和环境变化3个角度对节点城市的发展状况进行研究。结果表明,面向对象随机森林分类方法能够有效提取不透水层,总体精度高于81%,Kappa系数高于0.82;在过去的15年间,各城市以“中心-边缘”或“沿河-内陆”的方向扩张,表现为“先聚集再扩散”“先扩散再聚集”“聚集”的3种扩张模式;城市中金边的夜间灯光指数增速最快,年变化率为1.71;金边、吉隆坡和新加坡为NO2柱浓度呈现负增长,而南宁、万象、河内和曼谷的NO2柱浓度呈现正增长。     

An Interactive Approach to Solving Correspondence Problems

Finding correspondences among objects in different images is a critical problem in computer vision. Even good correspondence procedures can fail, however, when faced with deformations, occlusions, and differences in lighting and zoom levels across images. We present a methodology for augmenting correspondence matching algorithms with a means for triaging the focus of attention and effort in assisting the automated matching. For guiding the mix of human and automated initiatives, we introduce a measure of the expected value of resolving correspondence uncertainties. We explore the value of the approach with experiments on benchmark data.