An active trinocular vision system of sensing indoor navigation environment for mobile robots

Intelligent autonomous mobile robots must be able to sense and recognize 3D indoor space where they live or work. However, robots are frequently situated in cluttered environments with various objects hard to be robustly perceived. Although the monocular and binocular vision sensors have been widely used for mobile robots, they suffer from image intensity variations, insufficient feature information and correspondence problems. In this paper, we propose a new 3D sensing system, in which the laser-structured-lighting method is basically utilized because of the robustness on the nature of the navigation environment and the easy extraction of feature information of interest. The proposed active trinocular vision system is composed of the flexible multi-stripe laser projector and two cameras arranged with a triangular shape. By modeling the laser projector as a virtual camera and using the trinocular epipolar constraints, the matching pairs of line features observed into two real camera images are established, and 3D information from one-shot image can be extracted on the patterned scene. For robust feature matching, here we propose a new correspondence matching technique based on line grouping and probabilistic voting. Finally, a series of experimental tests is performed to show the simplicity, efficiency, and accuracy of this proposed sensor system for 3D environment sensing and recognition.     

移动机器人运动目标跟踪系统设计

利用SONYEV-D31摄像机和自主研发的摄像机控制模块,构建了一套主动视觉子系统,并将该子系统应用于RIRA-Ⅱ型移动机器人上,实现了移动机器人运动目标自动跟踪功能。RIRA-Ⅱ移动机器人采用了由一组分布式行为模块和集中命令仲裁器组成的基于行为的分布式控制体系结构。各行为模块基于领域知识通过反应方式产生投票,由仲裁器产生动作指令,机器人完成相应的动作。在设置了障碍、窄通道以及模拟墙体的复杂环境下进行运动目标跟踪实验,实验表明运动目标跟踪系统运行可靠,具有较高的鲁棒性。     

Flexible calibration of structured-light systems projecting point patterns

Structured-light systems (SLSs) are widely used in active stereo vision to perform 3D modelling of a surface of interest. We propose a flexible method to calibrate SLSs projecting point patterns. The method is flexible in two respects. First, the calibration is independent of the number of points and their spatial distribution inside the pattern. Second, no positioning device is required since the projector geometry is determined in the camera coordinate system based on unknown positions of the calibration board. The projector optical center is estimated together with the 3D rays originating from the projector using a numerical optimization procedure. We study the 3D point reconstruction accuracy for two SLSs involving a laser based projector and a pico-projector, respectively, and for three point patterns. We finally illustrate the potential of our active vision system for a medical endoscopy application where a 3D cartography of the inspected organ (a large field of view surface also including image textures) can be reconstructed from a video acquisition using the laser based SLS.     

基于主动视觉和超声的机器人目标跟踪系统

运动目标跟踪技术是未知环境下移动机器人研究领域的一个重要研究方向。该文提出了一种基于主动视觉和超声信息的移动机器人运动目标跟踪设计方法,利用一台SONY EV-D31彩色摄像机、自主研制的摄像机控制模块、图像采集与处理单元等构建了主动视觉系统。移动机器人采用了基于行为的分布式控制体系结构,利用主动视觉锁定运动目标,通过超声系统感知外部环境信息,能在未知的、动态的、非结构化复杂环境中可靠地跟踪运动目标。实验表明机器人具有较高的鲁棒性,运动目标跟踪系统运行可靠。     

Real-time visuomotor update of an active binocular head

In order for a binocular head to perform optimal 3D tracking, it should be able to verge its cameras actively, while maintaining geometric calibration. In this work we introduce a calibration update procedure, which allows a robotic head to simultaneously fixate, track, and reconstruct a moving object in real-time. The update method is based on a mapping from motor-based to image-based estimates of the camera orientations, estimated in an offline stage. Following this, a fast online procedure is presented to update the calibration of an active binocular camera pair. The proposed approach is ideal for active vision applications because no image-processing is needed at runtime for the scope of calibrating the system or for maintaining the calibration parameters during camera vergence. We show that this homography-based technique allows an active binocular robot to fixate and track an object, whilst performing 3D reconstruction concurrently in real-time.     

Optimal Representative Blocks for the Efficient Tracking of a Moving Object

Optimal representative blocks are proposed for an efficient tracking of a moving object and it is verified experimentally by using a mobile robot with a pan‐tilt camera. The key idea comes from the fact that when the image size of a moving object is shrunk in an image frame according to the distance between the camera of mobile robot and the moving object, the tracking performance of a moving object can be improved by shrinking the size of representative blocks according to the object image size. Motion estimation using edge detection (ED) and block‐matching algorithm (BMA) are often used in the case of moving object tracking by vision sensors. However, these methods often miss the real‐time vision data since these schemes suffer from the heavy computational load. To overcome this problem and to improve the tracking performance, the optimal representative block that can reduce a lot of data to be computed is defined and optimized by changing the size of the representative block according to the size of object in the image frame. The proposed algorithm is verified experimentally by using a mobile robot with a two degree‐of‐freedom active camera. © 2004 Wiley Periodicals, Inc.     

基于面结构光的机械工件三维扫描系统设计

针对工业机器人对自动化装配过程生产效率的提高以及工件拾取对三维扫描技术的应用需求,设计了能够准确提取机械工件表面点云的视觉系统。扫描系统主要由计算机、投影仪和工业相机构成。基于光学测量和机器视觉的原理,主要研究设计了扫描系统工业相机和投影仪的标定策略、结构光栅编码解码的检测策略以及点云重构的几何策略。对于机械工件三维扫描重构的多余背景平面点云,研究设计了通过随机选取点云并反复迭代构造背景平面实现分割的有效方法。实验结果表明采用面结构光技术,由投影仪投影不同频率的结构光栅在机械工件上,工业相机同步采集被机械工件调制的结构光栅图像,对图像中的光栅条纹进行提取并计算,并利用三角检测算法提取机械工件表面点云的方案具有高准确性,能够有效重构机械工件表面点云。     

Towards autonomous robotic servicing: Using an integrated hand-arm-eye system for manipulating unknown objects

Executing complex robotic tasks including dexterous grasping and manipulation requires a combination of dexterous robots, intelligent sensors and adequate object information processing. In this paper, vision has been integrated into a highly redundant robotic system consisting of a tiltable camera and a three-fingered dexterous gripper both mounted on a puma-type robot arm. In order to condense the image data of the robot working space acquired from the mobile camera, contour image processing is used for offline grasp and motion planning as well as for online supervision of manipulation tasks. The performance of the desired robot and object motions is controlled by a visual feedback system coordinating motions of hand, arm and eye according to the specific requirements of the respective situation. Experiences and results based on several experiments in the field of service robotics show the possibilities and limits of integrating vision and tactile sensors into a dexterous hand-arm-eye system being able to assist humans in industrial or servicing environments.     

结构光测量系统的投影仪标定方法研究

传统单目视觉结构光测量系统通过解相位间接计算被测物的高度信息,系统约束性过强、不易操作、且标定精度较低.将双目立体视觉原理引入单目结构光视觉测量系统,根据投影仪图像坐标和摄像机图像坐标的对应关系,将投影仪当作一个逆向的相机,建立了投影仪模型,使用成熟的相机标定算法对投影仪进行标定.再配合四步相移法和基于多频外差原理的时域相位展开法,实现条纹图像的快速精确解相位.相对于传统的单目视觉结构光测量系统,本方法具有单目视觉系统操作简单、鲁棒性强的特点,同时也可以达到双目视觉测量的精度.实验结果表明,这种方法的投影仪标定精度达到了实际应用的要求.     

三维立体视觉机械臂智能抓取分类系统的开发

采用工业相机、工业投影机、普通摄像头、计算机和机械臂开发了一套具有三维立体视觉的机械臂智能抓取分类系统。该系统采用自编软件实现了对工业相机、工业投影机的自动控制和同步，通过前期研究提出的双波长条纹投影三维形貌测量法获取了物体的高度信息，结合opencv技术和普通摄像头获取的物体二维平行面信息，实现了物体的自动识别和分类；利用串口通信协议，将上述处理后的数据传送至机械臂，系统进行几何姿态解算，实现了智能抓取，并能根据抓手上压力反馈自动调节抓手张合程度，实现自适应抓取。经实验证明该系统能通过自带的快速三维形貌获取装置实现准确、快速的抓取工作范围内的任意形状的物体并实现智能分类。     

激光—相机系统语义栅格建图和路径规划

目的 SLAM（simultaneous localization and mapping）是移动机器人在未知环境进行探索、感知和导航的关键技术。激光SLAM测量精确，便于机器人导航和路径规划，但缺乏语义信息。而视觉SLAM的图像能提供丰富的语义信息，特征区分度更高，但其构建的地图不能直接用于路径规划和导航。为了实现移动机器人构建语义地图并在地图上进行路径规划，本文提出一种语义栅格建图方法。方法 建立可同步获取激光和语义数据的激光-相机系统，将采集的激光分割数据与目标检测算法获得的物体包围盒进行匹配，得到各物体对应的语义激光分割数据。将连续多帧语义激光分割数据同步融入占据栅格地图。对具有不同语义类别的栅格进行聚类，得到标注物体类别和轮廓的语义栅格地图。此外，针对语义栅格地图发布导航任务，利用路径搜索算法进行路径规划，并对其进行改进。结果 在实验室走廊和办公室分别进行了语义栅格建图的实验，并与原始栅格地图进行了比较。在语义栅格地图的基础上进行了路径规划，并采用了语义赋权算法对易移动物体的路径进行对比。结论 多种环境下的实验表明本文方法能获得与真实环境一致性较高、标注环境中物体类别和轮廓的语义栅格地图，且实验硬件结构简单、成本低、性能良好，适用于智能化机器人的导航和路径规划。     

Circular camera array system for 3‐D displays based on the reconstruction of parallax rays

Abstract— A circular camera system employing an image‐based rendering technique that captures light‐ray data needed for reconstructing three‐dimensional (3‐D) images by using reconstruction of parallax rays from multiple images captured from multiple viewpoints around a real object in order to display a 3‐D image of a real object that can be observed from multiple surrounding viewing points on a 3‐D display is proposed. An interpolation algorithm that is effective in reducing the number of component cameras in the system is also proposed. The interpolation and experimental results which were performed on our previously proposed 3‐D display system based on the reconstruction of parallax rays will be described. When the radius of the proposed circular camera array was 1100 mm, the central angle of the camera array was 40°, and the radius of a real 3‐D object was between 60 and 100 mm, the proposed camera system, consisting of 14 cameras, could obtain sufficient 3‐D light‐ray data to reconstruct 3‐D images on the 3‐D display.     

3DComplete: Efficient completeness inspection using a 2.5D color scanner

In this paper, we present a low-cost and highly configurable quality inspection system capable of capturing 2.5D color data, created using off-the-shelf machine vision components, open-source software libraries, and a combination of standard and novel algorithms for 2.5D data processing. The system uses laser triangulation to capture 3D depth, in parallel with a color camera and a line light projector to capture color texture, which are then combined into a color 2.5D model in real-time.     

3D model based tracking for omnidirectional vision: A new spherical approach

The current work addresses the problem of 3D model tracking in the context of monocular and stereo omnidirectional vision in order to estimate the camera pose. To this end, we track 3D objects modeled by line segments because the straight line feature is often used to model the environment. Indeed, we are interested in mobile robot navigation using omnidirectional vision in structured environments. In the case of omnidirectional vision, 3D straight lines are projected as conics in omnidirectional images. Under certain conditions, these conics may have singularities.In this paper, we present two contributions. We, first, propose a new spherical formulation of the pose estimation withdrawing singularities, using an object model composed of lines. The theoretical formulation and the validation on synthetic images thus show that the new formulation clearly outperforms the former image plane one. The second contribution is the extension of the spherical representation to the stereovision case. We consider in the paper a sensor which combines a camera and four mirrors. Results in various situations show the robustness to illumination changes and local mistracking. As a final result, the proposed new stereo spherical formulation allows us to localize online a robot indoor and outdoor whereas the classical formulation fails.     

基于人工神经网络的机器人视觉系统中三维信息的快速并行求解

文章论述的是在一种主动式激光差频扫描机器人三维视觉系统中,用人工神经网络对其系统空间扫描点三维坐标值进行大规模并行计算以便达到实时机器视觉,并论述了用模拟VLSI技术实现训练后神经网络芯片的可行性及意义。     

基于线结构光扫描的结冰冰形三维测量

现有三维扫描仪进行结冰冰形测量时需喷涂显影剂,无法满足结冰生长过程3D冰形在线测量需求。为此,基于线结构光扫描,提出了结冰冰形三维测量方法,搭建了基于传送带的线结构光扫描冰形测量装置,对线结构光视觉测量系统进行标定,获取了摄像机和激光平面方程参数,采用改进的梯度重心法实现了低对比度冰体图像激光光带亚像素中心位置提取,结合标定结果计算了冰体轮廓线三维坐标,再根据传送带运动距离,对冰体轮廓线逐行拼接,得到整个冰体三维测量点云。实验结果表明,对半径已知的圆柱形冰块进行轮廓线测量,相对误差为0.157 mm。     

远心结构光三维系统研究

结构光三维重建技术已广泛应用于工业检测领域,随着工业检测需求的不断提高,工业检测 技术的需求也愈发向微小化和高精度化方向发展。其中,远心镜头具备透视误差小、镜头畸变小、成像失真少等优秀特性,获得越来越多的关注。该文投影仪采用传统镜头,将传统结构光系统中的相机镜头替换为远心镜头,通过对传统两步标定法的改进,对相机的仿射模型进行标定,随后用已标定的相机对投影仪进行参数标定。实验结果显示,所开发的远心结构光系统具备能够实现小视野范围的高精度、高分辨率三维重建,并具有加大的测量景深,可用于半导体器件及微型零部件等目标的高精度 三维检测。     

Fusing Visual and Inertial Sensing to Recover Robot Ego‐motion

A method for estimating mobile robot ego‐motion is presented, which relies on tracking contours in real‐time images acquired with a calibrated monocular video system. After fitting an active contour to an object in the image, 3D motion is derived from the affine deformations suffered by the contour in an image sequence. More than one object can be tracked at the same time, yielding some different pose estimations. Then, improvements in pose determination are achieved by fusing all these different estimations. Inertial information is used to obtain better estimates, as it introduces in the tracking algorithm a measure of the real velocity. Inertial information is also used to eliminate some ambiguities arising from the use of a monocular image sequence. As the algorithms developed are intended to be used in real‐time control systems, considerations on computation costs are taken into account. © 2004 Wiley Periodicals, Inc.     

Robust active stereo vision using Kullback-Leibler divergence

Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.