Fusion of fixation and odometry for vehicle navigation

This paper deals with the problem of determining the position and orientation of an autonomous guided vehicle (AGV) by fusing odometry with the information provided by a vision system. The main idea is to exploit the ability of pointing a camera in different directions, to fixate on a point of the environment while the AGV is moving. By fixating on a landmark, one can improve the navigation accuracy even if the scene coordinates of the landmark are unknown. This is a major improvement over previous methods which assume that the coordinates of the landmark are known, since any point of the observed scene can be selected as a landmark, and not just pre-measured points. This work argues that fixation is basically a simpler procedure than previously mentioned methods. The simplification comes from the fact that only one point needs to be tracked as opposed to multiple points in other methods. This disposes of the need to be able to identify which of the landmarks is currently being tracked, through a matching algorithm or by other means. We support our findings with both experimental and simulation results     

Using parallel line information for vision-based landmark location estimation and an application to automatic helicopter landing

An approach to landmark location estimation by computer vision techniques is proposed. The objective is to derive the position and the orientation of the landmark with respect to the vehicle by a single image. Such information is necessary for automatic vehicle navigation. This approach requires lower hardware cost and simple computation. The vanishing points of the parallel lines on the landmark are used to detect the landmark orientation. The detected vanishing points are used to derive the relative orientation between the landmark and the camera, which is then utilized to compute the landmark orientation with respect to the vehicle. The size of the landmark is used to determine the landmark position. Sets of collinear points are extracted from the landmark and their inter-point distances are computed. The positions of the collinear point sets are evaluated and used to determine the landmark position. Landing site location estimation by using the identification marking H on the helicopter landing site for automatic helicopter landing is presented as an application of the proposed approach. Simulations and experiments have been conducted to prove the feasibility of the proposed approach.     

Using parallel line information for vision-based landmark location estimation and an application to automatic helicopter landing

An approach to landmark location estimation by computer vision techniques is proposed. The objective is to derive the position and the orientation of the landmark with respect to the vehicle by a single image. Such information is necessary for automatic vehicle navigation. This approach requires lower hardware cost and simple computation. The vanishing points of the parallel lines on the landmark are used to detect the landmark orientation. The detected vanishing points are used to derive the relative orientation between the landmark and the camera, which is then utilized to compute the landmark orientation with respect to the vehicle. The size of the landmark is used to determine the landmark position. Sets of collinear points are extracted from the landmark and their inter-point distances are computed. The positions of the collinear point sets are evaluated and used to determine the landmark position. Landing site location estimation by using the identification marking H on the helicopter landing site for automatic helicopter landing is presented as an application of the proposed approach. Simulations and experiments have been conducted to prove the feasibility of the proposed approach.     

Location estimation for indoor autonomous vehicle navigation by omni-directional vision using circular landmarks on ceilings

A novel approach to location estimation by omni-directional vision for autonomous vehicle navigation in indoor environments using circular landmark information is proposed. A circular-shaped landmark is attached on a ceiling and an omni-directional camera is equipped on a vehicle to take upward-looking omni-directional images of the landmark. This way of image taking reduces possible landmark shape occlusion and image noise creation, which come from the existence of nearby objects or humans surrounding the vehicle. It is shown that the perspective shape of the circular landmark in the omni-directional image may be approximated by an ellipse by analytic formulas with good shape-fitting effect and fast computation speed. The parameters of the ellipse are then used for estimating the location of the vehicle with good precision for navigation guidance. Both simulated and real images were tested and good experimental results confirm the feasibility of the proposed approach.     

基于视觉的三维重建关键技术研究综述

三维重建在视觉方面具有很高的研究价值, 在机器人视觉导航、智能车环境感知系统以及虚拟现实中被广泛应用.本文对近年来国内外基于视觉的三维重建方法的研究工作进行了总结和分析, 主要介绍了基于主动视觉下的激光扫描法、结构光法、阴影法以及TOF (Time of flight)技术、雷达技术、Kinect技术和被动视觉下的单目视觉、双目视觉、多目视觉以及其他被动视觉法的三维重建技术, 并比较和分析这些方法的优点和不足.最后对三维重建的未来发展作了几点展望.     

基于单目视觉的水下机器人悬停定位技术与实现

以7000m载人潜水器的水下悬停为应用背景,以单目CCD摄像机为传感器,提出了基于模型的单目视觉定位和基于特征的视觉伺服两种水下机器人视觉悬停技术方案,分别适用于已知模型的观察目标和未知模型的观察目标两种情况．以自行研制的水下机器人控制系统实验平台为实验载体,利用前视摄像机和水下人工目标,建立了水下演示实验验证系统,并在室内实验水池中,分别实现了两种悬停方法的水下演示实验．演示实验表明：在两种视觉悬停方法的闭环控制下,水下机器人能够抵抗恒定水流干扰和人工位置扰动,很好地实现水下悬停作业．     

In‐water visual ship hull inspection using a hover‐capable underwater vehicle with stereo vision

Underwater visual inspection is an important task for checking the structural integrity and biofouling of the ship hull surface to improve the operational safety and efficiency of ships and floating vessels. This paper describes the development of an autonomous in‐water visual inspection system and its application to visual hull inspection of a full‐scale ship. The developed system includes a hardware vehicle platform and software algorithms for autonomous operation of the vehicle. The algorithms for vehicle autonomy consist of the guidance, navigation, and control algorithms for real‐time and onboard operation of the vehicle around the hull surface. The environmental perception of the developed system is mainly based on optical camera images, and various computer vision and optimization algorithms are used for vision‐based navigation and visual mapping. In particular, a stereo camera is installed on the underwater vehicle to estimate instantaneous surface normal vectors, which enables high‐precision navigation and robust visual mapping, not only on flat areas but also over moderately curved hull surface areas. The development process of the vehicle platform and the implemented algorithms are described. The results of the field experiment with a full‐scale ship in a real sea environment are presented to demonstrate the feasibility and practical performance of the developed system.     

基于不确定性知识的实时道路场景理解

由于室外机器人的工作环境非常复杂，因此机器人的视觉导航必须具有足够的智能和鲁棒性，为此，提出了一种基于不确定性知识的实时道路理解算法，该算法通过不确定性知识推理来融合多种信息和知识，以满足在复杂道路环境下的鲁棒性要求，它即使在有强烈阴影、水迹等干扰下也能给出比较好的结果；通过图象边缘信息的提取可以得到精确的道路边界，以满足视觉导航的精确性要求；同时在算法设计时，兼顾了实时性要求；使得算法得以实时实现，该算法已在实际的机器人上进行了测试，并得到了很好的结果。     

Improved visual SLAM: a novel approach to mapping and localization using visual landmarks in consecutive frames

Pathfinding is becoming more and more common in autonomous vehicle navigation, robot localization, and other computer vision applications. In this paper, a novel approach to mapping and localization is presented that extracts visual landmarks from a robot dataset acquired by a Kinect sensor. The visual landmarks are detected and recognized using the improved scale-invariant feature transform (I-SIFT) method. The methodology is based on detecting stable and invariant landmarks in consecutive (red-green-blue depth) RGB-D frames of the robot dataset. These landmarks are then used to determine the robot path, and a map is constructed by using the visual landmarks. A number of experiments were performed on various datasets in an indoor environment. The proposed method performs efficient landmark detection in various environments, which includes changes in rotation and illumination. The experimental results show that the proposed method can solve the simultaneous localization and mapping (SLAM) problem using stable visual landmarks, but with less computation time.     

Efficient View-Based SLAM Using Visual Loop Closures

This paper presents a simultaneous localization and mapping algorithm suitable for large-scale visual navigation. The estimation process is based on the viewpoint augmented navigation (VAN) framework using an extended information filter. Cholesky factorization modifications are used to maintain a factor of the VAN information matrix, enabling efficient recovery of state estimates and covariances. The algorithm is demonstrated using data acquired by an autonomous underwater vehicle performing a visual survey of sponge beds. Loop-closure observations produced by a stereo vision system are used to correct the estimated vehicle trajectory produced by dead reckoning sensors.      

An autonomous tracked vehicle with omnidirectional sensing

Operation of an autonomous vehicle along a marked path, in an obstacle‐laden environment, requires path detection, relative position detection and control, and obstacle detection and avoidance. The design solution of the team from the U.S. Military Academy is a tracked vehicle operating open‐loop in response to position information from an omnidirectional mirror, and to obstacle‐detection input from the mirror and from a scanning laser. The use of a tracked rather than a wheeled vehicle is the team's open‐loop solution to the problem of wheeled‐vehicle slippage on wet and sandy surfaces. The vehicleresponds to sensor information from (1) a digital camera‐mounted parabolic omnidirectional mirror for visual inputs and (2) a scanning laser for detecting obstacles in relief. Raw sensor data is converted synchronously into a global virtual context, which places the vehicle's center at the origin of a 2‐D Cartesian coordinate system. A four‐phase process is used to convert the camera's inputs into the data structures needed to reason about the vehicle's position relative to the course. Development of the path plan proceeds incrementally, using a space‐sweeping algorithm to identify safe paths along waypoints within the course boundaries. An attempt is made to minimize translation errors by favoring paths which exhibit fewer sharp turns. Integration of Intel's OpenCV computer vision library and the Independent JPEG Group's JPEG library allow for very good encapsulation of the low‐level functions needed to do most of the image processing. Ada95 is the language of choice for the majority of the team‐developed software, except where needed to interface to motors and sensors. Use of an object‐oriented high‐level language has been invaluable in leveraging the efforts of previous years' development activities, and for maximizing the ability to log or otherwise respond to anomalous behavior. © 2004 Wiley Periodicals, Inc.     

Tightly-coupled stereo vision-aided inertial navigation using feature-based motion sensors

A tightly-coupled stereo vision-aided inertial navigation system is proposed in this work, as a synergistic incorporation of vision with other sensors. In order to avoid loss of information possibly resulting by visual preprocessing, a set of feature-based motion sensors and an inertial measurement unit are directly fused together to estimate the vehicle state. Two alternative feature-based observation models are considered within the proposed fusion architecture. The first model uses the trifocal tensor to propagate feature points by homography, so as to express geometric constraints among three consecutive scenes. The second one is derived by using a rigid body motion model applied to three-dimensional (3D) reconstructed feature points. A kinematic model accounts for the vehicle motion, and a Sigma-Point Kalman filter is used to achieve a robust state estimation in the presence of non-linearities. The proposed formulation is derived for a general platform-independent 3D problem, and it is tested and demonstrated with a real dynamic indoor data-set alongside of a simulation experiment. Results show improved estimates than in the case of a classical visual odometry approach and of a loosely-coupled stereo vision-aided inertial navigation system, even in GPS (Global Positioning System)-denied conditions and when magnetometer measurements are not reliable.     

基于卫星定位和惯导融合的模拟测绘系统设计

设计了一种基于北斗/GPS卫星定位和惯导融合的模拟测绘系统,用以对现有测绘方式进行补充。系统以履带车为信息采集移动平台,以STM32F103单片机为信息采集控制中心,用树莓派对测绘现场进行实时视频采集,以九轴MPU融合北斗/GPS双精度定位算法实现对采集点的精确定位,通过树莓派自带的WiFi和4G路由器实现数据上传服务器,通过地面站对履带车进行实时远程手动控制和自主路径规划。测试表明系统通过北斗/GPS卫星定位和惯导融合能满足一定的测绘需要,能对所测地面地形建立数字高程模型(digital elevation model)。     

Biologically inspired collective construction with visual landmarks

We describe our research in using environmental visual landmarks as the basis for completing simple robot construction tasks.Inspired by honeybee visual navigation behavior,a visual template mechanism is proposed in which a natural landmark serves as a visual reference or template for distance determination as well as for navigation during collective construction.To validate our proposed mechanism,a wall construction problem is investigated and a minimalist solution is given.Experimental results show that,using the mechanism of a visual template,a collective robotic system can successfully build the desired structure in a decentralized fashion using only local sensing and no direct communication.In addition,a particular variable,which defines tolerance for alignment of the structure,is found to impact the system performance.By decreasing the value of the variable,system performance is improved at the expense of a longer construction time.The visual template mechanism is appealing in that it can use a reference point or salient object in a natural environment that is new or unexplored and it could be adapted to facilitate more complicated building tasks.     

Robust estimation of planar surfaces using spatio-temporal RANSAC for applications in autonomous vehicle navigation

A fundamental problem in autonomous vehicle navigation is the identification of obstacle free space in cluttered and unstructured environments. Features such as walls, people, furniture, doors and stairs, etc are potential hazards. The approach taken in this paper is motivated by the recent development on infra-red time-of-flight cameras that provide video frame rate low resolution depth maps. We propose to exploit the temporal information content provided by the high refresh rate of such cameras to overcome the limitations due to low spatial resolution and high depth uncertainty and aim to provide robust and accurate estimates of planar surfaces in the environment. These surfaces’ estimates are then used to provide statistical tests to identify obstacles and dangers in the environment. Classical 3D spatial RANSAC is extended to 4D spatio-temporal RANSAC by developing spatio-temporal models of planar surfaces that incorporate a linear motion model as well as linear environment features. A 4D-vector product is used for hypotheses generation from data that is randomly sampled across both spatial and temporal variations. The algorithm is fully posed in the spatio-temporal representation and there is no need to correlate points or hypothesis between temporal images. The proposed algorithm is computationally fast and robust for estimation of planar surfaces in general and the ground plane in particular. There are potential applications in mobile robotics, autonomous vehicular navigation, and automotive safety systems. The claims of the paper are supported by experimental results obtained from real video data for a time-of-flight range sensor mounted on an automobile navigating in an undercover parking lot.     

越野环境下自主车辆导航地图自动创建方法研究*

针对越野环境下的地图创建问题,提出了一种自动创建自主车导航地图的方法。首先将车载摄像机获得的图像投影到车体坐标系,然后结合车辆行驶轨迹信息采用基于标记的分水岭算法判定可通行区域,最后融合局部俯视图信息生成全局一致地图,并在实时导航需求下对地图进行优化得到最终的导航地图。自主车实车实验结果表明,该方法生成的地图满足自主车实时导航需求,提高了路径规划效率。     

基于核心区域信息和经验知识的道路理解算法

针对户外移动机器人视觉导航, 提出了一种基于核心区域和经验知识的道路理解算法. 本算法引入核心区域信息融合经验知识加强道路理解的准确性, 引入时间影响因子增强道路理解系统的鲁棒性. 根据模糊原理对于不同的区域赋予相应的道路颜色隶属度, 为安全度要求不同的机器人导航提供更为精确的信息. 算法设计中通过优化算法, 兼顾了导航的实时性要求.     

Visual Navigation in Natural Environments: From Range and Color Data to a Landmark-Based Model

This paper concerns the exploration of a natural environment by a mobile robot equipped with both a video color camera and a stereo-vision system. We focus on the interest of such a multi-sensory system to deal with the navigation of a robot in an a priori unknown environment, including (1) the incremental construction of a landmark-based model, and the use of these landmarks for (2) the 3-D localization of the mobile robot and for (3) a sensor-based navigation mode.For robot localization, a slow process and a fast one are simultaneously executed during the robot motions. In the modeling process (currently 0.1 Hz), the global landmark-based model is incrementally built and the robot situation can be estimated from discriminant landmarks selected amongst the detected objects in the range data. In the tracking process (currently 4 Hz), selected landmarks are tracked in the visual data; the tracking results are used to simplify the matching between landmarks in the modeling process.Finally, a sensor-based visual navigation mode, based on the same landmark selection and tracking, is also presented; in order to navigate during a long robot motion, different landmarks (targets) can be selected as a sequence of sub-goals that the robot must successively reach.     

Visual Recognition of Workspace Landmarks for Topological Navigation

In this work, robot navigation is approached using visual landmarks. Landmarks are not preselected or otherwise defined a priori; they are extracted automatically during a learning phase. To facilitate this, a saliency map is constructed on the basis of which potential landmarks are highlighted. This is used in conjunction with a model-driven segregation of the workspace to further delineate search areas for landmarks in the environment. For the sake of robustness, no semantic information is attached to the landmarks; they are stored as raw patterns, along with information readily available from the workspace segregation. This subsequently facilitates their accurate recognition during a navigation session, when similar steps are employed to locate landmarks, as in the learning phase. The stored information is used to transform a previously learned landmark pattern, according to the current position of the observer, thus achieving accurate landmark recognition. Results obtained using this approach demonstrate its validity and applicability in indoor workspaces.     

自治/遥控水下机器人北极冰下导航

针对北极高纬度和科学考察中长期冰站对海冰在一定范围内连续重复观测的需求,设计了面向冰下环境的自治/遥控水下机器人(北极ARV)导航定位系统.提出基于海冰运动修正的水下机器人自主导航方法,通过定时引入海冰运动信息,实时准确获取水下机器人相对于海冰的位置信息,这不仅提高了观测数据的实际应用价值,还提高了冰下作业的安全性.仿真试验和北极冰下应用证明这一导航系统具有精度高、稳定性好等优点.