Monocular Vision for Mobile Robot Localization and Autonomous Navigation

This paper presents a new real-time localization system for a mobile robot. We show that autonomous navigation is possible in outdoor situation with the use of a single camera and natural landmarks. To do that, we use a three step approach. In a learning step, the robot is manually guided on a path and a video sequence is recorded with a front looking camera. Then a structure from motion algorithm is used to build a 3D map from this learning sequence. Finally in the navigation step, the robot uses this map to compute its localization in real-time and it follows the learning path or a slightly different path if desired. The vision algorithms used for map building and localization are first detailed. Then a large part of the paper is dedicated to the experimental evaluation of the accuracy and robustness of our algorithms based on experimental data collected during two years in various environments.     

基于平行线的室内视觉导航

根据一组空间直线上的无穷远线素集在视平面上所形成的消失点对直线的平移具有稳定性的特点,提出了基于平行线的摄像机参数标定和自主移动平台室内视觉导航算法．在对摄像机进行标定时,将摄像机模型简化成关于移动平台航向角和X方向距离的线性模型,并利用走廊左右踢脚线在视平面上的投影直线的斜率、消失点坐标来标定摄像机的内、外参数;在视觉导航时,视走廊左右踢脚线为一组平行线,由其在视平面上的投影直线的斜率、消失点坐标,控制自主移动平台行驶的X方向距离和航向角,实现平台的室内视觉导航．本文采用YIQ彩色模型分割楼道图像,由Hough变换与最小二乘法相结合的方法提取楼道踢脚线．     

New Visual Invariants for Terrain Navigation Without 3D Reconstruction

For autonomous vehicles to achieve terrain navigation, obstacles must be discriminated from terrain before any path planning and obstacle avoidance activity is undertaken. In this paper, a novel approach to obstacle detection has been developed. The method finds obstacles in the 2D image space, as opposed to 3D reconstructed space, using optical flow. Our method assumes that both nonobstacle terrain regions, as well as regions with obstacles, will be visible in the imagery. Therefore, our goal is to discriminate between terrain regions with obstacles and terrain regions without obstacles. Our method uses new visual linear invariants based on optical flow. Employing the linear invariance property, obstacles can be directly detected by using reference flow lines obtained from measured optical flow. The main features of this approach are: (1) 2D visual information (i.e., optical flow) is directly used to detect obstacles; no range, 3D motion, or 3D scene geometry is recovered; (2) knowledge about the camera-to-ground coordinate transformation is not required; (3) knowledge about vehicle (or camera) motion is not required; (4) the method is valid for the vehicle (or camera) undergoing general six-degree-of-freedom motion; (5) the error sources involved are reduced to a minimum, because the only information required is one component of optical flow. Numerous experiments using both synthetic and real image data are presented. Our methods are demonstrated in both ground and air vehicle scenarios.     

Avoiding Non-Manhattan Obstacles Based on Projection of Spatial Corners in Indoor Environment

Monocular vision-based navigation is a considerable ability for a home mobile robot. However, due to diverse disturbances, helping robots avoid obstacles, especially non-Manhattan obstacles, remains a big challenge. In indoor environments, there are many spatial right-corners that are projected into two dimensional projections with special geometric configurations. These projections, which consist of three lines, might enable us to estimate their position and orientation in 3D scenes. In this paper, we present a method for home robots to avoid non-Manhattan obstacles in indoor environments from a monocular camera. The approach first detects non-Manhattan obstacles. Through analyzing geometric features and constraints, it is possible to estimate posture differences between orientation of the robot and non-Manhattan obstacles. Finally according to the convergence of posture differences, the robot can adjust its orientation to keep pace with the pose of detected non-Manhattan obstacles, making it possible avoid these obstacles by itself. Based on geometric inferences, the proposed approach requires no prior training or any knowledge of the camera’s internal parameters, making it practical for robots navigation. Furthermore, the method is robust to errors in calibration and image noise. We compared the errors from corners of estimated non-Manhattan obstacles against the ground truth. Furthermore, we evaluate the validity of convergence of differences between the robot orientation and the posture of non-Manhattan obstacles. The experimental results showed that our method is capable of avoiding non-Manhattan obstacles, meeting the requirements for indoor robot navigation.      

Improved Telemanipulator Navigation During Display-Control Misalignments Using Augmented Reality Cues

An augmented reality (AR) cueing method designed to improve teleoperator performance under conditions of display-control misalignment is investigated. The teleoperation task was designed to mimic the operation of space robot arms, which are manipulated using hand controllers (HCs) to orient and translate the body-fixed coordinates at the end effector (EE). Cameras provide visual feedback. However, the pose of the EE seen through the camera hinders operator performance due to misalignments between the displayed EE axes and the HC axes. In this paper, the coordinate system of the EE is graphically overlaid in three dimensions on the video views with uniquely colored axes using AR. The same color scheme is used to label the corresponding axes on the HCs. Operators use these color cues to map each axis on the HCs to the corresponding colored axis of the augmented coordinates at the EE to obtain EE movement in the desired direction. Between-groups and within-participant experiments comparing EE trajectory distance, deviation from path, navigation errors, and HC axis usage were used to determine the effectiveness of the augmented coordinates over conventional teleoperation without augmented coordinates. Significant reductions in EE trajectory distance, deviation from path, navigation errors, and single-axis HC usage were observed when participants manipulated the remote robot with augmented coordinates. The results demonstrate that the use of simple AR cues in remote robot arm teleoperation is beneficial to the operator.     

Online world modeling and path planning for an unmanned helicopter

Mission scenarios beyond line of sight or with limited ground control station access require capabilities for autonomous safe navigation and necessitate a continuous extension of existing and potentially outdated information about obstacles. The presented approach is a novel synthesis of techniques for 3D environment perception and global path planning. A locally bounded sensor fusion approach is used to extract sparse obstacles for global incremental path planning in an anytime fashion. During the flight, a stereo camera checks the field of view along the flight path ahead by analyzing depth images. A 3D occupancy grid is built incrementally. To reduce the high data rate and storage demands of grid-type maps, an approximated polygonal world model is created. For a compacted representation, it uses prisms and ground planes. This enables the system to constantly renew and update its knowledge about obstacles. An incremental heuristic path planner uses both a-priori information as well as incremental obstacle updates to assure a collision-free path at any time. Mapping results from flight tests show the functionality of onboard world modeling from real sensor data. Path planning feasibility is demonstrated within a simulation environment considering world model changes inside the vehicle’s field of view.     

Mobile Robots Navigation Modeling in Known 2D Environment Based on Petri Nets

The paper deals with supervised robot navigation in known environments. The navigation task is divided into two parts, where one part of the navigation is done by the supervisor system i.e. the system sets the vector marks on the salient edges of the virtual environment map and guides the robot to reach these marks. Mobile robots have to perform a specific task according to the given paths and solve the local obstacles avoidance individually. The salient point’s detection, vector mark estimation and optimal path calculation are done on the supervisor computer using colored Petri nets. The proposed approach was extended to simulate a flexible manufacturing system consisting of swarm of 17 robots, 17 - warehouses and 17 - manufacturing places. Our experimental investigation showed that simulated mobile robots with proposed supervision system were efficiently moving on the planned path.     

Mining preferred navigation patterns by consolidating both selection and time preferences

Preferred navigation patterns (PNP) are those contiguous sequential patterns whose elements are preferred by users to be selected as the next steps between several different selections and are preferred by users to spend much time on. Such navigation path and time preferred patterns are more actionable than any other finds only considering either path or time in various web applications, such as web user navigation, targeted online advertising and recommendation. However, due to the conceptual confusion and limitation on navigation preference in the existing work, the corresponding algorithms cannot discover actionable preferred navigation patterns. In this paper, we study the problem of preferred navigation pattern mining by involving both navigation path and time length. Firstly, we carefully define the concepts of time preference and selection preference for time-related path sequences, which can well reflect user interests from the relative path selection and time consumption respectively. Secondly, we propose an efficient PNP-forest algorithm for identifying PNPs, by first introducing PNP-forest data structure, and then presenting PNP-forest growth and maintenance mechanism, associated with optimization strategies. Then we introduce a more efficient mining algorithm called PrefixSpan_Forest, which integrates the advantages of PrefixSpan and PNP-forest. The performance of these two algorithms are also evaluated and the results show that the algorithms can discover PNPs effectively.     

Algorithms for path planning, navigation and guidance of an AGV

Algorithms for path navigation and generation of guidance setpoints for an AGV are developed. Navigation is based on a simple flat world model of connected nodes using a suboptimal path solution for execution speed. The guidance algorithms use vision data from a stereo pair of linear image array cameras, which described an obstacle location and height where intersected by the vision system plane of view. The development of a map of the area on the AGV path allows the detection of obstacles, which may be passed subject to the aisle markings. The complete vision-guidance system can be implemented using inexpensive commercially available 16 bit microprocessors.     

Machine vision applied to vehicle guidance

Research on the semiautonomous operation of mobile robots in typical pathways is described. The image of the pathway will consist of two nearly vertical lines bounding a region with little texture (the pathway) after correction for perspective. In order to identify pathway boundaries, regions in the image space are examined using an edge detection algorithm, edges between regions are determined by the algorithm, and those corresponding to straight or nearly straight lines with large slope (path boundaries) are identified by means of the Hough transform. Once the path boundaries are identified, the horizontal distance from camera to road edge is determined. Next, a method to detect cubics in the roadway (i.e., obstacles) is presented. The region of interest in the roadway (from the camera to some predetermined distance in front of it) is known from the path boundary algorithm. The interior of this region is examined for edges. If edges are detected, it means that obstacles or shadows are present. A method to separate obstacles from shadows using stere vision is then presented.     

A New Mobile Robot Toolbox for Matlab

In this study, a wheeled mobile robot navigation toolbox for Matlab is presented. The toolbox includes algorithms for 3D map design, static and dynamic path planning, point stabilization, localization, gap detection and collision avoidance. One can use the toolbox as a test platform for developing custom mobile robot navigation algorithms. The toolbox allows users to insert/remove obstacles to/from the robot’s workspace, upload/save a customized map and configure simulation parameters such as robot size, virtual sensor position, Kalman filter parameters for localization, speed controller and collision avoidance settings. It is possible to simulate data from a virtual laser imaging detection and ranging (LIDAR) sensor providing a map of the mobile robot’s immediate surroundings. Differential drive forward kinematic equations and extended Kalman filter (EKF) based localization scheme is used to determine where the robot will be located at each simulation step. The LIDAR data and the navigation process are visualized on the developed virtual reality interface. During the navigation of the robot, gap detection, dynamic path planning, collision avoidance and point stabilization procedures are implemented. Simulation results prove the efficacy of the algorithms implemented in the toolbox.     

Autonomous navigation and obstacle avoidance using navigation laws with time-varying deviation functions

In this paper we introduce a new family of navigation functions for robot navigation and obstacle avoidance. The method can be used for both path finding and real-time path planning. Each navigation function is composed of three parts: a proportionality term, a deviation function and a deviation constant. Deviation functions are time-varying functions satisfying certain conditions. These functions and parameters are updated in real-time to avoid collision with obstacles. Our strategy uses polar kinematics equations to model the navigation problem in terms of the range and direction between the robot and the goal. The obstacles are mapped to polar planes, and represented by the range and the direction from the robot or the final goal in polar coordinates. This representation gives a certain weight to the obstacles based on their relative position from the robot and facilitates the design of the navigation law. There exists an infinite number of navigation functions obtained by changing the proportionality constant, the deviation constant or the deviation function. This offers an infinite number of possibilities for the robot's path. Our navigation strategy is illustrated using an extensive simulation where different navigation parameters are used.     

Path planning with obstacle avoidance by G1 PH quintic splines

We propose a two-step approach for the construction of planar smooth collision-free navigation paths. Obstacle avoidance techniques that rely on classical data structures are initially considered for the identification of piecewise linear paths having no intersection with the obstacles of a given scenario. Variations of the shortest piecewise linear path with angle-based criteria are proposed and discussed. In the second part of the scheme we rely on spline interpolation algorithms with tension parameters to provide a smooth planar control strategy. In particular, we consider the class of curves with Pythagorean structures, because they provide an exact computation of fundamental geometric quantities. A selection of test cases demonstrates the quality of the new motion planning scheme.     

Omnidirectional vision scan matching for robot localization in dynamic environments

The localization problem for an autonomous robot moving in a known environment is a well-studied problem which has seen many elegant solutions. Robot localization in a dynamic environment populated by several moving obstacles, however, is still a challenge for research. In this paper, we use an omnidirectional camera mounted on a mobile robot to perform a sort of scan matching. The omnidirectional vision system finds the distances of the closest color transitions in the environment, mimicking the way laser rangefinders detect the closest obstacles. The similarity of our sensor with classical rangefinders allows the use of practically unmodified Monte Carlo algorithms, with the additional advantage of being able to easily detect occlusions caused by moving obstacles. The proposed system was initially implemented in the RoboCup Middle-Size domain, but the experiments we present in this paper prove it to be valid in a general indoor environment with natural color transitions. We present localization experiments both in the RoboCup environment and in an unmodified office environment. In addition, we assessed the robustness of the system to sensor occlusions caused by other moving robots. The localization system runs in real-time on low-cost hardware.     

双足足球机器人实时避障导航系统研究

介绍了避障控制系统的体系结构和基本步态算法．其次，介绍了路径规划策略，包括避障开始时的全局导航策略和穿越障碍物时的局域导航策略．分析了红外传感器测距实验数据及局域导航实验结果，验证了红外线传感器精确度和导航策略能够满足比赛要求．     

A variational approach to problems in calibration of multiple cameras

This paper addresses the problem of calibrating camera parameters using variational methods. One problem addressed is the severe lens distortion in low-cost cameras. For many computer vision algorithms aiming at reconstructing reliable representations of 3D scenes, the camera distortion effects will lead to inaccurate 3D reconstructions and geometrical measurements if not accounted for. A second problem is the color calibration problem caused by variations in camera responses that result in different color measurements and affects the algorithms that depend on these measurements. We also address the extrinsic camera calibration that estimates relative poses and orientations of multiple cameras in the system and the intrinsic camera calibration that estimates focal lengths and the skew parameters of the cameras. To address these calibration problems, we present multiview stereo techniques based on variational methods that utilize partial and ordinary differential equations. Our approach can also be considered as a coordinated refinement of camera calibration parameters. To reduce computational complexity of such algorithms, we utilize prior knowledge on the calibration object, making a piecewise smooth surface assumption, and evolve the pose, orientation, and scale parameters of such a 3D model object without requiring a 2D feature extraction from camera views. We derive the evolution equations for the distortion coefficients, the color calibration parameters, the extrinsic and intrinsic parameters of the cameras, and present experimental results.     

一个基于全景视觉的移动机器人导航系统的设计与实现

针对移动机器人路径规划与导航的实际应用,设计了一个基于全景视觉的移动机器人路径规划导航系统．首先,对导航系统的体系结构和功能进行描述．然后,分别就如何采用全景视觉传感器进行环境探索与地图创建,基于回归神经网络的广度优先搜索法和Voronoi骨架图法两种路径规划算法原理,以及如何实现按规划路径实施导航这三个方面进行了详细阐述．最后,结合实际机器人进行导航实验,评估导航系统的性能和路径规划算法的有效性．     

有色噪声下的平方根UKF在天文自主导航中的应用

针对由星敏感器和光学导航相机组成的卫星天文自主导航系统, 传统的平方根UKF不能很好地解决测量噪声为有色噪声情况下的非线性滤波问题, 导致导航系统的精度下降. 为此, 提出了一种有色噪声情况下的平方根UKF方法. 同时, 为了避免在数值计算的过程中, 由于舍入误差而破坏误差协方差矩阵的正定性和对称性, 在整个递推计算过程中, 借鉴平方根Kalman滤波理论, 采用协方差矩阵平方根进行递推计算, 改善滤波算法的稳定性, 协方差矩阵的平方根更新用cholesky分解和qr分解来计算. 将该方法应用于卫星天文自主导航系统中, 实验仿真结果表明, 相对于传统的平方根UKF算法, 所设计的平方根UKF算法能够很好地解决测量噪声为有色噪声情况下估计精度低问题.