Appearance‐based landmark selection for visual localization

Visual localization in outdoor environments is subject to varying appearance conditions rendering it difficult to match current camera images against a previously recorded map. Although it is possible to extend the respective maps to allow precise localization across a wide range of differing appearance conditions, these maps quickly grow in size and become impractical to handle on a mobile robotic platform. To address this problem, we present a landmark selection algorithm that exploits appearance co‐observability for efficient visual localization in outdoor environments. Based on the appearance condition inferred from recently observed landmarks, a small fraction of landmarks useful under the current appearance condition is selected and used for localization. This allows to greatly reduce the bandwidth consumption between the mobile platform and a map backend in a shared‐map scenario, and significantly lowers the demands on the computational resources on said mobile platform. We derive a landmark ranking function that exhibits high performance under vastly changing appearance conditions and is agnostic to the distribution of landmarks across the different map sessions. Furthermore, we relate and compare our proposed appearance‐based landmark ranking function to popular ranking schemes from information retrieval, and validate our results on the challenging University of Michigan North Campus long‐term vision and LIDAR data sets (NCLT), including an evaluation of the localization accuracy using ground‐truth poses. In addition to that, we investigate the computational and bandwidth resource demands. Our results show that by selecting 20–30% of landmarks using our proposed approach, a similar localization performance as the baseline strategy using all landmarks is achieved.     

Benchmarking urban six‐degree‐of‐freedom simultaneous localization and mapping

Quite a number of approaches for solving the simultaneous localization and mapping (SLAM) problem exist by now. Some of them have recently been extended to mapping environments with six‐degree‐of‐freedom poses, yielding 6D SLAM approaches. To demonstrate the capabilities of the respective algorithms, it is common practice to present generated maps and successful loop closings in large outdoor environments. Unfortunately, it is nontrivial to compare different 6D SLAM approaches objectively, because ground truth data about the outdoor environments used for demonstration are typically unavailable. We present a novel benchmarking method for generating the ground truth data based on reference maps. The method is then demonstrated by comparing the absolute performance of some previously existing 6D SLAM algorithms that build a large urban outdoor map. © 2008 Wiley Periodicals, Inc.     

Visual‐inertial curve simultaneous localization and mapping: Creating a sparse structured world without feature points

We present a simultaneous localization and mapping (SLAM) algorithm that uses Bézier curves as static landmark primitives rather than feature points. Our approach allows us to estimate the full six degrees of freedom pose of a robot while providing a structured map that can be used to assist a robot in motion planning and control. We demonstrate how to reconstruct the three‐dimensional (3D) location of curve landmarks from a stereo pair and how to compare the 3D shape of curve landmarks between chronologically sequential stereo frames to solve the data association problem. We also present a method to combine curve landmarks for mapping purposes, resulting in a map with a continuous set of curves that contain fewer landmark states than conventional point‐based SLAM algorithms. We demonstrate our algorithm's effectiveness with numerous experiments, including comparisons to existing state‐of‐the‐art SLAM algorithms.     

Bag‐of‐words‐driven,single‐camera simultaneous localization and mapping

This paper describes BoWSLAM, a scheme for a robot to reliably navigate and map previously unknown environments, in real time, using monocular vision alone. BoWSLAM can navigate challenging dynamic and self‐similar environments and can recover from gross errors. Key innovations allowing this include new uses for the bag‐of‐words image representation; this is used to select the best set of frames from which to reconstruct positions and to give efficient wide‐baseline correspondences between many pairs of frames, providing multiple position hypotheses. A graph‐based representation of these position hypotheses enables the modeling and optimization of errors in scale in a dual graph and the selection of only reliable position estimates in the presence of gross outliers. BoWSLAM is demonstrated mapping a 25‐min, 2.5‐km trajectory through a challenging and dynamic outdoor environment without any other sensor input, considerably farther than previous single‐camera simultaneous localization and mapping (SLAM) schemes. © 2010 Wiley Periodicals, Inc.     

Summary Maps for Lifelong Visual Localization

Robots that use vision for localization need to handle environments that are subject to seasonal and structural change, and operate under changing lighting and weather conditions. We present a framework for lifelong localization and mapping designed to provide robust and metrically accurate online localization in these kinds of changing environments. Our system iterates between offline map building, map summary, and online localization. The offline mapping fuses data from multiple visually varied datasets, thus dealing with changing environments by incorporating new information. Before passing these data to the online localization system, the map is summarized, selecting only the landmarks that are deemed useful for localization. This Summary Map enables online localization that is accurate and robust to the variation of visual information in natural environments while still being computationally efficient. We present a number of summary policies for selecting useful features for localization from the multisession map, and we explore the tradeoff between localization performance and computational complexity. The system is evaluated on 77 recordings, with a total length of 30 kilometers, collected outdoors over 16 months. These datasets cover all seasons, various times of day, and changing weather such as sunshine, rain, fog, and snow. We show that it is possible to build consistent maps that span data collected over an entire year, and cover day‐to‐night transitions. Simple statistics computed on landmark observations are enough to produce a Summary Map that enables robust and accurate localization over a wide range of seasonal, lighting, and weather conditions.     

Dynamic Modeling of a Class of Continuum Manipulators in Fixed Orientation

A novel simultaneous localization and mapping (SLAM) technique based on independent particle filters for landmark mapping and localization for a mobile robot based on a high-frequency (HF)-band radio-frequency identification (RFID) system is proposed in this paper. SLAM is a technique for performing self-localization and map building simultaneously. FastSLAM is a standard landmark-based SLAM method. RFID is a robust identification system with ID tags and readers over wireless communication; further, it is rarely affected by obstacles in the robot area or by lighting conditions. Therefore, RFID is useful for self-localization and mapping for a mobile robot with a reasonable accuracy and sufficient robustness. In this study, multiple HF-band RFID readers are embedded in the bottom of an omnidirectional vehicle, and a large number of tags are installed on the floor. The HF-band RFID tags are used as the landmarks of the environment. We found that FastSLAM is not appropriate for this condition for two reasons. First, the tag detection of the HF-band RFID system does not follow the standard Gaussian distribution, which FastSLAM is supposed to have. Second, FastSLAM does not have a sufficient scalability, which causes its failure to handle a large number of landmarks. Therefore, we propose a novel SLAM method with two independent particle filters to solve these problems. The first particle filter is for self-localization based on Monte Carlo localization. The second particle filter is for landmark mapping. The particle filters are nonparametric so that it can handle the non-Gaussian distribution of the landmark detection. The separation of localization and landmark mapping reduces the computational cost significantly. The proposed method is evaluated in simulated and real environments. The experimental results show that the proposed method has more precise localization and mapping and a lower computational cost than FastSLAM.     

Multiple‐Robot Simultaneous Localization and Mapping: A Review

Simultaneous localization and mapping (SLAM) in unknown GPS‐denied environments is a major challenge for researchers in the field of mobile robotics. Many solutions for single‐robot SLAM exist; however, moving to a platform of multiple robots adds many challenges to the existing problems. This paper reviews state‐of‐the‐art multiple‐robot systems, with a major focus on multiple‐robot SLAM. Various issues and problems in multiple‐robot SLAM are introduced, current solutions for these problems are reviewed, and their advantages and disadvantages are discussed.     

Comparative Evaluation of the Consistency of Three‐dimensional Spatial Representations used in Autonomous Robot Navigation

An increasing number of robots for outdoor applications rely on complex three‐dimensional (3D) environmental models. In many cases, 3D maps are used for vital tasks, such as path planning and collision detection in challenging semistructured environments. Thus, acquiring accurate three‐dimensional maps is an important research topic of high priority for autonomously navigating robots. This article proposes an evaluation method that is designed to compare the consistency with which different representations model the environment. In particular, the article examines several popular (probabilistic) spatial representations that are capable of predicting the occupancy of any point in space, given prior 3D range measurements. This work proposes to reformulate the obtained environmental models as probabilistic binary classifiers, thus allowing for the use of standard evaluation and comparison procedures. To avoid introducing localization errors, this article concentrates on evaluating models constructed from measurements acquired at fixed sensor poses. Using a cross‐validation approach, the consistency of different representations, i.e., the likelihood of correctly predicting unseen measurements in the sensor field of view, can be evaluated. Simulated and real‐world data sets are used to benchmark the precision of four spatial models—occupancy grid, triangle mesh, and two variations of the three‐dimensional normal distributions transform (3D‐NDT)—over various environments and sensor noise levels. Overall, the consistency of representation of the 3D‐NDT is found to be the highest among the tested models, with a similar performance over varying input data. © 2013 Wiley Periodicals, Inc.     

Local map fusion for real‐time indoor simultaneous localization and mapping

Among the solutions to the simultaneous localization and mapping (SLAM) problem with probabilistic techniques, the extended Kalman filter (EKF) is a very common approach. There are several approaches to deal with its computational cost, usually based on an adequate selection of features to be updated in real time, while the whole map update is delayed or processed in a background task, allowing one to map larger environments or to carry out multirobot experiments. Although these solutions are theoretically sound, there is a great lack of real experiments in large indoor environments due to several previously unknown problems derived from the geometric model of the map features and the inconsistency of the SLAM‐EKF algorithm. For the first time, these problems are described and solved, and the implementation of the algorithms and solutions presented in this paper achieve excellent results in experiments in different real large indoor environments. © 2006 Wiley Periodicals, Inc.     

Continuous‐Time Three‐Dimensional Mapping for Micro Aerial Vehicles with a Passively Actuated Rotating Laser Scanner

The ability to generate accurate and detailed three‐dimensional (3D) maps of a scene from a mobile platform is an essential technology for a wide variety of applications from robotic navigation to geological surveying. In many instances, the best vantage point is from above, and as a result, there is a growing demand for low‐altitude mapping solutions from micro aerial vehicles such as small quadcopters. Existing lidar‐based 3D airborne mapping solutions rely on GPS/INS solutions for positioning, or focus on producing relatively low‐fidelity or locally focused maps for the purposes of autonomous navigation. We have developed a general‐purpose airborne 3D mapping system capable of continuously scanning the environment during flight to produce accurate and dense point clouds without the need for a separate positioning system. A key feature of the system is a novel passively driven mechanism to rotate a lightweight 2D laser scanner using the rotor downdraft from a quadcopter. The data generated from the spinning laser is input into a continuous‐time simultaneous localization and mapping (SLAM) solution to produce an accurate 6 degree‐of‐freedom trajectory estimate and a 3D point cloud map. Extensive results are presented illustrating the versatility of the platform in a variety of environments including forests, caves, mines, heritage sites, and industrial facilities. Comparison with conventional surveying methods and equipment demonstrates the high accuracy and precision of the proposed solution.     

What localizes beneath: A metric multisensor localization and mapping system for autonomous underground mining vehicles

Robustly and accurately localizing vehicles in underground mines is particularly challenging due to the unavailability of GPS, variable and often poor lighting conditions, visual aliasing in long tunnels, and airborne dust and water. In this paper, we present a novel, infrastructure‐less, multisensor localization method for robust autonomous operation within underground mines. The proposed method integrates with existing mine site commissioning and operation procedures and includes both an offline map‐building process and an online localization algorithm. The approach combines the strengths of visual‐based place recognition, LIDAR‐based localization, and odometry in a particle filter fusion process. We provide an extensive experimental validation using new large data sets acquired in two operational Australian underground hard‐rock mines (including a 600m‐deep multilevel mine with approximately 33 km of mapping data and 7 km of vehicle localization) by actual mining vehicles during production operations. We demonstrate a significant increase in localization accuracy over prior state‐of‐the‐art SLAM research systems and real‐time operation, with processing times in the order of 10 Hz. We present results showing a mean error of 0.68 m from the Queensland Mine data set and 1.32 m from the New South Wales Mine data set and at least 86% reduction in error compared with prior state of the art. We also analyze the impact of the particle filter parameters with respect to localization accuracy. Together this study represents a new approach to positioning systems for currently deployed autonomous vehicles within underground mine environments.     

Projection‐based head‐tracking 3‐D displays

Abstract— This paper describes the construction and operation of four 3‐D displays in which each display produces two images for each eye and thus fits into the category of projection‐based binocular stereoscopic displays. The four 3‐D displays described are pico‐projector‐based, liquid‐ crystal—on—silicon (LCOS) conventional projector‐based, 120‐Hz digital‐light‐processor (DLP) projector‐ based, and the HELIUM3D system. In the first three displays, images are produced on a direct‐view LCD whose conventional backlight is replaced with a projection illumination source that is controlled by a multi‐user head tracker; novel steering optics direct the projector output to regions referred to as exit pupils located at the viewers' eyes. In the HELIUM3D display, the image information is supplied by a horizontally scanned, fast, light valve whose output is controlled by a spatial light modulator (SLM) to direct images to the appropriate viewers' eyes. The current statu s and the multimodal potential of the HELIUM3D display are described.     

Parallel Multiple‐Bounce Irradiance Caching

Building designers rely on predictive rendering techniques to design naturally and artificially lit environments. However, despite decades of work on the correctness of global illumination rendering techniques, our ability to accurately predict light levels in buildings—and to do so in a short time frame as part of an iterative design process—remains limited. In this paper, we present a novel approach to parallelizing construction of an irradiance cache over multiple‐bounce paths. Relevant points for irradiance calculation based on one or multiple cameras are located by tracing rays through multiple‐bounce paths. Irradiance values are then saved to a cache in reverse bounce order so that the irradiance calculation at each bounce samples from previously calculated values. We show by comparison to high‐dynamic range photography of a moderately complex space that our method can predict luminance distribution as accurately as Radiance , the most widely validated tool used today for architectural predictive rendering of daylit spaces, and that it is faster by an order of magnitude.     

RANGE–Robust autonomous navigation in GPS‐denied environments

This paper addresses the problem of autonomous navigation of a micro air vehicle (MAV) in GPS‐denied environments. We present experimental validation and analysis for our system that enables a quadrotor helicopter, equipped with a laser range finder sensor, to autonomously explore and map unstructured and unknown environments. The key challenge for enabling GPS‐denied flight of a MAV is that the system must be able to estimate its position and velocity by sensing unknown environmental structure with sufficient accuracy and low enough latency to stably control the vehicle. Our solution overcomes this challenge in the face of MAV payload limitations imposed on sensing, computational, and communication resources. We first analyze the requirements to achieve fully autonomous quadrotor helicopter flight in GPS‐denied areas, highlighting the differences between ground and air robots that make it difficult to use algorithms developed for ground robots. We report on experiments that validate our solutions to key challenges, namely a multilevel sensing and control hierarchy that incorporates a high‐speed laser scan‐matching algorithm, data fusion filter, high‐level simultaneous localization and mapping, and a goal‐directed exploration module. These experiments illustrate the quadrotor helicopter's ability to accurately and autonomously navigate in a number of large‐scale unknown environments, both indoors and in the urban canyon. The system was further validated in the field by our winning entry in the 2009 International Aerial Robotics Competition, which required the quadrotor to autonomously enter a hazardous unknown environment through a window, explore the indoor structure without GPS, and search for a visual target. © 2011 Wiley Periodicals, Inc.     

Learning‐based Nonlinear Model Predictive Control to Improve Vision‐based Mobile Robot Path Tracking

This paper presents a Learning‐based Nonlinear Model Predictive Control (LB‐NMPC) algorithm to achieve high‐performance path tracking in challenging off‐road terrain through learning. The LB‐NMPC algorithm uses a simple a priori vehicle model and a learned disturbance model. Disturbances are modeled as a Gaussian process (GP) as a function of system state, input, and other relevant variables. The GP is updated based on experience collected during previous trials. Localization for the controller is provided by an onboard, vision‐based mapping and navigation system enabling operation in large‐scale, GPS‐denied environments. The paper presents experimental results including over 3 km of travel by three significantly different robot platforms with masses ranging from 50 to 600 kg and at speeds ranging from 0.35 to 1.2 m/s (associated video at http://tiny.cc/RoverLearnsDisturbances ). Planned speeds are generated by a novel experience‐based speed scheduler that balances overall travel time, path‐tracking errors, and localization reliability. The results show that the controller can start from a generic a priori vehicle model and subsequently learn to reduce vehicle‐ and trajectory‐specific path‐tracking errors based on experience.     

动态环境下基于路径规划的机器人同步定位与地图构建

针对动态环境下随机目标同时为特征点和障碍物的情况,提出一种基于路径规划的同步定位与地图构 建（SLAM）算法．机器人在同步定位与地图构建的同时,基于势场原理来规划机器人下一步的运动控制规律．利用 混合当前统计模型的交互式多模型（IMM）方法预测随机目标的轨迹,采用最近邻数据关联方法将动态随机目标关 联到地图中．算法构建的地图由静态特征点和随机目标的轨迹组成．仿真结果表明,提出的算法解决了动态环境中 存在的随机目标同时为障碍物时机器人的同步定位与地图构建问题,相关性能指标验证了算法的一致性估计．     

Vision‐based Localization and Robot‐centric Mapping in Riverine Environments

This paper presents a vision‐based localization and mapping algorithm developed for an unmanned aerial vehicle (UAV) that can operate in a riverine environment. Our algorithm estimates the three‐dimensional positions of point features along a river and the pose of the UAV. By detecting features surrounding a river and the corresponding reflections on the water's surface, we can exploit multiple‐view geometry to enhance the observability of the estimation system. We use a robot‐centric mapping framework to further improve the observability of the estimation system while reducing the computational burden. We analyze the performance of the proposed algorithm with numerical simulations and demonstrate its effectiveness through experiments with data from Crystal Lake Park in Urbana, Illinois. We also draw a comparison to existing approaches. Our experimental platform is equipped with a lightweight monocular camera, an inertial measurement unit, a magnetometer, an altimeter, and an onboard computer. To our knowledge, this is the first result that exploits the reflections of features in a riverine environment for localization and mapping.     

SOFT‐SLAM: Computationally efficient stereo visual simultaneous localization and mapping for autonomous unmanned aerial vehicles

Autonomous navigation of unmanned aerial vehicles (UAVs) in GPS‐denied environments is a challenging problem, especially for small‐scale UAVs characterized by a small payload and limited battery autonomy. A possible solution to the aforementioned problem is vision‐based simultaneous localization and mapping (SLAM), since cameras, due to their dimensions, low weight, availability, and large information bandwidth, circumvent all the constraints of UAVs. In this paper, we propose a stereo vision SLAM yielding very accurate localization and a dense map of the environment developed with the aim to compete in the European Robotics Challenges (EuRoC) targeting airborne inspection of industrial facilities with small‐scale UAVs. The proposed approach consists of a novel stereo odometry algorithm relying on feature tracking (SOFT), which currently ranks first among all stereo methods on the KITTI dataset. Relying on SOFT for pose estimation, we build a feature‐based pose graph SLAM solution, which we dub SOFT‐SLAM. SOFT‐SLAM has a completely separate odometry and mapping threads supporting large loop‐closing and global consistency. It also achieves a constant‐time execution rate of 20 Hz with deterministic results using only two threads of an onboard computer used in the challenge. The UAV running our SLAM algorithm obtained the highest localization score in the EuRoC Challenge 3, Stage IIa–Benchmarking, Task 2. Furthermore, we also present an exhaustive evaluation of SOFT‐SLAM on two popular public datasets, and we compare it to other state‐of‐the‐art approaches, namely ORB‐SLAM2 and LSD‐SLAM. The results show that SOFT‐SLAM obtains better localization accuracy on the majority of datasets sequences, while also having a lower runtime.     

Dual‐tracked mobile robot for motion in challenging terrains

In this paper we present a novel design for a dual‐tracked mobile robot. The robot is designed for safe, stable, and reliable motion in challenging terrains, tunnels, and confined spaces. It consists of two tracked platforms connected with a semipassive mechanism. Sensors attached to the connecting mechanism provide redundant localization data that improve the vehicle's autonomous dead reckoning. Each tracked platform mechanically backs up the other platform, resulting in a more robust and reliable operation. The load share between the platforms enables a high payload‐to‐weight ratio even on soft and slippery terrains. The robot's configurations make it suitable for motion in confined spaces such as underground tunnels, collapsed structures, pipes, and caves. The paper presents the mechanical design of the robot, its kinematic model, stability analysis, and a motion planner. Experimental results conducted on prototype models in various types of environments verify the robot capabilities to operate successfully on challenging terrains. The dual‐tracked robot is capable of climbing slopes 50% steeper than a single robot can. Moreover, the improved odometry system shows high accuracy with 2% error of the total travel distance. © 2011 Wiley Periodicals, Inc.     

Terrain prediction for an eight‐legged robot

Most legged robots must negotiate unknown environments with little or no terrain knowledge, as autonomous terrain mapping for robots is limited. A predictive terrain contour mapping strategy is proposed, which employs the use of a feed‐forward neural network to predict the contours in environments, based on the positions of the neighboring legs. The predicted performance is better than previous implementations. © 2002 John Wiley & Sons, Inc.