Simplified map representation and map learning system for autonomous navigation of mobile robots

Autonomous mobile robots need environmental maps to navigate to specific destinations, but there are difficulties in generating and acquiring efficient maps for them. Map learning systems and map representation for autonomous robot navigation are highly interrelated and need a total system design that combines these two factors. This study considers a combined simple map representation and map learning system. The proposed map representation includes geometrical relationships between important places and grid maps for these places, but not a total grid map of the environment. In particular, the study focuses on the ability to recognize places based on image features. Successful experiments on autonomous navigation with the proposed map representation using an actual mobile robot are described.     

Supervised semantic labeling of places using information extracted from sensor data

Indoor environments can typically be divided into places with different functionalities like corridors, rooms or doorways. The ability to learn such semantic categories from sensor data enables a mobile robot to extend the representation of the environment facilitating interaction with humans. As an example, natural language terms like “corridor” or “room” can be used to communicate the position of the robot in a map in a more intuitive way. In this work, we first propose an approach based on supervised learning to classify the pose of a mobile robot into semantic classes. Our method uses AdaBoost to boost simple features extracted from sensor range data into a strong classifier. We present two main applications of this approach. Firstly, we show how our approach can be utilized by a moving robot for an online classification of the poses traversed along its path using a hidden Markov model. In this case we additionally use as features objects extracted from images. Secondly, we introduce an approach to learn topological maps from geometric maps by applying our semantic classification procedure in combination with a probabilistic relaxation method. Alternatively, we apply associative Markov networks to classify geometric maps and compare the results with a relaxation approach. Experimental results obtained in simulation and with real robots demonstrate the effectiveness of our approach in various indoor environments.     

2D map-building and localization in outdoor environments

Determining the pose (position and orientation) of a vehicle at any time is termed localization and is of paramount importance in achieving reliable and robust autonomous navigation. Knowing the pose it is possible to achieve high level tasks such as path planning. A new map-based algorithm for the localization of vehicles operating in harsh outdoor environments is presented in this article. A map building algorithm using observations from a scanning laser rangefinder is developed for building a polyline map that adequately captures the geometry of the environment. Using this map, the Iterative Closest Point (ICP) algorithm is employed for matching laser range images from the rangefinder to the polyline map. Once correspondences are established, an Extended Kalman Filter (EKF) algorithm provides reliable vehicle state estimates using a nonlinear observation model based on the vertices of the polyline map. Data gathered during field trials in an outdoor environment is used to test the efficiency of the proposed ICP-EKF algorithm in achieving the localization of a four-wheel drive (4WD) vehicle. © 2005 Wiley Periodicals, Inc.     

DeepDSAIR: Deep 6-DOF camera relocalization using deblurred semantic-aware image representation for large-scale outdoor environments

Deep Learning methods can deploy a fast, robust and lightweight model to solve the problem of 6-DOF camera relocalization in large-scale outdoor environments. However, two significant characteristics of captured images in a large-scale outdoor environment are moving objects, which should not include in the representation of an environment, and also motion blur which widely exists in the images captured with moving cameras. None of the existing approaches study and investigate these two problems in their method. This paper, for the first time, proposes a deep network architecture that is trained based on the knowledge achieved by combining deblurring and semantic segmentation modules and examines the effect of this combination on a challenging dataset. Results show approximately 20 and 50% improvement in camera position and orientation re-localization error respectively.     

Relational Model for Robotic Semantic Navigation in Indoor Environments

The emergence of service robots in our environment raises the need to find systems that help the robots in the task of managing the information from human environments. A semantic model of the environment provides the robot with a representation closer to the human perception, and it improves its human-robot communication system. In addition, a semantic model will improve the capabilities of the robot to carry out high level navigation tasks. This paper presents a semantic relational model that includes conceptual and physical representation of objects and places, utilities of the objects, and semantic relation among objects and places. This model allows the robot to manage the environment and to make queries about the environment in order to do plans for navigation tasks. In addition, this model has several advantages such as conceptual simplicity and flexibility of adaptation to different environments. To test the performance of the proposed semantic model, the output for the semantic inference system is associate to the geometric and topological information of objects and places in order to do the navigation tasks.     

Coarse head pose estimation of construction equipment operators to formulate dynamic blind spots

Several hundred workers die in construction in the United States every year because equipment operators are unable to see their fellow workers during operation of their vehicle. In this paper we propose a step towards improving this situation by providing an automated method based on range imaging for estimating the coarse head orientation of a construction equipment operator. This research utilizes commercially-available low resolution range cameras to measure the continuously changing field-of-view (FOV) of an equipment operator in outdoor construction. This paper presents a methodology to measure so-called dynamic blind spot maps. The dynamic blind spot map is then projected on a known static equipment blind spot map that already exists to each construction vehicle. A robust computational coarse head pose estimation algorithm and results to three different pieces of construction equipment and multiple operators are presented. The developed method has the potential in automatically determining the spaces around vehicles that are currently not in the field-of-view of the vehicle operator thus providing eventually additional means and technology for improving safety in construction.     

The Stixel World: A medium-level representation of traffic scenes

Recent progress in advanced driver assistance systems and the race towards autonomous vehicles is mainly driven by two factors: (1) increasingly sophisticated algorithms that interpret the environment around the vehicle and react accordingly, and (2) the continuous improvements of sensor technology itself. In terms of cameras, these improvements typically include higher spatial resolution, which as a consequence requires more data to be processed. The trend to add multiple cameras to cover the entire surrounding of the vehicle is not conducive in that matter. At the same time, an increasing number of special purpose algorithms need access to the sensor input data to correctly interpret the various complex situations that can occur, particularly in urban traffic.By observing those trends, it becomes clear that a key challenge for vision architectures in intelligent vehicles is to share computational resources. We believe this challenge should be faced by introducing a representation of the sensory data that provides compressed and structured access to all relevant visual content of the scene. The Stixel World discussed in this paper is such a representation. It is a medium-level model of the environment that is specifically designed to compress information about obstacles by leveraging the typical layout of outdoor traffic scenes. It has proven useful for a multitude of automotive vision applications, including object detection, tracking, segmentation, and mapping.In this paper, we summarize the ideas behind the model and generalize it to take into account multiple dense input streams: the image itself, stereo depth maps, and semantic class probability maps that can be generated, e.g., by deep convolutional neural networks. Our generalization is embedded into a novel mathematical formulation for the Stixel model. We further sketch how the free parameters of the model can be learned using structured SVMs.     

Loop-closing: A typicality approach

Loop-closing has long been identified as a critical issue when building maps from local observations. Topological mapping methods abstract the problem of how loops are closed from the problem of how to determine the metrical layout of places in the map and how to deal with noisy sensors.The typicality problem refers to the identification of new classes in a general classification context. This typicality concept is used in this paper to help a robot acquire a topological representation of the environment during its exploration phase. The problem is addressed using the INCA statistic which follows a distance-based approach.In this paper we describe a place recognition approach based on match testing by means of the INCA test. We describe the theoretical basis of the approach and present extensive experimental results performed in both a simulated and a real robot-environment system; Behaviour Based philosophy is used to construct the whole control architecture. Obtained results show the validity of the approach.     

结合状态机和动态目标路径的无人驾驶决策仿真

目的 决策系统是无人驾驶技术的核心研究之一。已有决策系统存在逻辑不合理、计算效率低、应用场景局限等问题,因此提出一种动态环境下无人驾驶路径决策仿真。方法 首先,基于规则模型构建适于无人驾驶决策系统的交通有限状态机;其次,针对交通动态特征,提出基于统计模型的动态目标路径算法计算状态迁移风险;最后,将交通状态机和动态目标路径算法有机结合,设计出一种基于有限状态机的无人驾驶动态目标路径模型,适用于交叉口冲突避免和三车道换道行为。将全速度差连续跟驰模型运用到换道规则中,并基于冲突时间提出动态临界跟车距离。结果 为验证模型的有效性和高效性,对交通环境进行虚拟现实建模,模拟交叉口通行和三车道换道行为,分析文中模型对车流量和换道率的影响。实验结果显示,在交叉口通行时,自主车辆不仅可以检测冲突还可以根据风险评估结果执行安全合理的决策。三车道换道时,自主车辆既可以实现紧急让道,也可以通过执行换道达成自身驾驶期望。通过将实测数据和其他两种方法对比,当车流密度在0.20.5时,本文模型的平均速度最高分别提高32 km/h和22 km/h。当车流密度不超过0.65时,本文模型的换道成功率最高分别提升37%和25%。结论 实验结果说明本文方法不仅可以在动态城区环境下提高决策安全性和正确性,还可以提高车流量饱和度,缓解交通堵塞。     

A Small and Lightweight Autonomous Laser Mapping System without GPS

Autonomous mapping systems execute multiple tasks that include navigation, location, and map generation via the collaborative work of multiple sensors. They are the object of a substantial research focus in the fields of robotics and remote sensing. Although the state‐of‐the‐art mobile mapping systems typically found in ready‐made vehicles or robots are reliable, they are rather large and heavy, their cost is high, and they generally use GPS and an inertial measurement unit to position, so their working environments are limited. After reviewing the current state of autonomous mapping systems, we describe the design and development of a small and lightweight autonomous mapping system (ASQ‐6DMapSys) without GPS, which incorporates low‐cost sensors and components. We describe the layout and selection strategy for sensors and other components in detail, and we present the design methodology for each subsystem. The ASQ‐6DMapSys employs a two‐dimensional (2D) lidar, an inclinometer, and two wheel encoders, which constitute a pose subsystem that uses extended Kalman filtering and simultaneous localization and mapping techniques to compute the pose of the vehicle body. A low‐cost 3D lidar that we developed is also installed on the vehicle body, and the resultant data are aligned with the corresponding pose data of the vehicle body to build a 3D point cloud that describes the global geometry of the environment. We designed and developed every subsystem of the ASQ‐6DMapSys, including the robot vehicle, so it will be easy to expand its functions in the future. The ASQ‐6DMapSys performs well in indoor, outdoor, and tunnel environments, and the experiments in different environments show that the ASQ‐6DMapSys is an effective, small, and lightweight autonomous mapping system with a high performance/price ratio.     

An Intelligent and Integrated Driver Assistance System for Increased Safety and Convenience Based on All-around Sensing

Advanced driver assistance systems (ADAS) support the driver’s decision making to increase safety and comfort by providing an ergonomic display of the driving environment as well as issuing the warning signals or even exerting active control in case of dangerous conditions. Most previous research and products intend to offer only a single warning service, such as lane departure warning, collision warning, lane change assistance, etc. Although each component of these functions elevates the driving safety and convenience to a certain degree, a new type of ADAS will be developed to integrate all of the important functions with an efficient human–machine interface (HMI) framework for various driving conditions. We present an all-around sensing system based on an integrated ADAS that senses all directions using 2 cameras and 8 sonars, recognizes the driving environment via lane and vehicle detection, and construct a novel bird’s-eye view HMI of the environment for easy comprehension that even gives a proper warning signal in case of imminent danger. It was tested on our experimental vehicle with a good demonstration its working. Further, it has a good potential for commercial use by virtue of the low cost of the sensors used.     

基于Transformer和边缘解耦的鸟瞰地图语义分割算法

高精(high-definition, HD)地图可以提供准确的道路信息和丰富的语义信息,使自动驾驶系统引导车辆正确行驶。高精地图通常依赖人工标注,现有自动化标注方法在自动驾驶场景下的识别精度较低,导致高精地图标注效率低下。为了解决这一问题,提出了一种新的用于高精地图自动标注的语义分割方法MapFormer,包括一个多级特征融合模块,能够使模型聚合不同级别的细节和语义信息;一种新的边界解耦联合解码器用以提高模型处理类别间边界的能力。在鸟瞰图数据集上的实验验证了该模型不仅在分割精度上取得了优秀的表现,而且在对类别边界的处理上更为清晰。其mIoU为55.82%,高于SegFormer的mIoU 1.03%,该方法可提升高精地图标注效率与标注自动化率。     

A mapping and localization framework for scalable appearance-based navigation

This paper presents a vision framework which enables feature-oriented appearance-based navigation in large outdoor environments containing other moving objects. The framework is based on a hybrid topological–geometrical environment representation, constructed from a learning sequence acquired during a robot motion under human control. At the higher topological layer, the representation contains a graph of key-images such that incident nodes share many natural landmarks. The lower geometrical layer enables to predict the projections of the mapped landmarks onto the current image, in order to be able to start (or resume) their tracking on the fly. The desired navigation functionality is achieved without requiring global geometrical consistency of the underlying environment representation. The framework has been experimentally validated in demanding and cluttered outdoor environments, under different imaging conditions. The experiments have been performed on many long sequences acquired from moving cars, as well as in large-scale real-time navigation experiments relying exclusively on a single perspective vision sensor. The obtained results confirm the viability of the proposed hybrid approach and indicate interesting directions for future work.     

基于动态物体跟踪的语义SLAM

针对传统视觉SLAM在动态场景下容易出现特征匹配错误从而导致定位精度下降的问题,提出了一种基于动态物体跟踪的语义SLAM算法。基于经典的视觉SLAM框架,提取动态物体进行帧间跟踪,并利用动态物体的位姿信息来辅助相机自身的定位。首先,算法在数据预处理中使用YOLACT、RAFT以及SC-Depth网络,分别提取图像中的语义掩膜、光流向量以及像素深度值。其次,视觉前端模块根据所提信息,通过语义分割掩膜、运动一致性检验以及遮挡点检验算法计算概率图以平滑区分场景中的动态特征与静态特征。然后,后端中的捆集调整模块融合了物体运动的多特征约束以提高算法在动态场景中的位姿估计性能。最后,在KITTI和OMD数据集的动态场景中进行对比验证。实验表明,所提算法能够准确地跟踪动态物体,在室内外动态场景中具备鲁棒、良好的定位性能。     

Proximity Based Automatic Data Annotation for Autonomous Driving

The recent development in autonomous driving involves high-level computer vision and detailed road scene understanding.Today,most autonomous vehicles employ expensive high quality sensor-set such as light detection and ranging(LIDAR)and HD maps with high level annotations.In this paper,we propose a scalable and affordable data collection and annotation framework image-to-map annotation proximity(I2MAP),for affordance learning in autonomous driving applications.We provide a new driving dataset using our proposed framework for driving scene affordance learning by calibrating the data samples with available tags from online database such as open street map(OSM).Our benchmark consists of 40000 images with more than40 affordance labels under various day time and weather even with very challenging heavy snow.We implemented sample advanced driver-assistance systems(ADAS)functions by training our data with neural networks(NN)and cross-validate the results on benchmarks like KITTI and BDD100K,which indicate the effectiveness of our framework and training models.     

基于多目标协同演化算法的大规模自动驾驶策略

目前无人驾驶技术领域的研究重点主要集中在单车层面的感知、决策与控制,而缺少对多车 之间交互及博弈的研究,因此无法有效降低交通系统整体事故率并提升通行效率。该文提出一种基于 合作博弈理论的大规模自动驾驶策略涌现方法。通过建立面向网联汽车、多目标优化决策的合作博弈 演化平台,并构造了一种网格道路模型和车辆运动学模型,使得系统中各车辆之间以近邻博弈的方式 进行交互;同时系统采用分布式算法并具有间接交互的特点,最终模型计算复杂度与模拟车辆规模呈 线性关系。实验结果表明,最佳策略涌现后,事故率和平均速度均取得明显改善,其中事故率降低了 90%,模型计算速度提升了 30%。该方法可应用于包含数百万辆自动驾驶汽车的城市级智能交通规划 系统中。     

From topological knowledge to geometrical map

The behavioral approach to robot navigation, characterized by a representation of the environment that is topological and robot-environmental interactions that are reactive, is preferable to purely geometrical navigation because it is far more robust against unpredictable changes of the environment. Nevertheless, there is still a need to obtain geometrical maps. This paper considers a geometrical map reconstruction that relies on the topological knowledge and uses redundant odometric measurements taken while the robot moves along the paths of the topological map. Five methods are presented and compared, in experiments involving a Nomad200 mobile robot operating in a real environment.