State-chain sequential feedback reinforcement learning for path planning of autonomous mobile robots

This paper deals with a new approach based on Q-learning for solving the problem of mobile robot path planning in complex unknown static environments.As a computational approach to learning through interaction with the environment,reinforcement learning algorithms have been widely used for intelligent robot control,especially in the field of autonomous mobile robots.However,the learning process is slow and cumbersome.For practical applications,rapid rates of convergence are required.Aiming at the problem of slow convergence and long learning time for Q-learning based mobile robot path planning,a state-chain sequential feedback Q-learning algorithm is proposed for quickly searching for the optimal path of mobile robots in complex unknown static environments.The state chain is built during the searching process.After one action is chosen and the reward is received,the Q-values of the state-action pairs on the previously built state chain are sequentially updated with one-step Q-learning.With the increasing number of Q-values updated after one action,the number of actual steps for convergence decreases and thus,the learning time decreases,where a step is a state transition.Extensive simulations validate the efficiency of the newly proposed approach for mobile robot path planning in complex environments.The results show that the new approach has a high convergence speed and that the robot can find the collision-free optimal path in complex unknown static environments with much shorter time,compared with the one-step Q-learning algorithm and the Q(λ)-learning algorithm.     

基于传感器信息的智能移动机器人导航评述

导航是研究智能移动机器人技术中的一个重要领域,对自主导航技术的关键问题——路径规划进行了评述。路径规划一般可分为基于模型的环境信息完全知道的全局路径规划和基于传感器的环境信息完全未知或部分未知的局部路径规划2种类型。分别对各种方法的发展现状进行了总结,指出了各种方法的优点和不足。     

Hybrid autonomous control for multi mobile robots

Reinforcement learning can be an adaptive and flexible control method for autonomous system. It does not need a priori knowledge; behaviors to accomplish given tasks are obtained automatically by repeating trial and error. However, with increasing complexity of the system, the learning costs are increased exponentially. Thus, application to complex systems, like a many redundant d.o.f. robot and multi-agent system, is very difficult. In the previous works in this field, applications were restricted to simple robots and small multi-agent systems, and because of restricted functions of the simple systems that have less redundancy, effectiveness of reinforcement learning is restricted. In our previous works, we had taken these problems into consideration and had proposed new reinforcement learning algorithm, 'Q-learning with dynamic structuring of exploration space based on GA (QDSEGA)'. Effectiveness of QDSEGA for redundant robots has been demonstrated using a 12-legged robot and a 50-link manipulator. However, previous works on QDSEGA were restricted to redundant robots and it was impossible to apply it to multi mobile robots. In this paper, we extend our previous work on QDSEGA by combining a rule-based distributed control and propose a hybrid autonomous control method for multi mobile robots. To demonstrate the effectiveness of the proposed method, simulations of a transportation task by 10 mobile robots are carried out. As a result, effective behaviors have been obtained.     

Fault-tolerant flocking for a group of autonomous mobile robots

Consider a system composed of mobile robots that move on the plane, each of which independently executing its own instance of an algorithm. Given a desired geometric pattern, the flocking problem consists in ensuring that the robots form this pattern and maintain it while moving together on the plane. In this paper, we explore flocking in the presence of faulty robots, where the desired pattern is a regular polygon. We propose a distributed fault tolerant flocking algorithm assuming a semi-synchronous model with a k-bounded scheduler, in the sense that no robot is activated no more than k times between any two consecutive activations of any other robot.The algorithm is composed of three parts: failure detector, ranking assignment, and flocking algorithm. The role of the rank assignment is to provide a persistent and unique ranking for the robots. The failure detector identifies the set of currently correct robots in the system. Finally, the flocking algorithm handles the movement and reconfiguration of the flock, while maintaining the desired shape. The difficulty of the problem comes from the combination of the three parts, together with the necessity to prevent collisions and allow the rotation of the flock. We formally prove the correctness of our proposed solution.     

Fuzzy rule-based environment-aware autonomous mobile robots for actuated touring

Intelligent Service Robotics - The involvement of computer-programmed autonomous mobile robots in real-time activities is emerging in the recent years. The actuation and interaction of the robots...     

A chaotic path planning generator for autonomous mobile robots

This work presents a chaotic path planning generator which is used in autonomous mobile robots, in order to cover a terrain. The proposed generator is based on a nonlinear circuit, which shows chaotic behavior. The bit sequence, produced by the chaotic generator, is converted to a sequence of planned positions, which satisfies the requirements for unpredictability and fast scanning of the entire terrain. The nonlinear circuit and the trajectory-planner are described thoroughly. Simulation tests confirm that with the proposed path planning generator better results can be obtained with regard to previous works.     

Mobility characterization for autonomous mobile robots using machine learning

This paper presents a supervised learning approach to improving the autonomous mobility of wheeled robots through sensing the robot’s interaction with terrain ‘underfoot.’ Mobility characterization is cast as a hierarchical task, in which pre-immobilization detection is achieved using support vector machines in time to prevent full immobilization, and if a pre-immobilization condition is detected, the associated terrain feature affecting mobility is identified using a Hidden Markov model. These methods are implemented using a hierarchical, layered control scheme developed for the Yeti robot, a 73-kg, four-wheeled robot designed to perform autonomous medium-range missions in polar terrain. The methodology is motivated by the difficultly of visually recognizing terrain features that impact mobility in low contrast terrain. The efficacy of the approach is evaluated using data from a suite of proprioceptive sensors. Real-time implementation shows that Yeti can consistently detect pre-immobilization conditions, stop in time to avoid unrecoverable immobilization, identify the terrain feature presenting the mobility challenge, and execute an escape sequence to retreat from the condition.     

Development environments for autonomous mobile robots: A survey

Robotic Development Environments (RDEs) have come to play an increasingly important role in robotics research in general, and for the development of architectures for mobile robots in particular. Yet, no systematic evaluation of available RDEs has been performed; establishing a comprehensive list of evaluation criteria targeted at robotics applications is desirable that can subsequently be used to compare their strengths and weaknesses. Moreover, there are no practical evaluations of the usability and impact of a large selection of RDEs that provides researchers with the information necessary to select an RDE most suited to their needs, nor identifies trends in RDE research that suggest directions for future RDE development. This survey addresses the above by selecting and describing nine open source, freely available RDEs for mobile robots, evaluating and comparing them from various points of view. First, based on previous work concerning agent systems, a conceptual framework of four broad categories is established, encompassing the characteristics and capabilities that an RDE supports. Then, a practical evaluation of RDE  usability in designing, implementing, and executing robot architectures is presented. Finally, the impact of specific RDEs on the field of robotics is addressed by providing a list of published applications and research projects that give concrete examples of areas in which systems have been used. The comprehensive evaluation and comparison of the nine RDEs concludes with suggestions of how to use the results of this survey and a brief discussion of future trends in RDE design.

Deliberation and reactivity in autonomous mobile robots

This paper discusses issues related to the design of the control architectures for an autonomous mobile robot capable of performing tasks efficiently and intelligently, i.e. in a manner adapted to its environment, to its own state and to the execution status of its task. We present our developments and experimentations on mobile robot navigation and show how it is necessary to produce representations at several levels of abstraction, that are used by adequate processes for obstacle detection, target recognition, robot localization, and motion planning and control. We also show that deliberation is necessary for the robot in order to anticipate events, take efficient decisions, and react adequately to asynchronous events. We also discuss the organization of the system, i.e. the design of the control architecture.     

Dynamic omnidirectional vision for mobile robots

Mobile robotic devices hold great promise for a variety of applications in industry. A key step in the design of a mobile robot is to determine the navigation method for mobility control. The purpose of this paper is to describe a new algorithm for omnidirectional vision navigation. A prototype omnidirectional vision system and the implementation of the navigation techniques using this modern sensor and an advanced automatic image processor is described. The significance of this work is in the development of a new and novel approach—dynamic omnidirectional vision for mobile robots and autonomous guided vehicles.     

Dynamic feedback robust stabilization of nonholonomic mobile robots based on visual servoing

This paper investigates the visual servoing robust stabilization of nonholonomic mobile robots. The calibration of visual parameters is not only complicated, but also needs great consumption of calculated time so that the accurate calibration is impossible in some situations for high requirement of real timing. Hence, it is interesting and important to consider the design of stabilizing controllers for nonholonomic kinematic systems with uncalibrated visual parameters. A novel uncertain model of these nonholonomic kinematic systems is proposed. Based on this model, a stabilizing controller is discussed by using dynamic feedback and two-step techniques. The proposed robust controller makes the mobile robot image pose and the orientation converge to the desired configuration despite the lack of depth information and the lack of precise visual parameters. The stability of the closed loop system is rigorously proved. The simulation is given to show the effectiveness of the presented controllers.     

Models of computation for reactive control of autonomous mobile robots

The paper studies computation models for tasks performed by autonomous mobile robots. Such tasks can be accomplished by reactive control algorithms. Reactive control systems can be described using different models of computation which have as distinguishing feature the abstraction level of time. Thus, three computation models are defined: the untimed model, the synchronous model and the timed model. It is shown that the clocked-synchronous model of computation is more appropriate for describing the controller for a parallel parking task.     

自主移动机器人避障技术研究现状

对避障技术的核心内容暨避障算法展开归纳介绍,依照传统算法和智能算法分类,详细介绍了应用较多的各避障算法的基本原理、优缺点及部分改进方案;对自主移动机器人避障技术的应用前景和发展趋势作出了展望.     

自主移动机器人室内定位方法研究综述

自主移动机器人的室内定位作为机器人研究领域中最基本的问题已被广泛研究。根据定位技术和传感器的不同,将室内定位方法分为航迹推算定位、地图匹配定位和基于信标定位三类。详细介绍了超声波网络定位系统和基于无线射频识别（RFID）的定位方法。对几种基于概率的定位算法做了分析和对比,并对自主移动机器人室内定位方法的研究方向做了展望。     

Distributed algorithms for partitioning a swarm of autonomous mobile robots

A number of recent studies address systems of mobile autonomous robots from a distributed computing point of view. Although such systems employ robots that are relatively weak and simple (i.e., dimensionless, oblivious and anonymous), they are nevertheless expected to have strong fault tolerance capabilities as a group. This paper studies the partitioning problem, where n$n$ robots must divide themselves into k$k$ size-balanced groups, and examines the impact of common orientation on the solvability of this problem. First, deterministic crash-fault-tolerant algorithms are given for the problem in the asynchronous full-compass and semi-synchronous half-compass models, and a randomized algorithm is given for the semi-synchronous no-compass model. Next, the role of common orientation shared by the robots is examined. Necessary and sufficient conditions for the partitioning problem to be solvable are given in the different timing models. Finally, the problem is proved to be unsolvable in the no-compass synchronous model.     

Towards principled experimental study of autonomous mobile robots

We review the current state of research in autonomous mobile robots and conclude that there is an inadequate basis for predicting the reliability and behavior of robots operating in unengineered environments. We present a new approach to the study of autonomous mobile robot performance based on formal statistical analysis of independently reproducible experiments conducted on real robots. Simulators serve as models rather than experimental surrogates. We demonstrate three new results: 1) Two commonly used performance metrics (time and distance) are not as well correlated as is often tacitly assumed. 2) The probability distributions of these performance metrics are exponential rather than normal, and 3) a modular, object-oriented simulation accurately predicts the behavior of the real robot in a statistically significant manner.     

A method for combining odometry and distance sensor information for effective obstacle avoidance of autonomous mobile robots

In this paper, a concept for virtual sensors is proposed for efficient avoidance of obstacles during the motion of robots. The virtual sensor yields new data by combining encoder values and real distance data, and derives new sensor data that includes the mobility of the robot. Simulation on Windows XP is executed to illustrate the proposed approach with actually acquired distance from virtual and actual sensors. To facilitate comparison with the alternative results developed in this paper, we refer to the conventional artificial potential field method using actual distance. Data from virtual sensors show smoother and safer motion in obstacle avoidance traces in regards to obstacle and robot mobility.     

Moving formation convergence of a group of mobile robots via decentralised information feedback

In this article, a decentralised information feedback mechanism is introduced to a group of mobile robots such that the robots asymptotically converge to a given moving formation. It is assumed that the robots can exchange only position information according to a pre-specified communication graph. Each node represents a robot. Two robots are neighbours of each other if there is an edge between the two nodes. A feedback controller is performed for each robot by only using its own velocity information and the position information from its neighbours. It is proven that if the graph is connected, then the convergence to the moving formation of the closed-loop system is guaranteed. Several numerical simulations are presented to illustrate the results.