Neural networks based reinforcement learning for mobile robots obstacle avoidance

This study proposes a new approach for solving the problem of autonomous movement of robots in environments that contain both static and dynamic obstacles. The purpose of this research is to provide mobile robots a collision-free trajectory within an uncertain workspace which contains both stationary and moving entities. The developed solution uses Q-learning and a neural network planner to solve path planning problems. The algorithm presented proves to be effective in navigation scenarios where global information is available. The speed of the robot can be set prior to the computation of the trajectory, which provides a great advantage in time-constrained applications. The solution is deployed in both Virtual Reality (VR) for easier visualization and safer testing activities, and on a real mobile robot for experimental validation. The algorithm is compared with Powerbot's ARNL proprietary navigation algorithm. Results show that the proposed solution has a good conversion rate computed at a satisfying speed.     

基于BP神经网络的移动机器人避障设计及仿真

为了解决移动机器人在复杂环境中如何高效精确地躲避障碍物的问题,提出了一种基于BP神经网络的避障方法。建立了机器人的避障运动模型并设计了神经网络避障控制系统;分析了机器人在运动过程中与障碍物的位置关系,使用超声波传感器采集距离信息,进行BP神经网络输入、输出训练并采用Matlab工具进行仿真试验。结果表明,该方法可以高效精确地实现移动机器人的自主避障,运行相对稳定、轨迹连续平滑,达到了较为理想的避障效果。验证了方法的可行性和有效性,为移动机器人自主避障提供了一种新的控制方法。     

Artificial neural network based robot control: An overview

The current thrust of research in robotics is to build robots which can operate in dynamic and/or partially known environments. The ability of learning endows the robot with a form of autonomous intelligence to handle such situations. This paper focuses on the intersection of the fields of robot control and learning methods as represented by artificial neural networks. An in-depth overview of the application of neural networks to the problem of robot control is presented. Some typical neural network architectures are discussed first. The important issues involved in the study of robotics are then highlighted. This paper concentrates on the neural network applications to the motion control of robots involved in both non-contact and contact tasks. The current state of research in this area is surveyed and the strengths and weakness of the present approaches are emphasized. The paper concludes by indentifying areas which need future research work.     

On actively closing loops in grid-based FastSLAM

Acquiring models of the environment belongs to the fundamental tasks of mobile robots. In the past, several researchers have focused on the problem of simultaneous localization and mapping (SLAM). Classical SLAM approaches are passive in the sense that they only process the perceived sensor data and do not influence the motion of the mobile robot. In this paper, we present a novel integrated approach that combines autonomous exploration with simultaneous localization and mapping. Our method uses a grid-based version of the FastSLAM algorithm and considers at each point in time actions to actively close loops during exploration. By re-entering already visited areas, the robot reduces its localization error and in this way learns more accurate maps. Experimental results presented in this paper illustrate the advantage of our method over previous approaches that lack the ability to actively close loops.     

Development of A Behavior‐Based Cooperative Search Strategy for Distributed Autonomous Mobile Robots Using Zigbee Wireless Sensor Network

To achieve efficient and objective search tasks in an unknown environment, a cooperative search strategy for distributed autonomous mobile robots is developed using a behavior‐based control framework with individual and group behaviors. The sensing information of each mobile robot activates the individual behaviors to facilitate autonomous search tasks to avoid obstacles. An 802.15.4 ZigBee wireless sensor network then activates the group behaviors that enable cooperative search among the mobile robots. An unknown environment is dynamically divided into several sub‐areas according to the locations and sensing data of the autonomous mobile robots. The group behaviors then enable the distributed autonomous mobile robots to scatter and move in the search environment. The developed cooperative search strategy successfully reduces the search time within the test environments by 22.67% (simulation results) and 31.15% (experimental results).     

Evolutionary robotics: the Sussex approach

We give an overview of evolutionary robotics research at Sussex over the last five years. We explain and justify our distinctive approaches to (artificial) evolution, and to the nature of robot control systems that are evolved. Results are presented from research with evolved controllers for autonomous mobile robots, simulated robots, co-evolved animats, real robots with software controllers, and a real robot with a controller directly evolved in hardware.     

A geometric approach to deploying robot swarms

We discuss the fundamental problems and practical issues underlying the deployment of a swarm of autonomous mobile robots that can potentially be used to build mobile robotic sensor networks. For the purpose, a geometric approach is proposed that allows robots to configure themselves into a two-dimensional plane with uniform spatial density. Particular emphasis is paid to the hole repair capability for dynamic network reconfiguration. Specifically, each robot interacts selectively with two neighboring robots so that three robots can converge onto each vertex of the equilateral triangle configuration. Based on the local interaction, the self-configuration algorithm is presented to enable a swarm of robots to form a communication network arranged in equilateral triangular lattices by shuffling the neighbors. Convergence of the algorithms is mathematically proved using Lyapunov theory. Moreover, it is verified that the self-reparation algorithm enables robot swarms to reconfigure themselves when holes exist in the network or new robots are added to the network. Through extensive simulations, we validate the feasibility of applying the proposed algorithms to self-configuring a network of mobile robotic sensors. We describe in detail the features of the algorithm, including self-organization, self-stabilization, and robustness, with the results of the simulation.     

Region filling operations with random obstacle avoidance for mobile robots

The article presents a new topic in path planning for mobile robots, region filling. which involves a sweeping operation to fill a whole region with random obstacle avoidance. The approaches for global strip filling and local path searching driven by sensory data procedures are developed. A computer graphic simulation is used to verify the filling strategy available. The research was developed from the program for the design of a robot lawn mower. However, the solution appears generic. The significance is that a problem of wide application and generic solutions for general autonomous mobile robots have been developed.     

A neural-network committee machine approach to the inverse kinematics problem solution of robotic manipulators

In robotics, inverse kinematics problem solution is a fundamental problem in robotics. Many traditional inverse kinematics problem solutions, such as the geometric, iterative, and algebraic approaches, are inadequate for redundant robots. Recently, much attention has been focused on a neural-network-based inverse kinematics problem solution in robotics. However, the result obtained from the neural network requires to be improved for some sensitive tasks. In this paper, a neural-network committee machine (NNCM) was designed to solve the inverse kinematics of a 6-DOF redundant robotic manipulator to improve the precision of the solution. Ten neural networks (NN) were designed to obtain a committee machine to solve the inverse kinematics problem using separately prepared data set since a neural network can give better result than other ones. The data sets for the neural-network training were prepared using prepared simulation software including robot kinematics model. The solution of each neural network was evaluated using direct kinematics equation of the robot to select the best one. As a result, the committee machine implementation increased the performance of the learning.     

Maze exploration behaviors using an integrated evolutionary robotics environment

This paper presents results generated with a new evolutionary robotics (ER) simulation environment and its complementary real mobile robot colony research test-bed. Neural controllers producing mobile robot maze searching and exploration behaviors using binary tactile sensors as inputs were evolved in a simulated environment and subsequently transferred to and tested on real robots in a physical environment. There has been a considerable amount of proof-of-concept and demonstration research done in the field of ER control in recent years, most of which has focused on elementary behaviors such as object avoidance and homing. Artificial neural networks (ANN) are the most commonly used evolvable controller paradigm found in current ER literature. Much of the research reported to date has been restricted to the implementation of very simple behaviors using small ANN controllers. In order to move beyond the proof-of-concept stage our ER research was designed to train larger more complicated ANN controllers, and to implement those controllers on real robots quickly and efficiently. To achieve this a physical robot test-bed that includes a colony of eight real robots with advanced computing and communication abilities was designed and built. The real robot platform has been coupled to a simulation environment that facilitates the direct wireless transfer of evolved neural controllers from simulation to real robots (and vice versa). We believe that it is the simultaneous development of ER computing systems in both the simulated and the physical worlds that will produce advances in mobile robot colony research. Our simulation and training environment development focuses on the definition and training of our new class of ANNs, networks that include multiple hidden layers, and time-delayed and recurrent connections. Our physical mobile robot design focuses on maximizing computing and communications power while minimizing robot size, weight, and energy usage. The simulation and ANN-evolution environment was developed using MATLAB. To allow for efficient control software portability our physical evolutionary robots (EvBots) are equipped with a PC-104-based computer running a custom distribution of Linux and connected to the Internet via a wireless network connection. In addition to other high-level computing applications, the mobile robots run a condensed version of MATLAB, enabling ANN controllers evolved in simulation to be transferred directly onto physical robots without any alteration to the code. This is the first paper in a series to be published cataloging our results in this field.     

Real-time exploration of a multi-robot rescue system in disaster areas

A system procedure is proposed for a multi-robot rescue system that performs real-time exploration over disaster areas. Real-time exploration means that every robot exploring the area always has a communication path to human operators standing by at a base station and that the communication path is configured by ad hoc wireless networking. Real-time exploration is essential in multi-robot systems for USAR (urban search and rescue) because operators must communicate with every robot to support the victim detection process and ad hoc networking is suitable to configure a communication path among obstacles. The proposed system procedure consists of the autonomous classification of robots into search and relay types and behavior algorithms for each class of robot. Search robots explore the areas and relay robots act as relay terminals between search robots and the base station. The rule of the classification and the behavior algorithm refer to the forwarding table of each robot constructed for ad hoc networking. The table construction is based on DSDV (destination-sequenced distance vector) routing that informs each robot of its topological position in the network and other essentials. Computer simulations are executed with a specific exploration strategy of search robots. The results show that a multi-robot rescue system can perform real-time exploration with the proposed system procedure and reduce exploration time in comparison with the case where the proposed scheme is not adopted.     

高精度激光测距下的机器人自主避障控制

在机器人自主避障过程中,由于传感器数据的误差会降低机器人感知和决策的准确性,从而影响机器人自主避障能力。为此,提出高精度激光测距下的机器人自主避障控制方法。通过设计机器人体系结构,建立机器人运动学模型,为机器人避障控制提供依据。采用高精度激光测距技术,构建机器人移动场地地形。通过自适应阈值方法,完成机器人的自主避障控制。实验结果表明,所提方法的机器人自主避障控制效果好,且障碍物位置测试值与实际位置值的误差保持在0.5m以内,具有较高的避障控制精确度。     

基于轨迹引导的移动机器人导航策略优化算法

针对在杂乱、障碍物密集的复杂环境下移动机器人使用深度强化学习进行自主导航所面临的探索困难,进而导致学习效率低下的问题,提出了一种基于轨迹引导的导航策略优化（TGNPO）算法。首先,使用模仿学习的方法为移动机器人训练一个能够同时提供专家示范行为与导航轨迹预测功能的专家策略,旨在全面指导深度强化学习训练;其次,将专家策略预测的导航轨迹与当前时刻移动机器人所感知的实时图像进行融合,并结合坐标注意力机制提取对移动机器人未来导航起引导作用的特征区域,提高导航模型的学习性能;最后,使用专家策略预测的导航轨迹对移动机器人的策略轨迹进行约束,降低导航过程中的无效探索和错误决策。通过在仿真和物理平台上部署所提算法,实验结果表明,相较于现有的先进方法,所提算法在导航的学习效率和轨迹平滑方面取得了显著的优势。这充分证明了该算法能够高效、安全地执行机器人导航任务。     

Navigating mobile robots with a modular neural architecture

Neural architectures have been proposed to navigate mobile robots within several environment definitions. In this paper a new neural modular constructive approach to navigate mobile robots in unknown environments is presented. The problem, in its basic form, consists of defining and executing a trajectory to a pre-defined goal while avoiding all obstacles, in an unknown environment. Some crucial issues arise when trying to solve this problem, such as an overflow of sensorial information and conflicting objectives. Most neural network (NN) approaches to this problem focus on a monolithic system, i.e., a system with only one neural network that receives and analyses all available information, resulting in conflicting training patterns, long training times and poor generalisation. The work presented in this article circumvents these problems by the use of a constructive modular NN. Navigation capabilities were proven with the NOMAD 200 mobile robot.     

Efficient terrain coverage for deploying wireless sensor nodes on multi-robot system

Coverage and connectivity are the two main functionalities of wireless sensor network. Stochastic node deployment or random deployment almost always cause hole in sensing coverage and cause redundant nodes in area. In the other hand precise deployment of nodes in large area is very time consuming and even impossible in hazardous environment. One of solution for this problem is using mobile robots with concern on exploration algorithm for mobile robot. In this work an autonomous deployment method for wireless sensor nodes is proposed via multi-robot system which robots are considered as node carrier. Developing an exploration algorithm based on spanning tree is the main contribution and this exploration algorithm is performing fast localization of sensor nodes in energy efficient manner. Employing multi-robot system and path planning with spanning tree algorithm is a strategy for speeding up sensor nodes deployment. A novel improvement of this technique in deployment of nodes is having obstacle avoidance mechanism without concern on shape and size of obstacle. The results show using spanning tree exploration along with multi-robot system helps to have fast deployment behind efficiency in energy.     

基于RGB-D信息的移动机器人SLAM和路径规划方法研究与实现

移动机器人在各种辅助任务中需具备自主定位、建图、路径规划与运动控制的能力。本文利用RGB-D信息和ORB-SLAM算法进行自主定位,结合点云数据和GMapping算法建立环境栅格地图,基于二次规划方法进行平滑可解析的路径规划,并设计非线性控制器,实现了由一个运动底盘、一个RGB-D传感器和一个运算平台组成的自主移动机器人系统。经实验验证,这一系统实现了复杂室内环境下的实时定位与建图、自主移动和障碍物规避。由此,为移动机器人的推广应用提供了一个硬件结构简单、性能良好、易扩展、经济性好、开发维护方便的解决方案。     

Three‐dimensional map building for mobile robot navigation environments using a self‐organizing neural network

In recent years, mobile robots have been required to become more and more autonomous in such a way that they are able to sense and recognize the three‐dimensional space in which they live or work. In this paper, we deal with such an environment map building problem from three‐dimensional sensing data for mobile robot navigation. In particular, the problem to be dealt with is how to extract and model obstacles which are not represented on the map but exist in the real environment, so that the map can be newly updated using the modeled obstacle information. To achieve this, we propose a three‐dimensional map building method, which is based on a self‐organizing neural network technique called “growing neural gas network.” Using the obstacle data acquired from the 3D data acquisition process of an active laser range finder, learning of the neural network is performed to generate a graphical structure that reflects the topology of the input space. For evaluation of the proposed method, a series of simulations and experiments are performed to build 3D maps of some given environments surrounding the robot. The usefulness and robustness of the proposed method are investigated and discussed in detail. © 2004 Wiley Periodicals, Inc.     

Reactive navigation in dynamic environment using a multisensorpredictor

A reactive navigation system for an autonomous mobile robot in unstructured dynamic environments is presented. The motion of moving obstacles is estimated for robot motion planning and obstacle avoidance. A multisensor-based obstacle predictor is utilized to obtain obstacle-motion information. Sensory data from a CCD camera and multiple ultrasonic range finders are combined to predict obstacle positions at the next sampling instant. A neural network, which is trained off-line, provides the desired prediction on-line in real time. The predicted obstacle configuration is employed by the proposed virtual force based navigation method to prevent collision with moving obstacles. Simulation results are presented to verify the effectiveness of the proposed navigation system in an environment with multiple mobile robots or moving objects. This system was implemented and tested on an experimental mobile robot at our laboratory. Navigation results in real environment are presented and analyzed.     

Tracking control of a nonholonomic mobile robot using compound cosine function neural networks

The purpose of this paper is to propose a compound cosine function neural network with continuous learning algorithm for the velocity and orientation angle tracking control of a nonholonomic mobile robot with nonlinear disturbances. Herein, two neural network (NN) controllers embedded in the closed-loop control system have the simple continuous learning and rapid convergence capability without the dynamics information of the mobile robot to realize the adaptive control of the mobile robot. The neuron function of the hidden layer in the three-layer feed-forward network structure is on the basis of combining a cosine function with a unipolar sigmoid function. The developed neural network controllers have simple algorithm and fast learning convergence because the weight values are only adjusted between the nodes in hidden layer and the output nodes, while the weight values between the input layer and the hidden layer are one, i.e. constant, without the weight adjustment. Therefore, the main advantages of this control system are the real-time control capability and the robustness by use of the proposed neural network controllers for a nonholonomic mobile robot with nonlinear disturbances. Through simulation experiments applied to the nonholonomic mobile robot with the nonlinear disturbances which are considered as dynamics uncertainty and external disturbances, the simulation results show that the proposed NN control system of nonholonomic mobile robots has real-time control capability, better robustness and higher control precision. The compound cosine function neural network provides us with a new way to solve tracking control problems for mobile robots.     

Bayesian Landmark Learning for Mobile Robot Localization

To operate successfully in indoor environments, mobile robots must be able to localize themselves. Most current localization algorithms lack flexibility, autonomy, and often optimality, since they rely on a human to determine what aspects of the sensor data to use in localization (e.g., what landmarks to use). This paper describes a learning algorithm, called BaLL, that enables mobile robots to learn what features/landmarks are best suited for localization, and also to train artificial neural networks for extracting them from the sensor data. A rigorous Bayesian analysis of probabilistic localization is presented, which produces a rational argument for evaluating features, for selecting them optimally, and for training the networks that approximate the optimal solution. In a systematic experimental study, BaLL outperforms two other recent approaches to mobile robot localization.