Cyclic error correction based Q-learning for mobile robots navigation

International Journal of Control, Automation and Systems - Similar to control systems, reinforcement learning can capture notions of optimal behavior using natural interaction experience. In the...     

Incremental Q-learning strategy for adaptive PID control of mobile robots

Expert and intelligent systems are being developed to control many technological systems including mobile robots. However, the PID (Proportional-Integral-Derivative) controller is a fast low-level control strategy widely used in many control engineering tasks. Classic control theory has contributed with different tuning methods to obtain the gains of PID controllers for specific operation conditions. Nevertheless, when the system is not fully known and the operative conditions are variable and not previously known, classical techniques are not entirely suitable for the PID tuning. To overcome these drawbacks many adaptive approaches have been arisen, mainly from the field of artificial intelligent. In this work, we propose an incremental Q-learning strategy for adaptive PID control. In order to improve the learning efficiency we define a temporal memory into the learning process. While the memory remains invariant, a non-uniform specialization process is carried out generating new limited subspaces of learning. An implementation on a real mobile robot demonstrates the applicability of the proposed approach for a real-time simultaneous tuning of multiples adaptive PID controllers for a real system operating under variable conditions in a real environment.     

An approach to mobile software robots for the WWW

This paper describes a framework for developing mobile software robots by using the PLANET mobile object system, which is characterized by a language-neutral layered architecture, the native code execution of mobile objects, and asynchronous object passing. We propose an approach to implementing mobile Web search robots that takes full advantage of these characteristics, and we base our discussion of its effectiveness on experiments conducted in the Internet environment. The results show that the PLANET approach to mobile Web search robots significantly reduces the amount of data transferred via the Internet and that it enables the robots to work more efficiently than the robots in the conventional stationary scheme whenever nontrivial amounts of HTML files are processed     

Reliable and efficient landmark-based localization for mobile robots

This paper describes an efficient and robust localization system for indoor mobile robots and AGVs. The system utilizes a sensor that measures bearings to artificial landmarks, and an efficient triangulation method. We present a calibration method for the system components and overcome typical problems for sensors of the mentioned type, which are localization in motion and incorrect identification of landmarks. The resulting localization system was tested on a mobile robot. It consumes less than 4% of a Pentium4 3.2 GHz processing power while providing an accurate and reliable localization result every 0.5 s. The system was successfully incorporated within a real mobile robot system which performs many other computational tasks in parallel.     

An appearance-based visual compass for mobile robots

Localization is one of the most important basic skills of a mobile robot. Most approaches, however, still rely either on special sensors or require artificial environments. In this article, a novel approach is presented that can provide compass information for localization, purely based on the visual appearance of a room. A robot using such a visual compass can quickly learn a cylindrical map of the environment, consisting of simple statistical features that can be computed very quickly. The visual compass algorithm is efficient, scalable and can therefore be used in real-time on almost any contemporary robotic platform. Extensive experiments on a Sony Aibo robot have validated that the approach works in a vast variety of environments.     

A new multisensor fusion SLAM approach for mobile robots

This paper presents a novel method, which enhances the use of external mechanisms by considering a multisensor system, composed of sonars and a CCD camera. Monocular vision provides redundant information about the location of the geometric entities detected by the sonar sensors. To reduce ambiguity significantly, an improved and more detailed sonar model is utilized. Moreover, Hough transform is used to extract features from raw sonar data and vision image. Information is fused at the level of features. This technique significantly improves the reliability and precision of the environment observations used for the simultaneous localization and map building problem for mobile robots. Experimental results validate the favorable performance of this approach.     

基于置信度上界的移动机器人信息路径规划方法

路径规划是移动机器人未知环境探索的关键问题,路径点的合理规划对提高环境探索的效率和环境场预测的准确性至关重要.基于强化学习范式,提出一种适用于静态环境场探索的移动机器人在线信息路径规划方法.针对基于模型训练算法计算成本高的问题,通过机器人与环境的交互作用,采用动作价值评估的方法来学习所获取的环境场历史信息,提高机器人实时规划能力.为了提高环境预测准确性,引入基于置信度上界的动作选择方法来平衡探索未知区域与利用已有信息,鼓励机器人向更多未知区域进行全场特征探索,同时避免因探索区域有限而陷入局部极值.仿真实验中,环境场分别采用高斯分布和Ackley函数模型.结果表明,所提算法能够实现机器人环境探索路径点的在线决策,准确有效地捕捉全场和局部环境特征.     

Enhanced continuous valued Q-learning for real autonomous robots

A parallel-jaw gripper is a very useful tool for robot manipulation tasks due to its simple mechanism and control. This fact limits the range of successful grasps it can undergo, and also makes it unfeasible under uncertainties. Thus, it is desirable to improve its dexterity and manipulability. In this paper, we propose a new design of a two-fingered parallel gripper that utilizes rolling at the contacts for object repositioning and reorientation, aimed at effective firm grasps. We name it the scrollic gripper, an acronym for synchronously closing with rolling constraints. At first, the background to utilize the rolling constraints is described. Then, grasping and manipulation of the gripper are discussed. In grasp acquisition, we propose a quality function for evaluating grasp stability. The sophisticated hardware and functioning for the scrollic gripper consist, basically, on implementation of an additional degree-of-freedom to the conventional parallel-jaw gripper, leading to grasp acquisition and secure grasping.     

A hybrid control approach to action coordination for mobile robots

In this paper, the problem concerning how to coordinate the contributions from concurrent controllers, when controlling mobile robots, is investigated. It is shown how a behavior based control system for autonomous robots can be modeled as a hybrid automaton, where each node corresponds to a distinct robot behavior. This type of construction gives rise to chattering executions, but it is shown how regularized automata can be used to solve this problem. As an illustration, the obstacle-negotiation problem is solved by using a combination of a robust path-following behavior and a reactive obstacle-avoidance behavior that move the robot around a given obstacle at a predefined safety distance.     

Cognitive maps for mobile robots—an object based approach

Robots are rapidly evolving from factory work-horses to robot-companions. The future of robots, as our companions, is highly dependent on their abilities to understand, interpret and represent the environment in an efficient and consistent fashion, in a way that is comprehensible to humans. The work presented here is oriented in this direction. It suggests a hierarchical probabilistic representation of space that is based on objects. A global topological representation of places with object graphs serving as local maps is proposed. The work also details the first efforts towards conceptualizing space on the basis of the human compatible representation so formed. Such a representation and the resulting conceptualization would be useful for enabling robots to be cognizant of their surroundings. Experiments on place classification and place recognition are reported in order to demonstrate the applicability of such a representation towards understanding space and thereby performing spatial cognition. Further, relevant results from user studies validating the proposed representation are also reported. Thus, the theme of the work is — representation for spatial cognition.     

A decentralized approach for the conflict-free motion of multiple mobile robots

This article presents a novel approach to decentralized motion planning and conflict-resolution for multiple mobile robots. The proposed multi-robot motion planning is an on-line operation, based on cost wave propagation within a discretized configuration space-time. By use of the planning method, a framework for negotiation is developed, which permits quick decentralized and parallel decision making. The key objective of the negotiation procedure is dynamic assignment of robot motion priorities. Thus, robots involved in a local conflict situation cooperate in planning and execution of the lowest cost motion paths without application of any centralized components. The features required for individual and cooperative motion are embedded in a hybrid control architecture. Results obtained from realistic simulation of a multi-robot environment and also from experiments performed with two mobile robots demonstrate the flexibility and the efficiency of the proposed method.     

An effective posture stabilizer for differential drive mobile robots

A continuous time-invariant feedback controller for differential drive mobile robots is proposed. Its main contribution is to stabilize a differential drive mobile robot to a desired posture through only smooth trajectory without inversion of the drive direction. Experimental results are reported to show the validity of the controller.     

Adaptive task assignment for multiple mobile robots via swarm intelligence approach

This paper describes an adaptive task assignment method for a team of fully distributed mobile robots with initially identical functionalities in unknown task environments. A hierarchical assignment architecture is established for each individual robot. In the higher hierarchy, we employ a simple self-reinforcement learning model inspired by the behavior of social insects to differentiate the initially identical robots into “specialists” of different task types, resulting in stable and flexible division of labor; on the other hand, in dealing with the cooperation problem of the robots engaged in the same type of task, Ant System algorithm is adopted to organize low-level task assignment. To avoid using a centralized component, a “local blackboard” communication mechanism is utilized for knowledge sharing. The proposed method allows the robot team members to adapt themselves to the unknown dynamic environments, respond flexibly to the environmental perturbations and robustly to the modifications in the team arising from mechanical failure. The effectiveness of the presented method is validated in two different task domains: a cooperative concurrent foraging task and a cooperative collection task.     

An uncertainty compensator for robust control of wheeled mobile robots

This paper proposes an uncertainty compensator to design a novel robust control for mobile robots with dynamic and kinematic uncertainties. A novel gradient-based adaptive fuzzy estimator is developed to compensate uncertainties with minimum required feedback signals. As a novelty, the proposed approach uses the tracking error and its first time derivative to form the estimation error of uncertainty, and guarantees that both the estimation error and tracking error converge asymmetrically to ignorable value. Advantages of the proposed robust control are simplicity in design, robustness against uncertainties, guaranteed stability, and good control performance. The control approach is verified by stability analysis. Simulation results and experimental results illustrate the effectiveness of the proposed control. Experimental evaluation of the proposed controller is expressed for two different low-cost nonholonomic wheeled mobile robots. The proposed control design is compared with an adaptive control approach to confirm the superiority of the proposed approach in terms of precision, simplicity of design, and computations.     

Practically efficient array initialization

Initialization of an array, out of which only a small initially unknown portion will eventually be used, is a frequent need in programming. A folklore solution for initializing an array of n entries in constant time uses 2n?log2n? extra bits to realize a stack of back pointers to the actually used entries of the array. Navarro has given a succinct version of this technique, which requires only n + o(n) bits of auxiliary storage. We describe, analyze, and experimentally compare these solutions and their space‐efficient but theoretically suboptimal alternatives based on a simple bitmap for keeping track of the array entries which have been assigned a value. Experimental results suggest that each of the methods has its niche of excellence, which are roughly as follows: the theoretically optimal solutions based on a stack of back pointers perform in general best on sparse arrays, whose access frequency is less than 1% of the number of their entries. Brute‐force initialization of the entire array seems generally to give the best overall performance for dense arrays whose access frequency is over 10% of their size. For the remaining cases of arrays with 1–10% access frequency, the methods which use a simple bitmap appear to give the best performance. The experiments show that the choice of a suitable implementation may yield substantial, up to hundreds of times speed‐ups in the performance of initializable array operations. Copyright © 2015 John Wiley & Sons Ltd.     

Local communication of multiple mobile robots: design of group behavior for efficient communication

A novel design method of robot behavior is discussed to realize efficient local communication for cooperation of multiple mobile robots. Local communication is now increasingly utilized in cooperative many-robot systems because of its advantages of load distribution and simple implementation. In its usage, the design of each robot's behavior is a very important issue since it has a significant effect upon the communication efficiency in a collective manner. In this study, we introduce a simple group behavior and analyze how it improves the performance of local communication among many mobile robots. The performance is evaluated using the information transmission time that plays a crucial part in effective cooperation. Next, the optimal group size is analytically derived by minimizing the transmission time. The effectiveness of the analytical design method is verified by computer simulations of many-robot communication.     

Switching control approach for stable navigation of mobile robots in unknown environments

This paper presents a stable switching control strategy for the parking problem of non-holonomic mobile robots. First, it is proposed a positioning-orientation switching controller for the parking problem. With this strategy robot backwards motions are avoided and the robot heading is always in the direction of the goal point facilitating the obstacle handling. Second, the avoidance of unexpected obstacles is considered in a reactive way by following the contour of the obstacles. Next, the stability of the switching parking/obstacle-avoider controller is analyzed showing stability under reasonable conditions. Finally, the good performance and the feasibility of this approach are shown through several experimental results.     

A hybrid approach for autonomous navigation of mobile robots in partially-known environments

We propose a hybrid approach specifically adapted to deal with the autonomous-navigation problem of a mobile robot which is subjected to perform an emergency task in a partially-known environment. Such a navigation problem requires a method that is able to yield a fast execution time, under constraints on the capacity of the robot and on known/unknown obstacles, and that is sufficiently flexible to deal with errors in the known parts of the environment (unexpected obstacles). Our proposal includes an off-line task-independent preprocessing phase, which is applied just once for a given robot in a given environment. Its purpose is to build, within the known zones, a roadmap of near-time-optimal reference trajectories. The actual execution of the task is an online process that combines reactive navigation with trajectory tracking and that includes smooth transitions between these two modes of navigation. Controllers used are fuzzy-inference systems. Both simulation and experimental results are presented to test the performance of the proposed hybrid approach. Obtained results demonstrate the ability of our proposal to handle unexpected obstacles and to accomplish navigation tasks in relatively complex environments. The results also show that, thanks to its time-optimal-trajectory planning, our proposal is well adapted to emergency tasks as it is able to achieve shorter execution times, compared to other waypoint-navigation methods that rely on optimal-path planning.     

An information-based exploration strategy for environment mapping with mobile robots

The availability of efficient mapping systems to produce accurate representations of initially unknown environments is recognized as one of the main requirements for autonomous mobile robots. In this paper, we present an efficient mapping system that has been implemented on a mobile robot equipped with a laser range scanner. The system builds geometrical point-based maps of environments employing an information-based exploration strategy that determines the best observation positions by taking into account both the distance travelled and the information gathered. Our exploration strategy, being based on solid mathematical foundations, differs from many ad hoc exploration strategies proposed in literature. We present: (a) the theoretical aspects of the criterion for determining the best observation positions for a robot building a map, (b) the implementation of a mapping system that uses the proposed criterion, and (c) the experimental validation of our approach.