The visual homing problem: An example of robotics/biology cross fertilization

In this paper, we describe how a mobile robot under simple visual control can retrieve a particular goal location in an open environment. Our model neither needs a precise map nor to learn all the possible positions in the environment. The system is a neural architecture inspired by neurobiological analysis of how visual patterns named landmarks are recognized. The robot merges these visual informations and their azimuth to build a plastic representation of its location. This representation is used to learn the best movement to reach the goal. A simple and fast on-line learning of a few places located near the goal allows this goal to be reached from anywhere in its neighborhood. The system uses only a very rough representation of the robot environment and presents very high generalization capabilities. We describe an efficient implementation of autonomous and motivated navigation tested on our robot in real indoor environments. We show the limitations of the model and its possible extensions.     

Auditory learning: a developmental method

Motivated by the human autonomous development process from infancy to adulthood, we have built a robot that develops its cognitive and behavioral skills through real-time interactions with the environment. We call such a robot a developmental robot. In this paper, we present the theory and the architecture to implement a developmental robot and discuss the related techniques that address an array of challenging technical issues. As an application, experimental results on a real robot, self-organizing, autonomous, incremental learner (SAIL), are presented with emphasis on its audition perception and audition-related action generation. In particular, the SAIL robot conducts the auditory learning from unsegmented and unlabeled speech streams without any prior knowledge about the auditory signals, such as the designated language or the phoneme models. Neither available before learning starts are the actions that the robot is expected to perform. SAIL learns the auditory commands and the desired actions from physical contacts with the environment including the trainers.     

Autonomous knowledge acquisition based on artificial curiosity: Application to mobile robots in an indoor environment

This paper describes an autonomous system for knowledge acquisition based on artificial curiosity. The proposed approach allows a humanoid robot to discover, in an indoor environment, the world in which it evolves, and to learn autonomously new knowledge about it. The learning process is accomplished by observation and by interaction with a human tutor, based on a cognitive architecture with two levels. Experimental results of deployment of this system on a humanoid robot in a real office environment are provided. We show that our cognitive system allows a humanoid robot to gain increased autonomy in matters of knowledge acquisition.     

Modelling Shared Attention Through Relational Reinforcement Learning

Shared attention is a type of communication very important among human beings. It is sometimes reserved for the more complex form of communication being constituted by a sequence of four steps: mutual gaze, gaze following, imperative pointing and declarative pointing. Some approaches have been proposed in Human?Robot Interaction area to solve part of shared attention process, that is, the most of works proposed try to solve the first two steps. Models based on temporal difference, neural networks, probabilistic and reinforcement learning are methods used in several works. In this article, we are presenting a robotic architecture that provides a robot or agent, the capacity of learning mutual gaze, gaze following and declarative pointing using a robotic head interacting with a caregiver. Three learning methods have been incorporated to this architecture and a comparison of their performance has been done to find the most adequate to be used in real experiment. The learning capabilities of this architecture have been analyzed by observing the robot interacting with the human in a controlled environment. The experimental results show that the robotic head is able to produce appropriate behavior and to learn from sociable interaction.     

Adaptation technique for integrating genetic programming and reinforcement learning for real robots

We propose an integrated technique of genetic programming (GP) and reinforcement learning (RL) to enable a real robot to adapt its actions to a real environment. Our technique does not require a precise simulator because learning is achieved through the real robot. In addition, our technique makes it possible for real robots to learn effective actions. Based on this proposed technique, we acquire common programs, using GP, which are applicable to various types of robots. Through this acquired program, we execute RL in a real robot. With our method, the robot can adapt to its own operational characteristics and learn effective actions. In this paper, we show experimental results from two different robots: a four-legged robot "AIBO" and a humanoid robot "HOAP-1." We present results showing that both effectively solved the box-moving task; the end result demonstrates that our proposed technique performs better than the traditional Q-learning method.     

Learning sensor-based navigation of a real mobile robot in unknownworlds

In this paper, we address the problem of navigating an autonomous mobile robot in an unknown indoor environment. The parti-game multiresolution learning approach is applied for simultaneous and cooperative construction of a world model, and learning to navigate through an obstacle-free path from a starting position to a known goal region. The paper introduces a new approach, based on the application of the fuzzy ART neural architecture, for on-line map building from actual sensor data. This method is then integrated, as a complement, on the parti-game world model, allowing the system to make a more efficient use of collected sensor information. Then, a predictive on-line trajectory filtering method, is introduced in the learning approach. Instead of having a mechanical device moving to search the world, the idea is to have the system analyzing trajectories in a predictive mode, by taking advantage of the improved world model. The real robot will only move to try trajectories that have been predicted to be successful, allowing lower exploration costs. This results in an overall improved new method for goal-oriented navigation. It is assumed that the robot knows its own current world location-a simple dead-reckoning method is used for localization in our experiments. It is also assumed that the robot is able to perform sensor-based obstacle detection (not avoidance) and straight-line motions. Results of experiments with a real Nomad 200 mobile robot are presented, demonstrating the effectiveness of the discussed methods.     

Behavior-based intelligent mobile robot using an immunized reinforcement adaptive learning mechanism

In this paper, a novel immunized reinforcement adaptive learning mechanism employing a behavior-based knowledge and the on-line adapting capabilities of the immune system is proposed and applied to an intelligent mobile robot. Rather than building a detailed mathematical model of immune systems, we try to explore principles in the immune system focusing on its self-organization, adaptive capability and immune memory. Two levels of the immune system, underlying the ‘micro’ level of cell interactions, and emergent ‘macro’ level of the behavior of the system are investigated.To evaluate the proposed immunized architecture, a ‘food foraging work’ simulation environment containing a mobile robot, foods, with/without obstacles is created to simulate the real world. The simulation results validate several significant characteristics of the immunized architecture: adaptability, learning, self-organizing, and stable ecological niche approaching.     

Internet Control Architecture for Internet-Based Personal Robot

This paper proposes a novel direct internet control architecture for internet-based personal robot, which is insensitive to the inherent internet time delay. The personal robot can be controlled using a simulator provided at a local site. Since the internet time delay is affected by the number of nodes and the internet loads, it is variable and unpredictable so that a large internet delay makes some control inputs distorted. The proposed control architecture guarantees that the personal robot can avoid obstacles and reduce the path error and the time difference between a virtual robot at the local site and a real robot at the remote site. This architecture is extended for an uncertain environment. Simulations and experimental results in the real internet environment demonstrate the effectiveness and applicability of the proposed internet control architecture.     

Interleaving Planning and Robot Execution for Asynchronous User Requests

ROGUE is an architecture built on a real robot which provides algorithms for the integration of high-level planning, low-level robotic execution, and learning. ROGUE addresses successfully several of the challenges of a dynamic office gopher environment. This article presents the techniques for the integration of planning and execution.ROGUE uses and extends a classical planning algorithm to create plans for multiple interacting goals introduced by asynchronous user requests. ROGUE translates the planner';s actions to robot execution actions and monitors real world execution. ROGUE is currently implemented using the PRODIGY4.0 planner and the Xavier robot. This article describes how plans are created for multiple asynchronous goals, and how task priority and compatibility information are used to achieve appropriate efficient execution. We describe how ROGUE communicates with the planner and the robot to interleave planning with execution so that the planner can replan for failed actions, identify the actual outcome of an action with multiple possible outcomes, and take opportunities from changes in the environment.ROGUE represents a successful integration of a classical artificial intelligence planner with a real mobile robot.     

RoboCup3D仿真系统中的机器人自定位方法

在RoboCup3D仿真比赛中,机器人自定位非常重要,定位不准确会对仿真比赛产生严重的影响。为了模拟真实环境,比赛中加入了视觉噪声。这使机器人定位变得更加困难。本文针对RoboCup3D仿真中的机器人视觉特征,提出一种观测值加权融合的卡尔曼滤波方法来实现机器人自定位,采用此方法能得到更精确的观测值。仿真实验结果表明,此定位方法大大提高了机器人自定位的精度。     

Bio-insect and artificial robot interaction: learning mechanism and experiment

This paper addresses fuzzy-logic-based reinforcement learning architecture and experimental results for the interaction between an artificial robot and a living bio-insect. The main goal of this research is to drag the bio-insect towards the desired goal area without any human aid. To achieve the goal, we seek to design robot intelligence architecture such that the robot can drag the bio-insect using its own learning mechanism. The main difficulties of this research are to find an interaction mechanism between the robot and bio-insect and to design a robot intelligence architecture. In simple interaction experiment, the bio-insect does not react to stimuli such as light, vibration, or artificial robot motion. From various trials-and-error efforts, we empirically found an actuation mechanism for the interaction between the robot and bio-insect. Nevertheless, it is difficult to control the movement of the bio-insect due to its uncertain and complex behavior. For the artificial robot, we design a fuzzy-logic-based reinforcement learning architecture that helps the artificial robot learn how to control the movement of the bio-insect under uncertain and complex behavior. Here, we present the experimental results regarding the interaction between an artificial robot and a bio-insect.     

Cost-Sensitive Learning of Classification Knowledge and Its Applications in Robotics

Traditional learning-from-examples methods assume that examples are given beforehand and all features are measured for each example. However, in many robotic domains the number of features that could be measured is very large, the cost of measuring those features is significant, and thus the robot must judiciously select which features it will measure. Finding a proper tradeoff between theaccuracy (e.g., number of prediction errors) andefficiency (e.g., cost of measuring features) during learning (prior to convergence) is an important part of the problem. Inspired by such robotic domains, this article considers realistic measurement costs of features in the process of incremental learning of classification knowledge. It proposes a unified framework for learning-from-examples methods that trade off accuracy for efficiency during learning, and analyzes two methods (CS-ID3 and CS-IBL) in detail. Moreover, this article illustrates the application of such a cost-sensitive-learning method to a real robot designed for anapproach-recognize task. The resulting robot learns to approach, recognize, and grasp objects on a floor effectively and efficiently. Experimental results show that highly accurate classification procedures can be learned without sacrificing efficiency in the case of both synthetic and real domains.     

Incremental acquisition of behaviors and signs based on a reinforcement learning schemata model and a spike timing-dependent plasticity network

A novel integrative learning architecture based on a reinforcement learning schemata model (RLSM) with a spike timing-dependent plasticity (STDP) network is described. This architecture models operant conditioning with discriminative stimuli in an autonomous agent engaged in multiple reinforcement learning tasks. The architecture consists of two constitutional learning architectures: RLSM and STDP. RLSM is an incremental modular reinforcement learning architecture, and it makes an autonomous agent acquire several behavioral concepts incrementally through continuous interactions with its environment and/or caregivers. STDP is a learning rule of neuronal plasticity found in cerebral cortices and the hippocampus of the human brain. STDP is a temporally asymmetric learning rule that contrasts with the Hebbian learning rule. We found that STDP enabled an autonomous robot to associate auditory input with its acquired behaviors and to select reinforcement learning modules more effectively. Auditory signals interpreted based on the acquired behaviors were revealed to correspond to 'signs' of required behaviors and incoming situations. This integrative learning architecture was evaluated in the context of on-line modular learning.     

Adaptive behaviors of reactive mobile robot with Bayesian inference in nonstationary environments

This paper presents a technique for a reactive mobile robot to adaptively behave in unforeseen and dynamic circumstances. A robot in nonstationary environments needs to infer how to adaptively behave to the changing environment. Behavior-based approach manages the interactions between the robot and its environment for generating behaviors, but in spite of its strengths of fast response, it has not been applied much to more complex problems for high-level behaviors. For that reason many researchers employ a behavior-based deliberative architecture. This paper proposes a 2-layer control architecture for generating adaptive behaviors to perceive and avoid moving obstacles as well as stationary obstacles. The first layer is to generate reflexive and autonomous behaviors with behavior network, and the second layer is to infer dynamic situations of the mobile robot with Bayesian network. These two levels facilitate a tight integration between high-level inference and low-level behaviors. Experimental results with various simulations and a real robot have shown that the robot reaches the goal points while avoiding stationary or moving obstacles with the proposed architecture.     

ALECSYS and the AutonoMouse: Learning to Control a Real Robot by Distributed Classifier Systems

In this article we investigate the feasibility of using learning classifier systems as a tool for building adaptive control systems for real robots. Their use on real robots imposes efficiency constraints which are addressed by three main tools: parallelism, distributed architecture, and training. Parallelism is useful to speed up computation and to increase the flexibility of the learning system design. Distributed architecture helps in making it possible to decompose the overall task into a set of simpler learning tasks. Finally, training provides guidance to the system while learning, shortening the number of cycles required to learn. These tools and the issues they raise are first studied in simulation, and then the experience gained with simulations is used to implement the learning system on the real robot. Results have shown that with this approach it is possible to let the AutonoMouse, a small real robot, learn to approach a light source under a number of different noise and lesion conditions.This work was partially written while the author was at International Computer Science Institute, 1947 Center Street, Suite 600, Berkeley, 94704-1198 California, USA.     

Deep unsupervised network for multimodal perception,representation and classification

In this paper, we tackle the problem of multimodal learning for autonomous robots. Autonomous robots interacting with humans in an evolving environment need the ability to acquire knowledge from their multiple perceptual channels in an unsupervised way. Most of the approaches in the literature exploit engineered methods to process each perceptual modality. In contrast, robots should be able to acquire their own features from the raw sensors, leveraging the information elicited by interaction with their environment: learning from their sensorimotor experience would result in a more efficient strategy in a life-long perspective. To this end, we propose an architecture based on deep networks, which is used by the humanoid robot iCub to learn a task from multiple perceptual modalities (proprioception, vision, audition). By structuring high-dimensional, multimodal information into a set of distinct sub-manifolds in a fully unsupervised way, it performs a substantial dimensionality reduction by providing both a symbolic representation of data and a fine discrimination between two similar stimuli. Moreover, the proposed network is able to exploit multimodal correlations to improve the representation of each modality alone.     

A Fuzzy Perceptual Model for Ultrasound Sensors Applied to Intelligent Navigation of Mobile Robots

The way of understanding the role of perception along the intelligent robotic systems has evolved greatly since classic approaches to the reactive behavior-based approaches. Classic approaches tried to model the environment using a high level of accuracy while in reactive systems usually the perception is related to the actions that the robot needs to undertake so that such complex models are not generally necessary. Regarding hybrid approaches is likewise important to understand the role that has been assigned to the perception in order to assure the success of the system. In this work a new perceptual model based on fuzzy logic is proposed to be used in a hybrid deliberative-reactive architecture. This perceptual model deals with the uncertainty and vagueness underlying to the ultrasound sensor data, it is useful to carry out the data fusion from different sensors and it allows us to establish various levels of interpretation in the sensor data. Furthermore, using this perceptual model an approximate world model can be built so that the robot can plan its motions for navigating in an office-like environment. Then the navigation is accomplished using the hybrid deliberative-reactive architecture and taking into account the perceptual model to represent the robot's beliefs about the world. Experiments in simulation and in an real office-like environment are shown for validating the perceptual model integrated into the navigation architecture.     

An architecture for a VLSI sensory–motor system for obstacle avoidance

This paper presents a signal processing architecture for a sensory–motor system based on the smart sensor paradigm. The architecture is designed for an obstacle avoidance task by a mobile robot in an unstructured environment. Drawing inspiration from the field of behavior-based robotics, the development of the architecture is guided by an emphasis on the requirements of an obstacle avoidance behavior for a mobile robot. The architecture is simple enough for a smart sensor, but incorporates features which enable it to deal with realistic, unstructured environments. It differs from existing systems by using a special foveation scheme to facilitate the detection of real-world objects. From this, a motor control signal is produced by using a biologically inspired technique of aligning sensory and motor maps. The effectiveness of the architecture is explored through computer simulation, including an obstacle avoidance simulation in a 3D virtual environment.     

Geographical information systems for map based navigation in urban environments

In order to solve most of the existing mobile robotics applications, the robot needs some information about its spatial environment encoded in what it has been commonly called a map. The knowledge contained in such a map, whatever approach is used to obtain it, will mainly be used by the robot to gain the ability to navigate in a given environment. We are describing in this paper, a method that allows a robot or team of robots to navigate in large urban areas for which an existing map in a standard human understandable fashion is available. As detailed maps of most urban areas already exist, it will be assumed that a map of the zone where the robot is supposed to work is given, which has not been constructed using the robot’s own sensors. We propose in this paper, the use of an existing Geographical Information System based map of an urban zone so that a robot or a team of robots can connect to this map and use it for navigation purposes. Details of the implemented system architecture as well as a position tracking experiment in a real outdoor environment, a University Campus, are provided.     

Task Transfer by a Developmental Robot

Scaffolding is a process of transferring learned skills to new and more complex tasks through arranged experience in open-ended development. In this paper, we propose a developmental learning architecture that enables a robot to transfer skills acquired in early learning settings to later more complex task settings. We show that a basic mechanism that enables this transfer is sequential priming combined with attention, which is also the driving mechanism for classical conditioning, secondary conditioning, and instrumental conditioning in animal learning. A major challenge of this work is that training and testing must be conducted in the same program operational mode through online, real-time interactions between the agent and the trainers. In contrast with former modeling studies, the proposed architecture does not require the programmer to know the tasks to be learned and the environment is uncontrolled. All possible perceptions and actions, including the actual number of classes, are not available until the programming is finished and the robot starts to learn in the real world. Thus, a predesigned task-specific symbolic representation is not suited for such an open-ended developmental process. Experimental results on a robot are reported in which the trainer shaped the behaviors of the agent interactively, continuously, and incrementally through verbal commands and other sensory signals so that the robot learns new and more complex sensorimotor tasks by transferring sensorimotor skills learned in earlier periods of open-ended development