智能机器人的自适应自主Agents的建模

在问题求解期间及穿越动态的、不可预测的环境的过程中 ,真实世界的问题需要自适应的求解方法来‘裁剪’Agent的行为 ,并使其落入任务域 ,否则不完备的知识、不确定性、不确定的Agents及过程的存在、硬件失灵和不精确性都会引发不确定事件的发生。自适应自主Agents是一种存在于动态的、不可预测的环境中 ,并试图满足一系列随时间变化的目标或动机的系统 ,而且此类Agents在处理这些目标的过程中 ,能基于已有的经验改进其能力。自适应自主Agents为智能机器人实现这一目标提供了一种新的途径。主要讨论了智能机器人的自适应自主Agents的建模问题 ,并指出它的局限性和开问题     

Human-aware robot navigation: A survey

Navigation is a basic skill for autonomous robots. In the last years human–robot interaction has become an important research field that spans all of the robot capabilities including perception, reasoning, learning, manipulation and navigation. For navigation, the presence of humans requires novel approaches that take into account the constraints of human comfort as well as social rules. Besides these constraints, putting robots among humans opens new interaction possibilities for robots, also for navigation tasks, such as robot guides. This paper provides a survey of existing approaches to human-aware navigation and offers a general classification scheme for the presented methods.     

Efficient Reinforcement Learning through Symbiotic Evolution

This article presents a new reinforcement learning method called SANE (Symbiotic, Adaptive Neuro-Evolution), which evolves a population of neurons through genetic algorithms to form a neural network capable of performing a task. Symbiotic evolution promotes both cooperation and specialization, which results in a fast, efficient genetic search and discourages convergence to suboptimal solutions. In the inverted pendulum problem, SANE formed effective networks 9 to 16 times faster than the Adaptive Heuristic Critic and 2 times faster than Q-learning and the GENITOR neuro-evolution approach without loss of generalization. Such efficient learning, combined with few domain assumptions, make SANE a promising approach to a broad range of reinforcement learning problems, including many real-world applications.     

机器学习的主要策略综述

当前人工智能研究的主要障碍和发展方向之一就是机器学习。机器学习与计算机科学、心理学、认知科学等各种学科都有着密切的联系,牵涉的面比较广,许多理论及技术上的问题尚处于研究之中。对机器学习的一些主要策略的基本思想进行了较全面的介绍,同时介绍了一些最新的进展和研究热点。     

Aimy: An autonomous mobile robot navigation in unknown environment with infrared detector system

Aimy — an Autonomous Mobile Robot (AMR), capable of moving in an unknown environment filled with obstacles, has been developed. To avoid collision with unexpected obstacles, an Infrared Detector System (IDS) for providing multiple reading data was designed and implemented. A navigation/obstacle avoidance strategy for a mobile robot, which is based on the use of infrared detector data only, is discussed. Experiment results are also presented which exhibit the power of the developed algorithm and Infrared Detector System.     

A self-adaptive Multimeme Memetic Algorithm co-evolving utility scores to control genetic operators and their parameter settings

Memetic algorithms are a class of well-studied metaheuristics which combine evolutionary algorithms and local search techniques. A meme represents contagious piece of information in an adaptive information sharing system. The canonical memetic algorithm uses a fixed meme, denoting a hill climbing operator, to improve each solution in a population during the evolutionary search process. Given global parameters and multiple parameterised operators, adaptation often becomes a crucial constituent in the design of MAs. In this study, a self-adaptive self-configuring Steady-state Multimeme Memetic Algorithm (SSMMA) variant is proposed. Along with the individuals (solutions), SSMMA co-evolves memes, encoding the utility score for each algorithmic component choice and relevant parameter setting option. An individual uses tournament selection to decide which operator and parameter setting to employ at a given step. The performance of the proposed algorithm is evaluated on six combinatorial optimisation problems from a cross-domain heuristic search benchmark. The results indicate the success of SSMMA when compared to the static MAs as well as widely used self-adaptive Multimeme Memetic Algorithm from the scientific literature.     

An autonomous educational mobile robot mediator

So far, most of the applications of robotic technology to education have mainly focused on supporting the teaching of subjects that are closely related to the Robotics field, such as robot programming, robot construction, or mechatronics. Moreover, most of the applications have used the robot as an end or a passive tool of the learning activity, where the robot has been constructed or programmed. In this paper, we present a novel application of robotic technologies to education, where we use the real world situatedness of a robot to teach non-robotic related subjects, such as math and physics. Furthermore, we also provide the robot with a suitable degree of autonomy to actively guide and mediate in the development of the educational activity. We present our approach as an educational framework based on a collaborative and constructivist learning environment, where the robot is able to act as an interaction mediator capable of managing the interactions occurring among the working students. We illustrate the use of this framework by a 4-step methodology that is used to implement two educational activities. These activities were tested at local schools with encouraging results. Accordingly, the main contributions of this work are: i) A novel use of a mobile robot to illustrate and teach relevant concepts and properties of the real world; ii) A novel use of robots as mediators that autonomously guide an educational activity using a collaborative and constructivist learning approach; iii) The implementation and testing of these ideas in a real scenario, working with students at local schools.

Evolvable hardware techniques for on-chip automated reconfiguration of programmable devices

While complete automated design is a harder problem than computer-assisted design, automated hardware reconfiguration is an even more challenging problem, because it needs to adjust to limited resources and various factors, such as noise and parasitic capacitance, a resistance and inductance. This paper presents some experimental results of on-chip automated design and reconfiguration using evolvable hardware techniques. It describes a stand-alone board level evolvable system, and its use to demonstrate on-chip synthesis of new circuits in only a few seconds. The experiments presented here indicate a recovery capability in the case of extreme environmental conditions, such as extreme temperatures, that adversely affect electronics. Some of the difficulties of dealing with the real hardware are exposed, as well as challenges more generally related to automated evolution of complex electronic systems.The work described in this paper was performed at the Center for Integrated Space Microsystems, Jet Propulsion Laboratory, California Institute of Technology and was sponsored by the Defense Advanced Research Projects Agency and by the National Aeronautics and Space Administration.     

基于模型的机器人强化学习研究综述

基于模型的强化学习通过学习一个环境模型和基于此模型的策略优化或规划,实现机器人更接近于人类的学习和交互方式.文中简述机器人学习问题的定义,介绍机器人学习中基于模型的强化学习方法,包括主流的模型学习及模型利用的方法.主流的模型学习方法具体介绍前向动力学模型、逆向动力学模型和隐式模型.模型利用的方法具体介绍基于模型的规划、...     

Evolving non-trivial behaviors on real robots: A garbage collecting robot

Recently, a new approach involving a form of simulated evolution has been proposed to build autonomous robots. However, it is still not clear if this approach is adequate for real life problems. In this paper we show how control systems that perform a non-trivial sequence of behaviors can be obtained with this methodology by “canalizing” the evolutionary process in the right direction. In the experiment described in the paper, a mobile robot was successfully trained to keep clear an arena surrounded by walls by locating, recognizing, and grasping “garbage” objects and by taking collected objects outside the arena. The controller of the robot was evolved in simulation and then downloaded and tested on the real robot. We also show that while a given amount of supervision may canalize the evolutionary process in the right direction the addition of unnecessary constraints can delay the evolution of the desired behavior.     

Target recognition by components for mobile robot navigation

This paper presents a vision-based technique for detecting targets of the environment which have to be reached by an autonomous mobile robot during its navigational tasks. The targets the robot has to reach are the doors of the authors' office building. The detection of the door has been performed by detecting its most significant components in the image and it is based on data classification. Two neural classifiers have been trained for recognizing single components of the door. Then a combining algorithm, based on heuristic considerations, checks that they are in the proper geometric configuration of the structure of the door. The novelty of this work is to use together colour and shape information for identifying features and for detecting the components of the target. The approach, based on learning by components, is able to cleverly solve the problems of scale changes, perspective variations and partial occlusions. The obtained detecting system has been tested on a large test set of real images showing a high reliability and robustness: doors of different rooms, under different illumination conditions and by different viewpoints have been successfully recognized. Results in terms of door detection rate and false positive rate are presented throughout the paper.     

基于智能强化学习的遗传算法研究

采用结合智能强化学习和遗传算法来求解车间作业调度问题。     

New technique of mobile robot navigation using a hybrid adaptive fuzzy potential field approach

This paper presents a summary of the research aimed at developing a new reliable methodology for robot navigation and obstacle avoidance. This new approach is based on the artificial potential field (APF) method, which is used extensively for obstacle avoidance. The classical APF is dependent only on the separation distance between the robot and the surrounding obstacles. The new scheme introduces a variable, which is used to determine the importance that each obstacle has on the robot's future path. The importance variable is dependent on the obstacles position, both angle and distance, with respect to the robot. Simulation results are presented demonstrating the ability of the algorithm to perform successfully in simple environments.     

Shaping robot behavior using principles from instrumental conditioning

Shaping by successive approximations is an important animal training technique in which behavior is gradually adjusted in response to strategically timed reinforcements. We describe a computational model of this shaping process and its implementation on a mobile robot. Innate behaviors in our model are sequences of actions and enabling conditions, and shaping is a behavior editing process realized by multiple editing mechanisms. The model replicates some fundamental phenomena associated with instrumental learning in animals, and allows an RWI B21 robot to learn several distinct tasks derived from the same innate behavior.     

The Loss from Imperfect Value Functions in Expectation-Based and Minimax-Based Tasks

Many reinforcement learning (RL) algorithms approximate an optimal value function. Once the function is known, it is easy to determine an optimal policy. For most real-world applications, however, the value function is too complex to be represented by lookup tables, making it necessary to use function approximators such as neural networks. In this case, convergence to the optimal value function is no longer guaranteed and it becomes important to know to which extent performance diminishes when one uses approximate value functions instead of optimal ones. This problem has recently been discussed in the context of expectation based Markov decision problems. Our analysis generalizes this work to minimax-based Markov decision problems, yields new results for expectation-based tasks, and shows how minimax-based and expectation based Markov decision problems relate.     

The uses of fuzzy logic in autonomous robot navigation

 The development of techniques for autonomous navigation in real-world environments constitutes one of the major trends in the current research on robotics. An important problem in autonomous navigation is the need to cope with the large amount of uncertainty that is inherent of natural environments. Fuzzy logic has features that make it an adequate tool to address this problem. In this paper, we review some of the possible uses of fuzzy logic in the field of autonomous navigation. We focus on four issues: how to design robust behavior-producing modules; how to coordinate the activity of several such modules; how to use data from the sensors; and how to integrate high-level reasoning and low-level execution. For each issue, we review some of the proposals in the literature, and discuss the pros and cons of fuzzy logic solutions. Received: 31 March 1997 / Accepted: 24 September 1997     

进化强化学习及其在机器人路径跟踪中的应用

研究了一种基于自适应启发评价(AHC)强化学习的移动机器人路径跟踪控制方法.AHC的评价单元(ACE)采用多层前向神经网络来实现.将TD(λ)算法和梯度下降法相结合来更新神经网络的权值.AHC的动作选择单元(ASE)由遗传算法优化的模糊推理系统(FIS)构成.ACE网络的输出构成二次强化信号,用于指导ASE的学习.最后将所提出的算法应用于移动机器人的行为学习,较好地解决了机器人的复杂路径跟踪问题.     

Reinforcement learning for robot soccer

Batch reinforcement learning methods provide a powerful framework for learning efficiently and effectively in autonomous robots. The paper reviews some recent work of the authors aiming at the successful application of reinforcement learning in a challenging and complex domain. It discusses several variants of the general batch learning framework, particularly tailored to the use of multilayer perceptrons to approximate value functions over continuous state spaces. The batch learning framework is successfully used to learn crucial skills in our soccer-playing robots participating in the RoboCup competitions. This is demonstrated on three different case studies.

A model-based sound localization system and its application to robot navigation

This paper describes a mobile robot equipped with a real time sound localization system as well as a sonar system for obstacle detection. The sound localization method is based on a model of the precedence effect of the human auditory system to cope with echoes and reverberations. Sound localization and robot navigation experiments were conducted. The results show that the robot is capable of localizing sounding objects in a reverberant environment and approaching the objects without collisions, even when the objects were behind obstacles. Environment flexibility and error robustness of the system were discussed as well.