基于神经网络的进化机器人组合行为方法研究

为了克服传统机器人设计方法存在的局限性,提高机器人的自适应能力,采用神经网络方法实现了进化机器人避碰、趋近及其组合行为学习,首先,提出了新的机器人模拟环境和机器人模型,结合了采用神经网络实现进化学习系统的方法。其次,对具有进化学习机制的机器人基本行为和组合行为学习系统进行了仿真,并通过仿真证明了新模型不要求环境知识的完备性,机器人具有环境自适应学习能力,还具有结构简洁、易扩展等特点,最后,对仿真结果进行分析与讨论,并提出了进一步研究方向。     

自主式微小型移动机器人的自动避障行为研究

针对多微小型移动机器人工作环境的模型未知或不确定,以及该机器人本身 的某些限制,采用基于行为的研究方法,实现了自行设计的自主式微小型移动机器人在未知 、动态环境中的自动避障,设计了该机器人的障碍物回避行为,采用了电机神经元网络选择 机器人的自动避障动作,并用增强式学习的动作评判结果在线修改网络的权值,结合机器人 的漫步行为,采用机器人的安全漫步任务验证了该方法的有效性．     

基于平行进化的机器人智能控制研究

为增强机器人的实用性,使机器人在一定环境内能够自主完成指定任务,具备良好的环境适应能力需要在机器人系统的设计与开发中引入进化的机制与思想,在机器人对环境的反射中,通过学习促进机器人行为与功能的进化.在机器人进化思想的基础上,提出了一种机器人和软件人之问的平行进化方法,建立了平行进化过程的多重广义模型,给出了平行进化的具体算法,并在实际的变电站巡检机器人系统中加以实现与验证.实际应用情况表明平行进化的方法可以提升机器人智能化水平、扩展机器人功能.     

基于PSO算法的RoboCup2D机器人研究

对机器人体系结构、动作学习及行为的组织方式进行了研究,以演化计算为基本方法,以RoboCup2D为平台,设计了基于PSO算法的足球机器人的体系结构,解决感知、动作、和规划问题;在训练环境下,形成感知规则,优化感知相关参数,得到对信息高效快速的感知方法,并根据指定的粒度、功能、参数,对RoboCup2D机器人的原子动作进行了组合优化,得到一组带参数和执行效果描述的粒子动作;最后在赛场环境和任务驱动下,搜索粒子动作并进行组织规划,得到完成特定任务的机器人行为;RoboCup2D仿真实验表明,演化计算方法不仅能利用原子动作进行组合优化,得到适应于不同条件的粒子动作,而且能通过其在线搜索粒子动作,动态组成机器人行为;基于演化计算的足球机器人能更好地完成跑位、截球、带球、传球等任务,具有更强的适应性。     

基于强化学习的未知环境多机器人协作搜集

针对多机器人协作复杂搜集任务中学习空间大,学习速度慢的问题,提出了带共享区的双层强化学习算法。该强化学习算法不仅能够实现低层状态-动作对的学习,而且能够实现高层条件-行为对的学习。高层条件-行为对的学习避免了学习空间的组合爆炸,共享区的应用强化了机器人间协作学习的能力。仿真实验结果说明所提方法加快了学习速度,满足了未知环境下多机器人复杂搜集任务的要求。     

自组织映射神经网络量化机器人强化学习方法研究

强化学习一词来自于行为心理学，这门学科把行为学习看成反复试验的过程，从而把环境状态映射成相应的动作。在设计智能机器人过程中，如何来实现行为主义的思想，在与环境的交互中学习行为动作？文中把机器人在未知环境中为躲避障碍所采取的动作看作一种行为，采用强化学习方法来实现智能机器人避碰行为学习。为了提高机器人学习速度，在机器人局部路径规划中的状态空量化就显得十分重要。本文采用自组织映射网络的方法来进行空间的量化。由于自组织映射网络本身所具有的自组织特性，使得它在进行空间量化时就能够较好地解决适应性灵活性问题，本文在对状态空间进行自组织量化的基础方法上，采用强化学习。解决了机器人避碰行为的学习问题，取得了满意的学习结果。     

人工情感的进化控制系统实现

人工情感在机器人的研究中至关重要,简要概括了当前人工情感的应用.在借鉴情感学习控制的理论的基础上,融入了进化控制的思想,设计出了一种基于人工情感的控制体系结构,在此结构中包含有基于遗传算法的进化控制系统、神经和人工情感控制系统.机器人通过神经系统接受环境信息并进行行为决策,行为决策的效果通过情感学习模型进行反馈.情感学习模型根据机器人的内、外环境状态,产生情感因子（即生物激素）,再由情感因子来调节神经系统的记忆和行为决策,最后神经系统的记忆与行为模块又由进化系统得以继承.该控制结构加强了机器人在动态环境中的学习和自适应能力.仿真实验验证了该控制结构的有效性,仿真结果也表明机器人具有很强的学习和自适应能力.     

一种新的多智能体强化学习算法及其在多机器人协作任务中的应用

在多机器人系统中，评价一个机器人行为的好坏常常依赖于其它机器人的行为，此 时必须采用组合动作以实现多机器人的协作，但采用组合动作的强化学习算法由于学习空间 异常庞大而收敛得极慢．本文提出的新方法通过预测各机器人执行动作的概率来降低学习空 间的维数，并应用于多机器人协作任务之中．实验结果表明，基于预测的加速强化学习算法 可以比原始算法更快地获得多机器人的协作策略．     

基于人工情感控制系统的设计与实现

人工情感在机器人的研究中至关重要,文中简要概括了当前人工情感的应用;我们在借鉴生物系统控制理论的基础上,融入了进化控制的思想,设计了一种基于人工情感的控制体系结构,在此结构中包含有基于蚁群算法的进化控制系统、神经和人工情感控制系统;机器人通过神经系统接受环境信息并进行行为决策,行为决策的效果通过情感学习模型进行反馈;情感学习模型根据机器人的内、外环境状态,产生情感因子(即生物激素),再由情感因子来调节神经系统的记忆和行为决策,最后神经系统的记忆与行为模块又由进化系统得以继承;该控制结构加强了机器人在动态环境中的学习和自适应能力;为了验证该控制结构的有效性,文章做了仿真实验;仿真结果也表明机器人具有很强的学习和自适应能力.     

未知环境下自主机器人的行为学习研究

机器人为实现在未知环境下的探索任务,必须具有自主学习其行为策略的能力.本文提出了一种自主机器人行为学习机制.机器人通过与环境的交互,基于Q学习进行行为的自主学习.为降低学习时的计算复杂度,状态空间通过分段映射为不同的类别,从而减少状态-动作对的数量.自主机器人在未知环境中的行为学习是增量式的过程,本文将基于案例的学习与Q学习结合,使机器人在试错时获得的经验以案例的形式保存,并实现案例库的动态更新相关案例同时可以降低机器人行为学习时的计算复杂度和试错时的风险.在文中的最后给出了仿真结果.     

自主微小型移动机器人的协作学习

自主微小型移动机器人的协作学习研究是多智能体机器人系统理论的主要研究方向。因为单个微小型移动机器人能力有限，所以机器人之间的协作在某些重要的基础工业和生物医学领域方面显得非常重要。该文介绍了几种用于协作学习的方法并且比较了它们之间的优点和缺点。最后，简要介绍了一些研究工作。     

Auditory learning: a developmental method

Motivated by the human autonomous development process from infancy to adulthood, we have built a robot that develops its cognitive and behavioral skills through real-time interactions with the environment. We call such a robot a developmental robot. In this paper, we present the theory and the architecture to implement a developmental robot and discuss the related techniques that address an array of challenging technical issues. As an application, experimental results on a real robot, self-organizing, autonomous, incremental learner (SAIL), are presented with emphasis on its audition perception and audition-related action generation. In particular, the SAIL robot conducts the auditory learning from unsegmented and unlabeled speech streams without any prior knowledge about the auditory signals, such as the designated language or the phoneme models. Neither available before learning starts are the actions that the robot is expected to perform. SAIL learns the auditory commands and the desired actions from physical contacts with the environment including the trainers.     

群体自主微小型移动机器人的合作

单个微小型机器人由于自身能力的限制，因此必须多个机器人联合起来才可以完 成指定的任务，所以机器人之间的协作在微操作领域就显得尤其重要。该文利用增强式的 学 习方法，使得微小型机器人具有一定的学习能力，增强了对不确定环境的适应性，并采 用了 一种基于行为的群体自主式微小移动机器人的协作结构，用于机器人的故障排除，仿 真结果 说明了这种体系结构的有效性。     

Rapid, safe, and incremental learning of navigation strategies

In this paper we propose a reinforcement connectionist learning architecture that allows an autonomous robot to acquire efficient navigation strategies in a few trials. Besides rapid learning, the architecture has three further appealing features. First, the robot improves its performance incrementally as it interacts with an initially unknown environment, and it ends up learning to avoid collisions even in those situations in which its sensors cannot detect the obstacles. This is a definite advantage over nonlearning reactive robots. Second, since it learns from basic reflexes, the robot is operational from the very beginning and the learning process is safe. Third, the robot exhibits high tolerance to noisy sensory data and good generalization abilities. All these features make this learning robot's architecture very well suited to real-world applications. We report experimental results obtained with a real mobile robot in an indoor environment that demonstrate the appropriateness of our approach to real autonomous robot control.     

基于情感与环境认知的移动机器人自主导航控制

将基于情感和认知的学习与决策模型引入到基于行为的移动机器人控制体系中, 设计了一种新的自主导航控制系统. 将动力学系统方法用于基本行为设计, 并利用ART2神经网络实现对连续的环境感知状态的分类, 将分类结果作为学习与决策算法中的环境认知状态. 通过在线情感和环境认知学习, 形成合理的行为协调机制. 仿真表明, 情感和环境认知能明显地改善学习和决策过程效率, 提高基于行为的移动机器人在未知环境中的自主导航能力     

Incremental acquisition of behaviors and signs based on a reinforcement learning schemata model and a spike timing-dependent plasticity network

A novel integrative learning architecture based on a reinforcement learning schemata model (RLSM) with a spike timing-dependent plasticity (STDP) network is described. This architecture models operant conditioning with discriminative stimuli in an autonomous agent engaged in multiple reinforcement learning tasks. The architecture consists of two constitutional learning architectures: RLSM and STDP. RLSM is an incremental modular reinforcement learning architecture, and it makes an autonomous agent acquire several behavioral concepts incrementally through continuous interactions with its environment and/or caregivers. STDP is a learning rule of neuronal plasticity found in cerebral cortices and the hippocampus of the human brain. STDP is a temporally asymmetric learning rule that contrasts with the Hebbian learning rule. We found that STDP enabled an autonomous robot to associate auditory input with its acquired behaviors and to select reinforcement learning modules more effectively. Auditory signals interpreted based on the acquired behaviors were revealed to correspond to 'signs' of required behaviors and incoming situations. This integrative learning architecture was evaluated in the context of on-line modular learning.     

Autonomous reconfiguration of robot shape by using Q-learning

A modular robot can be built with a shape and function that matches the working environment. We developed a four-arm modular robot system which can be configured in a planar structure. A learning mechanism is incorporated in each module constituting the robot. We aim to control the overall shape of the robot by an accumulation of the autonomous actions resulting from the individual learning functions. Considering that the overall shape of a modular robot depends on the learning conditions in each module, this control method can be treated as a dispersion control learning method. The learning object is cooperative motion between adjacent modules. The learning process proceeds based on Q-learning by trial and error. We confirmed the effectiveness of the proposed technique by computer simulation.     

Autonomous knowledge acquisition based on artificial curiosity: Application to mobile robots in an indoor environment

This paper describes an autonomous system for knowledge acquisition based on artificial curiosity. The proposed approach allows a humanoid robot to discover, in an indoor environment, the world in which it evolves, and to learn autonomously new knowledge about it. The learning process is accomplished by observation and by interaction with a human tutor, based on a cognitive architecture with two levels. Experimental results of deployment of this system on a humanoid robot in a real office environment are provided. We show that our cognitive system allows a humanoid robot to gain increased autonomy in matters of knowledge acquisition.     

INTELLIGENT SENSOR FUSION AND LEARNING FOR AUTONOMOUS ROBOT NAVIGATION

ABSTRACT

This paper presents the design and implementation of an autonomous robot navigation system for intelligent target collection in dynamic environments. A feature-based multi-stage fuzzy logic (MSFL) sensor fusion system is developed for target recognition, which is capable of mapping noisy sensor inputs into reliable decisions. The robot exploration and path planning are based on a grid map oriented reinforcement path learning system (GMRPL), which allows for long-term predictions and path adaptation via dynamic interactions with physical environments. In our implementation, the MSFL and GMRPL are integrated into subsumption architecture for intelligent target-collecting applications. The subsumption architecture is a layered reactive agent structure that enables the robot to implement higher-layer functions including path learning and target recognition regardless of lower-layer functions such as obstacle detection and avoidance. The real-world application using a Khepera robot shows the robustness and flexibility of the developed system in dealing with robotic behaviors such as target collecting in the ever-changing physical environment.     

Interactive Teaching for Vision-Based Mobile Robots: A Sensory-Motor Approach

For the last decade, we have been developing a vision-based architecture for mobile robot navigation. Using our bio-inspired model of navigation, robots can perform sensory-motor tasks in real time in unknown indoor as well as outdoor environments. We address here the problem of autonomous incremental learning of a sensory-motor task, demonstrated by an operator guiding a robot. The proposed system allows for semisupervision of task learning and is able to adapt the environmental partitioning to the complexity of the desired behavior. A real dialogue based on actions emerges from the interactive teaching. The interaction leads the robot to autonomously build a precise sensory-motor dynamics that approximates the behavior of the teacher. The usability of the system is highlighted by experiments on real robots, in both indoor and outdoor environments. Accuracy measures are also proposed in order to evaluate the learned behavior as compared to the expected behavioral attractor. These measures, used first in a real experiment and then in a simulated experiment, demonstrate how a real interaction between the teacher and the robot influences the learning process.