Rapid Concept Learning for Mobile Robots

Concept learning in robotics is an extremely challenging problem: sensory data is often high-dimensional, and noisy due to specularities and other irregularities. In this paper, we investigate two general strategies to speed up learning, based on spatial decomposition of the sensory representation, and simultaneous learning of multiple classes using a shared structure. We study two concept learning scenarios: a hallway navigation problem, where the robot has to induce features such as opening or wall. The second task is recycling, where the robot has to learn to recognize objects, such as a trash can. We use a common underlying function approximator in both studies in the form of a feedforward neural network, with several hundred input units and multiple output units. Despite the high degree of freedom afforded by such an approximator, we show the two strategies provide sufficient bias to achieve rapid learning. We provide detailed experimental studies on an actual mobile robot called PAVLOV to illustrate the effectiveness of this approach.     

Rapid Concept Learning for Mobile Robots

Concept learning in robotics is an extremely challenging problem: sensory data is often high dimensional, and noisy due to specularities and other irregularities. In this paper, we investigate two general strategies to speed up learning, based on spatial decomposition of the sensory representation, and simultaneous learning of multiple classes using a shared structure. We study two concept learning scenarios: a hallway navigation problem, where the robot has to induce features such as opening or wall. The second task is recycling, where the robot has to learn to recognize objects, such as a trash can. We use a common underlying function approximator in both studies in the form of a feedforward neural network, with several hundred input units and multiple output units. Despite the high degree of freedom afforded by such an approximator, we show the two strategies provide sufficient bias to achieve rapid learning. We provide detailed experimental studies on an actual mobile robot called PAVLOV to illustrate the effectiveness of this approach.     

Robot Awareness in Cooperative Mobile Robot Learning

Most of the straight-forward learning approaches in cooperative robotics imply for each learning robot a state space growth exponential in the number of team members. To remedy the exponentially large state space, we propose to investigate a less demanding cooperation mechanism—i.e., various levels of awareness—instead of communication. We define awareness as the perception of other robots locations and actions. We recognize four different levels (or degrees) of awareness which imply different amounts of additional information and therefore have different impacts on the search space size ((0), (1), (N), o(N),¹ where N is the number of robots in the team). There are trivial arguments in favor of avoiding binding the increase of the search space size to the number of team members. We advocate that, by studying the maximum number of neighbor robots in the application context, it is possible to tune the parameters associated with a (1) increase of the search space size and allow good learning performance. We use the cooperative multi-robot observation of multiple moving targets (CMOMMT) application to illustrate our method. We verify that awareness allows cooperation, that cooperation shows better performance than a purely collective behavior and that learned cooperation shows better results than learned collective behavior.     

Bayesian Landmark Learning for Mobile Robot Localization

To operate successfully in indoor environments, mobile robots must be able to localize themselves. Most current localization algorithms lack flexibility, autonomy, and often optimality, since they rely on a human to determine what aspects of the sensor data to use in localization (e.g., what landmarks to use). This paper describes a learning algorithm, called BaLL, that enables mobile robots to learn what features/landmarks are best suited for localization, and also to train artificial neural networks for extracting them from the sensor data. A rigorous Bayesian analysis of probabilistic localization is presented, which produces a rational argument for evaluating features, for selecting them optimally, and for training the networks that approximate the optimal solution. In a systematic experimental study, BaLL outperforms two other recent approaches to mobile robot localization.     

Mobile Robot Learning by Self-Observation

The aim of the project described in this paper was to investigate robot learning at a most fundamental level. The project focused on the transition between organisms with innate behaviors and organisms that have the most rudimentary capability of learning through their personal interaction with their environment. It was assumed that the innate behaviors gave basic survival competence but no learning ability. By observing the interaction between their innate behaviors and the organism's environment it was reasoned that the organism should be able to learn how to modify its actions in a way that improves its performance. If a learning system is given more information than it requires then, when it is successful, it is difficult to say which pieces of information contribute to the success. For this reason the information available to the learning system was kept to an absolute minimum. In order to provide a practical test of the learning scheme developed in this project, the robot environment EDEN was constructed. Within EDEN a robot's actions influence its internal energy reserves. The environment incorporates sources of energy, and it also involves situations that use additional energy or reduce energy consumption. A successful learning scheme was developed purely based on the recorded history of the robot's interactions with its environment and the knowledge that the robot's innate behavior was reactive. This learning scheme allowed the robot to improve its energy management by exhibiting classical conditioning and a restricted form of operant conditioning.     

Situated Learning of a Behavior-Based Mobile Robot Path Planner

1TaskDecompositionWebuildourpathplannerbaJsedonBrook'ssubsumptionarchitecturel1].InBrook'sapproach,theoveralltaskisdecomposedilltoseveralconcurrelitbehaviors,eachbehaviorhasitsownaPplicabilityconditionsspecifyingwhenitisappropriate,andapriorityorderingispresdefinedtoresolveconflictsamongbehaviors.Inpathplannillg,thetaskoftherobotistoaPproachatargetwhileavoidingobstacles.Wedecomposethetaskintothreebehaviors.TheyaJreAvoid,SteerandAdvance.Fig.1illustratestheoverallstructureofourpathplanner.…     

Learning Concepts from Sensor Data of a Mobile Robot

Machine learning can be a most valuable tool for improving the flexibility and efficiency of robot applications. Many approaches to applying machine learning to robotics are known. Some approaches enhance the robot's high-level processing, the planning capabilities. Other approaches enhance the low-level processing, the control of basic actions. In contrast, the approach presented in this paper uses machine learning for enhancing the link between the low-level representations of sensing and action and the high-level representation of planning. The aim is to facilitate the communication between the robot and the human user. A hierarchy of concepts is learned from route records of a mobile robot. Perception and action are combined at every level, i.e., the concepts are perceptually anchored. The relational learning algorithm GRDT has been developed which completely searches in a hypothesis space, that is restricted by rule schemata, which the user defines in terms of grammars.     

基于改进在线多示例学习算法的机器人目标跟踪

提出基于改进的在线多示例学习算法(Improved multiple instance learning, IMIL)的移动机器人目标跟踪方法. 该方法利用射频识别系统(Radio frequency identification, RFID)粗定位IMIL算法的搜索区域, 然后应用IMIL算法实现目标跟踪. 该方法保证了机器人跟踪系统的连续性, 解决了目标突然转弯时的跟踪问题. IMIL算法采用从低维空间提取的压缩特征描述包中示例, 以降低算法耗时. 通过最大化弱分类器与极大似然概率的内积, 选择判别能力强的弱分类器, 避免了弱分类器选择过程中多次计算包概率和示例概率, 进一步提高算法的实时处理能力. 计算包概率时该算法平等对待各示例, 保证概率高的示例对包概率的贡献度, 克服跟踪漂移问题. 跟踪过程中, 结合当前跟踪结果与目标模板间的相似性分数在线实时调整分类器, 提高了算法的自适应能力. 最后将本文方法在视频和移动机器人上进行实验. 实验结果表明, 该方法在目标运动突变及外观改变时具有较强的鲁棒性和准确性, 并满足系统的实时性要求.     

基于深度学习的移动机器人目标跟踪系统

针对当前智能移动机器人在跟踪过程中常因目标发生外观形态上的变化而丢失跟踪目标的问题,利用Caffe深度学习框架和ROS机器人操作系统作为开发平台,设计一个高准确度及高实时性的移动机器人目标跟踪系统并进行了研究.使用对于目标形变、视角、轻微遮挡及光照变化具有鲁棒性的基于孪生卷积神经网络的GOTURN目标跟踪算法,通过ROS系统为桥梁使离线训练的跟踪模型实时应用于TurtleBot移动机器人上,并开展了详细的测试.实验结果表明,该目标跟踪系统不仅设计方案可行,实现了移动机器人在各种复杂场景下有效地跟踪目标,还具有成本低、性能高和易扩展等特点.     

改进强化学习算法应用于移动机器人路径规划

为了解决传统的强化学习算法应用于移动机器人未知环境的路径规划时存在收敛速度慢、迭代次数多、收敛结果不稳定等问题,提出一种改进的Q-learning算法。在状态初始化时引入人工势场法,使得越靠近目标位置状态值越大,从而引导智能体朝目标位置移动,减少算法初始阶段因对环境探索产生的大量无效迭代;在智能体选择动作时改进[ε]-贪婪策略,根据算法的收敛程度动态调整贪婪因子[ε],从而更好地平衡探索和利用之间的关系,在加快算法收敛速度的同时提高收敛结果的稳定性。基于Python的Tkinter标准化库搭建的格栅地图仿真结果表明,改进的Q-learning算法相较于传统算法在路径规划时间上缩短85.1%,收敛前迭代次数减少74.7%,同时算法的收敛结果稳定性也得到了提升。     

基于深度强化学习的移动机器人路径规划

为解决传统的深度[Q]网络模型下机器人探索复杂未知环境时收敛速度慢的问题,提出了基于竞争网络结构的改进深度双[Q]网络方法（Improved Dueling Deep Double [Q]-Network,IDDDQN）。移动机器人通过改进的DDQN网络结构对其三个动作的值函数进行估计,并更新网络参数,通过训练网络得到相应的[Q]值。移动机器人采用玻尔兹曼分布与[ε]-greedy相结合的探索策略,选择一个最优动作,到达下一个观察。机器人将通过学习收集到的数据采用改进的重采样优选机制存储到缓存记忆单元中,并利用小批量数据训练网络。实验结果显示,与基本DDQN算法比,IDDDQN训练的机器人能够更快地适应未知环境,网络的收敛速度也得到提高,到达目标点的成功率增加了3倍多,在未知的复杂环境中可以更好地获取最优路径。     

Acquiring Mobile Robot Behaviors by Learning Trajectory Velocities

The development of robots that learn from experience is a relentless challenge confronting artificial intelligence today. This paper describes a robot learning method which enables a mobile robot to simultaneously acquire the ability to avoid objects, follow walls, seek goals and control its velocity as a result of interacting with the environment without human assistance. The robot acquires these behaviors by learning how fast it should move along predefined trajectories with respect to the current state of the input vector. This enables the robot to perform object avoidance, wall following and goal seeking behaviors by choosing to follow fast trajectories near: the forward direction, the closest object or the goal location respectively. Learning trajectory velocities can be done relatively quickly because the required knowledge can be obtained from the robot's interactions with the environment without incurring the credit assignment problem. We provide experimental results to verify our robot learning method by using a mobile robot to simultaneously acquire all three behaviors.     

改进深度强化学习的室内移动机器人路径规划

为了解决传统深度强化学习在室内未知环境下移动机器人路径规划中存在探索能力差和环境状态空间奖励稀疏的问题,提出了一种基于深度图像信息的改进深度强化学习算法。利用Kinect视觉传感器直接获取的深度图像信息和目标位置信息作为网络的输入,以机器人的线速度和角速度作为下一步动作指令的输出。设计了改进的奖惩函数,提高了算法的奖励值,优化了状态空间,在一定程度上缓解了奖励稀疏的问题。仿真结果表明,改进算法提高了机器人的探索能力,优化了路径轨迹,使机器人有效地避开了障碍物,规划出更短的路径,简单环境下比DQN算法的平均路径长度缩短了21.4%,复杂环境下平均路径长度缩短了11.3%。     

基于RDC-Q学习算法的移动机器人路径规划

传统Q算法对于机器人回报函数的定义较为宽泛,导致机器人的学习效率不高。为解决该问题,给出一种回报详细分类Q(RDC-Q)学习算法。综合机器人各个传感器的返回值,依据机器人距离障碍物的远近把机器人的状态划分为20个奖励状态和15个惩罚状态,对机器人每个时刻所获得的回报值按其状态的安全等级分类,使机器人趋向于安全等级更高的状态,从而帮助机器人更快更好地学习。通过在一个障碍物密集的环境中进行仿真实验,证明该算法收敛速度相对传统回报Q算法有明显提高。     

可变形机器人协同转向运动研究

可变形机器人AMOEBA I具有多种构形和多种转向方式．为实现机器人转向性能的优化,提出了可变形机器人的协同转向方法,并建立了相应的数学模型,对不同构形下的协同转向方式进行了理论分析．设定了机器人三个模块在协同转向过程中的运动关系,在此基础上给出了可变形机器人协同转向性能的评价指标．通过理论和实验比较了不同构形下的协同转向方式,实验验证了协同转向方法的有效性．     

移动机器人路径规划方法研究

针对室内动态非结构化环境下的移动机器人路径规划问题,提出了一种能够将全局路径规划方法和局部路径规划方法相结合、将基于反应的行为规划和基于慎思的行为规划相结合的路径规划方法．全局路径规划器采用A*算法生成到达目标点的子目标节点序列;局部路径规划器采用改进的人工势场方法对子目标节点序列中相邻两节点进行路径平滑和优化处理．在考虑了移动机器人运动学约束的前提下,该方法不但能够充分利用已知环境信息生成全局最优路径,而且还能及时处理所遇到的随机障碍信息．仿真研究与在室内复杂环境下的实际运行结果验证了该方法的有效性．     

架空电力线路巡线机器人的研究综述

回顾了国内外架空电力线路巡线机器人的研究现状 ,分析了几种巡线机器人的结构特点及存在的问题 ,详细探讨了巡线机器人避障、工作电源及线路故障探测等关键技术 .最后 ,展望了架空线路巡线机器人的发展趋势和应用前景 .     

基于声纳的移动机器人沿墙导航控制

针对移动机器人提出了一种沿墙导航控制算法．算法首先对室内环境中墙的形状进行分类,并针对每一类型的墙设计相应的控制策略．然后在移动机器人运动过程中,基于有限状态机实现移动机器人沿墙状态的转移,从而使移动机器人采用不同的控制策略控制其运动．文中利用Pioneer 2DX移动机器人对此算法进行了实验研究,取得了理想的效果．     

Adaptive Neuro-fuzzy Network Control for a Mobile Robot

A control structure that makes possible the integration of a kinematic controller and a neuro-fuzzy network (NFN) dynamic controller for mobile robots is presented. A combined kinematic/dynamic control law is developed using backstepping and stability is guaranteed by Lyapunov theory. The NFN controller proposed in this work can deal with unmodeled bounded disturbances and/or unstructured unmodeled dynamic in the mobile robot. On-line NFN parameter tuning algorithms do no require off-line learning yet guarantee small tracking errors and bounded control signals are utilized.     

Evolutionary Learning of a Fuzzy Behavior Based Controller for a Nonholonomic Mobile Robot in a Class of Dynamic Environments

This paper presents an approach for evolving optimum behaviors for a nonholonomic mobile robot in a class of dynamic environments. A new evolutionary algorithm reflecting some powerful features in the natural evolutionary process to have flexibility to deal with changes in the environment is used to evolve optimum behaviors. Furthermore, a fuzzy set based multi-objective fitness evaluation function is adopted in the evolutionary algorithm. The multi-objective evaluation function is designed so that it allows incorporating complex linguistic features that a human observer would desire in the behaviors of the mobile robot movements. To illustrate the effectiveness of the proposed method, simulation results are compared using a conventional evolutionary algorithm.