State-chain sequential feedback reinforcement learning for path planning of autonomous mobile robots

This paper deals with a new approach based on Q-learning for solving the problem of mobile robot path planning in complex unknown static environments.As a computational approach to learning through interaction with the environment,reinforcement learning algorithms have been widely used for intelligent robot control,especially in the field of autonomous mobile robots.However,the learning process is slow and cumbersome.For practical applications,rapid rates of convergence are required.Aiming at the problem of slow convergence and long learning time for Q-learning based mobile robot path planning,a state-chain sequential feedback Q-learning algorithm is proposed for quickly searching for the optimal path of mobile robots in complex unknown static environments.The state chain is built during the searching process.After one action is chosen and the reward is received,the Q-values of the state-action pairs on the previously built state chain are sequentially updated with one-step Q-learning.With the increasing number of Q-values updated after one action,the number of actual steps for convergence decreases and thus,the learning time decreases,where a step is a state transition.Extensive simulations validate the efficiency of the newly proposed approach for mobile robot path planning in complex environments.The results show that the new approach has a high convergence speed and that the robot can find the collision-free optimal path in complex unknown static environments with much shorter time,compared with the one-step Q-learning algorithm and the Q(λ)-learning algorithm.     

基于深度Q网络和人工势场的移动机器人路径规划研究

随着移动机器人在各个领域的研究与发展,人们对移动机器人路径规划的能力提出了更高的要求;为了解决传统的深度Q网络算法在未知环境下,应用于自主移动机器人路径规划时存在的收敛速度慢、训练前期产生较大迭代空间、迭代的次数多等问题,在传统DQN算法初始化Q值时,加入人工势场法的引力势场来协助初始化环境先验信息,进而可以引导移动机器人向目标点运动,来减少算法在最初几轮探索中形成的大批无效迭代,进而减少迭代次数,加快收敛速度;在栅格地图环境中应用pytorch框架验证加入初始引力势场的改进DQN算法路径规划效果;仿真实验结果表明,改进算法能在产生较小的迭代空间且较少的迭代次数后,快速有效地规划出一条从起点到目标点的最优路径。     

Dyna-Q-based vector direction for path planning problem of autonomous mobile robots in unknown environments

Reinforcement learning (RL) is a popular method for solving the path planning problem of autonomous mobile robots in unknown environments. However, the primary difficulty faced by learning robots using the RL method is that they learn too slowly in obstacle-dense environments. To more efficiently solve the path planning problem of autonomous mobile robots in such environments, this paper presents a novel approach in which the robot’s learning process is divided into two phases. The first one is to accelerate the learning process for obtaining an optimal policy by developing the well-known Dyna-Q algorithm that trains the robot in learning actions for avoiding obstacles when following the vector direction. In this phase, the robot’s position is represented as a uniform grid. At each time step, the robot performs an action to move to one of its eight adjacent cells, so the path obtained from the optimal policy may be longer than the true shortest path. The second one is to train the robot in learning a collision-free smooth path for decreasing the number of the heading changes of the robot. The simulation results show that the proposed approach is efficient for the path planning problem of autonomous mobile robots in unknown environments with dense obstacles.     

不确定动态环境下移动机器人的完全遍历路径规划

基于生物激励神经网络、滚动窗口和启发式搜索,提出了一种新的完全遍历路径规划方法．该方法用Grossberg的生物神经网络实现移动机器人的局部环境建模,将滚动窗口的概念引入到局部路径规划,由启发式算法决定滚动窗口内的局域路径规划目标．该方法能在不确定动态环境中有效地实现机器人自主避障的完全遍历路径规划．仿真研究证明了该方法的可用性和有效性．     

基于深度强化学习的移动机器人动态路径规划算法

为了在复杂舞台环境下使用移动机器人实现物品搬运或者载人演出,提出了一种基于深度强化学习的动态路径规划算法。首先通过构建全局地图获取移动机器人周围的障碍物信息,将演员和舞台道具分别分类成动态障碍物和静态障碍物。然后建立局部地图,通过LSTM网络编码动态障碍物信息,使用社会注意力机制计算每个动态障碍物的重要性来实现更好的避障效果。通过构建新的奖励函数来实现对动静态障碍物的不同躲避情况。最后通过模仿学习和优先级经验回放技术来提高网络的收敛速度,从而实现在舞台复杂环境下的移动机器人的动态路径规划。实验结果表明,该网络的收敛速度明显提高,在不同障碍物环境下都能够表现出好的动态避障效果。     

异构多目标差分-动态窗口算法 及其在移动机器人中的应用

为了实现在多移动机器人和多窄通道的复杂动态环境中机器人的节能运动规划,提出异构多目标差分-动态窗口法(heterogeneous multi-objective differential evolution-dynamic window algorithm,HMODE-DWA).首先,建立行驶时间、执行器作用力和平滑度的3目标优化模型,设计具有碰撞约束的异构多目标差分进化算法来获得3个目标函数的最优解,进而在已知的静态环境中获得帕累托前沿,利用平均隶属度函数获得起点与终点间最优的全局路径;其次,定义基于环境缓冲区域的模糊动态窗口法使机器人完成动态复杂环境中避障,利用所提出的HMODE-DWA算法动态避障的同时实现节能规划.仿真和实验结果表明,所提出的混合路径规划控制策略能够有效降低移动机器人动态避障过程中的能耗.     

基于几何法的移动机器人路径规划

旨在解决动态环境中移动机器人与障碍物发生碰撞可能性的判断和避开障碍的路径规划。提出了采用几何计算的方法判断机器人和障碍物之间发生碰撞的条件,规划出机器人沿着收敛曲线运动到安全圆周,在安全圆周上作动态圆周运动,最后沿着圆弧退出圆周到达预定的避障路径。将基本的避开障碍的理论和几何算法有机地结合起来,获得了光滑的路径,提高了机器人避开障碍的效率。     

基于自适应动态窗口改进细菌算法与移动机器人路径规划

针对移动机器人在复杂环境下的路径规划问题,提出一种新的自适应动态窗口改进细菌算法,并将新算法应用于移动机器人路径规划。改进细菌算法继承了细菌算法与动态窗口算法（dynamic window algorithm, DWA）在避障时的优点,能较好实现复杂环境中移动机器人静态和动态避障。该改进算法主要分三步完成移动机器人路径规划。首先,利用改进细菌趋化算法在静态环境中得到初始参考规划路径。接着,基于参考路径,机器人通过自身携带的传感器感知动态障碍物进行动态避障并利用自适应DWA完成局部动态避障路径规划。最后,根据移动机器人局部动态避障完成情况选择算法执行步骤,如果移动机器人能达到最终目标点,结束该算法,否则移动机器人再重回初始路径,直至到达最终目标点。仿真比较实验证明,改进算法无论在收敛速度还是路径规划精确度方面都有明显提升。     

Collision-free motion planning of dual-arm reconfigurable robots

Dual-arm reconfigurable robot is a new type of robot. It can adapt to different tasks by changing its different end-effector modules which have standard connectors. Especially, in fast and flexible assembly, it is very important to research the collision-free planning of dual-arm reconfigurable robots. It is to find a continuous, collision-free path in an environment containing obstacles. A new approach to the real-time collision-free motion planning of dual-arm reconfigurable robots is used in the paper. This method is based on configuration space (C-Space). The method of configuration space and the concepts reachable manifold and contact manifold are successfully applied to the collision-free motion planning of dual-arm robot. The complexity of dual-arm robots’ collision-free planning will reduce to a search in a dispersed C-Space. With this algorithm, a real-time optimum path is found. And when the start point and the end point of the dual-arm robot are specified, the algorithm will successfully get the collision-free path real time. A verification of this algorithm is made in the dual-arm horizontal articulated robot SCARATES, and the simulation and experiment ascertain that the algorithm is feasible and effective.     

移动机器人路径规划综述

移动机器人是目前科学技术发展最活跃的领域之一,在工业、农业、医疗等行业广泛应用,还在城市安全、国防和空间探测领域得到更广的应用。要实现机器人在未知环境下自主作业,具备实时、自主、识别高风险区域和风险管理的能力,路径规划是一个重要环节,规划水平的高低,在一定程度上标志着机器人的智能水平,因此研究机器人路径规划对提高机器人的智能化水平、加快工程化应用具有重要的意义。文章重点分别从全局路径规划和局部路径规划角度对机器人路径规划的研究方法进行了分析与总结,同时分析研究了基于仿生学的智能算法的遗传算法、蚁群算法、粒子群算法,最后展望了移动机器人的未来发展趋势。     

基于改进蚁群算法的全向移动机器人全遍历路径规划

受全遍历环境影响, 现有方法规划得出的路径长度过长, 为提高路径规划性能, 获取最优路径, 提出基于改进蚁群算法的全向移动机器人全遍历路径规划方法. 在拓扑建模示意图的基础上, 依据移动机器人在原坐标系下的位置信息, 利用角度转换建立新的环境模型. 考虑蚁群算法存在的问题, 将递减系数引入到启发函数中, 更新局部信息素, 通过设定迭代阈值, 调节信息素的挥发系数. 最后通过路径规划流程设计, 实现对全向移动机器人全遍历路径的规划. 实验结果表明, 所设计方法不仅可以缩短全遍历路径长度, 还可以缩短路径规划时间, 获取最优路径, 从而提高了全向移动机器人的全遍历路径规划性能.     

一种基于粒子群算法的移动机器人路径规划方法*

针对结构化环境中移动机器人路径规划问题,提出一种基于粒子群的路径规划算法.该算法利用适应度函数描述环境约束及路径的距离信息,适应度函数通过神经网络计算;由路径节点构成粒子,通过混合粒子群算法进行寻优.最后,通过计算机仿真验证了该算法是合理的,并且可应用于机器人的实时导航.     

Deliberative On-Line Local Path Planning for Autonomous Mobile Robots

This paper describes a method for local path planning for mobile robots that combines reactive obstacle avoidance with on-line local path planning. Our approach is different to other model-based navigation approaches since it integrates both global and local planning processes in the same architecture while other methods only combine global path planning with a reactive method to avoid non-modelled obstacles. Our local planning is only triggered when an unexpected obstacle is found and reactive navigation is not able to regain the initial path. A new trajectory is then calculated on-line using only proximity sensor information. This trajectory can be improved during the available time using an anytime algorithm. The proposed method complements the reactive behaviour and allows the robot to navigate safely in a partially known environment during a long time period without human intervention.     

一种移动机器人路径规划新算法

快速搜索随机树（Rapidly-exploring random Tree Star,RRT*）算法在移动机器人实际应用中规划路径在转向部分存在较多的冗余转折点,导致移动机器人在移动转向过程中出现多次停顿与转向,为剔除规划路径中的冗余路径点,提高机器人移动流畅性,提出一种改进的 RRT*算法。算法将局部逆序试连法引入移动机器人路径规划,在确保RRT*算法概率完备性和渐进最优性的前提下,剔除规划路径中的冗余路径节点,使最终路径更加接近最短路径。通过MATLAB仿真实验证明,规划路径平均长度缩短4%,算法耗时缩短35%,改进后的RRT*算法能缩短规划路径且转向部分路径更加平滑。最后,使用改进后的RRT*算法在室内环境下进行移动机器人路径规划实验。实验结果表明：规划路径上无冗余路径点,且移动机器人沿路径移动流畅。     

引入势场及陷阱搜索的强化学习路径规划算法

移动机器人在复杂环境中移动难以得到较优的路径,基于马尔可夫过程的Q学习（Q-learning）算法能通过试错学习取得较优的路径,但这种方法收敛速度慢,迭代次数多,且试错方式无法应用于真实的环境中。在Q-learning算法中加入引力势场作为初始环境先验信息,在其基础上对环境进行陷阱区域逐层搜索,剔除凹形陷阱区域[Q]值迭代,加快了路径规划的收敛速度。同时取消对障碍物的试错学习,使算法在初始状态就能有效避开障碍物,适用于真实环境中直接学习。利用python及pygame模块建立复杂地图,验证加入初始引力势场和陷阱搜索的改进Q-learning算法路径规划效果。仿真实验表明,改进算法能在较少的迭代次数后,快速有效地到达目标位置,且路径较优。     

基于再励学习的多移动机器人协调避障路径规划方法

随着多移动机器人协调系统的应用向未知环境发展,一些依赖于环境模型的路径规划方法不再适用。而利用再励学习与环境直接交互,不需要先验知识和样本数据的特点,该文将再励学习应用于多机器人协调系统中,提出了基于再励学习的避障路径规划方法,并将再励函数设计为基于行为分解的无模型非均匀结构。计算机仿真实验结果表明该方法有效,并有较好的鲁棒性,新的再励函数结构使得学习速度得以提高。     

Efficient collision-free path-planning of multiple mobile robots system using efficient artificial bee colony algorithm

This paper aims to propose a novel design approach for on-line path planning of the multiple mobile robots system with free collision. Based on the artificial bee colony (ABC) algorithm, we propose an efficient artificial bee colony (EABC) algorithm for solving the on-line path planning of multiple mobile robots by choosing the proper objective function for target, obstacles, and robots collision avoidance. The proposed EABC algorithm enhances the performance by using elite individuals for preserving good evolution, the solution sharing provides a proper direction for searching, the instant update strategy provides the newest information of solution. By the proposed approach, the next positions of each robot are designed. Thus, the mobiles robots can travel to the designed targets without collision. Finally, simulation results of illustration examples are introduced to show the effectiveness and performance of the proposed approach.     

基于生物认知的移动机器人路径规划方法

针对移动机器人在非结构环境下的导航任务,受哺乳动物空间认知方式的启发,提出一种基于生物认知进行移动机器人路径规划的方法．结合认知地图特性,模拟海马体的情景记忆形成机理,构建封装了场景感知、状态神经元及位姿感知相关信息的情景认知地图,实现了机器人对环境的认知．基于情景认知地图,以最小事件距离为准则,提出事件序列规划算法用于实时导航过程．实验结果表明,该控制算法能使机器人根据不同任务选择最佳规划路径．     

人工免疫算法在机器人路径规划中的应用

人工免疫算法是一种新兴的优化方法,在计算、控制等各方面都已得到应用.将免疫算法应用于移动机器人路径规划,提出一种任意多边形障碍物复杂布局环境下的机器人路径规划的人工免疫算法,仿真证明该算法可以准确地找到全局最优路径,而且能够适应各种复杂的环境.     

直线拟合算法在移动机器人轨迹记录中的研究与应用

移动机器人在进行未知环境探索时,需要记录保存其行驶轨迹,以便进行定位、路径规划、地图更新等处理。本文给出一个直线拟合的优化算法,对移动机器人采集的原始坐标信息进行分析,可去除大量冗余信息,得到准确、精简、必要的运动轨迹信息。该算法在移动机器人轨迹记录中得到了有效应用,节省了存储空间,且满足移动机器人的实时性。仿真及实际测试均显示,该直线拟合优化算法是快速、高效的。