基于深度强化学习的移动机器人动态路径规划算法

为了在复杂舞台环境下使用移动机器人实现物品搬运或者载人演出,提出了一种基于深度强化学习的动态路径规划算法。首先通过构建全局地图获取移动机器人周围的障碍物信息,将演员和舞台道具分别分类成动态障碍物和静态障碍物。然后建立局部地图,通过LSTM网络编码动态障碍物信息,使用社会注意力机制计算每个动态障碍物的重要性来实现更好的避障效果。通过构建新的奖励函数来实现对动静态障碍物的不同躲避情况。最后通过模仿学习和优先级经验回放技术来提高网络的收敛速度,从而实现在舞台复杂环境下的移动机器人的动态路径规划。实验结果表明,该网络的收敛速度明显提高,在不同障碍物环境下都能够表现出好的动态避障效果。     

无人机反应式扰动流体路径规划

针对复杂三维障碍环境,提出一种基于深度强化学习的无人机(Unmanned aerial vehicles, UAV)反应式扰动流体路径规划架构.该架构以一种受约束扰动流体动态系统算法作为路径规划的基本方法,根据无人机与各障碍的相对状态以及障碍物类型,通过经深度确定性策略梯度算法训练得到的动作网络在线生成对应障碍的反应系数和方向系数,继而可计算相应的总和扰动矩阵并以此修正无人机的飞行路径,实现反应式避障.此外,还研究了与所提路径规划方法相适配的深度强化学习训练环境规范性建模方法.仿真结果表明,在路径质量大致相同的情况下,该方法在实时性方面明显优于基于预测控制的在线路径规划方法.     

INTELLIGENT SENSOR FUSION AND LEARNING FOR AUTONOMOUS ROBOT NAVIGATION

ABSTRACT

This paper presents the design and implementation of an autonomous robot navigation system for intelligent target collection in dynamic environments. A feature-based multi-stage fuzzy logic (MSFL) sensor fusion system is developed for target recognition, which is capable of mapping noisy sensor inputs into reliable decisions. The robot exploration and path planning are based on a grid map oriented reinforcement path learning system (GMRPL), which allows for long-term predictions and path adaptation via dynamic interactions with physical environments. In our implementation, the MSFL and GMRPL are integrated into subsumption architecture for intelligent target-collecting applications. The subsumption architecture is a layered reactive agent structure that enables the robot to implement higher-layer functions including path learning and target recognition regardless of lower-layer functions such as obstacle detection and avoidance. The real-world application using a Khepera robot shows the robustness and flexibility of the developed system in dealing with robotic behaviors such as target collecting in the ever-changing physical environment.     

融合动态障碍物运动信息的路径规划算法

传统的路径规划算法只能在障碍物不发生位置变化的环境中计算最优路径。但是随着机器人在商场、医院、银行等动态环境下的普及,传统的路径规划算法容易与动态障碍物发生碰撞等危险。因此,关于随机动态障碍物条件下的机器人路径规划算法需要得到进一步改善。为了解决在动态环境下的机器人路径规划问题,提出了一种融合机器人与障碍物运动信息的改进动态窗口法来解决机器人在动态环境下的局部路径规划问题,并且与优化A*算法相结合来实现全局最优路径规划。主要内容体现为：在全局路径规划上,采用优化A*算法求解最优路径。在局部路径规划上,以动态障碍物的速度作为先验信息,通过对传统动态窗口法的评价函数进行扩展,实现机器人在动态环境下的自主智能避障。实验证明,该算法可以实现基于全局最优路径的实时动态避障,具体表现为可以在不干涉动态障碍物的条件下减少碰撞风险、做出智能避障且路径更加平滑、长度更短、行驶速度更快。     

基于障碍物运动预测的移动机器人路径规划

针对移动机器人局部动态避障路径规划问题开展优化研究。基于动态障碍物当前历史位置轨迹,提出动态障碍物运动趋势预测算法。在移动机器人的动态避障路径规划过程中,考虑障碍物当前的位置,评估动态障碍物的移动轨迹;提出改进的D*Lite路径规划算法,大幅提升机器人动态避障算法的效率与安全性。搭建仿真验证环境,给出典型的单动态障碍物、多动态障碍物场景,对比验证了避障路径规划算法的有效性。     

人群环境中基于深度强化学习的移动机器人避障算法

为了控制移动机器人在人群密集的复杂环境中高效友好地完成避障任务,本文提出了一种人群环境中基于深度强化学习的移动机器人避障算法。首先,针对深度强化学习算法中值函数网络学习能力不足的情况,基于行人交互（crowd interaction）对值函数网络做了改进,通过行人角度网格（angel pedestrian grid）对行人之间的交互信息进行提取,并通过注意力机制（attention mechanism）提取单个行人的时序特征,学习得到当前状态与历史轨迹状态的相对重要性以及对机器人避障策略的联合影响,为之后多层感知机的学习提供先验知识;其次,依据行人空间行为（human spatial behavior）设计强化学习的奖励函数,并对机器人角度变化过大的状态进行惩罚,实现了舒适避障的要求;最后,通过仿真实验验证了人群环境中基于深度强化学习的移动机器人避障算法在人群密集的复杂环境中的可行性与有效性。     

融合改进A*算法和Morphin算法的移动机器人动态路径规划

在动态未知环境下对机器人进行路径规划,传统A*算法可能出现碰撞或者路径规划失败问题。为了满足移动机器人全局路径规划最优和实时避障的需求,提出一种改进A*算法与Morphin搜索树算法相结合的动态路径规划方法。首先通过改进A*算法减少路径规划过程中关键节点的选取,在规划出一条全局较优路径的同时对路径平滑处理。然后基于移动机器人传感器采集的局部信息,利用Morphin搜索树算法对全局路径进行动态的局部规划,确保更好的全局路径的基础上,实时避开障碍物行驶到目标点。MATLAB仿真实验结果表明,提出的动态路径规划方法在时间和路径上得到提升,在优化全局路径规划的基础上修正局部路径,实现动态避障提高机器人达到目标点的效率。     

On-line Planning for Collision Avoidance on the Nominal Path

In this paper a solution to the obstacle avoidance problem for a mobile robot moving in the two-dimensional Cartesian plane is presented. The robot is modelled as a linear time-invariant dynamic system of finite size enclosed by a circle and the obstacles are modelled as circles travelling along rectilinear trajectories. This work deals with the avoidance problem when the obstacles move in known trajectories. The robot starts its journey on a nominal straight line path with a nominal velocity. When an obstacle is detected to be on a collision course with the robot, the robot must devise a plan to avoid the obstacle whilst minimising a cost index defined as the total sum squared of the magnitudes of the deviations of its velocity from the nominal velocity. The planning strategy adopted here is adjustment of the robot's velocity on the nominal path based on the time of collision between the robot and a moving obstacle, and determination of a desired final state such that its Euclidean distance from the nominal final state is minimal. Obstacle avoidance by deviation from the nominal path in deterministic and random environments is based on the work presented here and is investigated in another paper.     

基于几何法的移动机器人路径规划

旨在解决动态环境中移动机器人与障碍物发生碰撞可能性的判断和避开障碍的路径规划。提出了采用几何计算的方法判断机器人和障碍物之间发生碰撞的条件,规划出机器人沿着收敛曲线运动到安全圆周,在安全圆周上作动态圆周运动,最后沿着圆弧退出圆周到达预定的避障路径。将基本的避开障碍的理论和几何算法有机地结合起来,获得了光滑的路径,提高了机器人避开障碍的效率。     

基于自适应动态窗口改进细菌算法与移动机器人路径规划

针对移动机器人在复杂环境下的路径规划问题,提出一种新的自适应动态窗口改进细菌算法,并将新算法应用于移动机器人路径规划。改进细菌算法继承了细菌算法与动态窗口算法（dynamic window algorithm, DWA）在避障时的优点,能较好实现复杂环境中移动机器人静态和动态避障。该改进算法主要分三步完成移动机器人路径规划。首先,利用改进细菌趋化算法在静态环境中得到初始参考规划路径。接着,基于参考路径,机器人通过自身携带的传感器感知动态障碍物进行动态避障并利用自适应DWA完成局部动态避障路径规划。最后,根据移动机器人局部动态避障完成情况选择算法执行步骤,如果移动机器人能达到最终目标点,结束该算法,否则移动机器人再重回初始路径,直至到达最终目标点。仿真比较实验证明,改进算法无论在收敛速度还是路径规划精确度方面都有明显提升。     

动态未知环境下一种新的机器人路径规划方法

针对机器人动态路径规划问题,提出了一种机器人在复杂动态环境中实时路径规划方法.该方法基于滚动窗口的路径规划和避障策略,通过设定可视点子目标、绕行障碍物和对动态障碍物的分析预测,实现机器人在复杂动态环境下的路径规划.针对障碍物分布情况,合理设计可视点法和绕行算法之间转换,有效地解决了局部路径规划的死循环与极小值问题.该方...     

An autonomous fuzzy mobile robot

Autonomous mobile robots should have the capability of recognizing their environments and manoeuvring through those environments on the basis of their own judgement. Fuzzy control is suitable for autonomous mobile robot control where the amount of information to be handled is limited as much as possible and the processing is simple. Autonomous mobile control of a robot is derived from two kinds of controls: for obstacle avoidance and for guidance following an appropriate path to a destination point. Fuzzy control of a robot for obstacle avoidance based on finding permissible passageways using the edges between the floor and the wall or obstacles obtained by processing the image from a CCD camera in front of the robot is developed. Furthermore, guidance control of the robot over paths that are specified in terms of maps may be developed by a process that treats a wrong path as a virtual obstacle on the screen, and the robot advances in the designated direction when it reaches intersections. An autonomous fuzzy robot based on the above method is fabricated as a trial and its usefulness is demonstrated.     

融合改进A*与DWA算法的移动机器人路径规划

针对移动机器人在复杂环境下实现全局路径最优、未知环境下动态实时避障这一路径规划需求,对传统A*（A-star）算法进行改进,并融合动态窗口法（DWA）实现动态实时避障。首先分析栅格环境下的障碍物占比,将障碍物占比引入传统A*算法,优化启发函数h(n),从而改进评价函数f(n),提高其在不同环境下的搜索效率;其次针对复杂栅格环境下传统A*算法优化后的轨迹与障碍物顶点相交问题,优化子节点选择方式,同时删除路径中的冗余节点,提高路径的平滑度;最后融合动态窗口法,实现复杂环境下移动机器人的动态实时避障。通过MATLAB下的对比仿真实验表明,改进算法在轨迹长度、轨迹平滑度以及历经时间上得到优化,满足全局最优且能实现动态实时避障,具有更优秀的路径规划效果。     

Reactive navigation in dynamic environment using a multisensorpredictor

A reactive navigation system for an autonomous mobile robot in unstructured dynamic environments is presented. The motion of moving obstacles is estimated for robot motion planning and obstacle avoidance. A multisensor-based obstacle predictor is utilized to obtain obstacle-motion information. Sensory data from a CCD camera and multiple ultrasonic range finders are combined to predict obstacle positions at the next sampling instant. A neural network, which is trained off-line, provides the desired prediction on-line in real time. The predicted obstacle configuration is employed by the proposed virtual force based navigation method to prevent collision with moving obstacles. Simulation results are presented to verify the effectiveness of the proposed navigation system in an environment with multiple mobile robots or moving objects. This system was implemented and tested on an experimental mobile robot at our laboratory. Navigation results in real environment are presented and analyzed.     

Destination Driven Motion Planning via Obstacle Motion Prediction and Multi-State Path Repair

This paper presents a real-time navigating system named Destination Driven Navigator for a mobile robot operating in unstructured static and dynamic environments. We have designed a new obstacle representation method named Cross-Line Obstacle Representation and a new concept work space to reduce the robot's search space and the environment storage cost, an Adapted Regression Model to predict dynamic obstacles' motion, Multi-State Path Repair rules to quickly translate an infeasible path into feasible one, and the path-planning algorithm to generate a path. A high-level Destination Driven Navigator uses these methods, models and algorithms to guide a mobile robot traveling in various environments while avoiding static and dynamic obstacles. A group of experiments has been conducted. The results exhibit that the Destination Driven Navigator is a powerful and effective paradigm for robot motion planning and obstacle avoidance.     

Learning Dynamic Obstacle Avoidance for a Robot Arm Using Neuroevolution

Neural networks can be evolved to control robot manipulators in tasks like target tracking and obstacle avoidance in complex environments. Neurocontrollers are robust to noise and can be adapted to different environments and robot configurations. In this paper, neurocontrollers were evolved to position the end effector of a robot arm close to a target in three different environments: environments without obstacles, environments with stationary obstacles, and environments with moving obstacles. The evolved neurocontrollers perform qualitatively like inverse kinematic controllers in environments with no obstacles and like path-planning controllers based on Rapidly-exploring random trees in environments with obstacles. Unlike inverse kinematic controllers and path planners, the approach reliably generalizes to environments with moving obstacles, making it possible to use it in natural environments.     

New technique of mobile robot navigation using a hybrid adaptive fuzzy potential field approach

This paper presents a summary of the research aimed at developing a new reliable methodology for robot navigation and obstacle avoidance. This new approach is based on the artificial potential field (APF) method, which is used extensively for obstacle avoidance. The classical APF is dependent only on the separation distance between the robot and the surrounding obstacles. The new scheme introduces a variable, which is used to determine the importance that each obstacle has on the robot's future path. The importance variable is dependent on the obstacles position, both angle and distance, with respect to the robot. Simulation results are presented demonstrating the ability of the algorithm to perform successfully in simple environments.     

Obstacle avoidance of redundant manipulators using neural networks based reinforcement learning

This paper proposes a new approach for solving the problem of obstacle avoidance during manipulation tasks performed by redundant manipulators. The developed solution is based on a double neural network that uses Q-learning reinforcement technique. Q-learning has been applied in robotics for attaining obstacle free navigation or computing path planning problems. Most studies solve inverse kinematics and obstacle avoidance problems using variations of the classical Jacobian matrix approach, or by minimizing redundancy resolution of manipulators operating in known environments. Researchers who tried to use neural networks for solving inverse kinematics often dealt with only one obstacle present in the working field. This paper focuses on calculating inverse kinematics and obstacle avoidance for complex unknown environments, with multiple obstacles in the working field. Q-learning is used together with neural networks in order to plan and execute arm movements at each time instant. The algorithm developed for general redundant kinematic link chains has been tested on the particular case of PowerCube manipulator. Before implementing the solution on the real robot, the simulation was integrated in an immersive virtual environment for better movement analysis and safer testing. The study results show that the proposed approach has a good average speed and a satisfying target reaching success rate.     

基于双粒子群算法的矿井搜救机器人路径规划

针对在复杂地形中标准的粒子群算法用于矿井搜救机器人路径规划存在迭代速度慢和求解精度低的问题,提出了一种基于双粒子群算法的矿井搜救机器人路径规划方法。首先将障碍物膨胀化处理为规则化多边形,以此建立环境模型,再以改进双粒子群算法作为路径寻优算法,当传感器检测到搜救机器人正前方一定距离内有障碍物时,开始运行双改进粒子群算法:改进学习因子的粒子群算法(CPSO)粒子步长大,适用于相对开阔地带寻找路径,而添加动态速度权重的粒子群算法(PPSO)粒子步长小,擅长在障碍物形状复杂多变地带寻找路径;然后评估2种粒子群算法得到的路径是否符合避障条件,若均符合避障条件,则选取最短路径作为最终路径;最后得到矿井搜救机器人在整个路况模型中的最优行驶路径。仿真结果表明,通过改进学习因子和添加动态速度权重提高了粒子群算法的收敛速度,降低了最优解波动幅度,改进的双粒子群算法能够与路径规划模型有效结合,在复杂路段能够寻找到最优路径,提高了路径规划成功率,缩短了路径长度。     

Velocity potential approach to path planning for avoiding moving obstacles

This paper describes the theory and an experiment of a velocity potential approach to path planning and avoiding moving obstacles for an autonomous mobile robot by use of the Laplace potential. This new navigation function for path planning is feasible for guiding a mobile robot avoiding arbitrarily moving obstacles and reaching the goal in real time. The essential feature of the navigation function comes from the introduction of fluid flow dynamics into the path planning. The experiment is conducted to verify the effectiveness of the navigation function for obstacle avoidance in a real world. Two examples of the experiment are presented; first, the avoidance of a moving obstacle in parallel line-bounded space, and second, the avoidance of one moving obstacle and another standing obstacle. The robot can reach the goal after successfully avoiding the obstacles in these cases.