Multi-agent Path Planning with Heterogenous Interactions in Tight Spaces

By starting with the assumption that motion is fundamentally a decision making problem, we use the world-line concept from Special Relativity as the inspiration for a novel multi-agent path planning method. We have identified a particular set of problems that have so far been overlooked by previous works. We present our solution for the global path planning problem for each agent and ensure smooth local collision avoidance for each pair of agents in the scene. We accomplish this by modelling the collision-free trajectories of the agents through 2D space and time as rods in 3D. We obtain smooth trajectories by solving a non-linear optimization problem with a quasi-Newton interior point solver, initializing the solver with a non-intersecting configuration from a modified Dijkstra's algorithm. This space–time formulation allows us to simulate previously ignored phenomena such as highly heterogeneous interactions in very constrained environments. It also provides a solution for scenes with unnaturally symmetric agent alignments without the need for jittering agent positions or velocities.     

Interactive Navigation of Heterogeneous Agents Using Adaptive Roadmaps

We present a novel algorithm for collision-free navigation of a large number of independent agents in complex and dynamic environments. We introduce adaptive roadmaps to perform global path planning for each agent simultaneously. Our algorithm takes into account dynamic obstacles and interagents interaction forces to continuously update the roadmap based on a physically-based dynamics simulator. In order to efficiently update the links, we perform adaptive particle-based sampling along the links. We also introduce the notion of 'link bands' to resolve collisions among multiple agents. In practice, our algorithm can perform real-time navigation of hundreds and thousands of human agents in indoor and outdoor scenes.     

Topology-preserving flocking of nonlinear agents using optimistic planning

We consider the generalized flocking problem in multiagent systems, where the agents must drive a subset of their state variables to common values, while communication is constrained by a proximity relationship in terms of another subset of variables. We build a flocking method for general nonlinear agent dynamics, by using at each agent a near-optimal control technique from artificial intelligence called optimistic planning. By defining the rewards to be optimized in a well-chosen way, the preservation of the interconnection topology is guaranteed, under a controllability assumption. We also give a practical variant of the algorithm that does not require to know the details of this assumption, and show that it works well in experiments on nonlinear agents.     

Fast approximate clearance evaluation for rovers with articulated suspension systems

We present a light‐weight body‐terrain clearance evaluation algorithm for the automated path planning of NASA's Mars 2020 rover. Extraterrestrial path planning is challenging due to the combination of terrain roughness and severe limitation in computational resources. Path planning on cluttered and/or uneven terrains requires repeated safety checks on all the candidate paths at a small interval. Predicting the future rover state requires simulating the vehicle settling on the terrain, which involves an inverse‐kinematics problem with iterative nonlinear optimization under geometric constraints. However, such expensive computation is intractable for slow spacecraft computers, such as RAD750, which is used by the Curiosity Mars rover and upcoming Mars 2020 rover. We propose the approximate clearance evaluation (ACE) algorithm, which obtains conservative bounds on vehicle clearance, attitude, and suspension angles without iterative computation. It obtains those bounds by estimating the lowest and highest heights that each wheel may reach given the underlying terrain, and calculating the worst‐case vehicle configuration associated with those extreme wheel heights. The bounds are guaranteed to be conservative, hence ensuring vehicle safety during autonomous navigation. ACE is planned to be used as part of the new onboard path planner of the Mars 2020 rover. This paper describes the algorithm in detail and validates our claim of conservatism and fast computation through experiments.     

Live path: adaptive agent navigation in the interactive virtual world

We present a novel approach to adaptive navigation in the interactive virtual world by using data from the user. Our method constructs automatically a navigation mesh that provides new paths for agents by referencing the user movements. To acquire accurate data samples from all the user data in the interactive world, we use the following techniques: an agent of interest (AOI), a region of interest (ROI) map, and a discretized path graph (DPG). Our method enables adaptive changes to the virtual world over time and provides user-preferred path weights for smart-agent path planning. We have tested the usefulness of our algorithm with several example scenarios from interactive worlds such as video games. In practice, our framework can be applied easily to any type of navigation in an interactive world. In addition, it may prove useful for solving previous pathfinding problems in static navigation planning.     

Distributed learning and cooperative control for multi-agent systems

This paper presents an algorithm and analysis of distributed learning and cooperative control for a multi-agent system so that a global goal of the overall system can be achieved by locally acting agents. We consider a resource-constrained multi-agent system, in which each agent has limited capabilities in terms of sensing, computation, and communication. The proposed algorithm is executed by each agent independently to estimate an unknown field of interest from noisy measurements and to coordinate multiple agents in a distributed manner to discover peaks of the unknown field. Each mobile agent maintains its own local estimate of the field and updates the estimate using collective measurements from itself and nearby agents. Each agent then moves towards peaks of the field using the gradient of its estimated field while avoiding collision and maintaining communication connectivity. The proposed algorithm is based on a recursive spatial estimation of an unknown field. We show that the closed-loop dynamics of the proposed multi-agent system can be transformed into a form of a stochastic approximation algorithm and prove its convergence using Ljung’s ordinary differential equation (ODE) approach. We also present extensive simulation results supporting our theoretical results.     

Continual planning and acting in dynamic multiagent environments

In order to behave intelligently, artificial agents must be able to deliberatively plan their future actions. Unfortunately, realistic agent environments are usually highly dynamic and only partially observable, which makes planning computationally hard. For most practical purposes this rules out planning techniques that account for all possible contingencies in the planning process. However, many agent environments permit an alternative approach, namely continual planning, i.e. the interleaving of planning with acting and sensing. This paper presents a new principled approach to continual planning that describes why and when an agent should switch between planning and acting. The resulting continual planning algorithm enables agents to deliberately postpone parts of their planning process and instead actively gather missing information that is relevant for the later refinement of the plan. To this end, the algorithm explictly reasons about the knowledge (or lack thereof) of an agent and its sensory capabilities. These concepts are modelled in the planning language (MAPL). Since in many environments the major reason for dynamism is the behaviour of other agents, MAPL can also model multiagent environments, common knowledge among agents, and communicative actions between them. For Continual Planning, MAPL introduces the concept of of assertions, abstract actions that substitute yet unformed subplans. To evaluate our continual planning approach empirically we have developed MAPSIM, a simulation environment that automatically builds multiagent simulations from formal MAPL domains. Thus, agents can not only plan, but also execute their plans, perceive their environment, and interact with each other. Our experiments show that, using continual planning techniques, deliberate action planning can be used efficiently even in complex multiagent environments.     

基于全局路径规划的相互速度障碍物人群疏散方法

针对相互速度障碍物（RVO）模型缺少全局路径规划,只依靠局部碰撞避免不能很好地模拟复杂的疏散场景问题,提出了一种剩余路径代价尽量小的动态全局路径选择方法。该方法包含路径预处理和路径实时更新两部分:第一部分使用快速最短路径算法（SPFA）求取场景最短路径（SSP）;第二部分根据SSP快速动态地计算每个个体的最优疏散路径,并使用KD树优化障碍物阻挡判断过程。最后将方法扩展到多楼层、多障碍物、多通道、多出口的复杂场景实现了近千人的仿真实验。实验结果表明,该方法在多个场景中都取得了良好的路径规划效果。     

A hybrid cognitive/reactive intelligent agent autonomous path planning technique in a networked-distributed unstructured environment for reinforcement learning

This paper proposes a path planning technique for autonomous agent(s) located in an unstructured networked distributed environment, where each agent has limited and not complete knowledge of the environment. Each agent has only the knowledge available in the distributed memory of the computing node the agent is running on and the agents share some information learned over a distributed network. In particular, the environment is divided into several sectors with each sector located on a single separate distributed computing node. We consider hybrid reactive-cognitive agent(s) where we use autonomous agent motion planning that is based on the use of a potential field model accompanied by a reinforcement learning as well as boundary detection algorithms. Potential fields are used for fast convergence toward a path in a distributed environment while reenforcement learning is used to guarantee a variety of behavior and consistent convergence in a distributed environment. We show how the agent decision making process is enhanced by the combination of the two techniques in a distributed environment. Furthermore, path retracing is a challenging problem in a distributed environment, since the agent does not have complete knowledge of the environment. We propose a backtracking technique to keep the distributed agent informed all the time of its path information and step count including when migrating from one node to another. Note that no node has knowledge of the entire global path from a source to a goal when such a goal resides on a separate node. Each agent has only knowledge of a partial path (internal to a node) and related number of steps corresponding to the portion of the path that agent traversed when running on the node. In particular, we show how each of the agents(s), starting in one of the many sectors with no initial knowledge of the environment, using the proposed distributed technique, develops its intelligence based on its experience and seamlessly discovers the shortest global path to the target, which is located in a different node, while avoiding any obstacle(s) it encounters in its way, including when transitioning and migrating from one distributed computing node to another. The agent(s) use (s) multiple-token-ring message passing interface (MPI) to perform internode communication. Finally, the experimental results of the proposed method show that single and multiagents sharing the same goal and running on the same or different nodes successfully coordinate the sharing of their respective environment states/information to collaboratively perform their respective tasks. The results also show that distributed multiagent sharing information increases by an order of magnitude the speed of convergence to the optimal shortest path to the goal in comparison with the single-agent case or noninformation sharing multiagent case.     

Bounded approximate decentralised coordination via the max-sum algorithm

In this paper we propose a novel approach to decentralised coordination, that is able to efficiently compute solutions with a guaranteed approximation ratio. Our approach is based on a factor graph representation of the constraint network. It builds a tree structure by eliminating dependencies between the functions and variables within the factor graph that have the least impact on solution quality. It then uses the max-sum algorithm to optimally solve the resulting tree structured constraint network, and provides a bounded approximation specific to the particular problem instance. In addition, we present two generic pruning techniques to reduce the amount of computation that agents must perform when using the max-sum algorithm. When this is combined with the above mentioned approximation algorithm, the agents are able to solve decentralised coordination problems that have very large action spaces with a low computation and communication overhead. We empirically evaluate our approach in a mobile sensor domain, where mobile agents are used to monitor and predict the state of spatial phenomena (e.g., temperature or gas concentration). Such sensors need to coordinate their movements with their direct neighbours to maximise the collective information gain, while predicting measurements at unobserved locations. When applied in this domain, our approach is able to provide solutions which are guaranteed to be within 2% of the optimal solution. Moreover, the two pruning techniques are extremely effective in decreasing the computational effort of each agent by reducing the size of the search space by up to 92%.     

Space‐time planning in changing environments : using dynamic objects for accessibility

Navigation is a critical task for agents populating virtual worlds. In the last years, numerous solutions have been proposed to solve the path planning problem in order to enhance the autonomy of virtual agents. Those solutions mainly focused on static environments, eventually populated with dynamic obstacles. However, dynamic objects are usually more than just obstacles as they can be used by an agent to reach new locations. In this paper, we propose an online path planning algorithm in dynamically changing environments with unknown evolution such as physically based‐environments. Our method represents objects in terms of obstacles but also in terms of navigable surfaces. This representation allows our algorithm to find temporal paths through disconnected and moving platforms. We will also show that the proposed method also enables several kinds of adaptations such as avoiding moving obstacles or adapting the agent postures to environmental constraints. Copyright © 2012 John Wiley & Sons, Ltd.     

Planning multiple paths with evolutionary speciation

This paper demonstrates a new approach to multidimensional path planning that is based on multiresolution path representation, where explicit configuration space computation is not required, and incorporates an evolutionary algorithm for solving the multimodal optimization problem, generating multiple alternative paths simultaneously. The multiresolution path representation reduces the expected search length for the path-planning problem and accordingly reduces the overall computational complexity. Resolution independent constraints due to obstacle proximity and path length are introduced into the evaluation function. The system can be applied for planning paths for mobile robots, assembly, and articulated manipulators. The resulting path-planning system has been evaluated on problems of two, three, four, and six degrees of freedom. The resulting paths are practical, consistent, and have acceptable execution times. The multipath algorithm is demonstrated on a number of 2D path-planning problems     

基于插值A*算法的路径规划

提出一个基于插值的路径规划算法-插值A*.此算法可以在每个栅格路径代价不一致的情况下生成一条平滑路径.由于大多数基于栅格算法规划的路径只能从一个栅格中心到另一栅格中心,也就限制了路径的方向只能是倍数,所以所谓最优路径其实是次优的.插值A*算法在路径规划时,使用线形插值来计算出更精确的路径代价,由此产生更优路径.     

Real time planning in N-dim state space

This paper presents the actual work in real-time planning as search [1] [2]. Based in this work we tried to solve the path planning in numerical state space. We found that precision, performance, and time were very linked. In real-time problem solving, the agent can fall in traps made of forbidden zones and to go out it, have to spend too much computing time. To solve this problem we propose a multilayer inference based in subgoals computation. An architecture based in two agents, one for low level task with the maximum precision and other for subgoals computation is proposed here.     

Randomized path planning for tasks requiring the release and regrasp of objects

This paper presents a randomized planning algorithm for manipulation tasks that require the robot to release and regrasp an object in different robot postures. Such problems arise, for example, in robotic suturing and knot tying, and in assembly tasks where parts must be guided through complex environments. Formulating the problem as one of planning on a foliated manifold, we present a randomized planning algorithm that, unlike existing methods, involves sampling and tree propagation primarily in the task space manifold; such an approach significantly improves computational efficiency by reducing the number of projections to the constraint manifold, without incurring any significant increases in the number of release-regrasp sequences. We also propose a post-processing topological exploration algorithm and path refinement procedure for reducing the number of release-regrasp sequences in a solution path, independent of the algorithm used to generate the path. Experiments involving spatial open chains with up to 10 degrees of freedom, operating in complex obstacle-filled environments, show that our algorithm considerably outperforms existing algorithms in terms of computation time, path length, and the number of release-regrasp operations.     

Dual-layer multi-robot path planning in narrow-lane environments under specific traffic policies

Collision-free path planning is indispensable for the multi-robot system. Many existing multi-robot path planning algorithms may no longer work properly in the narrow-lane environment. We propose in this paper a dual-layer algorithm to deal with the multi-robot path planning problem in the narrow-lane environment. In the first layer, the integer programming technique primarily based on distance metrics balances the optimality of the generated collision-free paths and the computation time of the algorithm. In the second layer, fast feasible heuristics are applied to make sure the solvability of the proposed integer programming approach in the first layer. In the dual-layer algorithm, specific traffic policies for each narrow lane are implemented to generate a collision-free path for every robot while maintaining the narrow lane free, besides the collision avoidance approach at the robotic level. With this, inter-robot collision in the narrow lane is avoided, and the algorithm’s efficiency in producing collision-free paths increases. Simulations have been launched considerably based on the proposed assessment metrics. According to the extensive simulation data, our algorithm suggests a higher overall performance in the narrow-lane environment when in contrast with the present optimal, sub-optimal, and polynomial-complexity algorithms.

     

Reinforcement learning technique using agent state occurrence frequency with analysis of knowledge sharing on the agent’s learning process in multiagent environments

Reinforcement learning techniques like the Q-Learning one as well as the Multiple-Lookahead-Levels one that we introduced in our prior work require the agent to complete an initial exploratory path followed by as many hypothetical and physical paths as necessary to find the optimal path to the goal. This paper introduces a reinforcement learning technique that uses a distance measure to the goal as a primary gauge for an autonomous agent’s action selection. In this paper, we take advantage of the first random walk to acquire initial information about the goal. Once the agent’s goal is reached, the agent’s first perceived internal model of the environment is updated to reflect and include said goal. This is done by the agent tracing back its steps to its origin starting point. We show in this paper, no exploratory or hypothetical paths are required after the goal is initially reached or detected, and the agent requires a maximum of two physical paths to find the optimal path to the goal. The agent’s state occurrence frequency is introduced as well and used to support the proposed Distance-Only technique. A computation speed performance analysis is carried out, and the Distance-and-Frequency technique is shown to require less computation time than the Q-Learning one. Furthermore, we present and demonstrate how multiple agents using the Distance-and-Frequency technique can share knowledge of the environment and study the effect of that knowledge sharing on the agents’ learning process.     

A fast propagation scheme for approximate geodesic paths

Geodesic paths on surfaces are indispensable in many research and industrial areas, including architectural and aircraft design, human body animation, robotic path planning, terrain navigation, and reverse engineering. 3D models in these applications are typically large and complex. It is challenging for existing geodesic path algorithms to process large-scale models with millions of vertices. In this paper, we focus on the single-source geodesic path problem, and present a novel framework for efficient and approximate geodesic path computation over triangle meshes. The algorithm finds and propagates paths based on a continuous Dijkstra strategy with a two-stage approach to compute a path for each propagating step. Starting from an initial path for each step, its shape is firstly optimized by solving a sparse linear system and then the output floating path is projected to the surface to obtain the refined one for further propagation. We have extensively evaluated our algorithms on a number of 3D models and also compared their performance against existing algorithms. Such evaluation and comparisons indicate our algorithm is fast and produces acceptable accuracy.     

Heuristic approaches in robot path planning: A survey

Autonomous navigation of a robot is a promising research domain due to its extensive applications. The navigation consists of four essential requirements known as perception, localization, cognition and path planning, and motion control in which path planning is the most important and interesting part. The proposed path planning techniques are classified into two main categories: classical methods and heuristic methods. The classical methods consist of cell decomposition, potential field method, subgoal network and road map. The approaches are simple; however, they commonly consume expensive computation and may possibly fail when the robot confronts with uncertainty. This survey concentrates on heuristic-based algorithms in robot path planning which are comprised of neural network, fuzzy logic, nature-inspired algorithms and hybrid algorithms. In addition, potential field method is also considered due to the good results. The strengths and drawbacks of each algorithm are discussed and future outline is provided.