Time-varying formation control of a collaborative heterogeneous multi agent system

In this paper, coherent formation control of a multi-agent system in the presence of time-varying formation is studied. For special application of rescue and surveillance, a set of agents, consisting of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) are considered. Due to different degrees of freedom of the UAVs and the UGVs, the collaboration between the agents confronts many problems. A Lyapunov based controller is presented to stabilize the swarming and lead the system to a rigid formation using decentralized control approach. In the proposed control signal of each agent, a signal of the neighbors’ error is considered to cope with variation in performance and to provide synchronization, which means that the state error of the agents converges to zero nearly at the same time. The decentralized approach provides reliability of the performance in unknown environment, since the controller of each agent is designed based on local knowledge. This algorithm is evaluated in simulation and the results approve the accepted performance of the proposed approach.     

旋翼无人机在移动平台降落的控制参数自学习调节方法

无人机设备能够适应复杂地形,但由于电池容量等原因,无人机无法长时间执行任务。无人机与其他无人系统（无人车、无人船等）协同能够有效提升无人机的工作时间,完成既定任务,当无人机完成任务后,将无人机迅速稳定地降落至移动平台上是一项必要且具有挑战性的工作。针对降落问题,文中提出了基于矫正纠偏COACH(corrective advice communicated humans)方法的深度强化学习比例积分微分(proportional-integral-derivative, PID)方法,为无人机降落至移动平台提供了最优路径。首先在仿真环境中使用矫正纠偏框架对强化学习模型进行训练,然后在仿真环境和真实环境中,使用训练后的模型输出控制参数,最后利用输出参数获得无人机位置控制量。仿真结果和真实无人机实验表明,基于矫正纠偏COACH方法的深度强化学习PID方法优于传统控制方法,且能稳定完成在移动平台上的降落任务。     

Asynchronous double-precision windows based unmanned aerial vehicle real-time path planning

A real-time path planning approach based on asynchronous double-precision windows is proposed for unmanned aerial vehicles (UAVs). In this proposed method, cursory paths and elaborate paths are planned respectively in the global and local windows. Specifically, global cursory path planner and local elaborate path planner are integrated by rolling two windows on different frequencies with different modes. Simulation results demonstrate that the proposed approach is effective for realizing a balance between t...     

Distributed hierarchical control for multiple vertical takeoff and landing UAVs with a distance‐based network topology

A distributed formation control algorithm is proposed for multiple vertical takeoff and landing (VTOL) unmanned aerial vehicles (UAVs) in this paper. The neighboring information is obtained by active measurements rather than by nonactive communications as done in the current literatures. By properly expanding the distance‐based network topology structure between each pair of UAVs, a distributed algorithm is proposed such that the switching topology remains connected along the time. Since a VTOL UAV system is typically underactuated, a hierarchical idea is introduced to derive the control design procedure. More specifically, the studied formation control problem of multiple VTOL UAVs is first transformed into the consensus problem of their corresponding error systems. Then, a command force and an applied torque are synthesized for each VTOL UAV such that the error systems reach a consensus. It is demonstrated in terms of Lyapunov theory that the proposed distributed hierarchical control algorithm guarantees the formation realization of multiple VTOL UAVs while maintaining the network connectivity. Finally, simulations are performed to validate the theoretical results.     

Multiple UAVs/UGVs heterogeneous coordinated technique based on Receding Horizon Control (RHC) and velocity vector control

Multiple unmanned air vehicles(UAVs)/unmanned ground vehicles(UGVs) heterogeneous cooperation provides a new breakthrough for the effective application of UAV and UGV.On the basis of introduction of UAV/UGV mathematical model,the characteristics of heterogeneous flocking is analyzed in detail.Two key issues are considered in multi-UGV subgroups,which are Reynolds Rule and Virtual Leader(VL).Receding Horizon Control(RHC) with Particle Swarm Optimization(PSO) is proposed for multiple UGVs flocking,and velocit...     

A class of intelligent agents for coordinated control of outdoor terrain mapping UGVs

This article develops a systems- and control-oriented intelligent agent framework called the hybrid intelligent control agent (HICA) and describes its composition into multiagent systems. It is essentially developed around a hybrid control system core so that knowledge-based planning and coordination can be integrated with verified hybrid control primitives to achieve the coordinated control of multiple multi-mode dynamical systems. The scheme is applied to the control of a team of unmanned ground vehicles (UGVs) engaged in an outdoor terrain mapping task. Results are demonstrated experimentally.     

Crowd simulation using DC model and density information

Realistic crowd simulation is an important issue for the production of virtual worlds for games, crowd management, public space design, education, entertainment or architectural and urban planning. In this paper, crowd simulation is considered from two aspects: intra-group simulation and inter-group simulation. We propose a unified framework for crowd simulation in real-time virtual environment. Based on this framework, for intra-group simulation, we propose a novel density-based information crowd simulation to collision-free. For inter-group simulation, we propose a novel discrete choice (DC) model to realistic simulation of crowds and path planning. Meanwhile, we also propose a variable bounding box method for intra-group/inter-groups intersection problem. The simulation results show that the developed framework allows different group structures to be easily modeled. And the proposed framework could be used for real-time navigation of many moving crowd in complicated virtual environments.     

IoT-Enabled Autonomous System Collaboration for Disaster-Area Management

Timely investigating post-disaster situations to locate survivors and secure hazardous sources is critical, but also very challenging and risky. Despite first responders putting their lives at risk in saving others, human-physical limits cause delays in response time, resulting in fatality and property damage. In this paper, we proposed and implemented a framework intended for creating collaboration between heterogeneous unmanned vehicles and first responders to make search and rescue operations safer and faster. The framework consists of unmanned aerial vehicles (UAVs), unmanned ground vehicles (UGVs), a cloud-based remote control station (RCS). A light-weight message queuing telemetry transport (MQTT) based communication is adopted for facilitating collaboration between autonomous systems. To effectively work under unfavorable disaster conditions, antenna tracker is developed as a tool to extend network coverage to distant areas, and mobile charging points for the UAVs are also implemented. The proposed framework’s performance is evaluated in terms of end-to-end delay and analyzed using architectural analysis and design language (AADL). Experimental measurements and simulation results show that the adopted communication protocol performs more efficiently than other conventional communication protocols, and the implemented UAV control mechanisms are functioning properly. Several scenarios are implemented to validate the overall effectiveness of the proposed framework and demonstrate possible use cases.      

Cooperative air and ground surveillance

Unmanned aerial vehicles (UAV) can be used to cover large areas searching for targets. However, sensors on UAVs are typically limited in their accuracy of localization of targets on the ground. On the other hand, unmanned ground vehicles (UGV) can be deployed to accurately locate ground targets, but they have the disadvantage of not being able to move rapidly or see through such obstacles as buildings or fences. In this paper, we describe how we can exploit this synergy by creating a seamless network of UAVs and UGVs. The keys to this are our framework and algorithms for search and localization, which are easily scalable to large numbers of UAVs and UGVs and are transparent to the specificity of individual platforms. We describe our experimental testbed, the framework and algorithms, and some results.     

Rolling Horizon Path Planning of an Autonomous System of UAVs for Persistent Cooperative Service: MILP Formulation and Efficient Heuristics

A networked system consisting of unmanned aerial vehicles (UAVs), automated logistic service stations (LSSs), customer interface software, system orchestration algorithms and UAV control software can be exploited to provide persistent service to its customers. With efficient algorithms for UAV task planning, the UAVs can autonomously serve the customers in real time. Nearly uninterrupted customer service may be accomplished via the cooperative hand-off of customer tasks from weary UAVs to ones that have recently been replenished at an LSS. With the goal of enabling the autonomy of the task planning tasks, we develop a mixed integer linear programming (MILP) formulation for the problem of providing simultaneous. UAV escort service to multiple customers across a field of operations with multiple sharable LSSs. This MILP model provides a formal representation of our problem and enables use in a rolling horizon planner via allowance of arbitrary UAV initial locations and consumable reservoir status (e.g., battery level). As such, it enables automation of the orchestration of system activities. To address computational complexity, we develop efficient heuristics to rapidly derive near optimal solutions. A receding horizon task assignment (RHTA) heuristic and sequential task assignment heuristic (STAH) are developed. STAH exploits properties observed in optimal solutions obtained for small problems via CPLEX. Numerical studies suggest that RHTA and STAH are 45 and 2100 times faster than solving the MILP via CPLEX, respectively. Both heuristics perform well relative to the optimal solution obtained via CPLEX. An example demonstrating the use of the approach for rolling horizon planning is provided.     

Decentralized planning and control for UAV–UGV cooperative teams

In this paper we study a symbiotic aerial vehicle-ground vehicle robotic team where unmanned aerial vehicles (UAVs) are used for aerial manipulation tasks, while unmanned ground vehicles (UGVs) aid and assist them. UGV can provide a UAV with a safe landing area and transport it across large distances, while UAV can provide an additional degree of freedom for the UGV, enabling it to negotiate obstacles. We propose an overall system control framework that includes high-accuracy motion planning for each individual robot and ad-hoc decentralized mission planning for complex missions. Experimental results obtained in a mockup arena for parcel transportation scenario show that the system is able to plan and execute missions in various environments and that the obtained plans result in lower energy consumption.     

Modeling and simulation for natural disaster contingency planning driven by high-resolution remote sensing images

Natural disasters occur unexpectedly and usually result in huge losses of life and property. How to effectively make contingency plans is an intriguing question constantly faced by governments and experts. Human rescue operations are the most critical issue in contingency planning. A natural disaster scenario is, in general, highly complicated and dynamic. Modeling and simulation technologies have been gaining considerable momentum in investigating natural disaster scenarios to enable contingency planning. However, existing M&S systems still suffer from two open problems: (1) a lack of real data on natural disasters; and (2) the absence of methods and platforms to describe the collective behaviors of people in disaster situations. Considering these problems, an M&S framework for human rescue operations in a typical natural disaster, i.e., a landslide, has been developed in this study. The framework consists of three modules: (1) remote sensing information extraction, (2) landslide simulation, and (3) crowd simulation. The crowd simulation module is driven by the real/virtual data provided by the former modules. A number of simulations (using the Zhouqu landslide as an example) have been performed to study human relief operations spontaneously and under manipulation, with the effect of contingency plans highlighted. The experimental results demonstrate that  (1) the simulation framework is an effective tool for contingency planning, and (2) real data can make the simulation outputs more meaningful.     

A Framework for Simulation and Testing of UAVs in Cooperative Scenarios

Today, Unmanned Aerial Vehicles (UAVs) have deeply modified the concepts of surveillance, Search&Rescue, aerial photogrammetry, mapping, etc. The kinds of missions grow continuously; missions are in most cases performed by a fleet of cooperating autonomous and heterogeneous vehicles. These systems are really complex and it becomes fundamental to simulate any mission stage to exploit benefits of simulations like repeatability, modularity and low cost. In this paper a framework for simulation and testing of UAVs in cooperative scenarios is presented. The framework, based on modularity and stratification in different specialized layers, allows an easy switching from simulated to real environments, thus reducing testing and debugging times, especially in a training context. Results obtained using the proposed framework on some test cases are also reported.     

Hierarchical control of cooperative nonlinear dynamical systems

Cooperative control methods that are scalable with low computational cost are crucial for networked dynamical systems to respond quickly in unknown or cluttered environments. In an attempt to make the problem tractable, many existing cooperative controls are designed with oversimplified assumptions and/or without the capabilities of rapidly handling different environmental and dynamical constraints. In this article, proposed is a two-level hierarchical, cooperative control framework using a divide-and-conquer strategy so that challenges can be separately handled at different levels. It is scalable and has low computational cost. Based on a simplified homogeneous double-integrator dynamic model, the top-level planner first computes cooperative trajectories satisfying obstacle avoidance requirements. Then at the lower level, state and control constraints, nonlinear dynamics and self-collision/obstacle avoidance as related to the real system are addressed through a bio-inspired fast trajectory planning algorithm. The stability of the overall hierarchical structure is proven. Two examples, a differential-drive ground vehicle formation control and an unmanned aerial vehicle formation flight, are used to illustrate the advantages of the proposed hierarchical framework.     

Trajectory Planning for Communication Relay Unmanned Aerial Vehicles in Urban Dynamic Environments

This paper proposes an optimal positioning and trajectory planning algorithm for unmanned aerial vehicles (UAVs) to improve a communication quality of a team of ground mobile nodes (vehicles) in a complex urban environment. In particular, a nonlinear model predictive control (NMPC)-based approach is proposed to find an efficient trajectory for UAVs with a discrete genetic algorithm while considering the dynamic constraints of fixed-wing UAVs. The advantages of using the proposed NMPC approach and the communication performance metrics are investigated through a number of scenarios with different horizon steps in the NMPC framework, the number of UAVs used, heading rates and speeds.     

Learning behavior patterns from video for agent-based crowd modeling and simulation

This paper proposes a novel data-driven modeling framework to construct agent-based crowd model based on real-world video data. The constructed crowd model can generate crowd behaviors that match those observed in the video and can be used to predict trajectories of pedestrians in the same scenario. In the proposed framework, a dual-layer architecture is proposed to model crowd behaviors. The bottom layer models the microscopic collision avoidance behaviors, while the top layer models the macroscopic crowd behaviors such as the goal selection patterns and the path navigation patterns. An automatic learning algorithm is proposed to learn behavior patterns from video data. The learned behavior patterns are then integrated into the dual-layer architecture to generate realistic crowd behaviors. To validate its effectiveness, the proposed framework is applied to two different real world scenarios. The simulation results demonstrate that the proposed framework can generate crowd behaviors similar to those observed in the videos in terms of crowd density distribution. In addition, the proposed framework can also offer promising performance on predicting the trajectories of pedestrians.     

强化学习的地-空异构多智能体协作覆盖研究

以无人机（unmanned aerial vehicle, UAV）和无人车（unmanned ground vehicle, UGV）的异构协作任务为背景,通过UAV和UGV的异构特性互补,为了扩展和改进异构多智能体的动态覆盖问题,提出了一种地-空异构多智能体协作覆盖模型。在覆盖过程中,UAV可以利用速度与观测范围的优势对UGV的行动进行指导;同时考虑智能体的局部观测性与不确定性,以分布式局部可观测马尔可夫（decentralized partially observable Markov decision processes,DEC-POMDPs）为模型搭建覆盖场景,并利用多智能体强化学习算法完成对环境的覆盖。仿真实验表明,UAV与 UGV间的协作加快了团队对环境的覆盖速度,同时强化学习算法也提高了覆盖模型的有效性。     

平行机器人与平行无人系统:框架、结构、过程、平台及其应用

本文将基于ACP（Artificial societies,computational experiments,parallel execution）的平行系统思想与机器人领域相结合,形成一种软硬件相结合的框架,为无人机、无人车、无人船在复杂环境中实验、学习与实际工作提供便捷、安全的平台,即平行无人系统.本文从平行机器人的基本概念出发,提出平行无人系统的基本框架,并介绍了各模块的基本功能与实现方法,探讨了其中的关键技术.然后本文围绕无人机、无人车、无人船三个方面展望了无人平行系统在实际中的应用和所面临的挑战,提出了平行无人系统的未来发展方向.     

Reinforcement learning in spacecraft control applications: Advances,prospects, and challenges

This paper presents and analyzes Reinforcement Learning (RL) based approaches to solve spacecraft control problems. Different application fields are considered, e.g., guidance, navigation and control systems for spacecraft landing on celestial bodies, constellation orbital control, and maneuver planning in orbit transfers. It is discussed how RL solutions can address the emerging needs of designing spacecraft with highly autonomous on-board capabilities and implementing controllers (i.e., RL agents) robust to system uncertainties and adaptive to changing environments. For each application field, the RL framework core elements (e.g., the reward function, the RL algorithm and the environment model used for the RL agent training) are discussed with the aim of providing some guidelines in the formulation of spacecraft control problems via a RL framework. At the same time, the adoption of RL in real space projects is also analyzed. Different open points are identified and discussed, e.g., the availability of high-fidelity simulators for the RL agent training and the verification of RL-based solutions. This way, recommendations for future work are proposed with the aim of reducing the technological gap between the solutions proposed by the academic community and the needs/requirements of the space industry.     

基于混沌多项式的指令鲁棒优化及在飞行控制中的应用

本文提出一种新的方法对随机系统进行运动预测和控制指令设计, 该方法可以充分利用已知信息设计控 制指令以提高闭环随机系统的鲁棒性. 首先采用混沌多项式对随机信息进行数学表述, 并利用Galerkin投影法将随 机变量的混沌多项式引入常微分方程中. 然后, 将随机变量的均值和方差考虑至优化问题的成本函数中, 并利用伪 谱法对控制指令进行鲁棒优化. 最后, 将该方法应用于飞行器的动力学预测以及控制指令设计. 仿真结果表明, 该 方法能够预测飞行器飞行过程中不确定性的演化, 其精度与蒙特卡罗方法相当, 并且计算效率更高. 此外, 获得的 控制指令对存在不确定参数或初始条件的随机系统具有强鲁棒性.