四旋翼无人机自主移动降落方法研究

近年来,四旋翼无人机在自主完成各种复杂任务中扮演着愈发重要的角色。移动降落技术是无人机智能处理系统的关键技术,包含3个环节:目标检测、目标跟踪、位置预估及降落。提出了一种基于Apriltags的跟踪降落算法,提升了识别性能,通过Kalman滤波及拟合函数等方法预估运动目标轨迹,采用PID算法控制无人机稳定飞行、快速响应,实现了无人机对移动目标的智能识别、稳定跟踪及移动降落。     

基于深度强化学习的移动机器人导航策略研究

针对移动机器人在复杂动态变化的环境下导航的局限性,采用了一种将深度学习和强化学习结合起来的深度强化学习方法。研究以在OpenCV平台下搭建的仿真环境的图像作为输入数据,输入至TensorFlow创建的卷积神经网络模型中处理,提取其中的机器人的动作状态信息,结合强化学习的决策能力求出最佳导航策略。仿真实验结果表明：在经过深度强化学习的方法训练后,移动机器人在环境发生了部分场景变化时,依然能够实现随机起点到随机终点的高效准确的导航。     

空地异构机器人系统协作巡逻路径规划方法EI北大核心CSCD

空地异构机器人系统由空中无人机和地面无人车组成,当两者协作执行持续巡逻任务时,使用无人车充当无人机的地面移动补给站能够解决无人机续航能力不足的问题.运动受限于路网中的无人车必须在适当地点为无人机补充能量,这使得两者的路径高度耦合,给空地协作路径规划带来了挑战.针对此问题,本文通过分析无人机能量、路网、空地汇合时间、巡逻任务全覆盖等多种约束,以无人机完成全部巡逻任务的总距离为代价,建立了空地协作巡逻路径规划模型.该模型可推广至多架无人机与多辆无人车协作的情形.然后,采用遗传算法与蚁群算法相融合的方法,对无人机巡逻路径和无人车能量补给路径进行优化求解.仿真实验表明,本文的方法不仅可以得到很好的路径规划结果,而且较其他算法具有更优的收敛性和执行速度.     

深度确定性策略梯度学习的火星无人机控制

为了降低控制器设计对火星无人机动力学模型的依赖,提高火星无人机控制系统的智能化水平,结合强化学习（reinforcement learning,RL）算法,提出了一种具有自主学习能力的火星无人机位置姿态控制器。该控制器由神经网络构成,利用深度确定性策略梯度（deep deterministic policy gradient,DDPG）算法进行学习,不断优化控制策略,最终获得满足控制要求的策略。仿真结果表明,在没有推导被控对象模型的前提下,基于DDPG算法的控制器通过学习,自主将火星无人机稳定控制到目标位置,且控制精度、调节时间等性能优于比例-积分-微分（proportion integration differentiation,PID）控制器的效果,验证了基于DDPG算法的控制器的有效性;此外,在被控对象模型改变或存在外部扰动的情况下,基于DDPG算法的控制器仍然能够稳定完成任务,控制效果优于PID控制器,表明基于DDPG算法的控制器具有良好的鲁棒性。     

智能车辆深度强化学习的模型迁移轨迹规划方法

针对智能驾驶车辆传统路径规划中出现车辆模型跟踪误差和过度依赖问题,提出一种基于深度强化学习的模型迁移的智能驾驶车辆轨迹规划方法.首先,提取真实环境的抽象模型,该模型利用深度确定性策略梯度(DDPG)和车辆动力学模型,共同训练逼近最优智能驾驶的强化学习模型;其次,通过模型迁移策略将实际场景问题迁移至虚拟抽象模型中,根据该环境中训练好的深度强化学习模型计算控制与轨迹序列;而后,根据真实环境中评价函数选择最优轨迹序列.实验结果表明,所提方法能够处理连续输入状态,并生成连续控制的转角控制序列,减少横向跟踪误差;同时通过模型迁移能够提高模型的泛化性能,减小过度依赖问题.     

基于强化学习的无人车组路径优化算法研究

针对传统单车路径规划算法在进行无人车组路径规划时存在的算法收敛性问题,采用强化学习方法,对传统Q-learning算法中的探索率进行改进,将每一个路程点作为每一段局部路径规划的目标点,通过传感器感知外界环境的信息,进行基于强化学习的在线局部路径规划,完成避障和寻径任务。构建了算法模型与仿真环境,并进行了仿真,结果表明无人车组能够在短时间内收敛到稳定状态并自主完成规划任务,证明了算法的有效性和可行性。上述算法在多无人战车协同的智能规划与控制中具有良好的应用前景。     

考虑无线充电的无人机路径在线规划

近年来, 无人机在物流、通信、军事任务、灾害救援等领域中展现出了巨大的应用潜力, 然而无人机的续航 能力是制约其使用的重大因素, 在无线充电技术不断突破和发展的背景下, 本文基于深度强化学习方法, 提出了一 种考虑无线充电的无人机路径在线优化方法, 通过无线充电技术提高无人机的任务能力. 首先, 对无人机功耗模型 和无线充电模型进行了构建, 根据无人机的荷电状态约束, 设计了一种基于动态上下文向量的深度神经网络模型, 通过编码器和解码器的模型架构, 实现无人机路径的直接构造, 通过深度强化学习方法对模型进行离线训练, 从而 应用于考虑无线充电的无人机任务路径在线优化. 文本通过与传统优化方法和深度强化学习方法进行实验对比, 所提方法在CPU算力和GPU算力下分别实现了4倍以及100倍以上求解速度的提升.     

基于深度强化学习的多旋翼无人机空中目标自主跟踪

针对空中对接任务中的目标自主跟踪问题,提出了一种基于深度强化学习的端到端的目标跟踪方法。该方法采用近端策略优化算法,Actor网络与Critic网络共享前两层的网络参数,将无人机所拍摄图像作为卷积神经网络的输入,通过策略网络控制多旋翼无人机电机转速,实现端到端的目标跟踪,同时采用shaping方法以加速智能体训练。通过物理引擎Pybullet搭建仿真环境并进行训练验证,仿真结果表明该方法能够达到设定的目标跟踪要求且具有较好的鲁棒性。     

无人机动态测试、仿真与训练系统设计

无人机动态测试、仿真与训练系统包括地勤检测平台、联调仿真环境和虚拟训练环境,能够对无人机进行动态测试和控制,满足日常操作训练要求.地勤检测平台使用了便捷式测试仪,在满足无人机地勤检测需要的同时,能够方便地勤准备,缩短准备时间;联调仿真环境利用分布式交互仿真技术和虚拟现实技术,构建虚拟无人机,通过遥测和遥控设备完成无人机各部件的动态测试和综合演练,同步生成三维视景,方便地面分析和控制;在联调仿真环境的基础上,虚拟训练环境对精密设备,大型设备以及电磁设备进行虚拟化处理,方便日常操作训练.     

基于强化学习的四旋翼无人机控制律设计

目前四旋翼无人机大部分都采用经典控制方法进行控制律的设计,然而控制参数的选择和对被控对象数学模型的依赖一直是经典控制方法设计中需要克服的问题;针对此问题,采用了一种基于深度强化学习算法Deep Q Network的无人机控制律设计方法,以四旋翼姿态角和姿态角速率作为三层神经网络的输入数据,最终输出动作值函数,再根据贪婪策略进行动作的选取,通过与环境的不断交互,智能体根据奖惩信息来更新神经网络的权值,使得智能体朝着获得累积回报最大值的方向选取动作;仿真结果表明在经过强化学习训练之后,四旋翼姿态角能够快速准确地跟踪上参考指令的变化,证明了基于强化学习的四旋翼无人机控制律的可行性,从而避免了传统控制方法对控制参数的选择与控制模型的依赖。     

Algorithms and experiments on routing of unmanned aerial vehicles with mobile recharging stations

We study the problem of planning a tour for an energy‐limited Unmanned Aerial Vehicle (UAV) to visit a set of sites in the least amount of time. We envision scenarios where the UAV can be recharged at a site or along an edge either by landing on stationary recharging stations or on Unmanned Ground Vehicles (UGVs) acting as mobile recharging stations. This leads to a new variant of the Traveling Salesperson Problem (TSP) with mobile recharging stations. We present an algorithm that finds not only the order in which to visit the sites but also when and where to land on the charging stations to recharge. Our algorithm plans tours for the UGVs as well as determines the best locations to place stationary charging stations. We study three variants for charging: Multiple stationary charging stations, single mobile charging station, and multiple mobile charging stations. As the problems we study are nondeterministic polynomial time (NP)‐Hard, we present a practical solution using Generalized TSP that finds the optimal solution that minimizes the total time, subject to the discretization of battery levels. If the UGVs are slower than the UAVs, then the algorithm also finds the minimum number of UGVs required to support the UAV mission such that the UAV is not required to wait for the UGV. Our simulation results show that the running time is acceptable for reasonably sized instances in practice. We evaluate the performance of our algorithm through simulations and proof‐of‐concept field experiments with a fully autonomous system of one UAV and UGV.     

强化学习的地-空异构多智能体协作覆盖研究

以无人机（unmanned aerial vehicle, UAV）和无人车（unmanned ground vehicle, UGV）的异构协作任务为背景,通过UAV和UGV的异构特性互补,为了扩展和改进异构多智能体的动态覆盖问题,提出了一种地-空异构多智能体协作覆盖模型。在覆盖过程中,UAV可以利用速度与观测范围的优势对UGV的行动进行指导;同时考虑智能体的局部观测性与不确定性,以分布式局部可观测马尔可夫（decentralized partially observable Markov decision processes,DEC-POMDPs）为模型搭建覆盖场景,并利用多智能体强化学习算法完成对环境的覆盖。仿真实验表明,UAV与 UGV间的协作加快了团队对环境的覆盖速度,同时强化学习算法也提高了覆盖模型的有效性。     

Precedence-constrained path planning of messenger UAV for air-ground coordination

This paper addresses an unmanned aerial vehicle (UAV) path planning problem for a team of cooperating heterogeneous vehicles composed of one UAV and multiple unmanned ground vehicles (UGVs). The UGVs are used as mobile actuators and scattered in a large area. To achieve multi-UGV communication and collaboration, the UAV serves as a messenger to fly over all task points to collect the task information and then flies all UGVs to transmit the information about tasks and UGVs. The path planning of messenger UAV is formulated as a precedence-constrained dynamic Dubins traveling salesman problem with neighborhood (PDDTSPN). The goal of this problem is to find the shortest route enabling the UAV to fly over all task points and deliver information to all requested UGVs. When solving this path planning problem, a decoupling strategy is proposed to sequentially and rapidly determine the access sequence in which the UAV visits task points and UGVs as well as the access location of UAV in the communication neighborhood of each task point and each UGV. The effectiveness of the proposed approach is corroborated through computational experiments on randomly generated instances. The computational results on both small and large instances demonstrate that the proposed approach can generate high-quality solutions in a reasonable time as compared with two other heuristic algorithms.     

A DDDAMS-based planning and control framework for surveillance and crowd control via UAVs and UGVs

A dynamic data driven adaptive multi-scale simulation (DDDAMS) based planning and control framework is proposed for effective and efficient surveillance and crowd control via UAVs and UGVs. The framework is mainly composed of integrated planner, integrated controller, and decision module for DDDAMS. The integrated planner, which is designed in an agent-based simulation (ABS) environment, devises best control strategies for each function of (1) crowd detection (vision algorithm), (2) crowd tracking (filtering), and (3) UAV/UGV motion planning (graph search algorithm). The integrated controller then controls real UAVs/UGVs for surveillance tasks via (1) sensory data collection and processing, (2) control command generation based on strategies provided by the decision planner for crowd detection, tracking, and motion planning, and (3) control command transmission via radio to the real system. The decision module for DDDAMS enhances computational efficiency of the proposed framework via dynamic switching of fidelity of simulation and information gathering based on the proposed fidelity selection and assignment algorithms. In the experiment, the proposed framework (involving fast-running simulation as well as real-time simulation) is illustrated and demonstrated for a real system represented by hardware-in-the-loop (HIL) real-time simulation integrating real UAVs, simulated UGVs and crowd, and simulated environment (e.g. terrain). Finally, the preliminary results successfully demonstrate the benefit of the proposed dynamic fidelity switching concerning the crowd coverage percentage and computational resource usage (i.e. CPU usage) under cases with two different simulation fidelities.     

路网约束下异构机器人系统路径规划方法

由无人机(Unmanned aerial vehicles, UAV)和地面移动机器人组成的异构机器人系统在协作执行任务时,可以充分发挥两类机器人各自的优势.无人机运动灵活,但通常续航能力有限;地面机器人载荷多,适合作为无人机的着陆平台和移动补给站,但运动受路网约束.本文研究这类异构机器人系统协作路径规划问题.为了降低完成任务的时间代价,提出一种由蚁群算法(Ant colony optimization, ACO)和遗传算法(Genetic algorithm, GA)相结合的两步法对地面机器人和无人机的路线进行解耦,同时规划地面机器人和无人机的路线.第1步使用蚁群算法为地面机器人搜索可行路线.第2步对无人机的最优路径建模,采用遗传算法求解并将无人机路径长度返回至第1步中,用于更新路网的信息素参数,从而实现异构协作系统路径的整体优化.另外,为了进一步降低无人机的飞行时间代价,研究了无人机在其续航能力内连续完成多任务的协作路径规划问题.最后,通过大量仿真实验验证了所提方法的有效性.     

Decentralized planning and control for UAV–UGV cooperative teams

In this paper we study a symbiotic aerial vehicle-ground vehicle robotic team where unmanned aerial vehicles (UAVs) are used for aerial manipulation tasks, while unmanned ground vehicles (UGVs) aid and assist them. UGV can provide a UAV with a safe landing area and transport it across large distances, while UAV can provide an additional degree of freedom for the UGV, enabling it to negotiate obstacles. We propose an overall system control framework that includes high-accuracy motion planning for each individual robot and ad-hoc decentralized mission planning for complex missions. Experimental results obtained in a mockup arena for parcel transportation scenario show that the system is able to plan and execute missions in various environments and that the obtained plans result in lower energy consumption.     

基于AprilTag的分体式飞行汽车自主导引对接方法研究

分体式飞行汽车作为一种新概念空中陆地交通工具,可解决当前频繁发生的城市交通拥堵、城市郊区交通不便捷等问题;针对分体式飞行汽车进行模态转换时涉及的模块间精准导引对接问题,提出了一种基于AprilTag的视觉定位导航方案,将AprilTag识别算法的解算结果进行坐标变换后,得到模块间的相对位姿,再结合基于无人机PID控制器的导引对接降落流程设计,解决了因GPS定位误差大而无法达到厘米级精度的导引对接任务需求的问题,并提升了导引对接降落过程的平稳性;最后,在ROS平台利用实物实验验证了该方案的可行性.     

Multiple UAVs/UGVs heterogeneous coordinated technique based on Receding Horizon Control (RHC) and velocity vector control

Multiple unmanned air vehicles(UAVs)/unmanned ground vehicles(UGVs) heterogeneous cooperation provides a new breakthrough for the effective application of UAV and UGV.On the basis of introduction of UAV/UGV mathematical model,the characteristics of heterogeneous flocking is analyzed in detail.Two key issues are considered in multi-UGV subgroups,which are Reynolds Rule and Virtual Leader(VL).Receding Horizon Control(RHC) with Particle Swarm Optimization(PSO) is proposed for multiple UGVs flocking,and velocit...     

Cooperative air and ground surveillance

Unmanned aerial vehicles (UAV) can be used to cover large areas searching for targets. However, sensors on UAVs are typically limited in their accuracy of localization of targets on the ground. On the other hand, unmanned ground vehicles (UGV) can be deployed to accurately locate ground targets, but they have the disadvantage of not being able to move rapidly or see through such obstacles as buildings or fences. In this paper, we describe how we can exploit this synergy by creating a seamless network of UAVs and UGVs. The keys to this are our framework and algorithms for search and localization, which are easily scalable to large numbers of UAVs and UGVs and are transparent to the specificity of individual platforms. We describe our experimental testbed, the framework and algorithms, and some results.