基于深度强化学习的多潜器编队控制算法设计

考虑水下未知信道与不确定模型参数,提出基于深度强化学习的多潜器编队控制算法.首先,提出基于环境采样数据的最小二乘估计器,用于预测在衰落环境下的未知信道;其次,根据信道预测估计器得出的信噪比(SNR),建立通信有效性与编队稳定性的联合优化问题,并给出基于深度强化学习-深度确定性策略梯度算法(DDPG)的编队控制算法;最后,通过仿真与实验结果验证所提出算法的有效性,参考仿真结果并相比于直接编队控制,考虑通信有效性的情况下所提出算法提升了13.5%的通信性能.     

Black‐box position and attitude tracking for underwater vehicles by second‐order sliding‐mode technique

In this work we address the tracking control problems for autonomous underwater vehicles (AUVs). The proposed solution is based on the variable structure systems (VSS) theory and, in particular, on the second‐order sliding‐mode (2‐SM) methodology. The tuning of the controller is carried out via black‐box approach, dispensing with the knowledge of the actual AUV parameters, by simply progressively increasing a single gain parameter. The presented stability analysis includes explicitly the unmodelled actuator dynamics and the presence of external uncertain disturbances. The good performance of the proposed scheme is verified by means of simulations on a 6‐DOF AUV. Copyright © 2009 John Wiley & Sons, Ltd.     

Mobile Underwater Sensor Networks for Protection and Security: Field Experience at the UAN11 Experiment

The EU‐funded project UAN (Underwater Acoustic Network) was aimed at conceiving, developing, and testing at sea an innovative and operational concept for integrating underwater and above‐water sensors in a unique communication system to protect offshore and coastline critical infrastructures. This work gives details on the underwater part of the project. It introduces a set of original security features and gives details on the integration of autonomous underwater vehicles (AUVs) as mobile nodes of the network and as surveillance assets, acoustically controlled by the command and control center to respond against intrusions. Field results are given of the final UAN project sea trial, UAN11, held in May 2011 in Norway. During the experimental activities, a UAN composed of four fixed nodes, two AUVs, and one mobile node mounted on the supporting research vessel was operated continuously and integrated into a global protection system. In this article, the communication performance of the network is reported in terms of round‐trip time, packet loss, and average delivery ratio. The major results of the experiment can be thus summarized: the implemented network structure was successful in continuously operating over five days with nodes seamlessly entering and exiting the network; the performance of the network varied greatly with fluctuations in the acoustic channel; the addition of security features induced a minor degradation in network performance with respect to channel variation; the AUVs were successfully controlled from a remote station through acoustic signals routed by the network. © 2013 Wiley Periodicals, Inc.     

Modified set-point controller for underwater vehicles

A modified PD (proportional-derivative) controller for autonomous underwater vehicles (AUVs) is presented in this paper. The controller is expressed in terms of first-order equations of motion with a unit inertia matrix. The main difference between the proposed controller and the classical one relies on that the dynamics of the system is taken into account. This property ensures fast error and force convergence to the end-value. The PD controller can be applied for fully actuated AUVs. It is worth noting that the regulator gain matrices are selected based on parameters of the tested vehicle. The stability of the presented control law is proved in the sense of Lyapunov. Moreover, some advantages and observations resulting from the use of the controller are given. The performance of the controller is shown via simulations on a 6-DOF underwater vehicle.     

Modified set-point controller for underwater vehicles

A modified PD (proportional-derivative) controller for autonomous underwater vehicles (AUVs) is presented in this paper. The controller is expressed in terms of first-order equations of motion with a unit inertia matrix. The main difference between the proposed controller and the classical one relies on that the dynamics of the system is taken into account. This property ensures fast error and force convergence to the end-value. The PD controller can be applied for fully actuated AUVs. It is worth noting that the regulator gain matrices are selected based on parameters of the tested vehicle. The stability of the presented control law is proved in the sense of Lyapunov. Moreover, some advantages and observations resulting from the use of the controller are given. The performance of the controller is shown via simulations on a 6-DOF underwater vehicle.     

Improving the coordination efficiency of limited‐communication multi–autonomus underwater vehicle operations using a multiagent architecture

This research addresses the problem of coordinating multiple autonomous underwater vehicle (AUV) operations. An intelligent mission executive has been created that uses multiagent technology to control and coordinate multiple AUVs in communication‐deficient environments. By incorporating real‐time vehicle prediction, blackboard‐based hierarchical mission plans, mission optimization, and a distributed multiagent–based paradigm in conjunction with a simple broadcast communication system, this research aims to handle the limitations inherent in underwater operations, namely poor communication, and intelligently control multiple vehicles. In this research, efficiency is evaluated and then compared to the current state of the art in multiple AUV control. The research is then validated in real AUV coordination trials. Results will show that compared to the state of the art, the control system developed and implemented in this research coordinates multiple vehicles more efficiently and is able to function in a range of poor communication environments. These findings are supported by in‐water validation trials with heterogeneous AUVs. © 2010 Wiley Periodicals, Inc.     

Cooperative search method for multiple AUVs based on target clustering and path optimization

Natural Computing - A search method for uncertain targets using multiple autonomous underwater vehicles (AUVs) is studied. To improve search efficiency, a cooperative search method based on target...     

多AUV协同导航问题的研究现状与进展

自主水下航行器(Autonomous underwater vehicle, AUV)协同导航是未来50年解决水下中间层区域AUV导航定位的重要方法. 本文针对多AUV协同导航, 对该领域相关问题的研究进展进行了综述, 包括: 1)论述多AUV协同导航领域的研究现状, 包括协同导航问题界定、特点综述与讨论; 2)分析多AUV协同导航系统模型及相关算法的研究进展, 包括基于优化的、 基于参数估计的和基于贝叶斯估计的滤波算法; 3)对协同导航网络下的误差建模与补偿方法的研究进展进行了综述, 包括未知洋流的影响、水声通信延迟补偿等; 4)从影响协同导航定位精度的角度出发, 对AUV协同导航的可观测性与编队最优构型设计的研究进展进行了一系列的分析; 5)陈述目前多AUV协同导航中存在的关键问题, 并讨论其发展趋势.     

基于凸优化算法的无人水下航行器协同定位

In this paper, a cooperative localization algorithm for autonomous underwater vehicles (AUVs) is proposed. A ``parallel" model is adopted to describe the cooperative localization problem instead of the traditional ``leader-follower" model, and a linear programming associated with convex optimization method is used to deal with the problem. After an unknown-but-bounded model for sensor noise is assumed, bearing and range measurements can be modeled as linear constraints on the configuration space of the AUVs. Merging these constraints induces a convex polyhedron representing the set of all configurations consistent with the sensor measurements. Estimates for the uncertainty in the position of a single AUV or the relative positions of two or more nodes can then be obtained by projecting this polyhedron onto appropriate subspaces of the configuration space. Two different optimization algorithms are given to recover the uncertainty region according to the number of the AUVs. Simulation results are presented for a typical localization example of the AUV formation. The results show that our positioning method offers a good localization accuracy, although a small number of low-cost sensors are needed for each vehicle, and this validates that it is an economical and practical positioning approach compared with the traditional approach.     

Globally Stable Adaptive Dynamic Surface Control for Cooperative Path Following of Multiple Underactuated Autonomous Underwater Vehicles

The cooperative path following problem of multiple underactuated autonomous underwater vehicles (AUVs) involves two tasks. The first one is to force each AUV to converge to the desired parameterized path. The second one is to satisfy the requirement of a cooperative behavior along the paths. In this paper, both of the tasks have been further studied. For the first one, a simplified path following controller is proposed by incorporating the dynamic surface control (DSC) technique to avoid the calculation of derivatives of virtual control signals. Besides, in order to handle the uncertain dynamics, a new type of neural network (NN) adaptive controller is derived, and then an NN based energy‐efficient path following controller is firstly proposed, which consists of an adaptive neural controller dominating in the neural active region and an extra robust controller working outside the neural active region. For the second one, in order to reduce the amount of communications between multiple AUVs, a distributed estimator for the reference common speed is firstly proposed as determined by the communications topology adopted, which means the global knowledge of the reference speed is relaxed for the problem of cooperative path following. The overall algorithm ensures that all the signals in the closed‐loop system are globally uniformly ultimately bounded (GUUB) and the output of the system converges to a small neighborhood of the reference trajectory by properly choosing the design parameters. Simulation results validate the performance and robustness of the proposed strategy.     

Imperialist Competitive Algorithm for AUV Path Planning in a Variable Ocean

An imperialist competitive algorithm (ICA) is introduced for solving the optimal path planning problem for autonomous underwater vehicles (AUVs) operating in turbulent, cluttered, and uncertain environments. ICA is a new sociopolitically inspired global search metaheuristic based on a form of competition between “imperialist” forces and opposing colonies. In this study, ICA is applied to optimize the coordinates of a set of control points for generating a curved spline path. The ICA-based path planner is tested to find an optimal trajectory for an AUV navigating through a variable ocean environment in the presence of an irregularly shaped underwater terrain. The genetic algorithm (GA) and quantum-behaved particle swarm optimization (QPSO) are described and evaluated with the ICA for the path optimization problem. Simulation results show that the proposed ICA approach is able to obtain a more optimized trajectory than the GA- or QPSO-based methods. Monte Carlo simulations demonstrate the robustness and superiority of the proposed ICA scheme compared with the GA and QPSO schemes.     

基于固定时间滑模干扰观测器的AUVs事件触发编队控制

研究多自主水下机器人(AUVs)编队控制过程中存在外部未知扰动、模型参数摄动和节约控制器传输网络能耗的问题,提出一种抗扰动的固定时间相对阈值事件触发编队控制方法.首先,给出自适应固定时间积分滑模干扰观测器(AFISMDO),在固定时间内用来估计系统的复合扰动;其次,基于所设计的AFISMDO,将非线性滤波器引入到反步算...     

基于多AUV间任务协作的水下多目标探测机制

利用自主式水下航行器（Autonomous Underwater Vehicle, AUV）对水下多目标进行协同探测是目前海洋技术领域的研究热点。本文主要研究在水下三维区间内的多AUV任务分配与协作探测机制,建立了以每个AUV能量耗费与能耗均衡为约束条件的水下三维空间中的多旅行商（Multiple Traveling Salesman Problem, MTSP）问题模型,利用遗传算法（Genetic Algorithm, GA）对该NP-Complete问题进行启发式求解,同时设计了考虑巡航总路径及访问目标数的适应度函数以提高多AUV间的能耗均衡性,实现多个AUV对多个水下目标的优化协同探测。最后本文利用Matlab R2014a软件对多AUV任务协作与多目标探测机制进行了仿真,仿真结果验证了该方法能够均衡多AUV多目标探测问题的能量消耗,进而提高巡航速度和生命周期。     

Underwater guidance of distributed autonomous underwater vehicles using one leader

Consider the case where autonomous underwater vehicles (AUVs) are deployed to monitor a 3D underwater environment. This paper tackles the problem of guiding all AUVs to the destination while not colliding with a priori unknown 3D obstacles. Suppose that among all AUVs, only the leader AUV has an ability of locating itself, while accessing a destination location. A follower, an AUV that is not a leader, has no sensors for locating itself. Every follower can only measure the relative position of its neighbor AUVs utilizing its sonar sensors. Our paper addresses distributed controls, so that multiple followers track the leader while preserving communication connectivity. We design controls, so that all AUVs reach the destination safely, while maintaining connectivity in cluttered 3D environments. To the best of our knowledge, our article is novel in developing 3D underwater guidance controls, so that all AUVs equipped with sonar sensors are guided to reach a destination in a priori unknown cluttered environments. MATLAB simulations are used to validate the proposed guidance scheme in underwater environments with many obstacles.     

Output feedback robust MPC with one free control move for the linear polytopic uncertain system with bounded disturbance

In this paper a previous approach, for the robust model predictive control (MPC) for a linear polytopic uncertain system, is extended to the case with bounded disturbance and unmeasurable state. The controller on-line optimizes a free control move followed by an output feedback control law based on the pre-specified state estimator. A key technique for this controller is an appropriate formulation of the estimation error bound which accounts for recursive feasibility of the optimization problem. The quadratic boundedness (QB) of the augmented state is guaranteed by the proposed approach. A numerical example is given to illustrate the effectiveness of the proposed controller.     

基于复合AUVs的水声传感网拓扑优化机制

为提高网络拓扑的健壮性和自适应性, 基于拓扑重构思想提出基于复合自主式水下航行器的水声传感网拓扑智能优化机制来提高网络的连通性; 进一步, 在发现水声传感网拓扑结构形成的内在规律的基础上, 提出三角淡化关键节点网络拓扑优化方法, 该方法有助于提高网络抗毁再生能力. 最后通过仿真实验对比分析验证了所提出机制的有效性, 实验结果表明, 在牺牲约8.5\%覆盖度的条件下, 抗毁性能提高约50\%.     

Naval Mine Countermeasure Missions

Undersea operations using autonomous underwater vehicles (AUVs) provide a different and in some ways a more challenging problem than tasks for unmanned aerial vehicles and unmanned ground vehicles. In particular, in undersea operations, communication windows are restricted, and bandwidth is limited. Consequently, coordination among agents is correspondingly more difficult. In traditional approaches, a central planner initially assigns subtasks to a set of AUVs to achieve the team goal. However, those initial task assignments may become inefficient during real-time execution because of the real-world issues such as failures. Therefore, initial task allocations are usually subject to change if efficiency is a high concern. Reallocations are needed and should be performed in a distributed manner. To provide such flexibility, we propose a distributed auction-based cooperation framework, distributed and efficient multirobot-cooperation framework (DEMiR-CF), which is an online dynamic task allocation (reallocation) system that aims to achieve a team goal while using resources effectively. DEMiR-CF, with integrated task scheduling and execution capabilities, can also respond to and recover from real-time contingencies such as communication failures, delays, range limitations, and robot failures.     

Cooperative bathymetry-based localization using low-cost autonomous underwater vehicles

We present a cooperative bathymetry-based localization approach for a team of low-cost autonomous underwater vehicles (AUVs), each equipped only with a single-beam altimeter, a depth sensor and an acoustic modem. The localization of the individual AUV is achieved via fully decentralized particle filtering, with the local filter’s measurement model driven by the AUV’s altimeter measurements and ranging information obtained through inter-vehicle communication. We perform empirical analysis on the factors that affect the filter performance. Simulation studies using randomly generated trajectories as well as trajectories executed by the AUVs during field experiments successfully demonstrate the feasibility of the technique. The proposed cooperative localization technique has the potential to prolong AUV mission time, and thus open the door for long-term autonomy underwater.     

An autonomous underwater vehicle and SUNSET to bridge underwater networks composed of multi-vendor modems

Nowadays, underwater acoustic communication is an interesting topic of research. This is especially true in the case of Autonomous Underwater Vehicles (AUVs): due to the limitations of the underwater environment, where WiFi and radio communications are limited, it is very important to rely on efficient and reliable tools for acoustic data exchange. Indeed, an underwater network of acoustic nodes can enhance the navigation and localization capabilities of one or more vehicles. However, cooperation and networking is not feasible in underwater networks composed of vehicles with different software architecture and multi-vendor modems due to the lack of communication standards. In this paper, the authors investigate the use of an AUV as a mobile bridging node within a network of fixed heterogeneous acoustic modems. In particular, MARTA AUV, developed and built by the Department of Industrial Engineering of the University of Florence, is employed to enable data transmission between modems of different manufacturers. The networking and the handling of multiple and multi-vendor modems are instead performed through the SUNSET Software Defined Communication Stack (SDCS) framework, developed by University of Rome La Sapienza jointly with WSENSE Srl. Different bridging tests have been performed in field to evaluate the feasibility of such approach, showing that AUVs can efficiently accomplish the role of gateway between the nodes of the network communicating through the SUNSET SDCS framework.     

An Empirical Evaluation of Co-ordination Strategies for an AUV and UAV

We propose a framework for cooperative search using a combination of an unmanned aerial vehicle (UAV) and an autonomous underwater vehicle (AUV). Such a combination allows search platforms to adapt to changes in both mission objectives and environmental parameters. We propose three strategies for coordination between an UAV and AUV to maximize the area explored while minimizing the idle time of the UAV and AUV. We evaluate the efficacy of these strategies while varying the speed, communication range and the number of targets. Preliminary results suggest the feasibility of our approach to combine UAVs and AUVs for effectively searching a given area.