A vision-based strategy for autonomous aerial refueling tasks

Autonomous aerial refueling is a key enabling technology for both manned and unmanned aircraft where extended flight duration or range are required. The results presented within this paper offer one potential vision-based sensing solution, together with a unique test environment. A hierarchical visual tracking algorithm based on direct methods is proposed and developed for the purposes of tracking a drogue during the capture stage of autonomous aerial refueling, and of estimating its 3D position. Intended to be applied in real time to a video stream from a single monocular camera mounted on the receiver aircraft, the algorithm is shown to be highly robust, and capable of tracking large, rapid drogue motions within the frame of reference. The proposed strategy has been tested using a complex robotic testbed and with actual flight hardware consisting of a full size probe and drogue. Results show that the vision tracking algorithm can detect and track the drogue at real-time frame rates of more than thirty frames per second, obtaining a robust position estimation even with strong motions and multiple occlusions of the drogue.     

Additive-state-decomposition-based station-keeping control for autonomous aerial refueling

Dear editor,Aerial refueling (AR) is an effective method of increasing the endurance and range of aircraft by refueling them in flight[1,2].Station-keeping con...     

Optimal scheduling for refueling multiple autonomous aerial vehicles

The scheduling, for autonomous refueling, of multiple unmanned aerial vehicles (UAVs) is posed as a combinatorial optimization problem. An efficient dynamic programming (DP) algorithm is introduced for finding the optimal initial refueling sequence. The optimal sequence needs to be recalculated when conditions change, such as when UAVs join or leave the queue unexpectedly. We develop a systematic shuffle scheme to reconfigure the UAV sequence using the least amount of shuffle steps. A similarity metric over UAV sequences is introduced to quantify the reconfiguration effort which is treated as an additional cost and is integrated into the DP algorithm. Feasibility and limitations of this novel approach are also discussed.     

自主空中加油时变质量无人作战飞机非线性控制

针对自主空中加油中无人作战飞机(UCAV)位置保持问题, 进行了时变质量UCAV的动力学建模与非线性控制设计. 综合考虑了燃油传输对UCAV的质量、惯性矩阵和质心位置的影响, 基于相对于惯性系的状态变量, 推导了UCAV时变质量动力学方程. 通过引入谱半径, 将局部化自适应边界指令滤波反推方法应用于UCAV的位置保持控制. 使用逼近器对未知模型不确定性进行在线逼近. 对于固有逼近误差和外部扰动, 采用局部化自适应边界进行补偿. 通过指令滤波反推, 设计了相对位置、速度、姿态角和角速度四个反馈回路来保证UCAV的稳定性. 最后, 三种不同加油方案下的非线性仿真验证了非线性飞行控制律的有效性.     

An autonomous vision-based mobile robot

This paper describes the theoretical development and experimental implementation of a complete navigation procedure for use in an autonomous mobile robot for structured environments. Estimates of the vehicle's position and orientation are based on the rapid observation of visual cues located at discrete positions within the environment. The extended Kalman filter is used to combine these visual observations with sensed wheel rotations to produce optimal estimates continuously. The complete estimation procedure, as well as the control algorithm, developed are time independent. A naturally suitable quantity involving wheel rotations is used as the independent variable. One consequence of this choice is that the vehicle speed can be specified independently of the estimation and control algorithms. Reference paths are “taught” by manually leading the vehicle through the desired path. Estimates produced by the extended Kalman filter during this teaching session are then used to represent the geometry of the path. The tracking of taught reference paths is accomplished by controlling the position and orientation of the vehicle relative to the reference path. Time-independence path tracking has necessitated the development of a novel, geometry-based means for advancing along the reference path     

Recognition of a landing platform for unmanned aerial vehicles by using computer vision-based techniques

The use of Unmanned Aerial Vehicles (UAVs) is growing significantly for many and varied purposes. During the mission, an outdoor UAV is guided by following the planned path using GPS signals. However, the GPS capability may become defective or the environment may be GPS-denied, and an additional safety aid is therefore required for the automatic landing phase that is independent of GPS data. Most UAVs are equipped with machine vision systems which, together with onboard analysis, can be used for safe, automatic landing. This contributes greatly to the overall success of autonomous flight.This paper proposes an automatic expert system, based on image segmentation procedures, that assists safe landing through recognition and relative orientation of the UAV and platform. The proposed expert system exploits the human experience that has been incorporated into the machine vision system, which is mapped into the proposed image processing modules. The result is an improved reliability capability that could be incorporated into any UAV, and is especially robust for rotary wing UAVs. This is clearly a desirable fail-safe capability.     

A contribution to vision-based autonomous helicopter flight in urban environments

A navigation strategy that exploits the optic flow and inertial information to continuously avoid collisions with both lateral and frontal obstacles has been used to control a simulated helicopter flying autonomously in a textured urban environment. Experimental results demonstrate that the corresponding controller generates cautious behavior, whereby the helicopter tends to stay in the middle of narrow corridors, while its forward velocity is automatically reduced when the obstacle density increases. When confronted with a frontal obstacle, the controller is also able to generate a tight U-turn that ensures the UAV’s survival. The paper provides comparisons with related work, and discusses the applicability of the approach to real platforms.     

A sonar approach to obstacle detection for a vision-based autonomous wheelchair

An advanced prototype Computer Controlled Power Wheelchair Navigation System or CCPWNS has been developed to provide autonomy for highly disabled users, whose mix of disabilities makes it difficult or impossible to control their own power chairs in their homes. The working paradigm is “teach and repeat” a mode of control for typical industrial holonomic robots. Ultrasound sensors, which during subsequent autonomous tracking will be used to detect obstacles, also are active during teaching. Based upon post-processed data collected during this teaching event, elaborate trajectories–which may involve multiple direction changes, pivoting and so on, depending upon the requirements of the typically restricted spaces within which the chair must operate–will later be called upon by the disabled rider. An off-line postprocessor assigns an ultrasound profile to the sequence of poses of any taught trajectory. Use of this profile during tracking obviates most of the inherent problems of using ultrasound to avoid obstacles while retaining the ability to near solid objects, such as when passing through a narrow doorway, where required by the environment and trajectory objectives. The work in this article describes a procedure to obtain consistent maps of sonar boundaries during the teaching process, and a preliminary approach to use this information during the tracking phase. The approach is illustrated by results obtained by using the CCPWNS prototype.     

基于L1自适应的自动空中加油对接段飞行控制技术

在自动空中加油(AAR,automated aerialre fueling)对接过程中,加油机后方拖出的加油软管锥套受到加油机和受油机的双重气动干扰,呈现不规则摆动运动,受油机要实现与加油锥套的精确对接,要求其飞行控制系统具有鲁棒性和快速的自适应能力,为此提出采用自适应控制器方案,该方案以线性二次调节器(linear quadraticregulator,LQR)比例积分型控制器作为稳定闭环,在此基础上加入自适应控制器,仿真结果表明,采用自适应控制器的受油机自动空中加油飞行控制系统可以实现规定时间内的精确加油对接,既满足瞬态性能要求,又满足稳态精度要求,同时,还解决了由自适应参数跳动带来的舵机操纵过于频繁的问题.该方法可有效提高对接过程中受油机飞行控制系统的抗干扰能力,能够满足自动空中加油对接段的飞行控制要求.     

A vision-based landing system for small unmanned aerial vehicles using an airbag

Statistics show that the landing accounts for the largest portion of all mishaps of unmanned aerial vehicles (UAVs) due to many difficulties including limited situational awareness of the external pilot and the limited maneuverability during the low speed flight before touchdown. In this paper, a vision-based automatic landing system using a dome-shaped airbag is proposed for small UAVs. Its isotropic shape allows airplanes to approach in any direction to avoid crosswind unlike net-assisted landing. The dome’s distinctive color improves the detection owing to its strong visual cue. Color- and shape-based detection vision algorithms are applied for robust detection under varying lighting conditions. Due to the insufficient accuracy of navigation sensors, a direct visual servoing is used for terminal guidance. The proposed algorithm is validated in a series of flight tests.     

Real-time machine-vision-based position sensing system for UAV aerial refueling

This paper describes the design of a Real Time Machine-Vision (MV) Position Sensing System for the problem of Semi-Autonomous Docking within Aerial Refueling (AR) for Unmanned Aerial Vehicles (UAVs). In this effort, techniques and algorithms have been developed and extensively tested in the MATLAB/Simulink^® Soft Real-Time environment as well as in Linux/RTAI Hard Real-Time environment. The overall MV software performs several tasks, such as the image acquisition from a real camera, the Feature Extraction (FE) from the acquired image, the Detection and Labeling (DAL) of the features, and the tanker-UAV Pose Estimation (PE). A Cyclic Asynchronous Buffer (CAB) mechanism was implemented for inter-process communication among Real Time and Non Real Time processes. The entire sensing system was tested using an 800 MHz PC-104 computer. The results confirmed the feasibility of executing image processing algorithms in real-time using off-the-shelf commercial hardware to obtain reliable relative position and orientation estimations.     

Omnidirectional vision-based ego-pose estimation for an autonomous in-pipe mobile robot

This paper describes the omnidirectional vision-based ego-pose estimation method of an in-pipe mobile robot. An in-pipe mobile robot has been developed for inspecting the inner surface of various pipeline configurations, such as the straight pipeline, the elbow and the multiple-branch. Because the proposed in-pipe mobile robot has four individual drive wheels, it has the ability of flexible motions in various pipelines. The ego-pose estimation is indispensable for the autonomous navigation of the proposed in-pipe robot. An omnidirectional camera and four laser modules mounted on the mobile robot are used for ego-pose estimation. An omnidirectional camera is also used for investigating the inner surface of the pipeline. The pose of the in-pipe mobile robot is estimated from the relationship equation between the pose of a robot and the pixel coordinates of four intersection points where light rays that emerge from four laser modules intersect the inside of the pipeline. This relationship equation is derived from the geometry analysis of an omnidirectional camera and four laser modules. In experiments, the performance of the proposed method is evaluated by comparing the result of our algorithm with the measurement value of a specifically designed sensor, which is a kind of a gyroscope.     

Development of an autonomous aerial vehicle: A case study

This paper describes the design, development, and deployment of an unmanned autonomous aerial vehicle developed at the Georgia Institute of Technology during 1990–1991. The approach taken, the system architecture, and the embedded intelligence of the project as conceived by a team of students, faculty, and industrial affiliates is reported. The project focused on engineering a vehicle which performed an intended mission in the time, space, and weight restrictions specified as part of an AUVS 1991 Competition. This paper documents the system and its various components and also provides a discussion of integration issues.The project demonstrated capabilities of existing and new technologies, but also highlighted many serious integration issues, particularly when using prototype components. The project also demonstrated the utility and mutual benefits of academic-industry projects. All members of the team benefited by working on a real and tangible project. Industrial participates gained first hand experience integrating their products with other components and many saw potential for their products and services in new markets.     

Introducing autonomous aerial robots in industrial manufacturing

Although ground robots have been successfully used for many years in manufacturing, the capability of aerial robots to agilely navigate in the often sparse and static upper part of factories makes them suitable for performing tasks of interest in many industrial sectors. This paper presents the design, development, and validation of a fully autonomous aerial robotic system for manufacturing industries. It includes modules for accurate pose estimation without using a Global Navigation Satellite System (GNSS), autonomous navigation, radio-based localization, and obstacle avoidance, among others, providing a fully onboard solution capable of autonomously performing complex tasks in dynamic indoor environments in which all necessary sensors, electronics, and processing are on the robot. It was developed to fulfill two use cases relevant in many industries: light object logistics and missing tool search. The presented robotic system, functionalities, and use cases have been extensively validated with Technology Readiness Level 7 (TRL-7) in the Centro Bahía de Cádiz (CBC) Airbus D&S factory in fully working conditions.     

不确定环境下的软管-锥套建模及控制研究

提出了一种能有效适应不确定性环境的锥套稳定控制方法。在对软管-锥套非线性动力学模型线性化及降阶处理的基础上,将大气紊流对软管-锥套的扰动视为随机扰动,并把这种扰动作为参数与软管-锥套系统降阶模型中的状态合并成增广的系统状态量,采用卡尔曼滤波方法对系统状态量进行实时估计,并设计LQG控制器,实现对锥套的稳定控制。仿真结果表明,该方法能实时跟踪大气紊流的变化,且具有较好的抗大气紊流干扰的能力,能保证锥套的稳定。     

软式空中加油半实物仿真系统研究

为模拟软式空中加油相对运动过程,研究加油机和受油机在近距离时的气动影响和飞行接近策略,提出了一种考虑尾涡流、头波效应影响的软管-加油伞运动模型,仿真实验表明该模型能够准确模拟软式空中加油相对运动过程,在某型飞机大型球幕飞行训练模拟器的基础上,开发了一套基于该运动模型的软式空中加油半实物仿真模拟训练系统,对加油伞组件控制...     

空中加油场景下的目标联合检测跟踪算法

针对自主空中加油对接阶段的目标跟踪问题,提出一种空中加油场景下的目标联合检测跟踪算法。该算法采用检测跟踪一体化的CenterTrack网络实现对锥套的追踪,而针对计算量较大、训练耗时过长的问题,分别从模型设计与网络优化两方面改善该网络。首先,在跟踪器中引入膨胀卷积组,以在不改变感受野大小的前提下使得网络轻量化;同时,将输出部分的卷积层替换为深度可分离卷积层,从而减少网络的参数量与计算量;然后,对网络进行进一步的优化,即将随机梯度下降（SGD）法与Adam算法相结合,使网络更快收敛至稳定状态;最后,利用真实的空中加油场景视频与地面模拟视频制作相应格式的数据集,并将其用于实验验证。分别在自制的锥套数据集和MOT17公共数据集上进行了训练与测试,证实了提出算法的有效性。相较于原CenterTrack网络,改进的网络Tiny-CenterTrack减少了约48.6%的训练时长,并在实时性方面提升了8.8%。实验结果表明,改进后的网络在不损失网络性能的前提下可有效节省计算资源并在一定程度上提升实时性。     

Addressing corner detection issues for machine vision based UAV aerial refueling

This paper describes the results of the analysis of specific ‘corner detection’ algorithms within a Machine Vision approach for the problem of aerial refueling for unmanned aerial vehicles. Specifically, the performances of the SUSAN and the Harris corner detection algorithms have been compared. A critical goal of this study was to evaluate the interface of these feature extraction schemes with the successive detection and labeling, and pose estimation schemes in the overall scheme. Closed-loop simulations were performed using a Simulink^®-based simulation environment to reproduce docking maneuvers using the US Air Force refueling boom.     

Comparison of point matching algorithms for the UAV aerial refueling problem

This paper describes the results of a study focused on the evaluation of the performance of specific algorithms for the “point matching” task within the general problem of applying machine vision-based control laws for the problem of aerial refueling (AR) for UAVs. Two different point matching algorithms for the identification of the corner-points of the tanker are proposed. A detailed study of the algorithms is performed with special emphasis on the correct matching and required computational effort. The results show the importance of a correct point-matching scheme for obtaining accurate pose estimation; furthermore, the analysis highlights the tradeoffs involved in the selection of the appropriate algorithms.