On the design and development of attitude stabilization, vision-based navigation, and aerial gripping for a low-cost quadrotor

This paper presents the design and development of autonomous attitude stabilization, navigation in unstructured, GPS-denied environments, aggressive landing on inclined surfaces, and aerial gripping using onboard sensors on a low-cost, custom-built quadrotor. The development of a multi-functional micro air vehicle (MAV) that utilizes inexpensive off-the-shelf components presents multiple challenges due to noise and sensor accuracy, and there are control challenges involved with achieving various capabilities beyond navigation. This paper addresses these issues by developing a complete system from the ground up, addressing the attitude stabilization problem using extensive filtering and an attitude estimation filter recently developed in the literature. Navigation in both indoor and outdoor environments is achieved using a visual Simultaneous Localization and Mapping (SLAM) algorithm that relies on an onboard monocular camera. The system utilizes nested controllers for attitude stabilization, vision-based navigation, and guidance, with the navigation controller implemented using a nonlinear controller based on the sigmoid function. The efficacy of the approach is demonstrated by maintaining a stable hover even in the presence of wind gusts and when manually hitting and pulling on the quadrotor. Precision landing on inclined surfaces is demonstrated as an example of an aggressive maneuver, and is performed using only onboard sensing. Aerial gripping is accomplished with the addition of a secondary camera, capable of detecting infrared light sources, which is used to estimate the 3D location of an object, while an under-actuated and passively compliant manipulator is designed for effective gripping under uncertainty. The quadrotor is therefore able to autonomously navigate inside and outside, in the presence of disturbances, and perform tasks such as aggressively landing on inclined surfaces and locating and grasping an object, using only inexpensive, onboard sensors.     

Visual servoing for underactuated VTOL UAVs: a linear,homography‐based framework

The paper concerns the control of vertical take‐off and landing (VTOL) underactuated aerial vehicles (UAVs) in hover flight, on the basis of measurements provided by an onboard video camera. The objective is to stabilize the vehicle to the equilibrium pose associated with an image of a planar target, using a minimal sensor suite and poor knowledge of the environment. By using the homography matrix computed from the camera measurements of the target, stabilizing feedback laws are derived on the basis of the visual data and gyrometer measurements only. Explicit stability conditions on the control parameters are provided, showing that a proper tuning of the control parameters ensures a large robustness margin with only planar target and visibility assumptions, although the target size and orientation, the UAV position, linear velocity and orientation are unknown. Additional issues, such as the use of accelerometers to improve the UAV's positioning in the case of unmodeled dynamics (such as wind), are also considered. Copyright © 2013 John Wiley & Sons, Ltd.     

A Cross-Platform Comparison of Visual Marker Based Approaches for Autonomous Flight of Quadrocopters

In this paper, we compare three different marker based approaches for six degrees of freedom (6DOF) pose estimation, which can be used for position and attitude control of micro aerial vehicles (MAV). All methods are able to achieve real time pose estimation onboard without assistance of any external metric sensor. Since these methods can be used in various working environments, we compare their performance by carrying out experiments across two different platforms: an AscTec Hummingbird and a Pixhawk quadrocopter. We evaluate each method’s accuracy by using an external tracking system and compare the methods with respect to their operating ranges and processing time. We also compare each method’s performance during autonomous takeoff, hovering and landing of a quadrocopter. Finally we show how the methods perform in an outdoor environment. The paper is an extended version of the one with the same title published at the ICUAS Conference 2013.     

Information fusion for automotive applications – An overview

This article focusses on the fusion of information from various automotive sensors like radar, video, and lidar for enhanced safety and traffic efficiency. Fusion is not restricted to data from sensors onboard the same vehicle but vehicular communication systems allow to propagate and fuse information with sensor data from other vehicles or from the road infrastructure as well. This enables vehicles to perceive information from regions that are hardly accessible otherwise and represents the basis for cooperative driving maneuvers. While the Bayesian framework builds the basis for information fusion, automobile environments are characterized by their a priori unknown topology, i.e., the number, type, and structure of the perceived objects is highly variable. Multi-object detection and tracking methods are a first step to cope with this challenge. Obviously, the existence or non-existence of an object is of paramount importance for safe driving. Such decisions are highly influenced by the association step that assigns sensor measurements to object tracks. Methods that involve multiple sequences of binary assignments are compared with soft-assignment strategies. Finally, fusion based on finite set statistics that (theoretically) avoid an explicit association are discussed.     

Vision‐based autonomous convoying with constant time delay

This paper describes the design and experimental validation of a vision‐based vehicle‐following system that uses only onboard sensors to enable a convoy of follower vehicles to autonomously track the trajectory of a manually driven lead vehicle. The tracking is done using the concept of a constant time delay, in which a follower tracks the delayed trajectory of its leader. This constant‐time‐delay approach allows for new techniques to be used to estimate the speed and heading of the leader. Experiments were conducted with full‐sized military vehicles on a 1.3‐km test track. Successful field trials with one follower for 10 laps and with two followers for 13.5 laps, totaling over 30 km, are presented. © 2010 Government of Canada. Exclusive worldwide publication rights in the article have been transferred to Wiley Periodicals, Inc., AWiley Company.     

Vision-based autonomous tracking and landing of a fully-actuated rotorcraft

Autonomous landing is a challenging phase of flight for an aerial vehicle, especially when attempting to land on a moving target. This paper presents vision-based tracking and landing of a fully-actuated tilt-augmented quadrotor on a moving target. A fully-actuated vehicle allows higher freedom in terms of control design and a larger flight envelope since the position and attitude states are decoupled. An adaptive control law is designed to track a moving target with only relative position information from a camera. Low-cost hardware is used, and experiments are carried out to validate the proposed methodology for targets moving at realistic speeds.     

Bioinspired design of a landing system with soft shock absorbers for autonomous aerial robots

One of the main challenges for autonomous aerial robots is to land safely on a target position on varied surface structures in real‐world applications. Most of current aerial robots (especially multirotors) use only rigid landing gears, which limit the adaptability to environments and can cause damage to the sensitive cameras and other electronics onboard. This paper presents a bioinpsired landing system for autonomous aerial robots, built on the inspire–abstract–implement design paradigm and an additive manufacturing process for soft thermoplastic materials. This novel landing system consists of 3D printable Sarrus shock absorbers and soft landing pads which are integrated with an one‐degree‐of‐freedom actuation mechanism. Both designs of the Sarrus shock absorber and the soft landing pad are analyzed via finite element analysis, and are characterized with dynamic mechanical measurements. The landing system with 3D printed soft components is characterized by completing landing tests on flat, convex, and concave steel structures and grassy field in a total of 60 times at different speeds between 1 and 2 m/s. The adaptability and shock absorption capacity of the proposed landing system is then evaluated and benchmarked against rigid legs. It reveals that the system is able to adapt to varied surface structures and reduce impact force by 540N at maximum. The bioinspired landing strategy presented in this paper opens a promising avenue in Aerial Biorobotics, where a cross‐disciplinary approach in vehicle control and navigation is combined with soft technologies, enabled with adaptive morphology.     

AUV 水下着陆策略研究

水下自航行器（AUV）的续航能力主要取决于其所携带能源总量．为了有效地减少能耗,提出了 一种具有变浮力系统、能够着陆坐底的小型AUV,它可以利用有限的能源实现长时间的海洋环境监测．首先 介绍了该AUV 的总体结构,然后对AUV 的着陆策略进行了详细研究．在对三种着陆策略进行对比的基础上, 选择下潜航行到位控制注水着陆策略作为最佳的水下着陆方案．该策略能够使着陆时的冲击力保持在安全范 围内,并且着陆时间最短．最后对着陆轨迹进行规划．仿真和水域实验的结果都证明采用该着陆策略能够安 全、平稳地实现水下着陆．     

A distributed architecture for a robotic platform with aerial sensor transportation and self‐deployment capabilities

This paper presents the architecture developed in the framework of the AWARE project for the autonomous distributed cooperation between unmanned aerial vehicles (UAVs), wireless sensor/actuator networks, and ground camera networks. One of the main goals was the demonstration of useful actuation capabilities involving multiple ground and aerial robots in the context of civil applications. A novel characteristic is the demonstration in field experiments of the transportation and deployment of the same load with single/multiple autonomous aerial vehicles. The architecture is endowed with different modules that solve the usual problems that arise during the execution of multipurpose missions, such as task allocation, conflict resolution, task decomposition, and sensor data fusion. The approach had to satisfy two main requirements: robustness for operation in disaster management scenarios and easy integration of different autonomous vehicles. The former specification led to a distributed design, and the latter was tackled by imposing several requirements on the execution capabilities of the vehicles to be integrated in the platform. The full approach was validated in field experiments with different autonomous helicopters equipped with heterogeneous devices onboard, such as visual/infrared cameras and instruments to transport loads and to deploy sensors. Four different missions are presented in this paper: sensor deployment and fire confirmation with UAVs, surveillance with multiple UAVs, tracking of firemen with ground and aerial sensors/cameras, and load transportation with multiple UAVs. © 2011 Wiley Periodicals, Inc.     

Maximizing Water Surface Target Localization Accuracy Under Sunlight Reflection with an Autonomous Unmanned Aerial Vehicle

Reflected sunlight can significantly impact the effectiveness of vision-based object detection and tracking algorithms, especially ones developed for an aerial platform operating over a marine environment. These algorithms often fail to detect water surface objects due to sunlight glitter or rapid course corrections of unmanned aerial vehicles (UAVs) generated by the laws of aerodynamics. In this paper, we propose a UAV path planning method that maximizes the stationary or mobile target detection likelihood during localization and tracking by minimizing the sunlight reflection influences. In order to better reduce sunlight reflection effects, an image-based sunlight reflection reception adjustment is also proposed. We validate our method using both stationary and mobile target tracking tests.     

Motion control and obstacle avoidance of a mobile robot with an onboard manipulator

Navigation and control of autonomous mobile vehicles with onboard manipulator systems are currently being investigated for intelligent manufacturing applications. A systematic approach for modeling and base motion control of a mobile vehicle with an onboard robot arm is presented. Feedback linearization is used to take into account the complete dynamics with non-holonomic constraints, yet methods from potential field theory are incorporated to provide resolution among possibly conflicting performance goals (e.g. path following and obstacle avoidance). The feedback linearization provides an inner loop that accounts for possible motion of the onboard arm. The two cases of maintaining a desired course and speed, and following a desired Cartesian trajectory are considered. The outer control loop is designed using potential field theory, with the two objectives of homing and avoiding an obstacle. This simple result obtained using potential functions provides very naturally the necessary intelligence for online resolution of conflicting performance objectives. It gives capabilities to these autonomous vehicles for maintaining a desired course and speed or tracking a Cartesian trajectory, avoiding obstacles during the course of travel, and initiating new online path planning when the size of the object is large so that unnecessary wandering in the work space is avoided.     

基于改进TLD算法的无人机自主精准降落

四旋翼无人机（Unmanned Aerial Vehicle,UAV）在航拍、测绘、环境监测、快递等航空领域的广泛应用,对四旋翼无人机的可用性和可靠性提出了更高的要求,而其实现自主精准降落的功能是必不可少的。对目标进行快速鲁棒性跟踪是实现降落的重要基础,TLD（Tracking Learning Detector）算法为这一问题提供了一种有效的解决办法,虽然许多学者对其进行了研究并对传统的TLD算法进行了改进,但算法的跟踪精度及速度仍然难以满足无人机的降落要求。提出了一种基于TLD框架的目标跟踪算法来实现无人机与特定降落目标之间的相对定位。该算法在TLD框架下,提出一种基于目标形状特征自主确定降落目标的算法,提高了降落流程的自主性;用核相关滤波器（Kernelized Correlation Filter,KCF） 实现了TLD框架中的跟踪器,提高了算法的实时性、精准度及鲁棒性;同时在降落过程中采用一种基于方向梯度直方图特征（Histogram of Gradient,HOG）和支持向量机（Support Vector Machine,SVM） 的目标识别方法,以实现目标检测自矫正,保证长时间准确跟踪目标。在七类模拟无人机进行降落的视频集下验证了该算法,与其他三种跟踪算法进行对比,并进行实际降落测试。测试结果表明,该算法的鲁棒性和精准度均优于其他算法,处理速度可达到31.47?f/s,故而在TLD框架下采用核相关滤波器作为跟踪器,对跟踪及检测结果进行有效融合并提高算法实时性的同时,增加的检测自矫正环节保证了长时间跟踪的准确度,从而有效地实现了无人机全自主精准降落。     

一种多传感器反直升机智能雷伺服跟踪系统

讨论一种基于多传感器的反直升机智能雷AHM(Anti-Helicopter Mine)系统.为了提高智能雷的全自动智能跟踪能力和打击精度,在传统的被动声探测技术的基础上,结合图像传感器的视觉信息和激光测距仪的深度信息,提出一种基于声-光-电多传感器联合的自动目标探测、识别、跟踪算法.首先将五元十字声源定位技术用于低空目...     

Navigation Aided Image Processing in UAV Surveillance: Preliminary Results and Design of an Airborne Experimental System

This paper describes an airborne reconfigurable measurement system being developed at Swedish Defence Research Agency (FOI), Sensor Technology, Sweden. An image processing oriented sensor management architecture for UAV (unmanned aerial vehicles) IR/EO-surveillance is presented. Some preliminary results of navigation aided image processing in UAV applications are demonstrated, such as SLAM (simultaneous localization and mapping), structure from motion and geolocation, target tracking, and detection of moving objects. The design goal of the measurement system is to emulate a UAV-mounted sensor gimbal using a stand-alone system. The minimal configuration of the system consists of a gyro-stabilized gimbal with IR and CCD sensors and an integrated high-performance navigation system. The navigation system combines dGPS real-time kinematics (RTK) data with data from an inertial measurement unit (IMU) mounted with reference to the optical sensors. The gimbal is to be used as an experimental georeferenced sensor platform, using a choice of carriers, to produce military relevant image sequences for studies of image processing and sensor control on moving surveillance and reconnaissance platforms. Furthermore, a high resolution synthetic environment, developed for sensor simulations in the visual and infrared wavelengths, is presented. © 2004 Wiley Periodicals, Inc.     

An on-board vision sensor system for small unmanned vehicle applications

This paper describes an on-board vision sensor system that is developed specifically for small unmanned vehicle applications. For small vehicles, vision sensors have many advantages, including size, weight, and power consumption, over other sensors such as radar, sonar, and laser range finder, etc. A vision sensor is also uniquely suited for tasks such as target tracking and recognition that require visual information processing. However, it is difficult to meet the computing needs of real-time vision processing on a small robot. In this paper, we present the development of a field programmable gate array-based vision sensor and use a small ground vehicle to demonstrate that this vision sensor is able to detect and track features on a user-selected target from frame to frame and steer the small autonomous vehicle towards it. The sensor system utilizes hardware implementations of the rank transform for filtering, a Harris corner detector for feature detection, and a correlation algorithm for feature matching and tracking. With additional capabilities supported in software, the operational system communicates wirelessly with a base station, receiving commands, providing visual feedback to the user and allowing user input such as specifying targets to track. Since this vision sensor system uses reconfigurable hardware, other vision algorithms such as stereo vision and motion analysis can be implemented to reconfigure the system for other real-time vision applications.     

Obstacle Avoidance for Unmanned Aerial Vehicles

This work is framed within the PITVANT project and aims to contribute to the development of obstacle avoidance techniques for unmanned aerial vehicles (UAVs). The paper describes the design, implementation and experimental evaluation of a potential field obstacle avoidance algorithm based on the fluid mechanics panel methods. Obstacles and the UAV goal position are modeled by harmonic functions thus avoiding the presence of local minima. Adaptations are made to apply the method to the automatic control of a fixed wing aircraft, relying only on a local map of the environment that is updated with information from sensors onboard the aircraft. Hardware-In-Loop simulations show the good performance of the proposed algorithm in the envisioned mission scenarios for the PITVANT vehicles.     

Nonlinear filtering in target tracking using cooperative mobile sensors

Collaborative signal processing and sensor deployment have been among the most important research tasks in target tracking using networked sensors. In this paper, the mathematical model is formulated for single target tracking using mobile nonlinear scalar range sensors. Then a sensor deployment strategy is proposed for the mobile sensors and a nonlinear convergent filter is built to estimate the trajectory of the target.     

视频图像中的车型识别

文章介绍一种在固定单摄像头拍摄交通图像序列中检测车辆的方法。处理过程大致分为以下三步:重建不含运动目标的自然背景及图像分割;摄像机标定;目标区域的跟踪和车型识别。实验证明方法是可行的。     

Recognition of a landing platform for unmanned aerial vehicles by using computer vision-based techniques

The use of Unmanned Aerial Vehicles (UAVs) is growing significantly for many and varied purposes. During the mission, an outdoor UAV is guided by following the planned path using GPS signals. However, the GPS capability may become defective or the environment may be GPS-denied, and an additional safety aid is therefore required for the automatic landing phase that is independent of GPS data. Most UAVs are equipped with machine vision systems which, together with onboard analysis, can be used for safe, automatic landing. This contributes greatly to the overall success of autonomous flight.This paper proposes an automatic expert system, based on image segmentation procedures, that assists safe landing through recognition and relative orientation of the UAV and platform. The proposed expert system exploits the human experience that has been incorporated into the machine vision system, which is mapped into the proposed image processing modules. The result is an improved reliability capability that could be incorporated into any UAV, and is especially robust for rotary wing UAVs. This is clearly a desirable fail-safe capability.     

A Vision for Supporting Autonomous Navigation in Urban Environments

The autonomous navigation systems (ANS), such as autonomous ground vehicles (AGVs), unmanned aerial vehicles (UAVs), and unmanned submersible vehicles (USVs), and modern vehicles with actuators, sensors, and computer control perform three basic functions: context gathering using sensors, processing, and action. Most researchers have put all three functions into the ANS or the robot itself to overcome occlusions and handle the environment's dynamics. However, this causes the ANS and robotic systems to be bulky and expensive. It also impedes the introduction of vehicles with ANS in urban environments, where they must coexist with existing cars and highways. The approach presented distributes the context-gathering and processing functions using sensor networks and wireless communications technologies to reduce costs and make ANS widespread. The system uses sensors mounted on moving vehicles and stationary objects such as lampposts, traffic lights, toll plazas, and buildings to gather information at different levels