Passive range estimation for rotorcraft low-altitude flight

The automation of rotorcraft low-altitude flight presents challenging problems in control, computer vision, and image understanding. A critical element in this problem is the ability to detect and locate obstacles, using on-board sensors, and to modify the nominal trajectory. This requirement is also necessary for the safe landing of an autonomous lander on Mars. This paper examines some of the issues in the location of objects, using a sequence of images from a passive sensor, and describes a Kalman filter approach to estimate range to obstacles. The Kalman filter is also used to track features in the images leading to a significant reduction of search effort in the feature-extraction step of the algorithm. The method can compute range for both straightline and curvilinear motion of the sensor. An experiment is designed in the laboratory to acquire a sequence of images along with the sensor motion parameters under conditions similar to helicopter flight. The paper presents range estimation results using this imagery.     

Vision-based autonomous tracking and landing of a fully-actuated rotorcraft

Autonomous landing is a challenging phase of flight for an aerial vehicle, especially when attempting to land on a moving target. This paper presents vision-based tracking and landing of a fully-actuated tilt-augmented quadrotor on a moving target. A fully-actuated vehicle allows higher freedom in terms of control design and a larger flight envelope since the position and attitude states are decoupled. An adaptive control law is designed to track a moving target with only relative position information from a camera. Low-cost hardware is used, and experiments are carried out to validate the proposed methodology for targets moving at realistic speeds.     

Vision-based control of near-obstacle flight

This paper presents a novel control strategy, which we call optiPilot, for autonomous flight in the vicinity of obstacles. Most existing autopilots rely on a complete 6-degree-of-freedom state estimation using a GPS and an Inertial Measurement Unit (IMU) and are unable to detect and avoid obstacles. This is a limitation for missions such as surveillance and environment monitoring that may require near-obstacle flight in urban areas or mountainous environments. OptiPilot instead uses optic flow to estimate proximity of obstacles and avoid them. Our approach takes advantage of the fact that, for most platforms in translational flight (as opposed to near-hover flight), the translatory motion is essentially aligned with the aircraft main axis. This property allows us to directly interpret optic flow measurements as proximity indications. We take inspiration from neural and behavioural strategies of flying insects to propose a simple mapping of optic flow measurements into control signals that requires only a lightweight and power-efficient sensor suite and minimal processing power.     

一种基于视觉的道路检测算法

计算机视觉导航使移动机器人可以工作于复杂环境中,而道路检测是其中的关键环节。针对道路的多样性和环境因素的影响,提出了一种稳健的道路检测算法。依据道路特征把图像分为3组区域：路、非路和不确定区域,然后对难以判断的不确定区域使用假设检验的策略,依据道路的形状、路宽和面积信息综合判断,把不确定区域合并到路或者非路区域,从而快速准确地检测出道路。该算法已经在移动机器人-ATRV上测试和使用。     

Vision-based vehicle detection for a driver assistance system

On the basis of a cost-effective embedded system, this work implements a preceding vehicle detection system by using computer vision technologies. The road scenes are acquired with a monocular camera. The features of the vehicle in front are extracted and recognized by the proposed refined image processing algorithm, and a tracking process based on optical flow is also applied for reducing the complexity of computing. The system also provides the longitudinal distance information for the further function of adaptive cruise control. Moreover, voice alerts and image recording will be activated if the distance is less than the safe range. A statistical base of 100 video road images are tested in our experiments; the natures of the vehicles include sedan, minivan, truck, and bus. The experimental results show that the proportion of correct identifications of proceeding vehicles is above 95.8%, testing on highways in the daytime. Experimental results also indicate that the system correctly identifies vehicles in real time.     

一种障碍物模型检测算法

针对观测场景结构复杂、建模困难的问题,提出了一种基于运动目标跟踪的场景障碍物模型检测方法.结合对目标的检测与跟踪过程,利用空间投影关系确定场景中障碍物的位置,并将其映射到场景模型上,即可得到场景观测模型.该方法只需要对目标运动进行简单地跟踪和统计,不需要进行三维结构恢复以及图像拼接处理,能够有效解决场景的在线自适应建模问题.     

Vision-based entrance detection in outdoor scenes

Doors are a significant object for the visually impaired and robots to enter and exit buildings. Although the accuracy of door detection is reported high in indoor scenes, it has become a difficult problem in outdoor scenes in computer vision. The reason may lie in the fact that such properties of a simple ordinary door such as handles, corners, and the gap between the door and the ground may not be visible due to the great variety of doors in outdoor environments. In this paper, we present a vision-based method for detecting building entrances in outdoor images. After extracting the lines and deleting the extra ones, regions between the vertical lines are specified and the features including height, width, location, color, texture and the number of lines inside the regions are obtained. Finally, some additional knowledge such as door existence at the bottom of the image, a reasonable height and width of a door, the difference between color and texture of the doors and those of the neighboring regions, and numerous lines on doors is used to decide on door detection. The method was tested on the eTRIMS dataset, door images from the ImageNet dataset, and our own dataset including doors of houses, apartments, and stores leading to acceptable results. The obtained results show that our approach outperforms comparable state-of-the-art approaches.     

Robot manipulator self-identification for surrounding obstacle detection

Obstacle detection plays an important role for robot collision avoidance and motion planning. This paper focuses on the study of the collision prediction of a dual-arm robot based on a 3D point cloud. Firstly, a self-identification method is presented based on the over-segmentation approach and the forward kinematic model of the robot. Secondly, a simplified 3D model of the robot is generated using the segmented point cloud. Finally, a collision prediction algorithm is proposed to estimate the collision parameters in real-time. Experimental studies using the Kinect_? sensor and the Baxter_? robot have been performed to demonstrate the performance of the proposed algorithms.     

An evaluation of stereo and laser‐based range sensing for rotorcraft unmanned aerial vehicle obstacle avoidance

We present an evaluation of stereo vision and laser‐based range sensing for rotorcraft unmanned aerial vehicle (RUAV) obstacle avoidance. Our focus is on sensors that are suitable for mini‐RUAV class vehicles in terms of weight and power consumption. The study is limited to the avoidance of large static obstacles such as trees. We compare two commercially available devices that are representative of the state of the art in two‐dimensional scanning laser and stereo‐based sensing. Stereo is evaluated with three different focal length lenses to assess the tradeoff between range resolution and field of view (FOV). The devices are evaluated in the context of obstacle avoidance through extensive flight trials with an RUAV. We discuss the merits and limitations of each sensor type, including sensing range, FOV, accuracy, and susceptibility to lighting conditions. We show that the stereo device fitted with 8‐mm lenses has a better sensing range and vertical FOV than the laser device; however, it relies on careful calibration and is affected by high‐contrast outdoor lighting conditions. The laser has a wider horizontal FOV and is more reliable at detecting obstacles that are within a 20‐m range. Overall the laser produced superior obstacle avoidance performance, with a success rate of 84% compared to 42% for 8‐mm stereo. © 2012 Wiley Periodicals, Inc.     

A diffusion mechanism for obstacle detection from size-changeinformation

A mechanism fur the visual detection of obstacles is presented. A new immediacy measure, representing the imminence of collision between an object and a moving observer, is defined. A diffusion process on the image domain, whose initial condition is determined by the motion field normal to the object's boundary, is shown to converge asymptotically to the immediacy measure. A network of locally connected cells, derived from a finite-difference approximation of the diffusion equation, estimates the immediacy measure from normal velocity and boundary information provided by a motion measurement and segmentation stage. The algorithm's performance on real image sequences is demonstrated     

A spatial sampling criterion for sonar obstacle detection

A spatial sampling criterion for sonar systems that allows all obstacles within a given radius from the sensor to be detected is described. The environment considered is a two-dimensional floor plan that is extended into the third dimension, in which the scanning is performed in the horizontal plane. In this environment, edgelike reflectors, such as edges of doors or doorways, and oblique surfaces are the most difficult to detect. By considering the physics of sound propagation, the sonar scanning density required to detect these objects is determined. An experimental verification is included. The limitations of detecting objects with sonar in a more general environment are discussed. These results can be used to determine the necessary spacing in a transducer ring array and the maximum step size that a mobile robot can translate without danger of collision     

Context-aware obstacle detection for navigation by visually impaired

This paper presents a context-aware smartphone-based based visual obstacle detection approach to aid visually impaired people in navigating indoor environments. The approach is based on processing two consecutive frames (images), computing optical flow, and tracking certain points to detect obstacles. The frame rate of the video stream is determined using a context-aware data fusion technique for the sensors on smartphones. Through an efficient and novel algorithm, a point dataset on each consecutive frames is designed and evaluated to check whether the points belong to an obstacle. In addition to determining the points based on the texture in each frame, our algorithm also considers the heading of user movement to find critical areas on the image plane. We validated the algorithm through experiments by comparing it against two comparable algorithms. The experiments were conducted in different indoor settings and the results based on precision, recall, accuracy, and f-measure were compared and analyzed. The results show that, in comparison to the other two widely used algorithms for this process, our algorithm is more precise. We also considered time-to-contact parameter for clustering the points and presented the improvement of the performance of clustering by using this parameter.     

双足机器人实时障碍检测视觉系统

提出一种双足步行机器人的实时障碍检测视觉系统．基于图像平面与机器人行走地面之间的映射变换矩阵的唯一性准则，判别图像中的像点是否位于地面上，高于或低于地面的点被认为是障碍点，为减少实际行走过程中系统外部和内部参数变化对映射变换矩阵的影响，系统加入了在线校正映射变换矩阵模块．在提取出障碍物体边缘后，通过对其三维信息的简单恢复，建立了机器人行走空间的障碍投影图．该系统计算量小，可靠性强，能基本满足双足步行机器人实时避障的要求。     

Spatio-temporal analysis for obstacle detection in agricultural videos

Autonomous mobile vehicles are becoming commoner in outdoor scenarios for agricultural applications. They must be equipped with a robot navigation system for sensing, mapping, localization, path planning, and obstacle avoidance. In autonomous vehicles, safety becomes a major challenge where unexpected obstacles in the working area must be conveniently addressed. Of particular interest are, people or animals crossing in front of the vehicle or fixed/moving uncatalogued elements in specific positions. Detection of unexpected obstacles or elements on video sequences acquired with a machine vision system on-board a tractor moving in cornfields makes the main contribution to this research. We propose a new strategy for automatic video analysis to detect static/dynamic obstacles in agricultural environments via spatial-temporal analysis. At a first stage obstacles are detected by using spatial information based on spectral colour analysis and texture data. At a second stage temporal information is used to detect moving objects/obstacles at the scene, which is of particular interest in camouflaged elements within the environment. A main feature of our method is that it does not require any training process. Another feature of our approach consists in the spatial analysis to obtain an initial segmentation of interesting objects; afterwards, temporal information is used for discriminating between moving and static objects. To the best of our knowledge in the field of agricultural image analysis, classical approaches make use of either spatial or temporal information, but not both at the same time, making an important contribution. Our method shows favourable results when tested in different outdoor scenarios in agricultural environments, which are really complex, mainly due to the high variability in the illumination conditions, causing undesired effects such as shadows and alternating lighted and dark areas. Dynamic background, camera vibrations and static and dynamic objects are also factors complicating the situation. The results are comparable to those obtained with other state-of-art techniques reported in literature.     

Autonomous quadrotor flight with vision-based obstacle avoidance in virtual environment

In this paper, vision-based autonomous flight with a quadrotor type unmanned aerial vehicle (UAV) is presented. Automatic detection of obstacles and junctions are achieved by the use of optical flow velocities. Variation in the optical flow is used to determine the reference yaw angle. Path to be followed is generated autonomously and the path following process is achieved via a PID controller operating as the low level control scheme. Proposed method is tested in the Google Earth® virtual environment for four different destination points. In each case, autonomous UAV flight is successfully simulated without observing collisions. The results show that the proposed method is a powerful candidate for vision based navigation in an urban environment. Claims are justified with a set of experiments and it is concluded that proper thresholding of the variance of the gradient of optical flow difference have a critical effect on the detectability of roads having different widths.     

Robust crossfeed design for hovering rotorcraft

Control law design for rotorcraft fly-by-wire systems normally attempts to decouple the angular responses using fixed-gain crossfeeds. This approach can lead to poor decoupling over the frequency range of pilot inputs and increase the load on the feedback loops. In order to improve the decoupling performance, dynamic crossfeeds should be adopted. Moreover, because of the large changes that occur in the aircraft dynamics due to small changes about the nominal design condition, especially for near-hovering flight, the crossfeed design must be ‘robust’. A new low-order matching method is presented here to design robust crossfeed compensators for multi-input, multi-output (MIMO) systems. The technique minimizes cross-coupling given an anticipated set of parameter variations for the range of flight conditions of concern. Results are presented in this paper of an analysis of the pitch/roll coupling of the UH-60 Black Hawk helicopter in near-hovering flight. A robust crossfeed is designed that shows significant improvement in decoupling perfomance and robustness over the fixed-gain or single point dynamic compensators. The design method and results are presented in an easily used graphical format that lends significant physical insight to the design procedure. This plant precompensation technique is an appropriate preliminary step to the design of robust feedback control laws for rotorcraft.     

Accurate 3D-vision-based obstacle detection for an autonomous train

In this paper we present a 3D-vision based obstacle detection system for an autonomously operating train in open terrain environments. The system produces dense depth data in real-time from a stereo camera system with a baseline of 1.4 m to fulfill accuracy requirements for reliable obstacle detection 80 m ahead. On an existing high speed stereo engine, several modifications have been applied to significantly improve the overall performance of the system. Hierarchical stereo matching and slanted correlation masks increased the quality of the depth data in a way that the obstacle detection rate increased from 89.4% to 97.75% while the false positive detection rate could be kept as low as 0.25%. The evaluation results have been obtained from extensive real-world test data. An additional stereo matching speed-up of factor 2.15 was achieved and the overall latency of obstacle detection is considerably faster than 300 ms.     

Vision-based approach towards lane line detection and vehicle localization

Localization of the vehicle with respect to road lanes plays a critical role in the advances of making the vehicle fully autonomous. Vision based road lane line detection provides a feasible and low cost solution as the vehicle pose can be derived from the detection. While good progress has been made, the road lane line detection has remained an open one, given challenging road appearances with shadows, varying lighting conditions, worn-out lane lines etc. In this paper, we propose a more robust vision-based approach with respect to these challenges. The approach incorporates four key steps. Lane line pixels are first pooled with a ridge detector. An effective noise filtering mechanism will next remove noise pixels to a large extent. A modified version of sequential RANdom Sample Consensus) is then adopted in a model fitting procedure to ensure each lane line in the image is captured correctly. Finally, if lane lines on both sides of the road exist, a parallelism reinforcement technique is imposed to improve the model accuracy. The results obtained show that the proposed approach is able to detect the lane lines accurately and at a high success rate compared to current approaches. The model derived from the lane line detection is capable of generating precise and consistent vehicle localization information with respect to road lane lines, including road geometry, vehicle position and orientation.     

基于Kinect的实时障碍物检测

传统的传感器在移动机器人障碍物检测领域都有其各自的局限性。文章提出基于Kinect的障碍物检测方法:利用Kinect传感器获取环境深度图像;通过Kinect标定配准之后获取校准参数;通过该参数获得图像像素点与空间三维坐标的对应关系;通过空间三维坐标确定地平面与障碍物区域,并将障碍物区域作为感兴趣区域;通过三维坐标在x轴和z轴的连续性对感兴趣区域进行处理,分割出各个障碍物。实验结果表明,文中算法可以有效且实时地检测到障碍物信息。