Passive stereo range imaging for semi-autonomous land navigation

Many navigation tasks, such as the use of unmanned vehicles for planetary exploration or defense reconnaissance, require onboard range sensors to automatically detect obstacles in the path of a vehicle. Passive ranging via stereo triangulation, or stereo vision, is a very attractive approach to obstacle detection because it is nonemissive, nonmechanical, nonscanning, and compatible with stereographic viewing by human operators. However, several problems have restricted the practicality of stereo ranging in the past, including limitations on the speed, reliability, and generality of existing stereo matching algorithms. This situated has changed, because the Jet Propulsion Laboratory (JPL) recently demonstrated the first semi-autonomous, robotic traverses of natural terrain to use stereo vision for obstacle detection, with all computing onboard the vehicle. This article reviews the main algorithmic paradigms for stereo vision, describes a near realtime stereo vision system developed at JPL, and presents experimental results that demonstrate the emerging practically of stereo vision for obstacle detection in semi-autonomous land navigation. © 1992 John Wiley & Sons, Inc.     

An evaluation of stereo and laser‐based range sensing for rotorcraft unmanned aerial vehicle obstacle avoidance

We present an evaluation of stereo vision and laser‐based range sensing for rotorcraft unmanned aerial vehicle (RUAV) obstacle avoidance. Our focus is on sensors that are suitable for mini‐RUAV class vehicles in terms of weight and power consumption. The study is limited to the avoidance of large static obstacles such as trees. We compare two commercially available devices that are representative of the state of the art in two‐dimensional scanning laser and stereo‐based sensing. Stereo is evaluated with three different focal length lenses to assess the tradeoff between range resolution and field of view (FOV). The devices are evaluated in the context of obstacle avoidance through extensive flight trials with an RUAV. We discuss the merits and limitations of each sensor type, including sensing range, FOV, accuracy, and susceptibility to lighting conditions. We show that the stereo device fitted with 8‐mm lenses has a better sensing range and vertical FOV than the laser device; however, it relies on careful calibration and is affected by high‐contrast outdoor lighting conditions. The laser has a wider horizontal FOV and is more reliable at detecting obstacles that are within a 20‐m range. Overall the laser produced superior obstacle avoidance performance, with a success rate of 84% compared to 42% for 8‐mm stereo. © 2012 Wiley Periodicals, Inc.     

Conflict-free navigation in unknown urban environments

This paper presents an autonomous exploration method in an unknown environment that uses model predictive control (MPC)-based obstacle avoidance with local map building by onboard sensing. An onboard laser scanner is used to build an online map of obstacles around the vehicle with outstanding accuracy. This local map is combined with a real-time MPC algorithm that generates a safe vehicle path, using a cost function that penalizes the proximity to the nearest obstacle. The adjusted trajectory is then sent to a position tracking layer in the hierarchical unmanned aerial vehicle (UAV) avionics architecture. In a series of experiments using a Berkeley UAV, the proposed approach successfully guided the vehicle safely through the urban canyon.     

基于多传感器的无人机避障方法研究及应用

随着无人机巡检作业方式应用越来越广泛,巡检过程中对障碍物检测并进行避障显得愈发关键。若无人机碰到杆塔或线路不仅会造成无人机自身的损坏,还会对居民用电造成影响,给检修带来麻烦。毫米波雷达、激光雷达、双目视觉传感器在机器人避障中有广泛应用。但是基于输电线路巡检的多旋翼无人机的实际情况,传感器器件的选型、尺寸、重量,以及障碍物信息与飞控的融合,显得尤为重要。通过对多旋翼无人机搭载毫米波雷达、双目视觉传感器、差分GPS进行了研究,采用多传感器融合方法检测障碍物,利用虚拟力场法(VFF)进行航迹重规划,并实际飞行验证。测试表明该方法对杆塔避障取得了较好的应用效果。     

室外崎岖地形下基于视差图的无人自主车障碍物识别

对于地面无人车和室外非结构化环境, 本文介绍了我们开发的基于立体视觉的障碍物快速识别系统. 为了使无人地面车适应于较复杂的地形, 根据V视差图, 我们提出了一种新的地面主视差图的估计方法. 通过地面主视差图和局部的三维重建, 本文给出了一种由粗到精的障碍物识别与定位方法. 在我们的无人地面车平台上, 我们对这一障碍物自动识别系统进行了相应的实际试验. 其试验结果验证了该系统的有效性.     

基于滚动速度障碍法的无人机山地航测避障路径规划研究

无人机在进行山地航测时,经常遭遇鸟类等动态障碍,若不能及时规避掉障碍,极容易发生坠机事故。为此,研究一种基于滚动速度障碍法的无人机山地航测避障路径规划方法。基于山地环境模型,结合飞行路径长度、路径平滑度建立一个综合目标函数并利用改进布谷鸟搜索算法求解,得到无人机山地航测的初始路径。对图像进行预处理后,识别无人机初始路径飞行过程中遇到的障碍物,并通过超声波测量无人机与障碍物之间的距离,以此建立速度障碍模型,实现速度障碍碰撞分析,通过滚动窗口的方式确定无人机与障碍物是否存在飞行冲突。基于滚动速度障碍避障方法实现滚动角度避障和速度避障,获取最终的优化路径,完成基于滚动速度障碍法的无人机山地航测避障路径规划。测试结果表明:航测避障路径长度为571.45m,平滑度为165.52,规划的方案更具合理性。     

Autonomous quadrotor flight with vision-based obstacle avoidance in virtual environment

In this paper, vision-based autonomous flight with a quadrotor type unmanned aerial vehicle (UAV) is presented. Automatic detection of obstacles and junctions are achieved by the use of optical flow velocities. Variation in the optical flow is used to determine the reference yaw angle. Path to be followed is generated autonomously and the path following process is achieved via a PID controller operating as the low level control scheme. Proposed method is tested in the Google Earth® virtual environment for four different destination points. In each case, autonomous UAV flight is successfully simulated without observing collisions. The results show that the proposed method is a powerful candidate for vision based navigation in an urban environment. Claims are justified with a set of experiments and it is concluded that proper thresholding of the variance of the gradient of optical flow difference have a critical effect on the detectability of roads having different widths.     

In-flight alignment of a strapdown inertial navigation system of an unmanned aerial vehicle

The alignment system of a strapdown inertial navigation system for an unmanned aerial vehicle installed on a carrier aircraft is studied. The aim is to develop alignment techniques for the strapdown inertial navigation system that improve the accuracy of alignment and reduce the alignment time. A technique for constructing the initial alignment system for a strapdown inertial navigation system of an unmanned aerial vehicle is proposed.     

基于ROS的无人配送车导航系统的设计与实现

针对无人配送车在自主导航过程中存在的寻路效率低、避障能力弱、转折幅度过大等问题,该文采用搭载机器人操作系统(ROS)的Turtlebot3机器人作为无人配送车,设计并实现了高效稳定的无人配送车自主导航系统。ROS是专门用于编写机器人软件的灵活框架,对其集成的SLAM算法进行改进,以完成无人配送车在封闭园区环境中的即时定位与地图构建,同时对ROS导航功能包集成的路径规划算法进行改进,使无人配送车在已知环境地图中规划生成出适合无人配送车工作的路径和有效避开障碍物。最后在Gazebo仿真环境中对无人配送车自主导航系统进行测试与验证。仿真试验结果表明,设计实现的无人配送车导航系统能够很好地满足无人配送车在封闭园区中的自主导航功能。     

A new neural real-time implementation for obstacle detection using linear stereo vision

This paper presents a neural vision system for real-time obstacle detection in front of a moving vehicle using linear stereo vision. The key problem is the correspondence task which consists of matching features in two stereo images that are projections of the same physical entity in the three-dimensional world. The linear stereo correspondence problem is formulated as an optimization task. An energy function, which represents the constraints on the solution, is mapped onto a Hopfield neural network for minimization. The system has been evaluated with experimental results on real stereo images.     

Aerial robotic contact-based inspection: planning and control

The challenge of aerial robotic contact-based inspection is the driving motivation of this paper. The problem is approached on both levels of control and path-planning by introducing algorithms and control laws that ensure optimal inspection through contact and controlled aerial robotic physical interaction. Regarding the flight and physical interaction stabilization, a hybrid model predictive control framework is proposed, based on which a typical quadrotor becomes capable of stable and active interaction, accurate trajectory tracking on environmental surfaces as well as force control. Convex optimization techniques enabled the explicit computation of such a controller which accounts for the dynamics in free-flight as well as during physical interaction, ensures the global stability of the hybrid system and provides optimal responses while respecting the physical limitations of the vehicle. Further augmentation of this scheme, allowed the incorporation of a last-resort obstacle avoidance mechanism at the control level. Relying on such a control law, a contact-based inspection planner was developed which computes the optimal route within a given set of inspection points while avoiding any obstacles or other no-fly zones on the environmental surface. Extensive experimental studies that included complex “aerial-writing” tasks, interaction with non-planar and textured surfaces, execution of multiple inspection operations and obstacle avoidance maneuvers, indicate the efficiency of the proposed methods and the potential capabilities of aerial robotic inspection through contact.     

基于双目视觉的无人船障碍物探测与地图构建

在无人船的巡航过程中，如何探测和躲避障碍物是必须解决的问题。由于无人船自主巡航依赖于其自身对所处环境的精确探测，而目前一般采用的超声波或雷达技术探测距离和精度较低，避障功能弱，因此需引入视觉方法来提升避障精度，并用于轨迹生成、定位或路径规划。本文为无人船搭建双目视觉系统，提出采用双目直接稀疏里程计算法构建无人船所处三维空间模型，将构建的三维点云图转化为二维网格图并标记障碍物，为避障系统进行路径规划提供障碍物数据。实验结果表明，该系统对真实河流环境进行了地图构建，并解决了水上环境构建存在的“虚拟障碍”等问题。     

ARDEA—An MAV with skills for future planetary missions

We introduce a prototype flying platform for planetary exploration: autonomous robot design for extraterrestrial applications (ARDEA). Communication with unmanned missions beyond Earth orbit suffers from time delay, thus a key criterion for robotic exploration is a robot's ability to perform tasks without human intervention. For autonomous operation, all computations should be done on‐board and Global Navigation Satellite System (GNSS) should not be relied on for navigation purposes. Given these objectives ARDEA is equipped with two pairs of wide‐angle stereo cameras and an inertial measurement unit (IMU) for robust visual‐inertial navigation and time‐efficient, omni‐directional 3D mapping. The four cameras cover a $240^{°}$ vertical field of view, enabling the system to operate in confined environments such as caves formed by lava tubes. The captured images are split into several pinhole cameras, which are used for simultaneously running visual odometries. The stereo output is used for simultaneous localization and mapping, 3D map generation and collision‐free motion planning. To operate the vehicle efficiently for a variety of missions, ARDEA's capabilities have been modularized into skills which can be assembled to fulfill a mission's objectives. These skills are defined generically so that they are independent of the robot configuration, making the approach suitable for different heterogeneous robotic teams. The diverse skill set also makes the micro aerial vehicle (MAV) useful for any task where autonomous exploration is needed. For example terrestrial search and rescue missions where visual navigation in GNSS‐denied indoor environments is crucial, such as partially collapsed man‐made structures like buildings or tunnels. We have demonstrated the robustness of our system in indoor and outdoor field tests.     

Two-frame structure from motion using optical flow probability distributions for unmanned air vehicle obstacle avoidance

See-and-avoid behaviors are an essential part of autonomous navigation for Unmanned Air Vehicles (UAVs). To be fully autonomous, a UAV must be able to navigate complex urban and near-earth environments and detect and avoid imminent collisions. While there have been significant research efforts in robotic navigation and obstacle avoidance during the past few years, this previous work has not focused on applications that use small autonomous UAVs. Specific UAV requirements such as non-invasive sensing, light payload, low image quality, high processing speed, long range detection, and low power consumption, etc., must be met in order to fully use this new technology. This paper presents single camera collision detection and avoidance algorithm. Whereas most algorithms attempt to extract the 3D information from a single optical flow value at each feature point, we propose to calculate a set of likely optical flow values and their associated probabilities—an optical flow probability distribution. Using this probability distribution, a more robust method for calculating object distance is developed. This method is developed for use on a UAV to detect obstacles, but it can be used on any vehicle where obstacle detection is needed.     

Fuzzy image processing scheme for autonomous navigation of human blind

The main objective of this work is to develop an electronic travel aid to assist the blinds for obstacle identification in their navigation. This navigation assistance for visually impaired (NAVI) system presented in this paper consists of a single board processing system (SBPS), a vision sensor mounted headgear and a pair of stereo earphones. The image environment in front of the blind is captured by the vision sensor. The image is processed by a new real time image processing scheme using fuzzy clustering algorithms. The processed image is mapped onto a specially structured stereo acoustic patterns and transferred to the stereo earphones in the system. Blind individuals were trained with NAVI system and tested for obstacle identification. Suggestions from the blind volunteers regarding pleasantness and discrimination of sound pattern were also incorporated in the prototype. The proposed processing methodology is found to be effective for object identification and for producing stereo sound patterns in the NAVI system.     

架空输电线路巡线机器人的视觉导航

巡线机器人沿相线行走时必须探测识别和定位各种障碍物,并根据障碍类型规划越障行为。针对220 kV架空输电线路的结构特点,利用视觉传感器,设计了基于结构约束的障碍识别算法,完成了障碍识别和分类。根据障碍物的结构特点,设计了一种自适应多窗口区域立体匹配算法,实现了障碍物的双目视觉定位。模拟线路实验结果表明,算法能可靠地从复杂背景中识别并定位出防振锤、悬垂线夹和耐张线夹等障碍物,满足了巡线机器人导航要求。     

无人机软式自主空中加油视觉导航方法

针对无人机软式自主空中加油问题,提出了用于无人机软式自主空中加油的视觉导航方法.在加油锥套外环平面上均匀布置8个近红外LED,在受油机上布置CCD相机及带通滤镜构成视觉系统,利用该系统对锥套进行近红外成像,对所获取的近红外图像进行特征点提取并利用迭代算法估计锥套与受油机之间的位姿参数.在不同距离下对该视觉导航方法进行了仿真验证.结果表明,所研究的无人机软式自主空中加油视觉导航方法精度较高,抗干扰能力较强,可以有效地用于无人机软式自主空中加油.     

基于彩色立体视觉的障碍物快速检测方法

Real-time obstacle detection method is a key technique for machine vision based mobile robot and au-tonomous land vehicle navigation in unstructured environments. In this paper o considering the real-time requirement for stereo matching algorithm, an adaptive color segmentation method for possible obstacle region detection is first developed based on the color feature, and a simple region based stereo matching algorithm of binocular vision for realobstacle recognition is also introduced. Obstacle detection is implemented by combining the road color adaptive seg-mentation method and region based stereovision method. Lots of experiment results show that the proposed approachcan detect obstacle quickly and effectively, and this algorithm is particularly suited for road environments in which the road is relatively flat and of roughly the same color.