BETWEEN LOCATION AWARENESS AND AWARE LOCATIONS: WHERE TO PUT THE INTELLIGENCE

This article presents a review on vision-aided systems and proposes an approach for visual rehabilitation using stereo vision technology. The proposed system utilizes stereo vision, image processing methodology, and a sonification procedure to support blind mobilization. The developed system includes wearable computer, stereo cameras as vision sensor, and stereo earphones, all molded in a helmet. The image of the scene in front of the visually handicapped is captured by the vision sensors. The captured images are processed to enhance the important features in the scene in front for mobilization assistance. The image processing is designed as a model of human vision by identifying the obstacles and their depth information. The processed image is mapped onto musical stereo sound for the blind's understanding of the scene in front. The developed method has been tested in the indoor and outdoor environments and the proposed image processing methodology is found to be effective for object identification.     

基于仿生双目的无人旋翼机自主着陆方法

基于仿生双目理论提出一种仿双眼异向运动的变视轴夹角双目视觉系统,通过对实时采集的目标图像的处理,研究其双目摄像机夹角的在线自标定,实现获取无人旋翼机的实时高度信息。实验结果表明,该方法获得的光轴夹角信息相对误差小于5%,能准确控制双目摄像机的转动,避免由于双目视觉盲区所造成的着陆误差,系统所标定出的旋翼机高度信息也为其安全着陆提供了有力保障。     

Passive stereo range imaging for semi-autonomous land navigation

Many navigation tasks, such as the use of unmanned vehicles for planetary exploration or defense reconnaissance, require onboard range sensors to automatically detect obstacles in the path of a vehicle. Passive ranging via stereo triangulation, or stereo vision, is a very attractive approach to obstacle detection because it is nonemissive, nonmechanical, nonscanning, and compatible with stereographic viewing by human operators. However, several problems have restricted the practicality of stereo ranging in the past, including limitations on the speed, reliability, and generality of existing stereo matching algorithms. This situated has changed, because the Jet Propulsion Laboratory (JPL) recently demonstrated the first semi-autonomous, robotic traverses of natural terrain to use stereo vision for obstacle detection, with all computing onboard the vehicle. This article reviews the main algorithmic paradigms for stereo vision, describes a near realtime stereo vision system developed at JPL, and presents experimental results that demonstrate the emerging practically of stereo vision for obstacle detection in semi-autonomous land navigation. © 1992 John Wiley & Sons, Inc.     

基于激光雷达多源数据融合的路障检测系统设计

车辆行驶模式的改变会导致道路障碍观测盲区的出现,如何在满足车辆运行安全的条件下,对道路障碍进行快速检测已经成为了一项亟待解决的问题;基于此引入激光雷达多源数据融合理论,设计路障检测系统;利用多源以太网接口电路,提供激光图像传感器、路障特征提取分类器、TVPS150PBS图像解码器所需的应用电子量,再借助可编程逻辑器件,对待融合多源数据进行整合,完成路障检测系统硬件设计;规划雷达工作模式的识别环境,利用多源异构传感器设备,对检测角点特征进行提取处理,整合上述所有数据信息,完成雷达脉冲组序列的建模处理,实现基于激光雷达多源数据融合路障检测系统的应用;实验结果表明,激光雷达多源数据融合原理支持下路障检测系统的PF指标最大数值与最小数值的差值为60 MHz;PR指标的差值为4800 MHz,PW指标的差值为700 MHz;以上指标表明所设计系统的观测盲区更小,可在满足车辆运行安全的条件下,实现对道路障碍的快速检测.     

Stereo vision for planetary rovers: Stochastic modeling to near real-time implementation

NASA scenarios for lunar and planetary missions include robotic vehicles that function in both teleoperated and semi-autonomous modes. Under teleoperation, on-board stereo cameras may provide 3-D scene information to human operators via stereographic displays; likewise, under semi-autonomy, machine stereo vision may provide 3-D information for obstacle avoidance. In the past, the slow speed of machine stereo vision systems has posed a hurdle to the semi-autonomous scenario; however, recent work at JPL and other laboratories has produced stereo systems with high reliability and near real-time performance for low-resolution image pairs. In particular, JPL has taken a significant step by achieving the first autonomous, cross-country robotic traverses (of up to 100 meters) to use stereo vision, with all computing on-board the vehicle. Here, we describe the stereo vision system, including the underlying statistical model and the details of the implementation. The statistical and algorithmic aspects employ random field models of the disparity map, Bayesian formulations of single-scale matching, and area-based image comparisons. The implementation builds bandpass image pyramids and produces disparity maps from the 60×64 level of the pyramids at rates of up to two seconds per image pair. All vision processing is done in one 68020 augmented with Datacube image processing boards. We argue that the overall approach provides a unifying paradigm for practical, domain-independent stereo ranging. We close with a discussion of practical and theoretical issues involved in evaluating and extending the performance of the stereo system.     

Robot Localization in Indoor Environments Using Radio Frequency Identification Technology and Stereo Vision

In this paper, a novel method of obstacle recognition and localization for mobile robots using radio frequency identification (RFID) technology and stereo vision is proposed. It is inexpensive, flexible and easy to use in practical environments. As information about the obstacles or environment can be written in ID tags, the proposed method can detect the obstacles easily and quickly compared with other methods. RF is not so stable, so the Bayes rule was introduced to calculate the probability where the ID tag exists in order to improve the accuracy of localizing obstacles with the ID tags. Then the stereo camera starts to process the image of the region of interest where the obstacle exists. The proposed method does not need to process all the image and easily gets some information about obstacles such as size, and color, and thus decreases the processing computation. Research on RFID technology integrating stereo vision to localize an indoor mobile robot has also been performed. This paper introduces the architecture of the proposed method and some experimental results.     

基于视差平面分割的移动机器人障碍物地图构建方法

作为自主移动机器人地表障碍物探测（GPOD）技术的一部分,提出了一种利用双目摄像机的视差图像 获取信息来构建机器人前方障碍物栅格地图的方法. 该方法融合了3 维立体视觉技术以及2 维图像处理技术,前者 依据视差图的直方图信息对视差图像进行自适应平面分割,把每个平面看作是3 维场景中的实物切片进而提取障碍 物3 维信息,后者通过计算各平面上的障碍物信息曲线来提取障碍物信息,把立体视觉数据从视差图像空间变换到 2 维的障碍物地图空间. 给出了该方法构建障碍物地图的整体过程,试验结果证明了该算法的有效性和精确性.     

Algorithms for path planning, navigation and guidance of an AGV

Algorithms for path navigation and generation of guidance setpoints for an AGV are developed. Navigation is based on a simple flat world model of connected nodes using a suboptimal path solution for execution speed. The guidance algorithms use vision data from a stereo pair of linear image array cameras, which described an obstacle location and height where intersected by the vision system plane of view. The development of a map of the area on the AGV path allows the detection of obstacles, which may be passed subject to the aisle markings. The complete vision-guidance system can be implemented using inexpensive commercially available 16 bit microprocessors.     

Stereo perception and dead reckoning for a prototype lunar rover

This paper describes practical, effective approaches to stereo perception and dead reckoning, and presents results from systems implemented for a prototype lunar rover operating in natural, outdoor environments.The stereo perception hardware includes a binocular head mounted on a motion-averaging mast. This head provides images to a normalized correlation matcher, that intelligently selects what part of the image to process (saving time), and subsamples the images (again saving time) without subsampling disparities (which would reduce accuracy). The implementation has operated successfully during long-duration field exercises, processing streams of thousands of images.The dead reckoning approach employs encoders, inclinometers, a compass, and a turn-rate sensor to maintain the position and orientation of the rover as it traverses. The approach integrates classical odometry with inertial guidance. The implementation succeeds in the face of significant sensor noise by virtue of sensor modelling, plus extensive filtering.The stereo and dead reckoning components are used by an obstacle avoidance planner that projects a finite number of arcs through the terrain map, and evaluates the traversability of each arc to choose a travel direction that is safe and effective. With these components integrated into a complete navigation system, a prototype rover has traversed over 1 km in lunar-like environments.     

基于主动式传感器的道路导航

由于林荫道中阴影的影响,目前常用的基于被动式传感器技术的视觉导航方法(如采用图像的方法),难以取得比较理想的结果。对于这种环境,主动式传感器具有一定的优势,文中采用了激光雷达传感器。首先利用传感器对路边的树木等物体作为障碍物来进行检测,并据此信息来推算道路边界,详细介绍了具体的方案和算法。实验结果表明,所设计的方法具有很好的鲁棒性,可以满足实际导航的需要。     

Accuracy of stereoscopic navigation system in flights above natural landscapes and urban land

Accuracy of stereoscopic navigation system designated for solution of problems of high accuracy navigation, guidance, and remote landscape probing is estimated. Linearization of photometric equations of the stereoscopic navigation system in the regime of stabilized horizontal flight of an aircraft is performed. Observation equations of the stereoscopic navigation system with account of the only stereo pair and a set of stereo images are determined. New algorithms of image processing in the stereoscopic navigation system based on the following two proved results: direct and inverse photogrammetry equations and the theory of joint optimal filtering, identification, and verification of hypotheses in a discrete dynamic system, are used to obtain formulas characterizing the accuracy of the stereoscopic navigation system in flights above natural landscapes and urban land. Procedure of choosing correction segments is considered.     

3D visual perception for self-driving cars using a multi-camera system: Calibration,mapping, localization,and obstacle detection

Cameras are a crucial exteroceptive sensor for self-driving cars as they are low-cost and small, provide appearance information about the environment, and work in various weather conditions. They can be used for multiple purposes such as visual navigation and obstacle detection. We can use a surround multi-camera system to cover the full 360-degree field-of-view around the car. In this way, we avoid blind spots which can otherwise lead to accidents. To minimize the number of cameras needed for surround perception, we utilize fisheye cameras. Consequently, standard vision pipelines for 3D mapping, visual localization, obstacle detection, etc. need to be adapted to take full advantage of the availability of multiple cameras rather than treat each camera individually. In addition, processing of fisheye images has to be supported. In this paper, we describe the camera calibration and subsequent processing pipeline for multi-fisheye-camera systems developed as part of the V-Charge project. This project seeks to enable automated valet parking for self-driving cars. Our pipeline is able to precisely calibrate multi-camera systems, build sparse 3D maps for visual navigation, visually localize the car with respect to these maps, generate accurate dense maps, as well as detect obstacles based on real-time depth map extraction.     

防伪指纹计算机视觉传感系统的研究

提出了一种防伪指纹计算机视觉传感系统。通过对伪指纹的分析,设计了系统的结构,并介绍了该系统的工作流程、防伪指纹算法、指纹图像处理算法。实验结果表明,该系统拥有较快的伪指纹辨别速度和100％的准确率,并能快速有效地对指纹图像进行处理。     

Vision and inertial sensor cooperation using gravity as a vertical reference

This paper explores the combination of inertial sensor data with vision. Visual and inertial sensing are two sensory modalities that can be explored to give robust solutions on image segmentation and recovery of 3D structure from images, increasing the capabilities of autonomous robots and enlarging the application potential of vision systems. In biological systems, the information provided by the vestibular system is fused at a very early processing stage with vision, playing a key role on the execution of visual movements such as gaze holding and tracking, and the visual cues aid the spatial orientation and body equilibrium. In this paper, we set a framework for using inertial sensor data in vision systems, and describe some results obtained. The unit sphere projection camera model is used, providing a simple model for inertial data integration. Using the vertical reference provided by the inertial sensors, the image horizon line can be determined. Using just one vanishing point and the vertical, we can recover the camera's focal distance and provide an external bearing for the system's navigation frame of reference. Knowing the geometry of a stereo rig and its pose from the inertial sensors, the collineations of level planes can be recovered, providing enough restrictions to segment and reconstruct vertical features and leveled planar patches.     

基于立体视觉的移动机器人导航算法

移动机器人立体视觉系统不仅提供三维地形图用于障碍规避和路径规划,其结果还可以用于视觉导航。以移动机器人立体视觉系统为基础,研究了基于前后两个位置上立体图对的视觉测量算法用于移动机器人的连续导航,讨论了影响导航精度的因素和改进方法;研究了基于局部和全局三维地形图的地形匹配算法用于定期校正位置误差,算法实现简便,定位精度取决于地形图精度。实验结果证明了两种方法的有效性,可以兼顾近距离和中远距离导航任务。     

基于彩色立体视觉的障碍物快速检测方法

Real-time obstacle detection method is a key technique for machine vision based mobile robot and au-tonomous land vehicle navigation in unstructured environments. In this paper o considering the real-time requirement for stereo matching algorithm, an adaptive color segmentation method for possible obstacle region detection is first developed based on the color feature, and a simple region based stereo matching algorithm of binocular vision for realobstacle recognition is also introduced. Obstacle detection is implemented by combining the road color adaptive seg-mentation method and region based stereovision method. Lots of experiment results show that the proposed approachcan detect obstacle quickly and effectively, and this algorithm is particularly suited for road environments in which the road is relatively flat and of roughly the same color.     

Some accuracy and resolution aspects of computer vision distance measurements

The distance between an object and stereo vision sensors can be measured using image processing and known system parameters. A detailed distance measurement synthesis procedure to meet system specifications is presented and illustrated with an example. An error analysis shows that error is proportional to distance. System parameters such as separation between sensor elements, sensor focal length, and sensor array dimensions are related in the design and error equations presented. The main desired design goal is to establish the smallest image sensor array size which will meet system operating specifications. Minimum and maximum distance, object height, optic parameters, scene shift, and sensor array parameters are related.     

健康服务机器人室内导航视觉传感器研究与开发*

针对健康服务机器人室内导航的应用需求,设计出一种基于STM32F429的嵌入式导航视觉传感器,内嵌500万像素CMOS图像传感器和拥有高性能无线SOC的ESP8266,此次开发的视觉传感器,不是一般的CMOS图像传感器,而是综合了图像感知,图像处理,无线通信,节电唤醒等等功能于一体的智能化微型装置,即眼睛加大脑。经测试,本导航视觉传感器能够以每秒3次的速度向服务机器人发送位置,完全满足室内移动机器人的移动需求。