基于立体视觉的3D地形拼接

为了从月球车上的全景相机所捕获的立体图像对重建月球车周围的3D地形,给地面科学家制定科学探测指令提供一个直观的可视化平台,预先展开了基于立体视觉的3D地形重建的研究,主要介绍了在3D地形拼接方面的研究进展。当恢复出多个局部3D地形模型时,首先基于边缘检测和图像匹配技术提取出相邻局部模型之间的公共数据点,然后采用分离旋转变换和平移矢量的策略拟合出相邻模型之间的坐标转换关系,之后就可以将局部模型统一在同一个坐标系下。通过室内和室外多次实验验证了该拼接方案。     

Deriving terrain and textural information from stereo RADARSAT data for mountainous land cover mapping

A new procedure is proposed for land cover classification in a mountainous area using stereo RADARSAT-1 data. The method integrates a few types of information that can be extracted from the same stereo RADARSAT images: (1) the Digital Elevation Model (DEM) generated from the stereo RADARSAT images; (2) terrain information (elevation, slope and aspect) extracted from the derived DEM; and (3) textural information derived from the same RADARSAT images. An Artificial Neural Network (ANN) classifier is applied for the land cover classification. Performance of the proposed method is evaluated using a mountainous study area in Southern Argentina, where there is a lack of up-to-date information for environmental monitoring. The results show that the integration of textural and terrain information can greatly improve the accuracy of the classification using the ANN classifier. It demonstrates that stereo RADARSAT images provide valuable data sources for land cover mapping, especially in mountainous areas where cloud cover is a problem for optical data collection and topographical data are not always available.     

Hopfield network for stereo vision correspondence

An optimization approach is used to solve the correspondence problem for a set of features extracted from a pair of stereo images. A cost function is defined to represent the constraints on the solution, which is then mapped onto a two-dimensional Hopfield neural network for minimization. Each neuron in the network represents a possible match between a feature in the left image and one in the right image. Correspondence is achieved by initializing (exciting) each neuron that represents a possible match and then allowing the network to settle down into a stable state. The network uses the initial inputs and the compatibility measures between the matched points to find a stable state.     

Simulation study of a constant time hybrid approach for large scale terrain mapping using satellite stereo imagery

This work addresses the problem of performing large scale SLAM (Simultaneous Localization And Mapping) with satellite stereo imagery for terrain mapping, using a constant time estimation approach. The approach adopts the relative bundle adjustment approach (RBA) and integrates with it a particle-based framework to obtain a constant time probabilistic pose estimation model. The approach further uses a concept of fuzzy landmark-based similarity between poses to make common landmark identification across poses easier, especially when landmarks are sparsely encountered. In order to achieve robustness under varying environmental conditions, we use Speeded Up Robust Features (SURF) for computing spatial and temporal landmark correspondences across time steps. Finally, we use a fast loop closure approach to reduce drifts and obtain global pose estimates. For simulation study, the robot images are cropped from stereo-pair satellite images at different time steps incorporating errors in the robot’s control information. Extensive experimentation has been carried out to study the robot trajectories and the determination of Digital Elevation Model (DEM), with encouraging findings. We have also compared our work with 6D FastSLAM 2.0 (Thrun et al. (2005)) as well as Relative SLAM (RSLAM) due to Mei et al. (2010).     

Accurate hardware-based stereo vision

To enable both accurate and fast real-time stereo vision in embedded systems, we propose a novel stereo matching algorithm that is designed for high efficiency when realized in hardware. We evaluate its accuracy using the Middlebury Stereo Evaluation, revealing its high performance at minimum tolerance. To outline the resource efficiency of the algorithm, we present its realization as an Intellectual Property (IP) core that is designed for the deployment in Field Programmable Gate Arrays (FPGAs) and Application Specific Integrated Circuits (ASICs).     

基于立体视觉的在线实时测量系统设计与实现

为了满足深空探测器实时测量天体表面形貌的需求,设计并实现了一套基于立体视觉的在线实时测量原型系统。该系统通过立体相机实时获取空间天体的立体影像,利用每次观测的一组立体影像来重建其局部表面形状;再对每次重建的局部模型进行连接,得到空间天体完整的表面形貌模型。通过仿真实验验证了该系统的可行性,数据处理的速度与精度可以满足对深空目标进行实时测量的需要。     

Fog effect for photography using stereo vision

Fog is an important factor in photography with a special aesthetic, emotional, or compositional meaning. We present a fog-simulation method for photo editing using binocular stereo vision. Given a stereo pair, we estimate the depth information by stereo matching followed by a process to refine depth results for the given photo editing purpose. Then, depth-aware fog effects can be applied on the base image, with optional interaction for control purposes. Besides homogeneous fog, we provide three tools to control the density of the fog media. Thus, various kinds of heterogeneous atmospheric effects can also been simulated. Experiments show that the proposed method can achieve more natural-looking results than manually drawn fog, our results are very close to the appearance of fog in the real world.     

机器人手眼立体视觉标定研究

以固高GRB-400型机器人和CCD组成机器人手眼系统,分析了摄像机的成像模型,采用了基于直接将图像坐标映射到机器人参考坐标的“黑箱”思想,从图像坐标直接计算出目标位置的方法,用于立体定位的摄像机手眼标定,该方法通过保持机器人末端执行器到机器人参考坐标系旋转矩阵来简化复杂的相机标定过程,最后通过实验验证了该方法的可行性,并分析了实验误差产生的原因,并提出了相应的解决措施。     

Onboard plane-wise 3D mapping using super-pixels and stereo vision for autonomous flight of a hexacopter

Intelligent Service Robotics - Micro Aerial Vehicles (MAVs) have become very popular with a wide range of applications in cinema, surveillance, search and rescue, among many others. For these...     

Visual terrain mapping for Mars exploration

One goal for future Mars missions is for a rover to be able to navigate autonomously to science targets not visible to the rover, but seen in orbital or descent images. This can be accomplished if accurate maps of the terrain are available for the rover to use in planning and localization. We describe techniques to generate such terrain maps using images with a variety of resolutions and scales, including surface images from the lander and rover, descent images captured by the lander as it approaches the planetary surface, and orbital images from current and future Mars orbiters. At the highest resolution, we process surface images captured by rovers and landers using bundle adjustment. At the next lower resolution (and larger scale), we use wide-baseline stereo vision to map terrain distant from a rover with surface images. Mapping the lander descent images using a structure-from-motion algorithm generates data at a hierarchy of resolutions. These provide a link between the high-resolution surface images and the low-resolution orbital images. Orbital images are mapped using similar techniques, although with the added complication that the images may be captured with a variety of sensors. Robust multi-modal matching techniques are applied to these images. The terrain maps are combined using a system for unifying multi-resolution models and integrating three-dimensional terrains. The result is a multi-resolution map that can be used to generate fixed-resolution maps at any desired scale.     

Active/dynamic stereo vision

Visual navigation is a challenging issue in automated robot control. In many robot applications, like object manipulation in hazardous environments or autonomous locomotion, it is necessary to automatically detect and avoid obstacles while planning a safe trajectory. In this context the detection of corridors of free space along the robot trajectory is a very important capability which requires nontrivial visual processing. In most cases it is possible to take advantage of the active control of the cameras. In this paper we propose a cooperative schema in which motion and stereo vision are used to infer scene structure and determine free space areas. Binocular disparity, computed on several stereo images over time, is combined with optical flow from the same sequence to obtain a relative-depth map of the scene. Both the time to impact and depth scaled by the distance of the camera from the fixation point in space are considered as good, relative measurements which are based on the viewer, but centered on the environment. The need for calibrated parameters is considerably reduced by using an active control strategy. The cameras track a point in space independently of the robot motion and the full rotation of the head, which includes the unknown robot motion, is derived from binocular image data. The feasibility of the approach in real robotic applications is demonstrated by several experiments performed on real image data acquired from an autonomous vehicle and a prototype camera head     

GLSV: Graphics library stereo vision for OpenGL

This work proposes the development of an auxiliary library for use with OpenGL, to facilitate the creation of graphic applications incorporating stereoscopic representation. This library, christened graphics library stereo vision (GLSV), is designed to remove all calculations involving knowledge of stereo vision theory from the task performed by the programmer without the latter having to change the way he/she has been working with the OpenGL library. The GLSV is distributed under the terms of the GNU Library General Public License agreement.     

双目视觉测量传感器研究

研究了按照激光三角测量原理由 1个线激光器及 2个CCD摄像机组成的双目视觉测量传感器。探讨了 2个CCD摄像机的摆放位置及其对测量精度的影响,确定了其几何参数。在此基础上,论述了其工作过程,包括其标定方法、标定过程、扫描图像采集、特征提取、特征匹配及三维数据生成。该双目视觉测量传感器配合扫描机构(如三坐标测量机 )即可对物体进行非接触式三维激光扫描测量,其测量速度快,精度比较高,可以应用于逆向工程领域。     

ASTER ratio indices for supraglacial terrain mapping

Supraglacial terrain, such as that found in the Himalayas, is typically composed of snow, ice, ice‐mixed‐debris (IMD) and debris. This letter presents a methodology for systematic discrimination and mapping of these supraglacial cover types using Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) data. The Normalized Difference Snow Index (NDSI) has been used previously for discrimination of snow/ice‐bearing zones versus debris. Two new indices, the Normalized Difference Glacier Index (NDGI) and the Normalized Difference Snow Ice Index (NDSII), are presented. The combination of all three indices allows discrimination of snow, ice and IMD in a systematic manner.     

双目立体图像的快速匹配值计算方法

立体视觉计算过程中的匹配值计算极其耗时,整幅图的匹配值由各个参考像点的匹配值计算构成,因此,如果能够将前后两次参考像点的匹配值计算中重复的部分提取出来,那么就可以节约部分计算时间。将前后两次匹配值计算所涉及的领域像素分布情况分析清楚,结合所选取的计算步骤,将各个子步骤的中间输出结果缓存起来,以便下一次计算时直接使用而不需重新计算。如果匹配所用的邻域子图与整幅图相比非常小,则这种类似于流水作业的时间重叠计算方式可以获得很好的加速性能,同时该加速性能也会随着邻域尺寸的加大而提高,这得益于所省略的中间计算步骤较多的原因。     

A GPU persistent grid mapping for terrain rendering

In this paper we present persistent grid mapping (PGM), a novel framework for interactive view-dependent terrain rendering. Our algorithm is geared toward high utilization of modern GPUs, and takes advantage of ray tracing and mesh rendering. The algorithm maintains multiple levels of the elevation and color maps to achieve a faithful sampling of the viewed region. The rendered mesh ensures the absence of cracks and degenerate triangles that may cause the appearance of visual artifacts. In addition, an external texture memory support is provided to enable the rendering of terrains that exceed the size of texture memory. Our experimental results show that the PGM algorithm provides high quality images at steady frame rates. Electronic supplementary material Supplementary material is available in the online version of this article at and is accessible for authorized users.     

完全在线的双目直接法视觉SLAM算法

视觉SLAM在机器人的室外作业如野外探索、定位侦察中扮演了重要角色.为了使得机器人可以更好地进行室外作业,提出一种不受词袋模型的固定词汇限制的完全在线实时双目直接法视觉SLAM算法.作为直接法视觉SLAM,所提到的系统可以利用任何具有足够强度梯度的图像像素,使其在缺少特征点的区域仍具有很强的鲁棒性.在系统算法中引入双目静态残差约束并去除遮挡的滑窗优化来增强系统的跟踪精度,增加闭环检测和位姿图优化模块,并建立在线词袋模型,使得系统在大规模且陌生的环境中依然可以进行工作.将此算法在公开的EuRoC数据集和KITTI数据集上进行性能评估,结果表明,所提出的系统的定位精度优于最先进的直接法视觉SLAM系统,且室内场景和室外场景均具有鲁棒性.     

A portable stereo vision system for whole body surface imaging

This paper presents a whole body surface imaging system based on stereo vision technology. We have adopted a compact and economical configuration which involves only four stereo units to image the frontal and rear sides of the body. The success of the system depends on a stereo matching process that can effectively segment the body from the background in addition to recovering sufficient geometric details. For this purpose, we have developed a novel sub-pixel, dense stereo matching algorithm which includes two major phases. In the first phase, the foreground is accurately segmented with the help of a predefined virtual interface in the disparity space image, and a coarse disparity map is generated with block matching. In the second phase, local least squares matching is performed in combination with global optimization within a regularization framework, so as to ensure both accuracy and reliability. Our experimental results show that the system can realistically capture smooth and complete whole body shapes with high accuracy.     

Interactive stereo vision telepresence for correct communication of spatial geometry

This paper presents a novel interactive stereo vision telepresence system for use in broadband Internet-2 communication environments. An algorithm for virtual distance correction is deployed in this system which allows us to present a correct nominal depth impression of a remote environment to a local human operator. The correctness properties around the nominal depth are analyzed and an error function is derived. The efficacy of the distance correction measures is tested by evaluation experiments.