Deriving terrain and textural information from stereo RADARSAT data for mountainous land cover mapping

A new procedure is proposed for land cover classification in a mountainous area using stereo RADARSAT-1 data. The method integrates a few types of information that can be extracted from the same stereo RADARSAT images: (1) the Digital Elevation Model (DEM) generated from the stereo RADARSAT images; (2) terrain information (elevation, slope and aspect) extracted from the derived DEM; and (3) textural information derived from the same RADARSAT images. An Artificial Neural Network (ANN) classifier is applied for the land cover classification. Performance of the proposed method is evaluated using a mountainous study area in Southern Argentina, where there is a lack of up-to-date information for environmental monitoring. The results show that the integration of textural and terrain information can greatly improve the accuracy of the classification using the ANN classifier. It demonstrates that stereo RADARSAT images provide valuable data sources for land cover mapping, especially in mountainous areas where cloud cover is a problem for optical data collection and topographical data are not always available.     

Hopfield network for stereo vision correspondence

An optimization approach is used to solve the correspondence problem for a set of features extracted from a pair of stereo images. A cost function is defined to represent the constraints on the solution, which is then mapped onto a two-dimensional Hopfield neural network for minimization. Each neuron in the network represents a possible match between a feature in the left image and one in the right image. Correspondence is achieved by initializing (exciting) each neuron that represents a possible match and then allowing the network to settle down into a stable state. The network uses the initial inputs and the compatibility measures between the matched points to find a stable state.     

Accurate hardware-based stereo vision

To enable both accurate and fast real-time stereo vision in embedded systems, we propose a novel stereo matching algorithm that is designed for high efficiency when realized in hardware. We evaluate its accuracy using the Middlebury Stereo Evaluation, revealing its high performance at minimum tolerance. To outline the resource efficiency of the algorithm, we present its realization as an Intellectual Property (IP) core that is designed for the deployment in Field Programmable Gate Arrays (FPGAs) and Application Specific Integrated Circuits (ASICs).     

Fog effect for photography using stereo vision

Fog is an important factor in photography with a special aesthetic, emotional, or compositional meaning. We present a fog-simulation method for photo editing using binocular stereo vision. Given a stereo pair, we estimate the depth information by stereo matching followed by a process to refine depth results for the given photo editing purpose. Then, depth-aware fog effects can be applied on the base image, with optional interaction for control purposes. Besides homogeneous fog, we provide three tools to control the density of the fog media. Thus, various kinds of heterogeneous atmospheric effects can also been simulated. Experiments show that the proposed method can achieve more natural-looking results than manually drawn fog, our results are very close to the appearance of fog in the real world.     

Onboard plane-wise 3D mapping using super-pixels and stereo vision for autonomous flight of a hexacopter

Intelligent Service Robotics - Micro Aerial Vehicles (MAVs) have become very popular with a wide range of applications in cinema, surveillance, search and rescue, among many others. For these...     

Visual terrain mapping for Mars exploration

One goal for future Mars missions is for a rover to be able to navigate autonomously to science targets not visible to the rover, but seen in orbital or descent images. This can be accomplished if accurate maps of the terrain are available for the rover to use in planning and localization. We describe techniques to generate such terrain maps using images with a variety of resolutions and scales, including surface images from the lander and rover, descent images captured by the lander as it approaches the planetary surface, and orbital images from current and future Mars orbiters. At the highest resolution, we process surface images captured by rovers and landers using bundle adjustment. At the next lower resolution (and larger scale), we use wide-baseline stereo vision to map terrain distant from a rover with surface images. Mapping the lander descent images using a structure-from-motion algorithm generates data at a hierarchy of resolutions. These provide a link between the high-resolution surface images and the low-resolution orbital images. Orbital images are mapped using similar techniques, although with the added complication that the images may be captured with a variety of sensors. Robust multi-modal matching techniques are applied to these images. The terrain maps are combined using a system for unifying multi-resolution models and integrating three-dimensional terrains. The result is a multi-resolution map that can be used to generate fixed-resolution maps at any desired scale.     

Active/dynamic stereo vision

Visual navigation is a challenging issue in automated robot control. In many robot applications, like object manipulation in hazardous environments or autonomous locomotion, it is necessary to automatically detect and avoid obstacles while planning a safe trajectory. In this context the detection of corridors of free space along the robot trajectory is a very important capability which requires nontrivial visual processing. In most cases it is possible to take advantage of the active control of the cameras. In this paper we propose a cooperative schema in which motion and stereo vision are used to infer scene structure and determine free space areas. Binocular disparity, computed on several stereo images over time, is combined with optical flow from the same sequence to obtain a relative-depth map of the scene. Both the time to impact and depth scaled by the distance of the camera from the fixation point in space are considered as good, relative measurements which are based on the viewer, but centered on the environment. The need for calibrated parameters is considerably reduced by using an active control strategy. The cameras track a point in space independently of the robot motion and the full rotation of the head, which includes the unknown robot motion, is derived from binocular image data. The feasibility of the approach in real robotic applications is demonstrated by several experiments performed on real image data acquired from an autonomous vehicle and a prototype camera head     

GLSV: Graphics library stereo vision for OpenGL

This work proposes the development of an auxiliary library for use with OpenGL, to facilitate the creation of graphic applications incorporating stereoscopic representation. This library, christened graphics library stereo vision (GLSV), is designed to remove all calculations involving knowledge of stereo vision theory from the task performed by the programmer without the latter having to change the way he/she has been working with the OpenGL library. The GLSV is distributed under the terms of the GNU Library General Public License agreement.     

ASTER ratio indices for supraglacial terrain mapping

Supraglacial terrain, such as that found in the Himalayas, is typically composed of snow, ice, ice‐mixed‐debris (IMD) and debris. This letter presents a methodology for systematic discrimination and mapping of these supraglacial cover types using Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) data. The Normalized Difference Snow Index (NDSI) has been used previously for discrimination of snow/ice‐bearing zones versus debris. Two new indices, the Normalized Difference Glacier Index (NDGI) and the Normalized Difference Snow Ice Index (NDSII), are presented. The combination of all three indices allows discrimination of snow, ice and IMD in a systematic manner.     

双目视觉测量传感器研究

研究了按照激光三角测量原理由 1个线激光器及 2个CCD摄像机组成的双目视觉测量传感器。探讨了 2个CCD摄像机的摆放位置及其对测量精度的影响,确定了其几何参数。在此基础上,论述了其工作过程,包括其标定方法、标定过程、扫描图像采集、特征提取、特征匹配及三维数据生成。该双目视觉测量传感器配合扫描机构(如三坐标测量机 )即可对物体进行非接触式三维激光扫描测量,其测量速度快,精度比较高,可以应用于逆向工程领域。     

双目立体图像的快速匹配值计算方法

立体视觉计算过程中的匹配值计算极其耗时,整幅图的匹配值由各个参考像点的匹配值计算构成,因此,如果能够将前后两次参考像点的匹配值计算中重复的部分提取出来,那么就可以节约部分计算时间。将前后两次匹配值计算所涉及的领域像素分布情况分析清楚,结合所选取的计算步骤,将各个子步骤的中间输出结果缓存起来,以便下一次计算时直接使用而不需重新计算。如果匹配所用的邻域子图与整幅图相比非常小,则这种类似于流水作业的时间重叠计算方式可以获得很好的加速性能,同时该加速性能也会随着邻域尺寸的加大而提高,这得益于所省略的中间计算步骤较多的原因。     

A GPU persistent grid mapping for terrain rendering

In this paper we present persistent grid mapping (PGM), a novel framework for interactive view-dependent terrain rendering. Our algorithm is geared toward high utilization of modern GPUs, and takes advantage of ray tracing and mesh rendering. The algorithm maintains multiple levels of the elevation and color maps to achieve a faithful sampling of the viewed region. The rendered mesh ensures the absence of cracks and degenerate triangles that may cause the appearance of visual artifacts. In addition, an external texture memory support is provided to enable the rendering of terrains that exceed the size of texture memory. Our experimental results show that the PGM algorithm provides high quality images at steady frame rates. Electronic supplementary material Supplementary material is available in the online version of this article at and is accessible for authorized users.     

A portable stereo vision system for whole body surface imaging

This paper presents a whole body surface imaging system based on stereo vision technology. We have adopted a compact and economical configuration which involves only four stereo units to image the frontal and rear sides of the body. The success of the system depends on a stereo matching process that can effectively segment the body from the background in addition to recovering sufficient geometric details. For this purpose, we have developed a novel sub-pixel, dense stereo matching algorithm which includes two major phases. In the first phase, the foreground is accurately segmented with the help of a predefined virtual interface in the disparity space image, and a coarse disparity map is generated with block matching. In the second phase, local least squares matching is performed in combination with global optimization within a regularization framework, so as to ensure both accuracy and reliability. Our experimental results show that the system can realistically capture smooth and complete whole body shapes with high accuracy.     

Interactive stereo vision telepresence for correct communication of spatial geometry

This paper presents a novel interactive stereo vision telepresence system for use in broadband Internet-2 communication environments. An algorithm for virtual distance correction is deployed in this system which allows us to present a correct nominal depth impression of a remote environment to a local human operator. The correctness properties around the nominal depth are analyzed and an error function is derived. The efficacy of the distance correction measures is tested by evaluation experiments.     

Robust active stereo vision using Kullback-Leibler divergence

Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.     

Multi-object detection and tracking by stereo vision

This paper presents a new stereo vision-based model for multi-object detection and tracking in surveillance systems. Unlike most existing monocular camera-based systems, a stereo vision system is constructed in our model to overcome the problems of illumination variation, shadow interference, and object occlusion. In each frame, a sparse set of feature points are identified in the camera coordinate system, and then projected to the 2D ground plane. A kernel-based clustering algorithm is proposed to group the projected points according to their height values and locations on the plane. By producing clusters, the number, position, and orientation of objects in the surveillance scene can be determined for online multi-object detection and tracking. Experiments on both indoor and outdoor applications with complex scenes show the advantages of the proposed system.     

Similarity measures for image matching despite occlusions in stereo vision

In the context of computer vision, matching can be done with similarity measures. This paper presents the classification of these measures into five families. In addition, 18 measures based on robust statistics, previously proposed in order to deal with the problem of occlusions, are studied and compared to the state of the art. A new evaluation protocol and new analyses are proposed and the results highlight the most efficient measures, first, near occlusions, the smooth median powered deviation, and second, near discontinuities, a non-parametric transform-based measure, CENSUS.     

Forward-backward study of the stereo vision problem

In stereo vision a pair of two-dimensional (2D) stereo images is given and the purpose is to find out the depth (disparity) of regions of the image in relation to the background, so that we can reconstruct the 3D structure of the image from the pair of 2D stereo images given. Using the Bayesian framework we implemented the Forward-Backward algorithm to unfold the disparity (depth) of a pair of stereo images. The results showed are very reasonable, but we point out there is room for improvement concerning the graph structure used.     

Robust outdoor stereo vision SLAM for heavy machine rotation sensing

The paper presents a robust outdoor stereo vision simultaneous localization and mapping (SLAM) algorithm. It estimates camera pose reliably in outdoor environments with directional sunlight illumination causing shadows and non-uniform scene lighting. The algorithm has been developed to measure a mining rope shovel’s rotation angle about its vertical axis (“swing” axis). A stereo camera is mounted externally to the shovel house (upper revolvable portion of the shovel), with a clear view of the shovel’s lower carbody. As the shovel house swings, the camera revolves with the shovel house in a planar circular orbit, seeing differing views of the carbody top. During the swing, the SLAM algorithm builds a map of observed 3D features on the carbody and simultaneously using these landmarks to estimate the camera position. This estimated camera position is then used to compute the shovel swing angle. Two novel techniques are employed to improve the SLAM algorithm’s robustness in outdoor environments. First, a “Locally Maximal” feature selection technique for Harris corners is used to select features more consistently in non-uniformly illuminated scenes. Another novel technique is the use of 3D “Feature Clusters” as SLAM landmarks rather than individual single features. The Feature Cluster landmarks improve the robustness of the landmark matching and allow significant reduction of the SLAM filter computational cost. This approach of estimating the shovel swing angle has a maximum error of ±1° upon SLAM map convergence. Results demonstrate the improvements of using the novel techniques compared to previous methods.