Fuzzy image processing scheme for autonomous navigation of human blind

The main objective of this work is to develop an electronic travel aid to assist the blinds for obstacle identification in their navigation. This navigation assistance for visually impaired (NAVI) system presented in this paper consists of a single board processing system (SBPS), a vision sensor mounted headgear and a pair of stereo earphones. The image environment in front of the blind is captured by the vision sensor. The image is processed by a new real time image processing scheme using fuzzy clustering algorithms. The processed image is mapped onto a specially structured stereo acoustic patterns and transferred to the stereo earphones in the system. Blind individuals were trained with NAVI system and tested for obstacle identification. Suggestions from the blind volunteers regarding pleasantness and discrimination of sound pattern were also incorporated in the prototype. The proposed processing methodology is found to be effective for object identification and for producing stereo sound patterns in the NAVI system.     

利用立体视觉的室内场景分类

场景分类的目标是为各种视觉处理任务建立语义上下文,尤其是为目标识别。双目视觉系统现已广泛配备在智能机器人上,然而场景分类的任务大多只是使用单目图像。由于室内场景的复杂性,使用单目图像进行场景分类的性能很低。提出了一种基于双目视觉的室内场景分类方法,使用在一些特定区域里拟合出的若干平面的参数作为场景的特征。采用层级的分类方法,依据视差图,场景被分为开放场所类和封闭场所类,利用提出的场景特征和Gist特征对上述两类进行细分。为了验证提出的方法,建立了一个包含四种场景类别的图像数据集。实验结果表明提出的方法取得了较好的分类性能。     

基于FPGA的双目立体视觉系统

立体视觉的目的之一就是为了获得周围场景的3维信息,其关键在于匹配算法。然而即便是使用目前先进的通用处理器,其计算致密视差图所需的时间仍无法满足高速自主导航的需求。为了解决这个问题,提出了一种基于现场可编程门阵列(FPGA)的双目立体视觉系统的设计方案,同时介绍了系统的硬件结构,并在讨论区域匹配的快速算法的基础上,提出了基于FPGA的像素序列和并行窗口算法框架,用以实现零均值像素灰度差平方和(ZSSD)的匹配算法。该算法是先将视频信号经解码芯片生成场景立体图像对,并由FPGA来完成立体图像对的几何校正和ZSSD匹配算法,然后将获得的致密视差图通过PC I总线发送至上位机。实践表明,该算法效果好、速度快,不仅具有较强的鲁棒性,并且硬件系统性能稳定、可靠。此外,该方案还适用于像素灰度差的绝对值和(SAD)和像素灰度差的平方和(SSD)等多种传统区域匹配算法的快速实现和实时处理。     

基于视差平面分割的移动机器人障碍物地图构建方法

作为自主移动机器人地表障碍物探测（GPOD）技术的一部分,提出了一种利用双目摄像机的视差图像 获取信息来构建机器人前方障碍物栅格地图的方法. 该方法融合了3 维立体视觉技术以及2 维图像处理技术,前者 依据视差图的直方图信息对视差图像进行自适应平面分割,把每个平面看作是3 维场景中的实物切片进而提取障碍 物3 维信息,后者通过计算各平面上的障碍物信息曲线来提取障碍物信息,把立体视觉数据从视差图像空间变换到 2 维的障碍物地图空间. 给出了该方法构建障碍物地图的整体过程,试验结果证明了该算法的有效性和精确性.     

FPGA based disparity map computation with vergence control

Depth estimation in a scene using image pairs acquired by a stereo camera setup, is one of the important tasks of stereo vision systems. The disparity between the stereo images allows for 3D information acquisition which is indispensable in many machine vision applications. Practical stereo vision systems involve wide ranges of disparity levels. Considering that disparity map extraction of an image is a computationally demanding task, practical real-time FPGA based algorithms require increased device utilization resource usage, depending on the disparity levels operational range, which leads to significant power consumption. In this paper a new hardware-efficient real-time disparity map computation module is developed. The module constantly estimates the precisely required range of disparity levels upon a given stereo image set, maintaining this range as low as possible by verging the stereo setup cameras axes. This enables a parallel-pipelined design, for the overall module, realized on a single FPGA device of the Altera Stratix IV family. Accurate disparity maps are computed at a rate of more than 320 frames per second, for a stereo image pair of 640 × 480 pixels spatial resolution with a disparity range of 80 pixels. The presented technique provides very good processing speed at the expense of accuracy, with very good scalability in terms of disparity levels. The proposed method enables a suitable module delivering high performance in real-time stereo vision applications, where space and power are significant concerns.     

Stereo vision for planetary rovers: Stochastic modeling to near real-time implementation

NASA scenarios for lunar and planetary missions include robotic vehicles that function in both teleoperated and semi-autonomous modes. Under teleoperation, on-board stereo cameras may provide 3-D scene information to human operators via stereographic displays; likewise, under semi-autonomy, machine stereo vision may provide 3-D information for obstacle avoidance. In the past, the slow speed of machine stereo vision systems has posed a hurdle to the semi-autonomous scenario; however, recent work at JPL and other laboratories has produced stereo systems with high reliability and near real-time performance for low-resolution image pairs. In particular, JPL has taken a significant step by achieving the first autonomous, cross-country robotic traverses (of up to 100 meters) to use stereo vision, with all computing on-board the vehicle. Here, we describe the stereo vision system, including the underlying statistical model and the details of the implementation. The statistical and algorithmic aspects employ random field models of the disparity map, Bayesian formulations of single-scale matching, and area-based image comparisons. The implementation builds bandpass image pyramids and produces disparity maps from the 60×64 level of the pyramids at rates of up to two seconds per image pair. All vision processing is done in one 68020 augmented with Datacube image processing boards. We argue that the overall approach provides a unifying paradigm for practical, domain-independent stereo ranging. We close with a discussion of practical and theoretical issues involved in evaluating and extending the performance of the stereo system.     

Practical Structure and Motion from Stereo When Motion is Unconstrained

This paper describes a system which robustly estimates motion, and the 3D structure of a rigid environment, as a stereo vision platform moves through it. The system can cope with any camera motion, and any scene structure and is successful even in the presence of large jumps in camera position between the capture of successive image pairs, and when point matching is ambiguous. The system was developed to provide robust obstacle avoidance for a partially sighted person.The process described attempts to maximise use of the abundant information present in a stereo sequence. Key features include the use of multiple stereo match hypotheses, efficient motion computation from three images, and the use of this motion to ensure reliable matching, and to eliminate multiple stereo matches. Points are reconstructed in 3D space and tracked in a static coordinate frame with a Kalman Filter.This results in good 3D scene reconstructions. Structure which is impossible to match with certainty is absent, rather than being incorrectly reconstructed. As a result, the system is appropriate for obstacle detection. The results of processing some indoor and outdoor scenes, are given in the paper, and practical issues are highlighted throughout.     

Depth estimation and image restoration using defocused stereo pairs

We propose a method for estimating depth from images captured with a real aperture camera by fusing defocus and stereo cues. The idea is to use stereo-based constraints in conjunction with defocusing to obtain improved estimates of depth over those of stereo or defocus alone. The depth map as well as the original image of the scene are modeled as Markov random fields with a smoothness prior, and their estimates are obtained by minimizing a suitable energy function using simulated annealing. The main advantage of the proposed method, despite being computationally less efficient than the standard stereo or DFD method, is simultaneous recovery of depth as well as space-variant restoration of the original focused image of the scene.     

Design and application of industrial machine vision systems

In this paper, the role and importance of the machine vision systems in the industrial applications are described. First understanding of the vision in terms of a universal concept is explained. System design methodology is discussed and a generic machine vision model is reported. Such a machine includes systems and sub-systems, which of course depend on the type of applications and required tasks. In general, expected functions from a vision machine are the exploitation and imposition of the environmental constraint of a scene, the capturing of the images, analysis of those captured images, recognition of certain objects and features within each image, and the initiation of subsequent actions in order to accept or reject the corresponding objects. After a vision system performs all these stages, the task in hand is almost completed. Here, the sequence and proper functioning of each system and sub-systems in terms of high-quality images is explained. In operation, there is a scene with some constraint, first step for the machine is the image acquisition, pre-processing of image, segmentation, feature extraction, classification, inspection, and finally actuation, which is an interaction with the scene under study. At the end of this report, industrial image vision applications are explained in detail. Such applications include the area of automated visual inspection (AVI), process control, parts identification, and important role in the robotic guidance and control. Vision developments in manufacturing that can result in improvements in the reliability, in the product quality, and enabling technology for a new production process are presented. The key points in design and applications of a machine vision system are also presented. Such considerations can be generally classified into the six different categories such as the scene constraints, image acquisition, image pre-processing, image processing, machine vision justification, and finally the systematic considerations. Each aspect of such processes is described here and the proper condition for an optimal design is reported.     

双目视觉模型移动目标跟踪系统

为了使机器人能够对目标进行自主智能的跟踪,基于仿生学原理建立双目视觉模型,设计出四自由度的仿人眼颈跟踪系统,通过上位机与下位机的配合以及2台摄像机与4台直流电机,实现双目视觉移动目标跟踪系统.该系统通过双目摄像机对目标进行图像采集,由PC机对目标进行特征提取以及一系列处理后计算出目标的移动范围,将目标移动数据传输至下位机后通过PID控制算法平滑控制直流电机的运转方向与运转速度,从而对移动目标实现良好的跟踪性能.     

基于新型双目机构的立体视觉系统标定

为了获得更大的摄像机视野和更高的系统鲁棒性，研制了一种新型的双目机构；并以此机构为核心搭建了立体视觉系统，之后对整个系统进行了标定．为有效地消除图像的畸变，提出了一种显—隐式畸变校正方法；此方法可以同时校正图像的径向和切向畸变，用最小二乘法即可求解．提出了一种简单而精确的分步旋转法来校准左右图像的极线．实验结果表明，经过显—隐式畸变校正方法和分步旋转法处理的立体图对，可以满足立体匹配的要求．     

A real-time fuzzy hardware structure for disparity map computation

Stereo images acquired by a stereo camera setup provide depth estimation of a scene. Numerous machine vision applications deal with retrieval of 3D information. Disparity map recovery from a stereo image pair involves computationally complex algorithms. Previous methods of disparity map computation are mainly restricted to software-based techniques on general-purpose architectures, presenting relatively high execution time. In this paper, a new hardware-implemented real-time disparity map computation module is realized. This enables a hardware-based fuzzy inference system parallel-pipelined design, for the overall module, implemented on a single FPGA device with a typical operating frequency of 138 MHz. This provides accurate disparity map computation at a rate of nearly 440 frames per second, given a stereo image pair with a disparity range of 80 pixels and 640 × 480 pixels spatial resolution. The proposed method allows a fast disparity map computational module to be built, enabling a suitable module for real-time stereo vision applications.     

基于FPGA的智能视频融合技术研究

针对夜间或恶劣环境下交通事故频发的情况,提出一种在夜间或恶劣环境中辅助驾驶员改善能见度的技术,本技术是采用在FPGA内部对CCD捕获的可见光视频流和红外摄像机捕获的红外视频流进行实时融合、处理以获得极低或极强光照条件下图像信息的新方法;整个系统流程都在FPGA内实时完成,经过处理的视频流直接显示在位于司机前方的平视显示器上,以协助驾驶者在能见度低的环境中安全地行驶,在各种实验条件下经初步验证都取得了不错的效果。     

Some accuracy and resolution aspects of computer vision distance measurements

The distance between an object and stereo vision sensors can be measured using image processing and known system parameters. A detailed distance measurement synthesis procedure to meet system specifications is presented and illustrated with an example. An error analysis shows that error is proportional to distance. System parameters such as separation between sensor elements, sensor focal length, and sensor array dimensions are related in the design and error equations presented. The main desired design goal is to establish the smallest image sensor array size which will meet system operating specifications. Minimum and maximum distance, object height, optic parameters, scene shift, and sensor array parameters are related.     

Stereo-motion with stereo and motion in complement

This paper presents a new approach of combining stereo vision and dynamic vision with the objective of retaining their advantages and removing their disadvantages. It is shown that, by assuming affine cameras, the stereo correspondences and motion correspondences, if organized in a particular way in a matrix, can be decomposed into: the 3D structure of the scene, the camera parameters, the motion parameters, and the stereo geometry. With this, the approach can infer stereo correspondences from motion correspondences, requiring only a time linear with respect to the size of the available image data. The approach offers the advantages of simpler correspondence, as in dynamic vision, and accurate reconstruction, as in stereo vision, even with short image sequences     

Feature transfer and matching in disparate stereo views through the use of plane homographies

Many vision tasks rely upon the identification of sets of corresponding features among different images. This paper presents a method that, given some corresponding features in two stereo images, matches them with features extracted from a second stereo pair captured from a distant viewpoint. The proposed method is based on the assumption that the viewed scene contains two planar surfaces and exploits geometric constraints that are imposed by the existence of these planes to first transfer and then match image features between the two stereo pairs. The resulting scheme handles point and line features in a unified manner and is capable of successfully matching features extracted from stereo pairs that are acquired from considerably different viewpoints. Experimental results are presented, which demonstrate that the performance of the proposed method compares favorably to that of epipolar and tensor-based approaches.     

Terrain mapping under extreme light conditions with direct stereo matching method through aggregating matching costs by weight

One of the biggest problems in applying stereo vision techniques in field robotics is how to acquire 3D terrain maps under extreme light conditions. Through multiple exposures, the dynamic range of images can be increased. In this paper, instead of using existing lighting enhancement methods such as exposure fusion to increase the texture of 2D image, we propose that the matching costs of the images grabbed with multiple exposures are directly summed by weight. Compared with the previous methods such as exposure fusion, with the proposed method, it is not necessary to fuse the 2D images captured with multiple exposures, and for each pixel of the matching image, the local information in its local window can be better retained. Since it is possible that the camera is moved between exposures when the images are grabbed, the images captured with multiple exposures are aligned to the image acquired with auto exposure. In order to evaluate the performance of the proposed method, two different stereo matching algorithms were used: a local window-based method and semi-global method. Through experiments in laboratory and outdoors with a stereo vision camera fixed on a tripod and held in the hand, it was verified that the proposed method consistently allowed more valid points to be obtained and the 3D model of terrain can be built more accurately. Especially when the local window-based method was used, the proposed method performed much better.     

Cloud-based approach for automatic CNC workpiece origin localization based on image analysis

Computer vision applications in the industry have been a constant field of research in academia. Industrial daily challenges such as quality inspection, object detection, and measurement are examples of situations where some automation could be done by using computer vision techniques. In this paper, a cloud-based approach of an automatic system based on stereo vision and image analysis has been developed to automate a daily routine present in machining companies: workpiece referencing. The proposed architecture uses two cameras mounted in the spindle of a machining center. All images are processed in custom software, running on the cloud, to return the position of the Workpiece Coordinate System (WCS) directly to the Computer Numerical Control (CNC) machine controller. Experimental results validate the application of the proposed architecture in a real machining process machine.