A Portable Bio-Inspired Architecture for Efficient Robotic Vergence Control

In stereoscopic vision, the ability of perceiving the three-dimensional structure of the surrounding environment is subordinated to a precise and effective motor control for the binocular coordination of the eyes/cameras. If, on the one side, the binocular coordination of camera movements is a complicating factor, on the other side, a proper vergence control, acting on the binocular disparity, facilitates the binocular fusion and the subsequent stereoscopic perception process. In real-world situations, an effective vergence control requires further features other than real time capabilities: real robot systems are indeed characterized by mechanical and geometrical imprecision that affect the binocular vision, and the illumination conditions are changeable and unpredictable. Moreover, in order to allow an effective visual exploration of the peripersonal space, it is necessary to cope with different gaze directions and provide a large working space. The proposed control strategy resorts to a neuromimetic approach that provides a distributed representation of disparity information. The vergence posture is obtained by an open-loop and a closed-loop control, which directly interacts with saccadic control. Before saccade, the open-loop component is computed in correspondence of the saccade target region, to obtain a vergence correction to be applied simultaneously with the saccade. At fixation, the closed-loop component drives the binocular disparity to zero in a foveal region. The obtained vergence servos are able to actively drive both the horizontal and the vertical alignment of the optical axes on the object of interest, thus ensuring a correct vergence posture. Experimental tests were purposely designed to measure the performance of the control in the peripersonal space, and were performed on three different robot platforms. The results demonstrated that the proposed approach yields real-time and effective vergence camera movements on a visual stimulus in a wide working range, regardless of the illumination in the environment and the geometry of the system.     

虚拟环境中的视觉感知

通过对现实世界,平面图像的视觉成象方式的比较,指出虚拟环境成象方式是二者相结合的产物。在论述虚拟环境所提供的视差对三维深度感知重要性的同时,结合视觉生理特点,分析总结了虚拟环境所具有的缺陷,即违背了视标所发出的入瞳光线的散开度与双眼所处的状态应该相辅相成这一规律,提出了虚拟环境给队造成头晕、眼睛疲劳等不良反应的根源是虚拟对象容易使双眼的辐辏和调节功能发生冲突。最后列举并分析了刷新延迟、分辩率、瞳距等各种可能给观察者带来不良反应的诸多因素。     

Purposeful gazing and vergence control for active vision

The purposeful-gazing capability of active vision offers advantages to many manufacturing tasks. This paper discusses the problems associated with purposeful gazing and fixation of attention for active vision. In binocular active vision, gazing at a selected target refers to directing the visual axes to capture the target in the (appropriate part of) the visual field by both sensors (cameras), and holding gaze refers to directing the visual axes of the sensors so as to maintain the target or point of interest in the visual field of both sensors. This paper proposes solutions to the important problems involved in gaze stabilization by developing techniques for vergence error extraction, and vergence servo control. Vergence is the tilt movement process of two visual sensors (in a binocular system) in opposite directions to fixate at a selected point. Binocular gazing is realized by decreasing the disparity which represents the vergence error. In order to obtain the disparity for extraction of vergence error, a phase-based approach that robustly and efficiently estimates vergence disparity is developed. To control vergence, we present a dual sampling-rate approach for vision-sensor-based dynamic servo control.     

Towards a unified visual framework in a binocular active robot vision system

This paper presents the results of an investigation and pilot study into an active binocular vision system that combines binocular vergence, object recognition and attention control in a unified framework. The prototype developed is capable of identifying, targeting, verging on and recognising objects in a cluttered scene without the need for calibration or other knowledge of the camera geometry. This is achieved by implementing all image analysis in a symbolic space without creating explicit pixel-space maps. The system structure is based on the ‘searchlight metaphor’ of biological systems. We present results of an investigation that yield a maximum vergence error of ～6.5 pixels, while ～85% of known objects were recognised in five different cluttered scenes. Finally a ‘stepping-stone’ visual search strategy was demonstrated, taking a total of 40 saccades to find two known objects in the workspace, neither of which appeared simultaneously within the field of view resulting from any individual saccade.     

Promising directions in active vision

Active vision systems have mechanisms that can actively control camera parameters such as position, orientation, focus, zoom, aperture, and vergence (in a two-camera system) in response to the requirements of the task and external stimuli. They may also have features such as spatially variant (foveal) sensors. More broadly, active vision encompassesattention, selective sensing in space, resolution, and time, whether it is achieved by modifying physical camera parameters or the way data is processed after leaving the camera.In the active-vision paradigm, the basic components of the visual system are visual behaviors tightly integrated with the actions they support; these behaviors may not require elaborate categorical representations of the 3-D world. Because the cost of generating and updating a complete, detailed model of most environments is too high, this approach to vision is vital for achieving robust, real-time perception in the real world. In addition, active control of imaging parameters has been shown to simplify scene interpretation by eliminating the ambiguity present in single images. Important research areas in active vision include attention, foveal sensing, gaze control, eye-hand coordination, and integration with robot architectures.The workshop was supported by NSF grant IRL-9119911, and by generous contributions from IBM and Hughes Research Laboratories. The editors received support from ONR grant N00014-91-J-1185.     

Super multi‐view 3D displays reduce conflict between accommodative and vergence responses

A conflict between accommodation and vergence is one possible cause of visual fatigue and discomfort while viewing conventional three‐dimensional displays. Previous studies have proposed the super multi‐view (SMV) display technique to solve the vergence–accommodation conflict, in which two or more parallax images enter the pupil of the eye with highly directional rays. We simultaneously measured accommodative, vergence, and pupillary responses to SMV three‐dimensional displays to examine whether they can reduce the conflict. For comparison, responses to two‐view stereo images and real objects were also measured. The results show that the range of the accommodative response was increased by the SMV images compared with the two‐view images. The slope of the accommodation–vergence response function for the SMV images was similar to that for the real objects rather than the two‐view images. We also found that enhancement of the accommodative range by the SMV images is noticeable with binocular viewing, indicating that vergence‐induced accommodation plays an important role in viewing SMV displays. These results suggest that SMV displays induced a more natural accommodative response than did conventional, two‐view stereo displays. As a result, SMV displays reduced the vergence–accommodation conflict.     

Analysis of Error in Depth Perception with Vergence and Spatially Varying Sensing

In stereo vision the depth of a 3-D point is estimated based on the position of its projections on the left and right images. The image plane of cameras that produces the images consists of discrete pixels. This discretization of images generates uncertainty in estimation of the depth at each 3-D point. In this paper, we investigate the effect of vergence and spatially varying resolution on the depth estimation error. First, vergence is studied when pairs of stereo images with uniform resolution are used. Then the problem is studied for a stereo system similar to that of humans, in which cameras have high resolution in the center and nonlinearly decreasing resolution toward the periphery. In this paper we are only concerned with error in depth perception, assuming that stereo matching is already done.     

基于双目视觉系统的列车轨道异物距离检测

在线检测列车与轨道异物问的距离并为驾驶员提供减速信息对于行车安全具有重要意义。本文提出了一种基于双目视觉系统的列车轨道异物距离检测方法。通过对在机车上架设的摄像机获得的双目图像中待测目标对应点的准确匹配,应用双目视觉系统原理,根据左右视觉图像的视差实现机车与异物间距离的计算。     

Performance analysis of stereo, vergence, and focus as depth cuesfor active vision

This paper compares the performances of the binocular cues of stereo and vergence, and the monocular cue of focus for range estimation using an active vision system. The performance of each cue is characterized in terms of sensitivity to errors in the imaging parameters. The effects of random, quantization errors are expressed in terms of the standard deviation of the resulting depth error. The effect of systematic, calibration errors on estimation using each cue is also studied. Performance characterization of each cue is utilized to evaluate the relative performance of the cues. Also discussed, based on such characterization, are ways to select a cue taking into account the computational and reliability aspects of the corresponding estimation process     

Real-time visuomotor update of an active binocular head

In order for a binocular head to perform optimal 3D tracking, it should be able to verge its cameras actively, while maintaining geometric calibration. In this work we introduce a calibration update procedure, which allows a robotic head to simultaneously fixate, track, and reconstruct a moving object in real-time. The update method is based on a mapping from motor-based to image-based estimates of the camera orientations, estimated in an offline stage. Following this, a fast online procedure is presented to update the calibration of an active binocular camera pair. The proposed approach is ideal for active vision applications because no image-processing is needed at runtime for the scope of calibrating the system or for maintaining the calibration parameters during camera vergence. We show that this homography-based technique allows an active binocular robot to fixate and track an object, whilst performing 3D reconstruction concurrently in real-time.     

Dynamic fixation and active perception

Fixation is the link between the physical environment and the visual observer, both of which can be dynamic. That is, dynamic fixation serves the task of preserving a reference point in the world, despite relative motion. In this respect, fixation is dynamical in two senses: in response to voluntary changes of fixation point or attentive cues-gaze shiftings, and in response to the desire to compensate for the retinal slip-gaze holding.The work presented here, addresses the vergence movement and preservation of binocular fixation during smooth pursuit. This movement is a crucial component of fixation. The two vergence processes, disparity vergence and accommodative vergence, are described; a novel algorithm for robust disparity vergence and an active approach for blur detection and depth from defocus are presented. The main characteristics of the disparity vergence technique are the simplicity of the algorithm, the influence of both left and right images in the course of fixation and the agreement with the fixation model of primates. The major characteristic of the suggested algorithm for blur detection is its active approach which makes it suitable for achieving qualitative and reasonable depth estimations without unrealistic assumptions about the structures in the images.The paper also covers the integration of the two processes disparity vergence and accommodation vergence which are in turn accomplished by an integration of the disparity and blur stimuli. This integration is accounted for in both static and dynamic experiments.     

Dynamic Vergence Using Log-Polar Images

Vergence provides robot vision systems with a crucial degree of freedom: it enables fixation of points in visual space at different distances from the observer. Vergence control, therefore, affects the performance of the stereo system as well as the results of motion estimation and tracking and, as such, must satisfy different requirements in order to be able to provide not only a stable fixation, but a stable binocular fusion, and a fast, smooth and accurate reaction to changes in the environment. To obtain this kind of performance the paper focuses specifically on the use of dynamic visual information to drive vergence control. In this context, moreover, the use of a space-variant, anthropomorphic sensor is described and some advantages in relation to vergence control are discussed to demonstrate the relevance of image plane geometry for this particular task. Expansion or contraction patterns and the temporal evolution of the degree of fusion measured in the log-polar domain are the inputs to the vergence control system and determine robust and accurate steering of the two cameras. Real-time experiments are presented to demonstrate the performance of the system covering different key situations.     

Real-time vergence control for binocular robots

In binocular systems,vergence is the process of adjusting the angle between the eyes (or cameras) so that both eyes are directed at the same world point. Its utility is most obvious for foveate systems such as the human visual system, but it is a useful strategy for nonfoveate binocular robots as well. Here, we discuss the vergence problem and outline a general approach to vergence control, consisting of a control loop driven by an algorithm that estimates the vergence error. As a case study, this approach is used to verge the eyes of the Rochester Robot in real time. Vergence error is estimated with the cepstral disparity filter. The cepstral filter is analyzed, and it is shown in this application to be equivalent to correlation with an adaptive prefilter; carrying this idea to its logical conclusion converts the cepstral filter into phase correlation. The demonstration system uses a PD controller in cascade with the error estimator. An efficient real-time implementation of the error estimator is discussed, and empirical measures of the performance of both the disparity estimator and the overall system are presented.     

基于EVision的双目视觉系统的设计与实现

设计和开发了一个排爆机器人的双目立体视觉系统。该系统使用Matlab7.0和机器视觉软件EVision6.2的EasyMultiCam进行图像捕获并和EasyMath模式匹配库进行图像特征匹配,能实时捕获排爆机器人周围的图像信息、进行摄像机标定、图像预处理、立体图像匹配、求取可疑目标物的特征点的立体坐标，并把图中实时显示在控制台上，控制排爆机器人准确地抓取可疑目标物。该机器人视觉系统成功的抓取实验，表明了它在精度上能够满足排爆机器人的项目要求。     

Preferred vertical gaze direction and observation distance

Hill and Kroemer (1986) and Kroemer and Hill (1986) found that the preferred vertical direction of gaze is lower with a nearer binocular stimulus than with a more distant one. A model is proposed that relates this phenomenon to characteristics of the resting state of the oculomotor system. Three predictions of the model were tested, based on measurements of the preferred vertical gaze direction and dark vergence in the same subject sample. On average the effect of observation distance on the preferred vertical gaze direction served to reduce the discrepancy between the resting state of vergence, operationally defined as dark vergence, and actual convergence during inspection of the binocular stimulus. Second, dark vergence data from individual subjects could be successfully used to predict whether they raised or lowered their eyes on inspection of a binocular stimulus as compared with the preferred vertical gaze direction while viewing a monocular stimulus. Finally, predictions of the size of the change of the preferred vertical gaze direction on introduction of a binocular stimulus produced only small and non-significant correlations.     

双目视觉模型移动目标跟踪系统

为了使机器人能够对目标进行自主智能的跟踪,基于仿生学原理建立双目视觉模型,设计出四自由度的仿人眼颈跟踪系统,通过上位机与下位机的配合以及2台摄像机与4台直流电机,实现双目视觉移动目标跟踪系统.该系统通过双目摄像机对目标进行图像采集,由PC机对目标进行特征提取以及一系列处理后计算出目标的移动范围,将目标移动数据传输至下位机后通过PID控制算法平滑控制直流电机的运转方向与运转速度,从而对移动目标实现良好的跟踪性能.     

An intensity-based cooperative bidirectional stereo matching with simultaneous detection of discontinuities and occlusions

This paper presents a new intensity-based stereo algorithm using cooperative bidirectional matching with a hierarchical multilevel structure. Based on a new model of piecewise smooth depth fields and the consistency constraint, the algorithm is able to estimate the 3-D structure and detect its discontinuities and the occlusion reliably with low computational costs. In order to find the global optimal estimates, we utilize a multiresolution two-stage algorithm minimizing nonconvex cost functions, which is equivalent to the MAP estimation. This basic framework computing the 3-D structure from binocular stereo images has been extended to the trinocular stereo vision for a further improvement of the performance. A few examples for the binocular and trinocular stereo problems are given to illustrate the performance of the new algorithms.     

A Novel Space Variant Image Representation

Traditionally, in machine vision images are represented using cartesian coordinates with uniform sampling along the axes. On the contrary, biological vision systems represent images using polar coordinates with non-uniform sampling. For various advantages provided by space-variant representations many researchers are interested in space-variant computer vision. In this direction the current work proposes a novel and simple space variant representation of images. The proposed representation is compared with the classical log-polar mapping. The log-polar representation is motivated by biological vision having the characteristic of higher resolution at the fovea and reduced resolution at the periphery. On the contrary to the log-polar, the proposed new representation has higher resolution at the periphery and lower resolution at the fovea. Our proposal is proved to be a better representation in navigational scenarios such as driver assistance systems and robotics. The experimental results involve analysis of optical flow fields computed on both proposed and log-polar representations. Additionally, an egomotion estimation application is also shown as an illustrative example. The experimental analysis comprises results from synthetic as well as real sequences.     

排爆机器人双目立体视觉系统的研究和开发

为了辅助公安人员更好地完成排爆工作，设计和开发了一个带双目立体视觉系统的排爆机器人。该视觉系统使用Matlab7.0和机器视觉软件EVision的EasyMatch库进行开发，能实时捕获排爆机器人周围的图像信息、进行摄像机标定、图像预处理、立体图像匹配、求取可疑目标物的特征点的立体坐标，并把图像实时显示在控制台，控制排爆机器人准确地抓取可疑目标物。该机器人视觉系统成功地抓取实验，表明了它在精度上能够满足排爆机器人的项目要求。