Model-driven active visual tracking

We have previously demonstrated that the performance of tracking algorithms can be improved by integrating information from multiple cues in a model-driven Bayesian reasoning framework. Here we extend our work to active vision tracking, with variable camera geometry. Many existent active tracking algorithms avoid the problem of variable camera geometry by tracking view independent features, such as corners and lines. However, the performance of algorithms based on those single features will greatly deteriorate in the presence of specularities and dense clutter. We show, by integrating multiple cues and updating the camera geometry on-line, that it is possible to track a complicated object moving arbitrarily in three-dimensional (3D) space.We use a four degree-of-freedom (4-DoF) binocular camera rig to track three focus features of an industrial object, whose complete model is known. The camera geometry is updated by using the rig control commands and kinematic model of the stereo head. The extrinsic parameters are further refined by interpolation from a previously sampled calibration of the head work space.The 2D target position estimates are obtained by a combination of blob detection, edge searching and gray-level matching, with the aid of model geometrical structure projection using current estimates of camera geometry. The information is represented in the form of a probability density distribution, and propagated in a Bayes Net. The Bayesian reasoning that is performed in the 2D image is coupled by the rigid model geometry constraint in 3D space.An αβ filter is used to smooth the tracking pursuit and to predict the position of the object in the next iteration of data acquisition. The solution of the inverse kinematic problem at the predicted position is used to control the position of the stereo head.Finally, experiments show that the target undertaking arbitrarily 3D motion can be successfully tracked in the presence of specularities and dense clutter.     

Calibration of an active binocular head

In this paper, we show how an active binocular head, the IIS head, can be easily calibrated with very high accuracy. Our calibration method can also be applied to many other binocular heads. In addition to the proposal and demonstration of a four-stage calibration process, there are three major contributions in this paper. First, we propose a motorized-focus lens (MFL) camera model which assumes constant nominal extrinsic parameters. The advantage of having constant extrinsic parameters is to having a simple head/eye relation. Second, a calibration method for the MFL camera model is proposed in this paper, which separates estimation of the image center and effective focal length from estimation of the camera orientation and position. This separation has been proved to be crucial; otherwise, estimates of camera parameters would be very noise-sensitive. Thirdly, we show that, once the parameters of the MFL camera model is calibrated, a nonlinear recursive least-square estimator can be used to refine all the 35 kinematic parameters. Real experiments have shown that the proposed method can achieve accuracy of one pixel prediction error and 0.2 pixel epipolar error, even when all the joints, including the left and right focus motors, are moved simultaneously. This accuracy is good enough for many 3D vision applications, such as navigation, object tracking and reconstruction     

基于双目立体视觉的船舶轨迹跟踪算法研究

双目立体视觉模型通过模拟人眼可以实现对目标距离的测量。为了获得水上船舶实时的运动状态,提出了一种基于双目立体视觉的船舶轨迹跟踪方法。首先,通过摄像机标定、线性空间点三维重建可以测得以相机为中心到船舶的距离,得到船舶的部分运动轨迹;其次,在双目立体视觉测距系统的基础上采用常速(Constant Velocity,CV)模型的方法对船舶运动建模;最后,对建立的船舶运动模型利用强跟踪卡尔曼滤波(Strong Tracking Kalman Filter,STKF)船舶轨迹跟踪的方法跟踪船舶的轨迹并估算目标船舶实时的运动状态。实验结果表明,基于双目立体视觉的船舶轨迹跟踪的方法能有效地跟踪船舶轨迹且适用于工程应用的需求。     

基于双目视觉的运动物体实时跟踪与测距

利用双目视觉信息系统实现三维空间中运动物体实时跟踪与测距．当运动目标超出视野范围时,可通 过控制摄像机云台转动搜索目标．此外,还研究了在摄像头运动情况下,无需重新标定,即可实现运动物体测距的 算法．这里,自适应背景建模法与CamShift 算法用于实现运动物体的辨识与跟踪．实验结果证明了所提出的算法能 够有效地追踪物体,并同时准确地测量它的三维位置．     

基于一阶径向畸变算法的双目摄像机多位姿标定方法

双目立体视觉中在对物体进行三维测量或精准定位时,需要对摄像机进行标定以获得其内外参数。研究径向畸变摄像机模型,构造了基于一阶径向畸变（RAC）算法的双目摄像机内外参数线性求解公式。考虑侧倾角、旋转角、俯仰角以及透镜的主要畸变因素,修正了传统RAC标定法中只考虑径向畸变、部分参数需要先验值的缺陷。利用标定所得内、外参数进行了多位姿双目摄像机三维重构实验。实验结果表明,该标定方法重投影误差分布在[-0.3,0.3],动态识别结果与实际运行轨迹重合率为96%,对降低双目立体视觉三维测量误差率有积极性影响。     

3D head tracking for fall detection using a single calibrated camera

The head trajectory is an interesting source of information for behavior recognition and can be very useful for video surveillance applications, especially for fall detection. Consequently, much work has been done to track the head in the 2D image plane using a single camera or in a 3D world using multiple cameras. Tracking the head in real-time with a single camera could be very useful for fall detection. Thus, in this article, an original method to extract the 3D head trajectory of a person in a room is proposed using only one calibrated camera. The head is represented as a 3D ellipsoid, which is tracked with a hierarchical particle filter based on color histograms and shape information. Experiments demonstrated that this method can run in quasi-real-time, providing reasonable 3D errors for a monocular system. Results on fall detection using the head 3D vertical velocity or height obtained from the 3D trajectory are also presented.     

基于AdaBoost的公交客流量统计算法

为解决复杂场景目标识别中伪目标的干扰问题,采用基于AdaBoost分类的方法分析疑似目标的三维轨迹,结合真实目标的共有的特征信息,进一步分类真实目标与伪目标。首先,根据深度相机获取的深度图像提取疑似目标的人头区域,利用Kalman滤波跟踪得到的二维轨迹。其次,通过摄像机标定将目标的二维轨迹转换为空间中的三维轨迹。最后,利用AdaBoost训练正负样本得到强分类器,进一步分类真实目标与伪目标。实验结果表明,该方法能够有效的提高目标识别的精度,对复杂场景下的目标识别具有良好的适应性。     

双线阵CCD相机的畸变校正和标定方法

在双线阵CCD的三维重建中,对线阵CCD相机的标定和镜头畸变校正是基础环节。提出了一种用于三维重建中的双线阵CCD标定及镜头畸变校正方法。根据左右相机间的单应性关系,以及线阵CCD的成像原理,将双目相机间的空间关系分解成姿态角与错切角的关系。通过靶图数据的拟合,对姿态角和镜头畸变进行校正,根据求出的错切角完成相机间的标定,实现对具有镜头畸变的双线阵CCD的标定。实验结果表明,标定和校正精度满足后续三维重建中图像匹配的需求。     

Keeping smart, omnidirectional eyes on you [adaptive panoramic stereovision]

An adaptive panoramic stereo approach for two cooperative mobile platform is presented. There are four key features in the approach: 1) omnidirectional stereovision with an appropriate vertical FOV, and a simple camera calibration method; 2) cooperative mobile platforms for mutual dynamic calibration and best view planning; 3) 3D matching after meaningful object (human subject) extraction; and 4) real-time performance. The integration of omnidirectional vision with mutual awareness and dynamic calibration strategies allows intelligent cooperation between visual agents. This provides an effective way to solve the problems of limited resources, view planning, occlusion, and motion detection of movable robotic platforms. Experiments have shown that this approach is quite promising.     

一种改进的相机标定方法

摄像机标定与图像畸变修正是摄影测量、视觉检测、计算机视觉等领域的重点研究课题之一,在测绘、工业控制、导航、军事等领域得到了极大的应用。研究了摄像机模型,摄像机标定等内容。对DLT的标定方法进行了改进,在摄像机模型中全面考虑了镜头的畸变,利用图像中心附近点畸变量较小的性质,提出一种摄像机内外部参数和像差修正参数分离的标定方法。并举例说明了基于同一物体的两幅图画三维重构的具体实验步骤和方法,分析了影响精度的因素。     

基于双目视觉的移动机器人空间测距研究

空间测距是移动机器人实时避障和路径规划的基础.研究了移动机器人双目摄像头由一般布置形式变换到标准布置形式进而实现空间测距的方法.首先对摄像头进行标定(对双目摄像头进行立体标定);然后基于标定结果对摄像头一般布置下所获得立体图像进行极线校正,并得到校正后新的摄像机相互位置参数;最后基于已校正图像和新的位置参数实现移动机器人移动过程中深度与特定目标空间尺寸测量.实验结果证实了本文方法能获得比较精确的测量结果.     

Real-time 3D face tracking based on active appearance model constrained by depth data

Active Appearance Model (AAM) is an algorithm for fitting a generative model of object shape and appearance to an input image. AAM allows accurate, real-time tracking of human faces in 2D and can be extended to track faces in 3D by constraining its fitting with a linear 3D morphable model. Unfortunately, this AAM-based 3D tracking does not provide adequate accuracy and robustness, as we show in this paper. We introduce a new constraint into AAM fitting that uses depth data from a commodity RGBD camera (Kinect). This addition significantly reduces 3D tracking errors. We also describe how to initialize the 3D morphable face model used in our tracking algorithm by computing its face shape parameters of the user from a batch of tracked frames. The described face tracking algorithm is used in Microsoft's Kinect system.     

基于格雷码和多步相移法的双目立体视觉三维测量技术研究

传统的被动式双目立体视觉三维测量技术,具有操作简单,使用灵活方便,相机标定技术成熟的优点,但是对于特征点稀疏图像,寻找匹配点困难,匹配精度低。编码结构光测量方式通过向待测物体投射特定的编码图案,获取编码图像进行解码求解物体的三维信息,具有着测量精度高,速度快的优点,但是存在着投影仪标定精度低,实现难度大的缺点。提出了将双目立体视觉和编码结构光相结合的三维测量方法,在完成双目校正的基础上,向待测物体投射格雷码图案和多步相移图案,给予被测物体容易识别和可控制的特征信息,最后求取物体的三维信息。而且通过实验论证了投射多步相移图案比起4步相移图案,测量精度更高,能够更好的体现物体细节。     

基于边的自适应实时三维跟踪

针对缺乏纹理特征的物体,提出了一种基于边的自适应实时三维跟踪方法。在已知物体三维模型的情况下,通过基于历史运动信息的物体边缘检测与跟踪,可以有效准确地求解出摄像机的外参。基于并扩展了现有的基于边的实时跟踪算法,其主要工作体现在以下三个方面： 1）提出自适应阈值和基于历史信息估计当前帧的运动趋势的方法,从而提高边匹配算法在快速运动时的稳定性;2）提出一种基于随机抽样一致性（RANSAC）的边匹配策略,可以有效剔除误匹配的边,从而提高复杂模型的跟踪稳定性;3）利用抽取轮廓边的算法将边跟踪算法从CAD模型扩展到一般的面片模型。实验结果证明了该方法的鲁棒高效,能够满足增强现实、虚拟装配等应用需求。     

Partial camera automation in an unmanned air vehicle

The present study focused on an intelligent, semiautonomous, interface for a camera operator of a simulated unmanned air vehicle (UAV). This interface used system “knowledge” concerning UAV motion in order to assist a camera operator in tracking an object moving through the landscape below. The semiautomated system compensated for the translations of the UAV relative to the earth. This compensation was accompanied by the appropriate joystick movements ensuring tactile (haptic) feedback of these system interventions. The operator had to superimpose self-initiated joystick manipulations over these system-initiated joystick motions in order to track the motion of a target (a driving truck) relative to the terrain. Tracking data showed that subjects performed substantially better with the active system. Apparently, the subjects had no difficulty in maintaining control, i.e. “following” the active stick while superimposing self-initiated control movements over the system-interventions. Furthermore, tracking performance with an active interface was clearly superior relative to the passive system. The magnitude of this effect was equal to the effect of update-frequency (2-5 Hz) of the monitor image. The benefits of update frequency enhancement and semiautomated tracking were the greatest under difficult steering conditions. Mental workload scores indicated that, for the difficult tracking-dynamics condition, both semiautomation and update frequency increase resulted in less experienced mental effort. For the easier dynamics this effect was only seen for update frequency     

双目立体视觉的目标识别与定位

双目立体视觉系统可以实现对目标的识别与定位.此系统包含摄像机标定、图像分割、立体匹配和三维测距4个模块.在摄像机标定部分,提出了基于云台转角的外参数估计方法.该方法可以精确完成摄像头旋转情况下外参的估计,增强了机器人的视觉功能.并利用广茂达机器人系统,基于改进的双目视觉系统进行目标识别与定位,以此结果作为依据控制机器人的手臂进行相应运动,最终实现了对目标物体的抓取,验证了提出方法的可行性.     

Binocular motion tracking by gaze fixation control and three-dimensional shape reconstruction

It is an easy task for the human visual system to gaze continuously at an object moving in three-dimensional (3-D) space. While tracking the object, human vision seems able to comprehend its 3-D shape with binocular vision. We conjecture that, in the human visual system, the function of comprehending the 3-D shape is essential for robust tracking of a moving object. In order to examine this conjecture, we constructed an experimental system of binocular vision for motion tracking. The system is composed of a pair of active pan-tilt cameras and a robot arm. The cameras are for simulating the two eyes of a human while the robot arm is for simulating the motion of the human body below the neck. The two active cameras are controlled so as to fix their gaze at a particular point on an object surface. The shape of the object surface around the point is reconstructed in real-time from the two images taken by the cameras based on the differences in the image brightness. If the two cameras successfully gaze at a single point on the object surface, it is possible to reconstruct the local object shape in real-time. At the same time, the reconstructed shape is used for keeping a fixation point on the object surface for gazing, which enables robust tracking of the object. Thus these two processes, reconstruction of the 3-D shape and maintaining the fixation point, must be mutually connected and form one closed loop. We demonstrate the effectiveness of this framework for visual tracking through several experiments.     

基于PnP问题求解的摄像机标定

三维人脸识别中,双目成象系统是初始工作中极为关键的一步.本文引入了PnP问题讨论双目成象系统中摄像机的标定算法.本文提出解出P4P问题的唯一解,即确定摄像机的标定,该方法简单快捷.最后本文给出具体标定过程.     

Fully Vision-based Calibration of a Hand-Eye Robot

This article is concerned with calibrating an anthropomorphic two-armed robot equipped with a stereo-camera vision system, that is estimating the different geometric relationships involved in the model of the robot. The calibration procedure that is presented is fully vision-based: the relationships between each camera and the neck and between each arm and the neck are determined using visual measurements. The online calculation of all the relationships involved in the model of the robot is obtained with satisfactory precision and, above all, without expensive calibration mechanisms. For this purpose, two new main algorithms have been developed. The first one implements a non-linear optimization method using quaternions for camera calibration from 2D to 3D point or line correspondences. The second one implements a real-time camera pose estimation method based on the iterative use of a paraperspective camera model.     

基于显著性检测的双目测距系统

为了提高双目视觉测距系统中图像匹配的实时性与测距的精度,提出一种将显著性检测与焦距拟合相结合的双目测距方法。首先对双目相机进行畸变矫正,并利用双目相机成像的特点拟合相机焦距与目标距离的关系,随后对所得图像进行显著性检测,并提取目标区域,最后,利用surf算子对提取出的区域进行特征匹配,将匹配点代入测距模型中得到目标物体的距离。结果表明:显著性检测方法明显提升算法执行速度,焦距拟合降低双目测距模型误差,明显提升双目测距精度。