Real-time appearance-based Monte Carlo localization

A new technique for vision processing is presented which lets a mobile robot equipped with an omnidirectional camera perform appearance-based global localization in real time. The technique is applied directly to the omnidirectional camera images, producing low-dimensional rotation invariant feature vectors without any training or set-up phase. Using the feature vectors, particle filters can accurately estimate the location of a continuously moving real robot, processing 5000 simultaneous localization hypotheses on-line. Estimated body positions overlap the actual ones in over 95% of the time steps. The feature vectors show a graceful degradation against increasing levels of simulated noise and occlusion.     

Accurate appearance-based Bayesian tracking for maneuvering targets

We propose a tracking algorithm that combines the Mean Shift search in a Particle Filtering framework and a target representation that uses multiple semi-overlapping color histograms. The target representation introduces spatial information that accounts for rotation and anisotropic scaling without compromising the flexibility typical of color histograms. Moreover, the proposed tracker can generate a smaller number of samples than Particle Filter as it increases the particle efficiency by moving the samples toward close local maxima of the likelihood using Mean Shift. Experimental results show that the proposed representation improves the robustness to clutter and that, especially on highly maneuvering targets, the combined tracker outperforms Particle Filter and Mean Shift in terms of accuracy in estimating the target size and position while generating only 25% of the samples used by Particle Filter.     

视线追踪系统眼部特征检测及视线方向计算方法研究

基于单视觉主动红外光源系统,提出了一种视线检测方法．在眼部特征检测阶段,采用投影法定位人脸;根据人脸对称性和五官分布的先验知识,确定瞳孔潜在区域;最后进行人眼特征的精确分割．在视线方向建模阶段,首先在头部静止的情况下采用非线性多项式建立从平面视线参数到视线落点的映射模型;然后采用广义回归神经网络对不同头部位置造成的视线偏差进行补偿,使非线性映射函数扩展到任何头部位置．实验结果及在交互式图形界面系统中的应用验证了该方法的有效性．     

视线追踪系统中视线估计方法研究

文中提出一种基于瞳孔—角膜反射(PCCR)的视线估计方法(GEMHSSO).针对现有PCCR存在的主要问题:限制使用者头部运动和个体标定问题,提出了一种单相机单光源条件下头部位置的补偿方法,实现了头部位置变化对瞳孔角膜向量影响的解析补偿,并建立一种个体差异的转化模型,进而简化标定过程为单点标定.以此为基础形成一种新的视线估计方法,本方法使精确视线估计的最小硬件要求降低到单相机(未标定)单光源,既不需要繁杂的系统标定,又实现了自然头动视线估计,并且简化用户标定为单点标定.该方法的各个环节都满足实时性要求,为面向人机交互的视线追踪系统提供了有效的解决方案.     

改进压缩特征的实时压缩跟踪算法

提出了一种改进的实时压缩跟踪算法（RCT）。该算法基于实时压缩跟踪算法,构造出一个改进的随机测量矩阵,使降维后得到的压缩特征包含的灰度特征信息和纹理特征信息比例相等。RCT算法首先将图像序列的特征用改进的随机测量矩阵转化为低维度特征,再用朴素贝叶斯分类器对低维特征进行目标和背景的分类,从而实现对目标的跟踪。将原始算法（CT）、一种改进算法（BCT）和该文创新的改进算法（RCT）进行对比,实验表明：RCT算法保持了原始算法的实时性,并且在各实验图像序列中跟踪目标的鲁棒性最好。     

A video-based door monitoring system using local appearance-based face models

In this paper, we present a real-time video-based face recognition system. The developed system identifies subjects while they are entering a room. This application scenario poses many challenges. Continuous, uncontrolled variations of facial appearance due to illumination, pose, expression, and occlusion of non-cooperative subjects need to be handled to allow for successful recognition. In order to achieve this, the system first detects and tracks the eyes for proper registration. The registered faces are then individually classified by a local appearance-based face recognition algorithm. The obtained confidence scores from each classification are progressively combined to provide the identity estimate of the entire sequence. We introduce three different measures to weight the contribution of each individual frame to the overall classification decision. They are distance-to-model (DTM), distance-to-second-closest (DT2ND), and their combination. We have conducted closed-set and open-set identification experiments on a database of 41 subjects. The experimental results show that the proposed system is able to reach high correct recognition rates. Besides, it is able to perform facial feature and face detection, tracking, and recognition in real-time.     

A Bayesian approach to object detection using probabilistic appearance-based models

In this paper, we introduce a Bayesian approach, inspired by probabilistic principal component analysis (PPCA) (Tipping and Bishop in J Royal Stat Soc Ser B 61(3):611–622, 1999), to detect objects in complex scenes using appearance-based models. The originality of the proposed framework is to explicitly take into account general forms of the underlying distributions, both for the in-eigenspace distribution and for the observation model. The approach combines linear data reduction techniques (to preserve computational efficiency), non-linear constraints on the in-eigenspace distribution (to model complex variabilities) and non-linear (robust) observation models (to cope with clutter, outliers and occlusions). The resulting statistical representation generalises most existing PCA-based models (Tipping and Bishop in J Royal Stat Soc Ser B 61(3):611–622, 1999; Black and Jepson in Int J Comput Vis 26(1):63–84, 1998; Moghaddam and Pentland in IEEE Trans Pattern Anal Machine Intell 19(7):696–710, 1997) and leads to the definition of a new family of non-linear probabilistic detectors. The performance of the approach is assessed using receiver operating characteristic (ROC) analysis on several representative databases, showing a major improvement in detection performances with respect to the standard methods that have been the references up to now.This revised version was published online in November 2004 with corrections to the section numbers.     

Eye and gaze tracking for interactive graphic display

This paper describes a computer vision system based on active IR illumination for real-time gaze tracking for interactive graphic display. Unlike most of the existing gaze tracking techniques, which often require assuming a static head to work well and require a cumbersome calibration process for each person, our gaze tracker can perform robust and accurate gaze estimation without calibration and under rather significant head movement. This is made possible by a new gaze calibration procedure that identifies the mapping from pupil parameters to screen coordinates using generalized regression neural networks (GRNNs). With GRNNs, the mapping does not have to be an analytical function and head movement is explicitly accounted for by the gaze mapping function. Furthermore, the mapping function can generalize to other individuals not used in the training. To further improve the gaze estimation accuracy, we employ a hierarchical classification scheme that deals with the classes that tend to be misclassified. This leads to a improvement in classification error. The angular gaze accuracy is about horizontally and vertically. The effectiveness of our gaze tracker is demonstrated by experiments that involve gaze-contingent interactive graphic display.Received: 21 July 2002, Accepted: 3 February 2004, Published online: 8 June 2004 Correspondence to: Qiang Ji     

融合检测和跟踪的实时人脸跟踪

目的 在实时人脸跟踪过程中,因光照变化、目标被遮挡以及跟踪时间长等因素,导致的误差累积都会影响系统的整体性能。针对这些问题,提出一种融合检测和跟踪技术的方法,其中包含了检测、控制和跟踪3个模块(简称DCT)。方法 在检测模块中,利用AdaBoost算法提取人脸的相关信息,并将信息传递给跟踪模块进行跟踪处理;在跟踪模块中,采用在线随机蕨和SURF(speeded up robust features)算法对目标进行跟踪。同时,在每次检测到目标之后,会通过控制模块对当前跟踪目标准确性进行判断。结果 选取国际标准数据组并与LBP+Camshift+Kalman滤波算法、SEMI算法、TLD(tracking-learning-detection)算法比较,实验结果表明,DCT方法在目标发生尺度较大变化、目标遮挡、旋转、形变以及光照发生变化时都具有良好的跟踪识别效果,DCT方法识别准确率在95%以上,平均误识别率和漏识别率分别为0.86%和0.78%。结论 DCT方法具有消除误差累积,跟踪失败后自动恢复等特点,同时可以消除环境中光照、遮挡和仿射变换的影响并满足系统跟踪的实时性要求,运用于视频人脸跟踪系统中能够提高系统的实时性及鲁棒性。     

Real-time vision-based tracking and reconstruction

Many of the recent real-time markerless camera tracking systems assume the existence of a complete 3D model of the target scene. Also the system developed in the MATRIS project assumes that a scene model is available. This can be a freeform surface model generated automatically from an image sequence using structure from motion techniques or a textured CAD model built manually using a commercial software. The offline model provides 3D anchors to the tracking. These are stable natural landmarks, which are not updated and thus prevent an accumulating error (drift) in the camera registration by giving an absolute reference. However, sometimes it is not feasible to model the entire target scene in advance, e.g. parts, which are not static, or one would like to employ existing CAD models, which are not complete. In order to allow camera movements beyond the parts of the environment modelled in advance it is desired to derive additional 3D information online. Therefore, a markerless camera tracking system for calibrated perspective cameras has been developed, which employs 3D information about the target scene and complements this knowledge online by reconstruction of 3D points. The proposed algorithm is robust and reduces drift, the most dominant problem of simultaneous localisation and mapping (SLAM), in real-time by a combination of the following crucial points: (1) stable tracking of longterm features on the 2D level; (2) use of robust methods like the well-known Random Sampling Consensus (RANSAC) for all 3D estimation processes; (3) consequent propagation of errors and uncertainties; (4) careful feature selection and map management; (5) incorporation of epipolar constraints into the pose estimation. Validation results on the operation of the system on synthetic and real data are presented.     

Real-time tracking using trust-region methods

Optimization methods based on iterative schemes can be divided into two classes: line-search methods and trust-region methods. While line-search techniques are commonly found in various vision applications, not much attention is paid to trust-region ones. Motivated by the fact that line-search methods can be considered as special cases of trust-region methods, we propose to establish a trust-region framework for real-time tracking. Our approach is characterized by three key contributions. First, since a trust-region tracking system is more effective, it often yields better performances than the outcomes of other trackers that rely on iterative optimization to perform tracking, e.g., a line-search-based mean-shift tracker. Second, we have formulated a representation model that uses two coupled weighting schemes derived from the covariance ellipse to integrate an object's color probability distribution and edge density information. As a result, the system can address rotation and nonuniform scaling in a continuous space, rather than working on some presumably possible discrete values of rotation angle and scale. Third, the framework is very flexible in that a variety of distance functions can be adapted easily. Experimental results and comparative studies are provided to demonstrate the efficiency of the proposed method.     

Pupil detection and gaze tracking using a deformable template

Multimedia Tools and Applications - This paper suggests a method for tracking gaze of a person at a distance around 2 m, using a single pan-tilt-zoom (PTZ) camera. In the suggested method,...     

视线追踪系统中特征参数提取方法研究

为了提取人眼的高精度亚像素特征参数,利用亮瞳现象,提出了一种基于多通道图像的高精度亚像素特征参数提取方法。该方法首先通过差分图像滤波获得瞳孔区域,进而检测瞳孔区域的边缘,并在眼睛区域附近基于灰度,搜索角膜反射区域; 然后求取其质心用于定位角膜反射区域中心,并对瞳孔边缘做滤波,以消除角膜反射对瞳孔边缘轮廓的影响,进而利用椭圆拟合来定位瞳孔中心;最后提取包括人眼特征和人脸位置的多个参数,另外,还建立了一个多特征参数提取的流程,为下一步的视线估计提供了参数依据。 实验结果及视线追踪系统最终的视线估计结果证明,该方法是有效的。     

人体自由运动状态下的视线追踪算法研究*

针对人体在大空间范围内自由运动时视线方向难以追踪的问题,构建了一套基于光学跟踪设备的头戴式视线追踪系统。系统通过被动式光学追踪设备和头戴式眼部摄像机获取使用者的头部运动状态与眼部图像,然后依据初始标定结果来估计使用者自由运动状态下的视线方向;最后对系统进行简化,得到了适用于同类环境、与具体硬件设备无关的视线跟踪三点三面三变换几何模型。对系统进行应用实验和误差分析表明,使用者在3.0 * 3.2 * 2.0 m的大工作空间内自由运动时视线追踪误差为1.69度,频率为20赫兹。     

Improving the robustness of gaze tracking under unconstrained illumination conditions

Multimedia Tools and Applications - In human-computer interaction (HCI) applications, the performance degradation of gaze trackers in real-world environments is a critical issue. Typically, gaze...     

Perceptual self-position estimation based on gaze tracking in virtual reality

Virtual Reality - The depth perception of human visual system is divergent between virtual and real space; this depth discrepancy affects the spatial judgment of the user in a virtual space, which...     

Eye gaze tracking using corneal imaging and active illumination devices

This paper shows a novel eye gaze tracking (EGT) technique using the corneal imaging technique. Compared to the existing pupil center and pupil reflection techniques, our approach directly finds PoG in the reflected scene image at the human corneal surface. As a result, it does not suffer from the parallax issue and does not require per-setup system calibrations. To achieve this system, we develop following techniques: First, we use the idea of the gaze-reflection point (GRP), where light from the PoG in the scene reflects at the corneal surface into an eye image. Second, illuminating the whole scene or particular objects using coded structured light enables robust and accurate matching at the GRP to obtain the PoG in a scene image. For this purpose, we show two implementations: a special high-power IR LED-array projector and active LED markers. Experimental evaluation shows that the proposed scheme achieves considerable accuracy and successfully supports depth-varying environments as well as practical applications including observation in a conversation scene. We believe the proposed EGT technique has considerably large potential to solve major issues of current EGT systems, expands the application fields of the EGT and increases the usability of interactive systems.     

Eye gaze tracking based directional control interface for interactive applications

This paper proposes an unobtrusive and calibration-free framework towards eye gaze tracking based interactive directional control interface for desktop environment using simple webcam under unconstrained settings. The proposed eye gaze tracking involved hybrid approach designed by combining two different techniques based upon both supervised and unsupervised methods wherein the unsupervised image gradients method computes the iris centers over the eye regions extracted by the supervised regression based algorithm. Experiments performed by the proposed hybrid approach to detect eye regions along with iris centers over challenging face image datasets exhibited exciting results. Similar approach for eye gaze tracking worked well in real-time by using a simple web camera. Further, PC based interactive directional control interface based upon iris position has been designed that works without needing any prior calibrations unlike other Infrared illumination based eye trackers.

The proposed work may be useful to the people with full body motor disabilities, who need interactive and unobtrusive eye gaze control based applications to live independently.