视线追踪系统眼部特征检测及视线方向计算方法研究

基于单视觉主动红外光源系统,提出了一种视线检测方法．在眼部特征检测阶段,采用投影法定位人脸;根据人脸对称性和五官分布的先验知识,确定瞳孔潜在区域;最后进行人眼特征的精确分割．在视线方向建模阶段,首先在头部静止的情况下采用非线性多项式建立从平面视线参数到视线落点的映射模型;然后采用广义回归神经网络对不同头部位置造成的视线偏差进行补偿,使非线性映射函数扩展到任何头部位置．实验结果及在交互式图形界面系统中的应用验证了该方法的有效性．     

改进压缩特征的实时压缩跟踪算法

提出了一种改进的实时压缩跟踪算法（RCT）。该算法基于实时压缩跟踪算法,构造出一个改进的随机测量矩阵,使降维后得到的压缩特征包含的灰度特征信息和纹理特征信息比例相等。RCT算法首先将图像序列的特征用改进的随机测量矩阵转化为低维度特征,再用朴素贝叶斯分类器对低维特征进行目标和背景的分类,从而实现对目标的跟踪。将原始算法（CT）、一种改进算法（BCT）和该文创新的改进算法（RCT）进行对比,实验表明：RCT算法保持了原始算法的实时性,并且在各实验图像序列中跟踪目标的鲁棒性最好。     

A Bayesian approach to object detection using probabilistic appearance-based models

In this paper, we introduce a Bayesian approach, inspired by probabilistic principal component analysis (PPCA) (Tipping and Bishop in J Royal Stat Soc Ser B 61(3):611–622, 1999), to detect objects in complex scenes using appearance-based models. The originality of the proposed framework is to explicitly take into account general forms of the underlying distributions, both for the in-eigenspace distribution and for the observation model. The approach combines linear data reduction techniques (to preserve computational efficiency), non-linear constraints on the in-eigenspace distribution (to model complex variabilities) and non-linear (robust) observation models (to cope with clutter, outliers and occlusions). The resulting statistical representation generalises most existing PCA-based models (Tipping and Bishop in J Royal Stat Soc Ser B 61(3):611–622, 1999; Black and Jepson in Int J Comput Vis 26(1):63–84, 1998; Moghaddam and Pentland in IEEE Trans Pattern Anal Machine Intell 19(7):696–710, 1997) and leads to the definition of a new family of non-linear probabilistic detectors. The performance of the approach is assessed using receiver operating characteristic (ROC) analysis on several representative databases, showing a major improvement in detection performances with respect to the standard methods that have been the references up to now.This revised version was published online in November 2004 with corrections to the section numbers.     

A video-based door monitoring system using local appearance-based face models

In this paper, we present a real-time video-based face recognition system. The developed system identifies subjects while they are entering a room. This application scenario poses many challenges. Continuous, uncontrolled variations of facial appearance due to illumination, pose, expression, and occlusion of non-cooperative subjects need to be handled to allow for successful recognition. In order to achieve this, the system first detects and tracks the eyes for proper registration. The registered faces are then individually classified by a local appearance-based face recognition algorithm. The obtained confidence scores from each classification are progressively combined to provide the identity estimate of the entire sequence. We introduce three different measures to weight the contribution of each individual frame to the overall classification decision. They are distance-to-model (DTM), distance-to-second-closest (DT2ND), and their combination. We have conducted closed-set and open-set identification experiments on a database of 41 subjects. The experimental results show that the proposed system is able to reach high correct recognition rates. Besides, it is able to perform facial feature and face detection, tracking, and recognition in real-time.     

Eye and gaze tracking for interactive graphic display

This paper describes a computer vision system based on active IR illumination for real-time gaze tracking for interactive graphic display. Unlike most of the existing gaze tracking techniques, which often require assuming a static head to work well and require a cumbersome calibration process for each person, our gaze tracker can perform robust and accurate gaze estimation without calibration and under rather significant head movement. This is made possible by a new gaze calibration procedure that identifies the mapping from pupil parameters to screen coordinates using generalized regression neural networks (GRNNs). With GRNNs, the mapping does not have to be an analytical function and head movement is explicitly accounted for by the gaze mapping function. Furthermore, the mapping function can generalize to other individuals not used in the training. To further improve the gaze estimation accuracy, we employ a hierarchical classification scheme that deals with the classes that tend to be misclassified. This leads to a improvement in classification error. The angular gaze accuracy is about horizontally and vertically. The effectiveness of our gaze tracker is demonstrated by experiments that involve gaze-contingent interactive graphic display.Received: 21 July 2002, Accepted: 3 February 2004, Published online: 8 June 2004 Correspondence to: Qiang Ji     

Real-time vision-based tracking and reconstruction

Many of the recent real-time markerless camera tracking systems assume the existence of a complete 3D model of the target scene. Also the system developed in the MATRIS project assumes that a scene model is available. This can be a freeform surface model generated automatically from an image sequence using structure from motion techniques or a textured CAD model built manually using a commercial software. The offline model provides 3D anchors to the tracking. These are stable natural landmarks, which are not updated and thus prevent an accumulating error (drift) in the camera registration by giving an absolute reference. However, sometimes it is not feasible to model the entire target scene in advance, e.g. parts, which are not static, or one would like to employ existing CAD models, which are not complete. In order to allow camera movements beyond the parts of the environment modelled in advance it is desired to derive additional 3D information online. Therefore, a markerless camera tracking system for calibrated perspective cameras has been developed, which employs 3D information about the target scene and complements this knowledge online by reconstruction of 3D points. The proposed algorithm is robust and reduces drift, the most dominant problem of simultaneous localisation and mapping (SLAM), in real-time by a combination of the following crucial points: (1) stable tracking of longterm features on the 2D level; (2) use of robust methods like the well-known Random Sampling Consensus (RANSAC) for all 3D estimation processes; (3) consequent propagation of errors and uncertainties; (4) careful feature selection and map management; (5) incorporation of epipolar constraints into the pose estimation. Validation results on the operation of the system on synthetic and real data are presented.     

Real-time tracking using trust-region methods

Optimization methods based on iterative schemes can be divided into two classes: line-search methods and trust-region methods. While line-search techniques are commonly found in various vision applications, not much attention is paid to trust-region ones. Motivated by the fact that line-search methods can be considered as special cases of trust-region methods, we propose to establish a trust-region framework for real-time tracking. Our approach is characterized by three key contributions. First, since a trust-region tracking system is more effective, it often yields better performances than the outcomes of other trackers that rely on iterative optimization to perform tracking, e.g., a line-search-based mean-shift tracker. Second, we have formulated a representation model that uses two coupled weighting schemes derived from the covariance ellipse to integrate an object's color probability distribution and edge density information. As a result, the system can address rotation and nonuniform scaling in a continuous space, rather than working on some presumably possible discrete values of rotation angle and scale. Third, the framework is very flexible in that a variety of distance functions can be adapted easily. Experimental results and comparative studies are provided to demonstrate the efficiency of the proposed method.     

Pupil detection and gaze tracking using a deformable template

Multimedia Tools and Applications - This paper suggests a method for tracking gaze of a person at a distance around 2 m, using a single pan-tilt-zoom (PTZ) camera. In the suggested method,...     

Real-time shape tracking of facial landmarks

Detection of facial landmarks and accurate tracking of their shape are essential in real-time applications such as virtual makeup, where users can see the makeup’s effect by moving their face in diverse directions. Typical face tracking techniques detect facial landmarks and track them using a point tracker such as the Kanade-Lucas-Tomasi (KLT) point tracker. Typically, 5 or 64 points are used for tracking a face. Even though these points are enough to track the approximate locations of facial landmarks, they are not sufficient to track the exact shape of facial landmarks. In this paper, we propose a method that can track the exact shape of facial landmarks in real-time by combining a deep learning technique and a point tracker. We detect facial landmarks accurately using SegNet, which performs semantic segmentation based on deep learning. Edge points of detected landmarks are tracked using the KLT point tracker. In spite of its popularity, the KLT point tracker suffers from the point loss problem. We solve this problem by executing SegNet periodically to recalculate the shape of facial landmarks. That is, by combining the two techniques, we can avoid the computational overhead of SegNet and the point loss problem of the KLT point tracker, which leads to accurate real-time shape tracking. We performed several experiments to evaluate the performance of our method and report some of the results herein.

     

Real-time fingertip tracking and gesture recognition

Augmented desk interfaces and other virtual reality systems depend on accurate, real-time hand and fingertip tracking for seamless integration between real objects and associated digital information. We introduce a method for discerning fingertip locations in image frames and measuring fingertip trajectories across image frames. We also propose a mechanism for combining direct manipulation and symbolic gestures based on multiple fingertip motions. Our method uses a filtering technique, in addition to detecting fingertips in each image frame, to predict fingertip locations in successive image frames and to examine the correspondences between the predicted locations and detected fingertips. This lets us obtain multiple complex fingertip trajectories in real time and improves fingertip tracking. This method can track multiple fingertips reliably even on a complex background under changing lighting conditions without invasive devices or color markers.     

基于卷积神经网络的实时跟踪算法

利用卷积神经网络(CNN)强大的特征学习能力,提出了一种基于卷积神经网络的实时跟踪算法.通过对双通道卷积神经网络进行离线训练,学习相邻两帧之间的差异,得到跟踪目标的表观特征与运动之间的普遍规律.在不需要对网络模型在线更新的情况下,直接通过网络回归得到对目标的位置和对应置信度的预测.在VOT2014数据集中进行实验,结果表明:提出的跟踪算法的性能达到了当前领先水平.同时,跟踪算法的运行速度可以达到90帧/s,表现出非常不错的实时性.     

Real-time vehicle tracking for driving assistance

Detecting car taillights at night is a task which can nowadays be accomplished very fast on cheap hardware. We rely on such detections to build a vision-based system that, coupling them in a rule-based fashion, is able to detect and track vehicles. This allows the generation of an interface that informs a driver of the relative distance and velocity of other vehicles in real time and triggers a warning when a potentially dangerous situation arises. We demonstrate the system using sequences shot using a camera mounted behind a car’s windshield.     

Real-time visual tracking of complex structures

Presents a framework for three-dimensional model-based tracking. Graphical rendering technology is combined with constrained active contour tracking to create a robust wire-frame tracking system. It operates in real time at video frame rate (25 Hz) on standard hardware. It is based on an internal CAD model of the object to be tracked which is rendered using a binary space partition tree to perform hidden line removal. A Lie group formalism is used to cast the motion computation problem into simple geometric terms so that tracking becomes a simple optimization problem solved by means of iterative reweighted least squares. A visual servoing system constructed using this framework is presented together with results showing the accuracy of the tracker. The paper then describes how this tracking system has been extended to provide a general framework for tracking in complex configurations. The adjoint representation of the group is used to transform measurements into common coordinate frames. The constraints are then imposed by means of Lagrange multipliers. Results from a number of experiments performed using this framework are presented and discussed     

Improving the robustness of gaze tracking under unconstrained illumination conditions

Multimedia Tools and Applications - In human-computer interaction (HCI) applications, the performance degradation of gaze trackers in real-world environments is a critical issue. Typically, gaze...     

Perceptual self-position estimation based on gaze tracking in virtual reality

Virtual Reality - The depth perception of human visual system is divergent between virtual and real space; this depth discrepancy affects the spatial judgment of the user in a virtual space, which...     

Eye gaze tracking based directional control interface for interactive applications

This paper proposes an unobtrusive and calibration-free framework towards eye gaze tracking based interactive directional control interface for desktop environment using simple webcam under unconstrained settings. The proposed eye gaze tracking involved hybrid approach designed by combining two different techniques based upon both supervised and unsupervised methods wherein the unsupervised image gradients method computes the iris centers over the eye regions extracted by the supervised regression based algorithm. Experiments performed by the proposed hybrid approach to detect eye regions along with iris centers over challenging face image datasets exhibited exciting results. Similar approach for eye gaze tracking worked well in real-time by using a simple web camera. Further, PC based interactive directional control interface based upon iris position has been designed that works without needing any prior calibrations unlike other Infrared illumination based eye trackers.

The proposed work may be useful to the people with full body motor disabilities, who need interactive and unobtrusive eye gaze control based applications to live independently.

     

Real-time and robust hand tracking with a single depth camera

In this paper, we introduce a novel, real-time and robust hand tracking system, capable of tracking the articulated hand motion in full degrees of freedom (DOF) using a single depth camera. Unlike most previous systems, our system is able to initialize and recover from tracking loss automatically. This is achieved through an efficient two-stage k-nearest neighbor database searching method proposed in the paper. It is effective for searching from a pre-rendered database of small hand depth images, designed to provide good initial guesses for model based tracking. We also propose a robust objective function, and improve the Particle Swarm Optimization algorithm with a resampling based strategy in model based tracking. It provides continuous solutions in full DOF hand motion space more efficiently than previous methods. Our system runs at 40 fps on a GeForce GTX 580 GPU and experimental results show that the system outperforms the state-of-the-art model based hand tracking systems in terms of both speed and accuracy. The work result is of significance to various applications in the field of human–computer-interaction and virtual reality.