A robust eye gaze tracking method based on a virtual eyeball model

Gaze positions can provide important cues for natural computer interfaces. In this paper, we describe a new gaze estimation method based on a three dimensional analysis of the human eye which can be used in head-mounted display (HMD) environments. This paper presents four advantages over previous works. First, in order to obtain accurate gaze positions, we used a virtual eyeball model based on the 3D characteristics of the human eyeball. Second, we calculated the 3D position of the virtual eyeball and gaze vector by using a camera and three collimated IR-LEDs. Third, three reference frames (the camera, the monitor and the eye reference frames) were unified, which simplified the complex 3D converting calculations and allowed for calculation of the 3D eye position and gaze position on a HMD monitor. Fourth, a simple user-dependent calibration method was proposed by gazing at one position based on Kappa compensation. Experimental results showed that the eye gaze estimation error of the proposed method was lower than 1°.

---

Study on eye gaze estimation

There are two components to the human visual line-of-sight: pose of human head and the orientation of the eye within their sockets. We have investigated these two aspects but will concentrate on eye gaze estimation. We present a novel approach called the "one-circle" algorithm for measuring the eye gaze using a monocular image that zooms in on only one eye of a person. Observing that the iris contour is a circle, we estimate the normal direction of this iris circle, considered as the eye gaze, from its elliptical image. From basic projective geometry, an ellipse can be back-projected into space onto two circles of different orientations. However, by using a geometric constraint, namely, that the distance between the eyeball's center and the two eye corners should be equal to each other, the correct solution can be disambiguated. This allows us to obtain a higher resolution image of the iris with a zoom-in camera, thereby achieving higher accuracies in the estimation. A general approach that combines head pose determination with eye gaze estimation is also proposed. The searching of the eye gaze is guided by the head pose information. The robustness of our gaze determination approach was verified statistically by the extensive experiments on synthetic and real image data. The two key contributions are that we show the possibility of finding the unique eye gaze direction from a single image of one eye and that one can obtain better accuracy as a consequence of this.     

Real-time estimation system of the gaze position based on an electrooculogram

This article describes an online eye-tracking method based on an electrooculogram (EOG) to estimate gaze position. The objective is to use a biomedical signal EOG as the input of a human-machine interface for both disabled and healthy people. In this study, features of horizontal and vertical EOG signals were extracted to estimate the gaze position. Compensation for time-shift and nonlinearity were applied to improve the performance of the estimation. The estimated locus of the moving eye was compared with the locus of the target, and the deviation was about 3 cm. The results can be applied in some raw tracking fields, such as online communication, EOG-based pointers, and so on.     

Collaborative eye tracking based code review through real-time shared gaze visualization

Code review is intended to find bugs in early development phases, improving code quality for later integration and testing. However, due to the lack of experience with algorithm design, or software development, individual novice programmers face challenges while reviewing code. In this paper, we utilize collaborative eye tracking to record the gaze data from multiple reviewers, and share the gaze visualization among them during the code review process. The visualizations, such as borders highlighting current reviewed code lines, transition lines connecting related reviewed code lines, reveal the visual attention about program functions that can facilitate understanding and bug tracing. This can help novice reviewers to make sense to confirm the potential bugs or avoid repeated reviewing of code, and potentially even help to improve reviewing skills. We built a prototype system, and conducted a user study with paired reviewers. The results showed that the shared real-time visualization allowed the reviewers to find bugs more efficiently.     

基于单幅图像的视线计算研究

提出了一种新的基于单幅眼睛图像的视线计算方法。通过假定两眼的视线向量以及左右虹膜中心点组成的向量共面,可以计算得到两眼的视线方向。与现有的三维视线计算方法相比,提出的视线计算方法不需要眼角坐标等脸部特征信息,另外也不需要已知相机的焦距大小。可以允许相机的焦距大小以及与用户的距离自由改变。最后实验的结果也比较理想,充分证明了提出的视线计算方法的可行性。     

A neural network based real-time gaze tracker

A real-time gaze-tracking system that estimates the user's eye gaze and computes the window of focused view on a computer monitor has been developed. This artificial neural network based system can be trained and customized for an individual. Unlike existing systems in which skin color features and/or other mountable equipment are needed, this system is based on a simple non-intrusive camera mounted on the monitor. Gaze point is accurately estimated within a 1 in. on a 19-in. monitor with a CCD camera having a 640×480 image resolution. The system performance is independent of user's forward and backward as well as upward and downward movements. The gaze-tracking system implementation and the factors affecting its performance are discussed and analyzed in detail. The features and implementation methods that make this system real-time are also explained.     

An auxiliary gaze point estimation method based on facial normal

Considering the main disadvantage of the existing gaze point estimation methods which restrict user’s head movement and have potential injury on eyes, we propose a gaze point estimation method based on facial normal and binocular vision. Firstly, we calibrate stereo cameras to determine the extrinsic and intrinsic parameters of the cameras; Secondly, face is quickly detected by Viola–Jones framework and the center position of the two irises can be located based on integro-differential operators; The two nostrils and mouth are detected based on the saturation difference and their 2D coordinates can be calculated; Thirdly, the 3D coordinates of these five points are obtained by stereo matching and 3D reconstruction; After that, a plane fitting algorithm based on least squares is adopted to get the approximate facial plane, then, the normal via the midpoint of the two pupils can be figured out; Finally, the point-of-gaze can be obtained by getting the intersection point of the facial normal and the computer screen. Experimental results confirm the accuracy and robustness of the proposed method.     

Parameter estimation of 2-D stochastic FM model based on two step estimation procedure

This paper considers the problem of estimating the parameters of two-dimensional (2-D) stochastic FM models. First, the estimators of the parameters of 2-D stochastic FM model are obtained using two step estimation procedures by Kronecker product and least square method. Then, the asymptotic properties of the estimators are given. It is shown that the estimators are to be consistent and to have a distribution which converges to that of a normally distributed random vector under fairly general conditions. Finally, the performance of the proposed methods is illustrated by examples.     

Robust and real-time torsional eye position calculation using a template-matching technique

Computation of eye rotation about the line of sight (torsion) using image processing techniques has traditionally used cross-correlation of iral signatures sampled from circular arcs centered on the pupil. We have developed a new algorithm that utilizes a template-matching technique to calculate torsional eye position. Iral signatures are obtained from two annuli centered on the pupil center. By assuming that torsional rotation of the eye is constrained between successive video frames (<2 degrees), only a small window of the previous reference signature is necessary to determine relative torsional eye displacement. This dramatically reduces the number of pixels needed for computing torsion. This algorithm is considerably faster, attains a higher accuracy, and exhibits considerably less noise than the cross-correlation technique. Running on a 800 MHz Intel-based Dual Processor Pentium III, with a Matrox frame grabber, the system is capable of processing three-dimensional eye position at a rate of 120 frames/s.     

3-D face recognition method based on optimum 3-D image measurement technology

Most face recognition systems employ 2-D color or gray-scale images. However, face recognition based on 2-D images is adversely affected by 3-D movement, variable lighting, and the use of cosmetics. 3-D image measurement technology has the potential to overcome these limitations of face recognition based on 2-D images since it can perform geometric analysis. We propose a method that is capable of recognizing a person from a 3-D facial image obtained using a 3-D shape measurement system by employing a technique that optimizes the intensity-modulation pattern projection. This face recognition method is based on the iterative closest point algorithm. It is robust to changes in reflectivity and color. Since the 3-D facial information can be registered, this method can estimate rotations and translations to compensate for different positions or directions. In order to prove the validity of the proposed technique, a verification experiment was conducted which used 105 sample 3-D images obtained from 15 subjects. It achieved a detection rate of 96% when heads were turned at an angle of 20° or less relative to the camera.     

A novel approach to 3-D gaze tracking using stereo cameras

A novel approach to three-dimensional (3-D) gaze tracking using 3-D computer vision techniques is proposed in this paper. This method employs multiple cameras and multiple point light sources to estimate the optical axis of user's eye without using any user-dependent parameters. Thus, it renders the inconvenient system calibration process which may produce possible calibration errors unnecessary. A real-time 3-D gaze tracking system has been developed which can provide 30 gaze measurements per second. Moreover, a simple and accurate calibration method is proposed to calibrate the gaze tracking system. Before using the system, each user only has to stare at a target point for a few (2-3) seconds so that the constant angle between the 3-D line of sight and the optical axis can be estimated. The test results of six subjects showed that the gaze tracking system is very promising achieving an average estimation error of under 1 degrees.     

The possibility of isolated target 3-D position estimation and optimal receiver position determination in SS-BSAR

Starting from the generalized ambiguity function of bistatic SAR （BSAR）, it is shown that 3-D point target estimation can be carried out in space-surface bistatic SAR （SS-BSAR）. Appropriate analytical equations, based on maximum likelihood estimation （MLE）, are derived and confirmed via computer simulation. Furthermore, the performance of the estimate using the Crammer-Rao bound is analyzed for the case in question, thus further revealing the possibility and potential of target 3-D position estimation. Setting the determinant maximum of the information matrix as the criterion, the optimal receiver position and multi-receiver configuration are analytically determined in the SS-BSAR system. Simulation results also validate the correctness of the analytical calculation.     

基于AAM和T型结构的人脸3D姿态估计

在分析已有的人脸姿态估计方法基础上,提出了一种基于主动表观模型(AAM)和T型结构的人脸3D姿态估计方法。对多姿态的人脸样本进行训练,得到多姿态的AAM模板集;利用训练得到的多姿态的AAM模板集进行最佳模板匹配,并对人脸的特征点进行精确定位;用人脸的双眼和嘴部构建T型模型,进行人脸3D姿态的参数估计。实验结果表明,该方法能适应较大的姿态旋转角度,并具有良好的姿态估计精度。     

Perceptual self-position estimation based on gaze tracking in virtual reality

Virtual Reality - The depth perception of human visual system is divergent between virtual and real space; this depth discrepancy affects the spatial judgment of the user in a virtual space, which...     

3-D reconstruction using mirror images based on a plane symmetryrecovering method

Three-dimensional reconstruction from a perspective 2D image using mirrors is addressed. The mirrors are used to form symmetrical relations between the direct image and mirror images. By finding correspondences between them, the 3D shape can be reconstructed by means of plane symmetry recovering method using the vanishing point. Two constraints are used in determining the correspondence. In the case where only one mirror is used, invisible parts both in the direct image and in the mirror image may still remain. Using multiple mirrors, however, occluded parts will decrease or disappear, and occlusion-free object reconstruction becomes possible     

基于形态学算子的磨粒三维重构模型

利用图像处理技术,结合显微镜焦平面成像原理,提出了在普通光学显微镜下对磨粒进行三维表面快速重构的方法,使磨粒的三维测量不再受昂贵的三维测量仪器在功能上和成本上的限制。从采集视频中提取帧图像,应用梯度算子进行滤波处理划分得到焦平面范围,同时,提出结合数学形态学方法在小范围内消除焦平面区域的不连续性,扩展焦平面范围,使得不能检测到的焦平面得到较好的处理。通过布尔代数运算将不同的显微镜焦平面图像重构成一幅完整的磨粒图像。该方法可以弥补电子扫描电镜难以得到物体表面的颜色的不足,同时相比较于激光共焦显微镜测量三维表面的方法更为简单且更容易得到磨粒表面的真实图像色彩。     

Comparitive study on photometric normalization algorithms for an innovative, robust and real-time eye gaze tracker

Eye gaze trackers (EGTs) are generally developed for scientific exploration in controlled environments or laboratories and data have been used in ophthalmology, neurology, psychology, and related areas to study oculomotor characteristics and abnormalities, and their relation to cognition and mental states. The illumination is one of the most restrictive limitation of the EGTs, due to a problem of pupil center estimation during illumination changes. Most of the current systems, indeed, work under controlled illumination conditions either in dark or indoor environments, e.g. using infrared sources or conforming the sources of light to fixed levels or pointing directions. This work is focused on exploring and comparing several photometric normalization techniques to improve EGT systems during light changes. In particular, a new wearable and wireless eye tracking system (HATCAM) is used for testing the different techniques in terms of real-time capability, eye tracking and pupil area detection. Embedding real-time image enhancement into the HATCAM can make it an innovative and robust system for eye tracking in different lighting conditions, i.e. darkness, sunlight, indoor and outdoor environments.     

A new data processing method based on a biological model of thecompound eye: direction quantization representation

This paper presents a new data representation method called direction quantization representation (DQR) which is motivated by a simplified geometric model of biological compound eye and used in describing the shape of convex hulls of objects. Advantages of DQR include high efficiency and stability in numerical computation, convenience for semidynamic maintenance, suitability for parallel implementation, and applicability to various convex set related problems. Several practical applications are presented which show the feasibility and powerfulness of DQR     

一种基于改进PointNet++网络的三维手姿估计方法

针对PointNet++网络处理点云局部特征时因分组范围区过大导致计算量较大的问题,提出一种改进的PointNet++网络的三维手姿估计方法。首先对手势点云进行基于Delaunay三角剖分算法与K中位数聚类算法相结合的三角剖分,得到手势点云的三角网格模型,并计算三角网格模型的边长均值;然后以三角网格模型边长均值为半径,对最远点采样(FPS)的采样点进行球查询搜索,再根据搜索到的采样点个数极值对采样点云进行K近邻分组,并最终输入PointNet网络,完成三维手姿的位置估计。改进后的PointNet++网络可以根据不同的点云密度自动调整网络分组区域的局部提取点个数。实验结果表明,在不影响三维手姿估计精度的情况下,该方法提高了PointNet++网络的模型训练速度,并在三维手姿估计中可有效减少特征提取的计算量,使计算机能够更快地捕捉手姿状态。