A Gaze‐enabled Graph Visualization to Improve Graph Reading Tasks

Performing typical network tasks such as node scanning and path tracing can be difficult in large and dense graphs. To alleviate this problem we use eye‐tracking as an interactive input to detect tasks that users intend to perform and then produce unobtrusive visual changes that support these tasks. First, we introduce a novel fovea based filtering that dims out edges with endpoints far removed from a user's view focus. Second, we highlight edges that are being traced at any given moment or have been the focus of recent attention. Third, we track recently viewed nodes and increase the saliency of their neighborhoods. All visual responses are unobtrusive and easily ignored to avoid unintentional distraction and to account for the imprecise and low‐resolution nature of eye‐tracking. We also introduce a novel gaze‐correction approach that relies on knowledge about the network layout to reduce eye‐tracking error. Finally, we present results from a controlled user study showing that our methods led to a statistically significant accuracy improvement in one of two network tasks and that our gaze‐correction algorithm enables more accurate eye‐tracking interaction.     

Iris segmentation in non-ideal images using graph cuts

A non-ideal iris image segmentation approach based on graph cuts is presented that uses both the appearance and eye geometry information. A texture measure based on gradients is computed to discriminate between eyelash and non-eyelash regions, combined with image intensity differences between the iris, pupil, and the background (region surrounding the iris) are utilized as cues for segmentation. The texture and intensity distributions for the various regions are learned from histogramming and explicit sampling of the pixels estimated to belong to the corresponding regions. The image is modeled as a Markov Random Field and the energy minimization is achieved via graph cuts to assign each image pixel one of the four possible labels: iris, pupil, background, and eyelash. Furthermore, the iris region is modeled as an ellipse, and the best fitting ellipse to the initial pixel based iris segmentation is computed to further refine the segmented region. As a result, the iris region mask and the parameterized iris shape form the outputs of the proposed approach that allow subsequent iris recognition steps to be performed for the segmented irises. The algorithm is unsupervised and can deal with non-ideality in the iris images due to out-of-plane rotation of the eye, iris occlusion by the eyelids and the eyelashes, multi-modal iris grayscale intensity distribution, and various illumination effects. The proposed segmentation approach is tested on several publicly available non-ideal near infra red (NIR) iris image databases. We compare both the segmentation error and the resulting recognition error with several leading techniques, demonstrating significantly improved results with the proposed technique.     

使用人眼几何特征的视线追踪方法

目的 视线追踪是人机交互的辅助系统,针对传统的虹膜定位方法误判率高且耗时较长的问题,本文提出了一种基于人眼几何特征的视线追踪方法,以提高在2维环境下视线追踪的准确率。方法 首先通过人脸定位算法定位人脸位置,使用人脸特征点检测的特征点定位眼角点位置,通过眼角点计算出人眼的位置。直接使用虹膜中心定位算法的耗时较长,为了使虹膜中心定位的速度加快,先利用虹膜图片建立虹膜模板,然后利用虹膜模板检测出虹膜区域的位置,通过虹膜中心精定位算法定位虹膜中心的位置,最后提取出眼角点、虹膜中心点等信息,对点中包含的角度信息、距离信息进行提取,组合成眼动向量特征。使用神经网络模型进行分类,建立注视点映射关系,实现视线的追踪。通过图像的预处理对图像进行增强,之后提取到了相对的虹膜中心。提取到需要的特征点,建立相对稳定的几何特征代表眼动特征。结果 在普通的实验光照环境中,头部姿态固定的情况下,识别率最高达到98.9%,平均识别率达到95.74%。而当头部姿态在限制区域内发生变化时,仍能保持较高的识别率,平均识别率达到了90%以上。通过实验分析发现,在头部变化的限制区域内,本文方法具有良好的鲁棒性。结论 本文提出使用模板匹配与虹膜精定位相结合的方法来快速定位虹膜中心,利用神经网络来对视线落点进行映射,计算视线落点区域,实验证明本文方法具有较高的精度。     

Study on eye gaze estimation

There are two components to the human visual line-of-sight: pose of human head and the orientation of the eye within their sockets. We have investigated these two aspects but will concentrate on eye gaze estimation. We present a novel approach called the "one-circle" algorithm for measuring the eye gaze using a monocular image that zooms in on only one eye of a person. Observing that the iris contour is a circle, we estimate the normal direction of this iris circle, considered as the eye gaze, from its elliptical image. From basic projective geometry, an ellipse can be back-projected into space onto two circles of different orientations. However, by using a geometric constraint, namely, that the distance between the eyeball's center and the two eye corners should be equal to each other, the correct solution can be disambiguated. This allows us to obtain a higher resolution image of the iris with a zoom-in camera, thereby achieving higher accuracies in the estimation. A general approach that combines head pose determination with eye gaze estimation is also proposed. The searching of the eye gaze is guided by the head pose information. The robustness of our gaze determination approach was verified statistically by the extensive experiments on synthetic and real image data. The two key contributions are that we show the possibility of finding the unique eye gaze direction from a single image of one eye and that one can obtain better accuracy as a consequence of this.     

多用户眼动跟踪数据的可视化共享与协同交互

随着数字图像处理技术的发展,以及计算机支持的协同工作研究的深入,眼动跟踪开始应用于多用户协同交互.但是已有的眼动跟踪技术主要针对单个用户,多用户眼动跟踪计算架构不成熟、标定过程复杂,眼动跟踪数据的记录、传输以及可视化共享机制都有待深入研究.为此,建立了基于梯度优化的协同标定模型,简化多用户的眼动跟踪标定过程;然后提出面向多用户的眼动跟踪计算架构,优化眼动跟踪数据的传输和管理.进一步地,探索眼动跟踪数据的可视化形式在协同交互环境下对用户视觉注意行为的影响,具体设计了圆点、散点、轨迹这3种可视化形式,并验证了圆点形式能够有效地提高多用户协同搜索任务的完成效率.在此基础上,设计与开发了基于眼动跟踪的代码协同审查系统,实现了代码审查过程中多用户眼动跟踪数据的同步记录、分发,以及基于实时注视点、代码行边框和背景灰度、代码行之间连线的可视化共享.用户实验结果表明,代码错误的平均搜索时间比没有眼动跟踪数据可视化分享时减少了20.1%,显著提高了协同工作效率,验证了该方法的有效性.     

轮廓初始点检测对眼睛虹膜区域反射光的去除

针对传统人眼反射光模型单一的缺陷,提出了一种自适应的对不规则形状人眼反射光的检测并去除算法。对含有人眼反射光区域,应用区域生长算法对人眼图像进行分割,并利用垂直边缘检测算子提取虹膜左右轮廓边界,拟合虹膜轮廓,对虹膜所在轮廓区域,采用提出的反射光轮廓初始点检测算法,反射光区域判定算法,对其轮廓的位置进行自动定位并提取,利用虹膜及瞳孔的纹理信息对人眼反射光覆盖区域进行修补。通过对不同人眼图像进行验证,结果表明该方法对检测不同噪声条件下的虹膜反射光具有鲁棒性,并且能够更加快速有效地对虹膜反射光进行去除。     

A new approach to the design of hybrid finer directional wavelet filter bank for iris feature extraction and classification using k-out-of-n:A post-classifier

This paper presents a novel iris feature extraction scheme using two-dimensional non-separable, non-redundant, multi-scale hybrid finer directional wavelet filter bank and classification using fused post-classifier under non-ideal environmental conditions. The proposed approach overcomes limited directionality of wavelet transform. The designed filter bank converts the wavelet basis functions to a set of directional basis elements by employing the combination of designed bi-orthogonal wavelet filter bank and modified checkerboard-shaped filter bank (designed using triplet of half-band filters). The false rejection rate is reduced with the help of k-out-of-n:A rule. The proposed technique is capable to handle possible artifacts especially inaccurate iris localization, occlusion of eyelids, pupil, eyelashes, reflection on iris, non-linear deformation, head-tilt, etc. The performance of the proposed scheme is validated using CASIA-Iris V2.0, MMU1, and UBIRIS databases. Finally, the performance of proposed approach is compared with other existing iris recognition algorithms to demonstrate its potential.     

A hybrid approach to urban land use/cover mapping using Landsat 7 Enhanced Thematic Mapper Plus (ETM+) images

A hybrid method that incorporates the advantages of supervised and unsupervised approaches as well as hard and soft classifications was proposed for mapping the land use/cover of the Atlanta metropolitan area using Landsat 7 Enhanced Thematic Mapper Plus (ETM+) data. The unsupervised ISODATA clustering method was initially used to segment the image into a large number of clusters of pixels. With reference to ground data based on 1?:?40?000 colour infrared aerial photographs in the form of Digital Orthophoto Quarter Quadrangle (DOQQ), homogeneous clusters were labelled. Clusters that could not be labelled because of mixed pixels were clipped out and subjected to a supervised fuzzy classification. A final land use/cover map was obtained by a union overlay of the two partial land use/cover maps. This map was evaluated by comparing with maps produced using unsupervised ISODATA clustering, supervised fuzzy and supervised maximum likelihood classification methods. It was found that the hybrid approach was slightly better than the unsupervised ISODATA clustering in land use/cover classification accuracy, most probably because of the supervised fuzzy classification, which effectively dealt with the mixed pixel problem in the low-density urban use category of land use/cover. It was suggested that this hybrid approach can be economically implemented in a standard image processing software package to produce land use/cover maps with higher accuracy from satellite images of moderate spatial resolution in a complex urban environment, where both discrete and continuous land cover elements occur side by side.     

A neural-based remote eye gaze tracker under natural head motion

A novel approach to view-based eye gaze tracking for human computer interface (HCI) is presented. The proposed method combines different techniques to address the problems of head motion, illumination and usability in the framework of low cost applications. Feature detection and tracking algorithms have been designed to obtain an automatic setup and strengthen the robustness to light conditions. An extensive analysis of neural solutions has been performed to deal with the non-linearity associated with gaze mapping under free-head conditions. No specific hardware, such as infrared illumination or high-resolution cameras, is needed, rather a simple commercial webcam working in visible light spectrum suffices. The system is able to classify the gaze direction of the user over a 15-zone graphical interface, with a success rate of 95% and a global accuracy of around 2 degrees , comparable with the vast majority of existing remote gaze trackers.     

Driver’s eye-based gaze tracking system by one-point calibration

The accuracies of driver’s gaze detection by previous researches are affected by the various sitting positions and heights of drivers in case that initial calibration of driver is not performed. By using dual cameras, the driver’s calibration can be omitted, but processing time with complexity is increased. In addition, the problem of disappearing corneal specular reflection (SR) in the eye image as the driver severely turns his/her head has not been dealt in previous researches. To consider these issues, we propose a gaze tracking method based on driver’s one-point calibration using both corneal SR and medial canthus (MC) based on maximum entropy criterion. An experiment with collected data from 26 subjects (wearing nothing, glasses, sunglasses, hat, or taking various hand pose) in a vehicle, showed that the accuracy of the proposed method is higher than that of other gaze tracking methods. In addition, we showed the effectiveness of our method in the real driving environment.

     

基于单幅图像的视线计算研究

提出了一种新的基于单幅眼睛图像的视线计算方法。通过假定两眼的视线向量以及左右虹膜中心点组成的向量共面,可以计算得到两眼的视线方向。与现有的三维视线计算方法相比,提出的视线计算方法不需要眼角坐标等脸部特征信息,另外也不需要已知相机的焦距大小。可以允许相机的焦距大小以及与用户的距离自由改变。最后实验的结果也比较理想,充分证明了提出的视线计算方法的可行性。     

A cascaded scheme for eye tracking and head movement compensation

The authors previously (1995) proposed an efficient method for tracking the eye movements. The proposed algorithm did not address the issue of compensating for head movements. Head movements are normally much slower than eye movements and can be compensated for using another tracking scheme for head position. In this paper, a hybrid method employing the two tracking schemes is developed. To this end, first a measurement model for the compensation of the head movement is formulated and then the overall tracking scheme is implemented by cascading two Kalman filters. The tracking of the iris movement is followed by the compensation of the head movement for each image frame. Experimental results are presented to demonstrate the accuracy aspects and the real-time applicability of the proposed approach     

Model-based approach to study of mechanisms of complex image viewing

A model-based approach to study complex image viewing mechanisms and the first results of its implementation are presented. The choice of the most informative regions (MIRs) is performed according to results of psychophysical tests with high-accuracy tracking of eye movements. For three test images, the MIRs were determined as image regions with maximal density of gaze fixations for the all subjects (n = 9). Individual image viewing scanpaths (n= 49) were classified into three basic types (i.e. “viewing”, “object-consequent”, and “object-returned” scanpaths). Task-related and temporal dynamics of eye movement parameters for the same subjects have been found. Artificial image scanpaths similar to experimental have been obtained by means of gaze attraction function. The article is published in the original.     

基于混合视线-脑机接口与共享控制的人-机器人交互系统

设计了一种基于混合视线-脑机接口与共享控制的人-机器人交互系统,以使得用户可通过视线和意念对机器人末端在2维空间进行连续的运动控制,并在避障和趋近目标的任务中获得机器智能的辅助.首先,按照用户运动意念的强度对机器人末端的运动速度大小进行等比例连续调节,以提高用户对机器人的控制感以及完成任务的参与性.然后,提出了机器人末端运动方向的一种共享控制策略,动态地融合基于视线追踪技术所得到的用户方向控制指令以及由机器人避障和趋近目标的行为设定所得到的机器人系统方向控制指令,自适应地调整机器人系统对用户的辅助力度,以减轻用户脑力负荷,提高任务完成成功率.最后,针对搭建的基于混合视线-脑机接口和共享控制的人-机器人交互平台,通过实验验证了所提系统的有效性.     

Neural Approximation-based Model Predictive Tracking Control of Non-holonomic Wheel-legged Robots

This paper proposes a neural approximation based model predictive control approach for tracking control of a nonholonomic wheel-legged robot in complex environments, which features mechanical model uncertainty and unknown disturbances. In order to guarantee the tracking performance of wheel-legged robots in an uncertain environment, effective approaches for reliable tracking control should be investigated with the consideration of the disturbances, including internal-robot friction and external physical interactions in the robot’s dynamical system. In this paper, a radial basis function neural network (RBFNN) approximation based model predictive controller (NMPC) is designed and employed to improve the tracking performance for nonholonomic wheel-legged robots. Some demonstrations using a BIT-NAZA robot are performed to illustrate the performance of the proposed hybrid control strategy. The results indicate that the proposed methodology can achieve promising tracking performance in terms of accuracy and stability.

     

Brain image segmentation using semi-supervised clustering

The objective of brain image segmentation is to partition the brain images into different non-overlapping homogeneous regions representing the different anatomical structures. Magnetic resonance brain image segmentation has large number of applications in diagnosis of neurological disorders like Alzheimer diseases, Parkinson related syndrome etc. But automatically segmenting the MR brain image is not an easy task. To solve this problem, several unsupervised and supervised based classification techniques have been developed in the literature. But supervised classification techniques are more time consuming and cost-sensitive due to the requirement of sufficient labeled data. In contrast, unsupervised classification techniques work without using any prior information but it suffers from the local trap problems. So, to overcome the problems associated with unsupervised and supervised classification techniques, we have proposed a new semi-supervised clustering technique using the concepts of multiobjective optimization and applied this technique for automatic segmentation of MR brain images in the intensity space. Multiple centers are used to encode a cluster in the form of a string. The proposed clustering technique utilizes intensity values of the brain pixels as the features. Additionally it also assumes that the actual class label information of 10% points of a particular image data set is also known. Three cluster validity indices are utilized as the objective functions, which are simultaneously optimized using AMOSA, a modern multiobjective optimization technique based on the concepts of simulated annealing. First two cluster validity indices are symmetry distance based Sym-index and Euclidean distance based I-index, which are based on unsupervised properties. Last one is a supervised information based cluster validity index, Minkowski Index. The effectiveness of this proposed semi-supervised clustering technique is demonstrated on several simulated MR normal brain images and MR brain images having some multiple sclerosis lesions. The performance of the proposed semi-supervised clustering technique is compared with some other popular image segmentation techniques like Fuzzy C-means, Expectation Maximization and some recent image clustering techniques like multi-objective based MCMOClust technique, and Fuzzy-VGAPS clustering techniques.     

A saliency map based on sampling an image into random rectangular regions of interest

In this article we propose a novel approach to compute an image saliency map based on computing local saliencies over random rectangular regions of interest. Unlike many of the existing methods, the proposed approach does not require any training bases, operates on the image at the original scale and has only a single parameter which requires tuning. It has been tested on the two distinct tasks of salient region detection (using MSRA dataset) and eye gaze prediction (using York University and MIT datasets). The proposed method achieves state-of-the-art performance on the eye gaze prediction task as compared with nine other state-of-the-art methods.     

基于瞳孔-角膜反射技术的视线跟踪方法

视线跟踪是基于多通道的人机交互技术的重要研究内容．而基于瞳孔-角膜反射技术的视线方向是目前应用最广泛的视线跟踪技术之一。瞳孔-角膜反射技术的主要目的是提取人眼图像中瞳孔-角膜反射向量作为视线方向计算模型所需的视觉信息,通过搭建红外光源设备提取瞳孔-角膜反射向量构建基于瞳孔-角膜反射技术的视线跟踪系统,为面向人机交互的视线跟踪研究提供可行的低成本解决方案。