Study of image quality using event‐related potentials and eye tracking measurement

The image quality was investigated using electroencephalography and eye‐tracking measurements simultaneously. Different image qualities were obtained by changing σ values in the Gaussian kernel function. The experimental results illustrated that the measurable event‐related potentials and eye‐tracking parameters varied with the change of the image quality. The more blurred the image, the higher and earlier the P300 (event‐related potentials at latency about 300–550 ms) was elicited. Meanwhile, the visual fixation duration decreased and the detection rate increased with the increase of σ. The blurred image quality could be evaluated based on the synthetic results of the reciprocal of P300 latency and the visual fixation duration. The result showed a good correlation with the subjective rating scores, which proved the feasibility of utilizing electroencephalography and eye‐tracking technology to evaluate the blurred image quality.     

Linear Quasi-Parallax SfM Using Laterally-Placed Eyes

A large class of visual systems in the biological world often has multiple eyes in simultaneous motion and yet has little or no overlap in the visual fields between the eyes. These systems include the lateral eyes found in many vertebrates and the compound eyes in insects. Instead of computing feature correspondences between the eyes, which might not even be possible due to the lack of overlap in the visual fields, we exploit the organizational possibility offered by the eye topography. In particular, we leverage on the pair of visual rays that are parallel to each other but opposite in direction, and compute what we call the quasi-parallax for translation recovery. Besides resulting in parsimonious visual processing, the quasi-parallax term also enhances the information pick-up for the translation, as it is almost rotation-free. The rotation is subsequently recovered from a pencil of visual rays using the individual epipolar constraints of each camera. As a result of using these different and appropriate aspects of visual rays for motion recovery, our method is numerically more effective in disambiguating the translation and rotation. In comparison to the gold standard solution obtained by the bundle adjustment (BA) technique, our method has a better Fisher information matrix for a lateral eye pair, as well as a superior experimental performance under the case of narrow field of view. For other eye configurations, the two methods achieve comparable performances, with our linear method slightly edging the nonlinear BA method when there exists imperfection in the calibration.     

多用户眼动跟踪数据的可视化共享与协同交互

随着数字图像处理技术的发展,以及计算机支持的协同工作研究的深入,眼动跟踪开始应用于多用户协同交互.但是已有的眼动跟踪技术主要针对单个用户,多用户眼动跟踪计算架构不成熟、标定过程复杂,眼动跟踪数据的记录、传输以及可视化共享机制都有待深入研究.为此,建立了基于梯度优化的协同标定模型,简化多用户的眼动跟踪标定过程;然后提出面向多用户的眼动跟踪计算架构,优化眼动跟踪数据的传输和管理.进一步地,探索眼动跟踪数据的可视化形式在协同交互环境下对用户视觉注意行为的影响,具体设计了圆点、散点、轨迹这3种可视化形式,并验证了圆点形式能够有效地提高多用户协同搜索任务的完成效率.在此基础上,设计与开发了基于眼动跟踪的代码协同审查系统,实现了代码审查过程中多用户眼动跟踪数据的同步记录、分发,以及基于实时注视点、代码行边框和背景灰度、代码行之间连线的可视化共享.用户实验结果表明,代码错误的平均搜索时间比没有眼动跟踪数据可视化分享时减少了20.1%,显著提高了协同工作效率,验证了该方法的有效性.     

视觉感知正反馈的显著性检测

目的 人类视觉系统性能远超当前机器视觉,模拟人类视觉机制改进当前算法是有效研究途径,为此提出一种视觉感知正反馈模型,通过循环迭代、重复叠加视觉刺激生成更符合人类感知的视觉显著性图。方法 首先用多种常规方法检测图像显著度,模拟人类视觉多通道特性,再组合这些显著图为综合显著图;利用显著度大的像素构建初始注视区。其次借助集成RVFL（随机向量功能网络）模拟人脑神经网络产生视觉刺激,对注视与非注视区内像素在线“随机采样—学习建模”,图像像素经模型分类获得新注视区。对新注视区与非注视区,可重复迭代进行“随机采样—学习建模—像素分类”;迭代中若注视区连续相同,则表明感知饱和,迭代终止。若将每次像素分类结果看做是一种视觉刺激,则多次视觉刺激输出叠加,可生成新的图像显著性图。最终的像素分类结果就是图像分割目标。结果 将本文算法与现有方法在标准图像数据库上进行对比评测,包括通过对6种算法在ECSSD、SED2和MSRA10K 3个图像数据库上的P-R曲线,F-measure值和平均绝对误差（MAE）值上进行定量分析,对6种模型生成的显著性图作定性比较。数据表明,本文算法在SED2和MSRA10K图象数据库中性能最好,在ECSSD图象数据库中稍低于BL（bootstrap learning）和RBD（robust background detection）算法。本文算法的显著图与人类视觉感知更接近。且算法的正反馈迭代过程一般可迅速饱和,并未显著增加算法负担。实验结果表明,本文方法可作为一种有效的后处理手段,显著提升常规显著性检测算法的性能。结论 提出了一种模拟人类视觉机制的数据驱动显著性检测算法,无需图像先验知识和事先的标记样本。面对多目标,背景复杂等情况,本文方法具有相对好的鲁棒性和适用性,并且能够较好解决现实环境中图像处理算法的通用性、可靠性和准确性问题。     

Models for the Design of Bioinspired Robot Eyes

Active vision has the goal of improving visual perception; therefore, the investigation of ocular motion strategies must play an important role in the design of humanoid robot eyes. Listing's law is a basic principle, which characterizes various ocular movements in humans, including saccades and smooth pursuit, and its neural or mechanical origin has been debated for a long time. Recent anatomical advances suggest that motions compatible with Listing's law could be mainly caused by the mechanical structure of the eye plant. In this paper, we present a bioinspired model of the eye plant, and we formally prove that according to the model, the implementation of Listing's law can be actually explained on the base of the geometry of the eye and of its actuation system. The proposed model is characterized by a limited number of geometric parameters, which can be easily used to set the guidelines for the design of humanoid, and possibly tendon-driven, robot eyes. Simulative and experimental tests performed on a robot prototype are eventually presented to perform a quantitative evaluation of the performance of the model, also in comparison with physiological data measured in humans and primates and reported in the literature.     

Utilization of Visual Information Perception Characteristics to Improve Classification Accuracy of Driver’s Visual Search Intention for Intelligent Vehicle

Understanding and predicting a driver’s behaviors in a vehicle is a prospective function embedded in a smart car. Beyond the patterns of observable behaviors, driver’s intention could be identified based on goal-driven behaviors. A computational model to classify driver intention in visual search which is finding a target with one’s eyes as moving selective attention across a search field, could improve the level of intelligence that a smart car could demonstrate. To develop a computational cognitive that explains the underlying cognitive process and reproduces drivers’ behaviors, particular parameters in human cognitive process should be specified. In this study, 2 issues are considered as influential factors on a driver’s eye movements: a driver’s visual information processing characteristics (VIPCs) and the purpose of visual search. To assess an individual’s VIPC, 4 psychological experiments—Donders’s reaction time, mental rotation, signal detection, and Stroop experiments—were utilized. Upon applying k-means clustering method, 114 drivers were divided into 9 driver groups. To investigate the influence of task goal on a driver’s eye movement, driving simulation was conducted to collect a driver’s eye movement data under the given purpose of visual search (perceptual and cognitive tasks). The empirical data showed that there were significant differences in a driver’s oculomotor behavior, such as response time, average fixation time, and average glance duration between the driver groups and the purposes of visual search. The effectiveness of using VIPC for grouping drivers was tested with task goal classification model by comparing the models’ performance when drivers were grouped by typical demographic data such as gender. Results show that grouping based on VIPC improves accuracy and stability of prediction of the model on a driver’s intention underlying visual search behaviors. This study would benefit future studies focusing on personalization and adaptive interfaces in the development of smart car.     

人脸特征的定位与提取

人脸的特征定位与特征提取的好坏对于人脸图象识别效果有直接的影响．本文根据人脸图象的灰度特性以及特征的几何形状提出了一种简便有效的特征定位和特征提取的方法。本文所提出的方法能够比较准确而快速的提取较多的特征，因而能够适用于在大量人像中进行特征提取和识别。     

A basic study of a face-to-face image presentation system

Recently, broadband networks have made great progress and various interactive visual communication systems including video conferencing, video telephones and video networks for meetings are becoming increasingly popular. However, unnatural eye contact in the displays used by these communication systems has presented a barrier to natural conversation between communicators. We confirmed the possibility of realizing the feeling of eye contact that is part of natural face-to-face conversation. This paper also describes experimental results on the permissible eye-line angle that gives a sense of eye contact and the influence of image resolution.     

Absolute radiometric and photometric measurements of near‐eye displays

The arrival of near‐eye displays has challenged the traditional methods that have been used to measure the optical properties of displays. Near‐eye displays typically create virtual images and are designed for the relatively small entrance pupil of the human eye. These two attributes result in optical measurement requirements that are substantially different from traditional flat panel displays. This paper discusses the optical system requirements needed to make absolute radiometric and photometric measurements of near‐eye displays. These guidelines are contrasted with the performance of current optical measurement instruments. An initial study was conducted using traditional and modified instruments and exhibited a significant variance in the results with different near‐eye display designs. The study demonstrated that some traditional optical instruments can yield erroneous results when used to measure near‐eye displays. Generic optical system design concepts were used to interpret the experimental results and helped to identify how current commercial designs could be modified to properly measure near‐eye displays.     

Visual remapping by vector subtraction: analysis of multiplicative gain field models

Saccadic eye movements remain spatially accurate even when the target becomes invisible and the initial eye position is perturbed. The brain accomplishes this in part by remapping the remembered target location in retinal coordinates. The computation that underlies this visual remapping is approximated by vector subtraction: the original saccade vector is updated by subtracting the vector corresponding to the intervening eye movement. The neural mechanism by which vector subtraction is implemented is not fully understood. Here, we investigate vector subtraction within a framework in which eye position and retinal target position signals interact multiplicatively (gain field). When the eyes move, they induce a spatial modulation of the firing rates across a retinotopic map of neurons. The updated saccade metric can be read from the shift of the peak of the population activity across the map. This model uses a quasi-linear (half-rectified) dependence on the eye position and requires the slope of the eye position input to be negatively proportional to the preferred retinal position of each neuron. We derive analytically this constraint and study its range of validity. We discuss how this mechanism relates to experimental results reported in the frontal eye fields of macaque monkeys.     

基于编码和二维条形码的数字水印

提出了一种基于编码和二维条形码的数字水印算法,基本思想是先对二维条形码进行0、1二值化,再对结果进行分块,并且对各个子块进行编码,最后将各子块合并以后的图像按照特定顺序排列作为水印。采用变换域方法实现水印嵌入,先对载体图像进行小波变换,再根据人眼的视觉特性将水印嵌入到小波变换的低频系数上。实验结果表明该方法具有较好的鲁棒性和不可见性。     

人眼灰度感知建模及其在图像增强中的应用

原直方图均衡化算法处理结果不理想,主要是图像灰度域上实现的均衡化并不代表人眼感知亮度域上的均衡化,对此提出人眼感知亮度域上的直方图均衡化算法,但由于使用的灰度人眼感知模型不准确,处理效果也没有得到明显的改善。提出一种灰度人眼感知模型的建模方法,利用实验测试的方法得到人眼视觉系统临界可见偏差曲线;再推导得到人眼对不同灰度背景下同等灰度差别的敏感度曲线;接着利用上一步结果再通过积分和归一化方法得到灰度人眼感知模型;根据以上灰度人眼感知模型对直方图均衡化算法进行改进。对比实验结果表明,提出算法相比于原直方图均衡化具有明显的改善效果,相比于CLAHE、BBHE以及HMF等直方图改进算法,也具有无需参数调节,增强效果显著和适应性强等优点。     

融合眼动数据和文本信息的可视化阅读辅助

研究阅读过程中的视觉注意行为特征,基于眼动数据和文本主题进行阅读行为特征和文档结构的分析,设计了眼动热区图、圆环图、节点链接图、词云等可视化形式.在此基础上开发了面向阅读辅助的可视分析原型系统,该系统记录专家用户（如老师）的眼动数据,然后通过可视化形式分享给新手用户（如学生）.用户实验结果表明,实验组用户的阅读理解客观题和主观题得分平均值分别比对照组用户提高了31.8%和55.0%,阅读和答题总用时比对照组用户平均减少了9.7%,可以有效帮助读者提高阅读效率、快速抓住文章重点和更好地理解文章内容,具有一定的有效性和可行性.     

基于音视一致性的音视人眼关注点检测

现有音视人眼关注点检测算法使用双流结构分别对音视信息进行特征提取,随后对音视特征融合得到最终的预测图。但数据集中的音频信息和视觉信息会有不相关的情况,因此在音视不一致时直接对音视特征进行融合会使得音频信息对视觉特征产生消极的影响。针对上述问题,本文提出一种基于音视一致性的音视人眼关注点检测网络（Audio-visual Consistency Network, AVCN）。为验证该网络的可靠性,本文在现有音视结合的人眼关注点检测模型上加入音视一致性网络,AVCN对提取的音、视频特征进行一致性二值判断,二者一致时,输出音视融合特征作为最终的预测图,反之则输出视觉占主导的特征作为最终结果。该算法在开放的6个数据集上进行实验,结果表明加入AVCN模型的整体指标会有所提高。     

基于支持向量机的人眼检测

人眼检测是计算机人脸识别的重要部分。复杂环境下人眼定位容易受到光照以及人不同姿态的影响。为了解决复杂环境下的人眼定位问题,采用基于支持向量机的人眼检测算法,首先对复杂环境下采集的不同人的人眼样本进行灰度化均衡以及小波变换,将变换结果表示成向量形式,运用序贯最小优化算法进行训练,得到一组支持向量,然后遍历待检测人脸图利用支持向量所构成的分类器进行人眼初检,最后根据先验知识完成信息融合,最终标定人眼。实验结果表明,该算法对各种复杂环境下的含人眼图像有普遍的适应性和有效性。     

Fixational eye movements, natural image statistics, and fine spatial vision

Perception and motor control are often regarded as two separate branches of neuroscience. Like most species, however, humans are not passively exposed to the incoming flow of sensory data, but actively seek useful information. By shaping input signals in ways that simplify perceptual tasks, behavior might play an important role in establishing efficient sensory representations in the brain. Under natural viewing conditions, the main source of motion of the stimulus on the retina is not the scene but our own behavior. The retinal image is never still, even during visual fixation, when small eye movements combine with movements of the head and body to continually perturb the location of gaze. This article examines the impact of the fixational motion of the retinal image on the statistics of visual input and the neural encoding of visual information. Building upon recent theoretical and experimental results, it is argued that an unstable fixation constitutes an efficient strategy for acquiring information from natural scenes. According to this theory, the fluctuations of luminance caused by the incessant motion of the eye equalize the power present at different spatial frequencies in the spatiotemporal stimulus on the retina. This phenomenon yields compact neural representations, emphasizes fine spatial detail, and might enable a temporal multiplexing of visual information from the retina to the cortex. This theory posits motor contributions to early visual representations and suggests that perception and behavior are more intimately tied than commonly thought.     

一种融合视觉决策理论预测用户选择的方法

为有效解决在眼动人机交互环境中由于难以判断用户决策而导致的米达斯接触问题, 结合视觉决策理论, 通过融合眼动指标与时间序列来预测用户行为。先记录用户浏览目标页面时的眼动数据及进行选择时的点击数据, 并对数据按时间轴进行采样, 得到注视序列; 然后根据注视序列拟合用户进行决策时的注视似然曲线, 确定相关参数与触发阈值。眼动实验结果表明, 单凭注视阈值预测准确率就可以达到78. 3%, 在人机交互环境中拟合的似然曲线与视觉决策理论耦合。方法的实施无须用户进行额外的配合, 不依赖预设的固定阈值; 相对于设置固定触发阈值, 或要用户进行眨眼、按键等操作的传统方法, 交互过程可更顺畅、自然。     

一种新的人眼视觉运动认知特性实验分析方法

视觉感知过程源于对象自身的大范围拓扑性质,而场景对象发生运动状态变化时,其所在空间的大范围拓扑关系也将随之改变。为探索大范围拓扑性质和运动性质同时改变时人眼视觉的认知优先特性,基于虚拟现实技术及眼动跟踪技术,通过构建虚拟驾驶及眼动轨迹分析实验平台,结合拓扑复杂度和空间拓扑关系概念分别探讨二维和三维空间中人眼视知觉对物体运动的认知特性。在此基础上,采用响应面分析方法设计场景中自变量因子的随机实验数值,通过可视化方法对比多组认知场景的无感反馈眼动监测结果。实验结果表明,在Tobii眼动仪采集的545个眼动数据中,发生在运动物体上的数据有516个,相比于物体自身属性的大范围拓扑性质变化,人眼视知觉过程更优先识别物体运动的大范围空间拓扑关系变化。该运动认知优先特性为可靠人工智能驾驶中的视觉注视预测技术提供了新的研究思路,同时验证了所提实验分析方法的有效性。     

Effect of disparity and viewing ambient illumination on stereoscopic viewing: An eye movement approach

Researchers have found that factors related to viewing content and viewing environment could influence viewers' stereoscopic viewing experiences. In this paper, the effect of both disparity and viewing ambient illumination condition on stereoscopic viewing was investigated by using an eye movement approach. The eye movement responses to random dot stereograms (RDS) stimuli in stereoscopic viewing were studied by performing a fusion range measurement experiment involving a judgment task. The fusion ranges under two ambient illumination conditions, eye movement responses to RDS stimuli and effects of disparity on eye movement parameters, were analyzed, respectively. The experimental results showed that fusion range under two ambient illumination conditions were different, and changes of eye movement parameters were different among various disparities when ambient illumination conditions varied. As a result, in order to improve viewers' viewing experiences when viewing different 3D contents, the arrangement of viewing ambient illumination conditions and the selection of disparity should be appropriately considered. The findings in this paper provide guidelines to reasonably arrange the viewing ambient illumination condition and efficiently produce healthy disparity in stereoscopic viewing.     

Prediction of human eye movements in facial discrimination tasks

Under natural viewing conditions, human observers selectively allocate their attention to subsets of the visual input. Since overt allocation of attention appears as eye movements, the mechanism of selective attention can be uncovered through computational studies of eyemovement predictions. Since top-down attentional control in a task is expected to modulate eye movements significantly, the models that take a bottom-up approach based on low-level local properties are not expected to suffice for prediction. In this study, we introduce two representative models, apply them to a facial discrimination task with morphed face images, and evaluate their performance by comparing them with the human eye-movement data. The result shows that they are not good at predicting eye movements in this task.