合成纹理图像的视觉相似性眼动分析

探索了当人眼视觉感知纹理图像相似性时纹理特征(整体特征和局部特征)对视觉注意过程及相似性结果选择的影响。采用PGPC纹理合成方法,分别控制纹理的整体特征和局部特征,合成具有相同整体特征和局部特征的纹理。实验分为3组纹理场景,每一组有3幅纹理图像(A,B和S),S为对比纹理图像;实验者带着“纹理A和纹理B,哪个与纹理S更相似”的视觉感知任务观察纹理场景;利用Tobii T60眼动仪记录实验过程中观察者的眼动,跟踪获取89位实验者的眼动数据(眼动数据度量指标包含感兴趣区域内注视点散度和不同感兴趣区域间的眼跳次数);通过Tobii studio导出眼动数据,并基于R软件进行了方差分析和配对T检验等统计分析。通过对3组纹理图像的注视轨迹图及热点图的分析,验证了纹理整体特征和局部特征对视觉感知过程中的注视模式和相似性结果的选择有影响。实验结果表明:1)对于全局特征与对比纹理图像一致的纹理,其眼动注视点分散,而对于局部特征与对比纹理图像一致的纹理,其眼动注视点集中。2)纹理组相似性的最终选择结果与纹理组内不同纹理图像间的视觉搜索行为有关。     

基于眼动轨迹分析的BPPV诊断算法研究

近年来,以机器学习模型辅助临床诊断已成为智慧医疗领域的一大研究热点。在良性阵发性位置性眩晕（Benign Paroxysmal Positional Vertigo,BPPV）的临床诊断上,眼动视频的解释往往是由医生观测得出的,但直接观测诊断的局限性在于难以捕捉细微的眼动特性而容易导致误诊。对此,论文提出了一种基于时序轨迹的BPPV诊断模型,将基于深度学习的目标检测器和基于时序数据分析的分类器相结合以实现BPPV诊断。具体地,该模型对眼动视频进行眼球目标检测,以提取眼球运动的时序轨迹,并结合数据增强对训练样本进行扩充,以准确分类并得到更好的诊断结果。实验结果表明,论文提出的模型可有效提取眼动时序轨迹,并在BPPV诊断上取得良好性能。     

基于混合特征映射的密集场景运动模式分析

密集场景分析是目前计算机视觉领域的热点和难点课题。提出了一种新的密集场景下集群目标运动模式的分析算法。该算法根据集群目标运动特有的规则获取集群目标的轨迹片段,对轨迹片段学习后验散度,得出产生式-判别式混合特征映射,该特征映射有效地将底层特征和运动模式的语义信息结合起来。通过对特征映射进行基于图模型算法的无监督分层聚类,挖掘出集群目标运动模式信息。实验结果准确地揭示了当前视频中运动模式的分布,证明了该算法的有效性。     

视觉感知式场景文字检测定位方法

针对自然场景中复杂背景干扰检测的问题,本文提出一种基于视觉感知机制的场景文字检测定位方法。人类视觉感知机制通常分为快速并行预注意步骤与慢速串行注意步骤。本文方法基于人类感知机制提出一种场景文字检测定位方法,该方法首先通过两种视觉显著性方法进行预注意步骤,然后利用笔画特征以及文字相互关系实现注意步骤。本文方法在ICDAR 2013与场景汉字数据集中均取得较有竞争力的结果,实验表明可以较好地用于复杂背景的自然场景英文和汉字的检测。     

一种动态场景中的视觉注意区域检测方法

利用心理学中有关视觉注意的研究成果,提出一种新的动态场景中的视觉注意区域检测算法.该算法利用视觉对场景的感知的特点,以特征点轨迹作为运动特征,计算特征点运动的显著性,并用运动显著的特征点作为"种子",结合空间分割方法产生运动显著图.为了兼顾静态场景,则利用颜色和亮度作为特征,以center-surround反差模型获得图像的静态显著图. 最后提出一种基于运动优先思想的方法将运动和空间显著图进行动态融合,生成视觉注意区域.与以往方法相比,该方法生成的视觉注意区域较为完整,并且具有更好的抗噪性.实验结果证明了该方法的有效性和稳定性.     

一种新的人眼视觉运动认知特性实验分析方法

视觉感知过程源于对象自身的大范围拓扑性质,而场景对象发生运动状态变化时,其所在空间的大范围拓扑关系也将随之改变。为探索大范围拓扑性质和运动性质同时改变时人眼视觉的认知优先特性,基于虚拟现实技术及眼动跟踪技术,通过构建虚拟驾驶及眼动轨迹分析实验平台,结合拓扑复杂度和空间拓扑关系概念分别探讨二维和三维空间中人眼视知觉对物体运动的认知特性。在此基础上,采用响应面分析方法设计场景中自变量因子的随机实验数值,通过可视化方法对比多组认知场景的无感反馈眼动监测结果。实验结果表明,在Tobii眼动仪采集的545个眼动数据中,发生在运动物体上的数据有516个,相比于物体自身属性的大范围拓扑性质变化,人眼视知觉过程更优先识别物体运动的大范围空间拓扑关系变化。该运动认知优先特性为可靠人工智能驾驶中的视觉注视预测技术提供了新的研究思路,同时验证了所提实验分析方法的有效性。     

基于多分类器融合的多视角目标检测算法

在视觉感知获得图像视觉信息的基础上,提出了一种自底向上的视觉搜索与自顶向下的语义判定相联系的模型,该模型使图像区域和图像块与图像语义和图像类别之间建立感知联系,模拟注意焦点的移动,通过动眼扫描和搜索图像区域,对感兴趣的内容进行索引和存储,并在任务事件的驱动下进行联想,使图像理解融入了视觉感知和语义解释两个方面,符合人类的认知规律。     

改进视皮层视觉机制的视觉注意力模型

人类的视觉注意机制是人类大脑感知事物的最直接的功能。提出了一种基于视皮层视觉机制的生物激励注意模型。利用HMAX(hierarchical maximization)模型的四层机制中的C1细胞单元图,构造独立成分分析(independent component analysis,ICA)滤波器组,进一步利用对尺度、平移等均具有不变性的C2细胞特征,以及香农熵理论,共同构造用于视觉显著性区域检测的测度。在心理学实验的自然场景图像以及心理学刺激模式上的结果表明,该方法与传统方法相比,更符合人眼的感知特性,从而进一步验证了该方法的有效性和准确性。     

基于多层BP神经网络的无参考视频质量客观评价

机器学习在视频质量评价(Video quality assessment, VQA)模型回归方面具有较大的优势, 能够较大地提高构建模型的精度. 基于此, 设计了合理的多层BP神经网络, 并以提取的失真视频的内容特征、编解码失真特征、传输失真特征及其视觉感知效应特征参数为输入, 通过构建的数据库中的样本对其进行训练学习, 构建了一个无参考VQA模型. 在模型构建中, 首先采用图像的亮度和色度及其视觉感知、图像的灰度梯度期望值、图像的模糊程度、局部对比度、运动矢量及其视觉感知、场景切换特征、比特率、初始时延、单次中断时延、中断频率和中断平均时长共11个特征, 来描述影响视频质量的4个主要方面, 并对建立的两个视频数据库中的大量视频样本, 提取其特征参数; 再以该特征参数作为输入, 对设计的多层BP神经网络进行训练, 从而构建VQA模型; 最后, 对所提模型进行测试, 同时与14种现有的VQA模型进行对比分析, 研究其精度、复杂性和泛化性能. 实验结果表明: 所提模型的精度明显高于其14种现有模型的精度, 其最低高出幅度为4.34 %; 且优于该14种模型的泛化性能, 同时复杂性处于该15种模型中的中间水平. 综合分析所提模型的精度、泛化性能和复杂性表明, 所提模型是一种较好的基于机器学习的VQA模型.     

用异质迁移学习构建跨被试脑电情感模型

由于脑电信号的个体差异性和非平稳特性对情感模型性能产生影响,如何构建跨被试脑电情感模型是情感脑-机接口领域的一个重要研究方向.本文提出一种新的从眼睛的扫视轨迹进行知识迁移的异质迁移学习方法,以提升跨被试脑电情感模型的性能.该方法的主要神经生理学依据是,被试的视觉注视诱发了大脑特定的神经活动,而这些神经活动产生的脑电信号可以为情绪识别提供重要的情境线索.为了量化不同被试之间的域差异,我们引入了基于扫视轨迹和基于脑电信号的核矩阵,并提出了改进的直推式参数迁移学习算法,以实现跨被试脑电情感模型的构建.该方法与传统方法相比,具有两个优点:一是利用了目标被试容易获取的眼动追踪数据进行被试迁移,二是在目标被试只有眼动追踪数据的情况下,仍然能够从其他被试的历史数据中学到脑电信号的情绪类别判别信息.为了验证所提方法的有效性,我们对本文提出的方法与已有的迁移方法在三类情绪识别的脑电和眼动数据集上进行了系统的对比实验研究.实验结果表明,基于眼动轨迹的迁移模型取得了与基于脑电信号的迁移模型相当的识别性能.相对于传统的通用分类器50.46%的平均准确率,基于眼动轨迹的迁移模型的平均准确率达到了69.72%.     

Automatic gaze-based user-independent detection of mind wandering during computerized reading

Mind wandering is a ubiquitous phenomenon where attention involuntarily shifts from task-related thoughts to internal task-unrelated thoughts. Mind wandering can have negative effects on performance; hence, intelligent interfaces that detect mind wandering can improve performance by intervening and restoring attention to the current task. We investigated the use of eye gaze and contextual cues to automatically detect mind wandering during reading with a computer interface. Participants were pseudorandomly probed to report mind wandering while an eye tracker recorded their gaze during the reading task. Supervised machine learning techniques detected positive responses to mind wandering probes from eye gaze and context features in a user-independent fashion. Mind wandering was detected with an accuracy of 72 % (expected accuracy by chance was 60 %) when probed at the end of a page and an accuracy of 67 % (chance was 59 %) when probed in the midst of reading a page. Global gaze features (gaze patterns independent of content, such as fixation durations) were more effective than content-specific local gaze features. An analysis of the features revealed diagnostic patterns of eye gaze behavior during mind wandering: (1) certain types of fixations were longer; (2) reading times were longer than expected; (3) more words were skipped; and (4) there was a larger variability in pupil diameter. Finally, the automatically detected mind wandering rate correlated negatively with measures of learning and transfer even after controlling for prior knowledge, thereby providing evidence of predictive validity. Possible improvements to the detector and applications that utilize the detector are discussed.     

Video-realistic image-based eye animation via statistically driven state machines

In this work we elaborate on a novel image-based system for creating video-realistic eye animations to arbitrary spoken output. These animations are useful to give a face to multimedia applications such as virtual operators in dialog systems. Our eye animation system consists of two parts: eye control unit and rendering engine, which synthesizes eye animations by combining 3D and image-based models. The designed eye control unit is based on eye movement physiology and the statistical analysis of recorded human subjects. As already analyzed in previous publications, eye movements vary while listening and talking. We focus on the latter and are the first to design a new model which fully automatically couples eye blinks and movements with phonetic and prosodic information extracted from spoken language. We extended the already known simple gaze model by refining mutual gaze to better model human eye movements. Furthermore, we improved the eye movement models by considering head tilts, torsion, and eyelid movements. Mainly due to our integrated blink and gaze model and to the control of eye movements based on spoken language, subjective tests indicate that participants are not able to distinguish between real eye motions and our animations, which has not been achieved before.     

Impact of mainstream classroom setting on attention of children with autism spectrum disorder: an eye-tracking study

It has long been reported that children with autism spectrum disorder (ASD) exhibit attention difficulties while learning. They tend to focus on irrelevant information and can easily be distracted. As a result, they are often confined to a one-to-one teaching environment, with fewer distractions and social interactions than would be present in a mainstream educational setting. In recent years, inclusive mainstream schools have been growing in popularity due to government policies on equality rights. Therefore, it is crucial to investigate attentional patterns of children with ASD in mainstream schools. This study aims to explore the attentional behaviors of children with ASD in a virtual reality simulated classroom. We analyzed four eye-gaze behaviors and performance scores of 45 children: children with ASD (ASD n = 20) and typically developing children (TD n = 25) when performing attention tasks. The gaze behaviors included time to first fixate (TTFF), first fixation duration (FFD), average fixation duration (AFD) and the sum of fixation count (SFC) on fourteen areas of interest (AOIs) in the classroom. Our results showed that children with ASD exhibit similar gaze behaviors to TD children, but with significantly lower performance scores and SFC on the target AOI. These findings showed that classroom settings can influence attentional patterns and the academic performance of children with ASD. Further studies are needed on different modalities for supporting the attention of children with ASD in a mainstream setting.

     

Bi-directional navigation intent communication using spatial augmented reality and eye-tracking glasses for improved safety in human–robot interaction

Safety, legibility and efficiency are essential for autonomous mobile robots that interact with humans. A key factor in this respect is bi-directional communication of navigation intent, which we focus on in this article with a particular view on industrial logistic applications. In the direction robot-to-human, we study how a robot can communicate its navigation intent using Spatial Augmented Reality (SAR) such that humans can intuitively understand the robot’s intention and feel safe in the vicinity of robots. We conducted experiments with an autonomous forklift that projects various patterns on the shared floor space to convey its navigation intentions. We analyzed trajectories and eye gaze patterns of humans while interacting with an autonomous forklift and carried out stimulated recall interviews (SRI) in order to identify desirable features for projection of robot intentions. In the direction human-to-robot, we argue that robots in human co-habited environments need human-aware task and motion planning to support safety and efficiency, ideally responding to people’s motion intentions as soon as they can be inferred from human cues. Eye gaze can convey information about intentions beyond what can be inferred from the trajectory and head pose of a person. Hence, we propose eye-tracking glasses as safety equipment in industrial environments shared by humans and robots. In this work, we investigate the possibility of human-to-robot implicit intention transference solely from eye gaze data and evaluate how the observed eye gaze patterns of the participants relate to their navigation decisions. We again analyzed trajectories and eye gaze patterns of humans while interacting with an autonomous forklift for clues that could reveal direction intent. Our analysis shows that people primarily gazed on that side of the robot they ultimately decided to pass by. We discuss implications of these results and relate to a control approach that uses human gaze for early obstacle avoidance.     

使用人眼几何特征的视线追踪方法

目的 视线追踪是人机交互的辅助系统,针对传统的虹膜定位方法误判率高且耗时较长的问题,本文提出了一种基于人眼几何特征的视线追踪方法,以提高在2维环境下视线追踪的准确率。方法 首先通过人脸定位算法定位人脸位置,使用人脸特征点检测的特征点定位眼角点位置,通过眼角点计算出人眼的位置。直接使用虹膜中心定位算法的耗时较长,为了使虹膜中心定位的速度加快,先利用虹膜图片建立虹膜模板,然后利用虹膜模板检测出虹膜区域的位置,通过虹膜中心精定位算法定位虹膜中心的位置,最后提取出眼角点、虹膜中心点等信息,对点中包含的角度信息、距离信息进行提取,组合成眼动向量特征。使用神经网络模型进行分类,建立注视点映射关系,实现视线的追踪。通过图像的预处理对图像进行增强,之后提取到了相对的虹膜中心。提取到需要的特征点,建立相对稳定的几何特征代表眼动特征。结果 在普通的实验光照环境中,头部姿态固定的情况下,识别率最高达到98.9%,平均识别率达到95.74%。而当头部姿态在限制区域内发生变化时,仍能保持较高的识别率,平均识别率达到了90%以上。通过实验分析发现,在头部变化的限制区域内,本文方法具有良好的鲁棒性。结论 本文提出使用模板匹配与虹膜精定位相结合的方法来快速定位虹膜中心,利用神经网络来对视线落点进行映射,计算视线落点区域,实验证明本文方法具有较高的精度。     

一种基于视频数据的服装显著性预测方法

人眼视觉注意机制表明当人眼观察目标时,注意力只会放在少数感兴趣的区域, 而自动忽略视野中大部分不感兴趣的其他区域。研究人类视觉注意机制,并构建有效的服装显 著性预测模型,可在后期用于指导更加逼真有效的服装运动建模,提高模拟效率。为此,对着 装人体运动视频数据进行分析,构造了种类多样的视频样本,并利用眼动技术采集真实人眼的 注视数据,采用高斯卷积生成视频帧的显著图作为训练模型所需的 Ground-truth。在进行视频特 征提取时,结合了底层图像特征、高层语义特征以及运动特征,共同构造特征向量和标签,并 通过支持向量机(SVM)训练得到基于服装视频的显著性预测模型。通过实验验证,该方法的性 能在服装显著性预测时,优于传统的显著性预测算法,具有一定的鲁棒性。     

A real-time gaze position estimation method based on a 3-D eye model.

This paper proposes a new gaze-detection method based on a 3-D eye position and the gaze vector of the human eyeball. Seven new developments compared to previous works are presented. First, a method of using three camera systems, i.e., one wide-view camera and two narrow-view cameras, is proposed. The narrow-view cameras use autozooming, focusing, panning, and tilting procedures (based on the detected 3-D eye feature position) for gaze detection. This allows for natural head and eye movement by users. Second, in previous conventional gaze-detection research, one or multiple illuminators were used. These studies did not consider specular reflection (SR) problems, which were caused by the illuminators when working with users who wore glasses. To solve this problem, a method based on dual illuminators is proposed in this paper. Third, the proposed method does not require user-dependent calibration, so all procedures for detecting gaze position operate automatically without human intervention. Fourth, the intrinsic characteristics of the human eye, such as the disparity between the pupillary and the visual axes in order to obtain accurate gaze positions, are considered. Fifth, all the coordinates obtained by the left and right narrow-view cameras, as well as the wide-view camera coordinates and the monitor coordinates, are unified. This simplifies the complex 3-D converting calculation and allows for calculation of the 3-D feature position and gaze position on the monitor. Sixth, to upgrade eye-detection performance when using a wide-view camera, the adaptive-selection method is used. This involves an IR-LED on/off scheme, an AdaBoost classifier, and a principle component analysis method based on the number of SR elements. Finally, the proposed method uses an eigenvector matrix (instead of simply averaging six gaze vectors) in order to obtain a more accurate final gaze vector that can compensate for noise. Experimental results show that the root mean square error of gaze detection was about 0.627 cm on a 19-in monitor. The processing speed of the proposed method (used to obtain the gaze position on the monitor) was 32 ms (using a Pentium IV 1.8-GHz PC). It was possible to detect the user's gaze position at real-time speed.     

Eye movement prediction by oculomotor plant Kalman filter with brainstem control

Our work addresses one of the core issues related to Human Computer Interaction （HCI） systems that use eye gaze as an input. This issue is the sensor, transmission and other delays that exist in any eye tracker-based system, reducing its performance. A delay effect can be compensated by an accurate prediction of the eye movement trajectories. This paper introduces a mathematical model of the human eye that uses anatomical properties of the Human Visual System to predict eye movement trajectories. The eye mathematical model is transformed into a Kalman filter form to provide continuous eye position signal prediction during all eye movement types. The model presented in this paper uses brainstem control properties employed during transitions between fast （saccade） and slow （fixations, pursuit） eye movements. Results presented in this paper indicate that the proposed eye model in a Kalman filter form improves the accuracy of eye movement prediction and is capable of a real-time performance. In addition to the HCI systems with the direct eye gaze input, the proposed eye model can be immediately applied for a bit-rate/computational reduction in real-time gaze-contingent systems     

Nonparametric discovery and analysis of learning patterns and autism subgroups from therapeutic data

The spectrum nature and heterogeneity within autism spectrum disorders (ASD) pose as a challenge for treatment. Personalisation of syllabus for children with ASD can improve the efficacy of learning by adjusting the number of opportunities and deciding the course of syllabus. We research the data-motivated approach in an attempt to disentangle this heterogeneity for personalisation of syllabus. With the help of technology and a structured syllabus, collecting data while a child with ASD masters the skills is made possible. The performance data collected are, however, growing and contain missing elements based on the pace and the course each child takes while navigating through the syllabus. Bayesian nonparametric methods are known for automatically discovering the number of latent components and their parameters when the model involves higher complexity. We propose a nonparametric Bayesian matrix factorisation model that discovers learning patterns and the way participants associate with them. Our model is built upon the linear Poisson gamma model (LPGM) with an Indian buffet process prior and extended to incorporate data with missing elements. In this paper, for the first time we have presented learning patterns deduced automatically from data mining and machine learning methods using intervention data recorded for over 500 children with ASD. We compare the results with non-negative matrix factorisation and K-means, which being parametric, not only require us to specify the number of learning patterns in advance, but also do not have a principle approach to deal with missing data. The F1 score observed over varying degree of similarity measure (Jaccard Index) suggests that LPGM yields the best outcome. By observing these patterns with additional knowledge regarding the syllabus it may be possible to observe the progress and dynamically modify the syllabus for improved learning.     

Drawing reveals hallmarks of children with autism

Autism spectrum disorder (ASD) as a kind of mental disorder, has become an internationally recognized serious public health problem. Paintings of autistic children have not been compared systematically to those from Typically Developed (TD) children. In this work, we construct an ASD painting database which contains 478 paintings drawn by ASD individuals and 490 drawn by the TD group. Through subjective and objective analysis, some significant hallmarks, such as structuring logic, face, repetitive structure, composition location, edge completeness, etc. are found within the ASD paintings. We further train a classifier of ASD and TD painters using those extracted features, which shows encouraging accuracy as a potential screen tool for ASD. This work sheds light on understanding the uniqueness of autistic children through their paintings. The database will be released to the public.