驾驶员疲劳检测系统的研究

为了减少由于驾驶员疲劳驾驶引起的交通事故,提出驾驶员疲劳状态检测系统的方案。使用3×3中值滤波去除噪声和光照对图像的影响,通过对AdaBoost算法的强分类器训练算法改进、级联分类器优化实现人脸的快速检测,在检测到的人脸区域,通过积分灰度投影和从粗到细改进的模板匹配方法对人眼进行准确定位;通过PERCLOS、眼睛闭合时间、眼睛眨眼频率、嘴巴张开程度、头部运动的计算,进行驾驶员疲劳程度的综合判定。实验结果表明,该方法准确率高,兼具了良好的实时性和鲁棒性。     

Evaluation of eye,head and trunk coordination during target tracking tasks

This study tested the feasibility of a method to synchronise and to evaluate eye, head and trunk movement patterns during target tracking tasks performed by 10 subjects. A projected central target was randomly repositioned at 40° and 70° of rotation to the left and right. Subjects were instructed to change gaze as quickly as possible. Head and trunk motion was measured using a motion analysis system, and eye movement was measured using an eye-tracker; all data were synchronised. For healthy subjects, the eye moved faster than the head, there was no trunk movement and the head moved more than the eye to reach further displaced targets. The method tested was feasible, and it could be used to evaluate eye, head and trunk movement patterns of subjects with injuries such as whiplash and concussions.

Practitioner Summary: Studies of eye, head and trunk movements using synchronised methods are needed. We tested the feasibility of a method to synchronise and evaluate eye, head and trunk movement patterns. The method tested was feasible, and it could be used to evaluate movement patterns of subjects with injuries such as whiplash.     

A cascaded scheme for eye tracking and head movement compensation

The authors previously (1995) proposed an efficient method for tracking the eye movements. The proposed algorithm did not address the issue of compensating for head movements. Head movements are normally much slower than eye movements and can be compensated for using another tracking scheme for head position. In this paper, a hybrid method employing the two tracking schemes is developed. To this end, first a measurement model for the compensation of the head movement is formulated and then the overall tracking scheme is implemented by cascading two Kalman filters. The tracking of the iris movement is followed by the compensation of the head movement for each image frame. Experimental results are presented to demonstrate the accuracy aspects and the real-time applicability of the proposed approach     

Simulation results for an innovative point-of-regard sensor using neural networks

A major obstacle in point-of-regard monitoring for human-computer interaction has been the contaminating effect of head movement. A novel solution to this problem has been simulated. A multilayer perceptron converts the distorted pattern of four infrared sources, reflected in the cornea of the user, into an estimate for the point-of-regard. Using an idealised model of the eye the simulation combined vertical, horizontal, pitch and yaw head movements with eye excursions typical of individuals seated in front of a computer display. Results show that the method is capable of dramatically reducing the effects of small head movements. Under these viewing conditions the error in the point-of-regard estimate exhibited a standard deviation of under 0.5 mm. It is concluded that such a scheme could form an attractive solution to the long-standing problem of head movement artefact.     

A model of head movement contribution for gaze transitions

Head posture has been associated with work-related neck symptoms and discomfort, but its relationship with visual tasks has received much less attention. Head movement amplitude is normally a fraction of the angular distance to a visual target, as gaze transition is usually achieved through the combination of both head and eye movement. In this study, the proportion of head orientation vs. target orientation, named head movement contribution ratio (HMCR), was quantified and modelled as a function of target location. Head movements were measured on subjects orienting and maintaining gaze for 2 s at randomly presented visual targets distributed along an arc placed horizontally or vertically. Bootstrap regression models showed that the horizontal HMCR was approximately 69% of target azimuth. The vertical HMCR was bilinear and corresponded to 52% and 8% and of target elevation for targets above and below the horizontal plane, respectively. The data also demonstrated that head orientation is affected by the kinematic coupling between horizontal and vertical components of head movement.

Statement of Relevance: Awkward head and neck posture is a risk factor for work-related musculoskeletal disorders. This study investigated the influence of visual target location on head orientation over a large range of target eccentricity, as an attempt to predict the head and neck posture required for visual target detection and identification.     

Utilization of Visual Information Perception Characteristics to Improve Classification Accuracy of Driver’s Visual Search Intention for Intelligent Vehicle

Understanding and predicting a driver’s behaviors in a vehicle is a prospective function embedded in a smart car. Beyond the patterns of observable behaviors, driver’s intention could be identified based on goal-driven behaviors. A computational model to classify driver intention in visual search which is finding a target with one’s eyes as moving selective attention across a search field, could improve the level of intelligence that a smart car could demonstrate. To develop a computational cognitive that explains the underlying cognitive process and reproduces drivers’ behaviors, particular parameters in human cognitive process should be specified. In this study, 2 issues are considered as influential factors on a driver’s eye movements: a driver’s visual information processing characteristics (VIPCs) and the purpose of visual search. To assess an individual’s VIPC, 4 psychological experiments—Donders’s reaction time, mental rotation, signal detection, and Stroop experiments—were utilized. Upon applying k-means clustering method, 114 drivers were divided into 9 driver groups. To investigate the influence of task goal on a driver’s eye movement, driving simulation was conducted to collect a driver’s eye movement data under the given purpose of visual search (perceptual and cognitive tasks). The empirical data showed that there were significant differences in a driver’s oculomotor behavior, such as response time, average fixation time, and average glance duration between the driver groups and the purposes of visual search. The effectiveness of using VIPC for grouping drivers was tested with task goal classification model by comparing the models’ performance when drivers were grouped by typical demographic data such as gender. Results show that grouping based on VIPC improves accuracy and stability of prediction of the model on a driver’s intention underlying visual search behaviors. This study would benefit future studies focusing on personalization and adaptive interfaces in the development of smart car.     

嵌入式疲劳驾驶监测系统的设计与实现

疲劳驾驶是引发恶性交通事故的重要原因之一,驾驶员疲劳监测技术近年来已逐步成为图像处理领域的一个研究热点。基于改进的和提出的新算法,设计了一个嵌入式驾驶员疲劳监测系统。由可见光/近红外摄像头采集视频,首先采用Haar特征的级联分类器从图像中检测出人脸区域,并用钻石搜索法跟踪人脸区域;然后提取一个新的图像差分统计特征,并结合3个准则判断疲劳状态;最后采用全变分模型消除图像中的非均匀光照,以便实现鲁棒的人眼定位和人脸识别。实验测试结果表明,本系统的疲劳状态监测准确率达到95%以上。     

Driver fatigue detection using a genetic algorithm

Nowadays, many traffic accidents occur due to driver fatigue. Driver fatigue detection based on computer vision is one of the most hopeful applications of image recognition technology. There are several factors that reflect driver's fatigue. Many efforts have been made to develop fatigue monitoring, but most of them focus on only a single behavior, a feature of the eyes, or a head motion, or mouth motion, etc. When fatigue monitoring is implemented on a real model, it is difficult to predict the driver fatigue accurately or reliably based only on a single driver behavior. Additionally, the changes in a driver's performance are more complicated and not reliable. In this article, we represent a model that simulates a space in a real car. A web camera as a vision sensor is located to acquire video-images of the driver. Three typical characteristics of driver fatigue are involved, pupil shape, eye blinking frequency, and yawn frequency. As the influences of these characteristics on driver fatigue are quite different from each other, we propose a genetic algorithm (GA)-based neural network (NN) system to fuse these three parameters. We use the GA to determine the structure of the neural network system. Finally, simulation results show that the proposed fatigue monitoring system detects driver fatigue probability more exactly and robustly. This work was presented in part at the 11th International Symposium on Artificial Life and Robotics, Oita, Japan, January 23–25, 2006     

Effect of vibration on eye,head and helmet movements while wearing a helmet‐mounted display

Previous research has demonstrated a loss of helmet‐mounted display (HMD) legibility for users exposed to whole body vibration. A pair of human factors studies was conducted to evaluate the effect of whole body vibration on eye, head, and helmet movements for seated users of a HMD while conducting simple fixation and smooth pursuit tracking tasks. These experiments confirmed that vertical eye motion can be demonstrated, that is consistent with the human visual systems' response to the vestibular–ocular reflex (VOR). Helmet slippage was also shown to occur, which could exacerbate loss of display legibility. The largest amplitudes in eye movements were observed during exposure to sinusoidal vibration in the 4–6 Hz range, which is consistent with the frequencies that past research has associated with whole‐body resonance and the largest decrease in display legibility. Further, the measured eye movements appeared to be correlated with both the angular acceleration of the user's head and the angular slippage of the user's helmet. This research demonstrates that the loss of legibility while wearing HMDs likely results from a combination of VOR‐triggered eye movements and movement of the display. Future compensation algorithms should consider adjusting the display in response to both VOR‐triggered eye and HMD motion.     

基于人类视觉皮层双通道模型的驾驶员眼动行为识别

驾驶员的危险行为会增加交通事故的发生率,目前对驾驶员行为的研究中大多通过面部识别等方法对异常行为如疲劳驾驶、接电话等进行识别.这种方法仅客观地对驾驶员行为进行分类,而忽略了他们在驾驶过程中的主观心理.眼动仪是记录和分析驾驶员眼动数据的有效工具,可以清晰地了解驾驶员的想法并总结其视觉认知模式.因为目前还没有针对驾驶员眼动...     

Driver glance behaviors and scanning patterns: Applying static and dynamic glance measures to the analysis of curve driving with secondary tasks

Performing secondary tasks (or non‐driving‐related tasks) while driving on curved roads may be risky and unsafe. The purpose of this study was to explore whether driving safety in situations involving curved roads and secondary tasks can be evaluated using multiple measures of eye movement. We adopted Markov‐based transition algorithms (i.e., transition/stationary probabilities, entropy) to quantify drivers’ dynamic eye movement patterns, in addition to typical static visual measures, such as frequency and duration of glances. The algorithms were evaluated with data from an experiment (Jeong & Liu, 2019) involving multiple road curvatures and stimulus‐response secondary task types. Drivers were more likely to scan only a few areas of interest with a long duration in sharper curves. Total head‐down glance time was longer in less sharp curves in the experiment, but the probability of head‐down glances was higher in sharper curves over the long run. The number of reliable transitions between areas of interest varied with the secondary task type. The visual scanning patterns for visually undemanding tasks were as random as those for visually demanding tasks. Markov‐based measures of dynamic eye movements provided insights to better understand drivers’ underlying mental processes and scanning strategies, compared with typical static measures. The presented methods and results can be useful for in‐vehicle systems design and for further analysis of visual scanning patterns in the transportation domain.     

Differences between experts and novices in the monitoring of automated systems

In the future of aviation, operators will have to work with highly automated systems. This necessitates operators monitoring appropriately (OMA). Psychophysiological studies support the idea that eye movements offer an appropriate means for measuring the acquisition of visual information. A normative model for adequate monitoring behaviour was developed and eye movement parameters were defined to identify OMA. The normative model was validated by eye tracking studies with airline pilots and air traffic controllers. The present eye-tracking study focused on differences in monitoring behaviour between experts and novices. Results from 21 experts (experienced pilots and air traffic controllers) and 33 novices (applicants for jobs in aviation) are reported. Significant differences in monitoring behaviour between experts and novices were found. Experts allocate their attention more efficiently. Learning from experts’ monitoring of automated processes will be helpful in learning how to select successful trainees and provide them with appropriate training.     

Differences between oculomotor and perceptual artifacts for temporally limited head mounted displays

We used perceptual and oculomotor measures to understand the negative impacts of low (phantom array) and high (motion blur) duty cycles with a high‐speed, AR‐likehead‐mounted display prototype. We observed large intersubject variability for the detection of phantom array artifacts but a highly consistent and systematic effect on saccadic eye movement targeting during low duty cycle presentations. This adverse effect on saccade endpoints was also related to an increased error rate in a perceptual discrimination task, showing a direct effect of display duty cycle on the perceptual quality. For high duty cycles, the probability of detecting motion blur increased during head movements, and this effect was elevated at lower refresh rates. We did not find an impact of the temporal display characteristics on compensatory eye movements during head motion (e.g., VOR). Together, our results allow us to quantify the tradeoff of different negative spatiotemporal impacts of user movements and make subsequent recommendations for optimized temporal HMD parameters.     

Stimuli fixation and manual response as a function of expectancies

Two experiments investigated effects of expectations on eye fixations and responses to expected and unexpected information in a laboratory setting. Stimulus fixation and manual responses to predefined targets among distractors were measured in a dynamic environment shown on a monitor. Participants were part of either a predictable condition (predictable sequence of targets and distractors) or a random condition. In the predictable condition, participants fixated expected distractors less than targets, with similar fixation times in the random condition. Responses to targets were faster in the predictable condition than in the random condition. Irregularities were either missed or responses were slow. This indicates that incorrect expectations may have negative consequences, with practical implications for skill-based tasks such as operator monitoring and driving. Actual or potential applications of this research include introducing operator support for tasks in which the risk of missing unexpected information or of slow responses may be dramatic.     

Simultaneous measurements of posture and movements of head and trunk by continuous three-dimensional registration

The postures of three groups of employees were measured: straddle carrier drivers, crane operators and office employees. This type of sedentary work can be characterized as being highly static. Using a continuous three-dimensional registration device, the postures and movements of head and trunk were recorded simultaneously. The results show that the adopted postures and patterns of movement were predominantly imposed by the workplace. The posture of the crane operators was the most static compared to the other occupations. The most adopted posture in the sagittal plane for crane operators was trunk flexion of 5° and head flexion of 60°. Typically, a straddle carrier driver rotated his head more than 45° to the left or right for 28% of the day, which far exceeded that of the other groups. The measuring device provides accurate and reproducible data that can subsequently be used for calculating the postural load and for ergonomic analysis.     

A Review of Eye Gaze in Virtual Agents,Social Robotics and HCI: Behaviour Generation,User Interaction and Perception

A person's emotions and state of mind are apparent in their face and eyes. As a Latin proverb states: ‘The face is the portrait of the mind; the eyes, its informers’. This presents a significant challenge for Computer Graphics researchers who generate artificial entities that aim to replicate the movement and appearance of the human eye, which is so important in human–human interactions. This review article provides an overview of the efforts made on tackling this demanding task. As with many topics in computer graphics, a cross‐disciplinary approach is required to fully understand the workings of the eye in the transmission of information to the user. We begin with a discussion of the movement of the eyeballs, eyelids and the head from a physiological perspective and how these movements can be modelled, rendered and animated in computer graphics applications. Furthermore, we present recent research from psychology and sociology that seeks to understand higher level behaviours, such as attention and eye gaze, during the expression of emotion or during conversation. We discuss how these findings are synthesized in computer graphics and can be utilized in the domains of Human–Robot Interaction and Human–Computer Interaction for allowing humans to interact with virtual agents and other artificial entities. We conclude with a summary of guidelines for animating the eye and head from the perspective of a character animator.     

Video-realistic image-based eye animation via statistically driven state machines

In this work we elaborate on a novel image-based system for creating video-realistic eye animations to arbitrary spoken output. These animations are useful to give a face to multimedia applications such as virtual operators in dialog systems. Our eye animation system consists of two parts: eye control unit and rendering engine, which synthesizes eye animations by combining 3D and image-based models. The designed eye control unit is based on eye movement physiology and the statistical analysis of recorded human subjects. As already analyzed in previous publications, eye movements vary while listening and talking. We focus on the latter and are the first to design a new model which fully automatically couples eye blinks and movements with phonetic and prosodic information extracted from spoken language. We extended the already known simple gaze model by refining mutual gaze to better model human eye movements. Furthermore, we improved the eye movement models by considering head tilts, torsion, and eyelid movements. Mainly due to our integrated blink and gaze model and to the control of eye movements based on spoken language, subjective tests indicate that participants are not able to distinguish between real eye motions and our animations, which has not been achieved before.     

基于Mean Shift算法和粒子滤波器的人眼跟踪

基于视觉的驾驶疲劳检测是人脸表情识别技术最有商业前途的应用之一,实时人眼跟踪是其中的关键部分。为了解决跟踪方法对眼睛的部分遮挡、人脸尺度变化等过于敏感的问题,提出了一种综合MeanShift算法和粒子滤波器的跟踪算法。利用粒子滤波器得到样本的观测值后,将MeanShift分析用于每一个粒子,使得粒子集中在测量模型的局部区域内,很好地克服了粒子滤波器的退化现象并有效缩短了计算时间。实验结果表明该算法实时性强,且具有良好的鲁棒性。     

基于车道控制的交叉口流向及信号组合优化模型*

为了提升城市道路交叉口运行效益,提出将流向禁止与信号控制相结合的交叉口优化模型。采用组合优化的方法,将禁止流向选择、车道功能、路段掉头位置以及信号配时等优化变量整合在一个统一的模型中,同步进行优化。该优化模型由混合整数非线性规划描述,并将其转换为一系列混合整数线性规划,采用分支定界法求解。通过算例及仿真分析,验证模型的效益。研究结果表明,禁止流向的选择对于交叉口运行效益有着很大影响,优化模型可根据交叉口几何及交通需求条件,生成最优方案。尤其当常规交叉口无法满足交通需求时,该优化方法可提高通行能力达25%。     

What's skill got to do with it? Vehicle automation and driver mental workload

Previous research has found that vehicle automation systems can reduce driver mental workload, with implications for attentional resources that can be detrimental to performance. The present paper considers how the development of automaticity within the driving task may influence performance in underload situations. Driver skill and vehicle automation were manipulated in a driving simulator, with four levels of each variable. Mental workload was assessed using a secondary task measure and eye movements were recorded to infer attentional capacity. The effects of automation on driver mental workload were quite robust across skill levels, but the most intriguing findings were from the eye movement data. It was found that, with little exception, attentional capacity and mental workload were directly related at all levels of driver skill, consistent with earlier studies. The results are discussed with reference to applied theories of cognition and the design of automation.