Using Haar classifiers to detect driver fatigue and provide alerts

Drowsiness is a transition state between being awake and asleep and can have serious consequences when occurring in tasks that require sustained attention such as driving. During the state of drowsiness, reaction time is slower, vigilance is reduced, and information processing is less efficient, which may cause accidents. The proposed Driver Fatigue Detection System (called FDS) aims to monitor the alertness of drivers to prevent them from falling asleep at the wheel. The system monitors the driver’s face using Haar feature classifiers with an increased training set to detect changes in the face of the driver quickly. A correlation matching algorithm is used to accurately provide the target’s position and track the target’s eyes according to the intensity, shape, and size of the pupils. FDS uses an IR illuminator to produce the desired bright pupil effect when the driver is wearing sunglasses. The resulting system operates in real-time, and is more accurate and less intrusive to the driver than other systems currently available.     

基于多特征融合的眼睛状态检测算法研究

疲劳驾驶检测算法研究对提升交通安全有着重要的意义.目前,已有大量关于疲劳驾驶的文献和成果.在疲劳驾驶检测算法中,眼睛开闭状态的判断起着至关重要的作用.深度级联卷积神经网络用来检测人脸和人脸特征,利用Dlib工具快速提取驾驶员人脸特征.基于眼睛特征计算眼睛宽高比,并将眼睛宽高比、传统人眼特征的人眼虹膜等用于判断眼睛开闭的...     

Driver drowsiness detection with eyelid related parameters by Support Vector Machine

Various investigations show that drivers’ drowsiness is one of the main causes of traffic accidents. Thus, countermeasure device is currently required in many fields for sleepiness related accident prevention. This paper intends to perform the drowsiness prediction by employing Support Vector Machine (SVM) with eyelid related parameters extracted from EOG data collected in a driving simulator provided by EU Project SENSATION. The dataset is firstly divided into three incremental drowsiness levels, and then a paired t-test is done to identify how the parameters are associated with drivers’ sleepy condition. With all the features, a SVM drowsiness detection model is constructed. The validation results show that the drowsiness detection accuracy is quite high especially when the subjects are very sleepy.     

基于脉搏波检测的驾驶员睡意预警装置

随着驾驶员人数的不断增加,不文明的驾驶行为也越来越多,其中由于疲劳驾驶所引发的交通事故占据相当大的比例,给人民的生命和财产造成了巨大的损失,因此,对于驾驶员睡意预警装置的技术研究具有非常重要的意义和实用价值。通过对人体脉搏波信号进行分析处理,采用能够反映驾驶员睡意状态的脉搏频率特征信号作为依据,由STC89C52单片机、按键、数码管、光电传感器、时钟模块、滤波电路、集成运放等构成系统,设计了驾驶员睡意预警装置。调试结果显示该装置识别准确率高,数值可靠,能够有效的检测驾驶员的睡意状态,并在睡意状态时发出预警。对比市场同类型产品,该装置具有成本低廉,操作简单,能够实现车载等特点,为驾驶员睡意预警技术的相关研究提供了一定的技术和实验基础。     

Music classification as a new approach for malware detection

Each year, a huge number of malicious programs are released which causes malware detection to become a critical task in computer security. Antiviruses use various methods for detecting malware, such as signature-based and heuristic-based techniques. Polymorphic and metamorphic malwares employ obfuscation techniques to bypass traditional detection methods used by antiviruses. Recently, the number of these malware has increased dramatically. Most of the previously proposed methods to detect malware are based on high-level features such as opcodes, function calls or program’s control flow graph (CFG). Due to new obfuscation techniques, extracting high-level features is tough, fallible and time-consuming; hence approaches using program’s bytes are quicker and more accurate. In this paper, a novel byte-level method for detecting malware by audio signal processing techniques is presented. In our proposed method, program’s bytes are converted to a meaningful audio signal, then Music Information Retrieval (MIR) techniques are employed to construct a machine learning music classification model from audio signals to detect new and unseen instances. Experiments evaluate the influence of different strategies converting bytes to audio signals and the effectiveness of the method.

     

基于边缘计算的疲劳驾驶检测方法

现有疲劳驾驶检测方法通常将驾驶过程中采集的数据传输至云端进行分析,然而在车辆移动过程中网络覆盖范围、响应速度等因素会造成检测实时性差。为在车载嵌入式设备上对驾驶人疲劳状态进行准确预警,提出一种基于边缘计算的疲劳驾驶检测方法。通过改进的多任务卷积神经网络确定人脸区域,根据人脸的面部比例关系定位驾驶人的眼部与嘴部区域,利用基于Ghost模块的轻量化AlexNet分类检测眼部与嘴部的开闭状态,并结合PERCLOS和PMOT指标值实现疲劳检测。在NHTU-DDD数据集上的实验结果表明,该方法在树莓派4B开发板上的检测准确率达到93.5%且单帧平均检测时间为180 ms,在保障检测准确率的同时大幅降低了计算量,能较好地满足疲劳驾驶的实时检测需求。     

Spatio-temporal filtering of thermal video sequences for heart rate estimation

In this paper, a novel method for non-contact measurement of heart rate using thermal imaging was proposed. Thermal videos were recorded from subjects’ faces. The measurements are performed on three different areas: the whole face, the upper half of the face and the supraorbital region. A tracker was used to track these regions to make the algorithm invulnerable to the subject's motion. After tracking, the videos were spatially filtered using a full Laplacian pyramid decomposition to increase the signal to noise ratio; next, the video frames were successively temporally filtered using an ideal bandpass filter for extracting the thermal variations caused by blood circulation. Finally, the heart rate was calculated by using two methods including zero crossing and Fast Fourier Transform. For evaluating the results, the complement of absolute normalized difference (CAND) index was used which was introduced by Pavlidis. This index was 99.42% in the best case and 92.472% in average for 22 subjects. These results showed a growth in CAND index in comparison with previous work. Zerocrossing outperformed FFT because of the nonstationary nature of thermal signals. Another benefit of our method is that, the videos are taken from the face unlike most of the studies that take it from the neck and Carotid. Neck and carotid are less accessible than faces. Finally, the optimum ROI for estimating the heart rate from face was identified.     

基于眼部特征的疲劳检测算法

PERCLOS值因其良好的非接触性和准确性而被广泛应用于疲劳检测,但通常只采用一种PERCLOS标准.针对这种情况,该文提出眼睛持续闭合时间和动态 PERCLOS 值两个参数进行疲劳检测.该算法首先利用Haar-like 分类器和 Adaboost 算法进行人脸检测和定位;然后利用人脸结构特征缩小人眼的搜索区域,进一步利用 Adaboost 算法定位人眼,避免了眉毛的影响;最后采用图像形态学等图像处理方法获取人眼的垂直高度即上下眼帘的距离,判断人眼是否闭合.在疲劳预测阶段,分时间段采用不同的 PERCLOS 值标准进行判断.该算法对每秒10帧视频帧中的人眼定位准确率达到86.14%,并达到实时性要求,能够提高预测疲劳驾驶的准确性.     

基于驾驶员面部时序数据的疲劳驾驶检测算法

针对传统疲劳驾驶检测方法识别准确率低、泛化能力差的问题,提出了一种基于CNNs和LSTM的端到端可训练网络,检测驾驶员的疲劳状态。根据驾驶员面部特征点提取ROI,将在其他计算机视觉任务上表现较好的深度网络迁移到疲劳检测任务中,并结合LSTM处理时序数据的能力,提出一种新的疲劳检测网络,该网络能够读入视频流中的时序数据并检测出驾驶员的疲劳状态。实验证明所提方法和模型在公开数据集中具有较高的识别准确率,并且在不同的数据集间具有很好的泛化能力,对于减少路面车祸、保障人身安全具有很重要的意义。     

Performance analysis of power and power variance for classification,detection and localization of epileptic multi-channel EEG

There are a large number of data sets of EEG signal for which, it is difficult to judge and monitor brain activity through observations. Epilepsy is a disorder in which a recurrent and sudden malfunction of the brain is characterized. It is proposed to classify, detect and localize Epileptic multi-channel EEG through various power and novel power variance features non-invasively. This work presents power spectral estimation (PSE) using time–frequency analysis of EEG signals in both parametric (FFT) and non-parametric methods (i.e. Welch, Burg, Covariance, MUSIC and Yule–Walker). To examine the robustness of power features for different methods, the analysis of p value is performed. The detection of epileptic seizure is classified using different kernels through SVM. It is observed from the PSE that the power features have higher values in epileptic subjects as compared to non-epileptic subjects. Amongst all the parametric and non-parametric methods, the MUSIC method gives the highest average power. Sensitivity, specificity, and classification accuracy are 100% for Welch, Burg, Covariance, and Yule–Walker methods while MUSIC and FFT methods deliver 98.73 and 99.52% respectively. The novelty is introduced through the quantification of power and power variance robust feature region/lobe-wise. This quantification is used for the localization of 25 epileptic subjects. Analysis of the parametric and non-parametric PSD methods for extraction of power and power variance features is not used by any study. These are effectively utilized for detection and localization of epilepsy non-invasively.

     

Real-time shape tracking of facial landmarks

Detection of facial landmarks and accurate tracking of their shape are essential in real-time applications such as virtual makeup, where users can see the makeup’s effect by moving their face in diverse directions. Typical face tracking techniques detect facial landmarks and track them using a point tracker such as the Kanade-Lucas-Tomasi (KLT) point tracker. Typically, 5 or 64 points are used for tracking a face. Even though these points are enough to track the approximate locations of facial landmarks, they are not sufficient to track the exact shape of facial landmarks. In this paper, we propose a method that can track the exact shape of facial landmarks in real-time by combining a deep learning technique and a point tracker. We detect facial landmarks accurately using SegNet, which performs semantic segmentation based on deep learning. Edge points of detected landmarks are tracked using the KLT point tracker. In spite of its popularity, the KLT point tracker suffers from the point loss problem. We solve this problem by executing SegNet periodically to recalculate the shape of facial landmarks. That is, by combining the two techniques, we can avoid the computational overhead of SegNet and the point loss problem of the KLT point tracker, which leads to accurate real-time shape tracking. We performed several experiments to evaluate the performance of our method and report some of the results herein.

     

非合作面部晃动情况下的心率检测

目的 心率是直接反映人体健康的重要指标之一,基于视频的非接触式心率检测在医疗健康领域具有广泛的应用前景。然而,现有的基于视频的方法不适用于复杂的现实场景,主要原因是没有考虑视频中目标晃动干扰和空间尺度特征,使得血液容积脉冲信号提取不准确,检测精度不尽人意。为了克服以上缺陷,提出一种抗人脸晃动干扰的非接触式心率检测方法。方法 本文方法主要包含3个步骤：首先,针对目标晃动干扰人脸区域选择的问题,利用判别响应图拟合检测参考图像的人脸区域及主要器官特征点,在人脸跟踪时首次引入倾斜校正思想,输出晃动干扰抑制后的人脸视频;然后,结合空间尺度的差异,采用颜色放大方法对晃动干扰抑制后的人脸视频进行时空处理,提取干净的血液容积脉冲信号;最后,考虑到小样本问题,通过傅里叶系数迭代插值的频域分析方法估计心率。结果 在人脸静止的合作情况以及人脸晃动的非合作情况下采集视频,对心率检测结果进行定量分析,本文方法在两种情况下的准确率分别为97.84%和97.30%,与经典和最新的方法相比,合作情况准确率提升大于1%,非合作情况准确率提升大于7%,表现了出色的性能。结论 提出了一种基于人脸视频处理的心率检测方法,通过有效分析人脸的晃动干扰和尺度特性,提取到干净的血液容积脉冲信号,提高了心率检测的精度和鲁棒性。     

基于眼部自商图—梯度图共生矩阵的疲劳驾驶检测

目的 疲劳驾驶是引发车辆交通事故的主要原因之一,针对现有方法在驾驶员面部遮挡情况下对眼睛状态识别效果不佳的问题,提出了一种基于自商图—梯度图共生矩阵的驾驶员眼部疲劳检测方法。方法 利用以残差网络（residual network,ResNet）为前置网络的SSD（single shot multibox detector）人脸检测器来获取视频中的有效人脸区域,并通过人脸关键点检测算法分割出眼睛局部区域图像;建立驾驶员眼部的自商图与梯度图共生矩阵模型,分析共生矩阵的数字统计特征,选取效果较好的特征用以判定人眼的开闭状态;结合眼睛闭合时间百分比（percentage of eyelid closure,PERCLOS）与最长闭眼持续时间（maximum closing duration,MCD）两个疲劳指标来判别驾驶员的疲劳状态。结果 在六自由度汽车性能虚拟仿真实验平台上模拟汽车驾驶,采集并分析驾驶员面部视频,本文方法能够有效识别驾驶员面部遮挡时眼睛的开闭状态,准确率高达99.12%,面部未遮挡时的识别精度为98.73%,算法处理视频的速度约为32帧/s。对比方法1采用方向梯度直方图特征与支持向量机分类器相结合的人脸检测算法,并以眼睛纵横比判定开闭眼状态,在面部遮挡时识别较弱;以卷积神经网络（convolutional neural network,CNN）判别眼睛状态的对比方法2虽然在面部遮挡情况下的准确率高达98.02%,但眨眼检测准确率效果不佳。结论 基于自商图—梯度图共生矩阵的疲劳检测方法能够有效识别面部遮挡时眼睛的开闭情况和驾驶员的疲劳状态,具有较快的检测速度与较高的准确率。     

A multimodal fusion method for sarcasm detection based on late fusion

Information on social media is multi-modal, most of which contains the meaning of sarcasm. In recent years, many people have studied the problem of sarcasm detection. Many traditional methods have been proposed in this field, but the study of deep learning methods to detect sarcasm is still insufficient. It is necessary to comprehensively consider the information of the text,the changes of the tone of the audio signal,the facial expressions and the body posture in the image to detect sarcasm. This paper proposes a multi-level late-fusion learning framework with residual connections, a more reasonable experimental data-set split and two model variants based on different experimental settings. Extensive experiments on the MUStARD show that our methods are better than other fusion models. In our speaker-independent experimental split, the multi-modality has a 4.85% improvement over the single-modality, and the Error rate reduction has an improvement of 11.8%. The latest code will be updated to this URL later: https://github.com/DingNing123/m_fusion

     

基于时序性面部动作信息的驾驶员状态检测框架

通过网络摄像头获取驾驶员面部视频输入网络进行检测的方法主要通过分析驾驶员口型等面部表情来判断是否疲劳驾驶,但说话等很多类似的状态也被误检为疲劳。针对以上问题提出了一种基于时序性面部动作信息的检测框架,对驾驶员状态进行检测,从而提高检测准确率、降低误检率。该框架通过检测视频中的脸部轮廓,提取脸部的多种特征,形成面部动作单元;通过训练对应的LSTM网络,形成时序性的面部动作单元,根据其相关性进行多种动作单元融合,检测最终驾驶员的状态。在公共YawDD数据集上的检测结果表明,相比于现有的方法,该检测方法的准确率提高到了93.1%,同时大幅降低了疲劳状态的误检率。     

Quantifying influence from form manipulation of artificial facial expression to viewers

Abstract

Creating facial expressions manually needs to be enhanced through a set of easy operating rules, which involves adjectives and the manipulating combinations of facial design elements. This article tries to analyse viewers' cognition of artificial facial expressions in an objective and scientific way. We chose four adjectives – ‘satisfied’, ‘sarcastic’, ‘disdainful’, and ‘nervous’ – as the experimental subjects. The manipulative key factors of facial expressions (eyebrows, eyes, pupils, mouth and head rotation) were used to create permutations and combinations and to make 81 stimuli of different facial expressions with a 3-D face model in order to conduct a survey. Next, we used Quantification Theory Type I to find the best combinations that participants agreed on as representing these adjectives. The conclusions of this research are that: (1) there are differences in adopting facial features between creating artificial characters' expressions and recognising real humans' expressions; (2) using survey and statistics can scientifically analyse viewers' cognition of facial expressions with form changing; and (3) the results of this research can promote designers' efficiency in working with subtler facial expressions.     

Driver’s eye-based gaze tracking system by one-point calibration

The accuracies of driver’s gaze detection by previous researches are affected by the various sitting positions and heights of drivers in case that initial calibration of driver is not performed. By using dual cameras, the driver’s calibration can be omitted, but processing time with complexity is increased. In addition, the problem of disappearing corneal specular reflection (SR) in the eye image as the driver severely turns his/her head has not been dealt in previous researches. To consider these issues, we propose a gaze tracking method based on driver’s one-point calibration using both corneal SR and medial canthus (MC) based on maximum entropy criterion. An experiment with collected data from 26 subjects (wearing nothing, glasses, sunglasses, hat, or taking various hand pose) in a vehicle, showed that the accuracy of the proposed method is higher than that of other gaze tracking methods. In addition, we showed the effectiveness of our method in the real driving environment.

     

Multivariate autoregressive models and kernel learning algorithms for classifying driving mental fatigue based on electroencephalographic

Long-term driving is a significant cause of fatigue-related accidents. Driving mental fatigue has major implications for transportation system safety. Monitoring physiological signal while driving can provide the possibility to detect the mental fatigue and give the necessary warning. In this paper an EEG-based fatigue countermeasure algorithm is presented to classify the driving mental fatigue. The features of multichannel electroencephalographic (EEG) signals of frontal, central and occipital are extracted by multivariate autoregressive (MVAR) model. Then kernel principal component analysis (KPCA) and support vector machines (SVM) are employed to identify three-class EEG-based driving mental fatigue. The results show that KPCA–SVM method is able to effectively reduce the dimensionality of the feature vectors, speed up the convergence in the training of SVM and achieve higher recognition accuracy (81.64%) of three driving mental fatigue states in 10 subjects. The KPCA–SVM method could be a potential tool for classification of driving mental fatigue.