An improved likelihood model for eye tracking

While existing eye detection and tracking algorithms can work reasonably well in a controlled environment, they tend to perform poorly under real world imaging conditions where the lighting produces shadows and the person’s eyes can be occluded by e.g. glasses or makeup. As a result, pixel clusters associated with the eyes tend to be grouped together with background-features. This problem occurs both for eye detection and eye tracking. Problems that especially plague eye tracking include head movement, eye blinking and light changes, all of which can cause the eyes to suddenly disappear. The usual approach in such cases is to abandon the tracking routine and re-initialize eye detection. Of course this may be a difficult process due to missed data problem. Accordingly, what is needed is an efficient method of reliably tracking a person’s eyes between successively produced video image frames, even in situations where the person’s head turns, the eyes momentarily close and/or the lighting conditions are variable. The present paper is directed to an efficient and reliable method of tracking a human eye between successively produced infrared interlaced image frames where the lighting conditions are challenging. It proposes a log likelihood-ratio function of foreground and background models in a particle filter-based eye tracking framework. It fuses key information from even, odd infrared fields (dark and bright-pupil) and their corresponding subtractive image into one single observation model. Experimental validations show good performance of the proposed eye tracker in challenging conditions that include moderate head motion and significant local and global lighting changes. The paper presents also an eye detector that relies on physiological infrared eye responses and a modified version of a cascaded classifier.     

基于运动注意力融合模型的目标检测与提取算法

针对全局运动场景下目标检测与提取方法的局限性,文中根据运动注意力形成机理,构建一种运动注意力时-空融合模型用于运动目标的检测与提取。该算法首先对运动矢量场进行叠加和滤波等预处理。然后根据运动矢量在时间和空间上的变化特点定义运动注意力融合模型,并采用该模型检测运动目标区域。最后利用形态学和边界跟踪方法对目标区域进行精确化提取。根据多个不同全局运动视频场景的测试结果,显示该算法比其它算法具有更好的准确性和实时性。     

融合运动特征和深度学习的跌倒检测算法

为了利用计算机视觉技术准确检测老年人的跌倒状况,针对现有跌倒检测算法中人为设计特征造成的不完备性以及跌倒检测过程中前后景分离困难、目标混淆、运动目标丢失、跌倒检测准确率低等问题,提出了一种融合人体运动信息的深度学习跌倒检测算法对人体跌倒状态进行检测.首先,通过改进YOLOv3网络进行前景与背景的分离,并根据YOLOv3...     

一种基于块的图像边缘分析方法

边缘检测是图像处理和计算机视觉中的基本问题,人眼视觉系统对图像中的边缘信息非常敏感,经常作为描述图像特征的一种重要手段。基于运动补偿和DCT变换编码的视频编码标准中,一幅图像分成大小相同不重叠的块进行编码,基于块的图像边缘分析在图像处理、去块效应滤波、模式选择以及基于内容的视频检索等方面有较广泛应用。提出了一种基于分析块边缘方向的边缘分析算法。实验结果表明,与其他算法相比,该算法在分析性能与计算复杂方面具有较明显的优越性。     

人体运动分析研究的若干新进展

人体运动视觉分析主要包括运动目标检测、运动目标分类、人体运动跟踪、人体行为识别与描述四个环节,在多领域具有广阔的应用前景.本文从上述四个方面综述了人体运动分析的研究现状,对人体运动分析的热点难点进行讨论,对可能的发展方向进行阐述和展望.     

基于音视一致性的音视人眼关注点检测

现有音视人眼关注点检测算法使用双流结构分别对音视信息进行特征提取,随后对音视特征融合得到最终的预测图。但数据集中的音频信息和视觉信息会有不相关的情况,因此在音视不一致时直接对音视特征进行融合会使得音频信息对视觉特征产生消极的影响。针对上述问题,本文提出一种基于音视一致性的音视人眼关注点检测网络（Audio-visual Consistency Network, AVCN）。为验证该网络的可靠性,本文在现有音视结合的人眼关注点检测模型上加入音视一致性网络,AVCN对提取的音、视频特征进行一致性二值判断,二者一致时,输出音视融合特征作为最终的预测图,反之则输出视觉占主导的特征作为最终结果。该算法在开放的6个数据集上进行实验,结果表明加入AVCN模型的整体指标会有所提高。     

On implementing motion-based Region of Interest detection on multi-core CELL

Region of Interest (ROI) detection is a well-studied problem in computer vision for applications such as video surveillance and vision-based robotics. ROI detection may be done using background subtraction schemes with change detection and background estimation. When the camera is not static, these schemes will be ineffective and hence there is a need for global motion estimation (GME) to compensate the camera motion. Robust GME algorithms often require high computation power, rendering them unsuitable for real-time, embedded vision applications. In this article, we use a multi-core processor platform – CELL, to meet the computational requirements of the ROI detection system and to explore the feasibility of potential usage of such heterogeneous processor architecture for vision applications. In particular, we analyze the algorithmic components of a typical GME-based ROI detection system and show how to make efficient use of the parallel and vector computation capabilities in the CELL cores for maximizing the gain on speed performance. We have also ported our system on a Sony PS3 system and promising results have been achieved. Based on the study, various design aspects and implementation challenges are discussed which are believed to be useful for future work in porting vision algorithms on multi-core architectures for real-time embedded applications.     

Intelligent visual surveillance — A survey

Detection, tracking, and understanding of moving objects of interest in dynamic scenes have been active research areas in computer vision over the past decades. Intelligent visual surveillance (IVS) refers to an automated visual monitoring process that involves analysis and interpretation of object behaviors, as well as object detection and tracking, to understand the visual events of the scene. Main tasks of IVS include scene interpretation and wide area surveillance control. Scene interpretation aims at detecting and tracking moving objects in an image sequence and understanding their behaviors. In wide area surveillance control task, multiple cameras or agents are controlled in a cooperative manner to monitor tagged objects in motion. This paper reviews recent advances and future research directions of these tasks. This article consists of two parts: The first part surveys image enhancement, moving object detection and tracking, and motion behavior understanding. The second part reviews wide-area surveillance techniques based on the fusion of multiple visual sensors, camera calibration and cooperative camera systems.     

Eye detection in a facial image under pose variation based on multi-scale iris shape feature

The accurate location of eyes in a facial image is important to many human facial recognition-related applications, and has attracted considerable research interest in computer vision. However, most prevalent methods are based on the frontal pose of the face, where applying them to non-frontal poses can yield erroneous results.In this paper, we propose an eye detection method that can locate the eyes in facial images captured at various head poses. Our proposed method consists of two stages: eye candidate detection and eye candidate verification. In eye candidate detection, eye candidates are obtained by using multi-scale iris shape features and integral image. The size of the iris in face images varies as the head pose changes, and the proposed multi-scale iris shape feature method can detect the eyes in such cases. Since it utilizes the integral image, its computational cost is relatively low. The extracted eye candidates are then verified in the eye candidate verification stage using a support vector machine (SVM) based on the feature-level fusion of a histogram of oriented gradients (HOG) and cell mean intensity features.We tested the performance of the proposed method using the Chinese Academy of Sciences' Pose, Expression, Accessories, and Lighting (CAS-PEAL) database and the Pointing'04 database. The results confirmed the superiority of our method over the conventional Haar-like detector and two hybrid eye detectors under relatively extreme head pose variations.     

局部与全局相融合的孪生低照度视频增强网络

在低照度环境下拍摄到的视频往往有对比度低、噪点多、细节不清晰等问题, 严重影响后续的目标检测、分割等计算机视觉任务. 现有的低照度视频增强方法大都是基于卷积神经网络构建的, 由于卷积无法充分利用像素之间的长程依赖关系, 生成的视频往往会有部分区域细节丢失、颜色失真的问题. 针对上述问题, 提出了一种局部与全局相融合的孪生低照度视频增强网络模型, 通过基于可变形卷积的局部特征提取模块来获取视频帧的局部特征, 并且设计了一个轻量级自注意力模块来捕获视频帧的全局特征, 最后通过特征融合模块对提取到的局部特征和全局特征进行融合, 指导模型能生成颜色更真实、更具细节的增强视频. 实验结果表明, 本方法能有效提高低照度视频的亮度, 生成颜色和细节都更丰富的视频, 并且在峰值信噪比和结构相似性等评价指标中也都优于近几年提出的方法.     

基于多尺度分析的运动注意力计算方法研究

由于光流估算的缺陷、噪声干扰以及现有运动注意力模型的局限性,导致运动注意力计算结果不能准确反映运动的显著性特征。本文提出了一种基于多尺度分析的运动注意力计算方法,该方法根据视觉注意力形成机理构建运动注意力模型;然后通过时间尺度滤波去除噪声影响;鉴于视觉观测对尺度的依赖性,进行空间多尺度动注意力融合,最终得到运动注意力计算结果。测试结果表明本文方法比同类方法更能真实有效地反映出视频场景中的运动显著性特征。     

Detection of swimmer using dense optical flow motion map and intensity information

A vision-based system that can locate individual swimmers and recognize the activities is applicable for swimming gait analysis, drowning event detection, etc. The system relies on accurate detection of swimmer’s body parts such as head and upper limbs. The swimmer detection problem can be regarded as background subtraction. Swimmer detection in the aquatic environment is very difficult due to a dynamic background with water ripples, splashes, specular reflections, etc. This paper presents a swimmer detection method which utilizes both local motion and intensity information estimated from the image sequence. Local motion information is obtained by computing dense optical flow and periodogram. We adopt a heuristic approach to generate a motion map characterizing the local motions (random/stationary, ripple or swimming) of image pixels over a short duration. Intensity information is modeled as a mixture of Gaussians. Finally, using the motion map and the Gaussian models, swimmers are detected in each video frame. We test the method on video sequences captured at daytime, and nighttime, and of different swimming styles (breaststroke, freestyle, backstroke). Our method can detect swimmers much better than that using intensity information alone. In addition, we compare our method with existing algorithms—codebook model and self-organizing artificial neural networks. The methods are tested on publicly available video sequence and our swimming video sequence. We show through the quantitative measures the superiority of our method.     

A survey on compressed domain video analysis techniques

Image and video analysis requires rich features that can characterize various aspects of visual information. These rich features are typically extracted from the pixel values of the images and videos, which require huge amount of computation and seldom useful for real-time analysis. On the contrary, the compressed domain analysis offers relevant information pertaining to the visual content in the form of transform coefficients, motion vectors, quantization steps, coded block patterns with minimal computational burden. The quantum of work done in compressed domain is relatively much less compared to pixel domain. This paper aims to survey various video analysis efforts published during the last decade across the spectrum of video compression standards. In this survey, we have included only the analysis part, excluding the processing aspect of compressed domain. This analysis spans through various computer vision applications such as moving object segmentation, human action recognition, indexing, retrieval, face detection, video classification and object tracking in compressed videos.     

An on-board vision sensor system for small unmanned vehicle applications

This paper describes an on-board vision sensor system that is developed specifically for small unmanned vehicle applications. For small vehicles, vision sensors have many advantages, including size, weight, and power consumption, over other sensors such as radar, sonar, and laser range finder, etc. A vision sensor is also uniquely suited for tasks such as target tracking and recognition that require visual information processing. However, it is difficult to meet the computing needs of real-time vision processing on a small robot. In this paper, we present the development of a field programmable gate array-based vision sensor and use a small ground vehicle to demonstrate that this vision sensor is able to detect and track features on a user-selected target from frame to frame and steer the small autonomous vehicle towards it. The sensor system utilizes hardware implementations of the rank transform for filtering, a Harris corner detector for feature detection, and a correlation algorithm for feature matching and tracking. With additional capabilities supported in software, the operational system communicates wirelessly with a base station, receiving commands, providing visual feedback to the user and allowing user input such as specifying targets to track. Since this vision sensor system uses reconfigurable hardware, other vision algorithms such as stereo vision and motion analysis can be implemented to reconfigure the system for other real-time vision applications.     

融合双目多维感知特征的立体视频显著性检测

目的 立体视频能提供身临其境的逼真感而越来越受到人们的喜爱,而视觉显著性检测可以自动预测、定位和挖掘重要视觉信息,可以帮助机器对海量多媒体信息进行有效筛选。为了提高立体视频中的显著区域检测性能,提出了一种融合双目多维感知特性的立体视频显著性检测模型。方法 从立体视频的空域、深度以及时域3个不同维度出发进行显著性计算。首先,基于图像的空间特征利用贝叶斯模型计算2D图像显著图;接着,根据双目感知特征获取立体视频图像的深度显著图;然后,利用Lucas-Kanade光流法计算帧间局部区域的运动特征,获取时域显著图;最后,将3种不同维度的显著图采用一种基于全局-区域差异度大小的融合方法进行相互融合,获得最终的立体视频显著区域分布模型。结果 在不同类型的立体视频序列中的实验结果表明,本文模型获得了80%的准确率和72%的召回率,且保持了相对较低的计算复杂度,优于现有的显著性检测模型。结论 本文的显著性检测模型能有效地获取立体视频中的显著区域,可应用于立体视频/图像编码、立体视频/图像质量评价等领域。     

Glaucoma visual field quantification with eye tracker

Glaucoma is a group of diseases that manifest as atrophy and depression of the optic papilla, visual field defects, and vision loss, representing one of the three leading causes of blindness worldwide. Traditional visual field examinations – an important diagnostic tool for glaucoma – present various challenges including patients’ inability to maintain fixed vision, delays in detecting vision loss, passive position detection, difficulty in detection, and limitations in reflecting physiological visual field damage. Early diagnosis and intervention are crucial for improving patients’ condition and enhancing their later-life abilities and life quality. Herein, we proposed two vision field detection systems to overcome these limitations. First, we establish a dynamic visual field detection system to reduce the complexity of traditional detection experiments and to enhance their operability. Instead of fixating on a central point, subjects are only required to search for the target in the picture. We analyze the heat map and trajectory map of visual attention for visual interpretation, and the analysis of experimental data reveals that the average finding time of subjects in the experimental task varies. In response to the scenario where visual field defects are not detected by the dynamic visual field detection system, we have developed a static visual field detection system based on the former. The system obtains eye movement data and automatically generates a map of the extent of the physiological blind spot without any action required from the patient. The experiment results provide evidence for the effectiveness of the static visual field detection system in detecting the physiological blind spot. Given the well-established association between glaucoma and an enlarged physiological blind spot, the use of an eye tracker to assess the extent of the subject’s blind spot represents an easy-to-use and reliable method for preliminary glaucoma screening.     

基于多特征融合的视频交通数据采集方法

提出了一种基于多特征融合的视频交通数据采集方法, 核心思想是: 在图像中设置虚拟线圈, 假设车辆从虚拟线圈上驶过时引起像素变化, 通过识别这种像素变化来检测车辆并估计车速. 与现有技术相比, 本文的贡献在于: 1) 综合利用虚拟线圈内的前景面积、纹理变化、像素运动等特征来检测车辆, 提出了有效的多特征融合方法, 显著提高了车辆检测精度; 2) 根据单个虚拟线圈内的像素运动向量来估计车速, 避免了双线圈测速法的错误匹配问题. 算法测试结果表明本文算法能够在复杂多样的交通场景和天气条件下, 准确地检测车辆和估计车速. 在算法研究的基础上, 研制了一款嵌入式交通视频检测器, 在路口长期采集交通数据, 为交通信号控制和交通规律分析提供决策依据.     

视频序列中人体运动目标的检测与跟踪研究

提出一种视频序列中人体运动目标的精确检测、提取以硬跟踪算法。该算法采用帧间差闽值法（简称TIFD）实现快速精确地检测和提取目标,使用扩展的Kalman滤波器预测运动目标下一时刻可能处于的区域,缩小了目标跟踪时的搜索范围。充分利用运行目标检测的结果,提高了目标的匹配效率及跟踪速度。同时给出了相应的实验结果,结果表明方法是比较实用的,能满足人体运动分析的基本要求。