Combination of local feature detection methods for digital video stabilization

Technological advances in compact and portable cameras have enabled the generation of large volumes of video sequences. However, videos captured by amateurs are subject to unwanted vibrations due to camera shaking. To overcome such problem, video stabilization aims to remove undesired motion from videos to enhance visual quality, improving applications such as detection and tracking of objects. In this work, we develop and analyze a consensual method for combining a set of local feature techniques for camera motion estimation. Several video sequences are used to evaluate the proposed methodology. Experimental results demonstrate the effectiveness of the combination method over individual local feature approaches.     

Savitzky-Golay滤波和L1范数优化相结合的视频稳像算法

近年来,越来越多的人利用手持设备拍摄视频表达自己或社会问题等,这类视频常存在抖动现象,如何消除视频序列中的随机抖动成为近年来的研究热点。本文面向视频去抖动应用场景,提出了一种Savitzky-Golay滤波和L1范数优化相结合的视频稳像算法。首先基于帧间配对特征点进行运动估计;然后利用Savitzky-Golay滤波算法进行路径平滑处理,去除运动路径中的高频抖动分量,避免了噪声对后续生成稳定路径的影响;进一步,对平滑后的有效路径信息进行基于优化的L1范数优化算法拟合,得到最终的稳定路径;最后对原始视频图像序列进行运动补偿,生成稳定的图像序列。对比实验结果表明,本文方法稳像效果优于原始L1范数优化算法,算法速度快于目前效果较好的二维稳像算法。      

基于子块运动补偿的运动目标检测

鱼眼相机成像视角大,获得信息丰富,在车载应用中具有广阔应用前景.本文提出了一种适用于移动单目鱼眼相机的运动目标检测方法.首先,提出一种子块运动补偿模型补偿图像背景运动,解决了现有运动补偿模型对强视差背景补偿效果不好的问题.其次,在子块运动补偿模型参数求解时,通过引入自车运动参数简化模型参数个数,并结合直接方法求解,避免了传统基于特征点匹配方法求解参数时易受误匹配特征点影响的问题.然后,针对鱼眼相机的成像形变问题,本文提出了一种三平面校正方法获取鱼眼图像的子块运动补偿图像.最后,利用鱼眼图像的子块运动补偿图像和真实拍摄图像的差异信息实现运动目标检测.多种测试场景下的实验结果表明了本文方法的有效性.     

Mosaic-based 3D scene representation and rendering

In this paper we address the problem of fusing images from many video cameras or a moving video camera. The captured images have obvious motion parallax, but they will be aligned and integrated into a few mosaics with a large field-of-view (FOV) that preserve 3D information. We have developed a compact geometric representation that can re-organize the original perspective images into a set of parallel projections with different oblique viewing angles. In addition to providing a wide field of view, mosaics with various oblique views well represent occlusion regions that cannot be seen in a usual nadir view. Stereo mosaic pairs can be formed from mosaics with different oblique viewing angles. This representation can be used as both an advanced interface for interactive 3D video and a pre-processing step for 3D reconstruction. A ray interpolation approach for generating the parallel-projection mosaics is presented, and efficient 3D scene/object rendering based on multiple parallel-projection mosaics is discussed. Several real-world examples are provided, with applications ranging from aerial video surveillance/environmental monitoring, ground mobile robot navigation, to under-vehicle inspection.     

家用摄录机的电子图像稳定系统

本文介绍采用模糊数学控制大规模数字处理电路的电子图像稳定新技术,克服手握一体化摄录机引起的图像晃动效应,以改善图像质量。该技术可广泛应用于视频处理各领域。     

车载视频下改进的核相关滤波跟踪算法

针对相关滤波跟踪算法在车载视频下由于环境复杂及目标尺度变化等情况下容易跟踪失败的问题,该文提出一种基于背景信息的尺度自适应相关滤波跟踪算法。首先利用背景感知相关滤波跟踪器融合方向梯度直方图特征预测目标下一帧位置,然后根据预测位置选取图像块进行检测,最后结合动态尺度比例金字塔模型对目标进行尺度估计。实验选取了KITTI数据库中23段车载视频和标注国内的4段车载视频进行测试,实验结果表明,该算法能有效降低车载环境的复杂背景、目标尺度变化等因素干扰,整体性能优于KCF, DSST, SAMF, SATPLE等主流相关滤波算法,对车载环境下复杂背景和尺度变化的目标跟踪具有鲁棒性。     

Homography-based block motion estimation for video coding of PTZ cameras

Due to the constrained movement of pan-tilt-zoom (PTZ) cameras, two frames in the video sequences captured by such cameras can be geometrically related by a relationship (homography). This geometric relationship is helpful for reducing the spatial redundancy in video coding. In this paper, by exploiting the homography between two frames with optical flow tracking algorithm, we propose a novel homography-based search (HBS) algorithm for block motion estimation in coding the sequences captured by PTZ cameras. In addition, adaptive thresholds are adopted in our method to classify different kinds of blocks. Compared with other traditional fast algorithms, the proposed HBS algorithm is proved to be more efficient for the sequences captured by PTZ cameras. And compared to our previous work in ICME (Cui et al., 2011), which only deals with pan-tilt (PT) camera and calculates the homography with mechanical devices, in this extended work we compute the homography by using information on images instead.     

Correcting Image Distortion for Adaptive Cruise Control

One of the main drivers for intelligent transportation systems is safety. Adaptive cruise control, as a common solution for traffic safety, has extended from radars to cameras. Due to high mobility of vehicles and unevenness of roads, the picture quality of cameras has been great challenges for camera-based adaptive cruise control. In this paper, an image distortion correction algorithm is addressed. Our method is based on optical flow technology which is normally applied in motion estimation and video compression research. We are the first to attempt to adapt it in image distortion correction. Two optical flow approaches, the Lucas-Kanade method and the Horn-Schunck method, are selected and compared. The procedure of image distortion correction using the optical flow method has been tested by both synthetic test images and camera images. The experimental results show that the Lucas-Kanade method is more suitable in the correction of image distortion.     

运动补偿在等离子体显示屏中的应用

等离子体显示屏在显示运动画面时会出现伪轮廓线等动态失真现象。一种解决方法是对子场显示信息进行运动补偿，但它需要对连续视频流进行准确运动估计。本文提出一种MPEG码流中运动矢量提取算法，并对运动矢量提取方法和全搜索块匹配法进行了比较。利用提取的运动矢量，讨论了部分子场运动补偿方法对等离子体显示屏图像质量的改进。仿真结果表明将运动矢量提取方法和部分子场补偿方法相结合可以明显的消除等离子体显示屏上的动态失真现象。     

A Novel Character Segmentation Method for Text Images Captured by Cameras

Due to the rapid development of mobile devices equipped with cameras, instant translation of any text seen in any context is possible. Mobile devices can serve as a translation tool by recognizing the texts presented in the captured scenes. Images captured by cameras will embed more external or unwanted effects which need not to be considered in traditional optical character recognition (OCR). In this paper, we segment a text image captured by mobile devices into individual single characters to facilitate OCR kernel processing. Before proceeding with character segmentation, text detection and text line construction need to be performed in advance. A novel character segmentation method which integrates touched character filters is employed on text images captured by cameras. In addition, periphery features are extracted from the segmented images of touched characters and fed as inputs to support vector machines to calculate the confident values. In our experiment, the accuracy rate of the proposed character segmentation system is 94.90%, which demonstrates the effectiveness of the proposed method.     

动态前景干扰下的电子稳像算法

针对存在动态前景目标的抖动视频序列,提出了一种以块匹配为基础的块差分平方前景目标去除的电子稳像方法。首先,将图像分成大小为16×16的特征块,通过设定阈值筛选出具有足够梯度信息来估计运动矢量的特征块,从而减少参与计算的特征块数量以提高处理速度;然后,采用归一化互相关(Normalized Cross Correlation,NCC)匹配算法和迭代搜索策略来完成特征块的匹配;最后,采用牛顿迭代法计算仿射变换模型下的帧间运动参数,并根据这些参数完成运动补偿,实现图像序列的稳定。实验表明,算法成功地去除了视频序列中动态前景目标的干扰,对存在平移及旋转运动的视频序列,能够快速有效地去除抖动,获得稳定的视频序列。     

一种解决波动式干扰影响的序列图像运动目标检测方法

为解决复杂环境下的诸如枝叶摇摆、摄像机抖动等波动式干扰对运动目标检测的影响问题,该文提出基于视频窗口切分与分类的序列图像运动目标检测算法。首先将序列图像切分为rc大小的视频窗口,然后提取窗口内区域图像累积帧间差矩阵的简单统计特征,针对每一帧序列图像,将视频窗口进行分类,把它们划分为运动目标窗口和非运动目标窗口(包括静止背景窗口和波动式干扰窗口),最后将运动目标窗口合并为运动目标。该方法的优点是无需已知背景模型和运动目标大小、形状等任何先验信息。实验表明该算法能在摄像机抖动以及枝叶干扰等复杂环境下快速有效的检测出运动目标。     

小型面阵航空相机系统的像移补偿

设计了一种小型面阵航空相机系统.该系统采用大面阵CMOS图像传感器作为成像器件,通过反射镜扫描进行像移补偿的方法,实现无人机飞行状态时清晰地数字成像.实验室前向像移补偿实验结果表明,该系统对于前向像移进行了较好的补偿,达到指标要求,成像清晰.     

基于SIFT和卡尔曼滤波的电子稳像技术研究

利用电子稳像方法去除抖动视频中的噪声是目前稳像技术的发展方向之一。文中首先介绍电子稳像的基本流程,随后使用SIFT提取图像中的特征点,以解决图像中存在缩放的问题。在经过运动一致性估计后计算运动参数,并使用Kalman滤波得到各帧的修正矢量。实验结果表明该方法能有效地去除随机运动分量而保持意向运动分量,得到稳定的视频。     

Learning discriminative motion feature for enhancing multi-modal action recognition

Video action recognition is an important topic in computer vision tasks. Most of the existing methods use CNN-based models, and multiple modalities of image features are captured from the videos, such as static frames, dynamic images, and optical flow features. However, these mainstream features contain much static information including object and background information, where the motion information of the action itself is not distinguished and strengthened. In this work, a new kind of motion feature is proposed without static information for video action recognition. We propose a quantization of motion network based on the bag-of-feature method to learn significant and discriminative motion features. In the learned feature map, the object and background information is filtered out, even if the background is moving in the video. Therefore, the motion feature is complementary to the static image feature and the static information in the dynamic image and optical flow. A multi-stream classifier is built with the proposed motion feature and other features, and the performance of action recognition is enhanced comparing to other state-of-the-art methods.     

利用全景图的视频稳像算法

针对具有场景变换、景深变换较小的抖动视频,本文提出了一种利用全景图的视频稳像方法。不同于传统稳像方法,本文所提算法利用原始视频序列生成广角视图,通过后期抽取视频帧,以合成方式输出稳定视频。通过区域块的运动估计,获取相机的运动轨迹;在此基础上,为消除视频抖动导致的原始运动轨迹的非平滑问题,提出了一种基于运动矢量统计的相机运动模式判决方法,并以此选择相应的平滑方式。最后,通过平滑后的运动轨迹对全景图进行抽帧,合成平滑、稳定视频。实验结果表明,本文提出的稳像方法能有效去除较大抖动,同时避免了对视频宽度的裁剪等后处理,具有较好的稳像能力。      

Moving object detection and tracking from video captured by moving camera

This paper presents an effective method for the detection and tracking of multiple moving objects from a video sequence captured by a moving camera without additional sensors. Moving object detection is relatively difficult for video captured by a moving camera, since camera motion and object motion are mixed. In the proposed method, the feature points in the frames are found and then classified as belonging to foreground or background features. Next, moving object regions are obtained using an integration scheme based on foreground feature points and foreground regions, which are obtained using an image difference scheme. Then, a compensation scheme based on the motion history of the continuous motion contours obtained from three consecutive frames is applied to increase the regions of moving objects. Moving objects are detected using a refinement scheme and a minimum bounding box. Finally, moving object tracking is achieved using a Kalman filter based on the center of gravity of a moving object region in the minimum bounding box. Experimental results show that the proposed method has good performance.     

Iris on the Move: Acquisition of Images for Iris Recognition in Less Constrained Environments

Iris recognition is one of the most powerful techniques for biometric identification ever developed. Commercial systems based on the algorithms developed by John Daugman have been available since 1995 and have been used in a variety of practical applications. However, all currently available systems impose substantial constraints on subject position and motion during the recognition process. These constraints are largely driven by the image acquisition process, rather than the particular pattern-matching algorithm used for the recognition process. In this paper we present results of our efforts to substantially reduce constraints on position and motion by means of a new image acquisition system based on high-resolution cameras, video synchronized strobed illumination, and specularity based image segmentation. We discuss the design tradeoffs we made in developing the system and the performance we have been able to achieve when the image acquisition system is combined with a standard iris recognition algorithm. The Iris on the Move (IOM) system is the first system to enable capture of iris images of sufficient quality for iris recognition while the subject is moving at a normal walking pace through a minimally confining portal     

无人机视频图像运动目标检测算法综述

运动目标检测是实现目标跟踪、交通监控、行为分析等任务的基础。但在无人机获取的视频图像中,无人机运动、旋翼震动或外界风力等客观因素使图像出现较为明显的背景、光照等变化,会对运动目标的检测产生影响。因此,如何降低干扰、提高检测精度,让无人机在运动目标检测领域发挥作用在信息时代具有相当重要的意义。无人机视频图像的运动目标检测相比传统运动目标检测,检测思路基本一致,但干扰因素众多。本文以此为切入点,分类综述了适用于无人机视频图像运动目标检测的算法及其改进,主要包括运动估计算法、帧间差法、背景建模法、光流法等传统算法和近年出现的新型算法;通过对无人机运动状态的划分探讨比较了上述方法的优缺点及适用场景。帧间差法更适合处理无人机悬停状态的数据,背景建模法、光流法及新型算法对无人机悬停及巡航状态的数据均可处理;上述算法均不能很好解决光照变化造成误检、漏检现象。所以处理无人机视频数据时,要根据其运动信息及数据特点选择合适的算法,才能获得好的检测结果。     

Vehicle counting based on double virtual lines

Traditionally, magnetic loop detectors are often used to count vehicles passing over them in intelligent transportation system. Real-time image sequences are captured by video surveillance system. Virtual loop, which emulates the functionality of inductive loop detectors, is placed on images. It is more convenient, but it occurs in false detection and discrimination when vehicles are lane departure due to overtaking or crossing. This paper presents an effective approach for vehicle counting based on double virtual lines (DVL). Double virtual lines are assigned on images, which are across bidirectional multi-lane. The region between DVL is the detection zone, rather than virtual loop zone in each lane, so as to reduce the proportion of false detection and misjudgment from lane departure for vehicles. Then, in the detection zone, the dual-template convolution is designed to detect and locate moving vehicles to eliminate the mapping of one to many, many to one. The effective rules are given in terms of the constraint of the horizontal and vertical distances to improve the accuracy of vehicle counting. Experimental comparisons with the other method demonstrate the performance of the proposed method.