Fast motion estimation for surveillance video compression

In this article, novel approaches to perform efficient motion estimation specific to surveillance video compression are proposed. These includes (i) selective (ii) tracker-based and (iii) multi-frame-based motion estimation. In selective approach, motion vector search is performed for only those frames that contain some motion activity. In another approach, contrary to performing motion estimation on the encoder side, motion vectors are calculated using information of a surveillance video tracker. This approach is quicker but for some scenarios it degrades the visual perception of the video compared with selective approach. In an effort to speed up multi-frame motion estimation, we propose a fast multiple reference frames-based motion estimation technique for surveillance videos. Experimental evaluation shows that significant reduction in computational complexity can be achieved by applying the proposed strategies.     

Automatic video activity detection using compressed domain motion trajectories for H.264 videos

Most automatic event detection methods for video surveillance detect target events based on features extracted in the pixel domain. However, in practice, surveillance videos are often compressed. It is desirable to perform automatic event detection in the compressed domain directly so that the video does not need to be decoded for analysis purpose. In this paper, we investigate the use of motion trajectories for video activity detection in the compressed domain. We show it is possible to extract reliable motion trajectories directly from compressed H.264 video streams. To overcome the problems caused by unreliable motion vectors, we propose to include the information from the compressed domain prediction residuals to make the tracking more robust. We use a real world application of detecting vacant or occupied parking spaces to demonstrate the effectiveness of our proposed approach. We also demonstrate the robustness of our approach to different encoder settings, and lighting conditions.     

一种针对监控视频场景的压缩域运动对象分割方法

提出了一种压缩域背景建模与运动对象分割框架:首先提取H.264视频压缩码流中的MB-bits与4×4块残差系数,基于MB-bits场进行压缩域Vibe背景建模分割出宏块级运动对象区域,然后结合最大熵模型提取4×4边缘块进行运动对象轮廓细化最终分割出压缩域运动对象。试验对比分析表明,提出的算法能快速、准确地提取压缩域运动对象,系统具有一定鲁棒性。     

Moving object detection in the H.264/AVC compressed domain for video surveillance applications

In this paper a novel method is presented to detect moving objects in H.264/AVC [T. Wiegand, G. Sullivan, G. Bjontegaard, G. Luthra, Overview of the H.264/AVC video coding standard, IEEE Transactions on Circuits and Systems for Video Technology, 13 (7) (2003) 560–576] compressed video surveillance sequences. Related work, within the H.264/AVC compressed domain, analyses the motion vector field to find moving objects. However, motion vectors are created from a coding perspective and additional complexity is needed to clean the noisy field. Hence, an alternative approach is presented here, based on the size (in bits) of the blocks and transform coefficients used within the video stream. The system is restricted to the syntax level and achieves high execution speeds, up to 20 times faster than the related work. To show the good detection results, a detailed comparison with related work is presented for different challenging video sequences. Finally, the influence of different encoder settings is investigated to show the robustness of our system.     

压缩域快速视频拷贝检测算法

为了在大规模数据下有效提升拷贝检测速度,提出了一种压缩域视频拷贝检测算法,采用2级匹配框架,利用压缩域上DCT系数顺序度量特征进行相似度匹配,显著减少了查询视频解码操作;并为DCT系数顺序度量特征建立倒排索引,以加快第一级匹配的速度.实验结果表明,与已有的算法相比,所提出算法在保持相当的检测精度条件的同时,大幅提升了检测速度.     

面向检索的压缩域空时视频目标检测与分割

面向语义视频检索,提出一种压缩域的目标分割新算法。它直接基于压缩码流中运动矢量和DCT系数,经过运动检测、矢量分水岭分割、目标融合与修正、后处理与跟踪等步骤提取空时视频目标。整个过程主要基于压缩域进行,无需视频码流的完全解码。对不同测试序列的实验测试结果显示算法能基于压缩域提取较为精确的空时视频目标,并具有较好的鲁棒性。     

Surveillance video synopsis in the compressed domain for fast video browsing

The traditional pixel-domain based video analysis methods have taken dominated places for long. However, due to the rapidly increasing volume and resolution of surveillance video, the desirable fast and scalable browsing encounters significant challenges in terms of efficiency and flexibility. Under this circumstance, operating surveillance video in compressed domain has aroused great concern in academy and industry. In order to perform the intelligent video analysis task on the premise of preserving accuracy and controlling complexity, this paper presents a compressed-domain approach for massive surveillance video synopsis generation, labeling and browsing. The main work and achievements include: (1) a compressed-domain scheme is established to condense the compressed surveillance video and record the synopsis results; (2) a background modeling method via the Motion Vector based Local Binary Pattern (MVLBP) is introduced to extract moving objects in an efficient way; (3) an object flags based synopsis labeling method is proposed to represent the object regions as well as their display modes in a flexible way. Experimental results show that the video analysis system based on this framework can provide not only efficient synopsis generation but also flexible scalable or playback browsing.     

压缩域内的视频图像处理算法

介绍了一些在压缩域内处理已压缩视频流的算法，主要关注如何对DCT系数直接进行处理的算法，对于带有运动估计的压缩系统(如MPEG视频压缩码)介绍了一些解码算法及相应压缩域内的重构操作，这些算法也应用于视频转码。     

Online video summarization on compressed domain

Recent advances in technology have increased the availability of video data, creating a strong requirement for efficient systems to manage those materials. Making efficient use of video information requires that data to be accessed in a user-friendly way. Ideally, one would like to understand a video content, without having to watch it entirely. This has been the goal of a quickly evolving research area known as video summarization. In this paper, we present a novel approach for video summarization that works in the compressed domain and allows the progressive generation of a video summary. The proposed method relies on exploiting visual features extracted from the video stream and on using a simple and fast algorithm to summarize the video content. Experiments on a TRECVID 2007 dataset show that our approach presents high quality relative to the state-of-the-art solutions and in a computational time that makes it suitable for online usage.     

PCA-based approach for video scene change detection on compressed video

An automatic, real-time detection approach to video scene change detection is presented. Owing to the high correlation of two consecutive video frames, it is proposed that only the eigenvector corresponding to the largest eigenvalue is retained in the principal component analysis (PCA) for video data. A one-dimensional PCA feature of video data is then generated from the PCA. It shows superior performance compared to the histogram feature and the pixel feature. The detection algorithm based on this PCA feature is then designed to detect both abrupt and gradual transitions. The proposed approach is tested on the TREC video test repository to validate its performance     

A segmentation scheme for head-shoulder video in MPEG compressed domain

More attention has been paid to the study of video object segmentation in compressed domain these years, which has already led to some practical technology. In this paper, a scheme is put forward for segmentation of head-shoulder video in MPEG （Motion Picture Experts Group） compressed domain. The conception of DCT （Discrete Cosine Transform） feature plane is defined. In the suggested scheme, firstly, the face region is detected by clustering skin-tone DCT feature points in the DCT feature plane. Secondly, the region of head-shoulder is approximately regarded as combination of the head rectangle and shoulder rectangle, and head rectangle is confirmed by double template matching. Thirdly, Canny operator and morphological operation are applied to the region of head-shoulder in feature plane to get the object mask and the region of object mask is rectified by correlation of DCT blocks to get high-quality segmentation.     

Image/video spatial scalability in compressed domain

Image/video compression is widely used in various applications with the advent of the Joint Photographic Expert Group (JPEG), Motion Picture Expert Group (MPEG), and H.261 standards. Hence, most visual data are stored in the compressed format. Spatial scalable encoding of visual data has several applications, including browsing visual databases, querying multimedia databases, interactive multimedia communications, etc. We propose two novel techniques, namely, format compatible (FC) discrete cosine transform (DCT) and format modified (FM) DCT to implement image/video spatial scalability directly in the DCT compressed domain. The FC-DCT technique can be used to manipulate the standard bit streams, such as JPEG, MPEG, etc., while the FM-DCT technique can be employed in a variety of applications which require fast processing. In contrast to the traditional spatial-domain techniques, the compressed-domain techniques remove the unnecessary decompression and recompression procedures, thus, they have the advantages of reduced computational complexity and storage requirements. Simulation results demonstrate that the proposed DCT-domain techniques can achieve a comparable performance at a much lower computational complexity compared to the spatial-domain techniques     

Moving object detection in wavelet compressed video

In many surveillance systems the video is stored in wavelet compressed form. In this paper, an algorithm for moving object and region detection in video which is compressed using a wavelet transform (WT) is developed. The algorithm estimates the WT of the background scene from the WTs of the past image frames of the video. The WT of the current image is compared with the WT of the background and the moving objects are determined from the difference. The algorithm does not perform inverse WT to obtain the actual pixels of the current image nor the estimated background. This leads to a computationally efficient method and a system compared to the existing motion estimation methods.     

压缩域运动补偿中的快速算法研究

研究了压缩域运动补偿的快速算法，着重从数学表达方式上研究了当进行宏块预测时，充分利用参考帧中公共块的计算方法，与DCT矩阵分解的快速算法相比，在不影响图像质量的同时。其计算效率提高了26．5％。     

压缩域视频拼接方法研究

本文主要研究实时视频应用的拼接方法并详述了其在云视频会议上的应用.不同于传统的基于转码的视频拼接,提出了基于压缩域码流的处理算法,并在云端架构视频处理器.以HEVC码流为例将码流处理的难点分为序列参数集、图片参数集、Slice头部和编码单元四个方面,对这些方面存在的问题给出了解决方法,从而实现了低复杂度的视频拼接算法.仿真结果表明,本文提出的压缩域视频拼接算法,在视频处理效率和质量上均有很大的提高.     

针对多视角视频编码的快速运动估计方法

为了提高多视角视频编码系统中运动估计速度,提出了一种基于马尔科夫链模型的快速运动矢量估计算法。该算法充分利用帧间、视角间相关性,建立预测矢量状态集合。通过马尔科夫链模型的状态转移概率,对预测矢量进行提前测试。然后利用提前退出准则,实现快速运动矢量估计。实验结果表明,与单独采用全搜索算法比较,该算法对于多视角视频序列运动估计速度可以提高99%以上,PSNR平均降低0.16dB。提出的算法对多视角视频编码系统的运动估计效率提升明显。     

一种适用于AVS-M的自适应快速运动估计算法

为了减小移动视频编码标准(AVS-M)中运动估计模块的复杂度,提出了一种快速、有效的块匹配运动估计算法.该算法充分利用了视频图像中运动矢量场的中心偏置特性和时空相关性,根据运动类型自适应的选择搜索起点和搜索策略,结合改进的搜索模板和高效搜索中止准则,有效地降低了运动估计的运算量.实验结果表明,该算法在保证搜索精度的同时,大大减少了搜索点数.     

Saliency detection in the compressed domain for adaptive image retargeting

Saliency detection plays important roles in many image processing applications, such as regions of interest extraction and image resizing. Existing saliency detection models are built in the uncompressed domain. Since most images over Internet are typically stored in the compressed domain such as joint photographic experts group (JPEG), we propose a novel saliency detection model in the compressed domain in this paper. The intensity, color, and texture features of the image are extracted from discrete cosine transform (DCT) coefficients in the JPEG bit-stream. Saliency value of each DCT block is obtained based on the Hausdorff distance calculation and feature map fusion. Based on the proposed saliency detection model, we further design an adaptive image retargeting algorithm in the compressed domain. The proposed image retargeting algorithm utilizes multioperator operation comprised of the block-based seam carving and the image scaling to resize images. A new definition of texture homogeneity is given to determine the amount of removal block-based seams. Thanks to the directly derived accurate saliency information from the compressed domain, the proposed image retargeting algorithm effectively preserves the visually important regions for images, efficiently removes the less crucial regions, and therefore significantly outperforms the relevant state-of-the-art algorithms, as demonstrated with the in-depth analysis in the extensive experiments.     

A learning-based approach for leaf detection in traffic surveillance video

Multidimensional Systems and Signal Processing - Traffic surveillance video is recorded in uncontrolled outdoor scenarios. If the camera view gets obstructed by the leaves, the video will fail to...     

Real-time DSP implementation for MRF-based video motion detection

This paper describes the real time implementation of a simple and robust motion detection algorithm based on Markov random field (MRF) modeling, MRF-based algorithms often require a significant amount of computations. The intrinsic parallel property of MRF modeling has led most of implementations toward parallel machines and neural networks, but none of these approaches offers an efficient solution for real-world (i.e., industrial) applications. Here, an alternative implementation for the problem at hand is presented yielding a complete, efficient and autonomous real-time system for motion detection. This system is based on a hybrid architecture, associating pipeline modules with one asynchronous module to perform the whole process, from video acquisition to moving object masks visualization. A board prototype is presented and a processing rate of 15 images/s is achieved, showing the validity of the approach.