视频分割技术概述

概述了基于内容的视频分割技术，主要介绍非压缩域和压缩域两个主要方面的镜头变换检测算法，展望了视频分割技术在自动化视频浏览和检索系统的应用前景。     

面向检索的压缩域空时视频目标检测与分割

面向语义视频检索,提出一种压缩域的目标分割新算法。它直接基于压缩码流中运动矢量和DCT系数,经过运动检测、矢量分水岭分割、目标融合与修正、后处理与跟踪等步骤提取空时视频目标。整个过程主要基于压缩域进行,无需视频码流的完全解码。对不同测试序列的实验测试结果显示算法能基于压缩域提取较为精确的空时视频目标,并具有较好的鲁棒性。     

Compressed video processing for cut detection

One of the challenging problems in video databases is the organisation of video information. Segmenting a video into a number of clips and characterising each clip has been suggested as one mechanism for organising video information. This approach requires a suitable method to automatically locate cut points (boundaries between consecutive camera shots in a video). Several existing techniques solve this problem using uncompressed video. Since video is increasingly being captured, moved, and stored in compressed form, there is a need for detecting shot boundaries directly in compressed video. The authors address this issue and show certain feature extraction steps in MPEG compressed video that allow the implementation of most of the existing cut detection methods developed for uncompressed video for MPEG video stream. They also examine the performance of three tests for cut detection by viewing the problem of cut detection as a statistical hypothesis testing problem. As the experimental results indicate, the statistical hypothesis testing approach permits fast and accurate detection of video cuts     

Super-resolution reconstruction of compressed video using transform-domain statistics

Considerable attention has been directed to the problem of producing high-resolution video and still images from multiple low-resolution images. This multiframe reconstruction, also known as super-resolution reconstruction, is beginning to be applied to compressed video. Super-resolution techniques that have been designed for raw (i.e., uncompressed) video may not be effective when applied to compressed video because they do not incorporate the compression process into their models. The compression process introduces quantization error, which is the dominant source of error in some cases. In this paper, we propose a stochastic framework where quantization information as well as other statistical information about additive noise and image prior can be utilized effectively.     

Object tracking in surveillance videos using compressed domain features from scalable bit-streams

Recent developments in the video coding technology brought new possibilities of utilising inherently embedded features of the encoded bit-stream in applications such as video adaptation and analysis. Due to the proliferation of surveillance videos there is a strong demand for highly efficient and reliable algorithms for object tracking. This paper presents a new approach for the fast compressed domain analysis utilising motion data from the encoded bit-streams in order to achieve low-processing complexity of object tracking in the surveillance videos. The algorithm estimates the trajectory of video objects by using compressed domain motion vectors extracted directly from standard H.264/MPEG-4 Advanced Video Coding (AVC) and Scalable Video Coding (SVC) bit-streams. The experimental results show comparable tracking precision when evaluated against the standard algorithms in uncompressed domain, while maintaining low computational complexity and fast processing time, thus making the algorithm suitable for real time and streaming applications where good estimates of object trajectories have to be computed fast.     

数字通信中数据压缩编码与解码技术

数据压缩编码与解压解码技术是目前应用的数字通信系统中的关键技术之一。本文介绍常用的数据压缩编码与解码技术及其最新研究进展,包括语音压缩编码、图像压缩编码与解码技术、多媒体数据压缩域处理技术以及差错控制编码与解码技术等。     

Digital television: making it work

Now that the US standard for digital TV has set technical parameters, equipment has to be built for production, editing and broadcast. The full-production HDTV facility must support existing NTSC equipment. When possible, it also must allow compressed operations (like storage and splicing) to avoid encoding and decoding penalties. The configuration resembles that being created as part of the NIST HDBT project. In this setup, a high-speed ATM computer network routes the compressed bit-stream around the studio. Besides the compressed video, the ATM net routes intercom, digital audio (compressed or not), and data. All the equipment (servers, encoders, and the off-line transcoder) interfaces to the ATM router, which is the studio's central switch, replacing the router of conventional studios. The transcoder's job is to convert one compressed format into another. All the devices on the computer network are controlled by the studio control workstation. This architecture also allows connection to be made to other TV studios over existing telecommunications networks. A network interface device has this job. In the early stages, video production will be performed on uncompressed video. As compressed technology advances, more production will be done on compressed video. Most studios will probably transition to compressed production but retain uncompressed elements to take advantage of some of its features     

A novel moving object segmentation framework utilizing camera motion recognition for H.264 compressed videos

This paper presents a novel coarse to fine moving object segmentation framework for H.264/AVC compressed videos. The proposed framework integrates the global motion estimation and global motion compensation steps in the segmentation pipeline unlike previous techniques which did not consider such an integration. The integration is based on testing for presence of global motion by classifying the interframe motion vectors into moving camera class and still camera class. The decision boundary separating these two classes is learnt from the training video data. The integration automates the moving object segmentation to be applicable for static, moving and combination of static/moving camera cases which to the best of our knowledge has not been carried out earlier. Further, a novel coarse segmentation technique is proposed by decomposing the inter-frame motion vectors into wavelet sub-bands and utilizing logical operations on LH, HL and HH sub-band wavelet coefficients. The premise is based on the fact that since the LH, HL and HH sub-bands contain the detail information pertaining to horizontal, vertical and diagonal moving blocks respectively, they can be exploited to identify the coarse moving boundaries. The coarse segmentation is fast in comparison to state-of-the-art coarse segmentation methods as demonstrated by our experiments. Finally, these coarse boundaries are modeled in an energy minimization framework and shown that by minimizing the energy using graph cut optimization the segmentation is refined to obtain the fine segmentation. The proposed framework is tested on a number of standard video sequences encoded with H.264/AVC JM encoder and comparison is carried out with state-of-the-art compressed domain moving object segmentation methods as well as with an existing state-of-the-art pixel domain method to establish and validate the proposed moving object segmentation framework.     

压缩域视频对象分割研究

讨论了压缩域视频对象分割技术的发展和研究现状,介绍了几种典型的分割算法,并具体指出一些需要深入研究的问题.     

A wavelet domain diversity method for transmission of images overwireless channels

We propose a wavelet domain diversity combining method to combat errors during image transmission on wireless channels. For images represented in the wavelet domain, diversity is used to obtain multiple data streams corresponding to the transmitted image at the receiver. These individual image data streams are combined to form a composite image with higher perceptual quality. Both uncompressed and compressed images are considered. The SPIHT algorithm is used for image compression. Diversity combining methods for both uncompressed and compressed images exploit the characteristics of the wavelet transform. For compressed images, unequal error protection is employed in conjunction with diversity combining. Simulation results demonstrate that the quality of the received image can be significantly improved     

A segmentation scheme for head-shoulder video in MPEG compressed domain

More attention has been paid to the study of video object segmentation in compressed domain these years, which has already led to some practical technology. In this paper, a scheme is put forward for segmentation of head-shoulder video in MPEG （Motion Picture Experts Group） compressed domain. The conception of DCT （Discrete Cosine Transform） feature plane is defined. In the suggested scheme, firstly, the face region is detected by clustering skin-tone DCT feature points in the DCT feature plane. Secondly, the region of head-shoulder is approximately regarded as combination of the head rectangle and shoulder rectangle, and head rectangle is confirmed by double template matching. Thirdly, Canny operator and morphological operation are applied to the region of head-shoulder in feature plane to get the object mask and the region of object mask is rectified by correlation of DCT blocks to get high-quality segmentation.     

Scene change detection algorithms for content-based video indexingand retrieval

There is an urgent need to extract key information from video automatically for the purposes of indexing, fast retrieval, and scene analysis. To support this vision, reliable scene change detection algorithms must be developed. Several algorithms have been proposed for both sudden and gradual scene change detection in uncompressed and compressed video. In this paper some common algorithms that have been proposed for scene change detection are reviewed. A novel algorithm for sudden scene change detection for MPEG-2 compressed video is then presented. This uses the number of interpolated macroblocks in B-frames to identify the sudden scene changes. A gradual scene change detection algorithm based on statistical features is also presented     

Moving object segmentation in DCT-based compressed video

A block-based automatic segmentation algorithm has been developed for detecting and tracking moving objects in DCT-based compressed video. The proposed algorithm segments moving objects with block resolution using the stochastic behaviour of the image blocks in the DCT domain     

Research on key thechnologies of pornographic image/video recognition in compressed domain

Pornographic image/video recognition plays a vital role in network information surveillance and management. In this paper, its key techniques, such as skin detection, key frame extraction, and classifier design, etc, are studied in compressed domain. A skin detection method based on data-mining in compressed domain is proposed firstly and achieves the higher detection accuracy as well as higher speed. Then, a cascade scheme of pornographic image recognition based on selective decision tree ensemble is proposed in order to improve both the speed and accuracy of recognition. A pornographic video oriented key frame extraction solution in compressed domain and an approach of pornographic video recognition are discussed respectively in the end.     

Blind MPEG-2 video watermarking robust against geometric attacks: a set of approaches in DCT domain.

A set of robust MPEG-2 video watermarking techniques is proposed, focusing on commonly used typical geometric processing for bit-rate reduction, cropping, removal of any rows, arbitrary-ratio downscaling, and frame dropping. Both the embedding and the extraction of watermarks are done in the compressed domain, so the computational cost is low. Moreover, the watermark extraction is blind, i.e., no original unwatermarked MPEG-2 video is needed for watermark extraction. The presented technique is applicable not only to MPEG-2 video, but also to other DCT-based coding videos. Selected experimental results validate our techniques.     

Manipulation and compositing of MC-DCT compressed video

Many advanced video applications require manipulations of compressed video signals. Popular video manipulation functions include overlap (opaque or semitransparent), translation, scaling, linear filtering, rotation, and pixel multiplication. We propose algorithms to manipulate compressed video in the compressed domain. Specifically, we focus on compression algorithms using the discrete cosine transform (DCT) with or without motion compensation (MC). Such compression systems include JPEG, motion JPEG, MPEG, and H.261. We derive a complete set of algorithms for all aforementioned manipulation functions in the transform domain, in which video signals are represented by quantized transform coefficients. Due to a much lower data rate and the elimination of decompression/compression conversion, the transform-domain approach has great potential in reducing the computational complexity. The actual computational speedup depends on the specific manipulation functions and the compression characteristics of the input video, such as the compression rate and the nonzero motion vector percentage. The proposed techniques can be applied to general orthogonal transforms, such as the discrete trigonometric transform. For compression systems incorporating MC (such as MPEG), we propose a new decoding algorithm to reconstruct the video in the transform domain and then perform the desired manipulations in the transform domain. The same technique can be applied to efficient video transcoding (e.g., from MPEG to JPEG) with minimal decoding     

Automatic video activity detection using compressed domain motion trajectories for H.264 videos

Most automatic event detection methods for video surveillance detect target events based on features extracted in the pixel domain. However, in practice, surveillance videos are often compressed. It is desirable to perform automatic event detection in the compressed domain directly so that the video does not need to be decoded for analysis purpose. In this paper, we investigate the use of motion trajectories for video activity detection in the compressed domain. We show it is possible to extract reliable motion trajectories directly from compressed H.264 video streams. To overcome the problems caused by unreliable motion vectors, we propose to include the information from the compressed domain prediction residuals to make the tracking more robust. We use a real world application of detecting vacant or occupied parking spaces to demonstrate the effectiveness of our proposed approach. We also demonstrate the robustness of our approach to different encoder settings, and lighting conditions.     

一种MPEG压缩视频的镜头边界检测算法

提出一种工作在MPEG压缩域的检测算法.首先从压缩视频中提取DC图像和重构参考帧,然后经过全局运动补偿之后进行基于边对象变化率的镜头分割点检测,最后结合DC图直方图差法构成联合检测算法.本算法能准确地检测到镜头渐变,镜头切变,能得到渐变类型等特性.用AdobePremiere5.1生成的各类镜头转换视频片段进行检测,实验结果验证了算法的有效性.     

Text detection,localization, and tracking in compressed video

Video text information plays an important role in semantic-based video analysis, indexing and retrieval. Video texts are closely related to the content of a video. Usually, the fundamental steps of text-based video analysis, browsing and retrieval consist of video text detection, localization, tracking, segmentation and recognition. Video sequences are commonly stored in compressed formats where MPEG coding techniques are often adopted. In this paper, a unified framework for text detection, localization, and tracking in compressed videos using the discrete cosines transform (DCT) coefficients is proposed. A coarse to fine text detection method is used to find text blocks in terms of the block DCT texture intensity information. The DCT texture intensity of an 8×8 block of an intra-frame is approximately represented by seven AC coefficients. The candidate text block regions are further verified and refined. The text block region localization and tracking are carried out by virtue of the horizontal and vertical block texture intensity projection profiles. The appearing and disappearing frames of each text line are determined by the text tracking. The final experimental results show the effectiveness of the proposed methods.     

Saliency detection in the compressed domain for adaptive image retargeting

Saliency detection plays important roles in many image processing applications, such as regions of interest extraction and image resizing. Existing saliency detection models are built in the uncompressed domain. Since most images over Internet are typically stored in the compressed domain such as joint photographic experts group (JPEG), we propose a novel saliency detection model in the compressed domain in this paper. The intensity, color, and texture features of the image are extracted from discrete cosine transform (DCT) coefficients in the JPEG bit-stream. Saliency value of each DCT block is obtained based on the Hausdorff distance calculation and feature map fusion. Based on the proposed saliency detection model, we further design an adaptive image retargeting algorithm in the compressed domain. The proposed image retargeting algorithm utilizes multioperator operation comprised of the block-based seam carving and the image scaling to resize images. A new definition of texture homogeneity is given to determine the amount of removal block-based seams. Thanks to the directly derived accurate saliency information from the compressed domain, the proposed image retargeting algorithm effectively preserves the visually important regions for images, efficiently removes the less crucial regions, and therefore significantly outperforms the relevant state-of-the-art algorithms, as demonstrated with the in-depth analysis in the extensive experiments.