Video saliency detection incorporating temporal information in compressed domain

Saliency detection is widely used to pick out relevant parts of a scene as visual attention regions for various image/video applications. Since video is increasingly being captured, moved and stored in compressed form, there is a need for detecting video saliency directly in compressed domain. In this study, a compressed video saliency detection algorithm is proposed based on discrete cosine transformation (DCT) coefficients and motion information within a visual window. Firstly, DCT coefficients and motion information are extracted from H.264 video bitstream without full decoding. Due to a high quantization parameter setting in encoder, skip/intra is easily chosen as the best prediction mode, resulting in a large number of blocks with zero motion vector and no residual existing in video bitstream. To address these problems, the motion vectors of skip/intra coded blocks are calculated by interpolating its surroundings. In addition, a visual window is constructed to enhance the contrast of features and to avoid being affected by encoder. Secondly, after spatial and temporal saliency maps being generated by the normalized entropy, a motion importance factor is imposed to refine the temporal saliency map. Finally, a variance-like fusion method is proposed to dynamically combine these maps to yield the final video saliency map. Experimental results show that the proposed approach significantly outperforms other state-of-the-art video saliency detection models.     

Image extraction in DCT domain

More and more digital images are being stored in compressed formats, among which the format using discrete cosine transform (DCT) coefficients is widely adopted (JPEG, MPEG, H.263 etc). To exploit those successful image processing techniques developed in the pixel domain, the authors propose a fast image extraction algorithm to allow images to be extracted directly from compressed DCT coefficients without full decompression. In the proposed technique the extracted images retain quality comparable with that of fully decoded images. However, the computing cost and the storage expense incurred by the proposed algorithm are very much lower than the costs of full decompression. The experiments also demonstrate that the proposed algorithm has tremendous potential in that all existing image processing techniques developed in the pixel domain can be directly applied to compressed images, without involving full decompression.     

基于数据挖掘的图像压缩域肤色检测算法

提出了一种直接在JPEG图像压缩域进行肤色检测的算法。该算法首先在熵解码后的DCT系数中提取图像块的颜色特征和纹理特征,然后利用数据挖掘建立用于表征压缩域图像特征和肤色检测结果之间关系的肤色模型,并利用该模型进行初步肤色检测,最后利用区域生长的方法分割出图像中的肤色区域。实验结果表明,与像素域的SPM (Skin Probability Map)肤色检测算法相比,本文方法可以获得更高的检测准确率和更快的检测速度。     

A novel deep learning framework for double JPEG compression detection of small size blocks

Double JPEG compression detection plays a vital role in multimedia forensics, to find out whether a JPEG image is authentic or manipulated. However, it still remains to be a challenging task in the case when the quality factor of the first compression is much higher than that of the second compression, as well as in the case when the targeted image blocks are quite small. In this work, we present a novel end-to-end deep learning framework taking raw DCT coefficients as input to distinguish between single and double compressed images, which performs superior in the above two cases. Our proposed framework can be divided into two stages. In the first stage, we adopt an auxiliary DCT layer with sixty-four 8 × 8 DCT kernels. Using a specific layer to extract DCT coefficients instead of extracting them directly from JPEG bitstream allows our proposed framework to work even if the double compressed images are stored in spatial domain, e.g. in PGM, TIFF or other bitmap formats. The second stage is a deep neural network with multiple convolutional blocks to extract more effective features. We have conducted extensive experiments on three different image datasets. The experimental results demonstrate the superiority of our framework when compared with other state-of-the-art double JPEG compression detection methods either hand-crafted or learned using deep networks in the literature, especially in the two cases mentioned above. Furthermore, our proposed framework can detect triple and even multiple JPEG compressed images, which is scarce in the literature as far as we know.     

Content Authentication Based on JPEG-to-JPEG Watermarking

A content authentication technique based on JPEG-to-JPEG watermarking is proposed in this paper. In this technique, each 88 block in a JPEG compressed image is first processed by entropy decoding, and then the quantized discrete cosine transform (DCT) is applied to generate DCT coefficients: one DC coefficient and 63 AC coefficients in frequency coefficients. The DCT AC coefficients are used to form zero planes in which the watermark is embedded by a chaotic map. In this way, the watermark information is embedded into JPEG compressed domain, and the output watermarked image is still a JPEG format. The proposed method is especially applicable to content authentication of JPEG image since the quantized coefficients are modified for embedding the watermark and the chaotic system possesses an important property with the high sensitivity on initial values. Experimental results show that the tamper regions are localized accurately when the watermarked JPEG image is maliciously tampered.     

Content Authentication Based on JPEG-to-JPEG Watermarking

A content authentication technique based on JPEG-to-JPEG watermarking is proposed in this paper. In this technique, each 8x8 block in a JPEG compressed image is first processed by entropy decoding, and then the quantized discrete cosine transform （DCT） is applied to generate DCT coefficients： one DC coefficient and 63 AC coefficients in frequency coefficients. The DCT AC coefficients are used to form zero planes in which the watermark is embedded by a chaotic map. In this way, the watermark information is embedded into JPEG compressed domain, and the output watermarked image is still a JPEG format. The proposed method is especially applicable to content authentication of JPEG image since the quantized coefficients are modified for embedding the watermark and the chaotic system possesses an important property with the high sensitivity on initial values. Experimental results show that the tamper regions are localized accurately when the watermarked JPEG image is maliciously tampered.     

一种基于DCT域统计特征的JPEG图像隐写分析

提出了一种基于DCT域统计特征的JPEG图像隐写分析算法。该算法在分析JPEG图像的DCT域统计特性的基础上,提取了8维特征向量,通过LSSVM分类器对待测图像进行分类,以达到检测载密图像的目的。算法实现简单、计算复杂度低。实验结果表明,该算法检测速度快,具有较高的检测正确率,能够实现针对各类JPEG图像信息隐藏算法的有效检测。     

融合颜色特征和对比度特征的图像显著性检测

图像显著性检测能够获取一幅图像的视觉显著性区域,是计算机视觉的研究热点之一。提出一种结合颜色特征和对比度特征的图像显著性检测方法。首先构造图像在HSV空间的颜色函数以获取图像颜色特征;然后使用SLIC超像素分割算法对图像进行预处理,基于超像素块的对比度特征计算图像显著性;最后将融合颜色特征和对比度特征的显著图经过导向滤波优化形成最终的显著图。使用本文算法在公开数据集MSRA-1000上进行图像显著性检测,并与其他6种算法进行比较。实验结果表明本文算法结合了图像像素点和像素块的信息,检测的图像显著性区域轮廓更加完整,优于其他方法。     

Two-stage salient region detection by exploiting multiple priors

Saliency detection has become a valuable tool for many image processing tasks, like image retargeting, object recognition, and adaptive compression. With the rapid development of the saliency detection methods, people have approved the hypothesis that “the appearance contrast between the salient object and the background is high”, and build their saliency methods on some priors that explain this hypothesis. However, these methods are not satisfactory enough. We propose a two-stage salient region detection method. The input image is first segmented into superpixels. In the first stage, two measures which measure the isolation and distribution of each superpixel are proposed, we consider that both of these two measures are important for finding the salient regions, thus the image-feature-based saliency map is obtained by combining the two measures. Then, in the second stage, we incorporate into the image-feature-based saliency map a location prior map to emphasize the foci of attention. In this algorithm, six priors that explain what is the salient region are exploited. The proposed method is compared with the state-of-the-art saliency detection methods using one of the largest publicly available standard databases, the experimental result indicates that the proposed method has better performance. We also demonstrate how the saliency map of the proposed method can be used to create high quality of initial segmentation masks for subsequent image processing, like Grabcut based salient object segmentation.     

Image postprocessing by Non-local Kuan’s filter

Blocking artifacts exist in images and video sequences compressed to low bit rates using block-based discrete cosine transform (DCT) compression standards. In order to reduce blocking artifacts, two image postprocessing techniques, DNLK filter and OCDNLK filter, are presented in this paper. A more accurate DCT domain Kuan’s filter based on Non-local parameter estimation was proposed from the linear minimum mean-square-error (MMSE) criterion. We analyze the required two assumptions for the filter theoretically. Then the DCT domain Kuan’s filter for low frequency coefficients and Non-local mean filter for high frequency AC coefficients constitute the proposed Non-local Kuan’s (NLK) filter. After that, we propose the Dual Non-local Kuan’s (DNLK) filter by applying the proposed filter in dual layer. The DNLK filter is extended to form the Overcomplete Dual Non-local Kuan’s (OCDNLK) filter by applying to the overcomplete DCT coefficients. Experimental results on coded images using test quantization tables and JPEG coded images show the effectiveness of the two methods.     

Compressed domain video saliency detection using global and local spatiotemporal features

A compressed domain video saliency detection algorithm, which employs global and local spatiotemporal (GLST) features, is proposed in this work. We first conduct partial decoding of a compressed video bitstream to obtain motion vectors and DCT coefficients, from which GLST features are extracted. More specifically, we extract the spatial features of rarity, compactness, and center prior from DC coefficients by investigating the global color distribution in a frame. We also extract the spatial feature of texture contrast from AC coefficients to identify regions, whose local textures are distinct from those of neighboring regions. Moreover, we use the temporal features of motion intensity and motion contrast to detect visually important motions. Then, we generate spatial and temporal saliency maps, respectively, by linearly combining the spatial features and the temporal features. Finally, we fuse the two saliency maps into a spatiotemporal saliency map adaptively by comparing the robustness of the spatial features with that of the temporal features. Experimental results demonstrate that the proposed algorithm provides excellent saliency detection performance, while requiring low complexity and thus performing the detection in real-time.     

The spatial relationship of DCT coefficients between a block andits sub-blocks

At present, almost all digital images are stored and transferred in their compressed format in which discrete cosine transform (DCT)-based compression remains one of the most important data compression techniques due to the efforts from JPEG. In order to save the computation and memory cost, it is desirable to have image processing operations such as feature extraction, image indexing, and pattern classifications implemented directly in the DCT domain. To this end, we present in this paper a generalized analysis of spatial relationships between the DCTs of any block and its sub-blocks. The results reveal that DCT coefficients of any block can be directly obtained from the DCT coefficients of its sub-blocks and that the interblock relationship remains linear. It is useful in extracting global features in the compressed domain for general image processing tasks such as those widely used in pyramid algorithms and image indexing. In addition, due to the fact that the corresponding coefficient matrix of the linear combination is sparse, the computational complexity of the proposed algorithms is significantly lower than that of the existing methods     

基于灰度预测和灰色关联分析的离散余弦变化图像隐藏算法

Traditional information hiding algorithms cannot maintain a good balance of capacity, invisibility and robustness. In this paper, a novel blind colour image information hiding algorithm based on grey prediction and grey relational analysis in the Discrete Cosine Tran-sform (DCT) domain is proposed. First, this algorithm compresses the secret image losslessly based on the improved grey predic-tion GM(1,1) (IGM) model. It then chooses the blocks of rich texture in the cover image as the embedding regions using Double-dimension Grey Relational Analysis (DGRA). Finally, it adaptively embeds the compressed secret bits stream into the DCT domain mid-frequency coefficients, which are decided by those blocks’ Double-Dimension Grey Correlation Degree (DGCD) and Human Visual System (HVS). This method can ensure an adequate balance between invisibility, capacity and robustness. Experimental results show that the proposed algorithm is robust against JPEG compression (46.724 6 dB when the compression quality factor is 90%), Gaussian noise (45.531 3 dB when the parameter is (0,0.000 5)) etc., and it is a blind information hiding algorithm that can be extracted without an original carrier.     

Moving object segmentation in DCT-based compressed video

A block-based automatic segmentation algorithm has been developed for detecting and tracking moving objects in DCT-based compressed video. The proposed algorithm segments moving objects with block resolution using the stochastic behaviour of the image blocks in the DCT domain     

Saliency detection for panoramic landscape images of outdoor scenes

Saliency detection has been researched for conventional images with standard aspect ratios, however, it is a challenging problem for panoramic images with wide fields of view. In this paper, we propose a saliency detection algorithm for panoramic landscape images of outdoor scenes. We observe that a typical panoramic image includes several homogeneous background regions yielding horizontally elongated distributions, as well as multiple foreground objects with arbitrary locations. We first estimate the background of panoramic images by selecting homogeneous superpixels using geodesic similarity and analyzing their spatial distributions. Then we iteratively refine an initial saliency map derived from background estimation by computing the feature contrast only within local surrounding area whose range and shape are changed adaptively. Experimental results demonstrate that the proposed algorithm detects multiple salient objects faithfully while suppressing the background successfully, and it yields a significantly better performance of panorama saliency detection compared with the recent state-of-the-art techniques.     

基于重组DCT系数子带能量直方图的图像检索

现在许多图像采用JPEG格式存储,检索这些图像通常要先解压缩,然后提取基于像素域的特征矢量进行图像检索。己有文献提出直接在DCT域进行图像检索的方法,这样可以降低检索的时间复杂度。本文提出对JPEG图像的DCT系数利用多分辨率小波变换的形式进行重组,对整个数据库中所有图像的DCT系数重组得到的若干子带,分别建立子带能量直方图,而后采用Morton顺序建立每幅图像的索引,并采用变形B树结构组织图像数据库用于图像检索。     

Joint Optimization of Run-Length Coding, Huffman Coding, and Quantization Table With Complete Baseline JPEG Decoder Compatibility

To maximize rate distortion performance while remaining faithful to the JPEG syntax, the joint optimization of the Huffman tables, quantization step sizes, and DCT indices of a JPEG encoder is investigated. Given Huffman tables and quantization step sizes, an efficient graph-based algorithm is first proposed to find the optimal DCT indices in the form of run-size pairs. Based on this graph-based algorithm, an iterative algorithm is then presented to jointly optimize run-length coding, Huffman coding, and quantization table selection. The proposed iterative algorithm not only results in a compressed bitstream completely compatible with existing JPEG and MPEG decoders, but is also computationally efficient. Furthermore, when tested over standard test images, it achieves the best JPEG compression results, to the extent that its own JPEG compression performance even exceeds the quoted PSNR results of some state-of-the-art wavelet-based image coders such as Shapiro's embedded zerotree wavelet algorithm at the common bit rates under comparison. Both the graph-based algorithm and the iterative algorithm can be applied to application areas such as web image acceleration, digital camera image compression, MPEG frame optimization, and transcoding, etc.      

非清晰区域抑制下的显著对象检测方法

基于上下文感知的显著区域检测模型（Context-Aware,CA）对于大目标和复杂背景图像中显著对象检测存在检测内容缺失和误检的问题.在CA模型的基础上,引入图像清晰度的视觉反差特性,提出非清晰区域抑制下的图像显著对象检测方法.该方法以离散度作为判断图像中是否存在清晰度差异的标准,并对存在差异的图像进行抑制.实验结果表明,非清晰区域抑制的CA方法可以在较好的解决大目标检测和复杂背景误检问题,提高了显著对象检测精度.     

Saliency detection based on singular value decomposition

Saliency detection has gained popularity in many applications, and many different approaches have been proposed. In this paper, we propose a new approach based on singular value decomposition (SVD) for saliency detection. Our algorithm considers both the human-perception mechanism and the relationship between the singular values of an image decomposed by SVD and its salient regions. The key concept of our proposed algorithms is based on the fact that salient regions are the important parts of an image. The singular values of an image are divided into three groups: large, intermediate, and small singular values. We propose the hypotheses that the large singular values mainly contain information about the non-salient background and slight information about the salient regions, while the intermediate singular values contain most or even all of the saliency information. The small singular values contain little or even none of the saliency information. These hypotheses are validated by experiments. By regularization based on the average information, regularization using the leading largest singular values or regularization based on machine learning, the salient regions will become more conspicuous. In our proposed approach, learning-based methods are proposed to improve the accuracy of detecting salient regions in images. Gaussian filters are also employed to enhance the saliency information. Experimental results prove that our methods based on SVD achieve superior performance compared to other state-of-the-art methods for human-eye fixations, as well as salient-object detection, in terms of the area under the receiver operating characteristic (ROC) curve (AUC) score, the linear correlation coefficient (CC) score, the normalized scan-path saliency (NSS) score, the F-measure score, and visual quality.     

一种基于图像DCT域信息熵的盲检测算法

提出了一种基于图像DCT域信息熵的盲检测算法。该算法通过分析JPEG图像隐写前后子块DCT系数信息熵的变化,提取JPEG图像子块DCT系数信息熵的64维特征向量,之后用LSSVM分类器对待测图像进行分类,最终达到检测载密图像的目的。实验表明,该算法能有效地针对各种隐写算法下的载密图像进行检测,同时对低嵌入比例下的载密图像也能达到较高的检测率。