Image deblocking via sparse representation

Image compression based on block-based Discrete Cosine Transform (BDCT) inevitably produces annoying blocking artifacts because each block is transformed and quantized independently. This paper proposes a new deblocking method for BDCT compressed images based on sparse representation. To remove blocking artifacts, we obtain a general dictionary from a set of training images using the K-singular value decomposition (K-SVD) algorithm, which can effectively describe the content of an image. Then, an error threshold for orthogonal matching pursuit (OMP) is automatically estimated to use the dictionary for image deblocking by the compression factor of compressed image. Consequently, blocking artifacts are significantly reduced by the obtained dictionary and the estimated error threshold. Experimental results indicate that the proposed method is very effective in dealing with the image deblocking problem from compressed images.     

Multi-scale attention network for image super-resolution

The power of convolutional neural networks (CNN) has demonstrated irreplaceable advantages in super-resolution. However, many CNN-based methods need large model sizes to achieve superior performance, making them difficult to apply in the practical world with limited memory footprints. To efficiently balance model complexity and performance, we propose a multi-scale attention network (MSAN) by cascading multiple multi-scale attention blocks (MSAB), each of which integrates a multi-scale cross block (MSCB) and a multi-path wide-activated attention block (MWAB). Specifically, MSCB initially connects three parallel convolutions with different dilation rates hierarchically to aggregate the knowledge of features at different levels and scales. Then, MWAB split the channel features from MSCB into three portions to further improve performance. Rather than being treated equally and independently, each portion is responsible for a specific function, enabling internal communication among channels. Experimental results show that our MSAN outperforms most state-of-the-art methods with relatively few parameters and Mult-Adds.     

Variable block-based deblocking filter for H.264/AVC on low-end and low-bit rates terminals

H.264/AVC supports variable block motion compensation, multiple reference frames, 1/4-pixel motion vector accuracy, and in-loop deblocking filter, compared with previous video coding standards. While these coding techniques are major functions for video compression improvement, they lead to high computational complexity at the same time. For the H.264 video coding techniques to be actually applied on low-end/low-bit rates terminals more extensively, it is essential to improve the coding efficiency. Currently the H.264 deblocking filter, which can improve the subjective quality of video, is hardly used on low-end terminals due to computational complexity.In this paper, we propose an enhanced method of deblocking filter that efficiently reduces the blocking artifacts occurring during the low-bit rates video coding. In the ‘variable block-based deblocking filter (VBDF)’ proposed in this paper, the temporal and spatial characteristics of moving pictures are extracted using the variable block-size information of motion compensation, the filter mode is classified into four different modes according to the moving-picture characteristics, and the adaptive filtering is executed in the separate modes. The proposed VBDF can reduce the blocking artifacts, prevent excessive blurring effects, and achieve about 30–40% computational speedup at about the same PSNR compared with the existing methods.     

A no-reference perceptual image quality assessment database for learned image codecs

The drastic growth of research in image compression, especially deep learning-based image compression techniques, poses new challenges to objective image quality assessment (IQA). Typical artifacts encountered in the emerging image codecs are significantly different from that produced by traditional block-based codecs, leading to inapplicability of the existing objective IQA algorithms. Towards advancing the development of objective IQA algorithms for recent compression artifacts, we built a learning-based compressed image quality assessment (LCIQA) database involving traditional block-based image codecs, hybrid neural network based image codecs, convolutional neural network based and generative adversarial network (GAN) based end-to-end optimized image coding approaches. Our study confirms the statistical difference and human perception difference between reconstructions of learned compression and traditional block-based compression. We propose a two-step deep learning model for learning-based compressed image quality assessment. Extensive experiments on LCIQA database demonstrate that our proposed model performs better than other counterparts on learning-based compressed images, especially on GAN compressed images, and achieves competitive performance to the state-of-the-art IQA metrics on traditional compressed images.     

Deep learning based HEVC in-loop filter and noise reduction

The lossy compression techniques at low bit rate often create ringing and contouring effects on the output images and introduce various blurring and distortion at block bounders. To overcome those compression artifacts different neural network based post-processing techniques have been experimented with over the last few years. The traditional loop-filter methods in the HEVC frame-work support two post-processing operations namely a de-blocking filter followed by a sample adaptive offset (SAO) filter. These operations usually introduce extra signaling bits and become overhead to the network with high-resolution video processing. In this study, we came up with a new deep learning-based algorithm for SAO filtering operations and substantiated the merits of the proposed method. We introduced a variable filter size sub-layered dense CNN (SDCNN) to improve the denoising operation and incorporated large stride deconvolution layers for further computation improvement. We demonstrate that our deconvolution model can effectively be trained by leveraging the high-frequency edge features learned in a shallow network using residual learning and data augmentation techniques. Extensive experiments show that our approach outperformed other state-of-the-art approaches in terms of SSIM, Bjøntegaard delta bit-rate (BD-BR), BD-PSNR measurements on the standard video test set and achieves an average of 8.73 % bit rate saving compared to HEVC baseline.     

Self-learning-based post-processing for image/video deblocking via sparse representation

Blocking artifact, characterized by visually noticeable changes in pixel values along block boundaries, is a common problem in block-based image/video compression, especially at low bitrate coding. Various post-processing techniques have been proposed to reduce blocking artifacts, but they usually introduce excessive blurring or ringing effects. This paper proposes a self-learning-based post-processing framework for image/video deblocking by properly formulating deblocking as an MCA (morphological component analysis)-based image decomposition problem via sparse representation. Without the need of any prior knowledge (e.g., the positions where blocking artifacts occur, the algorithm used for compression, or the characteristics of image to be processed) about the blocking artifacts to be removed, the proposed framework can automatically learn two dictionaries for decomposing an input decoded image into its “blocking component” and “non-blocking component.” More specifically, the proposed method first decomposes a frame into the low-frequency and high-frequency parts by applying BM3D (block-matching and 3D filtering) algorithm. The high-frequency part is then decomposed into a blocking component and a non-blocking component by performing dictionary learning and sparse coding based on MCA. As a result, the blocking component can be removed from the image/video frame successfully while preserving most original visual details. Experimental results demonstrate the efficacy of the proposed algorithm.     

A generic post-deblocking filter for block based image compression algorithms

In this paper we propose a new post-processing deblocking technique that is independent of the compression method used to encode the image. The development of this filter was motivated by the use of Multidimensional Multiscale Parser (MMP) algorithm, a generic lossy and lossless compression method. Since it employs an adaptive block size, it presents some impairments when using the deblocking techniques presented in the literature. This led us to the development of a new and more generic deblocking method, based on total variation and adaptive bilateral filtering.The proposed method was evaluated not only for still images, but also for video sequences, encoded using pattern matching and transform based compression methods. For all cases, both the objective and subjective quality of the reconstructed images were improved, showing the versatility of the proposed technique.     

Efficient quadtree based block-shift filtering for deblocking and deringing

The existing implementations of block-shift based filtering algorithms for deblocking are hard to achieve good smoothing performance and low computation complexity simultaneously due to their fixed block size and small shifting range. In this paper, we propose to integrate quadtree (QT) decomposition with the block-shift filtering for deblocking. By incorporating the QT decomposition, we can easily find the locations of uniform regions and determine the corresponding suitable block sizes. The variable block sizes generated by the QT decomposition facilitate the later block-shift filtering with low computational cost. In addition, large block based shift filtering can provide better deblocking results because the smoothing range of large blocks spans over the conventional 8 × 8 block size. Furthermore, we extend the proposed QT based block-shifting algorithm for deringing JPEG2000 coded images. Experimental results show the superior performance of our proposed algorithms.     

基于多尺度稠密残差网络的JPEG压缩伪迹去除方法

JPEG在高压缩比的情况下,解压缩后的图像会产生块效应、边缘振荡效应和模糊,严重影响了图像的视觉效果。为了去除JPEG压缩伪迹,该文提出了多尺度稠密残差网络。首先把扩张卷积引入到残差网络的稠密块中,利用不同的扩张因子,使其形成多尺度稠密块;然后采用4个多尺度稠密块将网络设计成包含2条支路的结构,其中后一条支路用于补充前一条支路没有提取到的特征;最后采用残差学习的方法来提高网络的性能。为了提高网络的通用性,采用具有不同压缩质量因子的联合训练方式对网络进行训练,针对不同压缩质量因子训练出一个通用模型。经实验表明,该文方法不仅具有较高的JPEG压缩伪迹去除性能,且具有较强的泛化能力。     

用于光子计数单像素成像的去块状采样网络

将光子计数技术和单像素成像结合,能实现高灵敏、低成本的光子计数成像,但存在采样时间和重建时间长的问题。基于深度学习的压缩采样和重建网络,将去除偏置和激活函数的全连接层作为测量矩阵,通过从数据中学得高效的测量矩阵和避免传统迭代算法带来的巨大计算量,实现了更快、更高质量的图像重建。但利用全连接层进行高分辨图像的分块压缩感知时,重建图像会产生块状效应。针对该问题提出了重叠分块采样网络（Os_net）、嵌套采样网络(Ns_net)、卷积采样网络（Cs_net）等三种方法以取代全连接层采样。在重建网络的设计中,使用线性映射网络对图像进行重建,设计实验结果表明Cs_net的去块状化效果最好。将Cs_net二值化后应用于光子计数单像素成像系统,实验结果表明Cs_net除块状化明显优于传统算法TVAL3,且Cs_net在重建质量上也同样取得了较好的效果。     

Design a deblocking filter with three separate modes in DCT-based coding

The reconstructed images from highly compressed data have noticeable image degradations, such as blocking artifacts near the block boundaries. Post-processing appears to be the most feasible solution because it does not require any existing standards to be changed. Markedly reducing blocking effects can increase compression ratios for a particular image quality or improve the quality of equally compressed images. In this work, a novel deblocking algorithm is proposed based on three filtering modes in terms of the activity across block boundaries. By properly considering the masking effect of the HVS (Human Visual System), an adaptive filtering decision is integrated into the deblocking process. According to three different deblocking modes appropriate for local regions with different characteristics, the perceptual and objective quality are improved without excessive smoothing the image details or insufficiently reducing the strong blocking effect on a flat region. According to the simulation results, the proposed method outperforms other deblocking algorithms in respect to PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural SIMilarity).     

Aligned and non-aligned double JPEG detection using convolutional neural networks

Due to the wide diffusion of JPEG coding standard, the image forensic community has devoted significant attention to the development of double JPEG (DJPEG) compression detectors through the years. The ability of detecting whether an image has been compressed twice provides paramount information toward image authenticity assessment. Given the trend recently gained by convolutional neural networks (CNN) in many computer vision tasks, in this paper we propose to use CNNs for aligned and non-aligned double JPEG compression detection. In particular, we explore the capability of CNNs to capture DJPEG artifacts directly from images. Results show that the proposed CNN-based detectors achieve good performance even with small size images (i.e., 64 × 64), outperforming state-of-the-art solutions, especially in the non-aligned case. Besides, good results are also achieved in the commonly-recognized challenging case in which the first quality factor is larger than the second one.     

Improved Method for the Macroblock‐Level Deblocking Scheme

This paper presents a deblocking method for video compression in which the blocking artifacts are effectively extracted and eliminated based on both spatial and frequency domain operations. Firstly, we use a probabilistic approach to analyze the performance of the conventional macroblock‐level deblocking scheme. Then, based on the results of the analysis, an algorithm to reduce the computational complexity is introduced. Experimental results show that the proposed algorithm outperforms the conventional video coding methods in terms of computation complexity while coding efficiency is maintained.     

Correlation filter via random-projection based CNNs features combination for visual tracking

Object tracking based on the Convolutional Neural Networks (CNNs) with multiple feature correlation filter (CF) has become one of the best object tracking frameworks. In this paper, we propose a novel approach of CNNs based CF, which combines deep features from CNNs into low-dimensional features. To achieve the dimensionality reduction, random-projection is used due to its data-independence and superior computational efficiency over other widely used. In our proposed approach, the spectral graph theory is applied to generate a random projection matrix. This method bypasses the time-consuming Gram–Schmidt orthogonalization, where the dimension of the feature is high. The combined features have very low dimensions, less than one tenth of the dimensions of the original deep features from CNNs, offering an improvement of tracking speed and without loss of performance simultaneously. Extensive experiments are conducted on large-scale benchmark datasets. The results demonstrate that the proposed algorithm outperforms the state-of-the-art methods.     

Optimized video compression with residual split attention and swin-block artifact contraction

Research in video compression has seen significant advancement in the last several years. However, the existing deep learning-based algorithms continue to be plagued by erroneous motion compression and ineffective motion compensation architectures, resulting in compression errors with a lower rate–distortion trade-off. To overcome these challenges, we present an end-to-end purely deep learning-based video compression method through a set of primary operations (e.g., motion estimation, motion compression, motion compensation, residual compression, and artifact contraction) differently. A deep residual attention split (DRAS) block is introduced for motion compression networks to pay more attention to certain image regions to create more effective features for the decoder while boosting the rate–distortion optimization (RDO) efficiency. A channel residual block (CRB) is proposed in motion compensation to yield a more accurate predicted frame, potentially improving the residual frame. To mitigate the compression errors, an artifact contraction module (ACM) by residual swin convolution UNet block is included in this model to improve the reconstruction quality. To improve the final frame, a buffer is added to fine-tune the previous reference frames. These modules combine with a loss function by assessing the trade-off and enhancing the decoded video quality. A comprehensive ablation study demonstrates the effectiveness of the proposed blocks and modules for video compression. Experimental results show the competitive performance of the proposed method on four benchmark datasets.     

Edge‐Preserving Algorithm for Block Artifact Reduction and Its Pipelined Architecture

This paper presents a new edge‐protection algorithm and its very large scale integration (VLSI) architecture for block artifact reduction. Unlike previous approaches using block classification, our algorithm utilizes pixel classification to categorize each pixel into one of two classes, namely smooth region and edge region, which are described by the edge‐protection maps. Based on these maps, a two‐step adaptive filter which includes offset filtering and edge‐preserving filtering is used to remove block artifacts. A pipelined VLSI architecture of the proposed deblocking algorithm for HD video processing is also presented in this paper. A memory‐reduced architecture for a block buffer is used to optimize memory usage. The architecture of the proposed deblocking filter is verified on FPGA Cyclone II and implemented using the ANAM 0.25 µm CMOS cell library. Our experimental results show that our proposed algorithm effectively reduces block artifacts while preserving the details. The PSNR performance of our algorithm using pixel classification is better than that of previous algorithms using block classification.     

MRANet: Multi-atrous residual attention Network for stereo image super-resolution

In recent years, stereo cameras have been widely used in various fields. Due to the limited resolution of real equipments, stereo image super-resolution (SR) is a very important and hot topic. Recent studies have shown that deep network structures can directly affect feature expression and extraction and thus influence the final results. In this paper, we propose a multi-atrous residual attention stereo super-resolution network (MRANet) with parallax extraction and strong discriminative ability. Specifically, we propose a multi-scale atrous residual attention (MARA) block to obtain receptive fields of different scales through a multi-scale atrous convolution and then combine them with attention mechanisms to extract more diverse and meaningful information. Moreover, we propose a stereo feature fusion unit for stereo parallax extraction and single viewpoint feature refinement and integration. Experiments on benchmark datasets show that MRANet achieves state-of-the-art performance in terms of quantitative metrics and visual quality compared with several SR methods.     

Traffic sign recognition based on deep convolutional neural network

Traffic sign recognition (TSR) is an important component of automated driving systems. It is a rather challenging task to design a high-performance classifier for the TSR system. In this paper, we propose a new method for TSR system based on deep convolutional neural network. In order to enhance the expression of the network, a novel structure (dubbed block-layer below) which combines network-in-network and residual connection is designed. Our network has 10 layers with parameters (block-layer seen as a single layer): the first seven are alternate convolutional layers and block-layers, and the remaining three are fully-connected layers. We train our TSR network on the German traffic sign recognition benchmark (GTSRB) dataset. To reduce overfitting, we perform data augmentation on the training images and employ a regularization method named “dropout”. The activation function we employ in our network adopts scaled exponential linear units (SELUs), which can induce self-normalizing properties. To speed up the training, we use an efficient GPU to accelerate the convolutional operation. On the test dataset of GTSRB, we achieve the accuracy rate of 99.67%, exceed-ing the state-of-the-art results.     

Adaptive deblocking method for low bitrate coded HEVC video

HEVC in-loop deblocking filter significantly improves the subjective quality of coded video by removing blocking artifact. However, there are still visible blocking artifacts in the complex videos with fast and chaotic motions coded at a low bitrate. In this paper, we propose a three-step deblocking filter scheme, which pre-processes video to remove undesired noise, next removes the corner outliers, and then suppresses the normal blocking artifacts with adaptive deblocking filters. The whole deblocking filtering process is applied on both luma and chroma components. Experimental results show that the proposed method could effectively improve the subjective quality for various videos, and outperform other typical post-processing deblocking methods.     

一种基于人类视觉系统的去块效应算法

基于块离散余弦变换的图像和视频压缩主要缺点就是在低比特率时会在块边界出现明显的方块效应。本文提出一种充分利用人类视觉特性,在图像的平滑区和纹理区分别采用一维DCT域滤波和空间域滤波的去块效应算法。实验结果表明该算法既能有效地去除方块效应又能保护图像的边缘信息。