Improved hybrid layered image compression using deep learning and traditional codecs

Recently deep learning-based methods have been applied in image compression and achieved many promising results. In this paper, we propose an improved hybrid layered image compression framework by combining deep learning and the traditional image codecs. At the encoder, we first use a convolutional neural network (CNN) to obtain a compact representation of the input image, which is losslessly encoded by the FLIF codec as the base layer of the bit stream. A coarse reconstruction of the input is obtained by another CNN from the reconstructed compact representation. The residual between the input and the coarse reconstruction is then obtained and encoded by the H.265/HEVC-based BPG codec as the enhancement layer of the bit stream. Experimental results using the Kodak and Tecnick datasets show that the proposed scheme outperforms the state-of-the-art deep learning-based layered coding scheme and traditional codecs including BPG in both PSNR and MS-SSIM metrics across a wide range of bit rates, when the images are coded in the RGB444 domain.     

Low-complexity compression of multispectral images based on classified transform coding

Compression of remote-sensing images can be necessary in various stages of the image life, and especially on-board a satellite before transmission to the ground station. Although on-board CPU power is quite limited, it is now possible to implement sophisticated real-time compression techniques, provided that complexity constraints are taken into account at design time. In this paper we consider the class-based multispectral image coder originally proposed in [Gelli and Poggi, Compression of multispectral images by spectral classification and transform coding, IEEE Trans. Image Process. (April 1999) 476–489 [5]] and modify it to allow its use in real time with limited hardware resources. Experiments carried out on several multispectral images show that the resulting unsupervised coder has a fully acceptable complexity, and a rate–distortion performance which is superior to that of the original supervised coder, and comparable to that of the best coders known in the literature.     

基于改进深度学习自编码的图像边沿畸变校正算法研究

图像边沿畸变校正中直线投影衍生边沿拟合度差,导致校正误差大,提出基于改进深度学习自编码的图像边沿畸变校正算法。采用自适应阈值小波去噪算法,对各级尺度参数实施自适应变换,完成噪声去除。根据图像去噪结果,使用费舍尔向量编码优化深度学习结果,提取图像的边沿畸变形态。并以边沿畸变形态提取结果为基础,获取校正目标优化函数,分析边沿断裂情况,实现直线投影衍生边沿拟合;通过确定图像边沿误差评价函数,判断图像边沿畸变校正方式,达到图像边沿畸变校正的目的。以含噪桶形畸变与枕形畸变图像为研究对象进行实验分析,结果表明,所提算法可有效校正图像的桶形畸变与枕形畸变,桶形、枕形图像边沿畸变校正后,图像中的边沿条数和原图一致均为5条,实现高精度、高效率的图像边沿畸变校正。     

Video and image applications on narrowband and broadband isdn

Two major ISDN applications which will undoubtedly affect world-wide telecommunications in the coming decade are discussed. They are: (1) video transmission and (2) image transmission. Brief reviews of videophone chronicle and the current video coding technologies are presented. The application of videophones using p × 64 (CCITT coding algorithm up to 1·5 Mb/s) and the DCT (discrete cosine transform) algorithm for narrowband ISDN are discussed. Broadcast TV quality DS3-45 MB/s video codecs are also briefly discussed as a probable videophone system in the broadband ISDN era. The explosive growth of facsimile services is reviewed, and the progress of image coding technologies and their standards are covered. The prospects of high resolution image transfer systems with ISDN are addressed.     

Shape adaptive wavelet transform for magnetic resonance images coding

Region-based coding is an important feature in today's image coding techniques as it follows different regions of the image that will be encoded at different bit rates and hence at different qualities rather than encoding the entire image with a single quality constraints. This article proposes an algorithm for the region-based coding of the brain magnetic resonance images in which the brain part will be encoded with more number of bits than the background. This method employs Shape Adaptive Discrete Wavelet Transform, which can transform the regions of interest and the background on the images independently and the coefficients can be encoded by using the SPIHT coding at different levels. This algorithm was compared with the existing wavelet-based coding techniques and a better PSNR was achieved for the same bit rate by reconstructing the region of interest with high quality than the background.     

一种基于低复杂度嵌入零块编码的遥感图像压缩编码方法

该文针对遥感图像的数据特点,提出了一种新的遥感图像编码方法。它基于一种改进的小波变换嵌入零块编码算法。新算法中改进了零块编码中四叉树分裂算法(quadtree),并设计亍新的链表生成和不重要集合排序策略。通过这些方法的改进,不仅提高了图像编码性能,同时还大大提高了运算效率。实验表明该文阐述的算法具有很低的复杂度和高的压缩率,PSNR和计算速度均超过SPIHT和SPECK。在1bpp下,该文方法的PSNR比SPIHT提高了0.3dB以上,计算速度比SPIHT提高了35％。     

改进的块截断图象编码算法

已经提出了的各种块截断算法本质上都是两电平量化器，由于它们将图象简单地量化（或截断）成两个电平值，所以在译码图象中出现矢真，为减少这种失真，本文在块截算法中采用于平滑滤波器，从而改进了现有块共怕算法的性能，明显提高了译码图象质量。     

A no-reference perceptual image quality assessment database for learned image codecs

The drastic growth of research in image compression, especially deep learning-based image compression techniques, poses new challenges to objective image quality assessment (IQA). Typical artifacts encountered in the emerging image codecs are significantly different from that produced by traditional block-based codecs, leading to inapplicability of the existing objective IQA algorithms. Towards advancing the development of objective IQA algorithms for recent compression artifacts, we built a learning-based compressed image quality assessment (LCIQA) database involving traditional block-based image codecs, hybrid neural network based image codecs, convolutional neural network based and generative adversarial network (GAN) based end-to-end optimized image coding approaches. Our study confirms the statistical difference and human perception difference between reconstructions of learned compression and traditional block-based compression. We propose a two-step deep learning model for learning-based compressed image quality assessment. Extensive experiments on LCIQA database demonstrate that our proposed model performs better than other counterparts on learning-based compressed images, especially on GAN compressed images, and achieves competitive performance to the state-of-the-art IQA metrics on traditional compressed images.     

Rate–distortion optimization of multi-exposure image coding for high dynamic range image coding

High dynamic range (HDR) images have many practical applications because they offer an extended dynamic range and a more realistic visual experience. A HDR image is usually stored in floating-point format, so pre-processing is required to make the HDR image compatible with coding standards. A transfer function is also used to achieve better coding efficiency. Typically, HDR images are generated using several low dynamic range (LDR) images with different exposures. Instead of compressing the HDR image when it is generated from images with multiple exposures, this study proposes a technique to compress the multi-exposure images. The HDR image generation, as well as the multi-exposure images fusion, can be realized in the decoder. The proposed framework encodes the multi-exposure images using MV-HEVC where the inter-view redundancy is well exploited when an accurate intensity-mapping function between the multi-exposure images has been established. Multi-exposure image coding is used to produce a high-quality HDR image so the rate–distortion optimization (RDO) is modified by considering both the reconstruction quality of the current block and its effect on the multi-exposure fused image. A Lagrange multiplier is modified to maintain a balance between the rate and the modified distortion during the RDO process. Compared to encoding the generated HDR image using HEVC range extension, the experimental results show that the proposed technique achieves significant bitrate savings for equivalent quality in terms of HDR-VDP-2.     

Foveation scalable video coding with automatic fixation selection

Image and video coding is an optimization problem. A successful image and video coding algorithm delivers a good tradeoff between visual quality and other coding performance measures, such as compression, complexity, scalability, robustness, and security. In this paper, we follow two recent trends in image and video coding research. One is to incorporate human visual system (HVS) models to improve the current state-of-the-art of image and video coding algorithms by better exploiting the properties of the intended receiver. The other is to design rate scalable image and video codecs, which allow the extraction of coded visual information at continuously varying bit rates from a single compressed bitstream. Specifically, we propose a foveation scalable video coding (FSVC) algorithm which supplies good quality-compression performance as well as effective rate scalability. The key idea is to organize the encoded bitstream to provide the best decoded video at an arbitrary bit rate in terms of foveated visual quality measurement. A foveation-based HVS model plays an important role in the algorithm. The algorithm is adaptable to different applications, such as knowledge-based video coding and video communications over time-varying, multiuser and interactive networks.     

Prioritized DCT for compression and progressive transmission ofimages

An approach is based on the block discrete cosine transform (DCT). The novelty of this approach is that the transform coefficients of all image blocks are coded and transmitted in absolute magnitude order. The resulting ordered-by-magnitude transmission is accomplished without sacrificing coding efficiency by using partition priority coding. Coding and transmission are adaptive to the characteristics of each individual image. and therefore, very efficient. Another advantage of this approach is its high progression effectiveness. Since the largest transform coefficients that capture the most important characteristics of images are coded and transmitted first, this method is well suited for progressive image transmission. Further compression of the image-data is achieved by multiple distribution entropy coding, a technique based on arithmetic coding. Experiments show that the approach compares favorably with previously reported DCT and subband image codecs.     

Noncausal predictive image codec

The paper describes a lossy image codec that uses a noncausal (or bilateral) prediction model coupled with vector quantization. The noncausal prediction model is an alternative to the causal (or unilateral) model that is commonly used in differential pulse code modulation (DPCM) and other codecs with a predictive component. We show how to obtain a recursive implementation of the noncausal image model without compromising its optimality and how to apply this in coding in much the same way as a causal predictor. We report experimental compression results that demonstrate the superiority of using a noncausal model based predictor over using traditional causal predictors. The codec is shown to produce high-quality compressed images at low bit rates such as 0.375 b/pixel. This quality is contrasted with the degraded images that are produced at the same bit rates by codecs using causal predictors or standard discrete cosine transform/Joint Photographic Experts Group-based (DCT/JPEG-based) algorithms.     

Hybrid model-and-object-based real-time conversational video coding

Bandwidth-constrained real-time conversational video communications (such as mobile teleconferencing) require video codecs with good rate-distortion characteristics at low bit-rates and modest computational complexity. While target-specific object-based and model-based coding methods have been proposed for low bit-rate conversational video coding, difficulties in generalization and high computational complexity hinder their practical utilization. In this paper, we propose a low bit-rate coding method for typical conversational video by combining two-dimensional model-based coding of face regions and object-based coding of non-face head-shoulder regions, achieving high-quality face reconstruction and low overall bit-rate with real-time encoding capability. Experiments on typical conversational test sequences confirm that, compared to other conversational video codecs, our model-and-object-based coding method offers superior rate-distortion performance at low bit-rates.     

Real-time 16K video coding on a GPU with complexity scalable BPC-PaCo

The advent of new technologies such as high dynamic range or 8K screens has enhanced the quality of digital images but it has also increased the codecs’ computational demands to process such data. This paper presents a video codec that, while providing the same coding features and performance as those of JPEG2000, can process 16K video in real time using a consumer-grade GPU. This high throughput is achieved with a technique that introduces complexity scalability to a bitplane coding engine, which is the most computationally complex stage of the coding pipeline. The resulting codec can trade throughput for coding performance depending on the user’s needs. Experimental results suggest that our method can double the throughput achieved by CPU implementations of the recently approved High-Throughput JPEG2000 and by hardwired implementations of HEVC in a GPU.     

率失真优化的压缩感知图像编码

针对基于压缩感知的图像编码系统,分析了系统中编码参数和码率以及失真的关系,在此基础上提出了基于压缩感知的图像编码系统的码率-失真模型.根据所提模型设计了率失真优化的压缩感知图像编码算法.在给定码率的条件下,优化编码参数,使得编码器失真最小.算法在Matlab的编码平台上进行了仿真和实验,结果证明提出的码率-失真模型能够很好地拟合实际率失真曲线,并且基于该模型的率失真优化算法有效的提高了压缩感知图像编码系统的性能.     

Multidimensional incremental parsing for universal source coding

A multidimensional incremental parsing algorithm (MDIP) for multidimensional discrete sources, as a generalization of the Lempel-Ziv coding algorithm, is investigated. It consists of three essential component schemes, maximum decimation matching, hierarchical structure of multidimensional source coding, and dictionary augmentation. As a counterpart of the longest match search in the Lempel-Ziv algorithm, two classes of maximum decimation matching are studied. Also, an underlying behavior of the dictionary augmentation scheme for estimating the source statistics is examined. For an m-dimensional source, m augmentative patches are appended into the dictionary at each coding epoch, thus requiring the transmission of a substantial amount of information to the decoder. The property of the hierarchical structure of the source coding algorithm resolves this issue by successively incorporating lower dimensional coding procedures in the scheme. In regard to universal lossy source coders, we propose two distortion functions, the local average distortion and the local minimax distortion with a set of threshold levels for each source symbol. For performance evaluation, we implemented three image compression algorithms based upon the MDIP; one is lossless and the others are lossy. The lossless image compression algorithm does not perform better than the Lempel-Ziv-Welch coding, but experimentally shows efficiency in capturing the source structure. The two lossy image compression algorithms are implemented using the two distortion functions, respectively. The algorithm based on the local average distortion is efficient at minimizing the signal distortion, but the images by the one with the local minimax distortion have a good perceptual fidelity among other compression algorithms. Our insights inspire future research on feature extraction of multidimensional discrete sources.     

Data-parallel intra decoding for block-based image and video coding on massively parallel architectures

With the increasing number of processor cores available in modern computing architectures, task or data parallelism is required to maximally exploit the available hardware and achieve optimal processing speed. Current state-of-the-art data-parallel processing methods for decoding image and video bitstreams are limited in parallelism by dependencies introduced by the coding tools and the number of synchronization points introduced by these dependencies, only allowing task or coarse-grain data parallelism. In particular, entropy decoding and data prediction are bottleneck coding tools for parallel image and video decoding. We propose a new data-parallel processing scheme for block-based intra sample and coefficient prediction that allows fine-grain parallelism and is suitable for integration in current and future state-of-the-art image and video codecs. Our prediction scheme enables maximum concurrency, independent of slice or tile configuration, while minimizing synchronization points. This paper describes our data-parallel processing scheme for one- and two-dimensional prediction and investigates its application to block-based image and video codecs using JPEG XR and H.264/AVC Intra as a starting point. We show how our scheme enables faster decoding than the state-of-the-art wavefront method with speedup factors of up to 21.5 and 7.9 for JPEG XR and H.264/AVC Intra coding tools respectively. Using the H.264/AVC Intra coding tool, we discuss the requirements of the algorithm and the impact on decoded image quality when these requirements are not met. Finally, we discuss the impact on coding rate in order to allow for optimal parallel intra decoding.     

基于灰度指纹图像信噪特征的无损压缩算法

本文提出了一个有效的针对灰度指纹图像的无损压缩算法.该算法通过将指纹图像的信噪特征融于新一代国际无损图像压缩标准的基本算法(基于上下文预测编码)而实现.该算法主要包括三个特色技术:基于纹线局部走向的分类预测、体现指纹微观纹理的扩展上下文以及基于成像仪器的分类熵编码器概率模型初始化.对一组真实数据及ISO标准图像的压缩结果表明该算法的压缩比居于国内外文献中的领先水平.     

Image segmentation towards new image representation methods

Very low bit-rate video coding has recently become one of the most important areas of image communication and a large variety of applications have already been identified. Since conventional approaches are reaching a saturation point, in terms of coding efficiency, a new generation of video coding techniques, aiming at a deeper “understanding” of the image, is being studied. In this context, image analysis, particularly the identification of objects or regions in images (segmentation), is a very important step. This paper describes a segmentation algorithm based on split and merge. Images are first simplified using mathematical morphology operators, which eliminate perceptually less relevant details. The simplified image is then split according to a quad tree structure and the resulting regions are finally merged in three steps: merge, elimination of small regions and control of the number of regions.     

Fractal coding of subbands with an oriented partition

We propose a new image compression scheme based on fractal coding of the coefficients of a wavelet transform, in order to take into account the self-similarity observed in each subband. The original image is first decomposed into subbands containing information in different spatial directions and at different scales, using an orthogonal wavelet-generated filter bank. Subbands are encoded using local iterated function systems (LIFS), with range and domain blocks presenting horizontal or vertical directionalities. Their sizes are defined according to the correlation lengths and resolution of each subband. The edge degradation and the blocking effects encountered at low bit-rates using conventional LIFS algorithm are reduced with this approach. The computation complexity is also greatly decreased by a 12:1 factor in comparison to fractal coding of the full resolution image. The proposed method is applied to standard test images. The comparison with other fractal coding approaches and with JPEG shows an important increase in terms of PPSNR/bit-rate. Especially for images presenting a privileged directionality, the use of adaptive partitions results in about 3 dB improvement in PPSNR. We also discuss the distorsion versus rate improvement obtained on high-frequency subbands when fractal coding instead of pyramidal vector quantization is used. Our approach achieves a real gain in PPSNR for low bit-rates between 0.3 and 1.2 bpp.