A neural network-based video bit-rate control algorithm for variable bit-rate applications of versatile video coding standard

Versatile video coding is a new video coding standard that has more capabilities and higher coding efficiency compared with its predecessor. Practical video storage and transmission applications face constrained buffer size and available bandwidth. It is necessary to design the appropriate rate control algorithm to overcome such challenges. In this paper, the non-linear relationship between consumed bits, buffer size, and quantization parameter is estimated by taking the advantages of artificial neural networks, and a rate control algorithm is developed for real-time variable bit rate applications of the versatile video coding standard. The proposed rate control algorithm performs the control action in only one step that results in faster control action. The experimental results show that the proposed algorithm controls the bit-rate as well as the buffer state. Also, the rate–distortion analysis shows that the well-known $\lambda$-domain algorithm has only 2.7% bit-rate reduction in comparison with the proposed method.     

Very low bit-rate wavelet video coding

An image sequence coding scheme for very low bit-rate coding is presented. The scheme is based on spatial redundancy reduction via the new edge-sensitive subband coding method and temporal redundancy reduction via windowed overlapped block-matching motion compensation. In this scheme, only significant regions of difference images are coded. Thus, the computational cost can be kept low. Significant regions are considered to be regions where motion is present (temporal activity) and/or regions where the image is very detailed (spatial activity). The significant regions are determined by an adaptive strategy based on both the motion vectors and an edge map of the current image. The scheme has the main advantage that there are no blocking effects in the decoded images at all due to the properties of both the temporal and the spatial coding used. The overall visual performance of the coding scheme is very good     

Motion vector replensihment for low bit-rate video coding

A low bit-rate video codec based on motion vector replenishment is described. Motion vectors are used to update pictures at full frame rate. In addition, part of each frame is conditionally updated with a strip of interframe video data. The video data fill the remaining channel capacity not used for motion vectors. Thus under most conditions, each frame is fully updated by motion vectors and partially with interframe video data. This method has a comparable compression efficiency with that of the frame dropping method, but does not introduce any picture ‘jerkiness’. Finally the application of the proposed method to packet video networks is examined.     

Wavelet-based very low bit-rate video coding using image warpingand overlapped block motion compensation

The authors present an algorithm for very low bit-rate video coding that combines new ideas in motion estimation, wavelet filter design, and wavelet-based coding techniques. A new motion compensation technique using image warping and overlapped block motion compensation is proposed to reduce temporal redundancies in a given image sequence. This combined motion model has the advantage of representing more complex motion than simple block matching schemes. To further improve the quality of the temporal prediction, an adaptive grid with variable density according to the varying motion activity of a given scene is generated. An adaptively switched high-quality texture interpolation is employed to cope with the problem of fractional displacements in such a way that both objective and subjective reconstruction quality is improved. Spatial decorrelation of the motion compensated residual images is performed using an one-parametric family of biorthogonal infinite impulse response (IIR) wavelet filters coupled with the highly efficient pre-coding scheme of `partitioning, aggregation and conditional coding' (PACC). Experimental results demonstrate significant improvements in objective quality of 1.0-2.3 dB PSNR in comparison to the H.263+ test model TMN10 using advanced coding options. In addition, the authors' intracoding method provides a performance gain of 0.5 dB PSNR on the average for a test suite of various still images when compared to the emerging still image coding standard JPEG-2000     

Variable bit-rate coding of video signals for ATM networks

Statistical characteristics of video signals for video packet coding, are clarified and a variable-bit-rate coding method for asynchronous transfer mode (ATM) networks is described that is capable of compensating for packet loss. ATM capabilities are shown to be greatly affected by delay, delay jitter, and packet loss probability. Packet loss has the greatest influence on picture quality. Packets may be lost either due to random bit error in a cell header or to network control when traffic is congested. A layered coding technique using discrete-cosine transform (DCT) coding is presented which is suitable for packet loss compensation. The influence of packet loss on picture quality is discussed, and decoded pictures with packet loss are shown. The proposed algorithm was verified by computer simulations     

Morphological shape representation for very low bit-rate video coding

Region-based coding schemes are among the most promising compression techniques for very low bit-rate applications. They consist of image segmentation, contour and texture coding. This paper deals with the use of the geodesic skeleton as a morphological tool for contour coding of segmented image sequences. In the geodesic case, already coded and known regions are taken into account for the coding of contours of unknown regions. A new technique is presented for the entropy coding of the coordinates of the skeleton points exploiting their special spatial distribution. Furthermore, a fast algorithm for the reconstruction of the skeleton points is given based on hierarchical queues. In the case of numerous isolated contour arcs (for example error coding in a motion prediction loop), the geodesic skeleton proofs higher efficiency than traditional methods. Results at very low bit-rates are presented and compared to standard methods confirming the validity of the chosen approach.     

Low bit-rate video coding using wavelet vector quantisation

A method for low bit-rate video coding based on wavelet vector quantisation is proposed. Motion estimation/compensation using overlapped block matching (OBM) is employed to eliminate the blocking effects in the prediction error introduced by conventional block matching. It is shown that OBM significantly increases the efficiency of the wavelet transform coder. The motion-compensated interframe prediction error is decomposed using a wavelet transform and a method is employed for the efficient coding of the wavelet coefficients. In this technique, the coefficients are coded with a zero-tree multistage lattice vector quantiser. Simulation results are provided to evaluate the coding performance of the described coding scheme for low bit-rate video coding. It provides constant bit rate, obviating the need for buffer, with just small fluctuations in PSNR. Moreover, comparison with the RM8 implementation of the standard H261 video coder shows that the presented codec provides improvements in both peak signal-to-noise ratio and picture quality     

Low bit-rate video coding with spatio-temporal geometric transforms

A low bit-rate video coding technique that uses spatio-temporal geometric transforms is presented. Motion compensation based on the bilinear transform is employed to reduce the temporal redundancy of the video. The spatial redundancy of the motion compensated error images is reduced by a combination of fractals and the DCT. It is shown that in the objects boundaries of the motion compensated error image fractals outperforms the DCT, while in the smooth areas the DCT is better than fractals. A hybrid combination of fractals and the DCT gives the best result. The performance of this hybrid codec with geometrically transformed motion compensation is compared against the H.261 standard video codec at 64 kbit/s     

Very low bit-rate video coding with DFD segmentation

A video coding system for applications requiring very low bit-rate is presented. This coding scheme uses an intraframe coder for the initial frame in the video sequence and subsequent frames are coded using an interframe coding method. A wavelet-based technique is used for intraframe coding. For interframe coding, displaced frame differences (DFD) are computed and coded using a segmentation-based method wherein the displaced frame difference is segmented into active and inactive regions using morphological operators. To meet the very low bit-rate requirements, the motion vectors are processed so as to reduce their contribution to the overall bit-rate. To reduce coding artifacts, a post-processing technique is developed for use at the decoder. Coding performance of the proposed coding scheme is evaluated at 16 kbit/s and 32 kbit/s using luminance component of several typical test sequences at QCIF resolution with a frame rate 8.3 frame/s.     

Spatio-temporal segmentation of image sequences for object-oriented low bit-rate image coding

This paper describes a new method of segmentation of time-varying image sequences whose goal is object-oriented image coding. The segmentation represents a partition of each frame of the sequence into a set of regions which are homogeneous with regard to motion criterion. The region borders correspond to spatial contours of objects in the frame. Each spatio-temporal region is characterized by its temporal component, which is a model-dependent vector of motion parameters, and a structural component representing the polygonal approximation of the spatial contour of the region.

The construction of spatio-temporal segmentation includes two phases: the initialization step and temporal tracking. The initialization step is based on the spatial segmentation of the first frame of the sequence. Then homogeneous spatial regions are merged through motion estimation in accordance with a motion-based criterion. The temporal tracking consists of the projection of the segmentation along the time axis, and its adjustment. Special attention is paid to the processing of occlusions.

A predictive coding scheme is proposed which is based on the temporal coherence of the segmentation. This scheme is promising for a low bit-rate image compression.

The results for teleconference and TV sequences show the high visual quality of reconstructed only by prediction images. Moreover, the bit-rates for motion coding are very low: from 0.002 to 0.007 bit/pixel for teleconference sequence and from 0.004 to 0.021 bit/pixel for complex TV sequence. A scheme for encoding of the structural information is proposed which requires 0.083 – 0.17 bit per pixel depending on the content of the sequence.     

两种实时、低码率视频编码的运动估计搜索算法

先提出实时、低码率视频编码对运动估计搜索算法在运算时间和准确度上的要求，总结归纳了钻石搜索法和两步法的具体算法，通过与几种经典算法作比较，说明这两种算法更优，更适合用于实时，低码率视频编码，最后，以H．263的实现为例，说明了钻石搜索算法的具体实现。     

一种甚低码率下的面向对象小波变换编码技术

本文结合小波分解和ＭＰＥＧ－４的图像分割技术的优点,提出面向对象的小波变换编码技术。本方案在甚低码率的信道下,解码端可以根据接收到的部分码流恢复出原始图像的结构信息,对带宽不固定的信道有很好的自适应性。文中还对如何在有噪声的信道下进行错误纠正进行了探讨。     

Very low bit-rate video coding using variable block-sizeentropy-constrained residual vector quantizers

We present a practical video coding algorithm for use at very low bit rates. For efficient coding at very low bit rates, it is important to intelligently allocate bits within a frame, and so a powerful variable-rate algorithm is required. We use vector quantization to encode the motion-compensated residue signal in an H.263-like framework. For a given complexity, it is well understood that structured vector quantizers perform better than unstructured and unconstrained vector quantizers. A combination of structured vector quantizers is used in our work to encode the video sequences. The proposed codec is a multistage residual vector quantizer, with transform vector quantizers in the initial stages. The transform-VQ captures the low-frequency information, using only a small portion of the bit budget, while the later stage residual VQ captures the high-frequency information, using the remaining bits. We used a strategy to adaptively refine only areas of high activity, using recursive decomposition and selective refinement in the later stages. An entropy constraint was used to modify the codebooks to allow better entropy coding of the indexes. We evaluate the performance of the proposed codec, and compare this data with the performance of the H.263-based codec. Experimental results show that the proposed codec delivered significantly better perceptual quality along with better quantitative performance     

Error resilient video coding techniques

We review error resilience techniques for real-time video transport over unreliable networks. Topics covered include an introduction to today's protocol and network environments and their characteristics, encoder error resilience tools, decoder error concealment techniques, as well as techniques that require cooperation between encoder, decoder, and the network. We provide a review of general principles of these techniques as well as specific implementations adopted by the H.263 and MPEG-4 video coding standards. The majority of the article is devoted to the techniques developed for block-based hybrid coders using motion-compensated prediction and transform coding. A separate section covers error resilience techniques for shape coding in MPEG-4     

A 3D-DCT video encoder using advanced coding techniques for low power mobile device

The three-dimensional discrete cosine transform (3D-DCT) has been researched as an alternative to existing dominant video standards based on motion estimation and compensation. Since it does not need to search macro block for inter/intra prediction, 3D-DCT has great advantages for complexity. However, it has not been developed well because of poor video quality while video standards such as H.263(+) and HEVC have been blooming. In this paper, we propose a new 3D-DCT video coding as a new video solution for low power mobile technologies such as Internet of Things (IoT) and Drone. We focus on overcoming drawbacks reported in previous research. We build a complete 3D-DCT video coding system by adopting existing advanced techniques and devising new coding algorithms to improve overall performance of 3D-DCT. Experimental results show proposed 3D-DCT outperforms H.264 low power profiles while offering less complexity. From GBD-PSNR, proposed 3D-DCT provides better performance by average 4.6 dB.     

Low bit-rate cloud-based image coding in the wavelet transform domain

The rapid growth of image resources on the Internet makes it possible to find some highly correlated images on some Web sites when people plan to transmit an image over the Internet. This study proposes a low bit-rate cloud-based image coding scheme, which utilizes cloud resources to implement image coding. Multiple- discrete wavelet transform was adopted to decompose the input image into a low-frequency sub-band and several high-frequency sub-bands. The low-frequency sub-band image was used to retrieve highly correlated images (HCOIs) in the cloud. The highly correlated regions in the HCOIs were used to reconstruct the high-frequency sub-bands at the decoder to save bits. The final reconstructed image was generated using multiple inverse wavelet transform from a decompressed low-frequency sub-band and reconstructed high-frequency sub-bands. The experimental results showed that the coding scheme performed well, especially at low bit rates. The peak signal-to-noise ratio of the reconstructed image can gain up to 7 and 1.69 dB over JPEG and JPEG2000 under the same compression ratio, respectively. By utilizing the cloud resources, our coding scheme showed an obvious advantage in terms of visual quality. The details in the image can be well reconstructed compared with both JPEG, JPEG2000, and intracoding of HEVC.     

Error-resilient image and video coding for wireless communicationsystems

Current techniques for coding images and video sources with resilience to channel errors can remove much of the need for complex high-redundancy channel coding and provide a graceful degradation of performance with decreasing channel quality. The main function of these error-resilient techniques is to reduce the propagation of errors within the decoded data. Two main techniques are discussed in detail: the error-resilient entropy code (EREC) and pyramid vector quantisation (PVQ). The paper concludes with a comparison of the relative merits of these systems and areas for further consideration     

An object-background image model for predictive video coding

The author proposes a three-state image model whose elements are named object, background, and new-scene after their representation of the corresponding structural situation. An interframe state transition diagram controlled by luminance differences is introduced. The model is then utilized in a background luminance updating algorithm based on a maximum a posteriori probability. Applications of the model and the updating algorithm in predictive video coding at low bit rate (64 kb/s and less) are suggested     

Low bit-rate image coding using adaptive geometric piecewise polynomial approximation.

We present a new image coding algorithm, the geometric piecewise polynomials (GPP) method, that draws on recent developments in the theory of adaptive multivariate piecewise polynomials approximation. The algorithm relies on a segmentation stage whose goal is to minimize a functional that is conceptually similar to the Mumford-Shah functional except that it measures the smoothness of the segmentation instead of the length. The initial segmentation is "pruned" and the remaining curve portions are lossy encoded. The image is then further partitioned and approximated by low order polynomials on the subdomains. We show examples where our algorithm outperforms state-of-the-art wavelet coding in the low bit-rate range. The GPP algorithm significantly outperforms wavelet based coding methods on graphic and cartoon images. Also, at the bit rate 0.05 bits per pixel, the GPP algorithm achieves on the test image Cameraman, which has a geometric structure, a PSNR of 21.5 dB, while the JPEG2000 Kakadu software obtains PSNR of 20 dB. For the test image Lena, the GPP algorithm obtains the same PSNR as JPEG2000, but with better visual quality at 0.03 bpp.