Performance enhancement of HEVC lossless mode using sample-based angular and planar predictions

High-efficiency video coding (HEVC) is the state-of-the-art video compression standard designed to handle the storage and transmission requirements of next-generation multimedia services. In the lossless mode HEVC, prevailing sample-based prediction algorithms in the literature have shown better prediction accuracy compared to the conventional block-based prediction within the HEVC anchor. This work proposes a sample-based prediction technique to modify the planar prediction mode of HEVC and a complete sample-based predictive encoder for the lossless mode of HEVC. In this work, we propose gradient adaptive sample-based intraprediction (GASP) as a replacement for the block-based planar prediction in HEVC. To obtain the benefits of sample-based prediction methods in both angular and planar prediction modes, we also propose a combination of ISAP, a sample-based angular prediction in the literature, with the newly proposed GASP (CIG). The experimental results demonstrate the superiority of GASP and CIG over other state-of-the-art sample-based prediction strategies in angular and planar prediction modes.     

Lossless video sequence compression using adaptive prediction.

We present an adaptive lossless video compression algorithm based on predictive coding. The proposed algorithm exploits temporal, spatial, and spectral redundancies in a backward adaptive fashion with extremely low side information. The computational complexity is further reduced by using a caching strategy. We also study the relationship between the operational domain for the coder (wavelet or spatial) and the amount of temporal and spatial redundancy in the sequence being encoded. Experimental results show that the proposed scheme provides significant improvements in compression efficiencies.     

Video-based body geometric aware network for 3D human pose estimation

Three-dimensional human pose estimation (3D HPE) has broad application prospects in the fields of trajectory prediction, posture tracking and action analysis. However, the frequent self-occlusions and the substantial depth ambiguity in two-dimensional (2D) representations hinder the further improvement of accuracy. In this paper, we propose a novel video-based human body geometric aware network to mitigate the above problems. Our network can implicitly be aware of the geometric constraints of the human body by capturing spatial and temporal context information from 2D skeleton data. Specifically, a novel skeleton attention (SA) mechanism is proposed to model geometric context dependencies among different body joints, thereby improving the spatial feature representation ability of the network. To enhance the temporal consistency, a novel multilayer perceptron (MLP)-Mixer based structure is exploited to comprehensively learn temporal context information from input sequences. We conduct experiments on publicly available challenging datasets to evaluate the proposed approach. The results outperform the previous best approach by 0.5 mm in the Human3.6m dataset. It also demonstrates significant improvements in HumanEva-I dataset.     

Lossless compression of video using temporal information

We consider the problem of lossless compression of video by taking into account temporal information. Video lossless compression is an interesting possibility in the line of production and contribution. We propose a compression technique which is based on motion compensation, optimal three-dimensional (3-D) linear prediction and context based Golomb-Rice (1966, 1979) entropy coding. The proposed technique is compared with 3-D extensions of the JPEG-LS standard for still image compression. A compression gain of about 0.8 bit/pel with respect to static JPEG-LS, applied on a frame-by-frame basis, is achievable at a reasonable computational complexity.     

Multi-view video coding with view interpolation prediction for 2D camera arrays

An efficient compression algorithm for multi-view video sequences, which are captured by two-dimensional (2D) camera arrays, is proposed in this work. First, we propose a novel prediction structure, called three-dimensional hierarchical B prediction (3DHBP), which can efficiently reduce horizontal inter-view redundancies, vertical inter-view redundancies, and temporal redundancies in multi-view videos. Second, we develop a view interpolation scheme based on the bilateral disparity estimation. The interpolation scheme yields high quality view frames by adapting disparity estimation and compensation procedures using the information in neighboring frames. Simulation results demonstrate that the proposed multi-view video coding algorithm provides significantly better rate–distortion (R–D) performance than the conventional algorithm, by employing the 3DHBP structure and using interpolated view frames as additional reference frames.     

Wavelet-based coding of time-varying vector fields of ocean-surface winds

Geoscience applications often produce sizable datasets that are vector-valued and increasingly in need of compression algorithms to reduce storage and transmission burdens, particularly when the data are time-varying. In this paper, several advanced interframe-compression techniques are extended from the traditional realm of natural video to the coding of time-varying vector fields. Although similar to natural video in some respects, time-varying vector-field sequences often possess complex temporal evolution of vector-valued features that are important to the analytic quality of the data yet defy the simple motion models widely employed for natural video. To improve coding performance, motion compensation with reduced resolution is proposed such that motion compensation is applied only at low spatial resolution, while high-resolution information, for which the motion model fails, is intraframe coded with no temporal decorrelation. In empirical results on datasets of ocean-surface winds, this reduced-resolution motion-compensation technique results in significant performance improvement and greater feature preservation.     

Lossless compression of color mosaic images.

Lossless compression of color mosaic images poses a unique and interesting problem of spectral decorrelation of spatially interleaved R, G, B samples. We investigate reversible lossless spectral-spatial transforms that can remove statistical redundancies in both spectral and spatial domains and discover that a particular wavelet decomposition scheme, called Mallat wavelet packet transform, is ideally suited to the task of decorrelating color mosaic data. We also propose a low-complexity adaptive context-based Golomb-Rice coding technique to compress the coefficients of Mallat wavelet packet transform. The lossless compression performance of the proposed method on color mosaic images is apparently the best so far among the existing lossless image codecs.     

Dynamic image data compression in spatial and temporal domains:theory and algorithm

Advanced medical imaging requires storage of large quantities of digitized clinical data. These data must be stored in such a way that their retrieval does not impair the clinician's ability to make a diagnosis. We propose a theory and algorithm for near lossless dynamic image data compression. Taking advantage of domain-specific knowledge related to medical imaging, medical practice and the dynamic imaging modality, a compression ratio greater than 80:1 is achieved. The high compression ratios are achieved by the proposed algorithm through three stages: (1) addressing temporal redundancies in the data through application of image optimal sampling, (2) addressing spatial redundancies in the data through cluster analysis, and (3) efficient coding of image data using standard still-image compression techniques. To illustrate the practicality of the algorithm, a simulated positron emission tomography (PET) study using the fluoro-deoxy-glucose (FDG) tracer is presented. Realistic dynamic image data are generated by virtual scanning of a simulated brain phantom as a real PET scanner. These data are processed using the conventional and proposed algorithms as well as the techniques for storage and analysis. The resulting parametric images obtained from the conventional and proposed approaches are subsequently compared to evaluate the proposed compression algorithm. The storage space for dynamic image data reduced by more than 95%, without loss in diagnostic quality. Therefore, the proposed theory and algorithm are expected to be very useful in medical image database management and telecommunication     

Lossless compression of video sequences

We investigate lossless compression schemes for video sequences. A simple adaptive prediction scheme is presented that exploits temporal correlations or spectral correlations in addition to spatial correlations. It is seen that even with motion compensation, schemes that utilize only temporal correlations do not perform significantly better than schemes that utilize only spectral correlations. Hence, we look at hybrid schemes that make use of both spectral and temporal correlations. The hybrid schemes give significant improvement in performance over other techniques. Besides prediction schemes, we also look at some simple error modeling techniques that take into account prediction errors made in spectrally and/or temporally adjacent pixels in order to efficiently encode the prediction residual. Implementation results on standard test sequences indicate that significant improvements can be obtained by the proposed techniques     

Piecewise DC prediction in HEVC

In this paper, we propose an improved DC prediction method for high-efficiency video coding (HEVC) intra coding. The technique involves the application of pixel-wise predictors rather than block-based predictor in the DC mode. In HEVC, the pixels of neighboring reconstructed blocks are used to support multiple directional spatial prediction modes and reduce spatial redundancy. For lossless coding, pixel-by-pixel differential pulse code modulation (DPCM) is applied to directional predictions. Consequently, residuals are reduced and coding efficiency is improved. Since DC prediction still employs block-based coding, pixel-wise DC prediction is needed for better coding efficiency. When compared to HEVC lossless intra coding, the proposed algorithm reduces the bit rate by 5.95%. Furthermore, with pixel-by-pixel DPCM, the average bit rate reduction is 10.64% when compared to HEVC lossless intra coding.     

Lossless compression of color sequences using optimal linear prediction theory

In this paper, we present a novel technique that uses the optimal linear prediction theory to exploit all the existing redundancies in a color video sequence for lossless compression purposes. The main idea is to introduce the spatial, the spectral, and the temporal correlations in the autocorrelation matrix estimate. In this way, we calculate the cross correlations between adjacent frames and adjacent color components to improve the prediction, i.e., reduce the prediction error energy. The residual image is then coded using a context-based Golomb-Rice coder, where the error modeling is provided by a quantized version of the local prediction error variance. Experimental results show that the proposed algorithm achieves good compression ratios and it is roboust against the scene change problem.     

Gradient-oriented directional predictor for HEVC planar and angular intra prediction modes to enhance lossless compression

Recent advancements in the capture and display technologies motivated the ITU-T Video Coding Experts Group and ISO/IEC Moving Picture Experts Group to jointly develop the High-Efficiency Video Coding (HEVC), a state-of-the-art video coding standard for efficient compression. The compression applications that essentially require lossless compression scenarios include medical imaging, video analytics, video surveillance, video streaming etc., where the content reconstruction should be flawless. In the proposed work, we present a gradient-oriented directional prediction (GDP) strategy at the pixel level to enhance the compression efficiency of the conventional block-based planar and angular intra prediction in the HEVC lossless mode. The detailed experimental analysis demonstrates that the proposed method outperforms the lossless mode of HEVC anchor in terms of bit-rate savings by 8.29%, 1.65%, 1.94% and 2.21% for Main-AI, LD, LDP and RA configurations respectively, without impairing the computational complexity. The experimental results also illustrates that the proposed predictor performs superior to the existing state-of-the-art techniques in the literature.     

Weighted finite automata for video compression

Weighted finite automata (WFA) exploit self-similarities within single pictures and also sequences of pictures to remove spatial and temporal redundancies. Their implementation then combines techniques from hierarchical methods related to quadtrees and from vector quantization to achieve performance results for low bit rates which can be put on a par with state-of-the-art codecs like embedded zero-tree wavelet coding. Due to their simple mathematical structure, WFA provide an ideal platform for efficient hybrid compression methods. Therefore, WFA were chosen as a starting point for a fractal-like video compression integrating a hierarchical motion compensation as well as an option to vary the compression quality between “centers of interest” and “background” in a flexible manner     

Rate-distortion-optimized predictive compression of dynamic 3D mesh sequences

Compression of computer graphics data such as static and dynamic 3D meshes has received significant attention in recent years, since new applications require transmission over channels and storage on media with limited capacity. This includes pure graphics applications (virtual reality, games) as well as 3DTV and free viewpoint video. Efficient compression algorithms have been developed first for static 3D meshes, and later for dynamic 3D meshes and animations. Standard formats are available for instance in MPEG-4 3D mesh compression for static meshes, and Interpolator Compression for the animation part. For some important types of 3D objects, e.g. human head or body models, facial and body animation parameters have been introduced. Recent results for compression of general dynamic meshes have shown that the statistical dependencies within a mesh sequence can be exploited well by predictive coding approaches. Coders introduced so far use experimentally determined or heuristic thresholds for tuning the algorithms. In video coding, rate-distortion (RD) optimization is often used to avoid fixed thresholds and to select the optimum prediction mode. We applied these ideas and present here an RD-optimized dynamic 3D mesh coder. It includes different prediction modes as well as an RD cost computation that controls the mode selection across all possible spatial partitions of a mesh to find the clustering structure together with the associated prediction modes. The general coding structure is derived from statistical analysis of mesh sequences and exploits temporal as well as spatial mesh dependencies. To evaluate the coding efficiency of the developed coder, comparative coding results for mesh sequences at different resolutions were carried out.     

Efficient Lossless Compression of 4-D Medical Images Based on the Advanced Video Coding Scheme

This paper presents an efficient lossless compression method for 4-D medical images based on the advanced video coding scheme (H.264/AVC). The proposed method efficiently reduces data redundancies in all four dimensions by recursively applying multiframe motion compensation. Performance evaluations on real 4-D medical images of varying modalities including functional magnetic resonance show an improvement in compression efficiency of up to three times that of other state-of-the-art compression methods such as 3D-JPEG2000.     

Estimation of depth fields suitable for video compression based on3-D structure and motion of objects

Intensity prediction along motion trajectories removes temporal redundancy considerably in video compression algorithms. In three-dimensional (3-D) object-based video coding, both 3-D motion and depth values are required for temporal prediction. The required 3-D motion parameters for each object are found by the correspondence-based E-matrix method. The estimation of the correspondences-two-dimensional (2-D) motion field-between the frames and segmentation of the scene into objects are achieved simultaneously by minimizing a Gibbs energy. The depth field is estimated by jointly minimizing a defined distortion and bit-rate criterion using the 3-D motion parameters. The resulting depth field is efficient in the rate-distortion sense. Bit-rate values corresponding to the lossless encoding of the resultant depth fields are obtained using predictive coding; prediction errors are encoded by a Lempel-Ziv algorithm. The results are satisfactory for real-life video scenes.     

Video compression using spatiotemporal regularity flow.

We propose a new framework in wavelet video coding to improve the compression rate by exploiting the spatiotemporal regularity of the data. A sequence of images creates a spatiotemporal volume. This volume is said to be regular along the directions in which the pixels vary the least, hence the entropy is the lowest. The wavelet decomposition of regularized data results in a fewer number of significant coefficients, thus yielding a higher compression rate. The directions of regularity of an image sequence depend on both its motion content and spatial structure. We propose the representation of these directions by a 3-D vector field, which we refer to as the spatiotemporal regularity flow (SPREF). SPREF uses splines to approximate the directions of regularity. The compactness of the spline representation results in a low storage overhead for SPREF, which is a desired property in compression applications. Once SPREF directions are known, they can be converted into actual paths along which the data is regular. Directional decomposition of the data along these paths can be further improved by using a special class of wavelet basis called the 3-D orthonormal bandelet basis. SPREF -based video compression not only removes the temporal redundancy, but it also compensates for the spatial redundancy. Our experiments on several standard video sequences demonstrate that the proposed method results in higher compression rates as compared to the standard wavelet based compression.     

Morphological shape representation for very low bit-rate video coding

Region-based coding schemes are among the most promising compression techniques for very low bit-rate applications. They consist of image segmentation, contour and texture coding. This paper deals with the use of the geodesic skeleton as a morphological tool for contour coding of segmented image sequences. In the geodesic case, already coded and known regions are taken into account for the coding of contours of unknown regions. A new technique is presented for the entropy coding of the coordinates of the skeleton points exploiting their special spatial distribution. Furthermore, a fast algorithm for the reconstruction of the skeleton points is given based on hierarchical queues. In the case of numerous isolated contour arcs (for example error coding in a motion prediction loop), the geodesic skeleton proofs higher efficiency than traditional methods. Results at very low bit-rates are presented and compared to standard methods confirming the validity of the chosen approach.     

Adaptive prediction trees for image compression.

This paper presents a complete general-purpose method for still-image compression called adaptive prediction trees. Efficient lossy and lossless compression of photographs, graphics, textual, and mixed images is achieved by ordering the data in a multicomponent binary pyramid, applying an empirically optimized nonlinear predictor, exploiting structural redundancies between color components, then coding with hex-trees and adaptive runlength/Huffman coders. Color palettization and order statistics prefiltering are applied adaptively as appropriate. Over a diverse image test set, the method outperforms standard lossless and lossy alternatives. The competing lossy alternatives use block transforms and wavelets in well-studied configurations. A major result of this paper is that predictive coding is a viable and sometimes preferable alternative to these methods.