Picture Coding

A survey of recent German research in the field of picture coding is presented. The described coding methods are mainly based on extended differential pulse-code modulation (DPCM) techniques. A unified model of a feedback switched quantizer for picture coding is explained. Theoretical results show a 9-dB gain in SNR over that of DPCM. A simple realization is shown. For encoding the 1-MHz videotelephone signals, a two-stage coding system consisting of a two-dimensional DPCM in the first stage and a dot interlaced frame repeating codec in the second stage is described. A DPCM combined with a relevancy detector and runlength coder is used for coding the 5-MHz video signals of a high-resolution videotelephone. A special scanning technique assures compatibility with ordinary videotelephones. Good picture quality is obtained by coding the luminance and chrominance signals of color TV separately with DPCM and switched quantization into a 34 Mbit/s signal.     

Bounds on Quantizer Performance in the Low Bit-Rate Region

This paper investigates the optimum entropy versus distortion performance of quantizers optimized for uniform, Gaussian, Laplacian, and gamma-distributed memoryless sources which are useful models of the quantizer input signals in speech or picture coding schemes. We list the maximally obtainable signal-to-quantization noise ratios for one-dimensional optimum (i.e. entropy-coded) quantizers in the important low bit-rate region. These results have been obtained by an iterative solution of a set of nonlinear equations. Additionally we have also computed the corresponding rate-distortion functions by employing the Blahut-algorithm. These latter results upperbound the performances of multi-dimensional quantization schemes, and a comparison with the former results indicates the penalty to be paid for restricting a coder to perform a one-dimensionai quantization. It will be shown that the differences can be significant in the low bit-rate region.     

On quantizers for DPCM coding of picture signals

A measure of picture quality for simple element, differentially coded pictures is developed based on certain subjective tests. The measure weights the quantization noise according to its visibility. It is shown that the measure correlates well with the picture quality determined on a standard impairment scale. Optimization of DPCM quantizers is done for this and for the mean-square measure of picture quality. Performance of the following types of quantizers is evaluated in terms of entropy of the quantized output and the picture quality: a) minimum mean-square error quantizers with a fixed number of levels, b) minimum mean-square error quantizers with fixed entropy, c) minimum mean-square subjective distortion quantizers with a fixed number of levels, d) minimum mean-square subjective distortion quantizers with fixed entropy, and e) uniform quantizers. It is concluded that for a fixed number of levels and a fixed word-length coding of the quantizer outputs, the quantizers in c) outperform those in a); and with variable length coding, the quantizers in d) perform better than all of the other quantizers having the same entropy. The sensitivity of the approach to variation of picture content is also investigated.     

On the Design of Quantizers for DPCM Coders: A Functional Relationship Between Visibility, Probability and Masking

Visibility functions measure the relative visibility of noise added to a picture at those points where some measure of local activity exceeds a given threshold. The functions are obtained from a series of subjective experiments and vary with the content of the picture. Visibility functions have been used to design quantizing characteristics for DPCM coding of monochrome and color signals and for three-dimensional transform coding. We consider an alternative approach to determining the visibility function that obviates the need for repeated picture-dependent subjective tests. The visibility function is assumed to consist of two parts, a picture-dependent component and viewer-dependent component (referred to as the masking function). The visibility function may be approximated by the quotient of a probability density function raised to a power and the masking function. The role of probability is found to be weaker where the viewer has more opportunity to scrutinize the picture.     

DPCM Quantization of Color Television Signals

The application of DPCM to the coding of color television signals calls for the design of the quantization characteristics for the luminance and the two color difference components. In this paper we describe quantizer designs based on visibility thresholds of quantization noise measured as a function of prediction error for a number of test slides. We assume a quantizer for the luminance component designed previously by a similar procedure and conduct psychovisual tests for theUandVcolor components. The results show that, mainly for granular noise, there is some visual superposition of quantization noise between the luminance and theUchrominance signals, while little or no visual interaction is evident between the luminance and theVsignal impairments. The quantizers for theUandVcomponents are designed such that, with the previously designed luminance quantizer, the number of levels are minimized without exceeding the visibility thresholds. We conclude that a total of 6 bits per color sample are required to code theUandVcomponents together at 4.4 MHz.     

Adaptive image coding with perceptual distortion control

This paper presents a discrete cosine transform (DCT)-based locally adaptive perceptual image coder, which discriminates between image components based on their perceptual relevance for achieving increased performance in terms of quality and bit rate. The new coder uses a locally adaptive perceptual quantization scheme based on a tractable perceptual distortion metric. Our strategy is to exploit human visual masking properties by deriving visual masking thresholds in a locally adaptive fashion. The derived masking thresholds are used in controlling the quantization stage by adapting the quantizer reconstruction levels in order to meet the desired target perceptual distortion. The proposed coding scheme is flexible in that it can be easily extended to work with any subband-based decomposition in addition to block-based transform methods. Compared to existing perceptual coding methods, the proposed perceptual coding method exhibits superior performance in terms of bit rate and distortion control. Coding results are presented to illustrate the performance of the presented coding scheme.     

Conditional Run-Length and Variable-Length Coding of Digital Pictures

In earlier publications, we have presented two coding schemes which take into account the conditional statistics of input signals. In the first scheme, the codewords are assigned in such a way as to provide a signal with long runs of zeros and ones. In the second scheme, each picture element is coded by variable-length codewords according to the values of previously transmitted PEL's. In this paper, by providing further results, we examine these coding schemes in greater detail. The performance of both schemes in terms of entropy and bit rate are compared with an optimum predictive coder. The simulation results indicate that these schemes have a significant advantage over standard predictive encoders. Methods to reduce the storage requirement for the encoder and decoder codebooks are also discussed.     

Just noticeable distortion model and its applications in video coding

We explore a new perceptually-adaptive video coding (PVC) scheme for hybrid video compression, in order to achieve better perceptual coding quality and operational efficiency. A new just noticeable distortion (JND) estimator for color video is first devised in the image domain. How to efficiently integrate masking effects together is a key issue of JND modelling. We integrate spatial masking factors with the nonlinear additivity model for masking (NAMM). The JND estimator applies to all color components and accounts for the compound impact of luminance masking, texture masking and temporal masking. Extensive subjective viewing confirms that it is capable of determining a more accurate visibility threshold that is close to the actual JND bound in human eyes. Secondly, the image-domain JND profile is incorporated into hybrid video encoding via the JND-adaptive motion estimation and residue filtering process. The scheme works with any prevalent video coding standards and various motion estimation strategies. To demonstrate the effectiveness of the proposed scheme, it has been implemented in the MPEG-2 TM5 coder and demonstrated to achieve average improvement of over 18% in motion estimation efficiency, 0.6 dB in average peak signal-to perceptual-noise ratio (PSPNR) and most remarkably, 0.17 dB in the objective coding quality measure (PSNR) on average. Theoretical explanation is presented for the improvement on the objective coding quality measure. With the JND-based motion estimation and residue filtering process, hybrid video encoding can be more efficient and the use of bits is optimized for visual quality.     

Human visual weighted quantization : a unified approach for subband and pyramidal image coders

In the field of the coding of digital video signals, the concept of hierarchical coding is of real interest. Decorrelation operators fulfilling that hierarchical requirement are subband operators and pyramidal systems. The aim of this paper is show that the same methodology can be used to solve the quantization problem in subband and pyramidal coders. The criterion considered here takes into account the eye sensitivity to spatial frequencies. An analytical approach is presented in that paper for the cases of fixed bit allocation and quantizer followed by an entropy coder.     

Predictive Coding of Speech at Low Bit Rates

Predictive coding is a promising approach for speech coding. In this paper, we review the recent work on adaptive predictive coding of speech signals, with particular emphasis on achieving high speech quality at low bit rates (less than 10 kbits/s). Efficient prediction of the redundant structure in speech signals is obviously important for proper functioning of a predictive coder. It is equally important to ensure that the distortion in the coded speech signal be perceptually small. The subjective loudness of quantization noise depends both on the short-time spectrum of the noise and its relation to the short-time spectrum of the Speech signal. The noise in the formant regions is partially masked by the speech signal itself. This masking of quantization noise by speech signal allows one to use low bit rates while maintaining high speech quality. This paper will present generalizations of predictive coding for minimizing subjective distortion in the reconstructed speech signal at the receiver. The quantizer in predictive coders quantizes its input on a sample-by-sample basis. Such sample-by-sample (instantaneous) quantization creates difficulty in realizing an arbitrary noise spectrum, particularly at low bit rates. We will describe a new class of speech coders in this paper which could be considered to be a generalization of the predictive coder. These new coders not only allow one to realize the precise optimum noise spectrum which is crucial to achieving very low bit rates, but also represent the important first step in bridging the gap between waveform coders and vocoders without suffering from their limitations.     

Synthetic high coding for the pal luminance signal

For the transmission of PAL TV signals, the data rate of the luminance signal should not exceed 3-3.4 bit/pel, and the quality should fulfil the TV signal transmission standard. We suggested a synthetic high coding combined with DPCM coding for the luminance signal to meet the requirement. A 1-D FIR filter is used for filtering out the low signals. The sampling rate of low signal is lowered by subsampling and the signals are then coded in the usual DPCM manner. The difference of the luminance signal of present picture element with its previous picture element, when it exceeds a threshold, is the edge signal. The edges are then separated into two kinds, the relevant edges and non-relevant edges, according to the masking effect of visual perception. Those edges that can be discarded, which can not be perceived visually when the reconstructed picture is compared with the original picture, are non-relevant edges. The relevant signals and their addresses are coded and called the high signal. We have obtained a coding rate of 3.3 bit/pel for the luminance signal. The quality of the coded picture is similar to an 8 bit/pel PCM picture by subjective test. The errors in different steps of the coding are discussed. The limitation of the method is also discussed.     

Visual fidelity criterion and modeling

Fidelity measures and criteria for visual communications are discussed. It is recognized that the basis of visual fidelity assessment is subjective judgement of reproduced pictures. However, in design and elsewhere there is also need for explicit evaluation of visual communication waveforms. The rate distortion theory model for such evaluations is brought out and two existing evaluations of limited scope, viz., weighted noise measurement and K rating, are reviewed in relation to that theory. It is suggested that video distortion can largely be identified with visibility of errors in reproduction. Relevant findings on luminance vision are examined. It is found that a comprehensive model of visibility would incorporate excitation, inhibition, and masking, and that considerable quantitative knowledge of these factors already exists. Looking ahead, a block schematic is given of a meter which might measure distortion produced by quantization noise in a video feed back quantizer.     

Conditional entropy-constrained residual VQ with application toimage coding

This paper introduces an extension of entropy constrained residual vector quantization (VQ) where intervector dependencies are exploited. The method, which we call conditional entropy-constrained residual VQ, employs a high-order entropy conditioning strategy that captures local information in the neighboring vectors. When applied to coding images, the proposed method is shown to achieve better rate-distortion performance than that of entropy-constrained residual vector quantization with less computational complexity and lower memory requirements, moreover, it can be designed to support progressive transmission in a natural way. It is also shown to outperform some of the best predictive and finite-state VQ techniques reported in the literature. This is due partly to the joint optimization between the residual vector quantizer and a high order conditional entropy coder as well as the efficiency of the multistage residual VQ structure and the dynamic nature of the prediction.     

Coding isotropic images

Rate-distortion functions for 2-dimensional homogeneous isotropic images are compared with the performance of five source encoders designed for such images. Both unweighted and frequency weighted mean-square error distortion measures are considered. The coders considered are a) differential pulse code modulation (DPCM) using six previous samples or picture elements (pels) in the prediction--herein called 6-pel DPCM, b) simple DPCM using single-sample prediction, c) 6-pel DPCM followed by entropy coding, d)8 times 8discrete cosine transform coding, and e)4 times 4Hadamard transform coding. Other transform coders were studied and found to have about the same performance as the two transform coders above. With the mean-square error distortion measure, 6-pel DPCM with entropy coding performed best. Next best was the8 times 8discrete cosine transform coder and the 6-pel DPCM--these two had approximately the same distortion. Next were the4 times 4Hadamard and simple DPCM, in that order. The relative performance of the coders changed slightly when the distortion measure was frequency weighted mean-square error. FromR = 1to 3 bits/pel, which was the range studied here, the performances of all the coders were separated by only about 4 dB.     

Alphabet-constrained vector quantization

Alphabet-constrained rate-distortion theory is extended to coding of sources with memory. Two different cases are considered: when only the size of the codebook is constrained and when the codevector values are also held fixed. For both cases, nth-order constrained-alphabet rate-distortion functions are defined and a convergent algorithm for their evaluation is presented. Specific simulations using AR(1) sources show that performance near the rate-distortion bound is possible using a reproduction alphabet consisting of a small number of codevectors. It is also shown that the additional constraint of holding the codevector values fixed does not degrade performance of the coder in relation to the size-only constrained case. This observation motivates the development of a fixed-codebook vector quantizer, called the alphabet- and entropy-constrained vector quantizer, the performance of which is comparable to the entropy-constrained vector quantizer. A number of examples using an AR(1) and a speech source are presented to corroborate the theory     

Data compression of stereopairs

Two fundamentally different techniques for compressing stereopairs are discussed. The first technique, called disparity-compensated transform-domain predictive coding, attempts to minimize the mean-square error between the original stereopair and the compressed stereopair. The second technique, called mixed-resolution coding, is a psychophysically justified technique that exploits known facts about human stereovision to code stereopairs in a subjectively acceptable manner. A method for assessing the quality of compressed stereopairs is also presented. It involves measuring the ability of an observer to perceive depth in coded stereopairs. It was found that observers generally perceived objects to be further away in compressed stereopairs than they did in originals. It is proved that the rate distortion limit for coding stereopairs cannot in general be achieved by a coder that first codes and decodes the right picture sequence independently of the left picture sequence, and then codes and decodes the left picture sequence given the decoded right picture sequence     

Locally adaptive perceptual image coding

Most existing efforts in image and video compression have focused on developing methods to minimize not perceptual but rather mathematically tractable, easy to measure, distortion metrics. While nonperceptual distortion measures were found to be reasonably reliable for higher bit rates (high-quality applications), they do not correlate well with the perceived quality at lower bit rates and they fail to guarantee preservation of important perceptual qualities in the reconstructed images despite the potential for a good signal-to-noise ratio (SNR). This paper presents a perceptual-based image coder, which discriminates between image components based on their perceptual relevance for achieving increased performance in terms of quality and bit rate. The new coder is based on a locally adaptive perceptual quantization scheme for compressing the visual data. Our strategy is to exploit human visual masking properties by deriving visual masking thresholds in a locally adaptive fashion based on a subband decomposition. The derived masking thresholds are used in controlling the quantization stage by adapting the quantizer reconstruction levels to the local amount of masking present at the level of each subband transform coefficient. Compared to the existing non-locally adaptive perceptual quantization methods, the new locally adaptive algorithm exhibits superior performance and does not require additional side information. This is accomplished by estimating the amount of available masking from the already quantized data and linear prediction of the coefficient under consideration. By virtue of the local adaptation, the proposed quantization scheme is able to remove a large amount of perceptually redundant information. Since the algorithm does not require additional side information, it yields a low entropy representation of the image and is well suited for perceptually lossless image compression.     

Luminance Adaptive Coding of Chrominance Signals

We describe two techniques for digital coding of the chrominance components of a color television signal. Both techniques make use of an observation that in color pictures most of the locations of large spatial changes in the chrominance are coincident with large spatial changes in the luminance. This allows us to predict the chrominance samples more efficiently using the previously transmitted chrominance and luminance samples, and the present luminance sample. In general, we determine which of the previous luminance samples best represents the present luminance sample and use the corresponding previous chrominance sample to represent the present chrominance sample. We present results of computer simulations of two such coding schemes. The first scheme, in which the chrominance components are coded by a DPCM coder, uses adaptive prediction of the chrominance components based on the luminance. In the second scheme, the chrominance signal is adaptively extrapolated from its past using the luminance signal for adaptation. Only those chrominance samples where the extrapolation error is more than a threshold are transmitted to the receiver. The addresses of such samples are derived from the luminance signal and therefore need not be transmitted. Our computer simulations on videotelephone type of pictures, indicate that, for the predictive coding, the entropy of the coded chrominance signals can he reduced by about 15 to 20 percent by adaptation. This results in a bit rate of 0.55 bits/ luminance pel, for transmission of chrominance information. Using adaptive extrapolation, only about 20 percent of the chrominance samples need to be transmitted which results in a bit rate of approximately 0.58 bits/luminance pel.     

Design of DPCM Quantizers for Video Signals Using Subjective Tests

The application of differential pulse code modulation (DPCM) for broadcast color television signals requires a design which produces no visible impairments under normal viewing conditions. This paper describes a quantizer design which is based on measured visibility thresholds of the various kinds of DPCM impairments such as granular noise, edge busyness, and slope overload. The visibility thresholds are determined by subjective tests based on comparisons of DPCM and PCM encoded pictures. Constructions of quantizers are carried out such that the number of levels is minimized without exceeding the measured visibility thresholds. Besides nonadaptive quantizers, adaptive quantizers are also constructed which are controlled by the signal changes of surrounding picture elements. These investigations show that for component coding of color video signals with two-dimensional prediction, a transmission rate of 31.7 Mbits/s is possible for natural types of test pictures without visible impairments using constant word length coding.     

Motion-Compensated Coder

In an earlier paper, an extension of the pel recursive techniques of Netravali and Robbins [2] and Cafforio and Rocca [3] was introduced. Here a method is provided to realize the algorithm in hardware, with some approximations. The prediction error distribution allows the use of quantized variables to a lookup table of reasonable size. The algorithm is then incorporated into a simple multimode coder capable of 1.5 bits/pel on the sequence examined. The coder incorporates a spot filter, quantizer, block run length coding, and variable word length coding and subsampling. Simulation results are presented, including bit rate, buffer status, and mode control analysis.