Optimum pulse amplitude modulation--II: Inclusion of timing jitter

A method is presented for specifying transmitter waveforms and receiver filters for pulse amplitude modulation (PAM) systems which are subject to distortion from additive noise, intersymbol interference, and timing jitter. A criterion of minimum mean-squared error is used. For the case of band-limited transmission, specific procedures are developed for computing optimum transmitters, optimum receivers, and jointly optimum transmitters and receivers. For two special jitter distributions the joint optimization problem is solved without a bandwidth limitation, and the resulting systems are found to be band-limited. In cases of localized timing jitter and high signal-to-noise ratio, we are led to system pulse responses of the form[sin x/x]^{2}. A nonparametric method of reducing the effects of timing jitter is briefly discussed. In this approach the noise interference is minimized subject to the constraints that the intersymbol interference be zero and that the system pulse response have zero derivative at each nominal sampling point.     

Source and Channel Coding for Cooperative Relaying

User cooperation is a powerful tool to combat fading and increase robustness for communication over wireless channels. Although it is doubtless a promising technique for enhancing channel reliability, its performance in terms of average source distortion is not clear since source-channel separation theorem fails under the most common nonergodic slow-fading channel assumption, when channel state information (CSI) is only available at the receiving terminals. This work sheds some light on the end-to-end performance of joint source-channel coding for cooperative relay systems in the high signal-to-noise ratio (SNR) regime. Considering distortion exponent as a figure of merit, we propose various strategies for cooperative source and channel coding that significantly improve the performance compared to the conventional scheme of source coding followed by cooperative channel coding. We characterize the optimal distortion exponent of a full-duplex relay channel for all bandwidth ratios. For the half-duplex relay channel, we provide an upper bound which is tight for small and large bandwidth ratios. We consider the effect of correlated side information on the distortion exponent as well.     

Minimum mean-squared error transform coding and subband coding

Knowledge of the power spectrum of a stationary random sequence can be used for quantizing the signal efficiently and with minimum mean-squared error. A multichannel filter is used to transform the random sequence into an intermediate set of variables that are quantized using independent scalar quantizers, and then inverse-filtered, producing a quantized version of the original sequence. Equal word-length and optimal word-length quantization at high bit rates is considered. An analytical solution for the filter that minimizes the mean-squared quantization error is obtained in terms of its singular value decomposition. The performance is characterized by a set of invariants termed second-order modes, which are derived from the eigenvalue decomposition of the matrix-valued power spectrum. A more general rank-reduced model is used for decreasing distortion by introducing bias. The results are specialized to the case when the vector-valued time series is obtained from a scalar random sequence, which gives rise to a filter bank model for quantization. The asymptotic performance of such a subband coder is derived and shown to coincide with the asymptotic bound for transform coding. Quantization employing a single scalar pre- and postfilter, traditional transform coding using a square linear transformation, and subband coding in filter banks, arise as special cases of the structure analyzed here     

Hybrid Digital-Analog Coding with Bandwidth Compression for Gaussian Source-Channel Pairs

Three hybrid digital-analog (HDA) systems, denoted by HDA-I, HDA* and HDA-II, for the coding of a memoryless discrete-time Gaussian source over a discrete-time additive memoryless Gaussian channel under bandwidth compression are studied. The systems employ simple linear coding in their analog component and superimpose their analog and digital signals before channel transmission. Information-theoretic upper bounds on the asymptotically optimal mean squared error distortion of the systems are obtained under both matched and mismatched channel conditions. Allocation schemes for distributing the channel input power between the analog and the digital signals are also examined. It is shown that systems HDA* and HDA-II can asymptotically achieve the optimal Shannon-limit performance under matched channel conditions. Low-complexity and low-delay versions of systems HDA-I and HDA-II are next designed and implemented without the use of error correcting codes. The parameters of these HDA systems, which employ vector quantization in conjunction with binary phase-shift keying modulation in their digital part, are optimized via an iterative algorithm similar to the design algorithm for channel-optimized vector quantizers. Both systems have low complexity and low delay, and guarantee graceful performance improvements for high CSNRs. For memoryless Gaussian sources the designed HDA-II system is shown to be superior to the HDA-I designed system. When applied to a Gauss-Markov source under Karhunen-Loeve processing, the HDA-I system is shown to provide considerably better performance.     

Tree coding of image subbands

The authors consider the encoding of image subbands with a tree code that is asymptotically optimal for Gaussian sources and the mean squared error (MSE) distortion measure. They first prove that optimal encoding of ideally filtered subbands of a Gaussian image source achieves the rate distortion bound for the MSE distortion measure. The optimal rate and distortion allocation among the subbands is a by-product of this proof. A bound is derived which shows that subband coding is closer than full-band coding to the rate distortion bound for a finite length sequence. The tree codes are then applied to encode the image subbands, both nonadaptively and adaptively. Since the tree codes are stochastic and the search of the code tree is selective, a relatively few reproduction symbols may have an associated squared error a hundred times larger than the target for the subband. Correcting these symbols through a postcoding procedure improves the signal-to-noise ratio and visual quality significantly, with a marginal increase in total rate.     

Variable-rate trellis source encoding

The fixed slope lossy algorithm derived from the kth-order adaptive arithmetic codeword length function is extended to finite-state decoders or trellis-structured decoders. When this algorithm is used to encode a stationary, ergodic source with a continuous alphabet, the Lagrangian performance converges with probability one to a quantity computable as the infimum of an information-theoretic functional over a set of auxiliary random variables and reproduction levels, where λ>0 and -λ are designated to be the slope of the rate distortion function R(D) of the source at some D; the quantity is close to R(D)+λD when the order k used in the arithmetic coding or the number of states in the decoders is large enough, An alternating minimization algorithm for computing the quantity is presented; this algorithm is based on a training sequence and in turn gives rise to a design algorithm for variable-rate trellis source codes. The resulting variable-rate trellis source codes are very efficient in low-rate regions. With k=8, the mean-squared error encoding performance at the rate 1/2 bits/sample for memoryless Gaussian sources is comparable to that afforded by trellis-coded quantizers; with k=8 and the number of states in the decoder=32, the mean-squared error encoding performance at the rate 1/2 bits/sample for memoryless Laplacian sources is about 1 dB better than that afforded by the trellis-coded quantizers with 256 states, with k=8 and the number of states in the decoder=256, the mean-squared error encoding performance at the rates of a fraction of 1 bit/sample for highly dependent Gauss-Markov sources with correlation coefficient 0.9 is within about 0.6 dB of the distortion rate function     

Source and channel rate allocation for channel codes satisfying theGilbert-Varshamov or Tsfasman-Vladut-Zink bounds

We derive bounds for optimal rate allocation between source and channel coding for linear channel codes that meet the Gilbert-Varshamov or Tsfasman-Vladut-Zink (1984) bounds. Formulas giving the high resolution vector quantizer distortion of these systems are also derived. In addition, we give bounds on how far below the channel capacity the transmission rate should be for a given delay constraint. The bounds obtained depend on the relationship between channel code rate and relative minimum distance guaranteed by the Gilbert-Varshamov bound, and do not require sophisticated decoding beyond the error correction limit. We demonstrate that the end-to-end mean-squared error decays exponentially fast as a function of the overall transmission rate, which need not be the case for certain well-known structured codes such as Hamming codes     

Progressive transmission of images over memoryless noisy channels

An embedded source code allows the decoder to reconstruct the source progressively from the prefixes of a single bit stream. It is desirable to design joint source-channel coding schemes which retain the capability of progressive reconstruction in the presence of channel noise or packet loss. Here, we address the problem of joint source-channel coding of images for progressive transmission over memoryless bit error or packet erasure channels. We develop a framework for encoding based on embedded source codes and embedded error correcting and error detecting channel codes. For a target transmission rate, we provide solutions and an algorithm for the design of optimal unequal error/erasure protection. Three performance measures are considered: the average distortion, the average peak signal-to-noise ratio, and the average useful source coding rate. Under the assumption of rate compatibility of the underlying channel codes, we provide necessary conditions for progressive transmission of joint source-channel codes. We also show that the unequal error/erasure protection policies that maximize the average useful source coding rate allow progressive transmission with optimal unequal protection at a number of intermediate rates     

Probability of error for the fixed-length lossy coding of general sources

In this correspondence, we are interested in the error exponent of fixed-length lossy source coding, where the sources are general sources, in the sense of Han and Verduacute, including all nonstationary and/or nonergodic sources. The aim of the correspondence is to establish a unified formula for the minimum (D, r)-achievable rate which is the minimum achievable coding rate under asymptotic constraints of the form epsiv_n(D) ~ e^-nr (n rarr infin), where r is the prescribed error exponent, epsiv_n(D) is the probability of the distortion exceeding a level D, and n is the code-length. For the stationary memoryless source with finite alphabet, Marton (1974) obtained a formula for the reliability function which is expressed in terms of the minimum (D,r)- achievable rate. Recently, Ihara and Kubo proved that the Marton's formula remains true for the stationary memoryless Gaussian source under a mean-squared fidelity criterion. In this correspondence, it is shown that a formula similar to Marton's formula holds for the general sources. The error exponent of correct decoding is also investigated and a formula for the minimum achievable rate of correct decoding in lossy coding is established     

Resource Allocation for Error Resilient Video Coding Over AWGN Using Optimization Approach

The number of slices for error resilient video coding is jointly optimized with 802.11a-like media access control and the physical layers with automatic repeat request and rate compatible punctured convolutional code over additive white gaussian noise channel as well as channel times allocation for time division multiple access. For error resilient video coding, the relation between the number of slices and coding efficiency is analyzed and formulated as a mathematical model. It is applied for the joint optimization problem, and the problem is solved by a convex optimization method such as the primal-dual decomposition method. We compare the performance of a video communication system which uses the optimal number of slices with one that codes a picture as one slice. From numerical examples, end-to-end distortion of utility functions can be significantly reduced with the optimal slices of a picture especially at low signal-to-noise ratio.      

The Effectiveness and Efficiency of Hybrid Transform/DPCM Interframe Image Coding

We seek to evaluate the efficiency of hybrid transform/ DPCM interframe image coding relative to an optimal scheme that minimizes the mean-squared error in encoding a stationary Gaussian image sequence. The stationary assumption leads us to use the asymptotically optimal discrete Fourier transform (DFT) on the full frame of an image. We encode an actual image sequence with full-frame DFT/DPCM at several rates and compare it to previous interframe coding results with the same sequence. We also encode a single frame at these same rates using a full-frame DFT to demonstrate the inherent coding gains of interframe transform DPCM over intraframe coding. We then generate a pseudorandom image sequence with precise Gauss-Markov statistics and encode it by hybrid full-frame DFT/DPCM at various rates. We compare the signal-to-noise ratios (SNR's) of these reconstructions to the optimal ones calculated from the rate-distortion function. We conclude that in a medium rate range below 1 bit/pel/frame where reconstructions for hybrid transform/ DPCM may be unsatisfactory, there is enough margin for improvement to consider more sophisticated coding schemes.     

Transform and Hybrid Transform/DPCM Coding of Images Using Pyramid Vector Quantization

The transform and hybrid transform/DPCM methods of image coding are generalized to allow pyramid vector quantization of the transform coefficients. An asymptotic mean-squared error performance expression is derived for the pyramid vector quantizer and used to determine the optimum rate assignment for encoding the various transform coefficients. Coding simulations for two images at average rates of 0.5-1 bit/pixel demonstrate a 1-3 dB improvement in signal-to-noise ratio for the vector quantization approach in the hybrid coding, with more modest improvements in signal-to-noise ratio in the transform coding. However, this improvement is quite noticeable in image quality, particularly in reducing "blockiness" in the low bit rate encoded images.     

Achievable Performance of Orthogonal STBC Over Spatially Correlated Rician Channels

The effect of spatial correlation on the performance of orthogonal space-time block codes (OSTBCs) over multiple-input-multiple-output (MIMO) Rician fading channels is studied. Asymptotic error-rate formulas for OSTBC with high average signal-to-noise ratios (ASNRs) over arbitrarily correlated Rician MIMO channels are derived in terms of the diversity and coding gains. Our results show that, in correlated fading, the phase vector phi of the channel line-of-sight (LOS) components affects the effective Rice K-factor at the OSTBC receiver output and, hence, may result in a coding gain that is significantly higher than that for independent Rician MIMO channels. Furthermore, when the channel covariance matrix is rank deficient and under some additional mild conditions, the error and outage probabilities of OSTBC achieve those in a nonfading additive-white-Gaussian-noise channel. For both cases of full-rank and rank-deficient channel covariance matrices, analytical expressions of optimal and worst case phase vectors phi, and exact upper and lower bounds of OSTBC performance are derived. These results provide new insights into the achievable performance of OSTBC over correlated Rician MIMO channels and, if incorporated into future multiple antenna systems design, will bring about significant performance enhancement     

Asymptotic Distortion Performance of Source-Channel Diversity over Multihop and Relay Channels

A key challenge in the design of real-time wireless multimedia systems is the presence of fading coupled with strict delay constraints. A very effective answer to this problem is the use of diversity achieving techniques to overcome the fading nature of the wireless channels caused by the mobility of the nodes. The mobility of the nodes gives rise to the need of cooperation among the nodes to enhance the system performance. This paper focuses on comparing systems that exhibit diversity of three forms: source coding diversity, channel coding diversity, and user cooperation diversity implemented through multihop or relay channels with amplify-and-forward or decode-and-forward protocols. Commonly used in multimedia communications, performance is measured in terms of the distortion exponent, which measures the rate of decay of the end-to-end distortion at asymptotically high signal-to-noise ratio (SNR). For the case of repetition coding at the relay nodes, we prove that having more relays is not always beneficial. For the general case of having a large number of relays that can help the source using repetition coding, the optimum number of relay nodes that maximizes the distortion exponent is determined in this paper. This optimum number of relay nodes will depend on the system bandwidth as well as the channel quality. The derived result shows a trade-off between the quality (resolution) of the source encoder and the amount of cooperation (number of relay nodes). Also, the performances of the channel coding diversity-based scheme and the source coding diversity-based scheme are compared. The results show that for both relay and multihop channels, channel coding diversity provides the best performance, followed by the source coding diversity.     

A sequence-based approximate MMSE decoder for source coding overnoisy channels using discrete hidden Markov models

In previous work on source coding over noisy channels it was recognized that when the source has memory, there is typically “residual redundancy” between the discrete symbols produced by the encoder, which can be capitalized upon by the decoder to improve the overall quantizer performance. Sayood and Borkenhagen (1991) and Phamdo and Farvardin (see IEEE Trans. Inform. Theory, vol.40, p.186-93, 1994) proposed “detectors” at the decoder which optimize suitable criteria in order to estimate the sequence of transmitted symbols. Phamdo and Farvardin also proposed an instantaneous approximate minimum mean-squared error (IAMMSE) decoder. These methods provide a performance advantage over conventional systems, but the maximum a posteriori (MAP) structure is suboptimal, while the IAMMSE decoder makes limited use of the redundancy. Alternatively, combining aspects of both approaches, we propose a sequence-based approximate MMSE (SAMMSE) decoder. For a Markovian sequence of encoder-produced symbols and a discrete memoryless channel, we approximate the expected distortion at the decoder under the constraint of fixed decoder complexity. For this simplified cost, the optimal decoder computes expected values based on a discrete hidden Markov model, using the wellknown forward/backward (F/B) algorithm. Performance gains for this scheme are demonstrated over previous techniques in quantizing Gauss-Markov sources over a range of noisy channel conditions. Moreover, a constrained delay version is also suggested     

nd-convolutional codes. II. Structural analysis

For pt.I see ibid., vol.43, no.2, p.558-75 (1997). The structural properties of a noncoherent coded system, which incorporates convolutional codes in conjunction with multiple symbol noncoherent detection, is presented in this second part of a two-part paper, where the performance analysis was provided in Part I. These convolutional codes are referred to as nd-convolutional codes and they provide a general framework for various noncoherent coding systems, including differential systems, for several practical models of the carrier phase. The exponential rate in which the error probability decays to zero, derived in Part I of the paper, is used here to obtain the free equivalent distance of nd-codes, which is the single parameter dominating the error performance at large signal-to-noise ratios. The free equivalent distance is upper-bounded by the free nd-distance, which constitutes a more convenient and practical parameter to work with, and it is the basis for a computer search for optimal nd-codes. The resultant codes of the computer search are compared to codes which are optimal for coherent detection, and it is verified that the latter codes are not necessarily optimal for noncoherent detection since they exhibit in many cases a relatively small nd-distance. The ambiguity problem, inherent to noncoherent systems, is also treated in this paper in the general framework of nd-catastrophic codes, and necessary and sufficient conditions for catastrophic error propagation are identified     

On Joint Source and Channel Coding Using Trellis Coded CPM: Analytical Bounds on the Channel Distortion

Joint source and channel coding (JSCC) using trellis coded quantization (TCQ) in conjunction with trellis coded continuous phase modulation (CPM) is studied. The channel is assumed to be the additive white gaussian noise (AWGN) channel. Analytical bounds on the channel distortion for the investigated systems with maximum-likelihood sequence detection (MLSD) are developed. The bounds are based on the transfer function technique, which was modified and generalized to include continuous-amplitude discrete-time signals. For a memoryless uniform source, the constructed bounds for the investigated systems are shown to be asymptotically tight for increasing channel signal-to-noise ratio (SNR) values. For a memoryless nonuniform source, the constructed bounds are not as tight as the one for the uniform source, however, it still can be used as an indication to how the system performs. It is concluded that the minimum Euclidean distance of the system alone is not enough to evaluate the performance of the considered systems. The number of error events having minimum Euclidean distance and the total distortion caused by those error events also affect the asymptotic performance. This work provides an analysis tool for the investigated systems. The analysis method is very general. It may be applied to any trellis based JSCC schemes.     

Joint and tandem source-channel coding with complexity and delay constraints

Two common source-channel coding strategies, joint and tandem, are compared on the basis of distortion versus complexity and distortion versus delay by analyzing specific representatives of each when transmitting analog data samples across a binary symmetric channel. Channel-optimized transform coding is the joint source-channel strategy; transform coding plus Reed-Solomon coding is the tandem strategy. For each strategy, formulas for the mean-squared error, computational complexity, and delay are found and used to minimize distortion subject to constraints on complexity and delay, for source data modeled as Gauss-Markov. The results of such optimizations suggest there is a complexity threshold such that when the number of operations per data sample available for encoding and decoding is greater than this threshold, tandem coding is better, and when less, channel-optimized transform coding is better. Similarly, the results suggest there is also a delay threshold such that tandem coding is better than joint coding when the permissible encoding and decoding delay is greater than this threshold.     

Multiuser receivers for randomly spread signals: fundamental limitswith and without decision-feedback

Synchronous code-division multiple-access (CDMA) communication systems with randomly chosen spreading sequences and capacity-achieving forward error correction coding are analyzed in terms of spectral efficiency. Emphasis is on the penalties paid by applying single-user coding in conjunction with suboptimal multiuser receivers as opposed to optimal joint decoding which involves complexity that is exponential in the number of users times the code word length. The conventional, the decorrelating, and the (re-encoded) decorrelating decision-feedback detectors are analyzed in the nonasymptotic case for spherical random sequences. The re-encoded minimum mean-squared error (MMSE) decision-feedback receiver achieving the same performance as joint multiuser decoding for equal power users is shown to be suboptimal in the case of equal rates