Detection of binary Markov sources over channels with additiveMarkov noise

We consider maximum a posteriori (MAP) detection of a binary asymmetric Markov source transmitted over a binary Markov channel. The MAP detector observes a long (but finite) sequence of channel outputs and determines the most probable source sequence. In some cases, the MAP detector can be implemented by simple rules such as the “believe what you see” rule or the “guess zero (or one) regardless of what you see” rule. We provide necessary and sufficient conditions under which this is true. When these conditions are satisfied, the exact bit error probability of the sequence MAP detector can be determined. We examine in detail two special cases of the above source: (i) binary independent and identically distributed (i.i.d.) source and (ii) binary symmetric Markov source. In case (i), our simulations show that the performance of the MAP detector improves as the channel noise becomes more correlated. Furthermore, a comparison of the proposed system with a (substantially more complex) traditional tandem source-channel coding scheme portrays superior performance for the proposed scheme at relatively high channel bit error rates. In case (ii), analytical as well as simulation results show the existence of a “mismatch” between the source and the channel (the performance degrades as the channel noise becomes more correlated). This mismatch is reduced by the use of a simple rate-one convolutional encoder     

A sequence-based approximate MMSE decoder for source coding overnoisy channels using discrete hidden Markov models

In previous work on source coding over noisy channels it was recognized that when the source has memory, there is typically “residual redundancy” between the discrete symbols produced by the encoder, which can be capitalized upon by the decoder to improve the overall quantizer performance. Sayood and Borkenhagen (1991) and Phamdo and Farvardin (see IEEE Trans. Inform. Theory, vol.40, p.186-93, 1994) proposed “detectors” at the decoder which optimize suitable criteria in order to estimate the sequence of transmitted symbols. Phamdo and Farvardin also proposed an instantaneous approximate minimum mean-squared error (IAMMSE) decoder. These methods provide a performance advantage over conventional systems, but the maximum a posteriori (MAP) structure is suboptimal, while the IAMMSE decoder makes limited use of the redundancy. Alternatively, combining aspects of both approaches, we propose a sequence-based approximate MMSE (SAMMSE) decoder. For a Markovian sequence of encoder-produced symbols and a discrete memoryless channel, we approximate the expected distortion at the decoder under the constraint of fixed decoder complexity. For this simplified cost, the optimal decoder computes expected values based on a discrete hidden Markov model, using the wellknown forward/backward (F/B) algorithm. Performance gains for this scheme are demonstrated over previous techniques in quantizing Gauss-Markov sources over a range of noisy channel conditions. Moreover, a constrained delay version is also suggested     

An error resilient scheme for image transmission over noisychannels with memory

This article addresses the use of a joint source-channel coding strategy for enhancing the error resilience of images transmitted over a binary channel with additive Markov noise. In this scheme, inherent or residual (after source coding) image redundancy is exploited at the receiver via a maximum a posteriori (MAP) channel detector. This detector, which is optimal in terms of minimizing the probability of error, also exploits the larger capacity of the channel with memory as opposed to the interleaved (memoryless) channel. We first consider MAP channel decoding of uncompressed two-tone and bit-plane encoded grey-level images. Next, we propose a scheme relying on unequal error protection and MAP detection for transmitting grey-level images compressed using the discrete cosine transform (DCT), zonal coding, and quantization. Experimental results demonstrate that for various overall (source and channel) operational rates, significant performance improvements can be achieved over interleaved systems that do not incorporate image redundancy.     

Efficient source decoding over memoryless noisy channels using higher order Markov models

Exploiting the residual redundancy in a source coder output stream during the decoding process has been proven to be a bandwidth-efficient way to combat noisy channel degradations. This redundancy can be employed to either assist the channel decoder for improved performance or design better source decoders. In this work, a family of solutions for the asymptotically optimum minimum mean-squared error (MMSE) reconstruction of a source over memoryless noisy channels is presented when the redundancy in the source encoder output stream is exploited in the form of a /spl gamma/-order Markov model (/spl gamma//spl ges/1) and a delay of /spl delta/,/spl delta/>0, is allowed in the decoding process. It is demonstrated that the proposed solutions provide a wealth of tradeoffs between computational complexity and the memory requirements. A simplified MMSE decoder which is optimized to minimize the computational complexity is also presented. Considering the same problem setup, several other maximum a posteriori probability (MAP) symbol and sequence decoders are presented as well. Numerical results are presented which demonstrate the efficiency of the proposed algorithms.     

Optimal causal coding - decoding problems

The symbols produced by a finite Markov source are causally encoded so as to be transmitted through a noisy memoryless channel. The encoder is assumed to have channel feedback information and the decoder to be causal. The feedback information is shown to be useful in general. Separation results are derived and used to prove that encoding is useless for a class of symmetric channels.     

Markov model aided decoding for image transmission usingsoft-decision-feedback

Soft-decision-feedback MAP decoders are developed for joint source/channel decoding (JSCD) which uses the residual redundancy in two-dimensional sources. The source redundancy is described by a second order Markov model which is made available to the receiver for row-by-row decoding, wherein the output for one row is used to aid the decoding of the next row. Performance can be improved by generalizing so as to increase the vertical depth of the decoder. This is called sheet decoding, and entails generalizing trellis decoding of one-dimensional data to trellis decoding of two-dimensional data (2-D). The proposed soft-decision-feedback sheet decoder is based on the Bahl algorithm, and it is compared to a hard-decision-feedback sheet decoder which is based on the Viterbi algorithm. The method is applied to 3-bit DPCM picture transmission over a binary symmetric channel, and it is found that the soft-decision-feedback decoder with vertical depth V performs approximately as well as the hard-decision-feedback decoder with vertical depth V+1. Because the computational requirement of the decoders depends exponentially on the vertical depth, the soft-decision-feedbark decoder offers significant reduction in complexity. For standard monochrome Lena, at a channel bit error rate of 0.05, the V=1 and V=2 soft-decision-feedback decoder JSCD gains in RSNR are 5.0 and 6.3 dB, respectively.     

Uncoded optimal binary communication for sources with memory using the Burrows Wheeler Transform

Given an AWGN channel, we look at the problem of designing an optimal binary uncoded communication system for transmitting blocks of binary symbols generated by a stationary source with memory modelled by a Markov chain (MC) or a hidden Markov model (HMM). The goal is to minimize the average SNR required for a given block error rate. The particular case where the binary source is memoryless with nonuniform symbol probabilities has been studied by Korn et al. [Optimal binary communication with nonequal probabilities. IEEE Trans Commun 2003;51:1435–8] [1] by optimally allocating the energies of the transmitted signals. In this paper we generalize the previous work to include the important case of sources with memory. The proposed system integrates the block sorting Burrows Wheeler Transform (BWT, [Burrows M, Wheeler D. A block sorting lossless data compression algorithm. Research report 124. Digital Systems Center, 1994]) [2] with an optimal energy allocation scheme based on the first order probabilities of the transformed symbols. Analytical expressions are derived for the energy gain obtained with the proposed system when compared either with the optimal blockwise MAP receiver or with a standard source coded system consisting of an optimal source encoder followed by an optimal uncoded binary communication system, i.e. by a symbol-by-symbol MAP detector.     

Joint source-channel decoding of variable-length codes for convolutional codes and turbo codes

Several recent publications have shown that joint source-channel decoding could be a powerful technique to take advantage of residual source redundancy for fixed- and variable-length source codes. This letter gives an in-depth analysis of a low-complexity method recently proposed by Guivarch et al., where the redundancy left by a Huffman encoder is used at a bit level in the channel decoder to improve its performance. Several simulation results are presented, showing for two first-order Markov sources of different sizes that using a priori knowledge of the source statistics yields a significant improvement, either with a Viterbi channel decoder or with a turbo decoder.     

Sequence MAP decoding of trellis codes for Gaussian and Rayleighchannels

This paper considers the use of sequence maximum a posteriori (MAP) decoding of trellis codes. A MAP receiver can exploit any “residual redundancy” that may exist in the channel encoded signal in the form of memory and/or a nonuniform distribution, thereby providing enhanced performance over very noisy channels, relative to maximum likelihood (ML) decoding. The paper begins with a first-order two-state Markov model for the channel encoder input. A variety of different systems with different source parameters, different modulation schemes, and different encoder complexities are simulated. Sequence MAP decoding is shown to substantially improve performance under very noisy channel conditions for systems with low-to-moderate redundancy, with relative gain increasing as the rate increases. As a result, coding schemes with multidimensional constellations are shown to have higher MAP gains than comparable schemes with two-dimensional (2-D) constellations. The second part of the paper considers trellis encoding of the code-excited linear predictive (CELP) speech coder's line spectral parameters (LSPs) with four-dimensional (4-D) QPSK modulation. Two source LSP models are used. One assumes only intraframe correlation of LSPs while the second one models both intraframe and interframe correlation. MAP decoding gains (over ML decoding) as much as 4 dB are achieved. Also, a comparison between the conventionally designed codes and an I-Q QPSK scheme shows that the I-Q scheme achieves better performance even though the first (sampler) LSP model is used     

“Turbo DPSK”: iterative differential PSK demodulationand channel decoding

Multiple symbol differential detection is known to fill the gap between conventional differential detection of MPSK (M-DPSK) and coherent detection of M-PSK with differential encoding (M-DEPSK). Emphasis has been so far on soft-input/hard-output detectors applied in uncoded systems. In this paper, we investigate a receiver structure suitable for coded DPSK signals on static and time-varying channels. The kernel is an a posteriori probability (APP) DPSK demodulator. This demodulator accepts a priori information and produces reliability outputs. Due to the availability of reliability outputs, an outer soft-decision channel decoder can be applied. Due to the acceptance of a priori information, if the outer channel decoder also outputs reliability information, iterative (“turbo”) processing can be done. The proposed “APP DPSK demodulator” uses linear prediction and per-survivor processing to estimate the channel response. The overall transmission scheme represents a type of serial “turbo code,” with a differential encoder concatenated with a convolutional code, separated by interleaving. The investigated system has the potential to improve the performance of coherent PSK without differential encoding and perfect channel estimation for fading cases! Only a small number of iterations are required. The receiver under investigation can be applied to several existing standards without changing the transmission format. Results are presented for uncoded and convolutionally coded 4-DPSK modulation transmitted over the Gaussian channel and the Rayleigh flat-fading channel, respectively     

Analysis and design of trellis codes optimized for a binarysymmetric Markov source with MAP detection

We consider the problem of transmitting a binary symmetric Markov source (BSMS), over the additive white Gaussian noise (AWGN) channel. The coding technique considered is trellis-coded modulation (TCM), where we utilize decoders which implement the maximum-likelihood (ML) and maximum a posteriori (MAP) criteria. Employing 8-PSK Ungerboeck codes on a BSMS with state transition probability 0.1, we first show that the MAP decoder realizes a 0.8-2.1-dB coding gain over the ML decoder. Motivated by these gains, we consider the design of trellis codes optimized for the BSMS/AWGN/MAP system. An approximate union bound is established for this system. Using this bound, we found codes which exhibit additional 0.4-1.1-dB gains over Ungerboeck codes. Finally, we compare the proposed TCM system with a tandem coding system. At normalized signal-to-noise ratio (SNR) of 10.8 dB and below, the proposed system significantly outperforms the tandem system     

Progressive transmission of images using MAP detection overchannels with memory

We propose a new maximum a posteriori (MAP) detector, without the need for explicit channel coding, to lessen the impact of communication channel errors on compressed image sources. The MAP detector exploits the spatial correlation in the compressed bitstream as well as the temporal memory in the channel to correct channel errors. We first present a technique for computing the residual redundancy inherent in a compressed grayscale image (compressed using VQ). The performance of the proposed MAP detector is compared to that of a memoryless MAP detector. We also investigate the dependence of the performance on memory characteristics of the Gilbert-Elliott channel as well as average channel error rate. Finally, we study the robustness of the proposed MAP detector's performance to estimation errors.     

Autocorrelation properties of channel encoded sequences-applicability to blind equalization

Many blind channel equalization/identification algorithms are derived assuming the transmitted information sequence to be white. In practical communication systems, redundancy is added to the source sequence in order to detect and correct symbol errors in the receiver. It is not obvious how channel encoding will affect the assumption of whiteness. The autocorrelation function of some commonly used channel codes is analyzed in order to study the validity of assumptions used in blind equalization. The codes are presented in terms of a Markov model for which the autocorrelation is analytically expressed. The various encoded sequences are used in a prediction error based blind equalizer, and the performance is empirically compared with the case of unencoded data. A blind equalization example using a practical GSM speech encoder combined with a convolutional channel encoder is also given.     

Grammar-based lossless universal refinement source coding

A sequence y=(y/sub 1/,...,y/sub n/) is said to be a coarsening of a given finite-alphabet source sequence x=(x/sub 1/,...,x/sub n/) if, for some function /spl phi/, y/sub i/=/spl phi/(x/sub i/) (i=1,...,n). In lossless refinement source coding, it is assumed that the decoder already possesses a coarsening y of a given source sequence x. It is the job of the lossless refinement source encoder to furnish the decoder with a binary codeword B(x|y) which the decoder can employ in combination with y to obtain x. We present a natural grammar-based approach for finding the binary codeword B(x|y) in two steps. In the first step of the grammar-based approach, the encoder furnishes the decoder with O(/spl radic/nlog/sub 2/n) code bits at the beginning of B(x|y) which tell the decoder how to build a context-free grammar G/sub y/ which represents y. The encoder possesses a context-free grammar G/sub x/ which represents x; in the second step of the grammar-based approach, the encoder furnishes the decoder with code bits in the rest of B(x|y) which tell the decoder how to build G/sub x/ from G/sub y/. We prove that our grammar-based lossless refinement source coding scheme is universal in the sense that its maximal redundancy per sample is O(1/log/sub 2/n) for n source samples, with respect to any finite-state lossless refinement source coding scheme. As a by-product, we provide a useful notion of the conditional entropy H(G/sub x/|G/sub y/) of the grammar G/sub x/ given the grammar G/sub y/, which is approximately equal to the length of the codeword B(x|y).     

Robust transmission of MELP-compressed speech: an illustrative example of joint source-channel decoding

A mixed-excitation linear predictive (MELP) speech coder was selected as the US federal standard for 2400 b/s speech compression. This paper examines the quality of MELP-compressed speech when transmitted over noisy communication channels in conjunction with a variety of error-control schemes. The focus is on channel decoders that exploit the "residual redundancy" inherent in the MELP bitstream. This residual redundancy, which is manifested by the correlation in time and the nonuniform distribution of various MELP parameters, can be quantified by modeling the parameters as one-step Markov chains and computing the entropy rate of the Markov chains based on the relative frequencies of transitions. Moreover, this residual redundancy can be exploited by an appropriately "tuned" channel decoder to provide substantial coding gain when compared with decoders that do not exploit it. Channel coding schemes include conventional binary convolutional codes and iteratively-decoded parallel concatenated convolutional (turbo) codes.     

Quantizers with uniform decoders and channel-optimized encoders

Scalar quantizers with uniform decoders and channel-optimized encoders are studied for a uniform source on [0,1] and binary symmetric channels. Two families of affine index assignments are considered: the complemented natural code (CNC), introduced here, and the natural binary code (NBC). It is shown that the NBC never induces empty cells in the quantizer encoder, whereas the CNC can. Nevertheless, we show that the asymptotic distributions of quantizer encoder cells for the NBC and the CNC are equal and are uniform over a proper subset of the source's support region. Empty cells act as a form of implicit channel coding. An effective channel code rate associated with a quantizer designed for a noisy channel is defined and computed for the codes studied. By explicitly showing that the mean-squared error (MSE) of the CNC can be strictly smaller than that of the NBC, we also demonstrate that the NBC is suboptimal for a large range of transmission rates and bit error probabilities. This contrasts with the known optimality of the NBC when either both the encoder and decoder are not channel optimized, or when only the decoder is channel optimized.     

Reconstruction of predictively encoded signals over noisy channels using a sequence MMSE decoder

In this paper, we consider the problem of decoding predictively encoded signal over a noisy channel when there is residual redundancy (captured by a /spl gamma/-order Markov model) in the sequence of transmitted data. Our objective is to minimize the mean-squared error (MSE) in the reconstruction of the original signal (input to the predictive source coder). The problem is formulated and solved through minimum mean-squared error (MMSE) decoding of a sequence of samples over a memoryless noisy channel. The related previous works include several maximum a posteriori (MAP) and MMSE-based decoders. The MAP-based approaches are suboptimal when the performance criterion is the MSE. On the other hand, the previously known MMSE-based approaches are suboptimal, since they are designed to efficiently reconstruct the data samples received (the prediction residues) rather than the original signal. The proposed scheme is set up by modeling the source-coder-produced symbols and their redundancy with a trellis structure. Methods are presented to optimize the solutions in terms of complexity. Numerical results and comparisons are provided, which demonstrate the effectiveness of the proposed techniques.     

Low-complexity maximum-likelihood detection of coded signals sentover finite-state Markov channels

We propose a decision-feedback decoder for coded signals transmitted over finite-state Markov channels. The decoder achieves maximum-likelihood sequence detection (in the absence of feedback errors) with very low complexity by exploiting previous bit decisions and the Markov structure of the channel. We also propose a similar decoder, the output-feedback decoder, that does not use previous bit decisions and therefore does not suffer from error propagation. The decoder performance is determined using a new sliding window analysis technique as well as by simulation. Both decoders exhibit excellent bit error rate performance with a relatively low complexity that is independent of the channel decorrelation time     

Optimal quantizer design for noisy channels: An approach to combined source - channel coding

We present an analysis of the zero-memory quantization of memoryless sources when the quantizer output is to be encoded and transmitted across a noisy channel. Necessary conditions for the joint optimization of the quantizer and the encoder/decoder pair are presented, and an iterative algorithm for obtaining a locally optimum system is developed. The performance of this locally optimal system, obtained for the class of generalized Gaussian distributions and the binary symmetric channel, is compared against the optimum performance theoretically attainable (using rate-distortion theoretic arguments), as well as against the performance of Lloyd-Max quantizers encoded using the natural binary code and the folded binary code. It is shown that this optimal design could result in substantial performance improvements. The performance improvements are more noticeable at high bit rates and for broad-tailed densities.