Variations on the Gallager bounds, connections, and applications

There has been renewed interest in deriving tight bounds on the error performance of specific codes and ensembles, based on their distance spectrum. We discuss many reported upper bounds on the maximum-likelihood (ML) decoding error probability and demonstrate the underlying connections that exist between them. In addressing the Gallager bounds and their variations, we focus on the Duman and Salehi (see IEEE Trans. Commun., vol.46, p.717-723, 1998)variation, which originates from the standard Gallager bound. A large class of efficient bounds (or their Chernoff versions) is demonstrated to be a special case of the generalized second version of the Duman and Salehi bounds. Implications and applications of these observations are pointed out, including the fully interleaved fading channel, resorting to either matched or mismatched decoding. The proposed approach can be generalized to geometrically uniform nonbinary codes, finite-state channels, bit interleaved coded modulation systems, and it can be also used for the derivation of upper bounds on the conditional decoding error probability.     

Improved upper bounds on the ML decoding error probability ofparallel and serial concatenated turbo codes via their ensemble distancespectrum

The ensemble performance of parallel and serial concatenated turbo codes is considered, where the ensemble is generated by a uniform choice of the interleaver and of the component codes taken from the set of time-varying recursive systematic convolutional codes. Following the derivation of the input-output weight enumeration functions of the ensembles of random parallel and serial concatenated turbo codes, the tangential sphere upper bound is employed to provide improved upper bounds on the block and bit error probabilities of these ensembles of codes for the binary-input additive white Gaussian noise (AWGN) channel, based on coherent detection of equi-energy antipodal signals and maximum-likelihood decoding. The influence of the interleaver length and the memory length of the component codes is investigated. The improved bounding technique proposed here is compared to the conventional union bound and to a alternative bounding technique by Duman and Salehi (1998) which incorporates modified Gallager bounds. The advantage of the derived bounds is demonstrated for a variety of parallel and serial concatenated coding schemes with either fixed or random recursive systematic convolutional component codes, and it is especially pronounced in the region exceeding the cutoff rate, where the performance of turbo codes is most appealing. These upper bounds are also compared to simulation results of the iterative decoding algorithm     

Coding for Parallel Channels: Gallager Bounds and Applications to Turbo-Like Codes

The transmission of coded communication systems is widely modeled to take place over a set of parallel channels. This model is used for transmission over block-fading channels, rate-compatible puncturing of turbo-like codes, multicarrier signaling, multilevel coding, etc. New upper bounds on the maximum-likelihood (ML) decoding error probability are derived in the parallel-channel setting. We focus on the generalization of the Gallager-type bounds and discuss the connections between some versions of these bounds. The tightness of these bounds for parallel channels is exemplified for structured ensembles of turbo codes, repeat-accumulate (RA) codes, and some of their recent variations (e.g., punctured accumulate-repeat-accumulate codes). The bounds on the decoding error probability of an ML decoder are compared to computer simulations of iterative decoding. The new bounds show a remarkable improvement over the union bound and some other previously reported bounds for independent parallel channels. This improvement is exemplified for relatively short block lengths, and it is pronounced when the block length is increased. In the asymptotic case, where we let the block length tend to infinity, inner bounds on the attainable channel regions of modern coding techniques under ML decoding are obtained, based solely on the asymptotic growth rates of the average distance spectra of these code ensembles.     

Tight exponential upper bounds on the ML decoding error probability of block codes over fully interleaved fading channels

We derive tight exponential upper bounds on the decoding error probability of block codes which are operating over fully interleaved Rician fading channels, coherently detected and maximum-likelihood decoded. It is assumed that the fading samples are statistically independent and that perfect estimates of these samples are provided to the decoder. These upper bounds on the bit and block error probabilities are based on certain variations of the Gallager bounds. These bounds do not require integration in their final version and they are reasonably tight in a certain portion of the rate region exceeding the cutoff rate of the channel. By inserting interconnections between these bounds, we show that they are generalized versions of some reported bounds for the binary-input additive white Gaussian noise channel.     

Coded modulation in the block-fading channel: coding theorems and code construction

We consider coded modulation schemes for the block-fading channel. In the setting where a codeword spans a finite number N of fading degrees of freedom, we show that coded modulations of rate R bit per complex dimension, over a finite signal set /spl chi//spl sube//spl Copf/ of size 2/sup M/, achieve the optimal rate-diversity tradeoff given by the Singleton bound /spl delta/(N,M,R)=1+/spl lfloor/N(1-R/M)/spl rfloor/, for R/spl isin/(0,M/spl rfloor/. Furthermore, we show also that the popular bit-interleaved coded modulation achieves the same optimal rate-diversity tradeoff. We present a novel coded modulation construction based on blockwise concatenation that systematically yields Singleton-bound achieving turbo-like codes defined over an arbitrary signal set /spl chi//spl sub//spl Copf/. The proposed blockwise concatenation significantly outperforms conventional serial and parallel turbo codes in the block-fading channel. We analyze the ensemble average performance under maximum-likelihood (ML) decoding of the proposed codes by means of upper bounds and tight approximations. We show that, differently from the additive white Gaussian noise (AWGN) and fully interleaved fading cases, belief-propagation iterative decoding performs very close to ML on the block-fading channel for any signal-to-noise ratio (SNR) and even for relatively short block lengths. We also show that, at constant decoding complexity per information bit, the proposed codes perform close to the information outage probability for any block length, while standard block codes (e.g., obtained by trellis termination of convolutional codes) have a gap from outage that increases with the block length: this is a different and more subtle manifestation of the so-called "interleaving gain" of turbo codes.     

Performance analysis of convolutionally coded systems over quasi-static fading channels

This paper presents an improved upper bound on the performance of convolutionally coded systems over quasi-static fading channels (QSFC). The bound uses a combination of a classical union bound when the fading channel is in a high signal-to-noise ratio (SNR) state together with a new upper bound for the low SNR state. This new bounding approach is applied to both BPSK convolutional and turbo codes, as well as serially concatenated BPSK convolutional/turbo and space-time block codes. The new analytical technique produces bounds which are usually about 1 dB tighter than existing bounds. Finally, based on the proposed bound, we introduce an improved design criterion for convolutionally coded systems in slow flat fading channels. Simulation results are included to confirm the improved ability of the proposed criterion to search for convolutional codes with good performance over a QSFC.     

New Gallager Bounds in Block-Fading Channels

In this paper, we propose a new upper bound on the error performance of binary linear codes over block-fading channels by employing Gallager's first- and second-bounding techniques. As the proposed bound is numerically intensive in its general form, we consider two special cases, namely, the spherical bound and the DS2-exponential bound, which are found to be tight in nonergodic and near-ergodic block-fading channels, respectively. The tightness of the proposed bounds is demonstrated for turbo codes. Many existing bounds for quasistatic or fully interleaved fading channels can be viewed as special cases of the proposed Gallager bound     

Performance bounds for coded free-space optical communications through atmospheric turbulence channels

Error-control codes can help to mitigate atmospheric turbulence-induced signal fading in free-space optical communication links using intensity modulation/direct detection (IM/DD). Error performance bound analysis can yield simple analytical upper bounds or approximations to the bit-error probability. We first derive an upper bound on the pairwise codeword-error probability for transmission through channels with correlated turbulence-induced fading, which involves complicated multidimensional integration. To simplify the computations, we derive an approximate upper bound under the assumption of weak turbulence. The accuracy of this approximation under weak turbulence is verified by numerical simulation. Its invalidity when applied to strong turbulence is also shown. This simple approximate upper bound to the pairwise codeword-error probability is then applied to derive an upper bound to the bit-error probability for block codes, convolutional codes, and turbo codes for free-space optical communication through weak atmospheric turbulence channels. We also discuss the choice of interleaver length in block codes and turbo codes based on numerical evaluation of our performance bounds.     

BEAST decoding of block codes obtained via convolutional codes

BEAST is a bidirectional efficient algorithm for searching trees. In this correspondence, BEAST is extended to maximum-likelihood (ML) decoding of block codes obtained via convolutional codes. First it is shown by simulations that the decoding complexity of BEAST is significantly less than that of the Viterbi algorithm. Then asymptotic upper bounds on the BEAST decoding complexity for three important ensembles of codes are derived. They verify BEAST's high efficiency compared to other algorithms. For high rates, the new asymptotic bound for the best ensemble is in fact better than previously known bounds.     

Tight error bounds for space-time orthogonal block codes under slow Rayleigh flat fading

The performance of space-time orthogonal block (STOB) codes over slow Rayleigh fading channels and maximum-likelihood (ML) decoding is investigated. Two Bonferroni-type bounds (one upper bound and one lower bound) for the symbol error rate (SER) and bit error rate (BER) of the system are obtained. The bounds are expressed in terms of the pairwise error probabilities (PEPs) and the two-dimensional pairwise error probabilities (2-D PEPs) of the transmitted symbols. Furthermore, the bounds can be efficiently evaluated and they hold for arbitrary (nonstandard) signaling schemes and mappings. Numerical results demonstrate that the bounds are very accurate in estimating the performance of STOB codes. In particular, the upper and lower bounds often coincide even at low channel signal-to-noise ratios, large constellation sizes, and large diversity orders.     

On Achievable Rates and Complexity of LDPC Codes Over Parallel Channels: Bounds and Applications

A variety of communication scenarios can be modeled by a set of parallel channels. Upper bounds on the achievable rates under maximum-likelihood (ML) decoding, and lower bounds on the decoding complexity per iteration of ensembles of low-density parity-check (LDPC) codes are presented. The communication of these codes is assumed to take place over statistically independent parallel channels where the component channels are memoryless, binary-input, and output-symmetric. The bounds are applied to ensembles of punctured LDPC codes where the puncturing patterns are either random or possess some structure. Our discussion is concluded by a diagram showing interconnections between the new theorems and some previously reported results     

Closed form expression and improved bound on pairwise error probability for performance analysis of turbo codes over Rician fading channels

This letter provides derivations for an exact expression and a bound on pair wise error probability over fully interleaved Rician fading channels under the assumption of ideal channel state information. The derivation which is based on the probability distribution of the sum of squared Rician random variables leads to an improved upper bound in comparison with the only known bound in literature. Pairwise error probability plots together with average union upper bounds for turbo codes having (1,7/5,7/5) and (1,5/7,5/7) generator polynomials are presented to demonstrate the effectiveness of the new results.     

Tightened Upper Bounds on the ML Decoding Error Probability of Binary Linear Block Codes

The performance of maximum-likelihood (ML) decoded binary linear block codes is addressed via the derivation of tightened upper bounds on their decoding error probability. The upper bounds on the block and bit error probabilities are valid for any memoryless, binary-input and output-symmetric communication channel, and their effectiveness is exemplified for various ensembles of turbo-like codes over the additive white Gaussian noise (AWGN) channel. An expurgation of the distance spectrum of binary linear block codes further tightens the resulting upper bounds     

Parity-check density versus performance of binary linear block codes over memoryless symmetric channels

We derive lower bounds on the density of parity-check matrices of binary linear codes which are used over memoryless binary-input output-symmetric (MBIOS) channels. The bounds are expressed in terms of the gap between the rate of these codes for which reliable communications is achievable and the channel capacity; they are valid for every sequence of binary linear block codes if there exists a decoding algorithm under which the average bit-error probability vanishes. For every MBIOS channel, we construct a sequence of ensembles of regular low-density parity-check (LDPC) codes, so that an upper bound on the asymptotic density of their parity-check matrices scales similarly to the lower bound. The tightness of the lower bound is demonstrated for the binary erasure channel by analyzing a sequence of ensembles of right-regular LDPC codes which was introduced by Shokrollahi, and which is known to achieve the capacity of this channel. Under iterative message-passing decoding, we show that this sequence of ensembles is asymptotically optimal (in a sense to be defined in this paper), strengthening a result of Shokrollahi. Finally, we derive lower bounds on the bit-error probability and on the gap to capacity for binary linear block codes which are represented by bipartite graphs, and study their performance limitations over MBIOS channels. The latter bounds provide a quantitative measure for the number of cycles of bipartite graphs which represent good error-correction codes.     

Performance Bounds for Nonbinary Linear Block Codes Over Memoryless Symmetric Channels

The performance of nonbinary linear block codes is studied in this paper via the derivation of new upper bounds on the block error probability under maximum-likelihood (ML) decoding. The transmission of these codes is assumed to take place over a memoryless and symmetric channel. The new bounds, which are based on the Gallager bounds and their variations, are applied to the Gallager ensembles of nonbinary and regular low-density parity-check (LDPC) codes. These upper bounds are also compared with sphere-packing lower bounds. This study indicates that the new upper bounds are useful for the performance evaluation of coded communication systems which incorporate nonbinary coding techniques.      

Reliable channel regions for good binary codes transmitted over parallel channels

We study the average error probability performance of binary linear code ensembles when each codeword is divided into J subcodewords with each being transmitted over one of J parallel channels. This model is widely accepted for a number of important practical channels and signaling schemes including block-fading channels, incremental redundancy retransmission schemes, and multicarrier communication techniques for frequency-selective channels. Our focus is on ensembles of good codes whose performance in a single channel model is characterized by a threshold behavior, e.g., turbo and low-density parity-check (LDPC) codes. For a given good code ensemble, we investigate reliable channel regions which ensure reliable communications over parallel channels under maximum-likelihood (ML) decoding. To construct reliable regions, we study a modifed 1961 Gallager bound for parallel channels. By allowing codeword bits to be randomly assigned to each component channel, the average parallel-channel Gallager bound is simplified to be a function of code weight enumerators and channel assignment rates. Special cases of this bound, average union-Bhattacharyya (UB), Shulman-Feder (SF), simplified-sphere (SS), and modified Shulman-Feder (MSF) parallel-channel bounds, allow for describing reliable channel regions using simple functions of channel and code spectrum parameters. Parameters describing the channel are the average parallel-channel Bhattacharyya noise parameter, the average channel mutual information, and parallel Gaussian channel signal-to-noise ratios (SNRs). Code parameters include the union-Bhattacharyya noise threshold and the weight spectrum distance to the random binary code ensemble. Reliable channel regions of repeat-accumulate (RA) codes for parallel binary erasure channels (BECs) and of turbo codes for parallel additive white Gaussian noise (AWGN) channels are numerically computed and compared with simulation results based on iterative decoding. In addition, an examp     

Improvement of Gallager Upper Bound and its Variations for Discrete Channels

A new tight upper bound on the maximum-likelihood (ML) word and bit-error decoding probabilities for specific codes over discrete channels is presented. It constitutes an enhanced version of the Gallager upper bound and its variations resulting from the Duman–Salehi second bounding technique. An efficient technique is developed that, in the case of symmetric channels, overcomes the difficulties associated with the direct computation of the proposed bound. Surprisingly, apart from the distance and input–output weight enumerating functions (IOWEFs), the bound depends also on the coset weight distribution of the code.      

On the performance of rate 1/2 convolutional codes with QPSK onRician fading channels

Consideration is given to the bit error probability performance of rate 1/2 convolutional codes in conjunction with quaternary phase shift keying (QPSK) modulation and maximum-likelihood Viterbi decoding on fully interleaved Rician fading channels. Applying the generating function union bounding approach, an asymptotically tight analytic upper bound on the bit error probability performance is developed under the assumption of using the Viterbi decoder with perfect fading amplitude measurement. Bit error probability performance of constraint length K=3-7 codes with QPSK is numerically evaluated using the developed bound. Tightness of the bound is examined by means of computer simulation. The influence of perfect amplitude measurement on the performance of the Viterbi decoder is observed. A performance comparison with rate 1/2 codes with binary phase shift keying (BPSK) is provided     

List decoding of turbo codes

List decoding of turbo codes is analyzed under the assumption of a maximum-likelihood (ML) list decoder. It is shown that large asymptotic gains can be achieved on both the additive white Gaussian noise (AWGN) and fully interleaved flat Rayleigh-fading channels. It is also shown that the relative asymptotic gains for turbo codes are larger than those for convolutional codes. Finally, a practical list decoding algorithm based on the list output Viterbi algorithm (LOVA) is proposed as an approximation to the ML list decoder. Simulation results show that the proposed algorithm provides significant gains corroborating the analytical results. The asymptotic gain manifests itself as a reduction in the bit-error rate (BER) and frame-error rate (FER) floor of turbo codes     

Capacity-achieving codes for finite-state channels with maximum-likelihood decoding

Codes on sparse graphs have been shown to achieve remarkable performance in point-to-point channels with low decoding complexity. Most of the results in this area are based on experimental evidence and/or approximate analysis. The question of whether codes on sparse graphs can achieve the capacity of noisy channels with iterative decoding is still open, and has only been conclusively and positively answered for the binary erasure channel. On the other hand, codes on sparse graphs have been proven to achieve the capacity of memoryless, binary-input, output-symmetric channels with finite graphical complexity per information bit when maximum likelihood (ML) decoding is performed. In this paper, we consider transmission over finite-state channels (FSCs). We derive upper bounds on the average error probability of code ensembles with ML decoding. Based on these bounds we show that codes on sparse graphs can achieve the symmetric information rate (SIR) of FSCs, which is the maximum achievable rate with independently and uniformly distributed input sequences. In order to achieve rates beyond the SIR, we consider a simple quantization scheme that when applied to ensembles of codes on sparse graphs induces a Markov distribution on the transmitted sequence. By deriving average error probability bounds for these quantized code ensembles, we prove that they can achieve the information rates corresponding to the induced Markov distribution, and thus approach the FSC capacity.