On the Probability of Undetected Error for Linear Block Codes

The problem of computing the probability of undetected error is considered for linear block codes used for error detection. The recent literature is first reviewed and several results are extended. It is pointed out that an exact calculation can be based on either the weight distribution of a code or its dual. Using the dual code formulation, the probability of undetected error for the ensemble of all nonbinary linear block codes is derived as well as a theorem that shows why the probability of undetected error may not be a monotonic function of channel error rate for some poor codes. Several bounds on the undetected error probability are then presented. We conclude with detailed examples of binary and nonbinary codes for which exact results can be obtained. An efficient technique for measuring an unknown weight distribution is suggested and exact results are compared with experimental results.     

A lower bound on the undetected error probability and strictlyoptimal codes

Error detection is a simple technique used in various communication and memory systems to enhance reliability. We study the probability that a q-ary (linear or nonlinear) block code of length n and size M fails to detect an error. A lower bound on this undetected error probability is derived in terms of q, n, and M. The new bound improves upon other bounds mentioned in the literature, even those that hold only for linear codes. Block codes whose undetected error probability equals the new lower bound are investigated. We call these codes strictly optimal codes and give a combinatorial characterization of them. We also present necessary and sufficient conditions for their existence. In particular, we find all values of n and M for which strictly optimal binary codes exist, and determine the structure of all of them. For example, we construct strictly optimal binary-coded decimal codes of length four and five, and we show that these are the only possible lengths of such codes     

Coding theorems for the nonsynchronized channel

Capacity and error bounds are derived for a memoryless binary symmetric channel with the receiver having no a priori information as to the starting time of the code words. The channel capacity is the same as the capacity of the synchronized channel. For all rates below capacity, the minimum probability of error for the nonsynchronized channel decreases exponentially with the code-block length. For rates near channel capacity, the exponent in the upper bound on the probability of error for the nonsynchronized channel is the same as the corresponding exponent for the synchronized channel. For low rates, the largest exponent obtained for the nonsynchronized channel with conventional block coding is inferior to the exponent obtained for the synchronized channel. Stronger results are obtained for a new form of coding that allows for a Markov dependency between successive code words. Bounds on the minimum probability of error are obtained for unconstrained binary codes and for several classes of parity-check codes and are used to obtain asymptotic distance properties for various classes of binary codes. At certain rates there exist codes whose minimum distance, in the comma-free sense, is not only greater than one, but is proportional to the block length.     

A comparison of known codes, random codes, and the best codes

This paper calculates new bounds on the size of the performance gap between random codes and the best possible codes. The first result shows that, for large block sizes, the ratio of the error probability of a random code to the sphere-packing lower bound on the error probability of every code on the binary symmetric channel (BSC) is small for a wide range of useful crossover probabilities. Thus even far from capacity, random codes have nearly the same error performance as the best possible long codes. The paper also demonstrates that a small reduction k-k˜ in the number of information bits conveyed by a codeword will make the error performance of an (n,k˜) random code better than the sphere-packing lower bound for an (n,k) code as long as the channel crossover probability is somewhat greater than a critical probability. For example, the sphere-packing lower bound for a long (n,k), rate 1/2, code will exceed the error probability of an (n,k˜) random code if k-k˜>10 and the crossover probability is between 0.035 and 0.11=H^-1(1/2). Analogous results are presented for the binary erasure channel (BEC) and the additive white Gaussian noise (AWGN) channel. The paper also presents substantial numerical evaluation of the performance of random codes and existing standard lower bounds for the BEC, BSC, and the AWGN channel. These last results provide a useful standard against which to measure many popular codes including turbo codes, e.g., there exist turbo codes that perform within 0.6 dB of the bounds over a wide range of block lengths     

Some randomized code constructions from group actions

We study in this paper randomized constructions of binary linear codes that are invariant under the action of some group on the bits of the codewords. We study a non-Abelian randomized construction corresponding to the action of the dihedral group on a single copy of itself as well as a randomized Abelian construction based on the action of an Abelian group on a number of disjoint copies of itself. Cyclic codes have been extensively studied over the last 40 years. However, it is still an open question as to whether there exist asymptotically good binary cyclic codes. We argue that by using a slightly more complex group than a cyclic group, namely, the dihedral group, the existence of asymptotically good codes that are invariant under the action of the group on itself can be guaranteed. In particular, we show that, for infinitely many block lengths, a random ideal in the binary group algebra of the dihedral group is an asymptotically good rate-half code with a high probability. We argue also that a random code that is invariant under the action of an Abelian group G of odd order on k disjoint copies of itself satisfies the binary Gilbert-Varshamov (GV) bound with a high probability for rate 1/k under a condition on the family of groups. The underlying condition is in terms of the growth of the smallest dimension of a nontrivial F/sub 2/-representation of the group and is satisfied by roughly most Abelian groups of odd order, and specifically by almost all cyclic groups of prime order.     

Cyclotomy and duadic codes of prime lengths

We present a cyclotomic approach to the construction of all binary duadic codes of prime lengths. We calculate the number of all binary duadic codes for a given prime length and that of all duadic codes that are not quadratic residue codes. We give necessary and sufficient conditions for p such that all binary duadic codes of length p are quadratic residue (QR) codes. We also show how to determine some weights of duadic codes with the help of cyclotomic numbers     

On computing the weight spectrum of cyclic codes

Two deterministic algorithms of computing the weight spectra of binary cyclic codes are presented. These algorithms have the lowest known complexity for cyclic codes. For BCH codes of lengths 63 and 127, several first coefficients of the weight spectrum in number sufficient to evaluate the bounded distance decoding error probability are computed     

Is the class of cyclic codes asymptotically good?

There is the long-standing question whether the class of cyclic codes is asymptotically good. By an old result of Lin and Weldon, long Bose-Chaudhuri-Hocquenhem (BCH) codes are asymptotically bad. Berman proved that cyclic codes are asymptotically bad if only finitely many primes are involved in the lengths of the codes. We investigate further classes of cyclic codes which also turn out to be asymptotically bad. Based on reduction arguments we give some evidence that there are asymptotically good sequences of binary cyclic codes in which all lengths are prime numbers provided there is any asymptotically good sequence of binary cyclic codes.     

Turbo codes with rate-m/(m+1) constituent convolutional codes

The original turbo codes (TCs), presented in 1993 by Berrou et al., consist of the parallel concatenation of two rate-1/2 binary recursive systematic convolutional (RSC) codes. This paper explains how replacing rate-1/2 binary component codes by rate-m/(m+1) binary RSC codes can lead to better global performance. The encoding scheme can be designed so that decoding can be achieved closer to the theoretical limit, while showing better performance in the region of low error rates. These results are illustrated with some examples based on double-binary (m=2) 8-state and 16-state TCs, easily adaptable to a large range of data block sizes and coding rates. The double-binary 8-state code has already been adopted in several telecommunication standards.     

A new upper bound on the ML decoding error probability of linear binary block codes in AWGN interference

Performance evaluation of maximum-likelihood (ML) soft-decision-decoded binary block codes is usually carried out using bounding techniques. Many tight upper bounds on the error probability of binary codes are based on the so-called Gallager's first bounding technique (GFBT). The tangential sphere bound (TSB) of Poltyrev which has been believed for many years to offer the tightest bound developed for binary block codes is an example. Within the framework of the TSB and GFBT, we apply a new method referred to as the "added-hyper-plane" (AHP) technique, to the decomposition of the error probability. This results in a bound developed upon the application of two stages of the GFBT with two different Gallager regions culminating in a tightened upper bound beyond the TSB. The proposed bound is simple and only requires the spectrum of the binary code.     

The performance of incremental redundancy schemes based on convolutional codes in the block-fading Gaussian collision channel

The throughput performance of incremental redundancy (INR) schemes, based on short constraint length convolutional codes, is evaluated for the block-fading Gaussian collision channel. Results based on simulations and union bound computations are compared to estimates of the achievable throughput performance with random binary and Gaussian coding in the limit of large block lengths, obtained through information outage considerations. For low channel loads, it is observed that INR schemes with binary convolutional codes and limited block length may provide throughput close to the achievable performance for binary random coding. However, for these low loads, compared to binary random coding, Gaussian random coding may provide significantly better throughput performance, which prompts the use of larger modulation constellations. For high channel loads, a relatively large gap in throughput performance between binary convolutional codes and binary random codes indicates a potential for extensive performance improvement by alternative coding strategies. Only small improvements of the throughput have been observed by increasing the complexity through increased state convolutional coding.     

Finite-Dimensional Bounds on Zm and Binary LDPC Codes With Belief Propagation Decoders

This paper focuses on finite-dimensional upper and lower bounds on decodable thresholds of Zopf_m and binary low-density parity-check (LDPC) codes, assuming belief propagation decoding on memoryless channels. A concrete framework is presented, admitting systematic searches for new bounds. Two noise measures are considered: the Bhattacharyya noise parameter and the soft bit value for a maximum a posteriori probability (MAP) decoder on the uncoded channel. For Zopf _m LDPC codes, an iterative m-dimensional bound is derived for m-ary-input/symmetric-output channels, which gives a sufficient stability condition for Zopf_m LDPC codes and is complemented by a matched necessary stability condition introduced herein. Applications to coded modulation and to codes with nonequiprobably distributed codewords are also discussed. For binary codes, two new lower bounds are provided for symmetric channels, including a two-dimensional iterative bound and a one-dimensional noniterative bound, the latter of which is the best known bound that is tight for binary-symmetric channels (BSCs), and is a strict improvement over the existing bound derived by the channel degradation argument. By adopting the reverse channel perspective, upper and lower bounds on the decodable Bhattacharyya noise parameter are derived for nonsymmetric channels, which coincides with the existing bound for symmetric channels     

Tightened Upper Bounds on the ML Decoding Error Probability of Binary Linear Block Codes

The performance of maximum-likelihood (ML) decoded binary linear block codes is addressed via the derivation of tightened upper bounds on their decoding error probability. The upper bounds on the block and bit error probabilities are valid for any memoryless, binary-input and output-symmetric communication channel, and their effectiveness is exemplified for various ensembles of turbo-like codes over the additive white Gaussian noise (AWGN) channel. An expurgation of the distance spectrum of binary linear block codes further tightens the resulting upper bounds     

A method for convergence analysis of iterative probabilisticdecoding

A novel analytical approach to performance evaluation of soft-decoding algorithms for binary linear block codes based on probabilistic iterative error correction is presented. A convergence condition establishing the critical noise rate below which the expected bit-error probability tends to zero is theoretically derived. It explains the capability of iterative probabilistic decoding of binary linear block codes with sparse parity-check matrices to correct, with probability close to one, error patterns with the number of errors (far) beyond half the code minimum distance. Systematic experiments conducted on truncated simplex codes seem to agree well with the convergence condition. The method may also be interesting for the theoretical analysis of the so-called turbo codes     

Two topics on linear unequal error protection codes: Bounds on their length and cyclic code classes

It is possible for a linear block code to provide more protection for selected positions in the input message words than is guaranteed by the minimum distance of the code. Linear codes having this property are called linear unequal error protection (LUEP) codes. Bounds on the length of a LUEP code that ensures a given unequal error protection are derived. A majority decoding method for certain classes of cyclic binary UEP codes is treated. A list of short (i.e., of length less than 16) binary LUEP codes of optimal (i.e., minimal) length and a list of all cyclic binary UEP codes of length less than 40 are included.     

Fixed-length strategies for the binary multiplying channel

An achievable rate region for the binary multiplying channel is established such that block codes exist at all rate pairs within this region, with error probability that decreases exponentially with increasing block length. The result are generalized to asymmetrical rate pair     

Coding for Parallel Channels: Gallager Bounds and Applications to Turbo-Like Codes

The transmission of coded communication systems is widely modeled to take place over a set of parallel channels. This model is used for transmission over block-fading channels, rate-compatible puncturing of turbo-like codes, multicarrier signaling, multilevel coding, etc. New upper bounds on the maximum-likelihood (ML) decoding error probability are derived in the parallel-channel setting. We focus on the generalization of the Gallager-type bounds and discuss the connections between some versions of these bounds. The tightness of these bounds for parallel channels is exemplified for structured ensembles of turbo codes, repeat-accumulate (RA) codes, and some of their recent variations (e.g., punctured accumulate-repeat-accumulate codes). The bounds on the decoding error probability of an ML decoder are compared to computer simulations of iterative decoding. The new bounds show a remarkable improvement over the union bound and some other previously reported bounds for independent parallel channels. This improvement is exemplified for relatively short block lengths, and it is pronounced when the block length is increased. In the asymptotic case, where we let the block length tend to infinity, inner bounds on the attainable channel regions of modern coding techniques under ML decoding are obtained, based solely on the asymptotic growth rates of the average distance spectra of these code ensembles.     

Distributed Arithmetic Coding

We propose a distributed binary arithmetic coder for Slepian-Wolf coding with decoder side information, along with a soft joint decoder. The proposed scheme provides several advantages over existing schemes, and its performance is equal to or better than that of an equivalent scheme based on turbo codes at short and medium block lengths.     

On general Gilbert bounds

We define a distance measure for block codes used over memoryless channels and show that it is related to upper and lower bounds on the low-rate error probability in the same way as Hamming distance is for binary block codes used over the binary symmetric channel. We then prove general Gilbert bounds for block codes using this distance measure. Some new relationships between coding theory and rate-distortion theory are presented.     

A performance comparison of the binary quadratic residue codes withthe 1/2-rate convolutional codes

The 1/2-rate binary quadratic residue (QR) codes, using binary phase-shift keyed (BPSK) modulation and hard decoding, are presented as an efficient system for reliable communication. Performance results of error correction are obtained both theoretically and by means of computer calculations for a number of binary QR codes. These results are compared with the commonly used 1/2-rate convolutional codes with constraint lengths from 3 to 7 for the hard-decision case. The binary QR codes of different lengths are shown to be equivalent in error-correction performance to some 1/2-rate convolutional codes, each of which has a constraint length K that corresponds to the error-control rate d/n and the minimum distance d of the QR codes