Binomial moments of the distance distribution: bounds andapplications

We study a combinatorial invariant of codes which counts the number of ordered pairs of codewords in all subcodes of restricted support in a code. This invariant can be expressed as a linear form of the components of the distance distribution of the code with binomial numbers as coefficients. For this reason we call it a binomial moment of the distance distribution. Binomial moments appear in the proof of the MacWilliams (1963) identities and in many other problems of combinatorial coding theory. We introduce a linear programming problem for bounding these linear forms from below. It turns out that some known codes (1-error-correcting perfect codes, Golay codes, Nordstrom-Robinson code, etc.) yield optimal solutions of this problem, i.e., have minimal possible binomial moments of the distance distribution. We derive several general feasible solutions of this problem, which give lower bounds on the binomial moments of codes with given parameters, and derive the corresponding asymptotic bounds. Applications of these bounds include new lower bounds on the probability of undetected error for binary codes used over the binary-symmetric channel with crossover probability p and optimality of many codes for error detection. Asymptotic analysis of the bounds enables us to extend the range of code rates in which the upper bound on the undetected error exponent is tight     

Performance of a powerful error-correcting and -detecting codingscheme for the North American Basic Teletext System (NABTS) for randomindependent errors: methods, equations, calculations and results

For a powerful layered, upward- and downward-compatible error-correcting and error-detecting scheme for NABTS, various bit error rate (BER) related performance measures are derived and calculated for random independent errors. The methods, equations, calculations and results are given for the least powerful one-byte suffix codes, for the two-byte suffix code, called code C, and for the double and single bundle codes formed by using code C for each data block (i.e. horizontally) and also vertically, thus forming a product code, for a specified, but variable, number of data blocks. Performance bounds and equations for probabilities of correct decoding of error and of decoding failure are given. The weight enumerators for a number of one-byte suffix codes are calculated, and those of weight four are classified into types depending on the number of ones occurring in a byte, and in other arrangements. Performance analyses and comparisons with a code for Japanese teletext are included. Analyses used in computer simulation studies are described     

Error analysis of a statistical decision method

For the recognition of patterns as members of certain classes it is assumed that the probability distributions of certain characteristic features are known. For a decision based on the maximum likelihood criterion bounds on the statistical error are derived. In the case of normally distributed and statistically independent features these bounds are evaluated. Conditions are given under which the error tends to zero as the number of characteristic features goes to infinity.     

New closed-form bounds on the performance of coding in correlated Rayleigh fading

New, simple bounds are presented for the probability of error in a binary hypothesis test for communications using diversity signaling in correlated Rayleigh fading. The bounds are developed in the context of pairwise error-event probabilities in decoding an error-correction code. A long-standing conjecture regarding the form of worst-case error events in exponentially correlated Rayleigh fading is also proven. The utility of the results is illustrated by their application to transfer-function bounds on the probability of bit error for a system using a convolutional code. The closed-form transfer-function bounds are shown to be tighter than previously developed transfer-function bounds for communications in exponentially correlated Rayleigh fading.     

Source and channel rate allocation for channel codes satisfying theGilbert-Varshamov or Tsfasman-Vladut-Zink bounds

We derive bounds for optimal rate allocation between source and channel coding for linear channel codes that meet the Gilbert-Varshamov or Tsfasman-Vladut-Zink (1984) bounds. Formulas giving the high resolution vector quantizer distortion of these systems are also derived. In addition, we give bounds on how far below the channel capacity the transmission rate should be for a given delay constraint. The bounds obtained depend on the relationship between channel code rate and relative minimum distance guaranteed by the Gilbert-Varshamov bound, and do not require sophisticated decoding beyond the error correction limit. We demonstrate that the end-to-end mean-squared error decays exponentially fast as a function of the overall transmission rate, which need not be the case for certain well-known structured codes such as Hamming codes     

On Universal Properties of Capacity-Approaching LDPC Code Ensembles

This paper is focused on the derivation of some universal properties of capacity-approaching low-density parity-check (LDPC) code ensembles whose transmission takes place over memoryless binary-input output-symmetric (MBIOS) channels. Properties of the degree distributions, graphical complexity, and the number of fundamental cycles in the bipartite graphs are considered via the derivation of information-theoretic bounds. These bounds are expressed in terms of the target block/bit error probability and the gap (in rate) to capacity. Most of the bounds are general for any decoding algorithm, and some others are proved under belief propagation (BP) decoding. Proving these bounds under a certain decoding algorithm, validates them automatically also under any suboptimal decoding algorithm. A proper modification of these bounds makes them universal for the set of all MBIOS channels which exhibit a given capacity. Bounds on the degree distributions and graphical complexity apply to finite-length LDPC codes and to the asymptotic case of an infinite block length. The bounds are compared with capacity-approaching LDPC code ensembles under BP decoding, and they are shown to be informative and are easy to calculate. Finally, some interesting open problems are considered.     

A comparison of known codes, random codes, and the best codes

This paper calculates new bounds on the size of the performance gap between random codes and the best possible codes. The first result shows that, for large block sizes, the ratio of the error probability of a random code to the sphere-packing lower bound on the error probability of every code on the binary symmetric channel (BSC) is small for a wide range of useful crossover probabilities. Thus even far from capacity, random codes have nearly the same error performance as the best possible long codes. The paper also demonstrates that a small reduction k-k˜ in the number of information bits conveyed by a codeword will make the error performance of an (n,k˜) random code better than the sphere-packing lower bound for an (n,k) code as long as the channel crossover probability is somewhat greater than a critical probability. For example, the sphere-packing lower bound for a long (n,k), rate 1/2, code will exceed the error probability of an (n,k˜) random code if k-k˜>10 and the crossover probability is between 0.035 and 0.11=H^-1(1/2). Analogous results are presented for the binary erasure channel (BEC) and the additive white Gaussian noise (AWGN) channel. The paper also presents substantial numerical evaluation of the performance of random codes and existing standard lower bounds for the BEC, BSC, and the AWGN channel. These last results provide a useful standard against which to measure many popular codes including turbo codes, e.g., there exist turbo codes that perform within 0.6 dB of the bounds over a wide range of block lengths     

Performance Analysis and Code Design for Minimum Hamming Distance Fusion in Wireless Sensor Networks

Distributed classification fusion using error-correcting codes (DCFECC) has recently been proposed for wireless sensor networks operating in a harsh environment. It has been shown to have a considerably better capability against unexpected sensor faults than the optimal likelihood fusion. In this paper, we analyze the performance of a DCFECC code with minimum Hamming distance fusion. No assumption on identical distribution for local observations, as well as common marginal distribution for the additive noises of the wireless links, is made. In addition, sensors are allowed to employ their own local classification rules. Upper bounds on the probability of error that are valid for any finite number of sensors are derived based on large deviations technique. A necessary and sufficient condition under which the minimum Hamming distance fusion error vanishes as the number of sensors tends to infinity is also established. With the necessary and sufficient condition and the upper error bounds, the relation between the fault-tolerance capability of a DCFECC code and its pair-wise Hamming distances is characterized, and can be used together with any code search criterion in finding the code with the desired fault-tolerance capability. Based on the above results, we further propose a code search criterion of much less complexity than the minimum Hamming distance fusion error criterion adopted earlier by the authors. This makes the code construction with acceptable fault-tolerance capability for a network with over a hundred of sensors practical. Simulation results show that the code determined based on the new criterion of much less complexity performs almost identically to the best code that minimizes the minimum Hamming distance fusion error. Also simulated and discussed are the performance trends of the codes searched based on the new simpler criterion with respect to the network size and the number of hypotheses     

Performance bounds for optimum multiuser DS-CDMA systems

In this letter we estimate the bit error probability (BEP) of optimum multiuser detection for synchronous and asynchronous code division multiple access (CDMA) systems on Gaussian and fading channels. We first compute an upper bound and a lower bound on the bit error probability for a given spreading code, then average the bounds over a few thousand sets of spreading codes. These bounds are obtained from a partial distance spectrum. On Gaussian channels, the upper bound converges to the lower bound at moderate to large signal-to-noise ratios. However, on fading channels the upper bound does not converge, hence we present our results for the lower bound only. The numerical results show that: 1) the BEP of a 31-user CDMA system with binary random spreading codes of length 31 is only two to four times higher than the BEP of the single user system; 2) the number of users that can be accommodated in an asynchronous CDMA system is larger than the processing gain; and 3) optimum multiuser detection outperforms linear detection (e.g., the decorrelating detector) by about 2.8 to 5.7 dB     

Undetected error probabilities of binary primitive BCH codes forboth error correction and detection

We investigate the undetected error probabilities for bounded-distance decoding of binary primitive BCH codes when they are used for both error correction and detection on a binary symmetric channel. We show that the undetected error probability of binary linear codes can be simplified and quantified if the weight distribution of the code is binomial-like. We obtain bounds on the undetected error probability of binary primitive BCH codes by applying the result to the code and show that the bounds are quantified by the deviation factor of the true weight distribution from the binomial-like weight distribution     

On upper bounds for unrestricted binary-error-correcting codes

In this paper improved upper bounds are obtained forA(n,d), the maximum number of binary code vectors in a code of block lengthnand minimum distanced. Known bounds are presented in a unified way and then refined, giving improvement over best previously published results in almost all cases. Finally, tabulations of the improved results are given. Asymptotically, the new bounds agree with those given by Elias.     

On Randomized Linear Network Codes and Their Error Correction Capabilities

Randomized linear network code for single source multicast was introduced and analyzed in Ho et al. (IEEE Transactions on Information Theory, October 2006) where the main results are upper bounds for the failure probability of the code. In this paper, these bounds are improved and tightness of the new bounds is studied by analyzing the limiting behavior of the failure probability as the field size goes to infinity. In the linear random coding setting for single source multicast, the minimum distance of the code defined in Zhang, (IEEE Transactions on Information Theory, January 2008) is a random variable taking nonnegative integer values that satisfy the inequality in the Singleton bound recently established in Yeung and Cai (Communications in Information and Systems, 2006) for network error correction codes. We derive a bound on the probability mass function of the minimum distance of the random linear network code based on our improved upper bounds for the failure probability. Codes having the highest possible minimum distance in the Singleton bound are called maximum distance separable (MDS). The bound on the field size required for the existence of MDS codes reported in Zhang, (IEEE Transactions on Information Theory, January 2008) and Matsumoto (arXiv:cs.IT/0610121, Oct. 2006) suggests that such codes exist only when field size is large. Define the degradation of a code as the difference between the highest possible minimum distance in the Singleton bound and the actual minimum distance of the code. The bound for the probability mass function of the minimum distance leads to a bound on the field size required for the existence of network error correction codes with a given maximum degradation. The result shows that allowing minor degradation reduces the field size required dramatically.     

Recursive decoding and its performance for low-rate Reed-Muller codes

Recursive decoding techniques are considered for Reed-Muller (RM) codes of growing length n and fixed order r. An algorithm is designed that has complexity of order nlogn and corrects most error patterns of weight up to n(1/2-/spl epsiv/) given that /spl epsiv/ exceeds n/sup -1/2r/. This improves the asymptotic bounds known for decoding RM codes with nonexponential complexity. To evaluate decoding capability, we develop a probabilistic technique that disintegrates decoding into a sequence of recursive steps. Although dependent, subsequent outputs can be tightly evaluated under the assumption that all preceding decodings are correct. In turn, this allows us to employ second-order analysis and find the error weights for which the decoding error probability vanishes on the entire sequence of decoding steps as the code length n grows.     

Improved bounds on the word error probability of RA(2) codes with linear-programming-based decoding

This paper deals with the linear-programming-based decoding algorithm of Feldman and Karger for repeat-accumulate "turbo-like" codes. We present a new structural characterization that captures the event that decoding fails. Based on this structural characterization, we develop polynomial algorithms that, given an RA(2) code, compute upper and lower bounds on the word error probability P/sub w/ for the binary-symmetric and the additive white Gaussian noise (AWGN) channels. Our experiments with an implementation of these algorithms for bounding P/sub w/ demonstrate in many interesting cases an improvement in the upper bound on the word error probability by a factor of over 1000 compared to the bounds by Feldman et al.. The experiments also indicate that the improvement in upper bound increases as the codeword length increases and the channel noise decreases. The computed lower bounds on the word error probability in our experiments are roughly ten times smaller than the upper bound.     

Error Exponents for Two Soft-Decision Decoding Algorithms of Reed–Muller Codes

Error exponents are studied for recursive and majority decoding of general Reed–Muller (RM) codes ${rm RM}(r,m)$ used on the additive white Gaussian noise (AWGN) channels. Both algorithms have low complexity and correct many error patterns whose weight exceeds half the code distance. Decoding consists of multiple consecutive steps, which repeatedly recalculate the input symbols and determine different information symbols using soft-decision majority voting. For any code ${rm RM}(r,m)$, we estimate the probabilities of the information symbols obtained in these recalculations and derive the analytical upper bounds for the block error rates of the recursive and majority decoding. In the case of a low noise, we also obtain the lower bounds and show that the upper bounds are tight. For a higher noise, these bounds closely approach our simulation results.      

Error Probability Performance of M-ary CPSK Systems With Intersymbol Interference

We derive upper and lower bounds on the probability of error ofM-ary CPSK systems subject to intersymbol interference and additive Gaussian noise. The bounds are expressed in terms of the error probability obtained with a finite pulse train and some parameters associated with the residual pulse train. Methods are given to compute the probability of error with a finite number of interference terms and it is shown that the difference between the upper and the lower bounds is a monotone decreasing function of the number of pulses in the finite pulse train. The applicability of the method to compute the probability of error with any desired degree of accuracy is illustrated by examples for quaternary and octonary systems.     

Quantum error detection .II. Bounds

For pt.I see ibid., vol.46, no.3, p.778-88 (2000). In Part I of this paper we formulated the problem of error detection with quantum codes on the depolarizing channel and gave an expression for the probability of undetected error via the weight enumerators of the code. In this part we show that there exist quantum codes whose probability of undetected error falls exponentially with the length of the code and derive bounds on this exponent. The lower (existence) bound is proved for stabilizer codes by a counting argument for classical self-orthogonal quaternary codes. Upper bounds are proved by linear programming. First we formulate two linear programming problems that are convenient for the analysis of specific short codes. Next we give a relaxed formulation of the problem in terms of optimization on the cone of polynomials in the Krawtchouk basis. We present two general solutions of the problem. Together they give an upper bound on the exponent of undetected error. The upper and lower asymptotic bounds coincide for a certain interval of code rates close to 1     

Capacity results for the discrete memoryless network

A discrete memoryless network (DMN) is a memoryless multiterminal channel with discrete inputs and outputs. A sequence of inner bounds to the DMN capacity region is derived by using code trees. Capacity expressions are given for three classes of DMNs: (1) a single-letter expression for a class with a common output, (2) a two-letter expression for a binary-symmetric broadcast channel (BC) with partial feedback, and (3) a finite-letter expression for push-to-talk DMNs. The first result is a consequence of a new capacity outer bound for common output DMNs. The third result demonstrates that the common practice of using a time-sharing random variable does not include all time-sharing possibilities, namely, time sharing of channels. Several techniques for improving the bounds are developed: (1) causally conditioned entropy and directed information simplify the inner bounds, (2) code trellises serve as simple code trees, (3) superposition coding and binning with code trees improves rates. Numerical computations show that the last technique enlarges the best known rate regions for a multiple-access channel (MAC) and a BC, both with feedback. In addition to the rate bounds, a sequence of inner bounds to the DMN reliability function is derived. A numerical example for a two-way channel illustrates the behavior of the error exponents.     

ERROR-DETECTING CODES AND UNDETECTED ERROR PROBABILITIES

The definition of good codes for error-detection is given. It is proved that a (n, k) linear block code in GF(q) are the good code for error-detection, if and only if its dual code is also. A series of new results about the good codes for error-detection are derived. New lower bounds for undetected error probabilities are obtained, which are relative to n and k only, and not the weight structure of the codes.     

The effects of phase noise on trellis coded modulation overGaussian and fading channels

In this paper, error performance bounds are derived for 8-PSK trellis coded modulation (TCM) over Gaussian and fading channels in the presence of phase noise. Coherent demodulation combined with both phase-locked loop (PLL) and transparent tone in band technique (TTIB) for phase recovery is assumed. The analysis is then applied to a number of Ungerboeck 8-PSK trellis codes and the results for the bit error rate (BER) performance, obtained both analytically and by Monte Carlo simulation, are presented. The derived bounds can be directly extended to any M-PSK trellis code