The empirical distribution of good codes |
| |
Authors: | Shamai S Verdu S |
| |
Affiliation: | Dept. of Electr. Eng., Technion-Israel Inst. of Technol., Haifa; |
| |
Abstract: | Let the kth-order empirical distribution of a code be defined as the proportion of k-strings anywhere in the codebook equal to every given k-string. We show that for any fixed k, the kth-order empirical distribution of any good code (i.e., a code approaching capacity with vanishing probability of error) converges in the sense of divergence to the set of input distributions that maximize the input/output mutual information of k channel uses. This statement is proved for discrete memoryless channels as well as a large class of channels with memory. If k grows logarithmically (or faster) with blocklength, the result no longer holds for certain good codes, whereas for other good codes, the result can be shown for k growing as fast as a certain fraction of blocklength |
| |
Keywords: | |
|
|