The Kullback-Leibler divergence rate between Markov sources |
| |
Authors: | Rached Z Alajaji F Campbell LL |
| |
Affiliation: | Dept. of Math. & Stat., Queen's Univ., Kingston, Ont., Canada; |
| |
Abstract: | In this work, we provide a computable expression for the Kullback-Leibler divergence rate lim/sub n/spl rarr//spl infin//1/nD(p/sup (n)//spl par/q/sup (n)/) between two time-invariant finite-alphabet Markov sources of arbitrary order and arbitrary initial distributions described by the probability distributions p/sup (n)/ and q/sup (n)/, respectively. We illustrate it numerically and examine its rate of convergence. The main tools used to obtain the Kullback-Leibler divergence rate and its rate of convergence are the theory of nonnegative matrices and Perron-Frobenius theory. Similarly, we provide a formula for the Shannon entropy rate lim/sub n/spl rarr//spl infin//1/nH(p/sup (n)/) of Markov sources and examine its rate of convergence. |
| |
Keywords: | |
|
|