Cross-entropy minimization given fully decomposable subset and aggregate constraints (Corresp.)

The principle of maximum entropy and the principle of minimum cross-entropy (minimum directed divergence, minimum discrimination information) have been applied recently to problems in queuing theory and computer-system performance modeling. These information-theoretic principles estimate probability distributions based on information in the form of known expected values. In the case of queuing theory and computer-system modeling, the known expected values arise from rate balance equations. This correspondence concerns situations in which the system state probabilities decompose into disjoint subsets and in which the known expected values are either expectations conditional on a specific subset or expectations involving aggregate subset probabilities. New properties of minimum cross-entropy distributions are derived and an efficient method of computing these distributions is derived. Computational examples are included. In the case of queuing theory and computer-system modeling, the disjoint subsets correspond to internal device states, and the aggregate probabilities correspond to overall device states. The results here apply when one has both rate balance equations for device equilibrium involving internal device state probabilities, as well as rate balance equations for system equilibrium involving aggregate device state probabilities.