Call admission control in cellular networks: A reinforcement learning solution

In this paper, we address the call admission control (CAC) problem in a cellular network that handles several classes of traffic with different resource requirements. The problem is formulated as a semi‐Markov decision process (SMDP) problem. We use a real‐time reinforcement learning (RL) neuro‐dynamic programming (NDP)] algorithm to construct a dynamic call admission control policy. We show that the policies obtained using our TQ‐CAC and NQ‐CAC algorithms, which are two different implementations of the RL algorithm, provide a good solution and are able to earn significantly higher revenues than classical solutions such as guard channel. A large number of experiments illustrates the robustness of our policies and shows how they improve quality of service (QoS) and reduce call‐blocking probabilities of handoff calls even with variable traffic conditions. Copyright © 2004 John Wiley & Sons, Ltd.