Learning with handoff cost constraint for network selection in heterogeneous wireless networks

In heterogeneous wireless networks, network selection algorithms provide the user with the optimum network access choice. The optimal network is evaluated according to network parameters. Considering that the network parameters are dynamic and unavailable for the user in realistic heterogeneous wireless network environments, most existing network selection algorithms cannot work effectively. Learning‐based algorithms can address the problem of uncertain network parameters, while they commonly need considerable network handoff, resulting in unbearable handoff cost. In order to tackle the uncertainty of network parameters, we formulate the network selection problem as a multi‐armed bandit problem. Moreover, two online learning‐based network selection algorithms with a special consideration on reducing network handoff cost are proposed. By updating in a block manner, both algorithms achieve optimal logarithmic‐order regret and limited network handoff cost. The simulation indicates that the two algorithms can significantly reduce the network handoff cost and improve the transmission performance compared with existing algorithms, simultaneously. Copyright © 2014 John Wiley & Sons, Ltd.