Adaptive decentralized control under non-uniqueness of the optimal control

We study the problem of decentralization of flow control in packet-switching networks under the isarithmic scheme. An incoming packet enters the network only if there are permits available at the entry port when it arrives. The actions of the controllers refer to the routing of permits in the network and the control variables are the corresponding probabilities. We study the behavior of adaptive algorithms implemented at the controllers to update these probabilities and seek optimal performance. This problem can be stated as a routing problem in a closed queueing network. The centralized version of a learning automation is a general framework presented along with the proof of asymptotic optimality. Decentralization of the controller gives rise to non-uniqueness of the optimal control parameters. Non-uniqueness can be exploited to construct asymptotically optimal learning algorithms that exhibit different behavior. We implement two different algorithms for the parallel operation and discuss their differences. Convergence is established using the weak convergence methodology. In addition to our theoretical results, we illustrate the main results using the flow control problem as a model example and verify the predicted behavior of the two proposed algorithms through computer simulations, including an example of tracking.The work of this author was partially supported by a grant from the Canadian Institute for Telecommunications Research under the NCE program of the Government of Canada, and partially supported by NSERC grant WFA 0139015 and FCAR-Québec grant 95-NC-1375.The work of this author was supported by a grant from the CITR under the NCE program of the Government of Canada.