5G 异构网络中基于多目标 Actor-Critic的资源分配

在 5G 异构网络(heterogeneous network,HetNet)中广泛部署小基站可以提高网络容量和用户速率，但密集部署也会 产生严重干扰和更高能耗问题。为了最大化网络能量效率(energy efficiency,EE)并保证用户服务质量(quality of service, QoS),提出了一种在小蜂窝基站中嵌入能量收集器供电的资源分配方案。首先，针对网络系统的下行链路，将频谱和小基站 发射功率分配问题建模为联合优化系统能效和用户满意度的多目标优化问题。其次，提出了基于深度强化学习的多目标演 员-评论家(multi-objective actor-critic,MAC)资源分配算法求解所建立的优化模型。最后，仿真结果表明，相比于其他传统学 习算法，能量效率提高了11.96%～12.37%,用户满意度提高了11.45%～27.37%。

Resource allocation based on multi-objective Actor-Critic in 5G heterogeneous networks

The widespread deployment of small base stations in 5G heterogeneous networks(HetNet)can improve network capacity and user rates,but the dense deployment will also cause severe interference and higher energy consumption problems.In order to maximize the network energy efficiency(EE)and guarantee the user quality of service(QoS),this paper presents a resource allocation scheme that embeds energy harvester power supply in small cell base stations.Firstly,for the downlink of the network system,the spectrum and small base station transmit power allocation problem was modeled as a multi-objective optimization problem to jointly optimize system energy efficiency and user satisfaction.Secondly,a multi-objective actor-critic(MAC)resource allocation algorithm based on deep reinforcement learning was proposed to solve the established optimization model.Finally,simulation results show that compared with other traditional learning algorithms,the energy efficiency of the proposed algorithm is improved by 11.96%～12.37%,and the user satisfaction is improved by 11.45%～27.37%.