基于深度强化学习的边缘网络内容协作缓存与传输方案研究

为了应对第五代无线通信网络中数据吞吐量急剧增加的问题，移动边缘缓存成为了一种有效的解决方案。它通过在边缘设备上存储网络内容，减轻回程链路和核心网络的负担，缩短服务时延。到目前为止，大多数边缘缓存研究主要在协作内容缓存的优化方面，忽略了内容传输的效率。研究超密集网络的内容协作边缘缓存与无线带宽资源的分配问题，通过余弦相似度和高斯相似度求解基站之间总的相似度，将网络中的小基站根据总相似度进行分组，把缓存和无线带宽分配问题建模成一个长期混合整数的非线性规划问题（LT-MINLP），进而将协作边缘缓存与带宽分配问题转变为一个带约束的马尔可夫决策过程，并利用深度确定性策略梯度DDPG模型，提出了一种基于深度强化学习的内容协作边缘缓存与带宽分配算法CBDDPG。提出的基站分组方案增加了基站之间文件共享的机会，提出的CBDDPG算法的缓存方案利用DDPG双网络机制能更好地捕捉用户的请求规律，优化缓存部署。将CBDDPG算法与三种基线算法（RBDDPG、LCCS和CB-TS）进行了对比实验，实验结果表明所提方案能够有效地提高内容缓存命中率，降低内容传递的时延，提升用户体验。

Deep reinforcement learning based-edge network content cooperative caching and transmission scheme

In order to address the problem of rapid increase of data throughput in fifth-generation wireless communication networks, mobile edge caching has become a useful solution. It can reduce the burden on the backhaul link and core network, cut down service latency by storing network content on edge devices. So far, most edge caching solutions have mainly focused on optimizing cooperative content caching, and ignored the efficiency of content transmission. This paper studied cooperative edge caching and wireless bandwidth allocation problems in ultra-dense networks, calculated the overall similarity between the base stations by using cosine similarity and Gaussian similarity, and grouped the small base stations according to total similarity in the network. The caching and radio bandwidth allocation problems were modeled as a long-term mixed-integer non-linear programming(LT-MINLP). Then, the cooperative edge caching and wireless bandwidth allocation problem were transformed into a constrained Markov decision process. Finally, it proposed cooperative edge caching and radio resource allocation scheme by using the DDPG model. And it proposed deep reinforcement learning based-edge content cooperative caching and bandwidth allocation algorithm CBDDPG. The proposed base station group strategy increased the file sharing opportunity among base stations, the cache scheme of the proposed CBDDPG algorithm used DDPG dual-network mechanism, which could better capture the regularity of user requests and optimize cache deployment. The proposed CBDDPG algorithm was compared to three baseline algorithms RBDDPG, LCCS and CB-TS) in experiments. Experimental results show the proposed strategy can effectively enhance the content cache hit ratio, reduce the delay of content delivery and improve the user experience.