基于强化学习的5G网络切片虚拟网络功能迁移算法

针对5G网络切片架构下业务请求动态性引起的虚拟网络功能(VNF)迁移优化问题，该文首先建立基于受限马尔可夫决策过程(CMDP)的随机优化模型以实现多类型服务功能链(SFC)的动态部署，该模型以最小化通用服务器平均运行能耗为目标，同时受限于各切片平均时延约束以及平均缓存、带宽资源消耗约束。其次，为了克服优化模型中难以准确掌握系统状态转移概率及状态空间过大的问题，该文提出了一种基于强化学习框架的VNF智能迁移学习算法，该算法通过卷积神经网络(CNN)来近似行为值函数，从而在每个离散的时隙内根据当前系统状态为每个网络切片制定合适的VNF迁移策略及CPU资源分配方案。仿真结果表明，所提算法在有效地满足各切片QoS需求的同时，降低了基础设施的平均能耗。

Virtual Network Function Migration Algorithm Based on Reinforcement Learning for 5G Network Slicing

In order to solve the Virtual Network Function (VNF) migration optimization problem caused by the dynamicity of service requests on the 5G network slicing architecture, firstly, a stochastic optimization model based on Constrained Markov Decision Process (CMDP) is established to realize the dynamic deployment of multi-type Service Function Chaining (SFC). This model aims to minimize the average sum operating energy consumption of general servers, and is subject to the average delay constraint for each slicing as well as the average cache, bandwidth resource consumption constraints. Secondly, in order to overcome the issue of having difficulties in acquiring the accurate transition probabilities of the system states and the excessive state space in the optimization model, a VNF intelligent migration learning algorithm based on reinforcement learning framework is proposed. The algorithm approximates the behavior value function by Convolutional Neural Network (CNN), so as to formulate a suitable VNF migration strategy and CPU resource allocation scheme for each network slicing according to the current system state in each discrete time slot. The simulation results show that the proposed algorithm can effectively meet the QoS requirements of each slice while reducing the average energy consumption of the infrastructure.