基于深度确定性策略梯度的虚拟网络功能迁移优化算法

针对NFV/SDN架构下，服务功能链(SFC)的资源需求动态变化引起的虚拟网络功能(VNF)迁移优化问题，该文提出一种基于深度强化学习的VNF迁移优化算法。首先，在底层CPU、带宽资源和SFC端到端时延约束下，建立基于马尔可夫决策过程(MDP)的随机优化模型，该模型通过迁移VNF来联合优化网络能耗和SFC端到端时延。其次，由于状态空间和动作空间是连续值集合，提出一种基于深度确定性策略梯度(DDPG)的VNF智能迁移算法，从而得到近似最优的VNF迁移策略。仿真结果表明，该算法可以实现网络能耗和SFC端到端时延的折中，并提高物理网络的资源利用率。

Virtual Network Function Migration Optimization Algorithm Based on Deep Deterministic Policy Gradient

To solve the problem of Virtual Network Function (VNF) migration optimization, which is caused by the dynamic change of resource requirements of Service Function Chain (SFC) under Network Function Virtualization/ Software Defined Network (NFV/SDN) architecture, a VNF migration optimization algorithm is proposed based on deep reinforcement learning. Firstly, based on the underlying CPU, bandwidth resources and SFC end-to-end delay constraints, a Markov Decision Process (MDP) based stochastic optimization model is established. This model is used to optimize jointly network energy consumption and SFC end-to-end delay by migrating VNF. Secondly, since the state space and action space of this paper are continuous value sets, a VNF intelligent migration algorithm based on Deep Deterministic Policy Gradient (DDPG) is proposed to obtain an approximate optimal VNF migration strategy. The simulation results show that the algorithm can achieve the compromise between network energy consumption and SFC end-to-end delay, and improve the resource utilization of the physical network.