基于深度强化学习的多用户计算卸载优化模型和算法

在移动边缘计算(MEC)密集部署场景中,边缘服务器负载的不确定性容易造成边缘服务器过载,从而导致计算卸载过程中时延和能耗显著增加。针对该问题,该文提出一种多用户计算卸载优化模型和基于深度确定性策略梯度(DDPG)的计算卸载算法。首先,考虑时延和能耗的均衡优化建立效用函数,以最大化系统效用作为优化目标,将计算卸载问题转化为混合整数非线性规划问题。然后,针对该问题状态空间大、动作空间中离散和连续型变量共存,对DDPG深度强化学习算法进行离散化改进,基于此提出一种多用户计算卸载优化方法。最后,使用该方法求解非线性规划问题。仿真实验结果表明,与已有算法相比,所提方法能有效降低边缘服务器过载概率,并具有很好的稳定性。     

超密集网络中基于MEC的动态任务卸载方案

超密集网络(Ultra-dense Network,UDN)中集成移动边缘计算(Mobile Edge Computing,MEC),是5G中为用户提供计算资源的可靠方式,在多种因素影响下进行MEC任务卸载决策一直都是一个研究热点。目前已存在大量任务卸载相关的方案,但是这些方案中很少将重心放在用户在不同条件下的能耗需求差异上,无法有效提升用户体验质量(Quality of Experience,QoE)。在动态MEC系统中提出了一个考虑用户能耗需求的多用户任务卸载问题,通过最大化满意度的方式提升用户QoE,并将现有的深度强化学习算法进行了改进,使其更加适合求解所提优化问题。仿真结果表明,所提算法较现有算法在算法收敛性以及稳定性上具有一定提升。     

一种主从MEC服务器协作卸载与资源分配方案

移动边缘计算(Mobile Edge Computing,MEC)通过将云计算能力下沉至用户侧,提高了用户的任务执行能力.但在热点小区中,MEC服务器存在计算资源有限的问题.为了减少热点小区内任务执行总代价,提出了一种基于主从MEC系统的任务联合卸载方案.首先,方案随机生成卸载集,然后将卸载集内的任务分配至目标MEC服...     

一种面向随机计算卸载的两层无人机能耗优化方法

当物联网设备(Internet of Things Device,IoTD)面临随机到达且复杂度高的计算任务时,因自身计算资源和能力所限,无法进行实时高效的处理。为了应对此类问题,设计了一种两层无人机辅助的移动边缘计算(Mobile Edge Computing,MEC)模型。在该模型中,考虑到IoTD处理随机计算任务时的局限性,引入多架配备MEC服务器的下层无人机和单架上层无人机进行协同处理。为了实现系统能耗最优化,提出了一种资源优化和多无人机位置部署方案,根据计算任务到达的随机性,应用李雅普诺夫优化方法将能耗最小化问题转化为一个确定性问题,应用差分进化(Differential Evolution,DE)算法进行多次变异、交叉和选择取得无人机的优化部署方案;采用深度确定性策略梯度(Depth Deterministic policy Gradient,DDPG)算法对带宽分配、计算资源分配、传输功率分配和任务卸载分配进行联合优化。实验结果表明,该算法相较于对比算法系统能耗降低35%,充分验证了其可行性和有效性。     

基于深度强化学习的边缘视频任务卸载策略

随着物联网的发展以及智能设备的普及,视频处理技术已广泛应用于生活中。自动驾驶、产品质检等应用场景对视频处理技术的实时性需求逐步提高,移动边缘计算为计算能力不足和能源受限的设备提供计算资源以执行时延敏感性任务,为实时视频处理提供了新的计算架构。本文搭建了一个视频计算卸载场景,并以视频检测为任务,以系统时延为优化目标,建立了计算卸载模型和马尔可夫决策模型;考虑到计算卸载场景的复杂动态因素,如带宽波动、设备数量、任务大小等,以最小化系统时延为目标,提出了一种基于深度强化学习的计算卸载策略进行求解。实验表明,与其他基线方案相比,该卸载策略能够适应较复杂卸载场景,有效降低系统时延。     

基于多智能体元强化学习的车联网协同服务缓存和计算卸载

为了降低求解优化问题的难度,提出一种双层的多路侧单元(RSU)协同缓存框架将问题进行解耦.外层采用多智能体元强化学习方法,在每个本地智能体进行决策学习的同时,采用长短期记忆网络作为元智能体来平衡本地决策并加速学习过程,从而得到最优的RSU缓存策略;内层采用拉格朗日乘子法求解最佳协同卸载策略,实现RSU间的任务分配.基于...     

移动设备自适应计算卸载研究

通过卸载,移动云计算能够节省处理能量,这是最大限度降低功耗的最有效方法之一。自适应计算卸载属于局部计算卸载,它根据特定条件,将整项任务划分成更小的单元,且只有耗能大的任务单元被发送到云中进行卸载。文章分析移动云计算卸载的需求,给出移动云计算的3层架构,研究MACS(移动增强云服务)的架构,并基于3项限制条件来优化移动增强云服务。最后,文章提供5种云路径选择方法,讨论云路径选择中的带宽、价格、速度、安全性和可用性问题。     

协同边缘网络中智能计算卸载与资源优化算法

针对具有依赖关系的计算密集型应用任务面临的卸载决策难题,提出了一种基于优先级的深度优先搜索调度策略。考虑到用户能量受限和移动性,构建了一种联合用户下行能量捕获和上行计算任务卸载的网络模型,并在此基础上建立了端到端优化目标函数。结合任务优先级及时延约束,利用深度强化学习自学习的优势,将任务卸载决策问题建模为马尔科夫模型,并设计了基于任务相关性的Dueling Double DQN(D3QN)算法对问题进行求解。仿真数据表明,所提算法较其他算法能够满足更多用户的时延要求,并能减少9%～10%的任务执行时延。     

基于安全保障的边缘计算卸载方案

提出了基于安全管理的边缘计算卸载方案,并基于量子进化算法(QEA)设计了卸载决策方案。该方案保证了用户在边缘计算网络中进行计算卸载的安全性。仿真结果表明,与常规计算卸载方案对比,本方案能在保证计算卸载安全的情况下有效降低整个系统的开销。     

基于强化学习的移动边缘计算资源分配方法

当前的移动边缘计算资源分配结构多为单向形式,资源分配效率较低,导致资源分配比下降,文中设计了一种基于强化学习的移动边缘计算资源分配方法,并通过实验验证了其有效性。根据当前的测试需求,首先部署了资源采集节点,然后采用多阶的方式,提升整体的资源分配效率,构建多阶迁移资源分配结构,最后设计了移动边缘计算强化学习资源分配模型,采用动态化辅助协作处理的方式来实现资源分配。测试结果表明,对于选定的5个测试周期,经过3个分配组的测定及比对,最终得出的资源分配比均可以达到5.5以上,这说明在强化学习技术的辅助下,文中设计的移动边缘计算资源分配方法更加灵活、多变,针对性较强,具有实际的应用价值。     

基于深度强化学习的云边协同计算迁移研究

基于单一边缘节点计算、存储资源的有限性及大数据场景对高效计算服务的需求,本文提出了一种基于深度强化学习的云边协同计算迁移机制.具体地,基于计算资源、带宽和迁移决策的综合性考量,构建了一个最小化所有用户任务执行延迟与能耗权重和的优化问题.基于该优化问题提出了一个异步云边协同的深度强化学习算法,该算法充分利用了云边双方的计...     

MEC系统中基于多智能体深度增强学习的协作式任务卸载和资源分配

移动边缘计算(MEC)通过将计算任务卸载到MEC服务器上,在缓解智能移动设备计算负载的同时,可以降低服务时延。然而目前在MEC系统中,关于任务卸载和资源分配仍然存在以下问题：1)边缘节点间缺乏协作;2)计算任务到达与实际环境中动态变化的特征不匹配;3)协作式任务卸载和资源分配动态联合优化问题。为解决上述问题,文章在协作式MEC架构的基础上,提出了一种基于多智能体的深度确定性策略梯度算法(MADDPG)的任务卸载和资源分配算法,最小化系统中所有用户的长期平均成本。仿真结果表明,该算法可以有效降低系统的时延及能耗。     

基于k-匿名的隐私保护计算卸载方法

针对移动边缘计算(MEC)中用户的卸载任务及卸载频率可能使用户被攻击者锁定的问题,该文提出一种基于k-匿名的隐私保护计算卸载方法。首先,该方法基于用户间卸载任务及其卸载频率的差异性,提出隐私约束并建立基于卸载频率的隐私保护计算卸载模型;然后,提出基于模拟退火的隐私保护计算卸载算法(PCOSA)求得最优的k-匿名分组结果和组内各任务的隐私约束频率;最后,在卸载过程中改变用户原始卸载频率满足隐私约束,最小化终端能耗。仿真结果表明,PCOSA算法能找出用户所处MEC节点下与用户卸载表现最相近的k个用户形成匿名集,有效保护了所有用户隐私。     

基于Lyapunov优化的隐私感知计算卸载方法

移动边缘计算(MEC)中计算卸载决策可能暴露用户特征,导致用户被锁定。针对此问题,该文提出一种基于Lyapunov优化的隐私感知计算卸载方法。首先,该方法定义卸载任务中的隐私量,并引入隐私限制使各MEC节点上卸载任务的累积隐私量尽可能小;然后,提出假任务机制权衡终端能耗和隐私保护的关系,当系统因隐私限制无法正常执行计算卸载时,在MEC节点生成虚假的卸载任务以降低累积隐私量;最后,建立隐私感知计算卸载模型,并基于Lyapunov优化原理求解。仿真结果表明,基于Lyapunov优化的隐私感知卸载算法(LPOA)能使用户的累积隐私量稳定在0附近,且总卸载频率与不考虑隐私的决策一致,有效保护了用户隐私,同时保持了较低的平均能耗。

     

移动边缘计算中基于内容流行度的深度强化学习缓存机制

随着5G商用的推进,涌现出大量依赖高速率、低时延的新应用,混合现实(Mixed Reality,MR)就是其中之一.考虑到从中心云传输服务内容到MR设备会带来很大时延和能耗问题,引入移动边缘计算(Mobile Edge Compu?ting,MEC)技术,通过在MEC服务器上缓存用户的预渲染环境帧,以减少延迟和能耗.针...     

Efficient QoS Provisioning for Adaptive Multimedia in Mobile Communication Networks by Reinforcement Learning

The scarcity and large fluctuations of link bandwidth in wireless networks have motivated the development of adaptive multimedia services in mobile communication networks, where it is possible to increase or decrease the bandwidth of individual ongoing flows. This paper studies the issues of quality of service (QoS) provisioning in such systems. In particular, call admission control and bandwidth adaptation are formulated as a constrained Markov decision problem. The rapid growth in the number of states and the difficulty in estimating state transition probabilities in practical systems make it very difficult to employ classical methods to find the optimal policy. We present a novel approach that uses a form of discounted reward reinforcement learning known as Q-learning to solve QoS provisioning for wireless adaptive multimedia. Q-learning does not require the explicit state transition model to solve the Markov decision problem; therefore more general and realistic assumptions can be applied to the underlying system model for this approach than in previous schemes. Moreover, the proposed scheme can efficiently handle the large state space and action set of the wireless adaptive multimedia QoS provisioning problem. Handoff dropping probability and average allocated bandwidth are considered as QoS constraints in our model and can be guaranteed simultaneously. Simulation results demonstrate the effectiveness of the proposed scheme in adaptive multimedia mobile communication networks. This work is based in part on a paper presented at BroadNet's 04, San Jose, CA, Oct. 2004. Fei Yu received the M.S. degree in Computer Engineering from Beijing University of Posts and Telecommunications, P.R. China, in 1998, and the Ph.D. degree in Electrical Engineering from the University of British Columbia (UBC), Canada, in 2003. From 1998 to 1999, Dr. Yu was a system engineer at China Telecom, P.R. China, working on the planning, design and performance analysis of national SS7 and GSM networks. From 2002 to 2004, He was a research and development engineer at Ericsson Mobile Platforms, Sweden, where he worked on dual-mode UMTS/GPRS handsets. He is currently a postdoctoral research fellow at UBC. His research interests are quality of service, cross-layer design and mobility management in wireless networks. Vincent W.S. Wong (S'94-M'00) received the B.Sc. (with distinction) degree from the University of Manitoba, Winnipeg, MB, Canada, in 1994, the M.A.Sc. degree from the University of Waterloo, Waterloo, ON, Canada, in 1996, and the Ph.D. degree from the University of British Columbia (UBC), Vancouver, BC, Canada, in 2000, all in electrical engineering. From 2000 to 2001, he was a Systems Engineer at PMC-Sierra, Inc., Burnaby, BC. Since 2002, he has been with the Department of Electrical and Computer Engineering, UBC, where he is currently an Assistant Professor. His research interests are in wireless communications and networking. Dr. Wong received the Natural Science and Engineering Research Council (NSERC) postgraduate scholarship and the Fessenden Postgraduate Scholarship from Communications Research Centre, Industry Canada, during his graduate studies. Victor C.M. Leung received the B.A.Sc. (Hons.) degree in electrical engineering from the University of British Columbia (U.B.C.) in 1977, and was awarded the APEBC Gold Medal as the head of the graduating class in the Faculty of Applied Science. He attended graduate school at U.B.C. on a Natural Sciences and Engineering Research Council Postgraduate Scholarship and obtained the Ph.D. degree in electrical engineering in 1981. From 1981 to 1987, Dr. Leung was a Senior Member of Technical Staff at Microtel Pacific Research Ltd. (later renamed MPR Teltech Ltd.), specializing in the planning, design and analysis of satellite communication systems. He also held a part-time position as Visiting Assistant Professor at Simon Fraser University in 1986 and 1987. In 1988, he was a Lecturer in the Department of Electronics at the Chinese University of Hong Kong. He joined the Department of Electrical Engineering at U.B.C. in 1989, where he is a Professor, Associate Head of Graduate Affairs, holder of the TELUS Mobility Industrial Research Chair in Advanced Telecommunications Engineering, and a member of the Institute for Computing, Information and Cognitive Systems. His research interests are in the areas of architectural and protocol design and performance analysis for computer and telecommunication networks, with applications in satellite, mobile, personal communications and high speed networks. Dr. Leung is a Fellow of IEEE and a voting member of ACM. He is an editor of the IEEE Transactions on Wireless Communications, and an associate editor of the IEEE Transactions on Vehicular Technology. He has served on the technical program committees of numerous conferences, and is serving as the Technical Program Vice-Chair of IEEE WCNC 2005.     

一种基于深度增强学习的智能路由技术

随着网络规模的不断增大以及网络复杂度的不断提高,传统路由算法面对网络流量在时空分布上的剧烈波动难以兼顾计算复杂度和算法效率.近年来,随着软件定义网络和人工智能技术的兴起,基于机器学习的自动路由策略生成逐渐受到关注.本文提出一种基于深度增强学习的智能路由技术SmartPath,通过动态收集网络状态,使用深度增强学习自动生成路由策略,从而保证路由策略能够动态适应网络流量变化.实验结果表明,本文所提出的方案能够不依赖人工流量建模动态更新网络路由,在测试环境下比当前最优方案减少至少10%的平均端到端传输时延.