基于SAC强化学习的车联网频谱资源动态分配

针对车联网频谱资源稀缺问题，提出一种基于柔性致动-评价（SAC）强化学习算法的多智能体频谱资源动态分配方案。以最大化信道总容量与载荷成功交付率为目标，建立车辆-车辆（V2V）链路频谱资源分配模型。将每条V2V链路作为单个智能体，构建多智能体马尔科夫决策过程模型。利用SAC强化学习算法设计神经网络，通过最大化熵与累计奖励和以训练智能体，使得V2V链路经过不断学习优化频谱资源分配。仿真结果表明，与基于深度Q网络和深度确定性策略梯度的频谱资源分配方案相比，该方案可以更高效地完成车联网链路之间的频谱共享任务，且信道传输速率和载荷成功交付率更高。

Dynamic Spectrum Resource Allocation in Internet of Vehicles Based on SAC Reinforcement Learning

To address the scarcity of spectrum resources in Internet of Vehicles(IoV), a novel multi-agent dynamic spectrum allocation solution based on Soft Actor-Critic(SAC) reinforcement learning is proposed.The solution aims to maximize the total channel capacity and the success rate of payload delivery.To achieve this goal, a spectrum resource allocation model consisting of Vehicle-to-Vehicle(V2V) links is constructed.Each V2V link is regarded as an agent to model this problem as a Markov decision process.Then the SAC reinforcement learning algorithm is used to design a neural network.The agents are trained by maximum entropy and cumulative reward, so the V2V links can optimize the allocation of spectrum resources through rounds of learning.Simulation results show that compared with spectrum resource allocation scheme based on Deep Q-Network(DQN) and Deep Deterministic Policy Gradient(DDPG), the proposed scheme can more efficiently implement spectrum sharing between V2V links, and improves the channel transmission rate and the success rate of payload delivery.