一种采用联邦深度强化学习的车联网资源分配方法

在车联网中，为了充分利用可用资源，车到车(Vehicle to Vehicle,V2V)链路需要动态地复用固定分配给车到基础设施(Vehicle to Infrastructure,V2I)链路的信道。传统的集中式信道资源分配方法会产生较大的通信开销，也难以适应转瞬即逝的车辆环境。为此，提出了一种基于分布式联邦深度强化学习(Federated Deep Reinforcement Learning,FDRL)的信道资源分配方法。首先，所有V2V智能体基于局部观察的环境信息独立地训练自己的模型，但彼此间保持相同的奖励以激励它们相互协作进而达成全局最优方案；然后，这些V2V智能体通过基站的帮助聚合部分模型参数，以增加接入公平性并加快模型学习效率。通过上述两阶段的迭代训练，每个V2V智能体训练出独特的决斗深度神经网络信道接入决策模型。仿真结果表明，所提出的FDRL方法与现有的优化方法相比具有更高的V2I链路总容量和V2V链路传输成功率。

A Resource Allocation Method Using Federated Deep Reinforcement Learning in Vehicular Networks

In vehicular networks,to fully utilize the available resources,Vehicle to Vehicle(V2V) links need to dynamically multiplex channels that are fixedly allocated to Vehicle to Infrastructure(V2I) links.Traditional centralized channel resource allocation methods incur large communication overhead and are difficult to adapt to the rapidly changing vehicular environment.To this end,a federated deep reinforcement learning(FDRL) based channel resource allocation method is proposed.First,all V2V agents train their models independently based on locally observed environmental information,but keep the same reward among themselves to motivate them to collaborate with each other to reach a globally optimal solution.Then,these V2V agents aggregate some model parameters with the help of the base station to increase access fairness and accelerate model learning efficiency.Through the above two stages of iterative training,a unique dueling deep Q network channel access decision model is trained for each V2V agent.Simulation results show that the proposed FDRL method has a higher sum V2I links’ capacity and V2V links’ transmission success rate compared with existing optimization methods.