低时延传输的ERDQN数据调度算法

针对车载网络、远程医疗、工业控制等领域需要低时延、高可靠性的网络传输应用场景,提出了一种经验回放的DQN(experience replay DQN,ERDQN)数据传输调度算法。该算法的主要目的和任务是降低网络时延和提高网络传输的稳定性。ERDQN算法在最后期限感知的传输协议(deadline-aware transport protocol, DTP)的基础上优化了发送端的排队策略,充分考虑了数据块的优先级和截止日期(Deadline),将其作为计算进入等待队列顺序的重要因素,解决了数据块丢失Deadline的问题,降低了网络传输的排队延迟;同时在拥塞控制方面以当前时刻网络传输状态为特征向量,预测下一时刻网络传输状态参数,并赋予不同的奖励因子进行评估,通过ERDQN网络的迭代学习,自动调整到适合当前网络传输的最优参数,在后续的网络链路传输过程中,平均传输速率高且稳定,缓解了网络拥塞和传输不稳定的问题,降低了网络传输时延。实验结果表明ERDQN算法的平均排队时延和传输时延远远低于传统拥塞控制算法(Reno算法),在质量系数(quality of experience, QoE)方面远远高于传统的拥塞控制算法,能够最大程度减少网络传输速率波动、降低丢包率,提供稳定可靠的传输。

ERDQN data scheduling algorithm for low latency transmission

An experience replay DQN (ERDQN) data transmission scheduling algorithm was proposed for network transmission application scenarios that require low latency and high reliability in the fields of in-vehicle networks, telemedicine, and industrial control. The main purpose and task of this algorithm was to reduce network delay and improve the stability of network transmission. The ERDQN algorithm optimized the queuing strategy at the sender side based on the deadline-aware transport protocol (DTP), and gave fully consideration to the priority of data blocks and Deadline, taking them as important factors for calculating the order of entering the waiting queue, which avoided the problem of losing Deadline of data blocks and reduced the queuing delay of network transmission. Meanwhile, in the congestion control, the current network transmission state was used as the feature vector to predict the parameters of the next network transmission state and give different reward factors for evaluation. Through the iterative learning of the ERDQN network, the optimal parameters were automatically adjusted to fit the current network transmission. The average transmission rate was high and stable during the subsequent network link transmission process, which alleviated the problems of network congestion and transmission stability and reduced the network transmission delay. Experimental results show that the average queuing delay and transmission delay of the ERDQN algorithm were much lower than those of the traditional congestion control algorithm (Reno algorithm), and much higher than the traditional congestion control algorithm in terms of quality of experience (QoE), which could minimize the network transmission rate fluctuation, reduce the packet loss rate, and provide stable and reliable transmission.