基于边缘计算的分支神经网络模型推断延迟优化

针对云服务器上深度神经网络（DNN）模型推断任务延迟过高的问题，提出基于边缘计算的分支神经网络部署模型。分析了边缘计算场景中深度神经网络的分布式部署问题，证明该问题是NP-难的。设计了一种基于分支定界思想的部署算法（DBB），选择合适的边缘计算节点部署模型以减少推断任务的延迟。设计并实现了选择节点退出（SNE）算法，为不同任务选择合适的边缘计算节点来退出推断任务。仿真实验结果表明，与在云端部署神经网络模型的方法相比，基于边缘计算的分支神经网络模型的推断延迟平均降低了36%。

Inference delay optimization of branchy neural network model based on edge computing

Aiming at the long delay of inference tasks in Deep Neural Network （DNN） on cloud servers， a branchy neural network deployment model based on edge computing was proposed. The distributed deployment problem of DNNs in edge computing scenarios was analyzed， and was proved to be NP-hard. A Deployment algorithm based on Branch and Bound （DBB） was designed to select appropriate edge computing nodes to reduce inference delay. And a Selection Node Exit （SNE） algorithm was designed and implemented to select the appropriate edge computing nodes for different tasks to exit the inference task. The simulation results show that， compared with the approach of deploying neural network model on the cloud， the branchy neural network model based on edge computing reduces the inference delay by 36% on average.