面向边缘设备的高能效深度学习任务调度策略

近年来,深度学习在图像和自然语言处理等诸多领域表现出色,与深度学习相关的各类移动应用发展迅速,但由于移动网络状态的不稳定性及网络带宽的限制,基于云计算的深度模型任务可能出现较大响应延迟,严重影响用户体验.与此同时,深度模型对设备的计算及存储能力有较高的要求,无法直接在资源受限的移动设备中进行部署.因此,亟须设计一种新的计算模式,使得基于深度模型的移动应用能够满足用户对快速响应、低能耗及高准确率的期望.本文提出一种面向边缘设备的深度模型分类任务调度策略,该策略通过协同移动设备与边缘服务器,充分利用智能移动终端的便捷性和边缘服务器强大的计算能力,综合考虑分类任务的复杂度和用户期望,完成深度模型在移动设备和边缘服务器中的动态部署,并对推理任务进行动态调度,从而提升任务执行效率,降低深度学习模型推理开销.本文以基于卷积神经网络的图像识别应用为例,实验结果表明,在移动环境中,相比于准确率最高的深度模型,本文提出的高能效调度策略的推理能耗可降低93.2%、推理时间降低91.6%,同时准确率提升3.88%.

Energy-Efficient Deep Learning Task Scheduling Strategy for Edge Device

The deep neural network has made significant progress in many fields.Its powerful computing ability makes it an efficient tool to solve complex problems,and has been widely used in automatic driving,face recognition,and augmented reality.Due to the outstanding performance of deep learning in the fields of image recognition and natural language processing,applying the deep learning model on mobile application is inevitable.Typically,the deep learning model relies on high-performance servers equipped with strong computing processors and large storage.However,because of the unstable mobile networks and limited bandwidth,running deep learning on the cloud server may cause a response delay,which violates the quality of user experience,and running the inference task on the cloud also has the privacy problem.At the same time,the researcher tries to execute the inference task on the user’s own device,mainly focus on the on-device deep learning by using model compression techniques and develop the light-weight deep model,and all of them will sacrifice the model accuracy.Because of the limited resources of the mobile terminal(computing power,storage size,and battery capacity),the mobile device cannot satisfy the DNN model.We need to design a new computing paradigm so that the Deep Neural Network(DNN)based model can meet the user’s expectations for fast response,low energy consumption,and high accuracy.This paper proposes a novel scheduling strategy,Edge-based strategy,for deep learning inference tasks by using edge devices.The Edge-based strategy combines the mobility of the user’s mobile device with the powerful computing processors on edge server.Firstly,the strategy selects and deploys the appropriate DNN models by considering the inference time and accuracy.Specifically,the Edge-based strategy evaluates the candidate deep models on user mobile devices,and record the inference time and failure classification samples,the inference time is the first priority on mobile devices,then the strategy deploy the deep model with the least inference time on mobile devices,and input the failure sample to the other deep models and select the model with highest accuracy and deploy it on the edge device.After deploying the model on both devices,Edge-based strategy focuses on how to schedule the inference task between two devices to achieve the best performance.The core of task scheduling is the pre-trained classification model,it takes account of the input data complexity,and user expectations and schedule the inference task dynamically.This paper compares four typical machine learning techniques to train the classification model,and the random forest gives the highest accuracy.This paper takes the image recognition application as an example,and evaluate 12 popular CNN models on RaspberryPi3 B+,Jetson TX2 respectively,the experimental results show that in the mobile network environment,Edge-based strategy can effectively improve the performance of the deep model and reducing the overhead of inference,our approach outperforms the model with the highest accuracy by 93.2%,91.6%,and 3.88%for energy consumption,inference time and accuracy.