边缘智能:边缘计算驱动的深度学习加速技术
|
摘要
作为直接推动机器学习蓬勃发展的关键核心技术,深度学习已经迅速成为学术界与工业界关注的焦点。然而,由于深度学习模型的高精度需求往往会引发对计算资源的大量消耗,因此将一个深度学习模型部署到资源受限的移动设备面临着的巨大的挑战。本文介绍Edgent,一个基于边端协同的按需加速深度学习模型推理的优化框架,通过深度学习模型分割与模型精简实现加速。实验表明其能在网络边缘端高效支撑深度学习应用。
As a key core technology that directly promotes the flourishing of machine learning, deep learning has quickly become the focus of academic and industrial circles. However, because the high-precision requirements of deep learning models often lead to a large consumption of computing resources, deploying a deep learning model to resource-constrained mobile devices faces enormous challenges. This paper introduces Edgent, a device-edge synergy based optimization framework for ondemand deep learning model inference, which is accelerated by both model segmentation and model simplification. Experiments show that it can effectively support deep learning applications at the network edge.
As a key core technology that directly promotes the flourishing of machine learning, deep learning has quickly become the focus of academic and industrial circles. However, because the high-precision requirements of deep learning models often lead to a large consumption of computing resources, deploying a deep learning model to resource-constrained mobile devices faces enormous challenges. This paper introduces Edgent, a device-edge synergy based optimization framework for ondemand deep learning model inference, which is accelerated by both model segmentation and model simplification. Experiments show that it can effectively support deep learning applications at the network edge.
引文
[1] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]. Advances in neural information processing systems. 2012.
[2] China Telecom et al. 5G Mobile/Multi-Access Edge Computing[Z]. 2017.
[3] Iandola F N, Han S, Moskewicz M W, et al. Squeezenet:Alexnet-level accuracy with 50x fewer parameters and<0.5 mb model size[J]. ArXiv, 2016.
[4] Wu J, Leng C, Wang Y, et al. Quantized convolutional neural networks for mobile devices[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
[5] Han S, Mao H, Dally W J. Deep Compression:Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding[J]. Fiber. 2015.
[6] Kang Y, Hauswald J, Gao C, et al. Neurosurgeon:Collaborative Intelligence Between the Cloud and Mobile Edge[C].International Conference on Architectural Support for Programming Languages and Operating Systems. 2017.
[7] Kim, Yong-Deok, Park, Eunhyeok, Yoo, Sungjoo, et al. Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications[J]. Computer Science, 2015.
[8] Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images[R]. Technical report, University of Toronto,2009.
[9] Lane N, Bhattacharya S, Mathur A, et al. DXTK:Enabling Resource-efficient Deep Learning on Mobile and Embedded Devices with the DeepX Toolkit[C]. International Conference on Mobile Computing, Applications and Services. ICST(Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 2016.
[10] Oord A V D, Dieleman S, Zen H, et al. WaveNet:A Generative Model for Raw Audio[J]. ArXiv, 2016.
[11] Qualcomm. Augmented and Virtual Reality:the First Wave of 5G Killer Apps[Z]. 2017.
[12] Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
[13] Teerapittayanon S, McDanel B, Kung H T. Branchynet:Fast inference via early exiting from deep neural networks[C].International Conference on Pattern Recognition. 2016.
[14] Wang D, Nyberg E. A long short-term memory model for answer sentence selection in question answering[C].Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics(ACL)and the 7th International Joint Conference on Natural Language Processing(IJCNLP). 2015.
[15] En Li, Zhi Zhou, and Xu Chen. Edge Intelligence:On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy[C]. Proceedings of the Workshop on Mobile Edge Communications. 2018.
[2] China Telecom et al. 5G Mobile/Multi-Access Edge Computing[Z]. 2017.
[3] Iandola F N, Han S, Moskewicz M W, et al. Squeezenet:Alexnet-level accuracy with 50x fewer parameters and<0.5 mb model size[J]. ArXiv, 2016.
[4] Wu J, Leng C, Wang Y, et al. Quantized convolutional neural networks for mobile devices[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
[5] Han S, Mao H, Dally W J. Deep Compression:Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding[J]. Fiber. 2015.
[6] Kang Y, Hauswald J, Gao C, et al. Neurosurgeon:Collaborative Intelligence Between the Cloud and Mobile Edge[C].International Conference on Architectural Support for Programming Languages and Operating Systems. 2017.
[7] Kim, Yong-Deok, Park, Eunhyeok, Yoo, Sungjoo, et al. Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications[J]. Computer Science, 2015.
[8] Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images[R]. Technical report, University of Toronto,2009.
[9] Lane N, Bhattacharya S, Mathur A, et al. DXTK:Enabling Resource-efficient Deep Learning on Mobile and Embedded Devices with the DeepX Toolkit[C]. International Conference on Mobile Computing, Applications and Services. ICST(Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 2016.
[10] Oord A V D, Dieleman S, Zen H, et al. WaveNet:A Generative Model for Raw Audio[J]. ArXiv, 2016.
[11] Qualcomm. Augmented and Virtual Reality:the First Wave of 5G Killer Apps[Z]. 2017.
[12] Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
[13] Teerapittayanon S, McDanel B, Kung H T. Branchynet:Fast inference via early exiting from deep neural networks[C].International Conference on Pattern Recognition. 2016.
[14] Wang D, Nyberg E. A long short-term memory model for answer sentence selection in question answering[C].Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics(ACL)and the 7th International Joint Conference on Natural Language Processing(IJCNLP). 2015.
[15] En Li, Zhi Zhou, and Xu Chen. Edge Intelligence:On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy[C]. Proceedings of the Workshop on Mobile Edge Communications. 2018.