基于全注意力机制的多步网络流量预测
|
摘要
预测资源分配能有效利用无线网络的剩余资源服务非实时业务,其中的关键问题之一是剩余资源的预测,可转化为实时业务流量预测问题。本文把面向自然语言处理提出的全注意力机制引入到时间序列预测问题中,预测未来分钟级时间窗内秒级的流量,通过在每秒记录的实测流量数据集上进行训练和测试,与其他基于循环神经网络和线性、非线性预测模型的方法在复杂度(由训练和测试时间衡量)、预测精度(由平均相对百分比误差衡量)和预测误差统计特性(由预测误差的均值和标准差衡量)等方面进行比较。研究结果表明,与无注意力机制的循环神经网络相比,所设计的基于全注意力机制的方法计算复杂度低,由于多步预测的累积误差,在预测精度方面增益不明显。
Predictive resource allocation can harness residual resources in wireless networks to serve non-real-time service, where the prediction of residual resources plays a key role, which can be converted into the problem of predicting traffic load of real-time service. In this paper, a new neural network structure proposed for natural language processing, which is completely based on attention mechanism, is introduced to time sequence prediction, in particular traffic load prediction. By training and testing with a real traffic load dataset measured in each second, the multi-step prediction method with all-attention mechanism is compared with other methods using recurrent neural network(RNN) or linear and non-linear predictors, in terms of complexity(measured with training and testing time), prediction accuracy(measured with mean relative percentage error) and prediction error statistics(measured with the mean value and standard derivation of the prediction error). Simulation results show that the primary advantage of the designed neural network with all-attention mechanism over RNN without attention mechanism lies in low training and test complexity. Due to the accumulative errors in multi-step prediction, its gain in terms of prediction error is not obvious.
Predictive resource allocation can harness residual resources in wireless networks to serve non-real-time service, where the prediction of residual resources plays a key role, which can be converted into the problem of predicting traffic load of real-time service. In this paper, a new neural network structure proposed for natural language processing, which is completely based on attention mechanism, is introduced to time sequence prediction, in particular traffic load prediction. By training and testing with a real traffic load dataset measured in each second, the multi-step prediction method with all-attention mechanism is compared with other methods using recurrent neural network(RNN) or linear and non-linear predictors, in terms of complexity(measured with training and testing time), prediction accuracy(measured with mean relative percentage error) and prediction error statistics(measured with the mean value and standard derivation of the prediction error). Simulation results show that the primary advantage of the designed neural network with all-attention mechanism over RNN without attention mechanism lies in low training and test complexity. Due to the accumulative errors in multi-step prediction, its gain in terms of prediction error is not obvious.
引文
[1] Bui N,Widmer J.Data-driven Evaluation of Anticipatory Networking in LTE Network[J].IEEE Trans.Mobile Comput.,2018,17(10):2252-2265.
[2] Yao C,Yang C,Xiong Z.Energy-Saving Predictive Resource Planning and Allocation[J].IEEE Trans.Commun.,2016,64(12):5078-5095.
[3] Guo J,Yang C,Chih-Lin I.Exploiting Future Radio Resources with End-to-end Prediction by Deep Learning[J].IEEE Access,2018,6(1):75729-75747.
[4] Sadek N,Khotanzad A,Chen T.ATM Dynamic Bandwidth Allocation Using FARIMA Prediction Model[C]//IEEE ICCCN 2013,2013:359-363.
[5] Zhang G P.Time Series Forecasting Using a Hybrid ARIMA and Neural Network Model[J].Neurocomputing,2003,50(none):159-175.
[6] 高茜,李广侠,田湘,等.基于突发分离的自相似网络流量预测[J].信号处理,2012,28(2):158-165.Gao Q,Li G X,Tian X,et al.Self-similar Network Traffic Prediction Based on Burst Decomposition[J].Signal Processing,2012,28(2):158-165.(in Chinese)
[7] Xu Y,Xu W,Yin F,et al.High-Accuracy Wireless Traffic Prediction:A GP-Based Machine Learning Approach[C]//IEEE GLOBECOM 2017,2017:1- 6.
[8] Wang J,Tang J,Xu Z,et al.Spatiotemporal Modeling and Prediction in Cellular Networks:A Big Data Enabled Deep Learning Approach[C]//IEEE INFOCOM 2017,2017:1-9.
[9] Nie L,Jiang D,Yu S,et al.Network Traffic Prediction Based on Deep Belief Network in Wireless Mesh Backbone Networks[C]//IEEE WCNC 2017,2017:1-5.
[10] Li R,Zhao Z,Zheng J,et al.The Learning and Prediction of Application-Level Traffic Data in Cellular Networks[J].IEEE Trans.Wireless Commun.,2017,16(6):3899-3912.
[11] Vaswani A,Shazeer N,Parmar N,et al.Attention Is All You Need[C]//NIPS 2017,2017:5998- 6008.
[12] Neubig G.Neural Machine Translation and Sequence-to-sequence Models:A Tutorial[J].arXiv preprint arXiv:1703.01619,2017.
[13] Sutskever I,Vinyals O,Le Q V,et al.Sequence to Sequence Learning with Neural Networks[C]//NIPS 2014,2014:3104-3112.
[14] Zhang W,Liu T,Yang C.Trajectory Prediction with Recurrent Neural Networks For Predictive Resource Allocation[C]//IEEE ICSP 2018,2018:1- 6.
[15] Goodfellow I,Bengio Y,Courville A,et al.Deep learning[M].Cambridge:MIT Press,2016.
[16] Kingma D P,Ba J.Adam:A Method for Stochastic Optimization[J].arXiv preprint arXiv:1412.6980,2014.
[2] Yao C,Yang C,Xiong Z.Energy-Saving Predictive Resource Planning and Allocation[J].IEEE Trans.Commun.,2016,64(12):5078-5095.
[3] Guo J,Yang C,Chih-Lin I.Exploiting Future Radio Resources with End-to-end Prediction by Deep Learning[J].IEEE Access,2018,6(1):75729-75747.
[4] Sadek N,Khotanzad A,Chen T.ATM Dynamic Bandwidth Allocation Using FARIMA Prediction Model[C]//IEEE ICCCN 2013,2013:359-363.
[5] Zhang G P.Time Series Forecasting Using a Hybrid ARIMA and Neural Network Model[J].Neurocomputing,2003,50(none):159-175.
[6] 高茜,李广侠,田湘,等.基于突发分离的自相似网络流量预测[J].信号处理,2012,28(2):158-165.Gao Q,Li G X,Tian X,et al.Self-similar Network Traffic Prediction Based on Burst Decomposition[J].Signal Processing,2012,28(2):158-165.(in Chinese)
[7] Xu Y,Xu W,Yin F,et al.High-Accuracy Wireless Traffic Prediction:A GP-Based Machine Learning Approach[C]//IEEE GLOBECOM 2017,2017:1- 6.
[8] Wang J,Tang J,Xu Z,et al.Spatiotemporal Modeling and Prediction in Cellular Networks:A Big Data Enabled Deep Learning Approach[C]//IEEE INFOCOM 2017,2017:1-9.
[9] Nie L,Jiang D,Yu S,et al.Network Traffic Prediction Based on Deep Belief Network in Wireless Mesh Backbone Networks[C]//IEEE WCNC 2017,2017:1-5.
[10] Li R,Zhao Z,Zheng J,et al.The Learning and Prediction of Application-Level Traffic Data in Cellular Networks[J].IEEE Trans.Wireless Commun.,2017,16(6):3899-3912.
[11] Vaswani A,Shazeer N,Parmar N,et al.Attention Is All You Need[C]//NIPS 2017,2017:5998- 6008.
[12] Neubig G.Neural Machine Translation and Sequence-to-sequence Models:A Tutorial[J].arXiv preprint arXiv:1703.01619,2017.
[13] Sutskever I,Vinyals O,Le Q V,et al.Sequence to Sequence Learning with Neural Networks[C]//NIPS 2014,2014:3104-3112.
[14] Zhang W,Liu T,Yang C.Trajectory Prediction with Recurrent Neural Networks For Predictive Resource Allocation[C]//IEEE ICSP 2018,2018:1- 6.
[15] Goodfellow I,Bengio Y,Courville A,et al.Deep learning[M].Cambridge:MIT Press,2016.
[16] Kingma D P,Ba J.Adam:A Method for Stochastic Optimization[J].arXiv preprint arXiv:1412.6980,2014.