基于LightGBM的网络入侵检测系统
|
摘要
入侵检测系统(intrusion detection system,IDS)是一种能够发现疑似入侵行为并采取相应措施的网络安全设备.现有IDS通常采用传统的常用机器学习算法和简单的深度学习算法,但始终难以避免训练速度慢、准确率不够高的缺点.针对这种情况,提出了一种基于LightGBM算法的网络入侵检测系统,对疑似入侵行为样本进行准确分类,该方法可以对数据进行采样从而极大地减小了数据计算量.使用入侵检测系统的标准数据集KDD99数据集,准确率达到94. 7%,训练时间缩短至422s.实验结果表明:基于LightGBM算法的网络入侵检测系统相较于常用算法在取得更高准确率的同时训练模型的速度也提高10倍左右.
Intrusion detection system(IDS) is one class of network security device which can discovered suspected intrusion and take corresponding measures for captured traffic which is suspected of intrusion. The existing IDS is usually based on traditional machine learning or simple deep learning algorithms. However, these are too slow in training phase and has not achieved the expected detetion rate. Under these circumstances, this paper proposes a network IDS based on LightGBM. This algorithm can sample the datas and features, which are much less computationally intensive. In this experiment, the accuracy rate reached 94. 72 % and the training time was shortened to 422 seconds with KDD99 dataset. The experimental results show that LightGBM is at least ten times faster than the common algorithm in training phase, while the accuracy rate ishigher than the existing algorithms.
Intrusion detection system(IDS) is one class of network security device which can discovered suspected intrusion and take corresponding measures for captured traffic which is suspected of intrusion. The existing IDS is usually based on traditional machine learning or simple deep learning algorithms. However, these are too slow in training phase and has not achieved the expected detetion rate. Under these circumstances, this paper proposes a network IDS based on LightGBM. This algorithm can sample the datas and features, which are much less computationally intensive. In this experiment, the accuracy rate reached 94. 72 % and the training time was shortened to 422 seconds with KDD99 dataset. The experimental results show that LightGBM is at least ten times faster than the common algorithm in training phase, while the accuracy rate ishigher than the existing algorithms.
引文
[1] Kenkre P S, Pai A,Coltco L.Real time intrusion detection and prevention system[C]//Proc of the 3rd int Conf on Frontierl of Intelligent Computing:Theory and Applications(FICIA)2014. Berlin:Springer, 2015:405-411
[2] Anderson J P. Computer security threat monitoring and surveillance[R]. Washington:James P Anderson Company,1980
[3] Bai Yuebin, Kobayashi H, Intrusion detection systems:Technology and development[C]//Proc of the 17th Int Conf on Advanced Information Networking and Applications(AINA 2003). Piscataway, NJ:IEEE, 2003:710-715
[4] Sabhnani M,Serpen G. Application of machine learning algorithms to KDD intrusion detection dataset within misuse detection context[C]//Proc of Int Conf on Machine Learning; Models, Technologies and Applications. Las Vegas, Nevada:DBLP, 2003:209-215
[5] Mahmood H A. Network intrusion setection system(NIDS)in cloud environment based on hidden Naive Bayes multiclass classifier[J]. Al-Mustansiriyah Journal of Science, 2018, 28(2):134-142
[6] Nguyen S N,Nguyen V Q, Choi J, et al. Design and implementation of intrusion detection system using convolutional neural network for DoS detection[C]//Proc of the 2nd Int Conf on Machine Learning and Soft Computing. New York:ACM, 2018:34-38
[7] Mukkamala S,Janoski G,Sung A. Intrusion detection:Support vector machines and neural networks[C]//Proc of the IEEE Int Joint Conf on Neural Networks(ANNIE).Piscataway, NJ:IEEE, 2002:1702-1707
[8] Osuna E, Freund R, Girosit F. Training support vector machines:An application to face detection[C]//Proc of1997 IEEE Computer Society Conf on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 1997:130-136
[9] Zhang Chunlin, Jiang Ju, Kamel M. Intrusion detection using hierarchical neural networks[J]. Pattern Recognition Letters, 2005, 26(6):779-791
[10] Liu Yuchen, Liu Shengli, Zhao Xing. Intrusion detection algorithm based on convolutional neural network[J].DEStech Transaction of Beijing Institute of Technology,2017, 37(12):1271-1275
[11] Aslahi-Shahri B M, Rahmani R, Chizari M,et al. A hybrid method consisting of GA and SVM for intrusion detection system[J].Neural Computing and Applications,2016, 27(6):1669-1676
[12] Ke Guolin,Meng Qin, Finley T, et al. Lightgbm:A highly efficient gradient boosting decision tree[C]//Advances in Neural Information Processing Systems, Long Beach, CA:NIPS, 2017:3146-3154
[13] Jensen T R, Toft B. Graph Coloring Problems[M].Hoboken, NJ:John Wiley&Sons, 2011
[14] Lee W, Stolfo S J. A framework for constructing features and models for intrusion detection systems[J]. ACM Trans on Information&System Security, 2000,3(4):227-261
[15] Wang Ming, Li Jian. Network intrusion detection model based on convolutional neural network[J]. Journal of Information Security Research, 2017, 3(11):990-994
[16] Moustafa N, Slay J. The evaluation of network anomaly detection systems:Statistical analysis of the UNSW-NB15data set and the comparison with the KDD99 data set[J].Information Security Journal:A Global Perspective, 2016,25(1/2/3):18-31
[17] Polikar R. Ensemble Learning[M]//Ensemble Machine Learning. Berlin:Springer, 2012:1-34
[18] Lee K B,Goo H W. Quantitative image quality and histogram-based evaluations of an iterative reconstruction algorithm at low-to-ultralow radiation dose levels:A phantom study in chest CT[J]. Korean Journal of Radiology, 2018, 19(1):119-129
[19] Cheong S, Oh S H, Lee S Y. Support vector machines with binary tree architecture for multi-class classification[J]. Neural Information Processing-Letters and Reviews,2004, 2(3):47-51
[20]高妮,贺毅岳,高岭.海量数据环境下用于入侵检测的深度学习方法[J].计算机应用研究,2018, 35(3):1197-1200
[2] Anderson J P. Computer security threat monitoring and surveillance[R]. Washington:James P Anderson Company,1980
[3] Bai Yuebin, Kobayashi H, Intrusion detection systems:Technology and development[C]//Proc of the 17th Int Conf on Advanced Information Networking and Applications(AINA 2003). Piscataway, NJ:IEEE, 2003:710-715
[4] Sabhnani M,Serpen G. Application of machine learning algorithms to KDD intrusion detection dataset within misuse detection context[C]//Proc of Int Conf on Machine Learning; Models, Technologies and Applications. Las Vegas, Nevada:DBLP, 2003:209-215
[5] Mahmood H A. Network intrusion setection system(NIDS)in cloud environment based on hidden Naive Bayes multiclass classifier[J]. Al-Mustansiriyah Journal of Science, 2018, 28(2):134-142
[6] Nguyen S N,Nguyen V Q, Choi J, et al. Design and implementation of intrusion detection system using convolutional neural network for DoS detection[C]//Proc of the 2nd Int Conf on Machine Learning and Soft Computing. New York:ACM, 2018:34-38
[7] Mukkamala S,Janoski G,Sung A. Intrusion detection:Support vector machines and neural networks[C]//Proc of the IEEE Int Joint Conf on Neural Networks(ANNIE).Piscataway, NJ:IEEE, 2002:1702-1707
[8] Osuna E, Freund R, Girosit F. Training support vector machines:An application to face detection[C]//Proc of1997 IEEE Computer Society Conf on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 1997:130-136
[9] Zhang Chunlin, Jiang Ju, Kamel M. Intrusion detection using hierarchical neural networks[J]. Pattern Recognition Letters, 2005, 26(6):779-791
[10] Liu Yuchen, Liu Shengli, Zhao Xing. Intrusion detection algorithm based on convolutional neural network[J].DEStech Transaction of Beijing Institute of Technology,2017, 37(12):1271-1275
[11] Aslahi-Shahri B M, Rahmani R, Chizari M,et al. A hybrid method consisting of GA and SVM for intrusion detection system[J].Neural Computing and Applications,2016, 27(6):1669-1676
[12] Ke Guolin,Meng Qin, Finley T, et al. Lightgbm:A highly efficient gradient boosting decision tree[C]//Advances in Neural Information Processing Systems, Long Beach, CA:NIPS, 2017:3146-3154
[13] Jensen T R, Toft B. Graph Coloring Problems[M].Hoboken, NJ:John Wiley&Sons, 2011
[14] Lee W, Stolfo S J. A framework for constructing features and models for intrusion detection systems[J]. ACM Trans on Information&System Security, 2000,3(4):227-261
[15] Wang Ming, Li Jian. Network intrusion detection model based on convolutional neural network[J]. Journal of Information Security Research, 2017, 3(11):990-994
[16] Moustafa N, Slay J. The evaluation of network anomaly detection systems:Statistical analysis of the UNSW-NB15data set and the comparison with the KDD99 data set[J].Information Security Journal:A Global Perspective, 2016,25(1/2/3):18-31
[17] Polikar R. Ensemble Learning[M]//Ensemble Machine Learning. Berlin:Springer, 2012:1-34
[18] Lee K B,Goo H W. Quantitative image quality and histogram-based evaluations of an iterative reconstruction algorithm at low-to-ultralow radiation dose levels:A phantom study in chest CT[J]. Korean Journal of Radiology, 2018, 19(1):119-129
[19] Cheong S, Oh S H, Lee S Y. Support vector machines with binary tree architecture for multi-class classification[J]. Neural Information Processing-Letters and Reviews,2004, 2(3):47-51
[20]高妮,贺毅岳,高岭.海量数据环境下用于入侵检测的深度学习方法[J].计算机应用研究,2018, 35(3):1197-1200