网络数据流异常检测研究
|
摘要
网络数据流是典型的时间序列。具有动态性、高维性、无限性等特点。数据流每时每刻在不断变化,传统的处理方法存在多种弊端。根据网络数据流特性,提出基于DT-KNN的网络数据流异常检测算法,在理论和实验两个方面验证其高效性。具体地,在KNN算法中引入累计距离的概念,用以预测待检测未知点的变化趋势,并在流式处理平台Storm上加以实现,最终对基于DT-KNN的网络数据流异常检测算法进行MATLAB仿真。仿真结果表明,基于DT-KNN的网络数据流异常检测算法具有更高的准确率和更好的时效性。
Network data stream is a typical time series, which is characterized by dynamics, high dimensionality, and infinity. The data flow is constantly changing every moment, and the traditional processing methods have many drawbacks. According to the characteristics of network data stream, a network data flow anomaly detection algorithm based on DT-KNN is proposed, which is proved to be efficient in both theoretical and experimental aspects. Specifically, the concept of cumulative distance is introduced in the KNN algorithm to predict the trend of the unknown point to be detected, and is implemented on the streaming platform Storm. Finally, MATLAB simulation is performed on the DT-KNN-based network data flow anomaly detection algorithm. The simulation results indicate that the DT-KNN based network data flow anomaly detection algorithm has higher accuracy and better timeliness.
Network data stream is a typical time series, which is characterized by dynamics, high dimensionality, and infinity. The data flow is constantly changing every moment, and the traditional processing methods have many drawbacks. According to the characteristics of network data stream, a network data flow anomaly detection algorithm based on DT-KNN is proposed, which is proved to be efficient in both theoretical and experimental aspects. Specifically, the concept of cumulative distance is introduced in the KNN algorithm to predict the trend of the unknown point to be detected, and is implemented on the streaming platform Storm. Finally, MATLAB simulation is performed on the DT-KNN-based network data flow anomaly detection algorithm. The simulation results indicate that the DT-KNN based network data flow anomaly detection algorithm has higher accuracy and better timeliness.
引文
[1]汪成亮,陆志坚,庞栩.一种数据流趋势分析方法的研究与应用[J].计算机系统应用,2010,19(01):152-156.WANG Cheng-liang,LU Zhi-jian,PANG Xu.Research and Application of a Data Stream Trend Analysis Method[J].Computer System Application,2010,19(01):152-156.
[2]Canbay Y,Sagiroglu S.A Hybrid Method for Intrusion Detection[C].Machine Learning and Applications(ICMLA),2015 IEEE 14th International Conference on,2015:156-161.
[3]Salama M A,Eid H F,Ramadan R A,et al.Hybrid Intelligent Intrusion Detection Scheme[M].Berlin:Soft Computing in Industrial Applications,2011:293-303.
[4]马可,李玲娟.分布式实时流数据聚类算法及其基于Storm的实现[J].南京邮电大学学报(自然科学版),2016,36(02):104-110.MA Ke,LI Ling-juan.Distributed Real-Time Stream Data Clustering Algorithm and Its Implementation Based on Storm[J].Journal of Nanjing University of Posts and Telecommunications(Natural Science Edition),2016,36(02):104-110.
[5]费金龙.基于云模型的网络异常流量检测[J].计算机工程,2017,43(01):178-182.FEI Jin-long.Network Anomaly Traffic Detection Based on Cloud Model[J].Computer Engineeri ng,2017,43(01):178-182.
[6]王铭鑫.一种SDN中基于熵值计算的异常流量检测方法[J].电信科学,2015,31(09):83-89.WANG Ming-xin.An Abnormal Traffic Detection Method Based on Entropy Calculation in SDN[J].Telecommunications Science,2015,31(09):83-89.
[7]Sahu S,Mehtre B M.Network Intrusion Detection System Using J48 Decision Tree[C].Advances in Computing,Communications and Informatics(ICACCI),2015International Conference on,2015:2023-2026.
[8]Tran D,Mac H,Tong V,et al.A LSTM Based Framework for Handling Multiclass Imbalance in DGA Botnet Detection[J].Neurocomputing,2018(275):2401-2413.
[9]Ashfaq R A R,Wang X Z,Huang J Z,et al.Fuzziness Based Semi-supervised Learning Approach for Intrusion Detection System[J].Information Sciences,2017(378):484-497.
[10]Paulauskas N,Auskalnis J.Analysis of Data Preprocessing influence on Intrusion Detection Using NSL-KDD Dataset[C].Electrical,Electronic and Information Sciences(eStream),2017 Open Conference of IEEE,2017:1-5.
[11]庞景月.滑动窗口模型下的数据流自适应异常检测方法研究[D].哈尔滨:哈尔滨工业大学,2013.PANG Jing-yue.Study on Data Flow Adaptive Anomaly Detection Method under Sliding Window Model[D].Harbin:Harbin Institute of Technology,2013.
[2]Canbay Y,Sagiroglu S.A Hybrid Method for Intrusion Detection[C].Machine Learning and Applications(ICMLA),2015 IEEE 14th International Conference on,2015:156-161.
[3]Salama M A,Eid H F,Ramadan R A,et al.Hybrid Intelligent Intrusion Detection Scheme[M].Berlin:Soft Computing in Industrial Applications,2011:293-303.
[4]马可,李玲娟.分布式实时流数据聚类算法及其基于Storm的实现[J].南京邮电大学学报(自然科学版),2016,36(02):104-110.MA Ke,LI Ling-juan.Distributed Real-Time Stream Data Clustering Algorithm and Its Implementation Based on Storm[J].Journal of Nanjing University of Posts and Telecommunications(Natural Science Edition),2016,36(02):104-110.
[5]费金龙.基于云模型的网络异常流量检测[J].计算机工程,2017,43(01):178-182.FEI Jin-long.Network Anomaly Traffic Detection Based on Cloud Model[J].Computer Engineeri ng,2017,43(01):178-182.
[6]王铭鑫.一种SDN中基于熵值计算的异常流量检测方法[J].电信科学,2015,31(09):83-89.WANG Ming-xin.An Abnormal Traffic Detection Method Based on Entropy Calculation in SDN[J].Telecommunications Science,2015,31(09):83-89.
[7]Sahu S,Mehtre B M.Network Intrusion Detection System Using J48 Decision Tree[C].Advances in Computing,Communications and Informatics(ICACCI),2015International Conference on,2015:2023-2026.
[8]Tran D,Mac H,Tong V,et al.A LSTM Based Framework for Handling Multiclass Imbalance in DGA Botnet Detection[J].Neurocomputing,2018(275):2401-2413.
[9]Ashfaq R A R,Wang X Z,Huang J Z,et al.Fuzziness Based Semi-supervised Learning Approach for Intrusion Detection System[J].Information Sciences,2017(378):484-497.
[10]Paulauskas N,Auskalnis J.Analysis of Data Preprocessing influence on Intrusion Detection Using NSL-KDD Dataset[C].Electrical,Electronic and Information Sciences(eStream),2017 Open Conference of IEEE,2017:1-5.
[11]庞景月.滑动窗口模型下的数据流自适应异常检测方法研究[D].哈尔滨:哈尔滨工业大学,2013.PANG Jing-yue.Study on Data Flow Adaptive Anomaly Detection Method under Sliding Window Model[D].Harbin:Harbin Institute of Technology,2013.