基于局域波分析的网络流量异常检测方法研究
|
摘要
近几年来,计算机网络发展异常迅猛。网络规模不断扩大,复杂性不断增加,网络的异构性越来越高。导致网络出现各种性能问题的可能性增大,而且传播得更为广泛,发现和诊断问题的难度增大。另一方面用户对网络服务性能提出了更高的要求,这些都增加了网络管理的难度。因此如何实时检测这些网络异常成为目前重要的研究课题。对网络流量进行实时监测和管理,及时地发现网络流量异常,对提高网络的可靠性和安全性具有重要意义。
本文首先分析了国内外网络流量异常检测的研究现状,指出了所存在的问题,并提出了新的研究思路。
局域波分解方法是一种新兴的信号时频分析方法,它吸收了小波变换的多分辨分析的优点,同时又克服了小波变换需要选取小波基和分解层数的缺点,实现了基于信号局部时变特征的自适应时频分解。但原有的EMD方法分解速度慢、边缘出现失真、筛选条件不严格和缺乏对伪分量的判断。我们在分析EMD方法的基础上提出了线性均值分解方法,它有效的克服了这些缺点。进一步的,我们分析了采样频率对该分解方法精度的影响。
流量模型是流量分析的重要组成部分。本文分析了多种流量模型,并以自相似流量模型为基础,提出了基于局域波分析的自相似参数(Hurst参数)的估计方法。该方法能够更准确地估计出网络流量的自相似程度。
针对现有的网络流量异常检测方法实时性差、精度低和不具有自适应性等缺点,基于对局域波方法的研究,提出了一种基于局域波分解的流量异常检测方法。该方法能够根据流量信号的特点,而自适应的调节分辨率,达到对流量信号的更准确的分析。实验表明该方法具有更高的准确性。
With the rapid development of network, the scale and complexity are increasing and the heterogeneous is much higher than before. On the other hand, users advance higher request about network service, which makes the management of networks become more difficult. Consequently, it is difficult to detect anomalies accurately in real-time network management, which becomes an important research problem. Traffic anomalies can significantly disrupt and degrade network service. Therefore, making real-time monitor and management and finding out anomaly for network traffic has significant meanings in improving both robust and security for network.
At first, this paper analyzed the research actuality both here and abroad. We pointed out the deficiency of present research and some new research clews.
Local wave decomposition method is a new time frequency analysis method. It has some advantages of wavelet decomposition and conquered the disadvantages that the wavelet decomposition need to select a wavelet base and level. Local wave decomposition method can adaptively decomposed base on the local time-varing characteristics. The speed of the EMD method is slow, signal edge have distortion. The condition of screening is not strict. The EMD method do not judge the pseudo component. We proposed a new decompositionmethod——Linear mean decomposition method. This method had conquered theabove disadvantages. In addition, we researched the influence of sampling frequency.
Traffic model is one important part of traffic analysis. This paper analyzed some models. We proposed a new method base on the local wave decomposition to calculate the Hurst parameter. This method can estimate the degree of traffic self similarity.
The old anomaly detection is slow. It's results is not accurate. It can not decomposed adaptively. According to the disadvantages of the old anomaly detection, we proposed a new method base on the local wave decomposition. This method can decomposed base on the characteristics of signal. So it can get the more accurate results. Experiments had proved this.
本文首先分析了国内外网络流量异常检测的研究现状,指出了所存在的问题,并提出了新的研究思路。
局域波分解方法是一种新兴的信号时频分析方法,它吸收了小波变换的多分辨分析的优点,同时又克服了小波变换需要选取小波基和分解层数的缺点,实现了基于信号局部时变特征的自适应时频分解。但原有的EMD方法分解速度慢、边缘出现失真、筛选条件不严格和缺乏对伪分量的判断。我们在分析EMD方法的基础上提出了线性均值分解方法,它有效的克服了这些缺点。进一步的,我们分析了采样频率对该分解方法精度的影响。
流量模型是流量分析的重要组成部分。本文分析了多种流量模型,并以自相似流量模型为基础,提出了基于局域波分析的自相似参数(Hurst参数)的估计方法。该方法能够更准确地估计出网络流量的自相似程度。
针对现有的网络流量异常检测方法实时性差、精度低和不具有自适应性等缺点,基于对局域波方法的研究,提出了一种基于局域波分解的流量异常检测方法。该方法能够根据流量信号的特点,而自适应的调节分辨率,达到对流量信号的更准确的分析。实验表明该方法具有更高的准确性。
With the rapid development of network, the scale and complexity are increasing and the heterogeneous is much higher than before. On the other hand, users advance higher request about network service, which makes the management of networks become more difficult. Consequently, it is difficult to detect anomalies accurately in real-time network management, which becomes an important research problem. Traffic anomalies can significantly disrupt and degrade network service. Therefore, making real-time monitor and management and finding out anomaly for network traffic has significant meanings in improving both robust and security for network.
At first, this paper analyzed the research actuality both here and abroad. We pointed out the deficiency of present research and some new research clews.
Local wave decomposition method is a new time frequency analysis method. It has some advantages of wavelet decomposition and conquered the disadvantages that the wavelet decomposition need to select a wavelet base and level. Local wave decomposition method can adaptively decomposed base on the local time-varing characteristics. The speed of the EMD method is slow, signal edge have distortion. The condition of screening is not strict. The EMD method do not judge the pseudo component. We proposed a new decompositionmethod——Linear mean decomposition method. This method had conquered theabove disadvantages. In addition, we researched the influence of sampling frequency.
Traffic model is one important part of traffic analysis. This paper analyzed some models. We proposed a new method base on the local wave decomposition to calculate the Hurst parameter. This method can estimate the degree of traffic self similarity.
The old anomaly detection is slow. It's results is not accurate. It can not decomposed adaptively. According to the disadvantages of the old anomaly detection, we proposed a new method base on the local wave decomposition. This method can decomposed base on the characteristics of signal. So it can get the more accurate results. Experiments had proved this.
引文
[1]Lingsong Zhang,Singular Value.Decomposition with application on Network Traffic Modeling http://www-dirt.cs.unc.edu/netdata/svdp.pdf,Aug 13,2004.1-6P
[2]Chen-Mou Cheng,H.T.Kung and Koan-Sin Tan.Use of Spectral Analysis in Defense Against Dos Attacks.Proceedings of IEEE GIOBECOM 2002,Taiwan:Taipei,2002,243-248P
[3]V.Alarcon-Aquino,J.A.Barria.Anomaly Detection in Communication Networks Using Wavelets.IEEE Proc-Commun.Vol.148.No.6.December 2001,355-362P
[4]Anu Ramanathan.A Tool for Distribute Denial of Service Attack Detection.TAMU-ECU-2002-02,Master of Science Thesis,August 2002,20-26P
[5]P.Barford,J.Kline,D.Plonka,and A.Ron.A signal analysis of network traffic anomalies.In Internet Measurement Workshop,2002,71-82P
[6]Seong Soo Kim,A.L.Narasimha Reddy.Detecting Traffic Anomalies at the Source through aggregate analysis of packet header data.http://dropzone.tamu.edu/techpubs/2003/TAMU-ECU-2003-03.pdf 1-6P
[7]Lan Li and Gyungbo Lee.DDoS Attack Detection and Wavelets.Computer Communications and Networks,2003.ICCCN 2003.Proceedings.The 12~(th) International Conference on,20-22 Oct.2003,421-427P
[8]孙钦东,张德运,郑卫斌,胡国栋.基于时频分析的分布式拒绝服务攻击的自动监测.西安交通大学学报 2004 25页-30页
[9]A.Lakhina,M.Crovella,and C.Diot.Diagnosing Network-Wide Traffic Anomalies.In ACM SIGCOMM,Portland,August 2004, 219-230P
[10]A.Lakina,K.Papagiannaki,M.Crovella,C.Diot,E.D.Kolaczyk,and N.Taft.Structural Analysis of Network Traffic Flows.In ACM SIGMETRICS,New York,June 2004,61-72P
[11]A.Lakhina,M.Crovella.Amd C.Diot.Characterization of Network-Wide Anomalies in Traffic Flows.Technical Report BUCS-2004.1-5P
[12]M.Li,W.Jia,and W.Zhao.Decision analysis of network-based intrusion detection systems for denial-of-service attacks.In Proceedings,IEEE Conference on Info-tech and Info-net,200178-83P
[13]向渝.IP网络QoS和安全技术研究.电子科技大学博士论文.2003年,35页-36页
[14]W.Yah,E.Hou,and N.Ansari.Anomaly detection and traffic shaping under self-similar aggregated traffic in optical switched networks.In Proceeding of ICCT,2003,378-381P
[15]I.Katzela and M.Schwarz.Schemes for fault identification in communication networks.IEEE/ACM Trans.Networking,1995.vol.3,753-764P
[16]F.Feather and R.Maxion.Fault detection in an Ethernet network using anomaly signature matching.In Proc.ACM SIGCOMM,vol.23,SanFrancisco,CA,Sept.1993,279-288P
[17]S.Papavassiliou,M.Pace,A.Zawadzki,and L.Ho.Implementing enhanced network maintenance for transaction access services:Tools and applications.Proc.IEEE Int.Contr.Conf,2000,vol.1,211-215P
[18]温志贤,李小勇.基于支持向量机的网络流量异常检测.西北师范大学学报(自然科学版),2005.3:32页-36页
[19]第文军,薛丽军,将士奇.运用网络流量自相似分析的网络流量异常检测.兵工自动化,2003.6:31页-34页
[20]王欣,方滨兴.Hurst参数变化在网络流量异常检测中的应用.哈尔滨工业大学学报,2003.8:30页-33页
[21]应明,李建华,铁玲.基于条件规则库的流量异常检测系统设计.2003,10:108页-110页
[22]邹柏贤.一种网络异常实时检测方法.计算机学报,2003.8:45页-52页
[23]Shan Rongsheng,Li Jianhua,Wang Mingzheng.Anomaly detection for network traffic flow.Journal of Southeast University,2004.01:18-22P
[24]Huang N E,Shen Z,Long S R et al.The empirical mode decomposition and Hilbert spectrum for nonlinear and non-stationary time series analysis.Proc.Roy.Soc.London A,1998,454:904-995P
[25]Melvile W K.Wave modulation and breakdown.Journal of Fluid Mech.1983,128:489-506P
[26]盖强.局域波时频分析的理论与应用:(博士学位论文).大连:大连理工大学,2001.20页-24页
[27]Michael Jiang,Analysis of Wireless Data Network Traffic.http://www.ensc.sfu.calpeoplefaculty/ljiljalcnl/presentation s/michael/michael_thesis/index.htm.2002.1-5P
[28]S.F.Victor,M.Benjamin.Traffic modeling for telecommunications network.IEEE communications magazine,1994.1-6P
[29]Mandelbrot B,Ness JV.Fractional Brownian Motions,Fractional Noises and Applications.1995,15-20P
[30]Leland W.E.,Taqqu M.S.,Willinger W.On the self-similar nature of Ethernet traffic(extended version),IEEE/ACM Transactions on Networking,Volume 2,Issue 1,Feb.1994,1-15P
[31]Paxon,Flouds.Wide area traffic.The Failure of Poisson Modeling[J].IEEE/ACM Transactions on Networking,1995,226-244P
[32]Klivansky,Samukher jee,Csong.On Long Range Dependenceln NSFNET Traffic http://citeseer.ist.pus.edu/klivanskylongrange.html,1994,1-6P
[33]Jan Beran,Robert Sherman,Tagqu M.S.,Willinger.Long Range Dependence in Variable-Bit-Rate Video Traffic.IEEE Transactions On Communications,1995,43:2-4P
[34]Taqqu M.S.,Teverovsky V.,Willinger W.Estimators for Long Range Dependence:an Empirical Study.Fractals,1995,3(4):785-798P
[35]B.B.Mandelbrot and M.S.Taqqu.Roburst R/S Analysis of Long-run serial correlation.In Proceedings of the 42~(nd) session of the International Statistical Institute,Manila,1979,48:69-104P
[36]R.Fox and M.S.Taqqu.Large-Sample Properties of Parameter Estimates for Strongly Dependent Stationary Gaussian Time Series:The Annals of Statistics,1986,14:517-532P
[37]P.Abry and D.Veitch.Wavelet Analysis of Long-Range-Dependent traffic.IEEE/ACM Transactions on Networking,1998,44(1):2-15P
[38]P.Abry and D.Veitch.A Wavelet Based Joint Estimator of Parameters of Long-Range Dependence.IEEE/ACM Transactions on Information Theory,1999,43(1):16-24P
[39]P.Abry,L.Delbeke and P.Flandrin.Wavelet-Based Estimator for the Self-Similarity Parameter of a Stable Process.IEEE-ICASSP-99,Phoenix,1999.125-130P
[40]B.Mandelbrot.Self-Similar Error Clusters in Communication Systems and the Concept of Conditional Stationarity.IEEE Transaction on Communication Technology,1964,13(4):27-53P
[41]李浩.网络流量的自相似特性以及生成方法的研究:(硕士学位论文)长沙:国防科学技术大学.2004.68页-72页
[42]林原.基于网络自相似性的DDOS攻击检测.1998,7,16-21.72-73页
[2]Chen-Mou Cheng,H.T.Kung and Koan-Sin Tan.Use of Spectral Analysis in Defense Against Dos Attacks.Proceedings of IEEE GIOBECOM 2002,Taiwan:Taipei,2002,243-248P
[3]V.Alarcon-Aquino,J.A.Barria.Anomaly Detection in Communication Networks Using Wavelets.IEEE Proc-Commun.Vol.148.No.6.December 2001,355-362P
[4]Anu Ramanathan.A Tool for Distribute Denial of Service Attack Detection.TAMU-ECU-2002-02,Master of Science Thesis,August 2002,20-26P
[5]P.Barford,J.Kline,D.Plonka,and A.Ron.A signal analysis of network traffic anomalies.In Internet Measurement Workshop,2002,71-82P
[6]Seong Soo Kim,A.L.Narasimha Reddy.Detecting Traffic Anomalies at the Source through aggregate analysis of packet header data.http://dropzone.tamu.edu/techpubs/2003/TAMU-ECU-2003-03.pdf 1-6P
[7]Lan Li and Gyungbo Lee.DDoS Attack Detection and Wavelets.Computer Communications and Networks,2003.ICCCN 2003.Proceedings.The 12~(th) International Conference on,20-22 Oct.2003,421-427P
[8]孙钦东,张德运,郑卫斌,胡国栋.基于时频分析的分布式拒绝服务攻击的自动监测.西安交通大学学报 2004 25页-30页
[9]A.Lakhina,M.Crovella,and C.Diot.Diagnosing Network-Wide Traffic Anomalies.In ACM SIGCOMM,Portland,August 2004, 219-230P
[10]A.Lakina,K.Papagiannaki,M.Crovella,C.Diot,E.D.Kolaczyk,and N.Taft.Structural Analysis of Network Traffic Flows.In ACM SIGMETRICS,New York,June 2004,61-72P
[11]A.Lakhina,M.Crovella.Amd C.Diot.Characterization of Network-Wide Anomalies in Traffic Flows.Technical Report BUCS-2004.1-5P
[12]M.Li,W.Jia,and W.Zhao.Decision analysis of network-based intrusion detection systems for denial-of-service attacks.In Proceedings,IEEE Conference on Info-tech and Info-net,200178-83P
[13]向渝.IP网络QoS和安全技术研究.电子科技大学博士论文.2003年,35页-36页
[14]W.Yah,E.Hou,and N.Ansari.Anomaly detection and traffic shaping under self-similar aggregated traffic in optical switched networks.In Proceeding of ICCT,2003,378-381P
[15]I.Katzela and M.Schwarz.Schemes for fault identification in communication networks.IEEE/ACM Trans.Networking,1995.vol.3,753-764P
[16]F.Feather and R.Maxion.Fault detection in an Ethernet network using anomaly signature matching.In Proc.ACM SIGCOMM,vol.23,SanFrancisco,CA,Sept.1993,279-288P
[17]S.Papavassiliou,M.Pace,A.Zawadzki,and L.Ho.Implementing enhanced network maintenance for transaction access services:Tools and applications.Proc.IEEE Int.Contr.Conf,2000,vol.1,211-215P
[18]温志贤,李小勇.基于支持向量机的网络流量异常检测.西北师范大学学报(自然科学版),2005.3:32页-36页
[19]第文军,薛丽军,将士奇.运用网络流量自相似分析的网络流量异常检测.兵工自动化,2003.6:31页-34页
[20]王欣,方滨兴.Hurst参数变化在网络流量异常检测中的应用.哈尔滨工业大学学报,2003.8:30页-33页
[21]应明,李建华,铁玲.基于条件规则库的流量异常检测系统设计.2003,10:108页-110页
[22]邹柏贤.一种网络异常实时检测方法.计算机学报,2003.8:45页-52页
[23]Shan Rongsheng,Li Jianhua,Wang Mingzheng.Anomaly detection for network traffic flow.Journal of Southeast University,2004.01:18-22P
[24]Huang N E,Shen Z,Long S R et al.The empirical mode decomposition and Hilbert spectrum for nonlinear and non-stationary time series analysis.Proc.Roy.Soc.London A,1998,454:904-995P
[25]Melvile W K.Wave modulation and breakdown.Journal of Fluid Mech.1983,128:489-506P
[26]盖强.局域波时频分析的理论与应用:(博士学位论文).大连:大连理工大学,2001.20页-24页
[27]Michael Jiang,Analysis of Wireless Data Network Traffic.http://www.ensc.sfu.calpeoplefaculty/ljiljalcnl/presentation s/michael/michael_thesis/index.htm.2002.1-5P
[28]S.F.Victor,M.Benjamin.Traffic modeling for telecommunications network.IEEE communications magazine,1994.1-6P
[29]Mandelbrot B,Ness JV.Fractional Brownian Motions,Fractional Noises and Applications.1995,15-20P
[30]Leland W.E.,Taqqu M.S.,Willinger W.On the self-similar nature of Ethernet traffic(extended version),IEEE/ACM Transactions on Networking,Volume 2,Issue 1,Feb.1994,1-15P
[31]Paxon,Flouds.Wide area traffic.The Failure of Poisson Modeling[J].IEEE/ACM Transactions on Networking,1995,226-244P
[32]Klivansky,Samukher jee,Csong.On Long Range Dependenceln NSFNET Traffic http://citeseer.ist.pus.edu/klivanskylongrange.html,1994,1-6P
[33]Jan Beran,Robert Sherman,Tagqu M.S.,Willinger.Long Range Dependence in Variable-Bit-Rate Video Traffic.IEEE Transactions On Communications,1995,43:2-4P
[34]Taqqu M.S.,Teverovsky V.,Willinger W.Estimators for Long Range Dependence:an Empirical Study.Fractals,1995,3(4):785-798P
[35]B.B.Mandelbrot and M.S.Taqqu.Roburst R/S Analysis of Long-run serial correlation.In Proceedings of the 42~(nd) session of the International Statistical Institute,Manila,1979,48:69-104P
[36]R.Fox and M.S.Taqqu.Large-Sample Properties of Parameter Estimates for Strongly Dependent Stationary Gaussian Time Series:The Annals of Statistics,1986,14:517-532P
[37]P.Abry and D.Veitch.Wavelet Analysis of Long-Range-Dependent traffic.IEEE/ACM Transactions on Networking,1998,44(1):2-15P
[38]P.Abry and D.Veitch.A Wavelet Based Joint Estimator of Parameters of Long-Range Dependence.IEEE/ACM Transactions on Information Theory,1999,43(1):16-24P
[39]P.Abry,L.Delbeke and P.Flandrin.Wavelet-Based Estimator for the Self-Similarity Parameter of a Stable Process.IEEE-ICASSP-99,Phoenix,1999.125-130P
[40]B.Mandelbrot.Self-Similar Error Clusters in Communication Systems and the Concept of Conditional Stationarity.IEEE Transaction on Communication Technology,1964,13(4):27-53P
[41]李浩.网络流量的自相似特性以及生成方法的研究:(硕士学位论文)长沙:国防科学技术大学.2004.68页-72页
[42]林原.基于网络自相似性的DDOS攻击检测.1998,7,16-21.72-73页