网络流量监测技术在异常流量分析上的应用与研究
|
摘要
伴随着Internet的迅速发展，网络安全事件开始频繁发生，各种攻击手段层出不穷，计算机网络的保密性、完整性和可用性受到了严峻考验。分布式拒绝服务DDoS攻击就是目前一种危害极大的攻击方式，给网络服务器带来严重的威胁，也给千千万万的网络用户造成了损失。
网络流量监测技术已经成为保证现代网络管理性能的重要手段，在网络的配置管理、故障管理、性能管理、安全管理和计费管理等方面发挥着不可替代的作用。本文研究的目的是通过对网络流量监测技术进行研究，然后应用到DDoS等攻击、病毒造成的异常流量的监测和防御中。
本文对流量监测的应用范围进行了介绍，并对目前所流行的三种流量监测技术：基于网络流量全镜像的监测技术、基于SNMP的监测技术，基于Netflow的监测技术进行了分析与对比，通过分析对比，指出了这三种监测技术的特点及适用范围。
网络流量的采集是流量分析的基础，本文对现有两种常见的流量采集方式进行了介绍，并做了以下两项工作：一是认真了研究Unix环境下享誉盛名的BPF(Berkeley Packet Filter)，提出了一种基于BPF的数据包捕获模型，二是针对NetFlow技术的特点，对数据采集的采样率、采集点的设置、数据采集对网络的影响等多个方面进行了分析，并设计出了一种数据采集的系统组织方案。
异常流量的识别是对异常流量进行有效预防和处理的前提和基础，也是异常流量监测的主要工作之一。本文结合Netflow技术，从目前比较流行的异常流量的种类(如DoS、DDoS、网络蠕虫病毒等)入手，进行了大量的研究实践，通过对所采集的数据包的源地址、目的地址、源端口、目的端口、协议类型、字节数等关键参数的分析，给出了一套判断、分析异常流量的的方法。
最后，结合以上的研究，本文提出了从判断到采集、分析，到处理的一套系统的网络异常流量处理方法。
The Internet safe accidents frequently take place with Internet fast development. All kinds of attack measures constantly appear. The secrecy, integrity and usability of computer network are austerely tested out. Distributed Deny of Service name DDoS is a kind of attack measure making huge harm. DDoS badly threaten network server and make losing for thousands of network user.
Technology of net flow inspection has become an important measure to ensure modern network management performance. The technology has effect which can not be replaced at many network fields such as configuration management, failure management, performance management, safe management and expense management. The objective of paper is researching net flow inspection technology for using to inspect and defend exceptional flow which cased by DDoS attack and virus.
The application area of net flow inspection is introduced in this paper. The three kinds of flow inspection technologies such as inspection technology based on net flow total mirror, inspection technology based on SNMP and inspection technology based on Netflow are analyzed and compared in this paper. The characteristics and applying area of three kinds of technologies are brought forward by comparing and analyzing.
The net flow collection is base of flow analyzing. Two kinds of familiar flow collection method are introduced in the paper. Tow items of work are completed. The famous Berkeley Packet Filter on Unix environments is researched and a data packet capturing model based BPF is put forward. The many technologies such as data sampling frequency, setting of sampling point and affection of data sampling to network are analyzed. The data sampling scheme is designed.
The exceptional flow distinguish is basement and precondition of effectively prevented and managed exceptional flow. The measure is one of main work of net flow inspection. Large numbers of research practice about actual popular kind of exceptional flow such as DoS, DDoS and net worm virus is processed with Netflow technology in the paper. The pivotal parameters of collection data package is analyzed which is source address, destination address, source port, destination port, protocol type and byte number. The method of exactly distinguishing exceptional flow is brought forward.
The method is brought forward of exceptional flow management method from collection, analyzing and disposal with upwards research in the paper at last.
网络流量监测技术已经成为保证现代网络管理性能的重要手段，在网络的配置管理、故障管理、性能管理、安全管理和计费管理等方面发挥着不可替代的作用。本文研究的目的是通过对网络流量监测技术进行研究，然后应用到DDoS等攻击、病毒造成的异常流量的监测和防御中。
本文对流量监测的应用范围进行了介绍，并对目前所流行的三种流量监测技术：基于网络流量全镜像的监测技术、基于SNMP的监测技术，基于Netflow的监测技术进行了分析与对比，通过分析对比，指出了这三种监测技术的特点及适用范围。
网络流量的采集是流量分析的基础，本文对现有两种常见的流量采集方式进行了介绍，并做了以下两项工作：一是认真了研究Unix环境下享誉盛名的BPF(Berkeley Packet Filter)，提出了一种基于BPF的数据包捕获模型，二是针对NetFlow技术的特点，对数据采集的采样率、采集点的设置、数据采集对网络的影响等多个方面进行了分析，并设计出了一种数据采集的系统组织方案。
异常流量的识别是对异常流量进行有效预防和处理的前提和基础，也是异常流量监测的主要工作之一。本文结合Netflow技术，从目前比较流行的异常流量的种类(如DoS、DDoS、网络蠕虫病毒等)入手，进行了大量的研究实践，通过对所采集的数据包的源地址、目的地址、源端口、目的端口、协议类型、字节数等关键参数的分析，给出了一套判断、分析异常流量的的方法。
最后，结合以上的研究，本文提出了从判断到采集、分析，到处理的一套系统的网络异常流量处理方法。
The Internet safe accidents frequently take place with Internet fast development. All kinds of attack measures constantly appear. The secrecy, integrity and usability of computer network are austerely tested out. Distributed Deny of Service name DDoS is a kind of attack measure making huge harm. DDoS badly threaten network server and make losing for thousands of network user.
Technology of net flow inspection has become an important measure to ensure modern network management performance. The technology has effect which can not be replaced at many network fields such as configuration management, failure management, performance management, safe management and expense management. The objective of paper is researching net flow inspection technology for using to inspect and defend exceptional flow which cased by DDoS attack and virus.
The application area of net flow inspection is introduced in this paper. The three kinds of flow inspection technologies such as inspection technology based on net flow total mirror, inspection technology based on SNMP and inspection technology based on Netflow are analyzed and compared in this paper. The characteristics and applying area of three kinds of technologies are brought forward by comparing and analyzing.
The net flow collection is base of flow analyzing. Two kinds of familiar flow collection method are introduced in the paper. Tow items of work are completed. The famous Berkeley Packet Filter on Unix environments is researched and a data packet capturing model based BPF is put forward. The many technologies such as data sampling frequency, setting of sampling point and affection of data sampling to network are analyzed. The data sampling scheme is designed.
The exceptional flow distinguish is basement and precondition of effectively prevented and managed exceptional flow. The measure is one of main work of net flow inspection. Large numbers of research practice about actual popular kind of exceptional flow such as DoS, DDoS and net worm virus is processed with Netflow technology in the paper. The pivotal parameters of collection data package is analyzed which is source address, destination address, source port, destination port, protocol type and byte number. The method of exactly distinguishing exceptional flow is brought forward.
The method is brought forward of exceptional flow management method from collection, analyzing and disposal with upwards research in the paper at last.
引文
[1]晏家豪等，NetFlow网络业务流量监测技术的应用和设计[J]，邮电设计技术2006，1：11～12
[2]晏家豪等，互联网业务流量监测技术的应用和设计[J]，邮电设计技术2005．12：13～16
[3]金华敏等，异常流量监测技术的电信IP网应用[J]，通信世界，2005．5：31～38
[4]中国IT认证实验室DoS拒绝服务攻击专题[EB／OL]．http://www.chinaitiab.com/www/special/ciwDDoS.asp. 2006-08-14
[5]Harnedy S．简单网络管理协议教程[M]．北京．电子工业出版社1999：152，153，158
[6]段宗涛，林莎．基于SNMP的网络流量监控系统的设计与实现[J]，微型机与应用，2006．5：20～22
[7]张海波，网络流量监测与分析[D]，西安：西北工业大学，2001．
[8]陈丽华，高速IP网络数据处理与分析模型的设计与实现，[D]，西安：西北工业大学，2001
[9]杨策，张永智，庞正社，网络流量监测技术及性能分析IN]，空军工程大学学报，2003-11-24(6)
[10]Jaeyeon Jung, Balachander Krishnamurthy, and Michael Rabinovich.Flash crowds and denial of service attacks:Characterization and implication for cdns and web sites[N],WWW10,WWW2002, May 7-11, Honolulu, Hawaii, USA2002.
[11]P. Ferguson, D. Senie. Network ingress filtering:Defeating denial of service attacks which employ IP source address spllfing agreenients performance monitoring, RFC2827, May2000.
[12]Tobias Oeliker. mrtg-What is MRTG[J],2004:101～112
[13]Introduce to Cisco NetFlow Switching Software[M], 2001:23～26
[14]Phaal P, Panchen S, McKee N. A Method for Monitoring Traffic in Switched and Routed Networks[M]RFC3176,2001.
[15]Kihong Park and Heejo Lee. On the effectiveness of router-based packer filtering for distributed dos attack prevention in power-law internets. In Proceeding of 2001 ACM SIGCOMM Conference, San Diego,California, U.S.A.,August 2001.
[16]S Savage, D Wetherall, A Karlin et al. Network Support for IP Traceback[j],JEEE/ACM Transaction on Networking, 2001.
[17]Alex C Snoeren, Craig Partridge, Luis A Sanchez et. Hash-Based IP TraceBack[C].In: Proc ACM SICCOMM Conf, 2001-08.
[18]Drew Dean, Matt Franklin, Adam Stubblefield, An Algebraic Approach for IP TrackBack[C].In: Pro 2001 Network and Distributed System Security Symp,2001-02.
[19]Jianxin Yan, Stephen Early. The XenoService: A Distributed Defera for Distributed Denial of Service[C].
[20]李桂成，杨玉森．魏晓丽。测量误差及数据处理原理[M]．长春：吉林大学出版社，1990，121～126
[21]孟学军，吴黎兵，石岗．基于NetFlow网络流量分析的研究反应用[J]．华中科技大学学报．2003，31(10)：253—255。
[22]陈健．张亚平，李艳，基于流量分析的入侵检测系组研究[J]．天津理工学院学报．2004，20(2)：86—88。
[23]胡小新，王颖，罗旭斌．一种DDOS攻8的防御方案[J]．计算机工程与应用，2004(12)：160—163．
[24]薛丽军，第文军，蒋世奇．一种新的网络流量异常检测方法．2003，16(3)：45-49．
[25]陈丽华，张凌，汤力群，许勇．基于流量的网络性能分析模型设计与实现．计算机工程，2000，26：304—308．
[26]邹柏贤．一种网络异常实时检测方法[J]．计算机学报，2003，26(8)：941-947．
[27]黄美莹，郑纬民，汪东升，杨厂文．基于B／S模式和JSP编程指南[M]．电子工业出版社，2001．
[28]Karl Avedel，Danny Ayers，Timothy Briggs．北京：JSP编程指南[M]．电子工业出版社，2001，77～82
[29]Rahim Adazia．Faiz Arni，Kyle Gabhart．背景：EJB编程指南[M]．电子工业出版社，2002，65～72
[30]刘芳，陈泰伟，戴葵．分布式拒绝服务攻击预警系统的设计与实现[J]．计算机工程，2001，27(5)：129—130．
[31]刘特，徐迎晓等，基于Java Servlet的网络流量采集与监控技术．计算机工程．2002，28(5)：167—169．
[32]何飞，李健等．基于流量工程的网络性能监测和控制系统．计算机工程与应用，2001，16：50—53．
[33]Case, J., Fedor, M. Schoffstall,M, and J. Davin. A Simple Network Management Protocol(SNMP). RFC 1157, SNM P Research, Performance Systems International, Performance Systems International, and M IT Laboratory for Computer Science, May 1990
[34]Rose M, and K. Mc Cloghrie. Structure and Identification of Management Information for TCP/IP-based internets RFC 1155, Performance Systems International, Hughes LAN Systems,May 1990
[35]M. Rose. Management Information Base for Net-work Management of TCP/IP-based internets:M IB-Ⅱ RFC 1158, Performance Systems International, May 1990
[36]Conne Xions, SMP/SNMP Version 2: the Evolution of SNMP,1992,6(10):3～5
[37]岑贤道，安常青．网络管理协议及应用开发[M]．北京：清华大学出版社，1998：35—43
[38]杨家海，任宪坤，王沛瑜编．网络管理原理与实现技术[M]．北京：清华大学出版社出版，2000：66—75
[39]胡谷雨等．简单网络管理协议教程第2版[M]，电子工业出版社，1999：114—119
[40]王斌程明．基于SNMP协议的网络流量侦测系统设计及实现[N]．天津理工 大学学报，2005-6
[41]杨洁，窦伊男，雷振明．IP网络流量测量的研究与实现[J]．现代电信科技，2005—11：33-36
[42]Peter Phaal, Sonia Panchen和Neil McKee。RFC 3176, InMon Corporation's sFlow: A Method for Monitoring Traffic in a Switched and Routed Networks。
[43]Kimberly C. Claffy, George C. Polyzos 和 Hans-Werner Braun。Applications of Sampling Methodologies to Network Traffic Characterization。
[44]Jonathan Jedwab，Peter Phaal，Bob Pinna，(科罗拉多电信分公司)管理、数学和安全部，惠普布里斯托尔实验室。Traffic Estimation for the Largest Sources on a network, Using Packet Sampling with Limited Storage。
[45]J. Case, M. Fedor, M. Schoffstall and J. Davin, A simple Network Management Protocol(SNMP), RFC 1157,1990
[46]ITU—T Rec. M. 3010, Principles for a Telecommunication Management Network(TMN), Study Groupl Ⅳ, 1996
[47]K. McCloghrie and M. Rose, Su-ucture and Identification of Management Information for TCP/IP based Intemets, RFC 1 155, 1990
[48]沈俊，顾冠群，罗军舟．网络管理的研究和发展[J]．计算机研究和发展，2002，39(10)：1153～1167
[49]徐志光．网络管理的应用[M]．北京：人民出版社，1999：142～151
[2]晏家豪等，互联网业务流量监测技术的应用和设计[J]，邮电设计技术2005．12：13～16
[3]金华敏等，异常流量监测技术的电信IP网应用[J]，通信世界，2005．5：31～38
[4]中国IT认证实验室DoS拒绝服务攻击专题[EB／OL]．http://www.chinaitiab.com/www/special/ciwDDoS.asp. 2006-08-14
[5]Harnedy S．简单网络管理协议教程[M]．北京．电子工业出版社1999：152，153，158
[6]段宗涛，林莎．基于SNMP的网络流量监控系统的设计与实现[J]，微型机与应用，2006．5：20～22
[7]张海波，网络流量监测与分析[D]，西安：西北工业大学，2001．
[8]陈丽华，高速IP网络数据处理与分析模型的设计与实现，[D]，西安：西北工业大学，2001
[9]杨策，张永智，庞正社，网络流量监测技术及性能分析IN]，空军工程大学学报，2003-11-24(6)
[10]Jaeyeon Jung, Balachander Krishnamurthy, and Michael Rabinovich.Flash crowds and denial of service attacks:Characterization and implication for cdns and web sites[N],WWW10,WWW2002, May 7-11, Honolulu, Hawaii, USA2002.
[11]P. Ferguson, D. Senie. Network ingress filtering:Defeating denial of service attacks which employ IP source address spllfing agreenients performance monitoring, RFC2827, May2000.
[12]Tobias Oeliker. mrtg-What is MRTG[J],2004:101～112
[13]Introduce to Cisco NetFlow Switching Software[M], 2001:23～26
[14]Phaal P, Panchen S, McKee N. A Method for Monitoring Traffic in Switched and Routed Networks[M]RFC3176,2001.
[15]Kihong Park and Heejo Lee. On the effectiveness of router-based packer filtering for distributed dos attack prevention in power-law internets. In Proceeding of 2001 ACM SIGCOMM Conference, San Diego,California, U.S.A.,August 2001.
[16]S Savage, D Wetherall, A Karlin et al. Network Support for IP Traceback[j],JEEE/ACM Transaction on Networking, 2001.
[17]Alex C Snoeren, Craig Partridge, Luis A Sanchez et. Hash-Based IP TraceBack[C].In: Proc ACM SICCOMM Conf, 2001-08.
[18]Drew Dean, Matt Franklin, Adam Stubblefield, An Algebraic Approach for IP TrackBack[C].In: Pro 2001 Network and Distributed System Security Symp,2001-02.
[19]Jianxin Yan, Stephen Early. The XenoService: A Distributed Defera for Distributed Denial of Service[C].
[20]李桂成，杨玉森．魏晓丽。测量误差及数据处理原理[M]．长春：吉林大学出版社，1990，121～126
[21]孟学军，吴黎兵，石岗．基于NetFlow网络流量分析的研究反应用[J]．华中科技大学学报．2003，31(10)：253—255。
[22]陈健．张亚平，李艳，基于流量分析的入侵检测系组研究[J]．天津理工学院学报．2004，20(2)：86—88。
[23]胡小新，王颖，罗旭斌．一种DDOS攻8的防御方案[J]．计算机工程与应用，2004(12)：160—163．
[24]薛丽军，第文军，蒋世奇．一种新的网络流量异常检测方法．2003，16(3)：45-49．
[25]陈丽华，张凌，汤力群，许勇．基于流量的网络性能分析模型设计与实现．计算机工程，2000，26：304—308．
[26]邹柏贤．一种网络异常实时检测方法[J]．计算机学报，2003，26(8)：941-947．
[27]黄美莹，郑纬民，汪东升，杨厂文．基于B／S模式和JSP编程指南[M]．电子工业出版社，2001．
[28]Karl Avedel，Danny Ayers，Timothy Briggs．北京：JSP编程指南[M]．电子工业出版社，2001，77～82
[29]Rahim Adazia．Faiz Arni，Kyle Gabhart．背景：EJB编程指南[M]．电子工业出版社，2002，65～72
[30]刘芳，陈泰伟，戴葵．分布式拒绝服务攻击预警系统的设计与实现[J]．计算机工程，2001，27(5)：129—130．
[31]刘特，徐迎晓等，基于Java Servlet的网络流量采集与监控技术．计算机工程．2002，28(5)：167—169．
[32]何飞，李健等．基于流量工程的网络性能监测和控制系统．计算机工程与应用，2001，16：50—53．
[33]Case, J., Fedor, M. Schoffstall,M, and J. Davin. A Simple Network Management Protocol(SNMP). RFC 1157, SNM P Research, Performance Systems International, Performance Systems International, and M IT Laboratory for Computer Science, May 1990
[34]Rose M, and K. Mc Cloghrie. Structure and Identification of Management Information for TCP/IP-based internets RFC 1155, Performance Systems International, Hughes LAN Systems,May 1990
[35]M. Rose. Management Information Base for Net-work Management of TCP/IP-based internets:M IB-Ⅱ RFC 1158, Performance Systems International, May 1990
[36]Conne Xions, SMP/SNMP Version 2: the Evolution of SNMP,1992,6(10):3～5
[37]岑贤道，安常青．网络管理协议及应用开发[M]．北京：清华大学出版社，1998：35—43
[38]杨家海，任宪坤，王沛瑜编．网络管理原理与实现技术[M]．北京：清华大学出版社出版，2000：66—75
[39]胡谷雨等．简单网络管理协议教程第2版[M]，电子工业出版社，1999：114—119
[40]王斌程明．基于SNMP协议的网络流量侦测系统设计及实现[N]．天津理工 大学学报，2005-6
[41]杨洁，窦伊男，雷振明．IP网络流量测量的研究与实现[J]．现代电信科技，2005—11：33-36
[42]Peter Phaal, Sonia Panchen和Neil McKee。RFC 3176, InMon Corporation's sFlow: A Method for Monitoring Traffic in a Switched and Routed Networks。
[43]Kimberly C. Claffy, George C. Polyzos 和 Hans-Werner Braun。Applications of Sampling Methodologies to Network Traffic Characterization。
[44]Jonathan Jedwab，Peter Phaal，Bob Pinna，(科罗拉多电信分公司)管理、数学和安全部，惠普布里斯托尔实验室。Traffic Estimation for the Largest Sources on a network, Using Packet Sampling with Limited Storage。
[45]J. Case, M. Fedor, M. Schoffstall and J. Davin, A simple Network Management Protocol(SNMP), RFC 1157,1990
[46]ITU—T Rec. M. 3010, Principles for a Telecommunication Management Network(TMN), Study Groupl Ⅳ, 1996
[47]K. McCloghrie and M. Rose, Su-ucture and Identification of Management Information for TCP/IP based Intemets, RFC 1 155, 1990
[48]沈俊，顾冠群，罗军舟．网络管理的研究和发展[J]．计算机研究和发展，2002，39(10)：1153～1167
[49]徐志光．网络管理的应用[M]．北京：人民出版社，1999：142～151