基于Data Mining的网络异常流量检测系统的研究
|
摘要
随着计算机网络规模和应用领域的日益增大，网络复杂性和异构性也不断增加，通过网络传播的病毒和人为破坏越来越多，严重干扰了正常的网络运行秩序。在这种情况下，准确、快速地检测网络流量的异常，并做出合理的响应成为保证网络有效运行的关键问题之一。为了解决上述问题，本文设计了一个能够实时分析网络流量的异常检测系统。
为了保证有较高的检测率和较低的误报率，文中采用数据挖掘技术，从网络历史审计数据里分别得到正常与异常行为规则厍，用实时网络流量数据与其比对，从而判别网络流量数据的异常行为。当发现有不能识别的数据出现时，采用人为干预的方式，更新规则库，增强对未知数据的识别能力。为了避免因主机之间行为存在较大差异而引起的误判，文中以网络历史审计数据为数据源，统计网络中各主机单位时间内访问量，依据聚类算法将主机按访问量聚类建立IP群以指导网络审计数据的分流，用分流后的审计数据分别建立规则库。文中就整个检测系统架构系统各组成部分功能及实现做了详细描述最后，我们将实现的系统原型置于校园网络的出口节点，实时监控出入校园网的网络流量。通过进行模拟网络攻击，我们发现原型系统能够有效地识别已知攻击类型数据并对未知数据有良好的识别能力，实现了对网络异常流量地实时检测。
With the scale of the computer network and application fields growing, network hasbecome an important part of the daily life and work. However, due to increasing thenetwork complexity and heterogeneity, the number of the internet virusand various human factors become more and more through transmitting, whichprobably influence the function of the internet and seriously disturb the normaloperation of the network order. Under such circumstances, accurate and rapiddetection of abnormal network traffic and rational response is one of the key issues toensure the effective functioning of the network. In order to solve the above problem,this paper designs an abnormal detection system to analysis of network trafficcorrectly.
In order to guarantee a higher detection rate and lower false alarm rate, we usedata mining technology. From the historical data network Lane, we can get astorehouse of normal and abnormal behavior. Compared with the use real-timenetwork data flow, we can discriminate the abnormal behavior network traffic data.
If it is found that the data fail to be recognized we can use artificial intervention bythe way, update the rule storehouse and enhance the ability to identify the unknowndata.
To avoid actions between hosts there is a big difference caused by themisjudgment, the paper is based on the historical network data as the audit datasources. We count the number of visiting between the network mainframes accordingto clustering algorithm and we will build up clustering IP group in order to guide thenetwork audit data streaming, after the audit data separately for the establishment ofrules. The paper make a detailed description of the whole detection system, thevarious components of the system functions and realization.
In the experiment, we install the implemented prototype system in the outlet ofcampus network, and real-time access to the campus network monitoring networktraffic. Then, we make several network attacks to a server in the campus network, andfound that the prototype system can be effective in identifying known attack types ofdata as well as data unknown good recognition ability and the implementation of thenetwork traffic in real time abnormal detection.
为了保证有较高的检测率和较低的误报率，文中采用数据挖掘技术，从网络历史审计数据里分别得到正常与异常行为规则厍，用实时网络流量数据与其比对，从而判别网络流量数据的异常行为。当发现有不能识别的数据出现时，采用人为干预的方式，更新规则库，增强对未知数据的识别能力。为了避免因主机之间行为存在较大差异而引起的误判，文中以网络历史审计数据为数据源，统计网络中各主机单位时间内访问量，依据聚类算法将主机按访问量聚类建立IP群以指导网络审计数据的分流，用分流后的审计数据分别建立规则库。文中就整个检测系统架构系统各组成部分功能及实现做了详细描述最后，我们将实现的系统原型置于校园网络的出口节点，实时监控出入校园网的网络流量。通过进行模拟网络攻击，我们发现原型系统能够有效地识别已知攻击类型数据并对未知数据有良好的识别能力，实现了对网络异常流量地实时检测。
With the scale of the computer network and application fields growing, network hasbecome an important part of the daily life and work. However, due to increasing thenetwork complexity and heterogeneity, the number of the internet virusand various human factors become more and more through transmitting, whichprobably influence the function of the internet and seriously disturb the normaloperation of the network order. Under such circumstances, accurate and rapiddetection of abnormal network traffic and rational response is one of the key issues toensure the effective functioning of the network. In order to solve the above problem,this paper designs an abnormal detection system to analysis of network trafficcorrectly.
In order to guarantee a higher detection rate and lower false alarm rate, we usedata mining technology. From the historical data network Lane, we can get astorehouse of normal and abnormal behavior. Compared with the use real-timenetwork data flow, we can discriminate the abnormal behavior network traffic data.
If it is found that the data fail to be recognized we can use artificial intervention bythe way, update the rule storehouse and enhance the ability to identify the unknowndata.
To avoid actions between hosts there is a big difference caused by themisjudgment, the paper is based on the historical network data as the audit datasources. We count the number of visiting between the network mainframes accordingto clustering algorithm and we will build up clustering IP group in order to guide thenetwork audit data streaming, after the audit data separately for the establishment ofrules. The paper make a detailed description of the whole detection system, thevarious components of the system functions and realization.
In the experiment, we install the implemented prototype system in the outlet ofcampus network, and real-time access to the campus network monitoring networktraffic. Then, we make several network attacks to a server in the campus network, andfound that the prototype system can be effective in identifying known attack types ofdata as well as data unknown good recognition ability and the implementation of thenetwork traffic in real time abnormal detection.
引文
[1] 程光，龚俭，丁伟．网络流量宏观行为分析的一种时序分解模型[J]．电子学报，2002，30(11)，1633～1637．
[2] Tom M．Mitchell．机器学习(曾华君，张银奎等译)[M]．北京：机械工业出版社，2003，30～38．
[3] 阎平凡，张长水．人工神经网络与模拟进化计算[M]．北京：清华大学出版社，2005，82～96．
[4] 何飞，李健等．基于流量工程的网络性能监测和控制系统[J]．计算机工程与应用，2001，37(16)；50～53．
[5] 段海新，杨家海．基于Web和数据库的网络管理系统的设计与实现[J]．软件学报，2000，11(4)：468～472．
[6] 李之棠，杨红云．模糊入侵检测模型[J]．计算机工程与科学，2000，22(2)：49～53．
[7] 马皓，张晓军，崔建．DMRTG：网络性能监测的动态扩展[J]．小型微型计算机系统，2005，26(12)2105～2108．
[8] 邹柏贤．一种网络异常实时检测方法[J]计算机学报．2003．26(89)；40～947．
[9] 肖晓丽，田悦宏，陈川．基于改进否定选择匹配算法的异常检测[J]．计算机应用，2005，25(2)；383～385．
[10] 陈金阳，蒋建中，郭军利等．网络攻击技术研究与发展趋势探讨[J]．信息安全与通信保密，2004，59(12)：50～51．
[11] 方勇，龚海澎，胡勇等．一种SYN-Flooding攻击的防范对策[J]．四川大学学报(自然科学版)2004，41(1)；71～76．
[12] MRTG: The Multi Router Traffic Grapher[EB/OL]. http://www.mrtg.org，2004．
[13] 张朝贵．用MRTG监视网络流量[J]．四川轻化工学院学报，2002，15(3)：59～63．
[14] 谢喜秋，粱洁等．网络流量采集工具的分析和比较[J]．电信科学，2002，18(04)：63～66．
[15] 岑贤道，安常清．网络管理协议及应用开发[M]．北京：清华大学出版社，1998．
[16] 卢苇，严斌宇，郑畅等．MRTG软件在校园网状态参数监测中的应用[J]．四川大学学报(自然科学版)，2001，38(2)：189～192．
[17] 丛锁，吴甘沙，张伟等．网络状态参数监测与MRTG的应用[J]．微型电脑应用，2000，16(4)；51～53．
[18] Susan C.L., David V H. Training a neural-network based intrusion detector[J]. IEEE Transactions on systems, man and cybernetics-part a: System andHumans, 2001, 31(4): 294～299.
[19] Manganaris S, Christensen M.A data mining analysis of RTID alarms [J]. Computer Networks. 2000, 34 (4); 571～577.
[20] Sahami M.Using Machine Learning to Improve Information Access [EB/OL].http://ai.stanford.edu/～sahami/bio.html, 1998.
[21] 严大虎，刘毅．一种基于数据挖掘技术的入侵检测模型研究[J]．微机发展，2005，15(2)；47～49．
[22] Balajinath B., Raghavan S.V.Intrusion detection through learning behavior model [J]. Computer Communications, 2001, 24(12): 1202～1212.
[23] Denning D E.An Intrusion-Detection Model [J]. IEEE Transactions on Software Engineering, 1987, 13 (02); 222～232.
[24] Rabiner L R. A tutorial on hidden Markov models and selected application in speech recognition [J]. Proceeding of IEEE, 1989, 77(2): 257～285.
[25] 连一峰，戴英侠，王航．基于模式挖掘的用户行为异常检测[J]．计算机学报，2002，25(3)：325～330．
[26] 胡侃，夏绍玮．基于大型数据仓库的数据采掘研究综述[J]．软件学报，1998，9(01)：53～61．
[27] Dipankar Dasgupta, Fabio Gonzalez.An immunity-based technique to characterize intrusions in computer networks [J]. IEEE Tram Evolutionary Computation, 2002, 6(3): 281～291.
[28] 马江洪，张文修等．数据挖掘与数据库知识发现：统计学的观点[J]．工程数学学报，2002，19(1)：1～13．
[29] Han J, Kamber Morgan Kaufmann M. Data Mining: Concepts and Techniques [M]. New York, Series Editor Morgan Kaufmann Publishers, 2006.
[30] Cecilia M Procopiue. Clustering problems and their applications (survey) [EB/OL]. http://www, cs.duke.edu/magda, Department of Computer Science, Duke University 1997.
[31] 刘军．数据挖掘技术在网络入侵检测中的应用[J]．南京工业大学学报(自然科学版)，2006，28(2)79～84．
[32] U.Fayyad, G Piatetsky-Shapiro. The KDD process of extracting useful knowledge from volumes of data [J]. Communications of the ACM, 1996, 39(11): 27～34.
[33] JianPin Zhu.The ordinal cluster analysis for contingency data and itsapplication [J]. Application of Statistics and Management, 2002,4: 28～33.
[2] Tom M．Mitchell．机器学习(曾华君，张银奎等译)[M]．北京：机械工业出版社，2003，30～38．
[3] 阎平凡，张长水．人工神经网络与模拟进化计算[M]．北京：清华大学出版社，2005，82～96．
[4] 何飞，李健等．基于流量工程的网络性能监测和控制系统[J]．计算机工程与应用，2001，37(16)；50～53．
[5] 段海新，杨家海．基于Web和数据库的网络管理系统的设计与实现[J]．软件学报，2000，11(4)：468～472．
[6] 李之棠，杨红云．模糊入侵检测模型[J]．计算机工程与科学，2000，22(2)：49～53．
[7] 马皓，张晓军，崔建．DMRTG：网络性能监测的动态扩展[J]．小型微型计算机系统，2005，26(12)2105～2108．
[8] 邹柏贤．一种网络异常实时检测方法[J]计算机学报．2003．26(89)；40～947．
[9] 肖晓丽，田悦宏，陈川．基于改进否定选择匹配算法的异常检测[J]．计算机应用，2005，25(2)；383～385．
[10] 陈金阳，蒋建中，郭军利等．网络攻击技术研究与发展趋势探讨[J]．信息安全与通信保密，2004，59(12)：50～51．
[11] 方勇，龚海澎，胡勇等．一种SYN-Flooding攻击的防范对策[J]．四川大学学报(自然科学版)2004，41(1)；71～76．
[12] MRTG: The Multi Router Traffic Grapher[EB/OL]. http://www.mrtg.org，2004．
[13] 张朝贵．用MRTG监视网络流量[J]．四川轻化工学院学报，2002，15(3)：59～63．
[14] 谢喜秋，粱洁等．网络流量采集工具的分析和比较[J]．电信科学，2002，18(04)：63～66．
[15] 岑贤道，安常清．网络管理协议及应用开发[M]．北京：清华大学出版社，1998．
[16] 卢苇，严斌宇，郑畅等．MRTG软件在校园网状态参数监测中的应用[J]．四川大学学报(自然科学版)，2001，38(2)：189～192．
[17] 丛锁，吴甘沙，张伟等．网络状态参数监测与MRTG的应用[J]．微型电脑应用，2000，16(4)；51～53．
[18] Susan C.L., David V H. Training a neural-network based intrusion detector[J]. IEEE Transactions on systems, man and cybernetics-part a: System andHumans, 2001, 31(4): 294～299.
[19] Manganaris S, Christensen M.A data mining analysis of RTID alarms [J]. Computer Networks. 2000, 34 (4); 571～577.
[20] Sahami M.Using Machine Learning to Improve Information Access [EB/OL].http://ai.stanford.edu/～sahami/bio.html, 1998.
[21] 严大虎，刘毅．一种基于数据挖掘技术的入侵检测模型研究[J]．微机发展，2005，15(2)；47～49．
[22] Balajinath B., Raghavan S.V.Intrusion detection through learning behavior model [J]. Computer Communications, 2001, 24(12): 1202～1212.
[23] Denning D E.An Intrusion-Detection Model [J]. IEEE Transactions on Software Engineering, 1987, 13 (02); 222～232.
[24] Rabiner L R. A tutorial on hidden Markov models and selected application in speech recognition [J]. Proceeding of IEEE, 1989, 77(2): 257～285.
[25] 连一峰，戴英侠，王航．基于模式挖掘的用户行为异常检测[J]．计算机学报，2002，25(3)：325～330．
[26] 胡侃，夏绍玮．基于大型数据仓库的数据采掘研究综述[J]．软件学报，1998，9(01)：53～61．
[27] Dipankar Dasgupta, Fabio Gonzalez.An immunity-based technique to characterize intrusions in computer networks [J]. IEEE Tram Evolutionary Computation, 2002, 6(3): 281～291.
[28] 马江洪，张文修等．数据挖掘与数据库知识发现：统计学的观点[J]．工程数学学报，2002，19(1)：1～13．
[29] Han J, Kamber Morgan Kaufmann M. Data Mining: Concepts and Techniques [M]. New York, Series Editor Morgan Kaufmann Publishers, 2006.
[30] Cecilia M Procopiue. Clustering problems and their applications (survey) [EB/OL]. http://www, cs.duke.edu/magda, Department of Computer Science, Duke University 1997.
[31] 刘军．数据挖掘技术在网络入侵检测中的应用[J]．南京工业大学学报(自然科学版)，2006，28(2)79～84．
[32] U.Fayyad, G Piatetsky-Shapiro. The KDD process of extracting useful knowledge from volumes of data [J]. Communications of the ACM, 1996, 39(11): 27～34.
[33] JianPin Zhu.The ordinal cluster analysis for contingency data and itsapplication [J]. Application of Statistics and Management, 2002,4: 28～33.