基于聚类的入侵检测模型及算法研究
|
摘要
随着网络技术的飞速发展和网络应用范围的不断扩大,对网络的各类攻击与破坏与日俱增。在网络安全问题日益突出的今天,如何迅速有效地发现各种入侵行为,对于保证系统和网络资源的安全显得十分重要。传统的防火墙、数据加密等静态防御方式已很难胜任网络安全的需要,而入侵检测系统是一种积极主动的安全防护技术,它是信息安全保护体系结构中的一个重要组成部分,针对入侵检测方法和技术的研究已经引起人们越来越多的重视。
入侵检测技术通过对入侵行为的过程与特征的研究,使安全系统对入侵事件和入侵过程能做出实时响应,常用检测方法一般可分为两种：误用入侵检测和异常入侵检测。误用检测是指运用已知攻击方法,根据已定义好的入侵模式,通过判断这些入侵模式是否出现来检测是否存在这些入侵行为,其优点是可以准确地检测已知的入侵行为,缺点是不能检测未知的入侵行为；而在异常入侵检测中,假定所有入侵行为都是与正常行为不同的,’这样,如果建立系统正常行为的轨迹,那么理论上可以把所有与正常轨迹不同的系统状态视为可疑企图。数据挖掘技术可被用来进行特征构造和检测,聚类分析作为数据挖掘中的一个热点研究领域,它可通过对大量数据的分析来对数据对象进行自动归类,适用于异常检测。
为了克服目前已有入侵检测方法的不足,本文重点研究如何把聚类技术应用于入侵检测领域,所做工作主要包括以下二个方面：
(1)深入研究了入侵检测和聚类的相关理论；
(2)基于质心Voronoi图,提出了一种新的异常检测算法。在该算法中,首先利用质心Voronoi图来对样本数据进行聚类,然后基于聚类结果,计算出各个样本点的点密度,并以此来判断样本数据是否异常。最后,通过基于KDD Cup 1999数据集的实验测试,仿真结果表明,新算法在具有较低的误报率同时,也具有良好的检测率。
With the fast development of the network technologies and continuous extension of the network application scale, various network attacks increase day by day. Considering the currently severe network security problem, how to discover and find all kinds of the intrusions rapidly and effectively becomes very important for ensuring the security of systems and network recourses. These traditional static protection methods such as firewall and data encryption are difficult to satisfy the need of network security, but since the intrusion detection technology is a kind of active and initiative security protection technology and it is an important part of information security protection system structure, then the study and research on the intrusion detection technologies has attracted more and more attention.
Through the study of the process and characteristics of the intrusion behaviors, the intrusion detection technologies can tell the security system to make real-time response to any intrusions and the process. Usually there are two kinds of detection methods, misuse intrusion detection and anomaly intrusion detection. the misuse intrusion detection uses known attack methods based on defined intrusion profiles to judge whether there is any these defined intrusion profiles in the intrusions. The advantage of misuse intrusion detection is that any known intrusion behaviors can be detected precisely; while the disadvantage is unknown intrusion behaviors cannot be detected. However, in the method of anomaly intrusion detction, all intrusion behaviors are supposed to be different from normal behaviors, so if normal behaviors are established, theoretically, all different behaviors are considered to be suspicious. Data mining technology can be used for characteristic construction and detection, and cluster analysis is one hot research field in data mining, which can analyze a great volume of data to classify objects automatically, and is suitable for exceptional intrusion detection.
To make up the disadvantage of known detection methods, this article focuses on how to apply cluster technology in intrusion detection field and the major contribution includes:
(1) Research deeply on the theories about intrusion detection and clustering.
(2) Based on the Centroidal Voronoi Diagram, a new algorithm of anomaly detection is proposed in this paper, in which, the Centroidal Voronoi Diagram is applied in the clustering of sample data first, and then, the point density is computed out according to the results of clustering for each sample point, which is used to determine that whether the sample data is abnormal or not. Finally, a series of experiments on well known KDD Cup 1999 dataset demonstrate that our new algorithm has low false positive rate while ensuring high detection rate.
入侵检测技术通过对入侵行为的过程与特征的研究,使安全系统对入侵事件和入侵过程能做出实时响应,常用检测方法一般可分为两种：误用入侵检测和异常入侵检测。误用检测是指运用已知攻击方法,根据已定义好的入侵模式,通过判断这些入侵模式是否出现来检测是否存在这些入侵行为,其优点是可以准确地检测已知的入侵行为,缺点是不能检测未知的入侵行为；而在异常入侵检测中,假定所有入侵行为都是与正常行为不同的,’这样,如果建立系统正常行为的轨迹,那么理论上可以把所有与正常轨迹不同的系统状态视为可疑企图。数据挖掘技术可被用来进行特征构造和检测,聚类分析作为数据挖掘中的一个热点研究领域,它可通过对大量数据的分析来对数据对象进行自动归类,适用于异常检测。
为了克服目前已有入侵检测方法的不足,本文重点研究如何把聚类技术应用于入侵检测领域,所做工作主要包括以下二个方面：
(1)深入研究了入侵检测和聚类的相关理论；
(2)基于质心Voronoi图,提出了一种新的异常检测算法。在该算法中,首先利用质心Voronoi图来对样本数据进行聚类,然后基于聚类结果,计算出各个样本点的点密度,并以此来判断样本数据是否异常。最后,通过基于KDD Cup 1999数据集的实验测试,仿真结果表明,新算法在具有较低的误报率同时,也具有良好的检测率。
With the fast development of the network technologies and continuous extension of the network application scale, various network attacks increase day by day. Considering the currently severe network security problem, how to discover and find all kinds of the intrusions rapidly and effectively becomes very important for ensuring the security of systems and network recourses. These traditional static protection methods such as firewall and data encryption are difficult to satisfy the need of network security, but since the intrusion detection technology is a kind of active and initiative security protection technology and it is an important part of information security protection system structure, then the study and research on the intrusion detection technologies has attracted more and more attention.
Through the study of the process and characteristics of the intrusion behaviors, the intrusion detection technologies can tell the security system to make real-time response to any intrusions and the process. Usually there are two kinds of detection methods, misuse intrusion detection and anomaly intrusion detection. the misuse intrusion detection uses known attack methods based on defined intrusion profiles to judge whether there is any these defined intrusion profiles in the intrusions. The advantage of misuse intrusion detection is that any known intrusion behaviors can be detected precisely; while the disadvantage is unknown intrusion behaviors cannot be detected. However, in the method of anomaly intrusion detction, all intrusion behaviors are supposed to be different from normal behaviors, so if normal behaviors are established, theoretically, all different behaviors are considered to be suspicious. Data mining technology can be used for characteristic construction and detection, and cluster analysis is one hot research field in data mining, which can analyze a great volume of data to classify objects automatically, and is suitable for exceptional intrusion detection.
To make up the disadvantage of known detection methods, this article focuses on how to apply cluster technology in intrusion detection field and the major contribution includes:
(1) Research deeply on the theories about intrusion detection and clustering.
(2) Based on the Centroidal Voronoi Diagram, a new algorithm of anomaly detection is proposed in this paper, in which, the Centroidal Voronoi Diagram is applied in the clustering of sample data first, and then, the point density is computed out according to the results of clustering for each sample point, which is used to determine that whether the sample data is abnormal or not. Finally, a series of experiments on well known KDD Cup 1999 dataset demonstrate that our new algorithm has low false positive rate while ensuring high detection rate.
引文
[1]中国互联网络信息中心.第二十一次中国互联网发展统计调查报告.http://tech.qq.com/zt/2008/cnnic21/,2008-01
[2]国家计算机网络应急技术处理协调中心.CNCERT/CC 2006年网络安全工作报告.http://www.cert.org.cn,2007-04-06
[3]唐正军,李建华.入侵检测技术.北京：清华大学出版社,2004,9-10
[4]罗守山.入侵检测.北京邮电大学出版社,2004,82-90
[5]廖光忠,陈志凤.入侵检测技术研究综述.网络安全技术与应用,2007,(2),31-33
[6]J.P Anderson, Computer Security Threat Monitoring and Surveil-lance. Fort Washington Pennsylvania, April:James P Anderson Co,1980,35-40
[7]D. Denning. An Intrusion Detection Model. IEEE Transactions on Software Engineering,1987,13(2),222-232
[8]张杰,戴英侠.入侵检测系统技术现状及其发展趋势.计算机与通信,2002,(6),28-30
[9]Phil Porras, Dan Schnackenberg. Stuart Staniford-Chen. The Common Intrusion Detection Framework Architecture,2006,11(6):16-28
[10]叶飞,石福斌.基于异常的入侵检测技术研究.网络安全技术与应用,2007,(6),44-47
[11]Lunt T.F., Tamaru A., Gilham F. et al. A Real-time intrusion detection expert system(IDES). Computer Science Laboratory SRI International, Menlo Park, California, Technical Report,1992,51-53
[12]Anderson Debra, Lunt Teresa F., Javitz Harold, etal. Detecting unusual program behavior using the statistical component of the next-generation expert system(NIDES). Computer Science Laboratory, SRI International, Menlo Park, CA, USA, Technical Report SRI-CSL-95-06,1995,36-40
[13]Smaha S E, Haystack:An intrusion detection system. USA, Orlando FL Dec, 1998,215-241
[14]Hochberg J, Jackson K, Stallings C, et al. NADIR:An Automated System for Detecting Network Intrusion and Misuse. Computers and Security,1993, 12(3):235-248
[15]Todd Heberlein L, Gihan Dias V.KarlLevittN. etal. A Network security monitor, 1991,5(7):101-113
[16]Snapp, S.Brentano, J.Dias, etal. Mansur, D DIDS (Distributed Intrusion Detection System)Motivation, Architecture, and an Early Prototype,1991, 11 (4):79-86
[17]PORTNOY L, ESKIN E, STOLFO S J. Intrusion detection with unlabeled data using clustering. Proceedings of the ACM CSS Workshop on Data Mining App lied to Security. New York,2001,56-60
[18]CH IMPHLEEW, ABDULLAH A H, NOORMD SAPM, et al. Integrating genetic algorithms and fuzzy c2means for anomaly detection. Annual IEEE IND ICON. Washington, DC:IEEE,2005,575-579
[19]KR ISHNAPURAM R, KELLER J M. A possibilistic app roach to clustering. IEEE Transactions on Fuzzy Systems,1993,1 (2):98-110
[20]李庆华,李新,蒋盛益.一种面向高维混合属性数据的异常挖掘算法.计算机应用,2005,25(6),353-356
[21]Babu G P, Murty M N. A near-optimal initial seed value selection in K-means algorithm using a genetic algorithm.Pattern Recognition Letters.1993,14(10): 763-769
[22]SHENG W G, LIU X H. A hybrid algorithm for k-medoid clustering of large data sets. Proceed-ings of IEEE Congress on Evolutionary Computa-tion,2004, 101-105
[23]Ng Raymond T, Jiawei Han. Efficient and effective clustering methods for spatial data mining//Proceedings of the 20th Very Large Databases Conference (VLDB 94), Santiago, Chile,1994,144-155
[24]T Zhang, R Ramakrishnan, M Livny, et al. An efficient data clustering method for very large databases. In:Proc of the 1996 ACM SIGMOD Int'1 Conf on Management of Data. Montreal, Quebec:ACM Press,1996,103-114
[25]S Guha, R Rastogi, K Shim, et al. An efficient clustering algorithm for large databases. In:Proc of 1998 ACM SIGMOD Int'1 Conf on Management of Data. Seattle, Washington:ACM Press,1998,73-84
[26]G Karypis, E H Han, V Kumarl CHAMEL EON. A hierarchical clustering algorithm using dynamic modeling. Computer,1999,32(8),68-75
[27]Rao Xian, Dong Cun-Xi, Yangshao-Quan. Statistic learning and intrusion detection Lecture Notes in Artificial Intelligence.Subseries of Lecture Notes in Computer Science,2003,652-659
[28]LiuZhen, Susan M.Bridges, Rayford B.Vaughn. Classification of Anomalous Traces of Privileged and Parallel Programs by Neural Networks, St.Louis Missouri,2003,1225-1230
[29]Chen Yuju, Huang Tsungchuan, Hwang Rey Chue.An effective learning of neural network by using RFBP learning algorithm.Information Sciences, December 2, 2004,167 (4),77-86
[30]Jin Yaochu, Sendhoff Bernhard. Extracting Interpretable fuzzy Rules from RBF Networks. Neural processing Letters,2003.4,17(2),149-164
[31]Liang Y.C., Feng D.P., Liu G.R., et al. Neural identification of rock Parameters using fuzzy adaptive learning Parameters. Computers and Structures, September, 2003,81(24-25),2373-2382
[32]Andrew Honig, Andrew Howard, Eleazar Eskin, et al. Adaptive Model Generation:An Architecture for Deployment of Data Mining-based intrusion Detection systems. Data Mining for Security Applications-Kluwer,2002
[33]Kaya M., Alhajj R.. Genetic algorithm based framework for mining fuzzy association rules. Fuzzy Sets and Systems, June 16,2005,152(3),587-601
[34]Matsuoka K, Ohya, Kawamoto M. A neural net for blind separation of nonstationary signals.Neural Networks,1995
[35]刘衍珩,田大新,余雪岗,王健.基于分布式学习的大规模网络入侵检测算 法.软件学报,2008,19(4)：993-1003
[36]宿娇娜,李程,李巍,唐发根,李云春.基于改进NB分类方法的网络异常检测模型.计算机工程,2008,34(5)：148-149
[37]李洋,方滨兴,郭莉,陈友.基于直推式方法的网络异常检测方法.软件学报,2007,18(10)：2595-2604
[38]王雷,林亚平,彭雅,李闻,基于认知学习和最小风险的朴素贝叶斯邮件过滤算法.系统仿真学报,2004,16(3)：413-416
[39]Krugel C, Toth T, Kirda E. Service specific anomaly detection for network intrusion detection. In:Lamont GB, Hadda Papadopoulos G, Panda B, eds. Proc. of the 2002 ACM Symp. on Applied Computing. New York:ACM Press, 2002.201-208
[40]Lee W, Stolfo SJ. A framework for constructing features and models for intrusion detection systems. ACM Trans. on Information and System Security, 2000,3(4):227-261
[41]李昆仑,黄厚宽,田盛丰,刘振鹏,刘志强.模糊多类支持向量机及其在入侵检测中的应用.计算机学报,2005,28(2)：274-280
[42]陈治平,王雷.基于密度梯度的聚类算法研究.计算机应用,2006,26(10)：2389-2392
[43]Du, Qiang, Gunzburger Max, Ju, Lili, Wang Xiaoqiang. Centroidal Voronoi Tessellation Algorithms for Image Compression, Segmentation, and Multichannel Restoration. Journal of Mathematical Imaging and Vision,2006, 24(2):177-194
[44]肖立中,邵志清,马汉华,王秀英,刘刚.网络入侵检测中的自动决定聚类数算法.软件学报,2008,19(8)：2140-2148
[45]Portnoy L, Eskin E, Stolfo SJ. Intrusion detection with unlabeled data using clustering. In:Proc. of the ACM CSS Workshop on Data Mining Applied to Security (DMSA 2001),2001.5-8
[2]国家计算机网络应急技术处理协调中心.CNCERT/CC 2006年网络安全工作报告.http://www.cert.org.cn,2007-04-06
[3]唐正军,李建华.入侵检测技术.北京：清华大学出版社,2004,9-10
[4]罗守山.入侵检测.北京邮电大学出版社,2004,82-90
[5]廖光忠,陈志凤.入侵检测技术研究综述.网络安全技术与应用,2007,(2),31-33
[6]J.P Anderson, Computer Security Threat Monitoring and Surveil-lance. Fort Washington Pennsylvania, April:James P Anderson Co,1980,35-40
[7]D. Denning. An Intrusion Detection Model. IEEE Transactions on Software Engineering,1987,13(2),222-232
[8]张杰,戴英侠.入侵检测系统技术现状及其发展趋势.计算机与通信,2002,(6),28-30
[9]Phil Porras, Dan Schnackenberg. Stuart Staniford-Chen. The Common Intrusion Detection Framework Architecture,2006,11(6):16-28
[10]叶飞,石福斌.基于异常的入侵检测技术研究.网络安全技术与应用,2007,(6),44-47
[11]Lunt T.F., Tamaru A., Gilham F. et al. A Real-time intrusion detection expert system(IDES). Computer Science Laboratory SRI International, Menlo Park, California, Technical Report,1992,51-53
[12]Anderson Debra, Lunt Teresa F., Javitz Harold, etal. Detecting unusual program behavior using the statistical component of the next-generation expert system(NIDES). Computer Science Laboratory, SRI International, Menlo Park, CA, USA, Technical Report SRI-CSL-95-06,1995,36-40
[13]Smaha S E, Haystack:An intrusion detection system. USA, Orlando FL Dec, 1998,215-241
[14]Hochberg J, Jackson K, Stallings C, et al. NADIR:An Automated System for Detecting Network Intrusion and Misuse. Computers and Security,1993, 12(3):235-248
[15]Todd Heberlein L, Gihan Dias V.KarlLevittN. etal. A Network security monitor, 1991,5(7):101-113
[16]Snapp, S.Brentano, J.Dias, etal. Mansur, D DIDS (Distributed Intrusion Detection System)Motivation, Architecture, and an Early Prototype,1991, 11 (4):79-86
[17]PORTNOY L, ESKIN E, STOLFO S J. Intrusion detection with unlabeled data using clustering. Proceedings of the ACM CSS Workshop on Data Mining App lied to Security. New York,2001,56-60
[18]CH IMPHLEEW, ABDULLAH A H, NOORMD SAPM, et al. Integrating genetic algorithms and fuzzy c2means for anomaly detection. Annual IEEE IND ICON. Washington, DC:IEEE,2005,575-579
[19]KR ISHNAPURAM R, KELLER J M. A possibilistic app roach to clustering. IEEE Transactions on Fuzzy Systems,1993,1 (2):98-110
[20]李庆华,李新,蒋盛益.一种面向高维混合属性数据的异常挖掘算法.计算机应用,2005,25(6),353-356
[21]Babu G P, Murty M N. A near-optimal initial seed value selection in K-means algorithm using a genetic algorithm.Pattern Recognition Letters.1993,14(10): 763-769
[22]SHENG W G, LIU X H. A hybrid algorithm for k-medoid clustering of large data sets. Proceed-ings of IEEE Congress on Evolutionary Computa-tion,2004, 101-105
[23]Ng Raymond T, Jiawei Han. Efficient and effective clustering methods for spatial data mining//Proceedings of the 20th Very Large Databases Conference (VLDB 94), Santiago, Chile,1994,144-155
[24]T Zhang, R Ramakrishnan, M Livny, et al. An efficient data clustering method for very large databases. In:Proc of the 1996 ACM SIGMOD Int'1 Conf on Management of Data. Montreal, Quebec:ACM Press,1996,103-114
[25]S Guha, R Rastogi, K Shim, et al. An efficient clustering algorithm for large databases. In:Proc of 1998 ACM SIGMOD Int'1 Conf on Management of Data. Seattle, Washington:ACM Press,1998,73-84
[26]G Karypis, E H Han, V Kumarl CHAMEL EON. A hierarchical clustering algorithm using dynamic modeling. Computer,1999,32(8),68-75
[27]Rao Xian, Dong Cun-Xi, Yangshao-Quan. Statistic learning and intrusion detection Lecture Notes in Artificial Intelligence.Subseries of Lecture Notes in Computer Science,2003,652-659
[28]LiuZhen, Susan M.Bridges, Rayford B.Vaughn. Classification of Anomalous Traces of Privileged and Parallel Programs by Neural Networks, St.Louis Missouri,2003,1225-1230
[29]Chen Yuju, Huang Tsungchuan, Hwang Rey Chue.An effective learning of neural network by using RFBP learning algorithm.Information Sciences, December 2, 2004,167 (4),77-86
[30]Jin Yaochu, Sendhoff Bernhard. Extracting Interpretable fuzzy Rules from RBF Networks. Neural processing Letters,2003.4,17(2),149-164
[31]Liang Y.C., Feng D.P., Liu G.R., et al. Neural identification of rock Parameters using fuzzy adaptive learning Parameters. Computers and Structures, September, 2003,81(24-25),2373-2382
[32]Andrew Honig, Andrew Howard, Eleazar Eskin, et al. Adaptive Model Generation:An Architecture for Deployment of Data Mining-based intrusion Detection systems. Data Mining for Security Applications-Kluwer,2002
[33]Kaya M., Alhajj R.. Genetic algorithm based framework for mining fuzzy association rules. Fuzzy Sets and Systems, June 16,2005,152(3),587-601
[34]Matsuoka K, Ohya, Kawamoto M. A neural net for blind separation of nonstationary signals.Neural Networks,1995
[35]刘衍珩,田大新,余雪岗,王健.基于分布式学习的大规模网络入侵检测算 法.软件学报,2008,19(4)：993-1003
[36]宿娇娜,李程,李巍,唐发根,李云春.基于改进NB分类方法的网络异常检测模型.计算机工程,2008,34(5)：148-149
[37]李洋,方滨兴,郭莉,陈友.基于直推式方法的网络异常检测方法.软件学报,2007,18(10)：2595-2604
[38]王雷,林亚平,彭雅,李闻,基于认知学习和最小风险的朴素贝叶斯邮件过滤算法.系统仿真学报,2004,16(3)：413-416
[39]Krugel C, Toth T, Kirda E. Service specific anomaly detection for network intrusion detection. In:Lamont GB, Hadda Papadopoulos G, Panda B, eds. Proc. of the 2002 ACM Symp. on Applied Computing. New York:ACM Press, 2002.201-208
[40]Lee W, Stolfo SJ. A framework for constructing features and models for intrusion detection systems. ACM Trans. on Information and System Security, 2000,3(4):227-261
[41]李昆仑,黄厚宽,田盛丰,刘振鹏,刘志强.模糊多类支持向量机及其在入侵检测中的应用.计算机学报,2005,28(2)：274-280
[42]陈治平,王雷.基于密度梯度的聚类算法研究.计算机应用,2006,26(10)：2389-2392
[43]Du, Qiang, Gunzburger Max, Ju, Lili, Wang Xiaoqiang. Centroidal Voronoi Tessellation Algorithms for Image Compression, Segmentation, and Multichannel Restoration. Journal of Mathematical Imaging and Vision,2006, 24(2):177-194
[44]肖立中,邵志清,马汉华,王秀英,刘刚.网络入侵检测中的自动决定聚类数算法.软件学报,2008,19(8)：2140-2148
[45]Portnoy L, Eskin E, Stolfo SJ. Intrusion detection with unlabeled data using clustering. In:Proc. of the ACM CSS Workshop on Data Mining Applied to Security (DMSA 2001),2001.5-8