基于遗传网络的分类规则挖掘在入侵检测系统中的应用
|
摘要
随着Internet的飞速发展,计算机网络安全成为了一个全球性的热点课题。计算机网络是全世界的重要信息基础设施,每年因为计算机网络的安全性遭到破坏而造成的损失都十分巨大。网络安全问题受到了全球的普遍关注。
网络入侵检测系统(Network Intrusion Detection System, NIDS)作为一种主动的信息安全保障措施,有效地弥补了访问控制、防火墙等传统安全保护技术的缺陷,能有效地检测到入侵企图和入侵行为,越来越受到产业界和学术界的关注。作为NIDS的研究,其重点与难点就在于:(1)如何对大规模的网络数据进行处理;(2)如何降低对已知攻击的误报率和漏报率;(3)如何提高对未知新攻击的检测率。
对于NIDS,有着许多不同的研究方法,其中包括智能IDS,例如神经网络、遗传算法、代理技术、免疫系统、数据挖掘等。本研究将提出一种基于遗传网络编程(Genetic Network Programming,GNP)的分类关联规则挖掘方法,并探究这一方法在网络入侵检测系统中的应用。研究内容主要基于NIDS本身的难点的问题,具体的研究工作体现在:(1)利用有效的方法来处理大规模的网络数据,其中包括子属性效用、连续性数据的模糊化等方法;(2)针对已知攻击与未知攻击分别提出了分类关联规则挖掘算法及其相应的分类算法;(3)针对有效降低误报率和漏报率这两项重要指标,提出了可行的算法。
基于上述算法,本文还进一步探究了NIDS系统的效率提高问题,主要包括利用子属性频率机制的特征选取以及模糊分类关联规则挖掘算法的比较与优化,使得NIDS系统的有效性得到提高。
Computer Systems are exposed to an increasing number and type of security threats due to the expanding of internet in recent years. How to detect network intrusions effectively becomes an important techniques. The purpose of our research is to propose a new data mining approach based on Genetic Network Programming (GNP) for the network intrusion detection problem with high detection rate.
This thesis presents a novel fuzzy class association rule mining method based on Genetic Network Programming(GNP), which can be flexibly applied to both misuse and anomaly detection in Network Intrusion Detection Problem. By combining fuzzy set theory with GNP, the proposed method can deal with the mixed database which contains both discrete and continuous attributes. In addition, sub-attribute utilization mechanism is proposed to avoid the information loss. Meanwhile, new GNP structure for association rule mining is build up so as to conduct the rule extraction step. What's more, a new fitness function which provides the fexibility of mining more new rules or mining rules with higher accuracy is given to adapt to different kinds of detection.
After the extraction of class association rules, these rules are used for classification. Two different kinds of classiers are built up respectively for new connection data classication in this research. Experimental results with KDD99Cup and DAPRA98 databases from MIT Lincoln Laboratory show that the proposed method provides a competitively high detection rate compared with other machine learning techniques.
Besides, this thesis will further explore the possibility of improving network intrusion detection systems' efficiency. By applying the characteristic choosing and optimizing Fuzzy class association rule mining algorithm based on GNP, systems' efficiency is gradually improved.
网络入侵检测系统(Network Intrusion Detection System, NIDS)作为一种主动的信息安全保障措施,有效地弥补了访问控制、防火墙等传统安全保护技术的缺陷,能有效地检测到入侵企图和入侵行为,越来越受到产业界和学术界的关注。作为NIDS的研究,其重点与难点就在于:(1)如何对大规模的网络数据进行处理;(2)如何降低对已知攻击的误报率和漏报率;(3)如何提高对未知新攻击的检测率。
对于NIDS,有着许多不同的研究方法,其中包括智能IDS,例如神经网络、遗传算法、代理技术、免疫系统、数据挖掘等。本研究将提出一种基于遗传网络编程(Genetic Network Programming,GNP)的分类关联规则挖掘方法,并探究这一方法在网络入侵检测系统中的应用。研究内容主要基于NIDS本身的难点的问题,具体的研究工作体现在:(1)利用有效的方法来处理大规模的网络数据,其中包括子属性效用、连续性数据的模糊化等方法;(2)针对已知攻击与未知攻击分别提出了分类关联规则挖掘算法及其相应的分类算法;(3)针对有效降低误报率和漏报率这两项重要指标,提出了可行的算法。
基于上述算法,本文还进一步探究了NIDS系统的效率提高问题,主要包括利用子属性频率机制的特征选取以及模糊分类关联规则挖掘算法的比较与优化,使得NIDS系统的有效性得到提高。
Computer Systems are exposed to an increasing number and type of security threats due to the expanding of internet in recent years. How to detect network intrusions effectively becomes an important techniques. The purpose of our research is to propose a new data mining approach based on Genetic Network Programming (GNP) for the network intrusion detection problem with high detection rate.
This thesis presents a novel fuzzy class association rule mining method based on Genetic Network Programming(GNP), which can be flexibly applied to both misuse and anomaly detection in Network Intrusion Detection Problem. By combining fuzzy set theory with GNP, the proposed method can deal with the mixed database which contains both discrete and continuous attributes. In addition, sub-attribute utilization mechanism is proposed to avoid the information loss. Meanwhile, new GNP structure for association rule mining is build up so as to conduct the rule extraction step. What's more, a new fitness function which provides the fexibility of mining more new rules or mining rules with higher accuracy is given to adapt to different kinds of detection.
After the extraction of class association rules, these rules are used for classification. Two different kinds of classiers are built up respectively for new connection data classication in this research. Experimental results with KDD99Cup and DAPRA98 databases from MIT Lincoln Laboratory show that the proposed method provides a competitively high detection rate compared with other machine learning techniques.
Besides, this thesis will further explore the possibility of improving network intrusion detection systems' efficiency. By applying the characteristic choosing and optimizing Fuzzy class association rule mining algorithm based on GNP, systems' efficiency is gradually improved.
引文
[1] A. El-Semaray, J. Edmonds, J. Gonzalez-Pino and M. Papa, "Applying Data Mining of Fuzzy Association Rules to Network Intrusion Detection", In Proc. of the 2006 IEEE Workshop on Information Assurance United States Militaray Academy, West Point, NY, 2006
[2] S. Forrest, S. A. Hofmeyr, A. Somayaji and T. A. Longstaff, "A sense of self for unix processes", In Proc. of the 1996 IEEE Symposium on Security and Privacy, pp. 120-128, Los Almaitos, CA, 1996
[3] J. Luo "Integrating Fuzzy Logic With Data Mining Methods for Intrusion Detection", Master's thesis, Department of Computer Science, Mississippi State University, 1999
[4] K. Shimada, K. Hirasawa and J. Hu, "Genetic Network Programming with Acquisition Mechanisms of Association Rules", Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol. 10, No. 1, pp.102-111, 2006.
[5] DARPA data, http://www.ll.mit.edu.
[6] T. Eguchi, K. Hirasawa, J. Hu and N. Ota, "A study of Evolutionary Multi-agent Models Based on Symbiosis", IEEE Trans. on Syst., Man and Cybernetics Part B, Vol.36, No.1, pp.179-193, 2006.
[7] S. Mabu, K. Hirasawa and J. Hu, "A Graph-Based Evolutionary Algorithm: Genetic Network Programming(GNP) and Its Extension Using Reinforcement Learning", Evolutionary Computation, MIT press, Vol.15, No.3, pp.369-398, 2007.
[8] K. Hirasawa, M. Okubo, H. Katagiri, J. Hu and J. Murata, "Comparison between Genetic Network Programming (GNP) and Genetic Programming(GP)", In Proc. of the Congress of Evolutionary Computation}, pp.1276-1282, 2001.
[9] K. Hirasawa, T. Eguchi, J. Zhou, L. Yu and S. Markon, "A Double-Deck Elevator GroupSupervisory Control System Using Genetic Network Programming", IEEE Trans. on Systems, Man and Cybernetics, PartC, Vol.38, No.4, pp.535-550, 2008.
[10] K. Shimada, K. Hirasawa and J. Hu, "`Class Association Rule Mining with Chi-Squared Test Using Genetic Network Programming", In Proc. of the IEEE SMC 2006, Taipei}, pp.5338-5344, 2006.
[11] W. Lee and S. J. Stolfo, "A Framework for Construction Features and Models for Intrusion Detection System"', ACM Transactions on Information and System security, Vol. 3, No. 4, pp.227-261, November 2000.
[12] Tcptrace software tool, www.tcptrace.org.
[13] W. Lee and S. J. Stolfo, "Data Mining Approaches for Intrusion Detection", In Proc. of the 1998 USENIX Security Symposium,1998.
[14] M. Crosbie and G. Spafford, "Applying genetic programming to intrusion detection", Technical Reportk, FS-95-01, AAAI Fall Symposium Series, AAAI Press, 1995.
[15] W. Lu and I. Traore, "Detecting new forms of network intrusion using genetic programming", Computational Intelligene, Vol. 20, No.3, pp.474-494, 2004.
[16] S. Manganaris, M. Christensen, D. Serkle, and K. Hermix, "A Data Mining Analysis of RTID Alarms", In the Proc. of the 2nd International Workshop on Recent Advances in Intrusion Detection, West Lafayette, 1999
[17] D. E. Denning, "An Intrusion Detection Model", IEEE Transactions on Software Engineering, SE-13:222-232, 1987
[18] R. P. Lippmann, D. J. Fried, I. Graf, J.w. Haines, K. P. Kendall, D.McClung, D. Weber, S.E. Webster, D. Wyschogrod, R. K. Cunningham, and M. A. Zissman, "Evaluating Intrusion Detection Systems: The 1998 DARPA Offline Intrusion Detection Evaluation", In the Proc. of DARPA Information Survivability Conference and Exposition, 2000, Vol. 2, pp. 12-26, IEEE Computer Society Press, 2000
[19] C. C. Aggarwal and P. Yu "Outliers Detection for High Dimensional Data", In the Proc. ofthe ACM SIGMOD Conference, 2001
[20] SNORT Intrusion Detection System. www.snort.org
[21]张然钱,德沛张,文杰,刘轶,奕钟治.入侵检测技术研究综述.小型微型算机系统.2003.7.24(7):P 1113-111
[22]夏煌,郎荣玲,戴冠中.入侵检测系统的智能检测技术研究综述.计算机工程与应用.2001.24:P32-34,118
[23]陈钢,秦茗,张红梅.基于数据挖掘的入侵检测研究.自动化仪表.2006.6.27(6):P14-17,21
[24]刘评,汤志国,于海峰.网络入检测的快速规则匹配算法.海军工程大学学报.2004.9,16(5)
[25]张帆,夏红霞,袁景凌,沈琦.入侵检测系统中关联规则的挖掘,湖北工业大学学报. 2006.3.21(3):PZ15-21
[26]姚淑萍,郑链,刘峰.基于概率模糊认知图的入侵检测警报融合机制.计算机工程, 2005.11.31(27):P118-120
[27]宋世杰.基于序列模式挖掘的误用入侵检测系统及其关键技术研究.国防科技大学工学博士学位论.2005.3:P 50-53
[28]傅忠谦,王再见,李斌,庄镇泉.基于改进GP的入侵检测规则自动提取算法研究.计算机工程.2006.1.32(2):P137-13
[29] [日]玄光男,遗传算法与工程设计,科学出版社,2000年1月第一版
[2] S. Forrest, S. A. Hofmeyr, A. Somayaji and T. A. Longstaff, "A sense of self for unix processes", In Proc. of the 1996 IEEE Symposium on Security and Privacy, pp. 120-128, Los Almaitos, CA, 1996
[3] J. Luo "Integrating Fuzzy Logic With Data Mining Methods for Intrusion Detection", Master's thesis, Department of Computer Science, Mississippi State University, 1999
[4] K. Shimada, K. Hirasawa and J. Hu, "Genetic Network Programming with Acquisition Mechanisms of Association Rules", Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol. 10, No. 1, pp.102-111, 2006.
[5] DARPA data, http://www.ll.mit.edu.
[6] T. Eguchi, K. Hirasawa, J. Hu and N. Ota, "A study of Evolutionary Multi-agent Models Based on Symbiosis", IEEE Trans. on Syst., Man and Cybernetics Part B, Vol.36, No.1, pp.179-193, 2006.
[7] S. Mabu, K. Hirasawa and J. Hu, "A Graph-Based Evolutionary Algorithm: Genetic Network Programming(GNP) and Its Extension Using Reinforcement Learning", Evolutionary Computation, MIT press, Vol.15, No.3, pp.369-398, 2007.
[8] K. Hirasawa, M. Okubo, H. Katagiri, J. Hu and J. Murata, "Comparison between Genetic Network Programming (GNP) and Genetic Programming(GP)", In Proc. of the Congress of Evolutionary Computation}, pp.1276-1282, 2001.
[9] K. Hirasawa, T. Eguchi, J. Zhou, L. Yu and S. Markon, "A Double-Deck Elevator GroupSupervisory Control System Using Genetic Network Programming", IEEE Trans. on Systems, Man and Cybernetics, PartC, Vol.38, No.4, pp.535-550, 2008.
[10] K. Shimada, K. Hirasawa and J. Hu, "`Class Association Rule Mining with Chi-Squared Test Using Genetic Network Programming", In Proc. of the IEEE SMC 2006, Taipei}, pp.5338-5344, 2006.
[11] W. Lee and S. J. Stolfo, "A Framework for Construction Features and Models for Intrusion Detection System"', ACM Transactions on Information and System security, Vol. 3, No. 4, pp.227-261, November 2000.
[12] Tcptrace software tool, www.tcptrace.org.
[13] W. Lee and S. J. Stolfo, "Data Mining Approaches for Intrusion Detection", In Proc. of the 1998 USENIX Security Symposium,1998.
[14] M. Crosbie and G. Spafford, "Applying genetic programming to intrusion detection", Technical Reportk, FS-95-01, AAAI Fall Symposium Series, AAAI Press, 1995.
[15] W. Lu and I. Traore, "Detecting new forms of network intrusion using genetic programming", Computational Intelligene, Vol. 20, No.3, pp.474-494, 2004.
[16] S. Manganaris, M. Christensen, D. Serkle, and K. Hermix, "A Data Mining Analysis of RTID Alarms", In the Proc. of the 2nd International Workshop on Recent Advances in Intrusion Detection, West Lafayette, 1999
[17] D. E. Denning, "An Intrusion Detection Model", IEEE Transactions on Software Engineering, SE-13:222-232, 1987
[18] R. P. Lippmann, D. J. Fried, I. Graf, J.w. Haines, K. P. Kendall, D.McClung, D. Weber, S.E. Webster, D. Wyschogrod, R. K. Cunningham, and M. A. Zissman, "Evaluating Intrusion Detection Systems: The 1998 DARPA Offline Intrusion Detection Evaluation", In the Proc. of DARPA Information Survivability Conference and Exposition, 2000, Vol. 2, pp. 12-26, IEEE Computer Society Press, 2000
[19] C. C. Aggarwal and P. Yu "Outliers Detection for High Dimensional Data", In the Proc. ofthe ACM SIGMOD Conference, 2001
[20] SNORT Intrusion Detection System. www.snort.org
[21]张然钱,德沛张,文杰,刘轶,奕钟治.入侵检测技术研究综述.小型微型算机系统.2003.7.24(7):P 1113-111
[22]夏煌,郎荣玲,戴冠中.入侵检测系统的智能检测技术研究综述.计算机工程与应用.2001.24:P32-34,118
[23]陈钢,秦茗,张红梅.基于数据挖掘的入侵检测研究.自动化仪表.2006.6.27(6):P14-17,21
[24]刘评,汤志国,于海峰.网络入检测的快速规则匹配算法.海军工程大学学报.2004.9,16(5)
[25]张帆,夏红霞,袁景凌,沈琦.入侵检测系统中关联规则的挖掘,湖北工业大学学报. 2006.3.21(3):PZ15-21
[26]姚淑萍,郑链,刘峰.基于概率模糊认知图的入侵检测警报融合机制.计算机工程, 2005.11.31(27):P118-120
[27]宋世杰.基于序列模式挖掘的误用入侵检测系统及其关键技术研究.国防科技大学工学博士学位论.2005.3:P 50-53
[28]傅忠谦,王再见,李斌,庄镇泉.基于改进GP的入侵检测规则自动提取算法研究.计算机工程.2006.1.32(2):P137-13
[29] [日]玄光男,遗传算法与工程设计,科学出版社,2000年1月第一版