警报关联分析工具的研究及改进
摘要
入侵检测的研究已经进行了20多年,入侵检测系统是一种主动保护计算机免受入侵者攻击的新型计算机网络安全系统,提供了对内部攻击外部攻击和误操作的实时保护,在网络系统受到危害之前拦截和响应入侵。但是传统的IDS与有两个主要的缺陷,一个是大多数IDS只检测底层的攻击和异常,尽管产生的警报在逻辑上是有关联的,但是IDS只单独的产生警报。另一个是IDS产生大量的误报警,混在真的警报中让人们无法分辨,IDS的误报和漏报问题一直没有很好的解决方法。因此,警报关联的研究越来越被人们所重视,在最近几年,警报关联的研究越来越活跃,一种基于先决条件的报警关联方法被提出,这种方法的优点是:第一,提供了相关警报的一个高层次表示,揭示了一系列攻击的结构;第二,由于只保持相关警报,能减少误报的影响;第三,可能用于预报正在进行中的攻击,允许入侵响应系统采取相应动作停止正在进行的攻击。
TIAA是一种离线警报分析工具,有三个子系统组成:警报收集子系统,警报相关子系统,交互式分析系统,他们以知识库和数据库为核心。本文从警报关联的研究背景和意义出发,分析了警报错误率高的原因,介绍了警报关联框架及警报关联相关知识,详细阐述了警报关联的几个效用的功能。详细介绍了TIAA的安装使用方法,并使用DARPA2000数据集进行了测试。由于分布式入侵检测系统中传感器数量和种类很多,产生的冗余警报比较多,为了减少冗余警报,本文改进了频繁闭模式数据挖掘算法,引入了模糊集合的概念,使用该算法对警报数据进行挖掘,达到减少冗余警报的目的。
Intrusion Detection has been studied for about twenty years. Intrusion Detection Systems (IDSs) are usually considered the second line of defense to protect against malicious activities along with the prevention-based security mechanisms such as authentication and access control. However, traditional IDSs have two major weaknesses. First, they usually focus on low-level attacks or anomalies, and raise alerts independently, though the re may be logical connections between them. Second, there are a lot of false alerts reported by traditional IDSs, which are mixed with true alerts. We propose a technique to correlate the alerts by using their prerequisites and consequences in order to solve the problem. The method using prerequisites and consequences has three advantages: 1) provide a higher level scenarios of correlated alerts and implicate the structure of the attack.2) It can reduce the rate of producing false alerts after the attention is focused on alerts that are correlated with others.3) while someone is attacking, it can preview the attacking and prevent the attacking by IDS.
We propose a framework of correlating alerts, which contains four parts: prerequisite and consequence of attacks, hyper-alert type and hyper- -alert, hyper-alert correlation graph, utilities for interactively analyzing alerts. Predicates are the basic constructs to represent prerequisites and consequences of attacks. For example, a scanning attack may discover UDP services vulnerable to a certain buffer overflow attack. We can use the predicate UDPVulnerableToBOF (VictimIP, VictimPort) to represent the attacker’s discovery. Similarly, if an attack requires a UDP service vulnerable to the buffer overflow attack, we can use the same predicate to represent the prerequisite. A hyper-alert type T is a triple (fact, prerequi- -site, consequence), where (1) fact is a set of attribute names, each with an associated domain of values, (2) prerequisite is a logical combination of predicates whose free variables are all in fact, and (3) consequence is a set of predicates such that all the free variables in consequence are in fact. The hyper-alert correlation graph is not only an intuitive representation of attack scenarios constructed through alert correlation, but also reveals opportunities to improve intrusion detection. First, the hyper-alert corre lation graph can potentially reveal the intrusion strategies behind the attacks, and lead to better understanding of the attacker’s intention. Second, assuming some attackers exhibit patterns in their strategies, we can use the hyper-alert correlation graph to profile previous attacks and identify on-going attacks by matching to the profiles. A partial match to the profile may indicate attacks possibly missed by the IDSs, and lead to human investigation and improvement of the IDSs. Utilities can help analysts get as much information as possible and make the best judgment. These utilities are then integrated into one system (which we will present in the next section), which provides human analysts platform to examine correlated intrusion alerts interactively and progressively. TIAA is an off-line toolkit for analyzing the alerts of IDS. TIAA contains three Subsystems: Alert Collection Subsystem, Alert Correlation Subsystem, and Interactive Alert Analysis Subsystem. TIAA is implemented in Java, with JDBC to access the database. To save development effort, TIAA uses the GraphViz package as the visualization engine to generate the graphical representation of the analysis results. TIAA relies on a knowledge base for prior knowledge about different types of alerts as well as implication relationships between predicates. Because of the need for human analysts to write and possibly revise the knowledge base, the knowledge base is represented in an XML format. TIAA uses the Apache Xerces2 Java Parser [Xer ] to facilitate the manipulation of the knowledge base.
Data mining is widely used in kinds of area. In the research of IDS, Data mining is also a very important subject. We improve the method of close frequent pattern to mining the data of alerts generated by IDS. It can reduce the misuse alerts of IDS.
This paper analysis the reason of the high rate of false alerts, presents the framework of alert collection and other related knowledge. This paper introduce the utilities of alert collection particularly, and the way of using and installing TIAA. This paper also presents a method of data mining which called close frequent pattern method to improve the function of TIAA.
TIAA是一种离线警报分析工具,有三个子系统组成:警报收集子系统,警报相关子系统,交互式分析系统,他们以知识库和数据库为核心。本文从警报关联的研究背景和意义出发,分析了警报错误率高的原因,介绍了警报关联框架及警报关联相关知识,详细阐述了警报关联的几个效用的功能。详细介绍了TIAA的安装使用方法,并使用DARPA2000数据集进行了测试。由于分布式入侵检测系统中传感器数量和种类很多,产生的冗余警报比较多,为了减少冗余警报,本文改进了频繁闭模式数据挖掘算法,引入了模糊集合的概念,使用该算法对警报数据进行挖掘,达到减少冗余警报的目的。
Intrusion Detection has been studied for about twenty years. Intrusion Detection Systems (IDSs) are usually considered the second line of defense to protect against malicious activities along with the prevention-based security mechanisms such as authentication and access control. However, traditional IDSs have two major weaknesses. First, they usually focus on low-level attacks or anomalies, and raise alerts independently, though the re may be logical connections between them. Second, there are a lot of false alerts reported by traditional IDSs, which are mixed with true alerts. We propose a technique to correlate the alerts by using their prerequisites and consequences in order to solve the problem. The method using prerequisites and consequences has three advantages: 1) provide a higher level scenarios of correlated alerts and implicate the structure of the attack.2) It can reduce the rate of producing false alerts after the attention is focused on alerts that are correlated with others.3) while someone is attacking, it can preview the attacking and prevent the attacking by IDS.
We propose a framework of correlating alerts, which contains four parts: prerequisite and consequence of attacks, hyper-alert type and hyper- -alert, hyper-alert correlation graph, utilities for interactively analyzing alerts. Predicates are the basic constructs to represent prerequisites and consequences of attacks. For example, a scanning attack may discover UDP services vulnerable to a certain buffer overflow attack. We can use the predicate UDPVulnerableToBOF (VictimIP, VictimPort) to represent the attacker’s discovery. Similarly, if an attack requires a UDP service vulnerable to the buffer overflow attack, we can use the same predicate to represent the prerequisite. A hyper-alert type T is a triple (fact, prerequi- -site, consequence), where (1) fact is a set of attribute names, each with an associated domain of values, (2) prerequisite is a logical combination of predicates whose free variables are all in fact, and (3) consequence is a set of predicates such that all the free variables in consequence are in fact. The hyper-alert correlation graph is not only an intuitive representation of attack scenarios constructed through alert correlation, but also reveals opportunities to improve intrusion detection. First, the hyper-alert corre lation graph can potentially reveal the intrusion strategies behind the attacks, and lead to better understanding of the attacker’s intention. Second, assuming some attackers exhibit patterns in their strategies, we can use the hyper-alert correlation graph to profile previous attacks and identify on-going attacks by matching to the profiles. A partial match to the profile may indicate attacks possibly missed by the IDSs, and lead to human investigation and improvement of the IDSs. Utilities can help analysts get as much information as possible and make the best judgment. These utilities are then integrated into one system (which we will present in the next section), which provides human analysts platform to examine correlated intrusion alerts interactively and progressively. TIAA is an off-line toolkit for analyzing the alerts of IDS. TIAA contains three Subsystems: Alert Collection Subsystem, Alert Correlation Subsystem, and Interactive Alert Analysis Subsystem. TIAA is implemented in Java, with JDBC to access the database. To save development effort, TIAA uses the GraphViz package as the visualization engine to generate the graphical representation of the analysis results. TIAA relies on a knowledge base for prior knowledge about different types of alerts as well as implication relationships between predicates. Because of the need for human analysts to write and possibly revise the knowledge base, the knowledge base is represented in an XML format. TIAA uses the Apache Xerces2 Java Parser [Xer ] to facilitate the manipulation of the knowledge base.
Data mining is widely used in kinds of area. In the research of IDS, Data mining is also a very important subject. We improve the method of close frequent pattern to mining the data of alerts generated by IDS. It can reduce the misuse alerts of IDS.
This paper analysis the reason of the high rate of false alerts, presents the framework of alert collection and other related knowledge. This paper introduce the utilities of alert collection particularly, and the way of using and installing TIAA. This paper also presents a method of data mining which called close frequent pattern method to improve the function of TIAA.
引文
[1]. James P. Anderson. Computer security threat monitoring and surveillance. Washington: Anderson Co, Tech. Rep.: TR80904,1980.
[2]. Mukherjee, B., Heberlein, L. T., and Levitt, K. N. 1994. Network intrusion detection. IEEE Network 8, 3 (May), 26–41.
[3]. Bace, R. 2000. Intrusion Detection. Macmillan Technology Publishing.
[4]. Staniford, S., Hoagland, J., and McAlerney, J. 2002. Practical automated detection of stealthy portscans. Journal of Computer Security 10, 1/2, 105–136.
[5]. Valdes, A. and Skinner, K. 2001. Probabilistic alert correlation. In Proceedings of the 4th International Symposium on Recent Advances in Intrusion Detection (RAID 2001). 54–68.
[6]. Cuppens, F. 2001. Managing alerts in a multi-intrusion detection environment. In Proceedings of the 17th Annual Computer Security Applications Conference.
[7]. Julisch, K. 2001. Mining alarm clusters to improve alarm handling efficiency. In Proceedings of the 17th Annual Computer Security Applications Conference (ACSAC). 12–21.
[8]. Debar, H. and Wespi, A. 2001. Aggregation and correlation of intrusion-detection alerts. In Recent Advances in Intrusion Detection. LNCS 2212. 85 – 103.
[9]. Templeton, S. and Levitt, K. 2000. A requires/provides model for computer attacks. In Proceedings of New Security Paradigms Workshop. ACM Press, 31 – 38.
[10]. Cuppens, F. and Miege, A. 2002. Alert correlation in a cooperative intrusion detection frame-work. In Proceedings of the 2002 IEEE Symposium on Security and Privacy.
[11]. STEVEN E, SMAHA. Haystack: an intrusion detection system [A]. Proceedings of the Fourth Aerospace Computer Security Applications Conference[C]. Washington: IEEE Computer Society Press, 1988.37-44.
[12]. Curry, D. and Debar, H. 2001. Intrusion detection message exchange format data model and extensible markup language (xml) document type definition. Internet Draft, draft-ietf-idwg-idmef-xml-03.txt.
[13]. Han J, Pei J, Yin W. Mining Frequent Paterns without Candidate Generation[ J].A CM SIGMOD Record,20 00,2 9(2):1-12.
[14]. .苏 超., Zuo Wan-li.左万利..基于关联规则的分类,吉林大学学报自然科学版, 2001, (1): 31～35.
[15]. Peng Ning, Yun Cui, Douglas S. Reeves, and Dingbang Xu North Carolina State University著《Techniques and Tools for Analyzing Intrusion Alerts》ACM Transactions on Information and System Security, May 2004,Vol. 7, No. 2.
[16]. North Carolina StateUniversity,《TIAA: A Toolkit for Intrusion Alert Analysis (Version1.0)Installation,2002 and Operation Manual》
[17]. Fabien Pouget, Marc Dacier 著《White Paper: Alert CorrelatioReview of the state of the art 》,November 28, 2003.
[18]. 李雪莹,刘宝旭,毕学尧,安德海,许榕生《对入侵检测警报关联分析的研究和实践》,计算机工程与应用2003,19.
[19]. PENG NING and DINGBANG XU .Hypothesizing and Reasoning about Attacks Missed by Intrusion Detection Systems .North Carolina State University.
[20]. Cui, Y. A toolkit for intrusion alerts correlation based on prerequisites and consequences of attacks. M.S. thesis, North Carolina StateUniversity. 2002.
[21]. 杨华兵,叶新郢,张宁蓉. 入侵检测中频繁模式的有效挖掘算法. 情报指挥控制系统与仿真技术2005,27(1).
[22]. 易月娥,林亚平,王永红. 基于FP—tree挖掘密集型数据最大频繁 模式算法. 湖南城市学院学报(自然科学版)2007,16(1).
[23]. 李忠哗,任春龙,何丕廉.种基于FP-树的最大频繁模式增量更新挖 掘算法. 计算机应用与软件.2007,24(5).
[24]. TIAA下载http://discovery.csc.ncsu.edu/software.html.
[25]. 陆 楠,周春光.基于F P-tree频集模式的FP-Growth算法对关联规则挖掘的影响. 吉林大学学报,2003,Vol.41.
[2]. Mukherjee, B., Heberlein, L. T., and Levitt, K. N. 1994. Network intrusion detection. IEEE Network 8, 3 (May), 26–41.
[3]. Bace, R. 2000. Intrusion Detection. Macmillan Technology Publishing.
[4]. Staniford, S., Hoagland, J., and McAlerney, J. 2002. Practical automated detection of stealthy portscans. Journal of Computer Security 10, 1/2, 105–136.
[5]. Valdes, A. and Skinner, K. 2001. Probabilistic alert correlation. In Proceedings of the 4th International Symposium on Recent Advances in Intrusion Detection (RAID 2001). 54–68.
[6]. Cuppens, F. 2001. Managing alerts in a multi-intrusion detection environment. In Proceedings of the 17th Annual Computer Security Applications Conference.
[7]. Julisch, K. 2001. Mining alarm clusters to improve alarm handling efficiency. In Proceedings of the 17th Annual Computer Security Applications Conference (ACSAC). 12–21.
[8]. Debar, H. and Wespi, A. 2001. Aggregation and correlation of intrusion-detection alerts. In Recent Advances in Intrusion Detection. LNCS 2212. 85 – 103.
[9]. Templeton, S. and Levitt, K. 2000. A requires/provides model for computer attacks. In Proceedings of New Security Paradigms Workshop. ACM Press, 31 – 38.
[10]. Cuppens, F. and Miege, A. 2002. Alert correlation in a cooperative intrusion detection frame-work. In Proceedings of the 2002 IEEE Symposium on Security and Privacy.
[11]. STEVEN E, SMAHA. Haystack: an intrusion detection system [A]. Proceedings of the Fourth Aerospace Computer Security Applications Conference[C]. Washington: IEEE Computer Society Press, 1988.37-44.
[12]. Curry, D. and Debar, H. 2001. Intrusion detection message exchange format data model and extensible markup language (xml) document type definition. Internet Draft, draft-ietf-idwg-idmef-xml-03.txt.
[13]. Han J, Pei J, Yin W. Mining Frequent Paterns without Candidate Generation[ J].A CM SIGMOD Record,20 00,2 9(2):1-12.
[14]. .苏 超., Zuo Wan-li.左万利..基于关联规则的分类,吉林大学学报自然科学版, 2001, (1): 31～35.
[15]. Peng Ning, Yun Cui, Douglas S. Reeves, and Dingbang Xu North Carolina State University著《Techniques and Tools for Analyzing Intrusion Alerts》ACM Transactions on Information and System Security, May 2004,Vol. 7, No. 2.
[16]. North Carolina StateUniversity,《TIAA: A Toolkit for Intrusion Alert Analysis (Version1.0)Installation,2002 and Operation Manual》
[17]. Fabien Pouget, Marc Dacier 著《White Paper: Alert CorrelatioReview of the state of the art 》,November 28, 2003.
[18]. 李雪莹,刘宝旭,毕学尧,安德海,许榕生《对入侵检测警报关联分析的研究和实践》,计算机工程与应用2003,19.
[19]. PENG NING and DINGBANG XU .Hypothesizing and Reasoning about Attacks Missed by Intrusion Detection Systems .North Carolina State University.
[20]. Cui, Y. A toolkit for intrusion alerts correlation based on prerequisites and consequences of attacks. M.S. thesis, North Carolina StateUniversity. 2002.
[21]. 杨华兵,叶新郢,张宁蓉. 入侵检测中频繁模式的有效挖掘算法. 情报指挥控制系统与仿真技术2005,27(1).
[22]. 易月娥,林亚平,王永红. 基于FP—tree挖掘密集型数据最大频繁 模式算法. 湖南城市学院学报(自然科学版)2007,16(1).
[23]. 李忠哗,任春龙,何丕廉.种基于FP-树的最大频繁模式增量更新挖 掘算法. 计算机应用与软件.2007,24(5).
[24]. TIAA下载http://discovery.csc.ncsu.edu/software.html.
[25]. 陆 楠,周春光.基于F P-tree频集模式的FP-Growth算法对关联规则挖掘的影响. 吉林大学学报,2003,Vol.41.