基于序列模式挖掘的故障管理系统设计与实现
|
摘要
随着通信和信息技术的飞速发展,现代网络的规模越来越大,网络复杂性日益提高,设备的分散性、接口的多样性以及各设备厂家之间设备的相对独立性,这一切使得建设具备跨平台性、高效等特性的网络故障管理系统的需求越来越明显。
本文参考国内外相关领域最新研究成果,在对现有网络故障管理系统分析的基础上,综合考虑了网络故障管理的任务、告警数据的特点等因素,利用数据挖掘技术中的序列模式挖掘来实现告警相关性分析,制定了较为合理的故障管理业务处理流程,在整体上实现了对网络资源的故障管理功能。本文主要做了如下工作:
(1)根据故障管理系统的功能需求,设计故障管理的业务处理流程、系统体系结构以及数据库结构;
(2)从原理上研究如何建立告警相关性分析模型,着重分析如何针对告警数据进行数据挖掘工作,提出一种新的序列模式挖掘算法,并且基于该算法的框架提出一种增量式告警关联规则挖掘算法,解决了挖掘参数不变、告警数据增加导致的重复挖掘情况,提高数据挖掘的效率;
(3)提出拓扑约束的方法,利用网络中设备之间的拓扑连接知识来对告警关联规则挖掘工作进行约束,减少同步时钟失调情况下挖掘结果的误差;
(4)对系统所采用的序列模式挖掘算法的性能与准确性进行测试,并对系统的各功能进行测试。
With the rapid development of the technology of communication and information, nowadays, the internet has increased fleetly in scale and complexity. The devices have the characteristic of decentralization, the interfaces are diversiform, and the devices from different manufactory are independent comparatively, all above make the demand of building a multi-platform and efficient network fault management system become more and more heavy.
This paper is achieved after consulting the latest pertinent research production and analysing the situation of the network fault management system. After considering the task of network fault management and the characteristic of alarm data synthetically, it completes alarm correlation analysis using mining sequential patterns of the data mining technology, establishes comparatively reasonable processing course of the operations in fault management, and actualize the function of fault management for network resources. In this paper, we innovate in the following points:
(1)Design processing course of the operations, the framework of the system, and the structure of the database, considering the function requirement of the system.
(2)Do research about how to establish the module of alarm correlation analysis, and analyse the method of data mining upon alarm data. Put forward a new mining arithmetic, and then bring forward an incremental mining arithmetic of alarm correlated rules. It resolves the repeated mining situation when the mining parameter is unaltered and alarm data is increased, and the efficiency of data mining is enhanced.
(3)Bring forward the method of topology restriction, restrict the correlated rules mining using the topology connection knowledge between the devices in the network. It can decrease the error of mining result under the circumstance synchoronization clock maladjustment.
(4)Make test to prove the performance and precise of the arithmetic adopted by the system, and test the functions of the system.
本文参考国内外相关领域最新研究成果,在对现有网络故障管理系统分析的基础上,综合考虑了网络故障管理的任务、告警数据的特点等因素,利用数据挖掘技术中的序列模式挖掘来实现告警相关性分析,制定了较为合理的故障管理业务处理流程,在整体上实现了对网络资源的故障管理功能。本文主要做了如下工作:
(1)根据故障管理系统的功能需求,设计故障管理的业务处理流程、系统体系结构以及数据库结构;
(2)从原理上研究如何建立告警相关性分析模型,着重分析如何针对告警数据进行数据挖掘工作,提出一种新的序列模式挖掘算法,并且基于该算法的框架提出一种增量式告警关联规则挖掘算法,解决了挖掘参数不变、告警数据增加导致的重复挖掘情况,提高数据挖掘的效率;
(3)提出拓扑约束的方法,利用网络中设备之间的拓扑连接知识来对告警关联规则挖掘工作进行约束,减少同步时钟失调情况下挖掘结果的误差;
(4)对系统所采用的序列模式挖掘算法的性能与准确性进行测试,并对系统的各功能进行测试。
With the rapid development of the technology of communication and information, nowadays, the internet has increased fleetly in scale and complexity. The devices have the characteristic of decentralization, the interfaces are diversiform, and the devices from different manufactory are independent comparatively, all above make the demand of building a multi-platform and efficient network fault management system become more and more heavy.
This paper is achieved after consulting the latest pertinent research production and analysing the situation of the network fault management system. After considering the task of network fault management and the characteristic of alarm data synthetically, it completes alarm correlation analysis using mining sequential patterns of the data mining technology, establishes comparatively reasonable processing course of the operations in fault management, and actualize the function of fault management for network resources. In this paper, we innovate in the following points:
(1)Design processing course of the operations, the framework of the system, and the structure of the database, considering the function requirement of the system.
(2)Do research about how to establish the module of alarm correlation analysis, and analyse the method of data mining upon alarm data. Put forward a new mining arithmetic, and then bring forward an incremental mining arithmetic of alarm correlated rules. It resolves the repeated mining situation when the mining parameter is unaltered and alarm data is increased, and the efficiency of data mining is enhanced.
(3)Bring forward the method of topology restriction, restrict the correlated rules mining using the topology connection knowledge between the devices in the network. It can decrease the error of mining result under the circumstance synchoronization clock maladjustment.
(4)Make test to prove the performance and precise of the arithmetic adopted by the system, and test the functions of the system.
引文
[1] G. Jakobson, M. D. Weissman. Alarm Correlation. IEEE Network Magazine, 1993, 7(6): 52-59
[2]邓歆,孟洛明.告警相关性分析模型在通信网故障诊断中的应用.北京邮电大学学报, 2006, 29(3): 66-69
[3]郑庆国,吕卫峰.通信网络中的告警相关性研究.计算机工程与应用, 2002, 38(2): 11-14
[4] R. N. Cronk, P. H. Callan, L. Bemstein. Rule Based Expert Systems for Network Management and Operations: An Introduction. IEEE Network Magazine, 1993, 2(5): 233-237
[5] J. Kolodner. Case-Base Reasoning. San Mateo: Morgan kaufmann publishers, 1993: 101-142
[6] S. Kliger, S. Yemini, Y. Yemini et al. in: A coding approach to event correlation. Proceedings of the fourth international symposium on integrated network management IV. London: Chapman & Hall Ltd, 1995: 266-277
[7] E. Aboelela, C. Douligeris. Fuzzy Temporal Reasoning Model for Event Correlation in Network Management. in: Proceedings of the 24th Annual IEEE Conference on Local Computer Networks. Massachusetts: IEEE Computer Society, 1999: 150-159
[8]苏利敏,侯朝桢,戴忠健.指挥控制系统的通信网络故障诊断专家系统.计算机工程, 2003, 29(13): 19-23
[9] Heckerman David, A. Mamdani, M. P. Wellman. Real-world applications of Bayesian networks. Communications of the ACM, 1995, 38(3): 24-26
[10]马江洪,张文修,徐宗本.数据挖掘与数据库知识发现:统计学的观点.工程数学学报, 2002, 19(1): 1-13
[11] Hewlett Packard. HP OpenView event correlation for the telecommunications environment. Technology brief, 1995, 9(1): 23-26
[12] Simona Brugnoni, Guido Bruno, Roberto Manione et al. in: An Expert System for Real-Time Fault Diagnosis of the Italian Telecommunications Network. Proc. 3rd International Symposium on Integrated Network Management. Amsterdam:North-Holland Publishing Co, 1993: 617-628
[13] Kimmo Hatonen, Mika Klemettinen, Heikki Mannila et al. TASA: Telecommunication Alarm Sequence Analyzer, or How to enjoy faults in your network. in: IEEE/IFIP Network Operations and Management Symposium (NOMS’96). Kyoto: IEEE Computer Society, 1996: 520-529
[14] Yechiam Yemini, Shaula Alexander Yemini, Eyal Mozesetal. High Speed and Robust Event Correlation. IEEE Communications Magazine, 1996, 34(5): 82-90
[15] R. D. Gardner, D. A. Harle. Methods and systems for alarm correlation. in: Proceedings of Globecom’96. London: IEEE Computer Society, 1996: 136-140
[16] L. Lewis. Implementing Policy in Enterprise Networks. IEEE Communications Magazine, 1996, 1(34): 50-55
[17] David Hand, Heikki manila, Padhraic Smyth.数据挖掘原理.第一版.北京:机械工业出版社, 2003: 10-107
[18] Jiawei Han, Micheline Kamber.数据挖掘概念与技术.第一版.北京:机械工业出版社, 2001: 149-183
[19]王小虎.关联规则挖掘综述.计算机工程与应用, 2003, 39(33): 190-193
[20] R. Agrawal, T. lmielinski, A. Swami. Mining association rules between sets of items in large databases. in: Peter Buneman, Sushil Jajodia. Proceedings of the ACM SIGMOD Conference on Management of data. New York: ACM Press, 1993: 207-216
[21] R. Agrawal, R. Srikant. Mining Sequential Patterns. in: Philps S. Yu, Arbee L. P. Chen. Proceedings of the Eleventh International Conference on Data Engineering. Taiwan: IEEE Computer Society, 1995: 3-14
[22] H. Mannila, H. Toivonen, A. I. Verkamo. Discovery of frequent episodes in event sequences. Data Mining and Knowledge Discovery, 1997, 1(3): 259-289
[23] David Gallardo, Ed Burnette, Robert Mc Govern. Eclipse in Action. USA: Manning Publications, 2003: 13-143
[24] Craig Walls, Ryan Breidenbach. Spring in Action. USA: Manning Publications, 2005: 3-133
[25]宋汉增,沈琳.利用Hibernate对象持久化服务简化Java数据库访问.计算机应用, 2003, 23(12): 46-48
[26] Steve Holzner. Ant: The Definitive Guide. Second Edition. USA: O’Reilly, 2002: 34-73
[27]陈刚. Eclipse从入门到精通.第一版.北京:清华大学出版社, 2005: 110-180
[28] Kenneth F. Krutsch, David S. Cargo, Virginia Howlett. Java用户界面编程指南.第一版.张伟.北京:电子工业出版社, 2004: 79-325
[29] Erich Gamma, Richard Helm, Ralph Johnson et al.设计模式:可复用面向对象软件的基础.第一版.李英军,马晓星,蔡敏等.北京:机械工业出版社, 2000: 32-45
[30] Rob Harrop, Jan Machacek. Pro Spring. USA: Apress, 2005: 32-240
[31]王全玉,闫波,李凤霞等.基于动态代理提高RMI应用的伸缩性.计算机工程与应用, 2004, 40(12): 94-95
[32] Dave Minter, Jeff Linwood. Pro Hibernate 3. USA: Apress, 2005: 15-78
[33]吴扬扬,陈怀南.基于关联规则的通信网络告警相关性分析模型.通讯和计算机, 2004, 1(1): 57-63
[34]孙朝晖,张德运,李庆海.网络故障管理中的自动告警关联.计算机工程, 2004, 30(5): 30-34
[35]王云岚,朱海平,赵银亮等.网络故障管理专家系统及知识发现系统.微电子学与计算机, 2002, 19(4): 57-59
[36]刘康平,李增智.网络告警序列中的频繁情景规则挖掘算法.小型微型计算机系统, 2003, 24(5): 891-894
[37]冯玉才,冯剑琳.关联规则的增量式更新算法.软件学报, 1998, 9(4): 201-206
[38]王云岚,李增智,屈科文.基于候选项集个数上阶的增量式关联规则更新算法.电子学报, 2004, 32(5): 731-734
[39] D. W. Cheung, J. Han, V. Ng et al. Maintenance of Discovered Association Rules in Large Databases: An Incremental Updating Technique. in: Y. W. Su. Stanley. Proceedings of the 12th International Conference on Data Engineering. New Orleans: IEEE Computer Society, 1996: 106-114
[40] S. D. Lee, D. W. Cheung, B. Kao. A General Incremental Technique for updating Discovered Association Rules. in: R. W. Topor, K. Tanaka. Proceedings of the Fifth International Conference on Database Systems for Advanced Applications. Australia: World Scientific Press, 1997: 185-194
[2]邓歆,孟洛明.告警相关性分析模型在通信网故障诊断中的应用.北京邮电大学学报, 2006, 29(3): 66-69
[3]郑庆国,吕卫峰.通信网络中的告警相关性研究.计算机工程与应用, 2002, 38(2): 11-14
[4] R. N. Cronk, P. H. Callan, L. Bemstein. Rule Based Expert Systems for Network Management and Operations: An Introduction. IEEE Network Magazine, 1993, 2(5): 233-237
[5] J. Kolodner. Case-Base Reasoning. San Mateo: Morgan kaufmann publishers, 1993: 101-142
[6] S. Kliger, S. Yemini, Y. Yemini et al. in: A coding approach to event correlation. Proceedings of the fourth international symposium on integrated network management IV. London: Chapman & Hall Ltd, 1995: 266-277
[7] E. Aboelela, C. Douligeris. Fuzzy Temporal Reasoning Model for Event Correlation in Network Management. in: Proceedings of the 24th Annual IEEE Conference on Local Computer Networks. Massachusetts: IEEE Computer Society, 1999: 150-159
[8]苏利敏,侯朝桢,戴忠健.指挥控制系统的通信网络故障诊断专家系统.计算机工程, 2003, 29(13): 19-23
[9] Heckerman David, A. Mamdani, M. P. Wellman. Real-world applications of Bayesian networks. Communications of the ACM, 1995, 38(3): 24-26
[10]马江洪,张文修,徐宗本.数据挖掘与数据库知识发现:统计学的观点.工程数学学报, 2002, 19(1): 1-13
[11] Hewlett Packard. HP OpenView event correlation for the telecommunications environment. Technology brief, 1995, 9(1): 23-26
[12] Simona Brugnoni, Guido Bruno, Roberto Manione et al. in: An Expert System for Real-Time Fault Diagnosis of the Italian Telecommunications Network. Proc. 3rd International Symposium on Integrated Network Management. Amsterdam:North-Holland Publishing Co, 1993: 617-628
[13] Kimmo Hatonen, Mika Klemettinen, Heikki Mannila et al. TASA: Telecommunication Alarm Sequence Analyzer, or How to enjoy faults in your network. in: IEEE/IFIP Network Operations and Management Symposium (NOMS’96). Kyoto: IEEE Computer Society, 1996: 520-529
[14] Yechiam Yemini, Shaula Alexander Yemini, Eyal Mozesetal. High Speed and Robust Event Correlation. IEEE Communications Magazine, 1996, 34(5): 82-90
[15] R. D. Gardner, D. A. Harle. Methods and systems for alarm correlation. in: Proceedings of Globecom’96. London: IEEE Computer Society, 1996: 136-140
[16] L. Lewis. Implementing Policy in Enterprise Networks. IEEE Communications Magazine, 1996, 1(34): 50-55
[17] David Hand, Heikki manila, Padhraic Smyth.数据挖掘原理.第一版.北京:机械工业出版社, 2003: 10-107
[18] Jiawei Han, Micheline Kamber.数据挖掘概念与技术.第一版.北京:机械工业出版社, 2001: 149-183
[19]王小虎.关联规则挖掘综述.计算机工程与应用, 2003, 39(33): 190-193
[20] R. Agrawal, T. lmielinski, A. Swami. Mining association rules between sets of items in large databases. in: Peter Buneman, Sushil Jajodia. Proceedings of the ACM SIGMOD Conference on Management of data. New York: ACM Press, 1993: 207-216
[21] R. Agrawal, R. Srikant. Mining Sequential Patterns. in: Philps S. Yu, Arbee L. P. Chen. Proceedings of the Eleventh International Conference on Data Engineering. Taiwan: IEEE Computer Society, 1995: 3-14
[22] H. Mannila, H. Toivonen, A. I. Verkamo. Discovery of frequent episodes in event sequences. Data Mining and Knowledge Discovery, 1997, 1(3): 259-289
[23] David Gallardo, Ed Burnette, Robert Mc Govern. Eclipse in Action. USA: Manning Publications, 2003: 13-143
[24] Craig Walls, Ryan Breidenbach. Spring in Action. USA: Manning Publications, 2005: 3-133
[25]宋汉增,沈琳.利用Hibernate对象持久化服务简化Java数据库访问.计算机应用, 2003, 23(12): 46-48
[26] Steve Holzner. Ant: The Definitive Guide. Second Edition. USA: O’Reilly, 2002: 34-73
[27]陈刚. Eclipse从入门到精通.第一版.北京:清华大学出版社, 2005: 110-180
[28] Kenneth F. Krutsch, David S. Cargo, Virginia Howlett. Java用户界面编程指南.第一版.张伟.北京:电子工业出版社, 2004: 79-325
[29] Erich Gamma, Richard Helm, Ralph Johnson et al.设计模式:可复用面向对象软件的基础.第一版.李英军,马晓星,蔡敏等.北京:机械工业出版社, 2000: 32-45
[30] Rob Harrop, Jan Machacek. Pro Spring. USA: Apress, 2005: 32-240
[31]王全玉,闫波,李凤霞等.基于动态代理提高RMI应用的伸缩性.计算机工程与应用, 2004, 40(12): 94-95
[32] Dave Minter, Jeff Linwood. Pro Hibernate 3. USA: Apress, 2005: 15-78
[33]吴扬扬,陈怀南.基于关联规则的通信网络告警相关性分析模型.通讯和计算机, 2004, 1(1): 57-63
[34]孙朝晖,张德运,李庆海.网络故障管理中的自动告警关联.计算机工程, 2004, 30(5): 30-34
[35]王云岚,朱海平,赵银亮等.网络故障管理专家系统及知识发现系统.微电子学与计算机, 2002, 19(4): 57-59
[36]刘康平,李增智.网络告警序列中的频繁情景规则挖掘算法.小型微型计算机系统, 2003, 24(5): 891-894
[37]冯玉才,冯剑琳.关联规则的增量式更新算法.软件学报, 1998, 9(4): 201-206
[38]王云岚,李增智,屈科文.基于候选项集个数上阶的增量式关联规则更新算法.电子学报, 2004, 32(5): 731-734
[39] D. W. Cheung, J. Han, V. Ng et al. Maintenance of Discovered Association Rules in Large Databases: An Incremental Updating Technique. in: Y. W. Su. Stanley. Proceedings of the 12th International Conference on Data Engineering. New Orleans: IEEE Computer Society, 1996: 106-114
[40] S. D. Lee, D. W. Cheung, B. Kao. A General Incremental Technique for updating Discovered Association Rules. in: R. W. Topor, K. Tanaka. Proceedings of the Fifth International Conference on Database Systems for Advanced Applications. Australia: World Scientific Press, 1997: 185-194