数据挖掘技术在网络故障诊断中的应用研究
|
摘要
随着网络规模不断扩大,网络复杂性不断增加,网络故障问题越来越突出。本文针对传统故障诊断中存在的问题,深入地研究了关联规则挖掘与分类挖掘两种数据挖掘方法,并应用于故障诊断中,实现了对网络故障的智能诊断。
首先在基于数据挖掘的网络故障诊断定位模型基础上,考虑了网络故障告警信息的群集特点,对基于频繁模式树(FP-tree)的关联规则挖掘算法进行改进,提出了基于频繁模式树的故障群集的诊断定位挖掘FP-treeC算法；根据网络故障告警信息增加的特点,提出了基于频繁模式树的告警数据库中数据增加时关联规则定时增量挖掘FP-treeCT算法；对ID3的决策树挖掘算法进行了改进,实验表明三个算法都在原算法上得到了效率的提高；最后设计了针对某些特定类型的网络故障的一种基于策略的网络故障修复模型PBFR。
As the networks have been expanding ever-increasing network complexity, network failures become increasingly acute. Aiming at analyzing the problems in traditional fault diagnosis, this paper presents two kinds of in-depth data mining methods:the association rule mining and classification mining. And they were applied in fault diagnosis to implement intelligent diagnosis of network faults.
In the basis of network fault diagnosis which is based on location model-based data mining, together with considering the network fault alarm cluster features, the association rule mining based frequent pattern tree (FP-tree) algorithm is improved, and FP-treeC mining algorithm is proposed to improve the fault diagnosis and location for cluster; according to the network fault features that alarm information increases continuously, association rules incremental update problem about fault diagnosis and location are studied, and an improved timer FP-treeCT incremental updating algorithm for mining association rules is proposed; based on the existing ID3 decision tree mining algorithms, the IID3 algorithm is applied to network fault diagnosis and location. The paper also proposes a plan-based fault restoration model for certain specific types of network fault.
首先在基于数据挖掘的网络故障诊断定位模型基础上,考虑了网络故障告警信息的群集特点,对基于频繁模式树(FP-tree)的关联规则挖掘算法进行改进,提出了基于频繁模式树的故障群集的诊断定位挖掘FP-treeC算法；根据网络故障告警信息增加的特点,提出了基于频繁模式树的告警数据库中数据增加时关联规则定时增量挖掘FP-treeCT算法；对ID3的决策树挖掘算法进行了改进,实验表明三个算法都在原算法上得到了效率的提高；最后设计了针对某些特定类型的网络故障的一种基于策略的网络故障修复模型PBFR。
As the networks have been expanding ever-increasing network complexity, network failures become increasingly acute. Aiming at analyzing the problems in traditional fault diagnosis, this paper presents two kinds of in-depth data mining methods:the association rule mining and classification mining. And they were applied in fault diagnosis to implement intelligent diagnosis of network faults.
In the basis of network fault diagnosis which is based on location model-based data mining, together with considering the network fault alarm cluster features, the association rule mining based frequent pattern tree (FP-tree) algorithm is improved, and FP-treeC mining algorithm is proposed to improve the fault diagnosis and location for cluster; according to the network fault features that alarm information increases continuously, association rules incremental update problem about fault diagnosis and location are studied, and an improved timer FP-treeCT incremental updating algorithm for mining association rules is proposed; based on the existing ID3 decision tree mining algorithms, the IID3 algorithm is applied to network fault diagnosis and location. The paper also proposes a plan-based fault restoration model for certain specific types of network fault.
引文
[1]International Standards Organization. Information processing systems; Open Systems Interconnection; basic reference model; Part 4:Management framework ISO7498-4 [S].1989.
[2]Denise W,Irfan Khan,Ogier R,et al.An Artificial Intelligence Approach to Network Fault Management[EB].http://citeseer.nj.nec.com/105695.html,2003.
[3]Lewis L.A Case-based Reasoning Approach to the Management of Faults in Communication Network[C].in INFOCOM'93.San Francisco:1993.
[4]梁循.数据挖掘算法与应用[M].北京：北京大学出版社,2006.4.
[5]Quinlan J R.Induction of Decision Trees[J].Machine Learning.1986,1:81-106.
[6]曲朝阳,高宇峰,聂欣.基于决策树的网络故障诊断专家系统模型[J].计算机工程,2008.11.
[7]王莎莎,孙继银,李琳琳.故障决策树模型在诊断专家系统中的应用[J].计算机应用,2005.12.
[8]Kliger S,Yemini S,Yemini Y,et al.A Coding Approach to Event Correlation[C].in Intelligent NetworkManagement. Santa Barbara,CA:1995.
[9]Martin T.Hagan,Howard B.Demuth,Mark H.Beale.Neural Network Design[M].PWS Publishing Company.1996.
[10]Marilly E,Aghasaryan A,Betge-Brezetz S,et al.Alarm Correlation for Complex Telecommunication Network Using Neural Network and Signal Processing[C].in IEEE Workshop on Operation and Management,2002.
[11]苏利敏,候朝桢,戴忠健,潘秀琴.基于神经网络的告警关联[J].北京理工大学学报,2002.6.
[12]李静.基于神经网络和遗传算法的Ad hoc网络故障管理模型研究[D].成都：电子科技大学,2008.4.
[13]肖伟.数据挖掘在网络故障诊断中的应用[D].南京：南京理工大学.2004.6.
[14]Deng R H,Lazar A A,Wang W.A Probabilistic Approach to Fault Diagnosis in Linear Lightwave Network[C].IEEE Journal on Selected Areas in Communications,1993,11 (9):1438.
[15]肖秦琨,高嵩,高晓光.动态贝叶斯网络推理学习理论及应用[M].北京：国防工业出版社,2007.
[16]李千目.战略互联网故障智能诊断策略研究[D].南京市：南京理工大学,2005.
[17]翁永前.基于免疫原理的网络故障检测系统模型的研究[D].成都：电子科技大学,2008.
[18]M.Thottan and C.Ji.Statistical Detectionof Enterprise NetWork Problems[J].Joumal of Network and Systems Management,1999.
[19]李鹏.基于事件关联的网络故障管理研究[D].湖南：中南大学,2008.
[20]Jakobson G.Weissman M.Alarm Correlation[J].IEEE Network,1993,7(6):52-59.
[21]Dupuy A,Schwartz J,Yemini Y,et al.Network Fault Management:A user's View[C].in Integrated Network Management.North Holland:Elsevier Science Publishers B.V,1989.
[22]Katzela.l.Fault Diagnosis in Telecommunications Network[D].Columbia University,1996.
[23]胡谷雨.网络管理技术教程[M].北京：希望电子出版社,2002.9.23-24.
[24]Miseha Schartz.Broadband Integrated Networks[M].Prentiee Hall,1998.
[25]郭军.网络管理[M].第3版.北京：北京邮电大学出版社,2008.247-249.
[26]A.Bouloutas,S.Calo,and A.Finkel.Alarm Correlation and Fault Identification in Communication Networks[J].IEEE Transactions on Communications, 42:523-533,1994.
[27]高旭麟.基于SNMPv3的网络视频监控管理系统研究与实现[D].江南大学硕士学位论文.2008.
[28]Rakesh Agrawal,Giuseppe psaila.Active Data Mining[C].1st International Conference on Knowledge Discovery and Da a Mining,Menlo Park,Calif,1995.
[29]毛国君,段立娟,王实等.数据挖掘原理与算法[M].北京：清华大学出版社,2007,10-21.
[30]中国科学技术协会.2006-2007计算机科学学科发展报告.北京：中国科学技术出版社,2007,138-143.
[31]Borzsonyi S,Kossmann D,Stocker K.The Skyline Operator[G].In:Proc.of the 17th Int'1 Conf.on Data Engineering. Heidelberg:IEEE Computer Society Press, 2001,421-430.
[32]夏海涛,詹志强.新一代网络管理技术[M].北京：北京邮电大学出版社,2003.
[33]林曼筠.可扩展的计算机网络管理系统技术研究[D].北京市：中国科学院研究生院(计算技术研究所),2002.
[34]郭道荣.基于数据挖掘的电信网络故障诊断技术的研究[D].重庆大学硕士学位论文.2003.
[35]Han J,Jian P,Yiwen Y.Mining Frequent Patterns Without Candidate Generation[C]. In:Proceedings of the 2000 ACM SIGMOD International Conference Management of Data. Dallas,2000.1-12.
[36]毛国君,段立娟,王实等.数据挖掘原理与算法[M].北京：清华大学出版社,2007,69.
[37]Han J,Kamber M.Data Mining:Concepts and Techniques[M].Beijing:High Education Press,2001.
[38]端义锋,胡谷雨,丁力.序列模式挖掘在网络告警分析中的应用[J].北京邮电大学学报.2004.27(12).
[39]Cheung D.W.Maintenance of Discovered Association Rules in Large Databases:An Incremental Updating Technique[C].In:Proceedings of the 1996 International Conference on Data Engineering, New Orleans,Louisiana,1996.
[40]Cheung D.W.et al.. A General Incremental Technique for Updating Discovered Assoeiation Rules[C].In:Proceedings of the 1997 International Conference on Databases Systems for Advanced Application.Melbourne,1997:185-194.
[41]冯玉才,冯剑琳.关联规则的增量式更新算法[J].软件学报,1998.9(4)：301-306.
[42]杨学兵,宋红梅.一种高效的关联规则增量式更新算法[J].计算机技术与发展.2007.
[43]宋余庆,朱玉全,孙志挥等.基于FP-Tree的最大频繁项目集挖掘及更新算法[J].软件学报.2003,14(9)：1586-1592.
[44]宋余庆,朱玉全,孙志挥等.一种基于频繁模式树的约束最大频繁项目集挖掘及其更新算法[J].计算机研究与发展,2005,42(5)：777-783.
[45]吉根林,杨明,宋余庆等.最大频繁项目集的快速更新[J].计算机学报,2005,28(1)：128-135.
[46]徐前方.基于数据挖掘的网络故障告警相关性研究[D].北京邮电大学.2007.
[47]Quinlan J R. Induction of decision trees[J].Machine Learning.1986,(4):81-106.
[48]Quinlan J R. C4.5 Programs for Machine Learning[M].San Mateo:Morgan Kaufmann Publishers, lnc,1993.
[49]王妍妍,王艳宁,王敏.基于单变量决策树的网络故障诊断方法[J].计算机机工程与设计,2007.
[50]刘康平.面向网络故障管理的知识发现方法[D].西安交通大学博士学位论文.2001.
[51]R.Srikant,R.Agrawal.Mining Generalized association rules[J].In:Proc.Of the 21st Int'l conf.On Very Large Database,Zurich,Switzerland,1995.
[52]P.Smyth and R.M.Goodman.An information theoretic approach to rule induction from databases[J].IEEE Transactions on Knowledge and DataEngineering.1992.
[53]郭军.网络管理[M].第3版.北京：北京邮电大学出版社,2008.295-297.
[54]刘康平.面向网络故障管理的知识发现方法[D].西安交通大学博士学位论文,2001.
[2]Denise W,Irfan Khan,Ogier R,et al.An Artificial Intelligence Approach to Network Fault Management[EB].http://citeseer.nj.nec.com/105695.html,2003.
[3]Lewis L.A Case-based Reasoning Approach to the Management of Faults in Communication Network[C].in INFOCOM'93.San Francisco:1993.
[4]梁循.数据挖掘算法与应用[M].北京：北京大学出版社,2006.4.
[5]Quinlan J R.Induction of Decision Trees[J].Machine Learning.1986,1:81-106.
[6]曲朝阳,高宇峰,聂欣.基于决策树的网络故障诊断专家系统模型[J].计算机工程,2008.11.
[7]王莎莎,孙继银,李琳琳.故障决策树模型在诊断专家系统中的应用[J].计算机应用,2005.12.
[8]Kliger S,Yemini S,Yemini Y,et al.A Coding Approach to Event Correlation[C].in Intelligent NetworkManagement. Santa Barbara,CA:1995.
[9]Martin T.Hagan,Howard B.Demuth,Mark H.Beale.Neural Network Design[M].PWS Publishing Company.1996.
[10]Marilly E,Aghasaryan A,Betge-Brezetz S,et al.Alarm Correlation for Complex Telecommunication Network Using Neural Network and Signal Processing[C].in IEEE Workshop on Operation and Management,2002.
[11]苏利敏,候朝桢,戴忠健,潘秀琴.基于神经网络的告警关联[J].北京理工大学学报,2002.6.
[12]李静.基于神经网络和遗传算法的Ad hoc网络故障管理模型研究[D].成都：电子科技大学,2008.4.
[13]肖伟.数据挖掘在网络故障诊断中的应用[D].南京：南京理工大学.2004.6.
[14]Deng R H,Lazar A A,Wang W.A Probabilistic Approach to Fault Diagnosis in Linear Lightwave Network[C].IEEE Journal on Selected Areas in Communications,1993,11 (9):1438.
[15]肖秦琨,高嵩,高晓光.动态贝叶斯网络推理学习理论及应用[M].北京：国防工业出版社,2007.
[16]李千目.战略互联网故障智能诊断策略研究[D].南京市：南京理工大学,2005.
[17]翁永前.基于免疫原理的网络故障检测系统模型的研究[D].成都：电子科技大学,2008.
[18]M.Thottan and C.Ji.Statistical Detectionof Enterprise NetWork Problems[J].Joumal of Network and Systems Management,1999.
[19]李鹏.基于事件关联的网络故障管理研究[D].湖南：中南大学,2008.
[20]Jakobson G.Weissman M.Alarm Correlation[J].IEEE Network,1993,7(6):52-59.
[21]Dupuy A,Schwartz J,Yemini Y,et al.Network Fault Management:A user's View[C].in Integrated Network Management.North Holland:Elsevier Science Publishers B.V,1989.
[22]Katzela.l.Fault Diagnosis in Telecommunications Network[D].Columbia University,1996.
[23]胡谷雨.网络管理技术教程[M].北京：希望电子出版社,2002.9.23-24.
[24]Miseha Schartz.Broadband Integrated Networks[M].Prentiee Hall,1998.
[25]郭军.网络管理[M].第3版.北京：北京邮电大学出版社,2008.247-249.
[26]A.Bouloutas,S.Calo,and A.Finkel.Alarm Correlation and Fault Identification in Communication Networks[J].IEEE Transactions on Communications, 42:523-533,1994.
[27]高旭麟.基于SNMPv3的网络视频监控管理系统研究与实现[D].江南大学硕士学位论文.2008.
[28]Rakesh Agrawal,Giuseppe psaila.Active Data Mining[C].1st International Conference on Knowledge Discovery and Da a Mining,Menlo Park,Calif,1995.
[29]毛国君,段立娟,王实等.数据挖掘原理与算法[M].北京：清华大学出版社,2007,10-21.
[30]中国科学技术协会.2006-2007计算机科学学科发展报告.北京：中国科学技术出版社,2007,138-143.
[31]Borzsonyi S,Kossmann D,Stocker K.The Skyline Operator[G].In:Proc.of the 17th Int'1 Conf.on Data Engineering. Heidelberg:IEEE Computer Society Press, 2001,421-430.
[32]夏海涛,詹志强.新一代网络管理技术[M].北京：北京邮电大学出版社,2003.
[33]林曼筠.可扩展的计算机网络管理系统技术研究[D].北京市：中国科学院研究生院(计算技术研究所),2002.
[34]郭道荣.基于数据挖掘的电信网络故障诊断技术的研究[D].重庆大学硕士学位论文.2003.
[35]Han J,Jian P,Yiwen Y.Mining Frequent Patterns Without Candidate Generation[C]. In:Proceedings of the 2000 ACM SIGMOD International Conference Management of Data. Dallas,2000.1-12.
[36]毛国君,段立娟,王实等.数据挖掘原理与算法[M].北京：清华大学出版社,2007,69.
[37]Han J,Kamber M.Data Mining:Concepts and Techniques[M].Beijing:High Education Press,2001.
[38]端义锋,胡谷雨,丁力.序列模式挖掘在网络告警分析中的应用[J].北京邮电大学学报.2004.27(12).
[39]Cheung D.W.Maintenance of Discovered Association Rules in Large Databases:An Incremental Updating Technique[C].In:Proceedings of the 1996 International Conference on Data Engineering, New Orleans,Louisiana,1996.
[40]Cheung D.W.et al.. A General Incremental Technique for Updating Discovered Assoeiation Rules[C].In:Proceedings of the 1997 International Conference on Databases Systems for Advanced Application.Melbourne,1997:185-194.
[41]冯玉才,冯剑琳.关联规则的增量式更新算法[J].软件学报,1998.9(4)：301-306.
[42]杨学兵,宋红梅.一种高效的关联规则增量式更新算法[J].计算机技术与发展.2007.
[43]宋余庆,朱玉全,孙志挥等.基于FP-Tree的最大频繁项目集挖掘及更新算法[J].软件学报.2003,14(9)：1586-1592.
[44]宋余庆,朱玉全,孙志挥等.一种基于频繁模式树的约束最大频繁项目集挖掘及其更新算法[J].计算机研究与发展,2005,42(5)：777-783.
[45]吉根林,杨明,宋余庆等.最大频繁项目集的快速更新[J].计算机学报,2005,28(1)：128-135.
[46]徐前方.基于数据挖掘的网络故障告警相关性研究[D].北京邮电大学.2007.
[47]Quinlan J R. Induction of decision trees[J].Machine Learning.1986,(4):81-106.
[48]Quinlan J R. C4.5 Programs for Machine Learning[M].San Mateo:Morgan Kaufmann Publishers, lnc,1993.
[49]王妍妍,王艳宁,王敏.基于单变量决策树的网络故障诊断方法[J].计算机机工程与设计,2007.
[50]刘康平.面向网络故障管理的知识发现方法[D].西安交通大学博士学位论文.2001.
[51]R.Srikant,R.Agrawal.Mining Generalized association rules[J].In:Proc.Of the 21st Int'l conf.On Very Large Database,Zurich,Switzerland,1995.
[52]P.Smyth and R.M.Goodman.An information theoretic approach to rule induction from databases[J].IEEE Transactions on Knowledge and DataEngineering.1992.
[53]郭军.网络管理[M].第3版.北京：北京邮电大学出版社,2008.295-297.
[54]刘康平.面向网络故障管理的知识发现方法[D].西安交通大学博士学位论文,2001.