数据挖掘在通信网络管理系统中的应用
|
摘要
现代通信网络的特点是规模大、结构复杂,管理的主要任务是对网络进行实时监控,确保网络的高可靠性和高可用性。我军当前缺乏跨专业网的综合网络维护与管理系统,通信网络运行维护部门难以进行跨专业网的故障监控、定位与处理。为了保障好部队集中化的核心业务系统能稳定运行,使复杂的业务能够得到全面、统一、及时、准确的监控和管理,通信网络综合管理系统的开发就显得尤为迫切。
本文设计了一种部队通信网综合管理系统,并应用了CHARM_SM和CHARM_DM算法作为告警相关性分析的主要算法。
告警相关性分析是通信网络管理中的重要内容,可用于冗余告警删除、故障定位和故障预测。传统告警相关性分析方法过于依赖专家知识而无法适应复杂、大型网络的不足。本文所设计的是通过一个平台实现对多个平台、多业务系统的有效、统一管理,面对海量告警数据,将数据挖掘技术应用到告警相关性分析中,发现通信网络中的告警相关性知识,重点研究了告警频繁闭项集增量更新算法CHARM_SM和告警数据增加时的告警频繁闭项集增量更新算法CHARM_DM,仿真效果表明告警数据挖掘性能提高显著,而后对无冗余关联规则挖掘算法进行了仿真,实现了频繁闭项集增量挖掘算法和无冗余告警关联规则产生算法在部队通信网综合网管告警数据处理中的应用。
Modem telecommunication networks are characterized with large scale and complexity. Network fault management focus on monitoring the status of network, which ensures the network to run reliably and efficiently. Currently, our army is lack of network maintenance and management system across comprehensive professional network. Communication network maintenance department is hard to make professional network fault monitoring, orientation and processing. In order to guarantee good forces centralized core business system can stably operated, make the complex business can be fully, unity, timely, accurate monitored and managed, integrated management system of telecommunication network development becomes more important and urgent.
Alarms correlation analysis can delete redundant alarms, locate faults and predict faults. Traditional methods can hardly work well when networks are complex and huge. This design is a platform for multiple through more business platform is effectively, unified managed. Facing massive warning data, the data mining technology was applied to alarm correlation analysis, found the alarm correlation of knowledge communication network. The paper studies several key problems including incremental mining of FCI(frequent closed item sets) - CHARM_SM and it’s improve updating algorithm CHARM_DM. Simulation results show that alarm data mining are improved significantly , then redundant algorithm for mining association rules is simulated, and maintenance of FCI and non-redundant association rules mining algorithm implemented in alarm data processing applications in military comprehensive communication network.
本文设计了一种部队通信网综合管理系统,并应用了CHARM_SM和CHARM_DM算法作为告警相关性分析的主要算法。
告警相关性分析是通信网络管理中的重要内容,可用于冗余告警删除、故障定位和故障预测。传统告警相关性分析方法过于依赖专家知识而无法适应复杂、大型网络的不足。本文所设计的是通过一个平台实现对多个平台、多业务系统的有效、统一管理,面对海量告警数据,将数据挖掘技术应用到告警相关性分析中,发现通信网络中的告警相关性知识,重点研究了告警频繁闭项集增量更新算法CHARM_SM和告警数据增加时的告警频繁闭项集增量更新算法CHARM_DM,仿真效果表明告警数据挖掘性能提高显著,而后对无冗余关联规则挖掘算法进行了仿真,实现了频繁闭项集增量挖掘算法和无冗余告警关联规则产生算法在部队通信网综合网管告警数据处理中的应用。
Modem telecommunication networks are characterized with large scale and complexity. Network fault management focus on monitoring the status of network, which ensures the network to run reliably and efficiently. Currently, our army is lack of network maintenance and management system across comprehensive professional network. Communication network maintenance department is hard to make professional network fault monitoring, orientation and processing. In order to guarantee good forces centralized core business system can stably operated, make the complex business can be fully, unity, timely, accurate monitored and managed, integrated management system of telecommunication network development becomes more important and urgent.
Alarms correlation analysis can delete redundant alarms, locate faults and predict faults. Traditional methods can hardly work well when networks are complex and huge. This design is a platform for multiple through more business platform is effectively, unified managed. Facing massive warning data, the data mining technology was applied to alarm correlation analysis, found the alarm correlation of knowledge communication network. The paper studies several key problems including incremental mining of FCI(frequent closed item sets) - CHARM_SM and it’s improve updating algorithm CHARM_DM. Simulation results show that alarm data mining are improved significantly , then redundant algorithm for mining association rules is simulated, and maintenance of FCI and non-redundant association rules mining algorithm implemented in alarm data processing applications in military comprehensive communication network.
引文
[1] H.Mannila,H.Toivonen,A.Verkamo. Discovery of frequent episodes in event sequences[C].Data Mining and Knowledge Discovery.Netherlands,1997:259-289.
[2] D.Gardner,A.Harle.Fault Resolution and Alarm correlation in High-speed networks using Database Mining Techniques.ICSP1997.
[3]郭军.网络管理[M].北京:北京邮电大学出版社,2003.
[4] Rakesh Agrawal,Ramakrishnan Srikant.Fast Algorithm for Mining Association Rules.In:Proceedings of the 20th VLDB Conference.1994.
[5] Jia wei Han,Micheline.数据挖掘概念与技术.机械工业出版社,2001.
[6] N.Pasquier,Y.Bastide,R.Taouil,et al.Discovering frequent closed itemsets for association rules[C].In Proceeding 7th International Conference Database Theory(ICDT’99), Jerusalem,Israel, 1999: 398-416.
[7] J.Zaki,C.Hsiao.CHARM:An efficient algorithm for closed itemset mining[C].Proceeding 2002 SIAM Internatioanl Conference.Data Ming(SDM’02), 2002: 457-473.
[8] J.Pei,J.Han, R.Mao. CLOSET:An efficient algorithm for mining frequent closed itemset[C].Proc.2000 ACM-SIGMOD Int’l workshop data mining and knowledge discovery(DMKD’00), 2000 :11-20.
[9] Claudio Lucchese,Salvatore Orlando,Raffaele Perego.Fast and memory efficient mining of frequent closed itemsets[J].IEEE Transaction on Knowledge and Data Engineering, 2005:1-15.
[10]马占宇,数据挖掘技术在电信网络管理系统中的应用[D].北京:北京邮电大学,2007.
[11] Jakobson G, Weissman M. Alarm correlation[J]. IEEE Network, 1993.
[12] M.Klemettinen,H.Mannila,H.Toivonen. Rule discovery in telemmunication alarm data[J].Jorunal of Network and Systems Management, 1999,7(4):108-119.
[13]单莘.基于知识发现的告警相关性关键问题研究[D].北京:北京邮电大学,2006.
[14]王小虎.关联规则挖掘综述.计算机工程与应用,2003.
[15]唐纯贞,严建民,鲁碧英.电信网与电信业务.人民邮电出版社,2003.
[16]石峰.通信网网关告警相关性分析机制的研究与应用.南昌,南昌大学,2006.
[17] J.Zaki,C.Hsiao.CHARM:An efficient algorithm for closed itemset mining[C].Proceeding 2002 SIAM Internatioanl Conference.Data Ming(SDM’02), 2002: 457-473.
[18]刘杰,朱磊等.支持度门限改变下的频繁闭项集增量挖掘.军事通信技术, 2008.
[19] BartGoethals. Su rveyon frequent patternwww.adrem.ua.ac.be/~goethals/software/publications. mining .
[20]徐丽霞,网络故障管理告警关联技术分析[J].电脑知识与技术, 2008,4 (28).
[21] J.Zaki.Mining Non-redundant association rules[C].Data Mining and Knowledge Discovery.Netherlands, 2004, 9(1):223-248.
[22] G.Das,K.Lin,H.Mannila,et al. Rule discovery from time series[C].Proceedings of the fourth international conference on knowledge discovery and data mining (KDD’98), New York, 1999:16-22.
[23]何光辉,多层高维频繁序列挖掘算法研究.重庆大学学位论文,2004.
[24]冯玉才,冯剑琳.关联规则的增量式更新算法[J].软件学报,1998,9(4):301-306.
[25]宋余庆,朱玉全,孙志辉等.基于FP-Tree的最大频繁项目集挖掘及更新算法[J].软件学报,2003,14(9):1586-1592.
[26]宋余庆,朱玉全,孙志辉等.一种基于频繁模式树的约束最大频繁项目集挖掘及其更新算法[J].计算机研究与发展,2005,42(5):777-783.
[27] B.Ganter and R.Wille.Formal Concept Analysis:Mathematical Foundations.Springer-Verlag,1999.
[28] N.Pasquier,Y.Bastide,R.Taouil,et al.Discovering frequent closed itemsets for association rules[C].In Proceeding 7th International Conference Database Theory(ICDT’99), Jerusalem,Israel, 1999: 398-416.
[29] Mika Klemettinen.A konwledge discovery methodology for telecommunication network alarm databases[D].University of Helsinki,1999.
[30]刘康平,李增智.网络告警序列中的频繁情景规则挖掘算法[J].小型微型计算机系统,2003,24(5):891-894.
[2] D.Gardner,A.Harle.Fault Resolution and Alarm correlation in High-speed networks using Database Mining Techniques.ICSP1997.
[3]郭军.网络管理[M].北京:北京邮电大学出版社,2003.
[4] Rakesh Agrawal,Ramakrishnan Srikant.Fast Algorithm for Mining Association Rules.In:Proceedings of the 20th VLDB Conference.1994.
[5] Jia wei Han,Micheline.数据挖掘概念与技术.机械工业出版社,2001.
[6] N.Pasquier,Y.Bastide,R.Taouil,et al.Discovering frequent closed itemsets for association rules[C].In Proceeding 7th International Conference Database Theory(ICDT’99), Jerusalem,Israel, 1999: 398-416.
[7] J.Zaki,C.Hsiao.CHARM:An efficient algorithm for closed itemset mining[C].Proceeding 2002 SIAM Internatioanl Conference.Data Ming(SDM’02), 2002: 457-473.
[8] J.Pei,J.Han, R.Mao. CLOSET:An efficient algorithm for mining frequent closed itemset[C].Proc.2000 ACM-SIGMOD Int’l workshop data mining and knowledge discovery(DMKD’00), 2000 :11-20.
[9] Claudio Lucchese,Salvatore Orlando,Raffaele Perego.Fast and memory efficient mining of frequent closed itemsets[J].IEEE Transaction on Knowledge and Data Engineering, 2005:1-15.
[10]马占宇,数据挖掘技术在电信网络管理系统中的应用[D].北京:北京邮电大学,2007.
[11] Jakobson G, Weissman M. Alarm correlation[J]. IEEE Network, 1993.
[12] M.Klemettinen,H.Mannila,H.Toivonen. Rule discovery in telemmunication alarm data[J].Jorunal of Network and Systems Management, 1999,7(4):108-119.
[13]单莘.基于知识发现的告警相关性关键问题研究[D].北京:北京邮电大学,2006.
[14]王小虎.关联规则挖掘综述.计算机工程与应用,2003.
[15]唐纯贞,严建民,鲁碧英.电信网与电信业务.人民邮电出版社,2003.
[16]石峰.通信网网关告警相关性分析机制的研究与应用.南昌,南昌大学,2006.
[17] J.Zaki,C.Hsiao.CHARM:An efficient algorithm for closed itemset mining[C].Proceeding 2002 SIAM Internatioanl Conference.Data Ming(SDM’02), 2002: 457-473.
[18]刘杰,朱磊等.支持度门限改变下的频繁闭项集增量挖掘.军事通信技术, 2008.
[19] BartGoethals. Su rveyon frequent patternwww.adrem.ua.ac.be/~goethals/software/publications. mining .
[20]徐丽霞,网络故障管理告警关联技术分析[J].电脑知识与技术, 2008,4 (28).
[21] J.Zaki.Mining Non-redundant association rules[C].Data Mining and Knowledge Discovery.Netherlands, 2004, 9(1):223-248.
[22] G.Das,K.Lin,H.Mannila,et al. Rule discovery from time series[C].Proceedings of the fourth international conference on knowledge discovery and data mining (KDD’98), New York, 1999:16-22.
[23]何光辉,多层高维频繁序列挖掘算法研究.重庆大学学位论文,2004.
[24]冯玉才,冯剑琳.关联规则的增量式更新算法[J].软件学报,1998,9(4):301-306.
[25]宋余庆,朱玉全,孙志辉等.基于FP-Tree的最大频繁项目集挖掘及更新算法[J].软件学报,2003,14(9):1586-1592.
[26]宋余庆,朱玉全,孙志辉等.一种基于频繁模式树的约束最大频繁项目集挖掘及其更新算法[J].计算机研究与发展,2005,42(5):777-783.
[27] B.Ganter and R.Wille.Formal Concept Analysis:Mathematical Foundations.Springer-Verlag,1999.
[28] N.Pasquier,Y.Bastide,R.Taouil,et al.Discovering frequent closed itemsets for association rules[C].In Proceeding 7th International Conference Database Theory(ICDT’99), Jerusalem,Israel, 1999: 398-416.
[29] Mika Klemettinen.A konwledge discovery methodology for telecommunication network alarm databases[D].University of Helsinki,1999.
[30]刘康平,李增智.网络告警序列中的频繁情景规则挖掘算法[J].小型微型计算机系统,2003,24(5):891-894.