通信网网管告警过滤机制的研究与应用
|
摘要
由于网络规模的不断扩大，网络管理研究成为IT业界的热门研究方向之一，通信运营商急需建设综合化、集中化、智能化的网管系统，对通信网进行有效的管理，为客户提供优质的服务，提高企业的竞争力。
通信网综合网管需要采集和处理来自交换、传输、数据、动力环境等专业网的海量告警数据，为了保证有效地防止告警风暴，保证关注告警，快速发现、定位和处理故障，必须设计和实施有效的告警过滤机制，以实现快速、准确地过滤瞬断告警、无效告警和次要告警，这是建设通信网综合网管的核心任务之一。
本文首先对通信网告警数据的特点进行了详细分析，并详细阐述了告警关联分析的常用方法以及关联规则挖掘算法。
其次，在比较现有告警关联分析方法后，探讨了如何采用数据挖掘技术获取告警关联规则，并在此基础上构建告警过滤规则知识库，以及各告警过滤模板的知识表示和知识获取更新方法。
最后，以上述研究分析结果为基础，提出了分层告警过滤模型和基于模板的告警过滤方案，阐述了知识库规则匹配及冲突消解方法，告警过滤机制的具体实现及其实际应用效果。
国内对于本地通信网综合网管的研究与开发工作是近几年才起步的，而将数据挖掘和知识库技术应用于告警过滤机制的研究工作则时间更短，因此本课题的研究内容与成果具有较强的理论、技术与应用价值。课题的主要创新体现在：
◇ 根据对告警数据特点和告警过滤需求的分析，定义了分层结构的告警过滤模型。
◇ 设计构建了告警关联知识库，实现了告警过滤模板的智能化生成。
◇ 通过分析经典Apriori算法并结合时间窗口，获得告警时间相关性规则并应用于实际系统。
◇ 采用了基于模板的复合过滤技术，有效地实现了告警数据的逐层收敛、告警过滤功能的灵活配置与实施。
With the continual expansion of the scale of network, network management research has become one of the most popular research directions in IT industry. Telecommunication service providers need to construct a integrated, centralized, intelligent management system, in order to carry through effective management to telecommunication, to provide qualified services to customers and to improve the competitiveness of enterprises.
Telecommunication integrated network management demands to collect and process the huge number of alarm data from the exchange network management, transmission network management ,data network management, dynamic environment network management and etc. It's one of the core tasks that designing and implementation effective filtering mechanisms of alarms to achieve filtering disrupt, void and secondary alarms rapidly and accurately, to ensure to prevent alarms storm effectively, to ensure the concern alarms and to detect, position and handle failure rapidly.
Firstly, this article analyses the characteristic of the alarm data of telecommunication network in detail and minutely clarifies the common methods of alarm relativity analysis and association rule mining algorithm.
Secondly, it probes into how to obtain the alarm association rule using data mining technology after the comparison of current alarm association analysis methods. On based of that this article constructs alarm filtering rule knowledge base and knowledge expression of every alarm filtering module and knowledge acquisition and updating methods.
Lastly, based on the former analysis results, this article puts forward stratified alarm filtering model and alarm filtering project basis of module. Further more, it illuminates knowledge base rule matching and the methods to settle the collisions, the concrete implementation of alarm filtering mechanism and its application effects.
In our country, it is not a long time to start the research and exploitation of local
通信网综合网管需要采集和处理来自交换、传输、数据、动力环境等专业网的海量告警数据，为了保证有效地防止告警风暴，保证关注告警，快速发现、定位和处理故障，必须设计和实施有效的告警过滤机制，以实现快速、准确地过滤瞬断告警、无效告警和次要告警，这是建设通信网综合网管的核心任务之一。
本文首先对通信网告警数据的特点进行了详细分析，并详细阐述了告警关联分析的常用方法以及关联规则挖掘算法。
其次，在比较现有告警关联分析方法后，探讨了如何采用数据挖掘技术获取告警关联规则，并在此基础上构建告警过滤规则知识库，以及各告警过滤模板的知识表示和知识获取更新方法。
最后，以上述研究分析结果为基础，提出了分层告警过滤模型和基于模板的告警过滤方案，阐述了知识库规则匹配及冲突消解方法，告警过滤机制的具体实现及其实际应用效果。
国内对于本地通信网综合网管的研究与开发工作是近几年才起步的，而将数据挖掘和知识库技术应用于告警过滤机制的研究工作则时间更短，因此本课题的研究内容与成果具有较强的理论、技术与应用价值。课题的主要创新体现在：
◇ 根据对告警数据特点和告警过滤需求的分析，定义了分层结构的告警过滤模型。
◇ 设计构建了告警关联知识库，实现了告警过滤模板的智能化生成。
◇ 通过分析经典Apriori算法并结合时间窗口，获得告警时间相关性规则并应用于实际系统。
◇ 采用了基于模板的复合过滤技术，有效地实现了告警数据的逐层收敛、告警过滤功能的灵活配置与实施。
With the continual expansion of the scale of network, network management research has become one of the most popular research directions in IT industry. Telecommunication service providers need to construct a integrated, centralized, intelligent management system, in order to carry through effective management to telecommunication, to provide qualified services to customers and to improve the competitiveness of enterprises.
Telecommunication integrated network management demands to collect and process the huge number of alarm data from the exchange network management, transmission network management ,data network management, dynamic environment network management and etc. It's one of the core tasks that designing and implementation effective filtering mechanisms of alarms to achieve filtering disrupt, void and secondary alarms rapidly and accurately, to ensure to prevent alarms storm effectively, to ensure the concern alarms and to detect, position and handle failure rapidly.
Firstly, this article analyses the characteristic of the alarm data of telecommunication network in detail and minutely clarifies the common methods of alarm relativity analysis and association rule mining algorithm.
Secondly, it probes into how to obtain the alarm association rule using data mining technology after the comparison of current alarm association analysis methods. On based of that this article constructs alarm filtering rule knowledge base and knowledge expression of every alarm filtering module and knowledge acquisition and updating methods.
Lastly, based on the former analysis results, this article puts forward stratified alarm filtering model and alarm filtering project basis of module. Further more, it illuminates knowledge base rule matching and the methods to settle the collisions, the concrete implementation of alarm filtering mechanism and its application effects.
In our country, it is not a long time to start the research and exploitation of local
引文
[1] 唐纯贞，严建民，鲁碧英．电信网与电信业务[M]．人民邮电出版社，2003
[2] ITU-T Rec. M. 30100:Generic Network Information Model[S]. 1992
[3] ITU-T Rec. M. 3400:TMN management functions[S]. 2001
[4] ITU-T Rec. M. 3200:TMN management services and communications managed areas: overview[S]. 2001
[5] 毛京丽．现代通信网[M]．北京邮电大学出版社，2002
[6] 赵宁，顾宏斌，米志超．信令监测系统在通信网中的应用研究[J]．科技情报开发与经济，2005
[7] 刘康平，李增智．网络告警序列中的频繁情景规则挖掘算法[J]．小型微型计算机系统，2003
[8] 中国电信集团公司部门文件．本地通信网集中告警系统功能需求规范V1[1]．10[S]，2005
[9] 郑庆国，吕卫峰．通信网络中的告警相关性研究[J]．计算机工程与应用，2002
[10] Jakobson G., Weissman M. Alarm Correlation[J]. IEEE Network, 1993
[11] R. Stenit, K. Adamson. E. P. Curran. Case-based Reasoning for Complex Telecommunication Systems. NIMES'98-Complex Systems[J]. Intelligent Systems and Interfaces, Nimes, France, 1998.
[12] Liang, Kai-Chih, Chyan, Daplme, Chang, Yue-Shan, Lo, Win-Tsung, Yuan, Shyan-Ming. A case- based reasoning approach to the management of faults in communication networks, Volume: 25, Issue: 4, August, 2002
[13] 孙朝晖，张德运，李庆海周络故障管理中的自动告警关联[J]．计算机工程，2004
[14] 苏利敏，侯朝桢，戴忠健，潘秀琴．基于神经网络的告警关联[J]．北京理工大学学报，2002
[15] R. Cronk, P. Callaban, L. Bernstein. Rule-based expert systems for network management and operation: an introduction[J]. IEEE Network, Vol. 2, No. 5, 1988
[16] M. Klemettinen, H. Mannila, H. Toivonen. Rule Discovery in Telecommunication Alarms Data[J]. Journal of Network and Systems Management, Vol. 7, No. 4, 1999
[17] 吕晓斌，邓琦皓，罗军勇，王清贤．基于模式匹配的告警关联[J]．微电子学与计算机，2005
[18] 周伟．王汝传．徐小龙．基于移动代理的网络故障管理的研究[J]．微型机与应用，2004
[19] 罗芳，蒲秋梅．关联规则在移动通信行业中的应用[J]．武汉理工大学学报，2005
[20] 邓丽艳，谢世波，杨永田，王书达．数据挖掘在电信网管中的应用研究[J]．哈尔滨商业大学学报，2005
[21] Jiawei Han Micheline Kamber．数据挖掘概念与技术[M]．机械工业出版社，2001
[22] R. Agrawal and R. Srikant. Fast algorithm for mining association rules. In Proc. 1994 Int. Conf.??Very Large Data Base, pp. 487-499, Santiago, Chile, Sept., 1994
[23] 武波，马玉祥等．专家系统[M]．北京理工大学出版社，2001
[24] 林尧瑞，马少平．人工智能导论[M]．清华大学出版社，1998
[25] 赵瑞清，王晖，邱涤虹．知识表示和推理[M]．气象出版社，1991
[26] 王永庆．人工智能原理与方法[M]．西安交通大学出版社，1998
[27] 吴伟民．基于关系数据库表示产生式规则[J]．四川联合大学学报(工程科学版)．1998
[28] 李晓强，崔德光．基于关系数据库的知识库结构设计．计算机工程与应用，2001
[29] W. A. Woods. What's Important about Knowledge Representation?[J]. IEEE Computer, 1983
[30] Ahmed Rameau, Hesham Hassen. Automatic knowledge acquisition tool for irrigation and fertilization expert systems[J]. Expert Systems with Applications, 2003
[31] R. Steystems. Intelligent Systems and Interfaces, Nimes, France, 1998.
[32] John McDonald, P. Eng. Over whelmed by alarms: blackout puts filtering and suppression technologies in the spotlight. Electricity Today, Issue 8, 2003
[33] Robert D. Gardner, David AiHarle. Pattern discovery and specification techniques for alarm correlation[J]. IEEE, 1998
[34] Anthony Edwards, Robert Whitaker. Fault management:A functional View of Root Cause Analysis and Correlation[J]. TAVVE Software Company, 2000
[35] Kettschau, H. -J., Bruck, S., Schefezik, P., LUCAS. an expert system for intelligent fault management and alarm correlation[J]. Network Operation and Management Symposium, 2002, NOMS 2002. 2002 IEEE/IFIP, 15-19 April 2002
[36] Qingguo Zheng, Ke Xu. Weifeng Lv, Shilong Ma. Intelligent research of correlated alarms from database containing noise data. Network Operations and Management Symposium, 2002. NOMS 2002. 2002 IEEE/IFIP. 15-19 April 2002
[37] 杨洋，马玉祥．基于关联规则的网管告警数据库知识发现[J]．微电子学与计算机，2004
[38] 乔庆泉，李红松，冯玉珉．通信行业综合决策支持系统[J]．铁道通信信号，2004
[39] 王玮．对通信综合网管系统故障管理的设计[J]．通信世界，2004
[40] 端义锋，胡谷雨，丁力．序列模式挖掘在网络告警分析中的应用[J]．北京邮电大学学报，2004
[41] 徐昊朗，熊齐邦．一种新的网络告警相关系统的体系结构[J]．计算机应用，2004
[42] 朱有产，窦炳琳，商李彪，余晓晔．网络故障诊断框架的研究及应用[J]．通信学报，2005
[2] ITU-T Rec. M. 30100:Generic Network Information Model[S]. 1992
[3] ITU-T Rec. M. 3400:TMN management functions[S]. 2001
[4] ITU-T Rec. M. 3200:TMN management services and communications managed areas: overview[S]. 2001
[5] 毛京丽．现代通信网[M]．北京邮电大学出版社，2002
[6] 赵宁，顾宏斌，米志超．信令监测系统在通信网中的应用研究[J]．科技情报开发与经济，2005
[7] 刘康平，李增智．网络告警序列中的频繁情景规则挖掘算法[J]．小型微型计算机系统，2003
[8] 中国电信集团公司部门文件．本地通信网集中告警系统功能需求规范V1[1]．10[S]，2005
[9] 郑庆国，吕卫峰．通信网络中的告警相关性研究[J]．计算机工程与应用，2002
[10] Jakobson G., Weissman M. Alarm Correlation[J]. IEEE Network, 1993
[11] R. Stenit, K. Adamson. E. P. Curran. Case-based Reasoning for Complex Telecommunication Systems. NIMES'98-Complex Systems[J]. Intelligent Systems and Interfaces, Nimes, France, 1998.
[12] Liang, Kai-Chih, Chyan, Daplme, Chang, Yue-Shan, Lo, Win-Tsung, Yuan, Shyan-Ming. A case- based reasoning approach to the management of faults in communication networks, Volume: 25, Issue: 4, August, 2002
[13] 孙朝晖，张德运，李庆海周络故障管理中的自动告警关联[J]．计算机工程，2004
[14] 苏利敏，侯朝桢，戴忠健，潘秀琴．基于神经网络的告警关联[J]．北京理工大学学报，2002
[15] R. Cronk, P. Callaban, L. Bernstein. Rule-based expert systems for network management and operation: an introduction[J]. IEEE Network, Vol. 2, No. 5, 1988
[16] M. Klemettinen, H. Mannila, H. Toivonen. Rule Discovery in Telecommunication Alarms Data[J]. Journal of Network and Systems Management, Vol. 7, No. 4, 1999
[17] 吕晓斌，邓琦皓，罗军勇，王清贤．基于模式匹配的告警关联[J]．微电子学与计算机，2005
[18] 周伟．王汝传．徐小龙．基于移动代理的网络故障管理的研究[J]．微型机与应用，2004
[19] 罗芳，蒲秋梅．关联规则在移动通信行业中的应用[J]．武汉理工大学学报，2005
[20] 邓丽艳，谢世波，杨永田，王书达．数据挖掘在电信网管中的应用研究[J]．哈尔滨商业大学学报，2005
[21] Jiawei Han Micheline Kamber．数据挖掘概念与技术[M]．机械工业出版社，2001
[22] R. Agrawal and R. Srikant. Fast algorithm for mining association rules. In Proc. 1994 Int. Conf.??Very Large Data Base, pp. 487-499, Santiago, Chile, Sept., 1994
[23] 武波，马玉祥等．专家系统[M]．北京理工大学出版社，2001
[24] 林尧瑞，马少平．人工智能导论[M]．清华大学出版社，1998
[25] 赵瑞清，王晖，邱涤虹．知识表示和推理[M]．气象出版社，1991
[26] 王永庆．人工智能原理与方法[M]．西安交通大学出版社，1998
[27] 吴伟民．基于关系数据库表示产生式规则[J]．四川联合大学学报(工程科学版)．1998
[28] 李晓强，崔德光．基于关系数据库的知识库结构设计．计算机工程与应用，2001
[29] W. A. Woods. What's Important about Knowledge Representation?[J]. IEEE Computer, 1983
[30] Ahmed Rameau, Hesham Hassen. Automatic knowledge acquisition tool for irrigation and fertilization expert systems[J]. Expert Systems with Applications, 2003
[31] R. Steystems. Intelligent Systems and Interfaces, Nimes, France, 1998.
[32] John McDonald, P. Eng. Over whelmed by alarms: blackout puts filtering and suppression technologies in the spotlight. Electricity Today, Issue 8, 2003
[33] Robert D. Gardner, David AiHarle. Pattern discovery and specification techniques for alarm correlation[J]. IEEE, 1998
[34] Anthony Edwards, Robert Whitaker. Fault management:A functional View of Root Cause Analysis and Correlation[J]. TAVVE Software Company, 2000
[35] Kettschau, H. -J., Bruck, S., Schefezik, P., LUCAS. an expert system for intelligent fault management and alarm correlation[J]. Network Operation and Management Symposium, 2002, NOMS 2002. 2002 IEEE/IFIP, 15-19 April 2002
[36] Qingguo Zheng, Ke Xu. Weifeng Lv, Shilong Ma. Intelligent research of correlated alarms from database containing noise data. Network Operations and Management Symposium, 2002. NOMS 2002. 2002 IEEE/IFIP. 15-19 April 2002
[37] 杨洋，马玉祥．基于关联规则的网管告警数据库知识发现[J]．微电子学与计算机，2004
[38] 乔庆泉，李红松，冯玉珉．通信行业综合决策支持系统[J]．铁道通信信号，2004
[39] 王玮．对通信综合网管系统故障管理的设计[J]．通信世界，2004
[40] 端义锋，胡谷雨，丁力．序列模式挖掘在网络告警分析中的应用[J]．北京邮电大学学报，2004
[41] 徐昊朗，熊齐邦．一种新的网络告警相关系统的体系结构[J]．计算机应用，2004
[42] 朱有产，窦炳琳，商李彪，余晓晔．网络故障诊断框架的研究及应用[J]．通信学报，2005