IP网管数据采集系统的设计与实现
|
摘要
近年来,随着以TCP/IP为核心协议的互连网的快速发展,IP技术作为一种承载业务和沟通传输的中间体得到了极大的发展。基于IP的业务应用发展迅速,网络的规模日趋庞大。网络的可用性与网络规模膨胀的矛盾显得更加突出。网络的正常高效的运转急需好的IP网管软件的支持。
面向对象技术和分布式计算技术的发展,特别是近来Java技术的快速发展和J2EE框架的不断完善,为分布式、统一的网络管理系统的实现提供了有效途径。因此,以J2EE技术为主要框架的IP网管是系统发展的方向,而数据采集子系统在网管系统中占据着重要的地位。本文详细阐述了IP网管数据采集子系统的设计和实现方法。
文章首先介绍了网络管理的一些基础知识,包括网络管理的基本概念、网络管理的五大功能以及网络管理的体系结构。
其次,文章分析了网管数据采集系统使用的主要协议,包括SNMP、NETFLOW以及CLI。
接着,文章介绍了J2EE的概念和框架结构,并讨论了把J2EE框架应用到分布式网络管理的技术优势。
文章最后详细描述了数据采集子系统的设计和实现。鉴于JMX具有完全的向后兼容性和良好的扩展性,我们在数据采集系统的设计上使用JMX实现了数据采集与协议相关的底层操作。在上层的实现中,把数据采集分为主动数据采集和被动数据采集。NETFLOW、SNMP TRAP属于被动采集的范畴,在JMX Mbean里面自动实现数据的采集、告警和入库操作。而SNMP、CLI属于主动采集,必须让采集服务器使用轮询操作来及时取得网管数据。设计中我们提出了在EJB容器内实现异步多线程轮询的解决方案。这一方案统一描述不同协议的轮询任务,通过轮询任务表集中控制轮询操作,并把轮询任务的执行分散到集群系统中,有效的解决了大规模网络轮询操作中网络设备数量庞大、协议多的问题。
为了提高数据接收的效率,我们用C语言编写了NETFLOW数据的接收程序,这是唯一一个没有在J2EE框架内实现的数据接收模块,文章会详细论述其中的原因和实现方法。
With the great development of Internet ,the IP network technology , as a carrier of business and communication, having got rapid growth in recent years. The business based on IP technology also make good time ,which make the network become more and more enormous, and increase the inconsistency between availability and size of the network. Today, excellent network management software is indispensable to high efficient network.
The great development of Object-Oriented technology and distributed computing, especially the fast growing of Java and J2EE framework, promote the implementation of distributed network management . So, the IP network management technology base on J2EE is the trend of future development, and the acquisition subsystem play a important role in network management system. This article will elaborate on the design and implement method of data acquisition subsystem.
This paper first introduces some base theory of network management technology, including base conception, main functions and framework of the network management.
Secondly, this paper analyzes some protocols used by data collectting, including SNMP, NETFLOW and CLI.
After that this paper gives a brief introduction to conception and framework of J2EE , demonstrate the great technology advantages to apply J2EE framework to distributed network management.
Finally ,the design and implemention of the data acquisition subsystem is discussed in detail at chapter 5. In this chapter, JMX is used to accomplish bottom layer’s function interacting with hardware device by its good scalability and compatibility .In upper level,data acquisition is divided into active acquisition and passive acquisition. NETFLOW,SNMP TRAP belong to passive acquisition, and the JMX Mbeans collect the data , save the data and transfer the data to alarm module. However, SNMP and CLI ,belonging to active acquisition , must be polled regularly by polling server. This paper brings up a solution of multi-thread asynchronous polling in EJB container. A single database is used to save all kinds of polling tasks, and by virtue of JBOSS clustering , a great number of polling tasks are balanced to the nodes of cluster. This way settles the problem of large number of device and complex protocols in data acquisition.
In order to promote the efficency of NETFLOW data receiving,we write the C language program receiving the NETFLOW data . The program is the only module implemented by none-J2EE framework. This paper expand on the cause and implementation.
面向对象技术和分布式计算技术的发展,特别是近来Java技术的快速发展和J2EE框架的不断完善,为分布式、统一的网络管理系统的实现提供了有效途径。因此,以J2EE技术为主要框架的IP网管是系统发展的方向,而数据采集子系统在网管系统中占据着重要的地位。本文详细阐述了IP网管数据采集子系统的设计和实现方法。
文章首先介绍了网络管理的一些基础知识,包括网络管理的基本概念、网络管理的五大功能以及网络管理的体系结构。
其次,文章分析了网管数据采集系统使用的主要协议,包括SNMP、NETFLOW以及CLI。
接着,文章介绍了J2EE的概念和框架结构,并讨论了把J2EE框架应用到分布式网络管理的技术优势。
文章最后详细描述了数据采集子系统的设计和实现。鉴于JMX具有完全的向后兼容性和良好的扩展性,我们在数据采集系统的设计上使用JMX实现了数据采集与协议相关的底层操作。在上层的实现中,把数据采集分为主动数据采集和被动数据采集。NETFLOW、SNMP TRAP属于被动采集的范畴,在JMX Mbean里面自动实现数据的采集、告警和入库操作。而SNMP、CLI属于主动采集,必须让采集服务器使用轮询操作来及时取得网管数据。设计中我们提出了在EJB容器内实现异步多线程轮询的解决方案。这一方案统一描述不同协议的轮询任务,通过轮询任务表集中控制轮询操作,并把轮询任务的执行分散到集群系统中,有效的解决了大规模网络轮询操作中网络设备数量庞大、协议多的问题。
为了提高数据接收的效率,我们用C语言编写了NETFLOW数据的接收程序,这是唯一一个没有在J2EE框架内实现的数据接收模块,文章会详细论述其中的原因和实现方法。
With the great development of Internet ,the IP network technology , as a carrier of business and communication, having got rapid growth in recent years. The business based on IP technology also make good time ,which make the network become more and more enormous, and increase the inconsistency between availability and size of the network. Today, excellent network management software is indispensable to high efficient network.
The great development of Object-Oriented technology and distributed computing, especially the fast growing of Java and J2EE framework, promote the implementation of distributed network management . So, the IP network management technology base on J2EE is the trend of future development, and the acquisition subsystem play a important role in network management system. This article will elaborate on the design and implement method of data acquisition subsystem.
This paper first introduces some base theory of network management technology, including base conception, main functions and framework of the network management.
Secondly, this paper analyzes some protocols used by data collectting, including SNMP, NETFLOW and CLI.
After that this paper gives a brief introduction to conception and framework of J2EE , demonstrate the great technology advantages to apply J2EE framework to distributed network management.
Finally ,the design and implemention of the data acquisition subsystem is discussed in detail at chapter 5. In this chapter, JMX is used to accomplish bottom layer’s function interacting with hardware device by its good scalability and compatibility .In upper level,data acquisition is divided into active acquisition and passive acquisition. NETFLOW,SNMP TRAP belong to passive acquisition, and the JMX Mbeans collect the data , save the data and transfer the data to alarm module. However, SNMP and CLI ,belonging to active acquisition , must be polled regularly by polling server. This paper brings up a solution of multi-thread asynchronous polling in EJB container. A single database is used to save all kinds of polling tasks, and by virtue of JBOSS clustering , a great number of polling tasks are balanced to the nodes of cluster. This way settles the problem of large number of device and complex protocols in data acquisition.
In order to promote the efficency of NETFLOW data receiving,we write the C language program receiving the NETFLOW data . The program is the only module implemented by none-J2EE framework. This paper expand on the cause and implementation.
引文
[1] 郭军.网络管理.北京邮电大学出版社,2001.
[2] 乐春晖.基于XML的三层C/S模型[J]. 计算机系统应用,2001
[3]Bruce Eckel. Thinking In Java, Second Edition.机械工业出版社,2002,9.
[4]W.Richard Stevens著,范建华,胥光辉,张涛等译,TCP/IP详解卷1:协议,2000.4
[5]李纯喜,郭宇春等,IP网络管理中的数据采集问题,电信科学,2001
[6] 郭军,网络管理,北京邮电大学出版社,2001.9
[7] .Linda G. DeMichiel, "Enterprise JavaBeansTM Specification,Version2.0",Sun Microsystems,Aug 2001.
[8] Ed Roman,"Mastering Enterprise JavaBeanTM(Seconde Edition)",2002.9.
[9] Cisco,"NetFlow Services and Applications", Cisco White Paper, Jun 2000.
[10] Sun, "Java Technology and XML", May 2002,http://java.sun.com/xm
[11] Management Information Base for Network Management of TCP/IP-based internets: MIB-II, RFC 1213, Performance Systems International, March 1991.
[12] W3C, "Extensible Markup Language (XML) 1.0", W3C Recommendation, May 2002,http://www.w3.org/TR/REC-xml.
[13] Scott Stark and Jboss Group, JBoss Administration and Development Second, April 14, 2003
[14] Marshall T. Rose: Management Information Base for Network Management of TCP/IP-based Internets: MIB-II, RFC 1213 (previously RFC 1158)[S], March 1991.
[15] Sun Microsystems, Inc,JavaTM Management Extensions (JMX) v1.2 Specification. http://jcp.org/aboutJava/communityprocess/final/jsr003/index3.html. September 2, 2002.
[16] OMG. The common object request broker[j].Architecture and Specification
[17] Scott Stark,Integrate security infrastructures with JbossSX,http://www.javaworld.com/
[18] Java2 Platform Enterprise Edition Specification,V1.4
[19] Floyd Marinescu,EJB Design Patterns,Wiley Computer Publishing
[20] Java Naming and Directory Interface specifications, http:// java.sun.com/products/jndi/.
[2] 乐春晖.基于XML的三层C/S模型[J]. 计算机系统应用,2001
[3]Bruce Eckel. Thinking In Java, Second Edition.机械工业出版社,2002,9.
[4]W.Richard Stevens著,范建华,胥光辉,张涛等译,TCP/IP详解卷1:协议,2000.4
[5]李纯喜,郭宇春等,IP网络管理中的数据采集问题,电信科学,2001
[6] 郭军,网络管理,北京邮电大学出版社,2001.9
[7] .Linda G. DeMichiel, "Enterprise JavaBeansTM Specification,Version2.0",Sun Microsystems,Aug 2001.
[8] Ed Roman,"Mastering Enterprise JavaBeanTM(Seconde Edition)",2002.9.
[9] Cisco,"NetFlow Services and Applications", Cisco White Paper, Jun 2000.
[10] Sun, "Java Technology and XML", May 2002,http://java.sun.com/xm
[11] Management Information Base for Network Management of TCP/IP-based internets: MIB-II, RFC 1213, Performance Systems International, March 1991.
[12] W3C, "Extensible Markup Language (XML) 1.0", W3C Recommendation, May 2002,http://www.w3.org/TR/REC-xml.
[13] Scott Stark and Jboss Group, JBoss Administration and Development Second, April 14, 2003
[14] Marshall T. Rose: Management Information Base for Network Management of TCP/IP-based Internets: MIB-II, RFC 1213 (previously RFC 1158)[S], March 1991.
[15] Sun Microsystems, Inc,JavaTM Management Extensions (JMX) v1.2 Specification. http://jcp.org/aboutJava/communityprocess/final/jsr003/index3.html. September 2, 2002.
[16] OMG. The common object request broker[j].Architecture and Specification
[17] Scott Stark,Integrate security infrastructures with JbossSX,http://www.javaworld.com/
[18] Java2 Platform Enterprise Edition Specification,V1.4
[19] Floyd Marinescu,EJB Design Patterns,Wiley Computer Publishing
[20] Java Naming and Directory Interface specifications, http:// java.sun.com/products/jndi/.