即时通文件传输审计系统的研究与实现
摘要
即时通讯软件(IM)作为一种便捷的网络通讯技术已成为广大网民工作生活所不可或缺的信息交流平台,与此同时即时通讯软件的安全问题越来越受到重视。设计实现能实时审计用户传输文件的即时通讯文件审计系统有重要的意义。
本文首先介绍了即时通讯软件标准通信模型,并对当前即时通讯软件两个通用协议XMPP (Extensible Messageing and Presence Protocol,可扩展消息与存在)协议及SIMPLE (SIP for Instant Messaging and Presence Leveraging Extensions)协议进行了分析对比。然后通过搭建协议分析环境,在大量测试分析的基础上,提取总结出了Fetion、QQ、MSN和雅虎通四种即时通讯软件文件传输的应用层特征。基于此,本文对Linux平台网络数据包捕获及处理技术进行了研究,并给出了Linux平台下基于LIBPCAP库进行数据包捕获及协议分析的过程和方法。在此基础上,通过对需求的分析和系统目标的总结,对即时通讯软件文件传输协议特征及特征匹配方法进行了深入的研究,设计出一个通用的即时通讯软件文件审计系统。基于即时通讯软件文件审计系统的基本功能,将系统模块化并逐步实现,其中涉及到深度包检测、文件重组还原中的协议切换、TCP重传、协议切换、TCP“粘包”等关键技术。本文在测试床上构建的网络环境及真实网络环境中对系统进行了测试,评定了系统的性能。系统可以有效的审计IM传输的文件,不受文件类型、文件大小及网络环境的限制。在网络环境突变、网络状况异常恶劣(网速<10K、丢包率10%)的条件下,也可正常审计文件。
本文最后对系统的研究与开发工作进行了简要总结,并简要阐述了下一步对该系统进行扩充与完善的工作。
The Instant messenger (IM), an essential tool to people's daily life, has brought users a lot of convenience. Howerver, it also imperils the nation and society's security, since the IMs make the spread of reaction and eroticism information easier.Therefore, research and designing of instant messenger files audit system is important to information security.
Firstly, this thesis categorizes and introduces standard communication model IMPP for IM and two well-known IM protocol XMPP and SIMPLE in accordance with the model. Most of commonly used IMs use their own proprietary which are developed by the service providers their own. In view of this, the characteristics of the application layer protocol are unknown. Based on the building of the environment of the protocol analysis, we have a deep analysis and research on all kinds of instant messenger protocols and combined with relevant technology, this paper summarized the protocol characters of the Fetion, MSN, QQ and yahoo using the contrasting method.
In the following, network packet capturing and processing technique on Linux platform is dicussed and the processes of protocol analysis based on Libpcap are presented. Base on this, we analyze the requirement and summarize the target of the system to proposal a solution for IM File Aduit system. With the basic functions, we partition the whole system to many modules and accomplish them step by step. The implementation refers to many important technologies, such as deep packet inspection, TCP stick package, data compression, TCP retransmission and so on. Then, some tests and analysis result of the system are given using test bed and and real network enviroment. It can be seen from the test results that FAudit can audit all kinds of files such as.txt,.doc,.pdf,.rmvb,.jpg and so on in various network environments no matter however bad it is.
Finally, the thesis summarizes the current existing open issues, and provides some further directions.
本文首先介绍了即时通讯软件标准通信模型,并对当前即时通讯软件两个通用协议XMPP (Extensible Messageing and Presence Protocol,可扩展消息与存在)协议及SIMPLE (SIP for Instant Messaging and Presence Leveraging Extensions)协议进行了分析对比。然后通过搭建协议分析环境,在大量测试分析的基础上,提取总结出了Fetion、QQ、MSN和雅虎通四种即时通讯软件文件传输的应用层特征。基于此,本文对Linux平台网络数据包捕获及处理技术进行了研究,并给出了Linux平台下基于LIBPCAP库进行数据包捕获及协议分析的过程和方法。在此基础上,通过对需求的分析和系统目标的总结,对即时通讯软件文件传输协议特征及特征匹配方法进行了深入的研究,设计出一个通用的即时通讯软件文件审计系统。基于即时通讯软件文件审计系统的基本功能,将系统模块化并逐步实现,其中涉及到深度包检测、文件重组还原中的协议切换、TCP重传、协议切换、TCP“粘包”等关键技术。本文在测试床上构建的网络环境及真实网络环境中对系统进行了测试,评定了系统的性能。系统可以有效的审计IM传输的文件,不受文件类型、文件大小及网络环境的限制。在网络环境突变、网络状况异常恶劣(网速<10K、丢包率10%)的条件下,也可正常审计文件。
本文最后对系统的研究与开发工作进行了简要总结,并简要阐述了下一步对该系统进行扩充与完善的工作。
The Instant messenger (IM), an essential tool to people's daily life, has brought users a lot of convenience. Howerver, it also imperils the nation and society's security, since the IMs make the spread of reaction and eroticism information easier.Therefore, research and designing of instant messenger files audit system is important to information security.
Firstly, this thesis categorizes and introduces standard communication model IMPP for IM and two well-known IM protocol XMPP and SIMPLE in accordance with the model. Most of commonly used IMs use their own proprietary which are developed by the service providers their own. In view of this, the characteristics of the application layer protocol are unknown. Based on the building of the environment of the protocol analysis, we have a deep analysis and research on all kinds of instant messenger protocols and combined with relevant technology, this paper summarized the protocol characters of the Fetion, MSN, QQ and yahoo using the contrasting method.
In the following, network packet capturing and processing technique on Linux platform is dicussed and the processes of protocol analysis based on Libpcap are presented. Base on this, we analyze the requirement and summarize the target of the system to proposal a solution for IM File Aduit system. With the basic functions, we partition the whole system to many modules and accomplish them step by step. The implementation refers to many important technologies, such as deep packet inspection, TCP stick package, data compression, TCP retransmission and so on. Then, some tests and analysis result of the system are given using test bed and and real network enviroment. It can be seen from the test results that FAudit can audit all kinds of files such as.txt,.doc,.pdf,.rmvb,.jpg and so on in various network environments no matter however bad it is.
Finally, the thesis summarizes the current existing open issues, and provides some further directions.
引文
[1]李远杰,刘渭锋,张玉清等.主流即时通信软件通信协议分析.计算机应用研究,2005,22(7)：243～245
[2]Yahoo Messenger. http:/cn.messenger.yahoo.com/
[3]MSN.http://messcnger.live.cn/
[4]QQ.http://im.qq.com/
[5]Fetion. http://www.fetion.com/
[6]LINCK.A study of monitoring technologies in instant messenger [EB/OL]. http://thesis.lib.ncu.edu.tw/ETD-db/ETD-search/getfile?URN=93523042%fil ename=93523042.pdf
[7]Xiao Z, Guo L, Tracey J. Understanding instant messaging traffic characteristics. In:proc. of 27th International Conference on Dislfibuted Computing Systems,2007:51-58
[8]Liu Z J, Linw L, Lin, et al. Detecting and filtering instant messaging sparn- a global and personalized approach. In:proc. of 1st/EEE ICNP Workshop on Secure Network Protocols.2005:9-24
[9]Sangkyun K, Hoon L. Implementation of the security system for instant messengers. In:proc. of The International Symposium on Computational and Information Sciences (cis'04).2004:739-744
[10]付安民.即时通实时监控系统的设计与实现.通信学报,2008,29(10):165～172
[11]付安民.即时通监控系统的设计与实现：[硕士学位论文].西安：西安电子科技大学,2008
[12]M. Day, J. Rosenberg, H. Sugano. RFC2778:A Model for Presence and Instant Messaging.2000
[13]M. Day, S. Aggarwal, G. Mohr, J. Vincent. RFC 2779:Instant Messaging/Presence Protocol Requirements.2000
[14]J. Peterson, NeuStar. RFC 3859:Common Profile for Presence (CPP).2004
[15]J. Peterson, NeuStar. Common Profile for Instant Messaging (CPIM).2004
[16]G. Klyne, D. Atkins. RFC 3860:Common Presence and Instant Messaging (CPIM):Message Format.2004
[17]H. Sugano, S. Fujimoto, G. Klyne, A. Bateman, W. Carr, J. Peterson. RFC3863: Presence Information Data Format (PIDF).2004
[18]J. Peterson, NeuStar. RFC 3861:Address Resolution for Instant Messaging and Presence.2004
[19]P. Saint-Andre, et al. RFC 3920:Extensible Messaging and Presence Protocol (XMPP):Core.2004
[20]P. Saint-Andre, et al. RFC 3921:Extensible Messaging and Presence Protocol (XMPP):Instant Messaging and Presence.2004
[21]P. Saint-Andre, et al. RFC 3922:Mapping the Extensible Messaging and Presence Protocol (XMPP) to Common Presence and Instant Messaging (CPIM).2004
[22]P. Saint-Andre, et al. RFC 3923:End-to-End Signing and Object Encryption for the Extensible Messaging and Presence Protocol (XMPP).2004
[23]杨斌.XMPP协议分析与应用探讨.微型机与应用,2005,8:10～11.
[24]J. Myers. RFC 2222:Simple Authentication and Security Layer (SASL).1997
[25]T. Dierks, C. Allen. RFC 2246:The TLS Protocol Version 1.0.1999
[26]B. Campbell, Ed, J. Rosenberg, H. Schulzrinne, C. Huitema, D. Gurle. RFC 3428:Session Initiation Protocol (SIP) Extension for Instant Messaging.2002
[27]J. Rosenberg. RFC 3856:A Presence Event Package for the Session Initiation Protocol (SIP).2004
[28]J. Rosenberg. RFC 3857:A Watcher Information Event Template-Package for the Session Initiation Protocol (SIP).2004
[29]J. Rosenberg. RFC 3858:An Extensible Markup Language (XML) Based Format for Watcher Information.2004
[30]H. Schulzrinne. RFC 3994:Indication of Message Composition for Instant Messaging.2005
[31]张云川.标准化的即时通信协议-SIMPLE和XMPP的对比研究.武汉科技大学学报,2005,28(3)：375～377
[32]刘志治,李晓峰.基于SIP协议的即时信息机制.北京邮电大学学报,2004,27(3)：137～142
[33]马严.IPv6下基于SIP/SIMPLE协议IM的研究与实现：[硕士学位论文].北京：北京邮电大学,2009
[34]A B Roach. RFC3265:Session initiation protocol (sip)-specific event notification.2002
[35]H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson. RFC 3550:RTP:A Transport Protocol for Real-Time Applications.2003
[36]Mourad Debbabi, Mahfuzur Rahman. The War of Presence and Instant Messaging:Right Protocols and APIs. In:proc. Of Consumer Communications and Networking Conference.2004:341-346
[37]Wendy A. Kellogg. A Conversation with Peter Ford. In:ACM Queue,2003, 1(8):18～27
[38]彭城.基于wireshark的协议分析研究与扩展实验：[硕士学位论文].成都：电子科技大学,2007
[39]Jacobson V, Leres C, McCanne S. Packet Capturing Library, Lawrence Berkeley National Laboratory [EB/OL]. http://www.org.ee.lbl.gov
[40]Kim H M, Michelena N F, Papalambros P Y, et al. Target cascading in optimal system design. ASME Journal of Mechanical Design,2003,125(3):475～480
[41]BPF数据包过滤机制http://www.csdn.net
[42]平镇宇Libpcap数据包捕获机制剖析与研究.信息网络安全,2008(8):37～39
[43]Eric A.Hall著,张金辉译Internet核心协议权威指南.北京：人民邮电出版社,2002
[44]Kenneth D.Reed著,孙坦,张学峰,扬琳等译.协议分析.北京：电子工业出版社,2002
[45]W.Richard Stevens,范建华等译TCP/IP lustracted Volume1:The Protocols,机械工业出版社,2000
[46]Behrouz A.Forouzan, Sophia Chung Fegan著,谢希仁译TCP/IP协议族.北京：清华大学出版社,2001
[47]Andrew S.Tanenbaum著,熊桂喜等译.计算机网络(第三版).北京：清华大学出版社,2003
[48]Sailesh Kumar, Jonathan Turner, John Williams. Advanced algorithms for fast and sealable deep packet inspection. In. Proc. of the 2006 ACM/IEEE symposium on Architecture for networking and communications systems. 2006:81～92
[49]牟乔.准确高效的应用层协议分析识别方法.计算机科学与工程,2010,32(8)：39～45
[50]刘胤.深度包检测技术的研究与设计.贵州：贵州大学,2008
[51]陈亮,龚俭,徐选.应用层协议识别算法综述.计算机科学,2007,34(7):73～75
[52]Wright G R, Stevens W R陆雪莹等(译)TCP/IP详解(卷2).北京：机械工业出版社,2008
[53]王丰锦,邵新宇,喻道远,李培根.基于Socket和多线程的应用程序间通信技术的研究.计算机应用技术,2000,20(6)：65～67
[54]沈枫.基于网络的实时仿真支撑平台研究.武汉：武汉理工大学,2007
[55]Behrouz A Forouzan, Sophia Chung Fegan谢希仁等(译).TCP/IP协议族(第3版).北京：清华大学出版社,2006
[56]Nagle J. RFC896:Congestion control in IP/TCP Internet works. Internet Engineering Task Force,1984
[57]Internet2 NetFlow:Weekly reports.http://netflow.internet2.edu/weekly/.2009, 11
[58]赵丽莉,孙伟.TCP协议乱序数据包处理算法综述.软件工程师,2010:61～63
[59]Bennett JCR, Partridge C, Shectman N. Packet Reordering is Not Pathological Network Behavior. IEEE/ACM Transactiona on Networking,1999,7(6): 789-798
[60]Allman M, Paxson V, Stevens w. RFC 2581:TCP Congestion Control. Internet Engineering Task Force,1999
[61]The Wide Area Network emulator.http://wanem.sourceforge.net/
[2]Yahoo Messenger. http:/cn.messenger.yahoo.com/
[3]MSN.http://messcnger.live.cn/
[4]QQ.http://im.qq.com/
[5]Fetion. http://www.fetion.com/
[6]LINCK.A study of monitoring technologies in instant messenger [EB/OL]. http://thesis.lib.ncu.edu.tw/ETD-db/ETD-search/getfile?URN=93523042%fil ename=93523042.pdf
[7]Xiao Z, Guo L, Tracey J. Understanding instant messaging traffic characteristics. In:proc. of 27th International Conference on Dislfibuted Computing Systems,2007:51-58
[8]Liu Z J, Linw L, Lin, et al. Detecting and filtering instant messaging sparn- a global and personalized approach. In:proc. of 1st/EEE ICNP Workshop on Secure Network Protocols.2005:9-24
[9]Sangkyun K, Hoon L. Implementation of the security system for instant messengers. In:proc. of The International Symposium on Computational and Information Sciences (cis'04).2004:739-744
[10]付安民.即时通实时监控系统的设计与实现.通信学报,2008,29(10):165～172
[11]付安民.即时通监控系统的设计与实现：[硕士学位论文].西安：西安电子科技大学,2008
[12]M. Day, J. Rosenberg, H. Sugano. RFC2778:A Model for Presence and Instant Messaging.2000
[13]M. Day, S. Aggarwal, G. Mohr, J. Vincent. RFC 2779:Instant Messaging/Presence Protocol Requirements.2000
[14]J. Peterson, NeuStar. RFC 3859:Common Profile for Presence (CPP).2004
[15]J. Peterson, NeuStar. Common Profile for Instant Messaging (CPIM).2004
[16]G. Klyne, D. Atkins. RFC 3860:Common Presence and Instant Messaging (CPIM):Message Format.2004
[17]H. Sugano, S. Fujimoto, G. Klyne, A. Bateman, W. Carr, J. Peterson. RFC3863: Presence Information Data Format (PIDF).2004
[18]J. Peterson, NeuStar. RFC 3861:Address Resolution for Instant Messaging and Presence.2004
[19]P. Saint-Andre, et al. RFC 3920:Extensible Messaging and Presence Protocol (XMPP):Core.2004
[20]P. Saint-Andre, et al. RFC 3921:Extensible Messaging and Presence Protocol (XMPP):Instant Messaging and Presence.2004
[21]P. Saint-Andre, et al. RFC 3922:Mapping the Extensible Messaging and Presence Protocol (XMPP) to Common Presence and Instant Messaging (CPIM).2004
[22]P. Saint-Andre, et al. RFC 3923:End-to-End Signing and Object Encryption for the Extensible Messaging and Presence Protocol (XMPP).2004
[23]杨斌.XMPP协议分析与应用探讨.微型机与应用,2005,8:10～11.
[24]J. Myers. RFC 2222:Simple Authentication and Security Layer (SASL).1997
[25]T. Dierks, C. Allen. RFC 2246:The TLS Protocol Version 1.0.1999
[26]B. Campbell, Ed, J. Rosenberg, H. Schulzrinne, C. Huitema, D. Gurle. RFC 3428:Session Initiation Protocol (SIP) Extension for Instant Messaging.2002
[27]J. Rosenberg. RFC 3856:A Presence Event Package for the Session Initiation Protocol (SIP).2004
[28]J. Rosenberg. RFC 3857:A Watcher Information Event Template-Package for the Session Initiation Protocol (SIP).2004
[29]J. Rosenberg. RFC 3858:An Extensible Markup Language (XML) Based Format for Watcher Information.2004
[30]H. Schulzrinne. RFC 3994:Indication of Message Composition for Instant Messaging.2005
[31]张云川.标准化的即时通信协议-SIMPLE和XMPP的对比研究.武汉科技大学学报,2005,28(3)：375～377
[32]刘志治,李晓峰.基于SIP协议的即时信息机制.北京邮电大学学报,2004,27(3)：137～142
[33]马严.IPv6下基于SIP/SIMPLE协议IM的研究与实现：[硕士学位论文].北京：北京邮电大学,2009
[34]A B Roach. RFC3265:Session initiation protocol (sip)-specific event notification.2002
[35]H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson. RFC 3550:RTP:A Transport Protocol for Real-Time Applications.2003
[36]Mourad Debbabi, Mahfuzur Rahman. The War of Presence and Instant Messaging:Right Protocols and APIs. In:proc. Of Consumer Communications and Networking Conference.2004:341-346
[37]Wendy A. Kellogg. A Conversation with Peter Ford. In:ACM Queue,2003, 1(8):18～27
[38]彭城.基于wireshark的协议分析研究与扩展实验：[硕士学位论文].成都：电子科技大学,2007
[39]Jacobson V, Leres C, McCanne S. Packet Capturing Library, Lawrence Berkeley National Laboratory [EB/OL]. http://www.org.ee.lbl.gov
[40]Kim H M, Michelena N F, Papalambros P Y, et al. Target cascading in optimal system design. ASME Journal of Mechanical Design,2003,125(3):475～480
[41]BPF数据包过滤机制http://www.csdn.net
[42]平镇宇Libpcap数据包捕获机制剖析与研究.信息网络安全,2008(8):37～39
[43]Eric A.Hall著,张金辉译Internet核心协议权威指南.北京：人民邮电出版社,2002
[44]Kenneth D.Reed著,孙坦,张学峰,扬琳等译.协议分析.北京：电子工业出版社,2002
[45]W.Richard Stevens,范建华等译TCP/IP lustracted Volume1:The Protocols,机械工业出版社,2000
[46]Behrouz A.Forouzan, Sophia Chung Fegan著,谢希仁译TCP/IP协议族.北京：清华大学出版社,2001
[47]Andrew S.Tanenbaum著,熊桂喜等译.计算机网络(第三版).北京：清华大学出版社,2003
[48]Sailesh Kumar, Jonathan Turner, John Williams. Advanced algorithms for fast and sealable deep packet inspection. In. Proc. of the 2006 ACM/IEEE symposium on Architecture for networking and communications systems. 2006:81～92
[49]牟乔.准确高效的应用层协议分析识别方法.计算机科学与工程,2010,32(8)：39～45
[50]刘胤.深度包检测技术的研究与设计.贵州：贵州大学,2008
[51]陈亮,龚俭,徐选.应用层协议识别算法综述.计算机科学,2007,34(7):73～75
[52]Wright G R, Stevens W R陆雪莹等(译)TCP/IP详解(卷2).北京：机械工业出版社,2008
[53]王丰锦,邵新宇,喻道远,李培根.基于Socket和多线程的应用程序间通信技术的研究.计算机应用技术,2000,20(6)：65～67
[54]沈枫.基于网络的实时仿真支撑平台研究.武汉：武汉理工大学,2007
[55]Behrouz A Forouzan, Sophia Chung Fegan谢希仁等(译).TCP/IP协议族(第3版).北京：清华大学出版社,2006
[56]Nagle J. RFC896:Congestion control in IP/TCP Internet works. Internet Engineering Task Force,1984
[57]Internet2 NetFlow:Weekly reports.http://netflow.internet2.edu/weekly/.2009, 11
[58]赵丽莉,孙伟.TCP协议乱序数据包处理算法综述.软件工程师,2010:61～63
[59]Bennett JCR, Partridge C, Shectman N. Packet Reordering is Not Pathological Network Behavior. IEEE/ACM Transactiona on Networking,1999,7(6): 789-798
[60]Allman M, Paxson V, Stevens w. RFC 2581:TCP Congestion Control. Internet Engineering Task Force,1999
[61]The Wide Area Network emulator.http://wanem.sourceforge.net/