基于穿越NAT网络的点到点数据通信的研究与实现
|
摘要
本文以家庭远程监控系统为研究背景,为在广域网上实现点到点高速可靠数据传输而展开研究。由于中国家庭的网络环境一般处于NAT之后,外网一般不能直接访问内网,为实现对等的点到点通信就必须过穿越NAT这一关。
本文先对NAT种类和特性进行介绍,并在点到点应用中实现对NAT的穿越,针对不同的网络环境采用不同的NAT穿越技术：从家庭监控设备有公网IP时的主动直连,到家庭位于NAT之后而监控者处于公网的反向连接,从双方都处于EIMNAT时的UDP打洞,到能够穿越任何NAT但大量消耗服务资源的服务器中转,依次将当今主流NAT穿透技术应用于家庭远程监控系统以实现点到点的数据传输。
在此基础上,为实现数据的可靠传输,在UDP打洞时,引入基于UDP的可靠数据传输协议UDT,在介绍如何使用UDT传输数据流的同时,对UDT进行改造以增强其对网络带宽的抢占能力。
当使用公网服务器协助点到点通信双方穿越NAT时,为实现服务程序的高并发、快响应、长稳定,为满足服务程序开发维护过程中的高敏捷、低成本、易扩展,使用半同步/半异步的线程池调度模式作为服务程序的架构。本文更进一步的将半同步/半异步调度模式针对多核系统进行优化和改进,提出了基于任务关联度的任务分配策略和基于资源利用率的任务调度算法,以提高服务器响应速度,提升多核处理器的使用效率,使多核处理器负载更加均衡。
This paper based on the background of home remote monitoring system, does research on the reliable and high efficient Peer-to-Peer Communication over WAN. In China, the communication endpoint located in home network is usually behind NAT, the endpoint from public network cannot directly access the inside one and have to establish P2P communication with the help of NAT traversal.
This paper firstly introduces the different types of NAT behavior, and proposes a complete set of solutions for NAT traversal used in P2P applications:the client in the private network can initiate connection to the device which in the public and vice versa; if the two peers behind different EIM NATs, we use UDP Hole Punching to traverse NAT; if all of the above method won't work effectively, we use a public server to relay messages to each other. All these technologies are widely used in nowadays, and we apply them to our home remote monitoring system.
Further more, we introduce a reliable UDP based application level data transport protocol—UDT to transfer data reliably after punching UDP hole to traverse NAT successfully. We not only learn to use UDT but also improve it to enhance the ability to seize the bandwidth.
When using a public server to assist the P2P communication to traverse NAT, how to implement high concurrency service app with response quickly and running stable is our purpose, how to develop agile service app with cost acceptable and easy to extend is our goal, the author according to the characteristics of multi-core processor, selected Half-Sync/Half-Async pattern as the research object, proposed the task allocating strategy based on correlation degree between tasks and the task scheduling algorithm based on resource utilization to improve Half-Sync/Half-Async pattern.
本文先对NAT种类和特性进行介绍,并在点到点应用中实现对NAT的穿越,针对不同的网络环境采用不同的NAT穿越技术：从家庭监控设备有公网IP时的主动直连,到家庭位于NAT之后而监控者处于公网的反向连接,从双方都处于EIMNAT时的UDP打洞,到能够穿越任何NAT但大量消耗服务资源的服务器中转,依次将当今主流NAT穿透技术应用于家庭远程监控系统以实现点到点的数据传输。
在此基础上,为实现数据的可靠传输,在UDP打洞时,引入基于UDP的可靠数据传输协议UDT,在介绍如何使用UDT传输数据流的同时,对UDT进行改造以增强其对网络带宽的抢占能力。
当使用公网服务器协助点到点通信双方穿越NAT时,为实现服务程序的高并发、快响应、长稳定,为满足服务程序开发维护过程中的高敏捷、低成本、易扩展,使用半同步/半异步的线程池调度模式作为服务程序的架构。本文更进一步的将半同步/半异步调度模式针对多核系统进行优化和改进,提出了基于任务关联度的任务分配策略和基于资源利用率的任务调度算法,以提高服务器响应速度,提升多核处理器的使用效率,使多核处理器负载更加均衡。
This paper based on the background of home remote monitoring system, does research on the reliable and high efficient Peer-to-Peer Communication over WAN. In China, the communication endpoint located in home network is usually behind NAT, the endpoint from public network cannot directly access the inside one and have to establish P2P communication with the help of NAT traversal.
This paper firstly introduces the different types of NAT behavior, and proposes a complete set of solutions for NAT traversal used in P2P applications:the client in the private network can initiate connection to the device which in the public and vice versa; if the two peers behind different EIM NATs, we use UDP Hole Punching to traverse NAT; if all of the above method won't work effectively, we use a public server to relay messages to each other. All these technologies are widely used in nowadays, and we apply them to our home remote monitoring system.
Further more, we introduce a reliable UDP based application level data transport protocol—UDT to transfer data reliably after punching UDP hole to traverse NAT successfully. We not only learn to use UDT but also improve it to enhance the ability to seize the bandwidth.
When using a public server to assist the P2P communication to traverse NAT, how to implement high concurrency service app with response quickly and running stable is our purpose, how to develop agile service app with cost acceptable and easy to extend is our goal, the author according to the characteristics of multi-core processor, selected Half-Sync/Half-Async pattern as the research object, proposed the task allocating strategy based on correlation degree between tasks and the task scheduling algorithm based on resource utilization to improve Half-Sync/Half-Async pattern.
引文
[1]李笑凇.基于IP技术的网络远程视频监控系统的研究与实现[学位论文].贵州大学,2007：6-8
[2]马义涛.基于P2P网络应用的NAT穿越方案的分析与设计[学位论文].上海交通大学,2008：9-11
[3]Ford. B., Srisuresh. P., Kegel. D. Peer-to-Peer Communication Across Network Address Translators. Proceedings of the USENIX Annual Technical Conference (Anaheim, CA), 2005,4: 179-192
[4]Ford. B. NAT Check—check Your Network Address Translator for Compatibility with Peer-to-Peer Protocols. http://midcom-p2p.sourceforge.net
[5]Jennings, C. NAT Classification Test Results. http://tools.ietf.org/id/draft-jennings-behave-test-results.
[6]Guha, Saikat. NAT STUNT Results. http://saikat.guha.cc/stunt-results.php
[7]Guha, S., and Francis, P. Characterization and Measurement of TCP Traversal through NATs and Firewalls. Proceedings of Internet Measurement Conference (IMC), Berkeley, CA, 2005,10: 199-211
[8]Biggadike, A., Ferullo, D., Wilson, G. Establishing TCP Connections Between Hosts Behind NATs. ACM SIGCOMM ASIA Workshop, Beijing, China, 2005,4:121-131
[9]Huitema, C. Teredo:Tunneling IPv6 over UDP through Network Address Translations (NATs). RFC 4380,2006,2:35-42
[10]Rosenberg, J. Interactive Connectivity Establishment (ICE):A Methodology for Network Address Translator (NAT) Traversal for Offer/Answer Protocols. RFC 5245,2007,10:23-27
[11]Rosenberg, J., Weinberger, J., Huitema, C. STUN - Simple Traversal of User Datagram Protocol (UDP) Through Network Address Translators (NATs). RFC 3489, 2003,3:41-45
[12]Rosenberg, J. TCP Candidates with Interactive Connectivity Establishment (ICE). RFC 6544, 2007,7:13-15
[13]Audet, F., Ed., C. Jennings. Network Address Translation (NAT) Behavioral Requirements for Unicast UDP. RFC 4787,2007,1:5-13
[14]Srisuresh. P, Ford. B. State of Peer-to-Peer (P2P) Communication across Network Address Translators (NATs).RFC 5128,2008,3:4-7
[15]彭中平,李慧霸,张红雷.基于UDT协议的NAT穿越技术探讨.科技信息,2009,2：6-8
[16]He K, Zhao Y, Huang WQ. A Clustering and Scheduling Algorithm Based on Task Duplication. Chinese Journal of Computers, 2008, 31(5):733-740
[17]吴佳俊.多核多线程处理器上任务调度技术研究[学位论文].中国科学院计算技术研究所,2006，9-11
[18]S.H Bokhari. Dual Processor Scheduling with Dynamic Reassignment. IEEE Transactions on Software Engineering, 1979, 7(4):341-349
[19]刘振英,方滨兴,胡铭曾,张毅.一个有效的动态负载平衡方法.软件学报,2001,12(4)：563-568
[20]梁根,郭小雪,秦勇.基于公平调度算法的分布式系统负载均衡研究.计算机工程与设计,2008,29(6)：1362-1365
[21]廖羽,戴瑜兴.基于内容的分布式web服务器负载平衡算法.电子学报,2006,34(6)：1053-1057
[22]S.Zhou, X.Z.Delisle. Utopia: a Load Sharing Facility for Large Heterogeneous Distributed Computer Systems. Software Practice and Experience, 1993:1305-1336
[23]李刚,金蓓弘.基于线程的并发控制技术研究与应用.计算机工程,2007,33(14)：43-45
[24]Schmidt D C, Stal M, Rohnert H面向模式的软件体系结构,卷2：用于并发和网络化对象的模式.北京：机械工业出版社,2003：115-117
[25]Douglas Schmidt, Carlos O'Ryan, Michael Kircher. Leader/Followers. http://www.cs.wustl.edu/-schmidt/PDF/lf.pdf
[26]Angeles Navarro, Rafael Asenjo, Siham Tabik. Analytical Modeling of Pipeline Parallelism. In PACT'09,2009:281-290
[27]Douglas Schmidt,. Half Sync/Half Async. http://www.cs.wustl.edu/~schmidt/PDF/HS-HA.pdf
[28]刘燚.半同步半异步线程池的设计与实现.计算机与现代化,2010,7：38-40
[29]李刚,金蓓弘.两种线程池的实现和性能评价.计算机工程与设计,2007,28(7)：1489-1492
[30]王华,马亮,顾明.线程池技术研究与应用.计算机应用研究,2005,22(11)：141-144
[31]欧阳志鹏,沈富可.基于改进线程池技术服务器程序的设计与实现.计算机与数字工程,2005,33(10)：133-136
[32]刘轶,张昕,李鹤.多核处理器大规模并行系统中的任务分配问题及算法.小型微型计算机系统,2008,29(5)：972-975
[33]陈华平,计永昶,陈国良.分布式动态负载平衡调度的一个通用模型.软件学报,1998,9(1)：25-29
[34]鞠九滨,杨鯤,徐高潮.使用资源利用率作为负载平衡系统的负载指标.软件学报,1996,7(4)：238-242
[35]王力生,毛昀波.多处理机系统的负载平衡模型设计.单片机与嵌入式系统应用,2008,4：10-13
[36]Mercury Interactive. LoadRunner Controller User's Guide Version 7.6[EB/OL]. http://genesis.co.kr/image/product/pds/LoadRunner/Manual/LoadRunner_Controller.pdf
[37]Mercury Interactive. LoadRunner Analysis User's Guide Version 7.6[EB/OL]. http://genesis.co.kr/image/product/pds/LoadRunner/Manual/LoadRunner_Analysis.pdf
[2]马义涛.基于P2P网络应用的NAT穿越方案的分析与设计[学位论文].上海交通大学,2008：9-11
[3]Ford. B., Srisuresh. P., Kegel. D. Peer-to-Peer Communication Across Network Address Translators. Proceedings of the USENIX Annual Technical Conference (Anaheim, CA), 2005,4: 179-192
[4]Ford. B. NAT Check—check Your Network Address Translator for Compatibility with Peer-to-Peer Protocols. http://midcom-p2p.sourceforge.net
[5]Jennings, C. NAT Classification Test Results. http://tools.ietf.org/id/draft-jennings-behave-test-results.
[6]Guha, Saikat. NAT STUNT Results. http://saikat.guha.cc/stunt-results.php
[7]Guha, S., and Francis, P. Characterization and Measurement of TCP Traversal through NATs and Firewalls. Proceedings of Internet Measurement Conference (IMC), Berkeley, CA, 2005,10: 199-211
[8]Biggadike, A., Ferullo, D., Wilson, G. Establishing TCP Connections Between Hosts Behind NATs. ACM SIGCOMM ASIA Workshop, Beijing, China, 2005,4:121-131
[9]Huitema, C. Teredo:Tunneling IPv6 over UDP through Network Address Translations (NATs). RFC 4380,2006,2:35-42
[10]Rosenberg, J. Interactive Connectivity Establishment (ICE):A Methodology for Network Address Translator (NAT) Traversal for Offer/Answer Protocols. RFC 5245,2007,10:23-27
[11]Rosenberg, J., Weinberger, J., Huitema, C. STUN - Simple Traversal of User Datagram Protocol (UDP) Through Network Address Translators (NATs). RFC 3489, 2003,3:41-45
[12]Rosenberg, J. TCP Candidates with Interactive Connectivity Establishment (ICE). RFC 6544, 2007,7:13-15
[13]Audet, F., Ed., C. Jennings. Network Address Translation (NAT) Behavioral Requirements for Unicast UDP. RFC 4787,2007,1:5-13
[14]Srisuresh. P, Ford. B. State of Peer-to-Peer (P2P) Communication across Network Address Translators (NATs).RFC 5128,2008,3:4-7
[15]彭中平,李慧霸,张红雷.基于UDT协议的NAT穿越技术探讨.科技信息,2009,2：6-8
[16]He K, Zhao Y, Huang WQ. A Clustering and Scheduling Algorithm Based on Task Duplication. Chinese Journal of Computers, 2008, 31(5):733-740
[17]吴佳俊.多核多线程处理器上任务调度技术研究[学位论文].中国科学院计算技术研究所,2006，9-11
[18]S.H Bokhari. Dual Processor Scheduling with Dynamic Reassignment. IEEE Transactions on Software Engineering, 1979, 7(4):341-349
[19]刘振英,方滨兴,胡铭曾,张毅.一个有效的动态负载平衡方法.软件学报,2001,12(4)：563-568
[20]梁根,郭小雪,秦勇.基于公平调度算法的分布式系统负载均衡研究.计算机工程与设计,2008,29(6)：1362-1365
[21]廖羽,戴瑜兴.基于内容的分布式web服务器负载平衡算法.电子学报,2006,34(6)：1053-1057
[22]S.Zhou, X.Z.Delisle. Utopia: a Load Sharing Facility for Large Heterogeneous Distributed Computer Systems. Software Practice and Experience, 1993:1305-1336
[23]李刚,金蓓弘.基于线程的并发控制技术研究与应用.计算机工程,2007,33(14)：43-45
[24]Schmidt D C, Stal M, Rohnert H面向模式的软件体系结构,卷2：用于并发和网络化对象的模式.北京：机械工业出版社,2003：115-117
[25]Douglas Schmidt, Carlos O'Ryan, Michael Kircher. Leader/Followers. http://www.cs.wustl.edu/-schmidt/PDF/lf.pdf
[26]Angeles Navarro, Rafael Asenjo, Siham Tabik. Analytical Modeling of Pipeline Parallelism. In PACT'09,2009:281-290
[27]Douglas Schmidt,. Half Sync/Half Async. http://www.cs.wustl.edu/~schmidt/PDF/HS-HA.pdf
[28]刘燚.半同步半异步线程池的设计与实现.计算机与现代化,2010,7：38-40
[29]李刚,金蓓弘.两种线程池的实现和性能评价.计算机工程与设计,2007,28(7)：1489-1492
[30]王华,马亮,顾明.线程池技术研究与应用.计算机应用研究,2005,22(11)：141-144
[31]欧阳志鹏,沈富可.基于改进线程池技术服务器程序的设计与实现.计算机与数字工程,2005,33(10)：133-136
[32]刘轶,张昕,李鹤.多核处理器大规模并行系统中的任务分配问题及算法.小型微型计算机系统,2008,29(5)：972-975
[33]陈华平,计永昶,陈国良.分布式动态负载平衡调度的一个通用模型.软件学报,1998,9(1)：25-29
[34]鞠九滨,杨鯤,徐高潮.使用资源利用率作为负载平衡系统的负载指标.软件学报,1996,7(4)：238-242
[35]王力生,毛昀波.多处理机系统的负载平衡模型设计.单片机与嵌入式系统应用,2008,4：10-13
[36]Mercury Interactive. LoadRunner Controller User's Guide Version 7.6[EB/OL]. http://genesis.co.kr/image/product/pds/LoadRunner/Manual/LoadRunner_Controller.pdf
[37]Mercury Interactive. LoadRunner Analysis User's Guide Version 7.6[EB/OL]. http://genesis.co.kr/image/product/pds/LoadRunner/Manual/LoadRunner_Analysis.pdf