基于JXTA的P2P网络搜索机制的研究
摘要
P2P(peer-to-peer)技术自由平等的思想使P2P文件共享系统得到广泛应用。有效地发现资源是充分利用P2P资源的前提,因此P2P搜索技术成为P2P领域研究的热点问题之一。目前P2P系统开发比较混乱且没有统一的标准,JXTA的出现为开发P2P系统提供了统一标准。
当前基于JXTA的P2P搜索技术采用DHT算法,该算法会导致搜索效率低、维护管理成本高等问题。针对当前基于JXTA的P2P搜索技术的缺陷与不足,本文分析了当前P2P和JXTA的各项技术,并对基于JXTA的P2P搜索技术进行了探讨。本文在提高搜索效率、降低管理成本方面做的主要工作如下:
①提出了自适应对等组的概念。具有相似兴趣的对等点组成自适应对等组,每个自适应对等组会维持在一定规模。根据自适应对等组的定义,提出了自适应对等组的规模调整策略,即当网络规模过大或过小时,自适应对等组可以进行拆分与合并。
②在自适应对等组的基础上,改进了Gnutella算法,使改进的搜索算法适用于在自适应对等组内执行。Gnutella算法是广播式的转发查询,只适用于小规模的网络,当网络规模过大时,容易引起广播风暴而占用太多网络带宽。改进的Gnutella算法在每个节点加入了辅助信息,在很多情况下可以替代广播式的转发而采用选择性转发,从而减少了不必要的网络资源浪费,并且可以更快搜索到资源。本文利用PeerSim仿真平台对改进的Gnutella算法进行了仿真测试,通过实验数据分析,表明改进的Gnutella算法在自适应对等组内执行可以提高搜索算法的性能。
③当搜索不满足条件时,应该允许搜索在其它对等组内进行。在这种情况下,为了使组外搜索更为有效,应该将搜索限制在与本地对等组具有相似兴趣的对等组内,所以提出了自适应对等组相似度算法。
④根据自适应对等组相似度算法的计算结果建立组等点视图(Group Peer View,GPV)。在自适应对等组、改进的Gnutella算法、组等点视图的基础上提出了一种优先级对等组搜索机制(Prioritized Adaptive Group Searching,PAGS)。PAGS搜索机制设计了组内外搜索控制的具体策略。
最后根据PAGS搜索机制的思想完成了搜索系统的总体设计,实现了P2P搜索系统的关键部分。该搜索系统可以支持简单的模糊查询,算法实现简单并且在一定程度上降低了管理成本,与目前基于JXTA的P2P搜索相比具有一定的优势。
The P2P (peer-to-peer) file sharing systems are widely applied because of its idea of freedom and equality of P2P technology. Effective way to find P2P resources is the premise of the full use of resources. Therefore, P2P searching technology becomes one of hot topic in P2P research field. Current development of P2P systems is a bit chaotic and there is no unified standard. JXTA provided unified standards for the development of P2P systems.
The DHT algorithm is used by P2P searching technology based on JXTA. This algorithm will cause some problems, such as low search efficiency and high management costs. In terms of the defects and deficiencies of P2P searching technology based on JXTA, the technologies of P2P and JXTA are analyzed in this thesis and further works were done in the P2P searching technology based on JXTA. The primary works about improving searching efficiency and reducing management costs in this thesis are described as follows:
Firstly, the concept of adaptive peer group is proposed. Adaptive peer group i is a collection of peers that have similar interests. Each adaptive peer group is maintained on certain scale. According to the definition of adaptive peer group, a strategy of the adjustment on adaptive peer group size is proposed. When the network size is too large or too small, adaptive peer group can be split or merged.
Secondly, the Gnutella algorithm is improved on the basis of adaptive peer group, and the improved searching algorithm is applied to run in adaptive peer group. Gnutella algorithm is broadcast forward to query and it is only applied to small-scale network. Gnutella algorithm will lead to broadcasting storm and take up too much network bandwidth easily when the network size is too large. The improved Gnutella algorithm adds the auxiliary information to each node. In many cases, it can replace the broadcast forwarding with the selective forwarding. Thereby,it reduces the unnecessary waste of network resources and can quickly find resources. In this thesis, the improved Gnutella algorithm is tested by PeerSim simulation platform. The experimental data indicates that the improved Gnutella algorithm running in adaptive peer group can improve the searching algorithm performance.
Thirdly, searching in other peer groups should be processed when the searching results do not meet for the conditions. In this case, to make search in other peer group more efficient, searching should be limited to peer group that has the similar interests with the local peer group. Therefore, a similarity algorithm of adaptive peer group is proposed.
Fourthly, group peer view is established by the results of the similarity algorithm. Prioritized adaptive group searching mechanism (PAGS) is proposed on the basis of adaptive peer group, the improved algorithm Gnutella and group peer view. PAGS searching mechanism of specific searching control strategies is designed.
Finally, according to the idea of PAGS searching mechanism, the overall design of the searching system is complemented and the key part of the searching system is accomplished. The searching system supports straightforward fuzzy queries, and its algorithm is simple and can reduce management costs to some extent. It has certain advantages compared with the traditional P2P searching based on JXTA.
当前基于JXTA的P2P搜索技术采用DHT算法,该算法会导致搜索效率低、维护管理成本高等问题。针对当前基于JXTA的P2P搜索技术的缺陷与不足,本文分析了当前P2P和JXTA的各项技术,并对基于JXTA的P2P搜索技术进行了探讨。本文在提高搜索效率、降低管理成本方面做的主要工作如下:
①提出了自适应对等组的概念。具有相似兴趣的对等点组成自适应对等组,每个自适应对等组会维持在一定规模。根据自适应对等组的定义,提出了自适应对等组的规模调整策略,即当网络规模过大或过小时,自适应对等组可以进行拆分与合并。
②在自适应对等组的基础上,改进了Gnutella算法,使改进的搜索算法适用于在自适应对等组内执行。Gnutella算法是广播式的转发查询,只适用于小规模的网络,当网络规模过大时,容易引起广播风暴而占用太多网络带宽。改进的Gnutella算法在每个节点加入了辅助信息,在很多情况下可以替代广播式的转发而采用选择性转发,从而减少了不必要的网络资源浪费,并且可以更快搜索到资源。本文利用PeerSim仿真平台对改进的Gnutella算法进行了仿真测试,通过实验数据分析,表明改进的Gnutella算法在自适应对等组内执行可以提高搜索算法的性能。
③当搜索不满足条件时,应该允许搜索在其它对等组内进行。在这种情况下,为了使组外搜索更为有效,应该将搜索限制在与本地对等组具有相似兴趣的对等组内,所以提出了自适应对等组相似度算法。
④根据自适应对等组相似度算法的计算结果建立组等点视图(Group Peer View,GPV)。在自适应对等组、改进的Gnutella算法、组等点视图的基础上提出了一种优先级对等组搜索机制(Prioritized Adaptive Group Searching,PAGS)。PAGS搜索机制设计了组内外搜索控制的具体策略。
最后根据PAGS搜索机制的思想完成了搜索系统的总体设计,实现了P2P搜索系统的关键部分。该搜索系统可以支持简单的模糊查询,算法实现简单并且在一定程度上降低了管理成本,与目前基于JXTA的P2P搜索相比具有一定的优势。
The P2P (peer-to-peer) file sharing systems are widely applied because of its idea of freedom and equality of P2P technology. Effective way to find P2P resources is the premise of the full use of resources. Therefore, P2P searching technology becomes one of hot topic in P2P research field. Current development of P2P systems is a bit chaotic and there is no unified standard. JXTA provided unified standards for the development of P2P systems.
The DHT algorithm is used by P2P searching technology based on JXTA. This algorithm will cause some problems, such as low search efficiency and high management costs. In terms of the defects and deficiencies of P2P searching technology based on JXTA, the technologies of P2P and JXTA are analyzed in this thesis and further works were done in the P2P searching technology based on JXTA. The primary works about improving searching efficiency and reducing management costs in this thesis are described as follows:
Firstly, the concept of adaptive peer group is proposed. Adaptive peer group i is a collection of peers that have similar interests. Each adaptive peer group is maintained on certain scale. According to the definition of adaptive peer group, a strategy of the adjustment on adaptive peer group size is proposed. When the network size is too large or too small, adaptive peer group can be split or merged.
Secondly, the Gnutella algorithm is improved on the basis of adaptive peer group, and the improved searching algorithm is applied to run in adaptive peer group. Gnutella algorithm is broadcast forward to query and it is only applied to small-scale network. Gnutella algorithm will lead to broadcasting storm and take up too much network bandwidth easily when the network size is too large. The improved Gnutella algorithm adds the auxiliary information to each node. In many cases, it can replace the broadcast forwarding with the selective forwarding. Thereby,it reduces the unnecessary waste of network resources and can quickly find resources. In this thesis, the improved Gnutella algorithm is tested by PeerSim simulation platform. The experimental data indicates that the improved Gnutella algorithm running in adaptive peer group can improve the searching algorithm performance.
Thirdly, searching in other peer groups should be processed when the searching results do not meet for the conditions. In this case, to make search in other peer group more efficient, searching should be limited to peer group that has the similar interests with the local peer group. Therefore, a similarity algorithm of adaptive peer group is proposed.
Fourthly, group peer view is established by the results of the similarity algorithm. Prioritized adaptive group searching mechanism (PAGS) is proposed on the basis of adaptive peer group, the improved algorithm Gnutella and group peer view. PAGS searching mechanism of specific searching control strategies is designed.
Finally, according to the idea of PAGS searching mechanism, the overall design of the searching system is complemented and the key part of the searching system is accomplished. The searching system supports straightforward fuzzy queries, and its algorithm is simple and can reduce management costs to some extent. It has certain advantages compared with the traditional P2P searching based on JXTA.
引文
[1]陈贵海,李振华.对等网络:结构、应用与设计[M].北京:清华人学出版社,2007.
[2] Dana Moore, John Hebeler.对等网(Peer-to-Peer ) [M].北京:清华大学出版社,2003.
[3] Robert Flenner, Michael Abbott, Toufic Boubez and etc. Java P2P技术内幕[M].北京:人民邮电出版社,2003.
[4] Stoica I, Morris R, Karger Dl. Chord a Scalable Peer-to-Peer Lookup Service for Internet Applications[C]. Proceedings of ACM SIGCOMM’01. 2001: 149.
[5] S. Ratnasamy, Paul Francis, Mark Handley, Richard Karp, Scott Schenker. A Scalable Content-Addressable Network[J]. Proceeding of ACM SIGCOMM,ACM Press, New York,2001.8:161- 172.
[6] B.Y Zhao, J. D. Kubiatowicz, and A. D. Joseph. Tapestry: An Infrastructure for Fault-Resilient Wide-area Location and Routing[R]. Technical Rept:UCB//CSD-011141, U.C.Berkeley, 2001.4.
[7] Peter Druschel, Anthony Rowstron. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems[C]. IFIP/ACM International Conference on Distributed Systems Platforms, Kluwer Academic Press,2001.11:329-350.
[8]王炯.对等网络中搜索算法与资源最优分布策略的研究与应用[D].合肥工业大学硕士学位论文,2005:15-22.
[9] Liu L. Query Routing in Large-scale Digital Library Systems[C]. Proceedings of ICDE’99, 19
[10]张俊,杨薇薇,张薇,邱功凯.李庆华基于移动代理技术的网格搜索服务的研究与实现[J].计算机应用与软件: 2005,22(5):11-12.
[11] Scott Oaks, Bernard Traversat & Li Gong. JXTATM[M].北京:清华人学出版社,2004.
[12]林姚,何巍.利用JXTA平台实现P2P服务的研究[J].武汉理工大学学报:信息与管理工程版.2005,27(2):38-41.
[13]许斌. JXTA-Java P2P网络编程技术[M].北京:清华人学出版社,2003.
[14]龙伟,刘志勤.基于JXTA的CMS内容共享[J].兵工自动化: 2004,23(6):86-88.
[15]林晨.异构环境下非结构化对等网络路由算法的研究[D].同济大学学位论文,2006:6-21.
[16]黄鑫.基于JXTA协议的Peer-to-Peer:网络查询研究[D].郑州大学硕士学位论文,2004.
[17] Bernard Traversat, Mohamed Abdelaziz, Mike Duigou, Jean-Christophe Hugly, Eric Pouyoul, Bill Yeage.ProjectJXTA Virtual Network[J].Sun Microsystems,2002,32(08):34-35.
[18] Dabek, Fralk,Brunskill, Emma,Kaashoek, M.Frans, Karger, David, Moris, Robert, Stoica, Ion, Balakrisan, Hari. Building Peer-to-Peer systems with chord, adistributed look upservice[C]. Proceedings of the Workshop on Hot To Picsin Operating Systems,2001,20 (5):81-86.
[19]杨大路,刘字宏,张文等. P2P网络技术原理与系统开发实例[M].北京:人民邮电出版社,2007.
[20]李进松. P2P研究及JXTA上的Java实现[D].大连海事大学硕士学位论文,2006:48-63.
[21] Andrew S, Tanenbaum. Distributed systems-principles and paradigms[J]. Prentice Ha11,2002,29(OS):26-27.
[22] White, Liam Quin, Linda Burman. XML从入门到精通[M].北京:电子工业出版社,2002
[23] Daniel Brookshier, Darren Govoni, Navaneeth Krishnan. JXTA: P2P Programming [J]. Sun Microsystems white paper 2002,34(10):49-50.
[24] Prasanna Ganesan, Krishna Gummadi, Hector Garcia-Molina, Canon in GMajor: Designing DHT with Hierarchical Structure[C]. Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS04) 1063-6927/04 2004 IEEE.
[25] Jeffrey Considine. Cluster-based Optimizations for Distributed Hash Tables[R]. Techrep, Computer Science Department, Boston University, November 1, 2002.
[26]李津生,洪佩琳.下一代Internet网络技术[M].北京:人民邮电出版社,2001.
[27] Peter Druschel, Anthony Rowstron. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems[C]. IFIP/ACM International Conference on Distributed Systems Platforms, Kluwer Academic Press,2001.11:329-350.
[28] Kunwadee S, Bruce M,Hui Z. Efficient content location using interest-based locality in peer-to-peer systems. INFOCOM Conference, San Francisco,CA,2003:2167-2175.
[29]李俊清,孙涛. IP多播技术在JXTA系统中的应用田[J].微计算机应用,2006,315(03):25-27.
[30] Byers, J. Considine, and Mitzenmacher, M. Simple Load Balancing for Distributed Hash Tables[R]. BU Computer Science Technical Report 2002.
[31] S. Ratnasamy, Paul Francis, Mark Handley, Richard Karp, Scott Schenker. A Scalable Content-Addressable Network[J]. Proceeding of ACM SIGCOMM,ACM Press, New York, 2001.8:161- 172.
[32] SteveW aterhouse, DavidM Doo lin. Distributed search in P2P networks[J]. IEEE Internet Computing, 2002, (122):68-72.
[33]张智,李瑞轩.基于JXTA的Web服务发现模型研究[J].计算机工程与应用,2005,381(09):137-139.
[34]崔韦.基于JXTA的分布式资源目录搜索模型的研究与实现[D].天津师范大学硕士学位论文,2006:14-35.
[35]黄明秀.基于JXTA的混合P2P网络中资源搜索的研究[D].大连海事大学硕士学位论文,2006:23-25.
[36]徐传福,陈海涛,黄遵国,胡华平,王正华.基于DHT的层次式P2P资源定位模型[J].计算机工程与应用.2004,40(18):156-158.
[37] Christos Gkantsidis, Milena Mihail, Amin Saberi. Hybrid Search Schemes for Unstructured Peer-to-Peer Networks[C]. IEEE Infocom 2005.
[38] Vana Kalogeraki,Dimitrios Gunopulos,D.ZeinalipourYazti.A Local Search Mechanism for PeertoPeer Networks[C]. In CIKM’02,November 4-9,2002.
[39]刘勇,陈志刚. P2P的新发展--JXTA及其部分功能的实现[J].计算机应用研究,2003 (4):149-151.
[40] Daniel Stutzbach, Reza Rejaie. Improving Lookup Performance over a Widely-Deployed DHT[J]. July 2006,31(3):179-190.
[2] Dana Moore, John Hebeler.对等网(Peer-to-Peer ) [M].北京:清华大学出版社,2003.
[3] Robert Flenner, Michael Abbott, Toufic Boubez and etc. Java P2P技术内幕[M].北京:人民邮电出版社,2003.
[4] Stoica I, Morris R, Karger Dl. Chord a Scalable Peer-to-Peer Lookup Service for Internet Applications[C]. Proceedings of ACM SIGCOMM’01. 2001: 149.
[5] S. Ratnasamy, Paul Francis, Mark Handley, Richard Karp, Scott Schenker. A Scalable Content-Addressable Network[J]. Proceeding of ACM SIGCOMM,ACM Press, New York,2001.8:161- 172.
[6] B.Y Zhao, J. D. Kubiatowicz, and A. D. Joseph. Tapestry: An Infrastructure for Fault-Resilient Wide-area Location and Routing[R]. Technical Rept:UCB//CSD-011141, U.C.Berkeley, 2001.4.
[7] Peter Druschel, Anthony Rowstron. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems[C]. IFIP/ACM International Conference on Distributed Systems Platforms, Kluwer Academic Press,2001.11:329-350.
[8]王炯.对等网络中搜索算法与资源最优分布策略的研究与应用[D].合肥工业大学硕士学位论文,2005:15-22.
[9] Liu L. Query Routing in Large-scale Digital Library Systems[C]. Proceedings of ICDE’99, 19
[10]张俊,杨薇薇,张薇,邱功凯.李庆华基于移动代理技术的网格搜索服务的研究与实现[J].计算机应用与软件: 2005,22(5):11-12.
[11] Scott Oaks, Bernard Traversat & Li Gong. JXTATM[M].北京:清华人学出版社,2004.
[12]林姚,何巍.利用JXTA平台实现P2P服务的研究[J].武汉理工大学学报:信息与管理工程版.2005,27(2):38-41.
[13]许斌. JXTA-Java P2P网络编程技术[M].北京:清华人学出版社,2003.
[14]龙伟,刘志勤.基于JXTA的CMS内容共享[J].兵工自动化: 2004,23(6):86-88.
[15]林晨.异构环境下非结构化对等网络路由算法的研究[D].同济大学学位论文,2006:6-21.
[16]黄鑫.基于JXTA协议的Peer-to-Peer:网络查询研究[D].郑州大学硕士学位论文,2004.
[17] Bernard Traversat, Mohamed Abdelaziz, Mike Duigou, Jean-Christophe Hugly, Eric Pouyoul, Bill Yeage.ProjectJXTA Virtual Network[J].Sun Microsystems,2002,32(08):34-35.
[18] Dabek, Fralk,Brunskill, Emma,Kaashoek, M.Frans, Karger, David, Moris, Robert, Stoica, Ion, Balakrisan, Hari. Building Peer-to-Peer systems with chord, adistributed look upservice[C]. Proceedings of the Workshop on Hot To Picsin Operating Systems,2001,20 (5):81-86.
[19]杨大路,刘字宏,张文等. P2P网络技术原理与系统开发实例[M].北京:人民邮电出版社,2007.
[20]李进松. P2P研究及JXTA上的Java实现[D].大连海事大学硕士学位论文,2006:48-63.
[21] Andrew S, Tanenbaum. Distributed systems-principles and paradigms[J]. Prentice Ha11,2002,29(OS):26-27.
[22] White, Liam Quin, Linda Burman. XML从入门到精通[M].北京:电子工业出版社,2002
[23] Daniel Brookshier, Darren Govoni, Navaneeth Krishnan. JXTA: P2P Programming [J]. Sun Microsystems white paper 2002,34(10):49-50.
[24] Prasanna Ganesan, Krishna Gummadi, Hector Garcia-Molina, Canon in GMajor: Designing DHT with Hierarchical Structure[C]. Proceedings of the 24th International Conference on Distributed Computing Systems (ICDCS04) 1063-6927/04 2004 IEEE.
[25] Jeffrey Considine. Cluster-based Optimizations for Distributed Hash Tables[R]. Techrep, Computer Science Department, Boston University, November 1, 2002.
[26]李津生,洪佩琳.下一代Internet网络技术[M].北京:人民邮电出版社,2001.
[27] Peter Druschel, Anthony Rowstron. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems[C]. IFIP/ACM International Conference on Distributed Systems Platforms, Kluwer Academic Press,2001.11:329-350.
[28] Kunwadee S, Bruce M,Hui Z. Efficient content location using interest-based locality in peer-to-peer systems. INFOCOM Conference, San Francisco,CA,2003:2167-2175.
[29]李俊清,孙涛. IP多播技术在JXTA系统中的应用田[J].微计算机应用,2006,315(03):25-27.
[30] Byers, J. Considine, and Mitzenmacher, M. Simple Load Balancing for Distributed Hash Tables[R]. BU Computer Science Technical Report 2002.
[31] S. Ratnasamy, Paul Francis, Mark Handley, Richard Karp, Scott Schenker. A Scalable Content-Addressable Network[J]. Proceeding of ACM SIGCOMM,ACM Press, New York, 2001.8:161- 172.
[32] SteveW aterhouse, DavidM Doo lin. Distributed search in P2P networks[J]. IEEE Internet Computing, 2002, (122):68-72.
[33]张智,李瑞轩.基于JXTA的Web服务发现模型研究[J].计算机工程与应用,2005,381(09):137-139.
[34]崔韦.基于JXTA的分布式资源目录搜索模型的研究与实现[D].天津师范大学硕士学位论文,2006:14-35.
[35]黄明秀.基于JXTA的混合P2P网络中资源搜索的研究[D].大连海事大学硕士学位论文,2006:23-25.
[36]徐传福,陈海涛,黄遵国,胡华平,王正华.基于DHT的层次式P2P资源定位模型[J].计算机工程与应用.2004,40(18):156-158.
[37] Christos Gkantsidis, Milena Mihail, Amin Saberi. Hybrid Search Schemes for Unstructured Peer-to-Peer Networks[C]. IEEE Infocom 2005.
[38] Vana Kalogeraki,Dimitrios Gunopulos,D.ZeinalipourYazti.A Local Search Mechanism for PeertoPeer Networks[C]. In CIKM’02,November 4-9,2002.
[39]刘勇,陈志刚. P2P的新发展--JXTA及其部分功能的实现[J].计算机应用研究,2003 (4):149-151.
[40] Daniel Stutzbach, Reza Rejaie. Improving Lookup Performance over a Widely-Deployed DHT[J]. July 2006,31(3):179-190.