Linux集群中套接字迁移的研究与实现
|
摘要
随着高性能工作站的出现和高性能网络技术的飞速发展，集群计算逐渐成为一种具有很高性价比的的并行／分布式计算系统。集群计算系统的核心问题是资源的共享及有效利用，只有平衡的负载才能达到最大的资源使用率。进程是计算机系统中使用资源的基本单元，使进程能够动态的在集群节点间迁移是实现负载平衡的一种基本而有效的方法，这样就能使系统能适应负载的动态变化。在现在以地址为中心的通信模型中，进程迁移必然带来通信地址的变化，进程间的网络通信不仅直接影响迁移的透明性，带来了迁移限制，而且导致极大的剩余相关性，对迁移后的性能和系统的健壮性有很大的负面影响。因此，必须使集群系统能够提供对网络通信迁移的支持，保证网络通信不因为进程的迁移而中断。我们将其称为套接字迁移。
本文首先研究了MOSIX进程迁移机制，然后通过源码阅读分析了Linux系统TCP和UDP套接字的实现，并在此基础上，讨论了这两种套接字迁移中的关键问题；然后给出了内核级的迁移套接字的完整方案，实现了一个套接字迁移在MOSIX集群系统上的系统原型。解决了集群环境下进程迁移如何透明的保持TCP和UDP通信的问题。
With the development of high performance workstation and network, cluster become an attractive platform for high end parellel and distributed computing. The core problem of cluster computing is resource sharing and its fully utilization,because the best performance can not be achieved when loads among nodes are unbalanced. A process is a basic unit of using resources in computer systems, so it is a basic and effective way to let processes migrate among nodes dynamically to achieve load balance, by which the cluster system can acclimatize itself to the changes of load. Transparence is a basic requirement for process migration. But in today's address-centered communcation model.the IP address of a process changes when it migrate from one node to another. So the network communcation not only affect the transparence of process migration but lead to residual dependency.which put bad effect on the performance and robustness of the system. Therefore we must present a mechanism to support the migration of network co
mmunication so that the communication between processes is not affected by process migraion. We call it socket migration.
In this paper, the process migration mechanism in MOSIX cluster is studied firstly .And then, the implementation of TCP and UDP socket in Linux are also studied by reading the Linux kernel source. On the basis of that , the key problems of socket migration are discussed. After that, a complete scheme of socket migration system in kernel level is presented and a prototype is implemented in MOSIX cluster which solved the problem of transparent TCP and UDP socket migration in Linux cluster.
本文首先研究了MOSIX进程迁移机制，然后通过源码阅读分析了Linux系统TCP和UDP套接字的实现，并在此基础上，讨论了这两种套接字迁移中的关键问题；然后给出了内核级的迁移套接字的完整方案，实现了一个套接字迁移在MOSIX集群系统上的系统原型。解决了集群环境下进程迁移如何透明的保持TCP和UDP通信的问题。
With the development of high performance workstation and network, cluster become an attractive platform for high end parellel and distributed computing. The core problem of cluster computing is resource sharing and its fully utilization,because the best performance can not be achieved when loads among nodes are unbalanced. A process is a basic unit of using resources in computer systems, so it is a basic and effective way to let processes migrate among nodes dynamically to achieve load balance, by which the cluster system can acclimatize itself to the changes of load. Transparence is a basic requirement for process migration. But in today's address-centered communcation model.the IP address of a process changes when it migrate from one node to another. So the network communcation not only affect the transparence of process migration but lead to residual dependency.which put bad effect on the performance and robustness of the system. Therefore we must present a mechanism to support the migration of network co
mmunication so that the communication between processes is not affected by process migraion. We call it socket migration.
In this paper, the process migration mechanism in MOSIX cluster is studied firstly .And then, the implementation of TCP and UDP socket in Linux are also studied by reading the Linux kernel source. On the basis of that , the key problems of socket migration are discussed. After that, a complete scheme of socket migration system in kernel level is presented and a prototype is implemented in MOSIX cluster which solved the problem of transparent TCP and UDP socket migration in Linux cluster.
引文
[1]Kuntz Bryan, Rajan Karthik MIGSOCK:Migratable TCP Socket in Linux[thesis],Carnegie Mellon http://www-2.cs.cmu.edu/～softagents/migsock.html 2002
[2]Bubak Marian, Zbik Dariusz, Albada Dick van:Portable library of Migratable Sockets, Proceedings of the SGI Users, Conference 2000.
[3]Barak Amnon, La'adan Oren, Shiloh Amnon: Scalable Cluster Computing with MOSIX for LINUX, Proc. Linux Expo' 99, pp.95--100, Raleigh, N.C., May 1999.
[4]Xun Qu, Jeffrey Xu Yu, Richard P. Brent: A Mobile TCP Socket http://cs.anu.edu.au/people/Jeffrey.X.Yu/research-comm.html 1997
[5]Florin Sultan, Kiran Srinivasan, Deepa Iyer: Migratory TCP:Highly Available Internet Services Using Connection Migration, http://www.cs.rutgers.edu/pub/technical-reports/2002
[6]David a Maltz ,Pravin Bhagwat MSOCKS: An Architecture for Transport Layer Mobility Phd thesis http://www-cgi.cs.cmu.edu/afs/cs.cmu.edu/user/dmaltz/www/home.html 1998
[7]F. Douglis:"Transparent Process Migration in the Sprite OperatingSystem" .PhD thesis, University of California, Berkeley, CA 94720, September 1990.
[8]Timothy S. Mitrovich, Kenneth M. Ford, and Niranjan Suri: Transparent Redirection of Network Sockets http://nomads.coginst.uwf.edu/
[9]Praveen Ramanan: Socket Swapping for Efficient Distributed Communication Between Communicating Process http://www.cis.ksu.edu/～pra4444/
[10]Barak Amnon, La'adan Oren The MOSIX Multicomputer Operating System for High Performance Cluster Computing, Journal of Future Generation Computer Svstems, April 1998
[11]Ammon Barak , Avner Braverman Ilia Gildername Performance of PVM with the MOSIX preemptive process migration scheme http://www.mosix.org/1996
[12]Lior Amar ,Amnon Barak and Amnon Shiloh The MOSIX Direct File System Access Method for Supporting Scalable Cluster File Systems http://www.mosix.org/2003
[13]Ravikanth Nasika Partha Dasgupta: Transparent Migration of Distributed Communicating Processes, http://www.isa.informatik.tu-darmstadt.de/VS/Publikationen/, 1998
[14]Ma Jin Ming JESSICA: Java-Enabled Single-System-Image Computing Architecture, http://www.rdt.monash.edu.au/～rajkumar/pdpta99/index.html, 1999
[15]Amnon Barak and Avner Braverman,Memory ushering in a scalable computing cluster http://www.mosix.org 1997
[16]Fred Douglis John Ousterhout: Transparent Process Migration Design Alternative and Sprite Implementation, http://www.research.att.com/～douglis/papers/1991
[17]高兰 集群系统UDP协议单拷贝技术的研究 华中科技大学硕士学位论文 2002．5
[18]毛德操、胡希明 Linux内核源代码情景分析[上、下]浙江大学出版社 2001
[19]蒋江 异构集群系统中基于进程迁移机制的负载平衡算法的研究 国防科技大学博士学位论文 2002
[20]黄翊：集群系统MOSIX分析 http://www-900.ibm.com/developerWorks/cn/linux/cluster/mosix/part1/index.shtml
[21]黄翊、蒋江、张民选 MOSIX进程迁移机制研究 计算机工程 2002年8月 第8期 卷28。
[22]周竞扬、陈韬略等 一种高效可靠的移动Agent间通信机制 软件学报 2003年第8期 卷14
[23]Alessandro Rubini & Jonathan Corbet等著魏永明等译 Linux设备驱动程序 第二版 中国电力出版社 2002
[24]赵军锁、周恩强 消息传递、PVM及MPI 电脑与信息技术 1998年2期 P11-P15
[25]Rajkumar Buyya著 郑纬民 汪东升 石威等译 高性能集群计算 卷一 电子工业出版社 2001-06-01 p2-p47
[26]Dougl Fred and Ousterhout John:"Process Migration in Sprite:A Status Report" IEEE TCOS Newsletter, Vol.3, No.1, Winter 1989, PP.8--10.
[27]Michael Litzkow, Miron Livny, and Matt Mutka: Condor-Ahunter of Idle Workstations, Proceeding of the 8th Internation Conference of Distributed Computing Systems ,pages 104-111, June, 1988
[2]Bubak Marian, Zbik Dariusz, Albada Dick van:Portable library of Migratable Sockets, Proceedings of the SGI Users, Conference 2000.
[3]Barak Amnon, La'adan Oren, Shiloh Amnon: Scalable Cluster Computing with MOSIX for LINUX, Proc. Linux Expo' 99, pp.95--100, Raleigh, N.C., May 1999.
[4]Xun Qu, Jeffrey Xu Yu, Richard P. Brent: A Mobile TCP Socket http://cs.anu.edu.au/people/Jeffrey.X.Yu/research-comm.html 1997
[5]Florin Sultan, Kiran Srinivasan, Deepa Iyer: Migratory TCP:Highly Available Internet Services Using Connection Migration, http://www.cs.rutgers.edu/pub/technical-reports/2002
[6]David a Maltz ,Pravin Bhagwat MSOCKS: An Architecture for Transport Layer Mobility Phd thesis http://www-cgi.cs.cmu.edu/afs/cs.cmu.edu/user/dmaltz/www/home.html 1998
[7]F. Douglis:"Transparent Process Migration in the Sprite OperatingSystem" .PhD thesis, University of California, Berkeley, CA 94720, September 1990.
[8]Timothy S. Mitrovich, Kenneth M. Ford, and Niranjan Suri: Transparent Redirection of Network Sockets http://nomads.coginst.uwf.edu/
[9]Praveen Ramanan: Socket Swapping for Efficient Distributed Communication Between Communicating Process http://www.cis.ksu.edu/～pra4444/
[10]Barak Amnon, La'adan Oren The MOSIX Multicomputer Operating System for High Performance Cluster Computing, Journal of Future Generation Computer Svstems, April 1998
[11]Ammon Barak , Avner Braverman Ilia Gildername Performance of PVM with the MOSIX preemptive process migration scheme http://www.mosix.org/1996
[12]Lior Amar ,Amnon Barak and Amnon Shiloh The MOSIX Direct File System Access Method for Supporting Scalable Cluster File Systems http://www.mosix.org/2003
[13]Ravikanth Nasika Partha Dasgupta: Transparent Migration of Distributed Communicating Processes, http://www.isa.informatik.tu-darmstadt.de/VS/Publikationen/, 1998
[14]Ma Jin Ming JESSICA: Java-Enabled Single-System-Image Computing Architecture, http://www.rdt.monash.edu.au/～rajkumar/pdpta99/index.html, 1999
[15]Amnon Barak and Avner Braverman,Memory ushering in a scalable computing cluster http://www.mosix.org 1997
[16]Fred Douglis John Ousterhout: Transparent Process Migration Design Alternative and Sprite Implementation, http://www.research.att.com/～douglis/papers/1991
[17]高兰 集群系统UDP协议单拷贝技术的研究 华中科技大学硕士学位论文 2002．5
[18]毛德操、胡希明 Linux内核源代码情景分析[上、下]浙江大学出版社 2001
[19]蒋江 异构集群系统中基于进程迁移机制的负载平衡算法的研究 国防科技大学博士学位论文 2002
[20]黄翊：集群系统MOSIX分析 http://www-900.ibm.com/developerWorks/cn/linux/cluster/mosix/part1/index.shtml
[21]黄翊、蒋江、张民选 MOSIX进程迁移机制研究 计算机工程 2002年8月 第8期 卷28。
[22]周竞扬、陈韬略等 一种高效可靠的移动Agent间通信机制 软件学报 2003年第8期 卷14
[23]Alessandro Rubini & Jonathan Corbet等著魏永明等译 Linux设备驱动程序 第二版 中国电力出版社 2002
[24]赵军锁、周恩强 消息传递、PVM及MPI 电脑与信息技术 1998年2期 P11-P15
[25]Rajkumar Buyya著 郑纬民 汪东升 石威等译 高性能集群计算 卷一 电子工业出版社 2001-06-01 p2-p47
[26]Dougl Fred and Ousterhout John:"Process Migration in Sprite:A Status Report" IEEE TCOS Newsletter, Vol.3, No.1, Winter 1989, PP.8--10.
[27]Michael Litzkow, Miron Livny, and Matt Mutka: Condor-Ahunter of Idle Workstations, Proceeding of the 8th Internation Conference of Distributed Computing Systems ,pages 104-111, June, 1988