系统仿真数据分布式计算环境的研究与实现
|
摘要
目前,系统仿真数据的计算都是在一台机器上进行计算的。系统仿真数据是用来评价支撑业务系统的网络系统的性能的,这就需要实时的计算系统仿真数据,以便向用户展现业务系统运行时的各个网络设备的负载情况,方便用户判断所选设备是否合理。然而由于仿真数据量大,在单机上很难满足向用户实时交互的需求。这就有必要通过分布式计算技术把地理上广泛分布的各种资源连成一个整体,共同完成计算任务。
利用网络主机的资源开展分布式计算已经成为一种重要的高性能计算方式,它提供了更好的性能价格比,更容易解决一些在单台计算机上难以解决的复杂问题。本文着重分析了系统仿真数据,针对系统仿真数据要求实时性、计算量大、单机处理速度慢等特点,采用了基于Master—Worker模型的三层体系结构的分布式计算处理方案;在实现技术上,采用了WebServices技术,在计算节点端采用Web Services技术将系统仿真数据的计算应用发布为Web服务。
首先论文给出了系统架构和各个模块的功能。由于任务计算需要很长的时间,计算节点发生错误的可能性增加,针对单个计算节点失效引起的计算上的巨大损失问题,采用了检查点功能。接着对系统中的任务调度算法进行了研究,对蚁群算法进行改进,即性能高的机器分配的任务比较多,性能低的机器分配的任务比较少,通过合理分配减少任务的运算时间。
At present, system simulation data is calculated on one computer. System simulation data is used to evaluate the performance of the network system which supports transaction system. This requires real-time computing system simulation data, in order to demonstrate network equipment's load to users when transaction system is running. Simulation data is so much as it is difficult of meeting real-time interaction to users when it is on one computer. It is necessary to connect different resources which are scattered on the internet into integrated one with the distributed computing technology. The new one will accomplish computing tasks together.
Distributed computation using free resource on the network has been an important way of high-performance computation. It is more convenient than one computer to solve complex problems with providing better performance-price ratio. It analyzes system simulation data in thesis In view of the characteristics of system simulation data, such as real-time, the amount of computation, low processing speed of one computer, it adopts a three-layer structure based on the Master-Worker. In the realization of technology, it adopts Web Services in thesis. The computation application of system simulation data is deployed as a computation Web Services on the Worker.
Firstly, it introduces the frameworks and the function of each module in thesis. Because it is a task which needs a long time to run, checkpoint function is adopted in order to avoid a great loss caused by one compute node's failure. Secondly, job assignment algorithms in the system are researched and an improved ant colony optimization is introduced which means that more tasks are assigned to high-performance computer and less tasks are assigned to low-performance computer and it will reduce the computing time by this way.
利用网络主机的资源开展分布式计算已经成为一种重要的高性能计算方式,它提供了更好的性能价格比,更容易解决一些在单台计算机上难以解决的复杂问题。本文着重分析了系统仿真数据,针对系统仿真数据要求实时性、计算量大、单机处理速度慢等特点,采用了基于Master—Worker模型的三层体系结构的分布式计算处理方案;在实现技术上,采用了WebServices技术,在计算节点端采用Web Services技术将系统仿真数据的计算应用发布为Web服务。
首先论文给出了系统架构和各个模块的功能。由于任务计算需要很长的时间,计算节点发生错误的可能性增加,针对单个计算节点失效引起的计算上的巨大损失问题,采用了检查点功能。接着对系统中的任务调度算法进行了研究,对蚁群算法进行改进,即性能高的机器分配的任务比较多,性能低的机器分配的任务比较少,通过合理分配减少任务的运算时间。
At present, system simulation data is calculated on one computer. System simulation data is used to evaluate the performance of the network system which supports transaction system. This requires real-time computing system simulation data, in order to demonstrate network equipment's load to users when transaction system is running. Simulation data is so much as it is difficult of meeting real-time interaction to users when it is on one computer. It is necessary to connect different resources which are scattered on the internet into integrated one with the distributed computing technology. The new one will accomplish computing tasks together.
Distributed computation using free resource on the network has been an important way of high-performance computation. It is more convenient than one computer to solve complex problems with providing better performance-price ratio. It analyzes system simulation data in thesis In view of the characteristics of system simulation data, such as real-time, the amount of computation, low processing speed of one computer, it adopts a three-layer structure based on the Master-Worker. In the realization of technology, it adopts Web Services in thesis. The computation application of system simulation data is deployed as a computation Web Services on the Worker.
Firstly, it introduces the frameworks and the function of each module in thesis. Because it is a task which needs a long time to run, checkpoint function is adopted in order to avoid a great loss caused by one compute node's failure. Secondly, job assignment algorithms in the system are researched and an improved ant colony optimization is introduced which means that more tasks are assigned to high-performance computer and less tasks are assigned to low-performance computer and it will reduce the computing time by this way.
引文
[1]邵堃,刘宗田,孙智勇.分布式计算环境的比较研究.计算机工程与应用.2001,13:26-29页
[2]L.Smam,C Catlett.Metacomputing[J].Communication of the ACM.1992,35(6):44-52P
[3]Dorian C.Amold,Henri Casanova,jack Dongarra.Innovations of the NetSolve Grid Computing System.Concurrency and Computation:Practice and Experience.2002,14(13-15):1457-1479P
[4]Enslow P H.What is a distributed data processing system?[J]IEEE Computers.1978,22(1):13-21P
[5]赵小建,方康玲.基于流套接字的RPC技术研究与应用.武汉科技大学学报.2007,30(1):68-70页
[6]王安莉,蒋外文.浅析分布式计算技术的发展.中国中医药现代远程教育.2007,5(04):49-51页
[7]Vadim Draluk.Discovering Web Services:An overview.In:Proc of the 27~(th) International Conference on Very Large Data Bases VLDB '01Published:Morgan Kaufrnann Publishers Inc.2001,637-640P
[8]Werner Vogels.Web Services are not distributed objects.Internet Computing,IEEE,2003,7(6):P59-66P
[9]Muthucumaru Maheswaran,Shoukat Ali,Howard Jay Siegel,et al.Dynamic Matching and scheduling of a Class of Independent Tasks onto Heterogeneous Computing Systems.In Proceedings of the 8~(th) IEEE Heterogeneous Computing Workshop(HCW'99).IEEE Computer Society Press.1999.30P
[10]Colorni A,Dorigo M,Maniezzo V.Distributed optimization by ant colonies.In:Varela F,Bourgine P,eds.Proc of the ECAL'91 European Conf of Artificial Life.Paris:Elsevier,1991,134-144P
[11]Dortgo M,Maniezzo V,Colomi A.Ant system:Optimization by a colony cooperating Agents.IEEE Trans.on Systems,Man,and Cybernetics-Part B:Cybernetics.1996,26(1):29-41P
[12]Dortgo M,Gambardella LM.Ant colony system:A cooperative learning approach to the traveling salesman problem[J].IEEE Transactions on Evolutionary Computation.1997,1(1):53-66P
[13]Merkle D,Mdiddendorf M,Schmeck H.Ant colony optimization for resource-constrained project scheduling.IEEE Transactions On Evolutionary Computation.2002,6(4):333-339P
[14]Parpinelli RS,Lopes HS,Freitas AA.Data mining with an ant colony optimization algorithm.IEEE Transactions on Evolutionary Computation.2002,6(4):321-328P
[15]Dorigo M.Special section on ant colony optimization.IEEE Trans.on Evolutionary Computation,2002,6(4):317-319P
[16]Hoshyar R,Jamali SH,Locus C.Ant colony algorithm for finding good interleaving pattern in turbo codes.IEE Proceedings?Communications,2000,147(5):257-262P
[17]Tsai CF,Tsai CW.A new approach for solving large traveling salesman problem using evolution ant rules.In:Neural Networks,IJCNN 2002,Proc.of the 2002 Int'l Joint Conf.on,Vol 2.Honolulu:IEEE Press.2002,1540-1545P
[18]吴斌,史忠植.一种基于蚁群算法的TSP问题分段求解算法.计算机学报.2001,24(12):1328-1333页
[19]王莉,窦曼,刘宗田,黄美丽.一种快速网格任务调度策略.计算机科学.2007,34(06):128-130页
[20]朱庆保,杨志军.基于变异和动态信息素更新的蚁群优化算法.软件学报.2004,15(02):185-191页
[21]钟一文,杨建刚.求解多任务调度问题的免疫蚁群算法.模式识别与人工智能.2006,19(01):73-78页
[22]Leeman M,De Florio V,Deconinck G,"A flexible library for dependable Master-Worker parallel programs," IEEE Comp.Soc.Press,Los Alamitos,CA.genoa,Italy,February 5-7,2003,PP.299-307P
[23]E.Heymann,M.A.Senar,E.Luque,and M.Livny."Adaptive scheduling for Master-Worker applications on the computational grid".In:Proc.Of the First IEEE/ACM Intemational Workshop on Grid Computing (Grid2000).2000,214- 227P
[24]吕苏,洪国辉.基于Java的轻量级元计算.计算机工程.2003,29(6):89-91页
[25]J.P.Goux et al.An Enabling Framework for Master-Worker Applicaions on The Computational Grid.In the 9th IEEE International Symposium on High Performance Distributed Computing.2000,43P
[26]林伟伟,齐德昱.树型网格环境TGrid的模型及算法.华南理工大学学报.2007,35(1):89-93页
[27]M.O.Neary,S.P.Brydon,P.Kmiec,et al.Javalin++:Scalability Issues in Global Computing.Concurrency:Practice and Experience,2001,12:727-753P
[28]L.Sarmenta,S.Chua,P.Echevarria,et al.Bayanihan computing.NET:Grid computing with XML Web Services.In:Proceedings of the 2~(nd)IEEE/ACM Int'l Conf.on Cluster Computing and the Grid(CCGRID 2002).Berlin:IEEE Computer Society Press,2002.404-405P
[29]Wayne Kelly,Paul Roe,Jiro Sumitomo.G2:A Grid Middleware for Cycle Donation Using.NET.In:Proceedings of the Interfiational Conference on Parallel and Distributed Processing Techniques and Applications.Las Vegas,Nevada,USA,2002,699-705P
[30]A.Duda.The effect of checkpointing on program excution time.Information Processing letters,1983,16(4):221-229P
[31]C.Germain,G.Fedak,V.Neri,er al.Global computing systems.Lecture Notes in Computer Science.2001,2179:218-227P
[32]李凯原,杨孝宗.减少检查点开销的一种方法.计算机工程与应用.2002(2):4-6页
[33]梅皓,沈志宇,廖湘科.基于Java的分布式并行计算关键技术.计算机工程与科学.2000,22(2):103-106页
[34]赵扬帆.基于遗传算法和蚁群算法的网格任务调度策略.中国海洋大学硕士学位论文.2006:35-36页
[35]Stutzle T,Hoos HH.MAX-MIN ant system and local search for the traveling salesman problem.In:IEEE Int'l Conf.on Evolutionary Computation.Indianapolis:IEEE Press,1997.309-314P
[2]L.Smam,C Catlett.Metacomputing[J].Communication of the ACM.1992,35(6):44-52P
[3]Dorian C.Amold,Henri Casanova,jack Dongarra.Innovations of the NetSolve Grid Computing System.Concurrency and Computation:Practice and Experience.2002,14(13-15):1457-1479P
[4]Enslow P H.What is a distributed data processing system?[J]IEEE Computers.1978,22(1):13-21P
[5]赵小建,方康玲.基于流套接字的RPC技术研究与应用.武汉科技大学学报.2007,30(1):68-70页
[6]王安莉,蒋外文.浅析分布式计算技术的发展.中国中医药现代远程教育.2007,5(04):49-51页
[7]Vadim Draluk.Discovering Web Services:An overview.In:Proc of the 27~(th) International Conference on Very Large Data Bases VLDB '01Published:Morgan Kaufrnann Publishers Inc.2001,637-640P
[8]Werner Vogels.Web Services are not distributed objects.Internet Computing,IEEE,2003,7(6):P59-66P
[9]Muthucumaru Maheswaran,Shoukat Ali,Howard Jay Siegel,et al.Dynamic Matching and scheduling of a Class of Independent Tasks onto Heterogeneous Computing Systems.In Proceedings of the 8~(th) IEEE Heterogeneous Computing Workshop(HCW'99).IEEE Computer Society Press.1999.30P
[10]Colorni A,Dorigo M,Maniezzo V.Distributed optimization by ant colonies.In:Varela F,Bourgine P,eds.Proc of the ECAL'91 European Conf of Artificial Life.Paris:Elsevier,1991,134-144P
[11]Dortgo M,Maniezzo V,Colomi A.Ant system:Optimization by a colony cooperating Agents.IEEE Trans.on Systems,Man,and Cybernetics-Part B:Cybernetics.1996,26(1):29-41P
[12]Dortgo M,Gambardella LM.Ant colony system:A cooperative learning approach to the traveling salesman problem[J].IEEE Transactions on Evolutionary Computation.1997,1(1):53-66P
[13]Merkle D,Mdiddendorf M,Schmeck H.Ant colony optimization for resource-constrained project scheduling.IEEE Transactions On Evolutionary Computation.2002,6(4):333-339P
[14]Parpinelli RS,Lopes HS,Freitas AA.Data mining with an ant colony optimization algorithm.IEEE Transactions on Evolutionary Computation.2002,6(4):321-328P
[15]Dorigo M.Special section on ant colony optimization.IEEE Trans.on Evolutionary Computation,2002,6(4):317-319P
[16]Hoshyar R,Jamali SH,Locus C.Ant colony algorithm for finding good interleaving pattern in turbo codes.IEE Proceedings?Communications,2000,147(5):257-262P
[17]Tsai CF,Tsai CW.A new approach for solving large traveling salesman problem using evolution ant rules.In:Neural Networks,IJCNN 2002,Proc.of the 2002 Int'l Joint Conf.on,Vol 2.Honolulu:IEEE Press.2002,1540-1545P
[18]吴斌,史忠植.一种基于蚁群算法的TSP问题分段求解算法.计算机学报.2001,24(12):1328-1333页
[19]王莉,窦曼,刘宗田,黄美丽.一种快速网格任务调度策略.计算机科学.2007,34(06):128-130页
[20]朱庆保,杨志军.基于变异和动态信息素更新的蚁群优化算法.软件学报.2004,15(02):185-191页
[21]钟一文,杨建刚.求解多任务调度问题的免疫蚁群算法.模式识别与人工智能.2006,19(01):73-78页
[22]Leeman M,De Florio V,Deconinck G,"A flexible library for dependable Master-Worker parallel programs," IEEE Comp.Soc.Press,Los Alamitos,CA.genoa,Italy,February 5-7,2003,PP.299-307P
[23]E.Heymann,M.A.Senar,E.Luque,and M.Livny."Adaptive scheduling for Master-Worker applications on the computational grid".In:Proc.Of the First IEEE/ACM Intemational Workshop on Grid Computing (Grid2000).2000,214- 227P
[24]吕苏,洪国辉.基于Java的轻量级元计算.计算机工程.2003,29(6):89-91页
[25]J.P.Goux et al.An Enabling Framework for Master-Worker Applicaions on The Computational Grid.In the 9th IEEE International Symposium on High Performance Distributed Computing.2000,43P
[26]林伟伟,齐德昱.树型网格环境TGrid的模型及算法.华南理工大学学报.2007,35(1):89-93页
[27]M.O.Neary,S.P.Brydon,P.Kmiec,et al.Javalin++:Scalability Issues in Global Computing.Concurrency:Practice and Experience,2001,12:727-753P
[28]L.Sarmenta,S.Chua,P.Echevarria,et al.Bayanihan computing.NET:Grid computing with XML Web Services.In:Proceedings of the 2~(nd)IEEE/ACM Int'l Conf.on Cluster Computing and the Grid(CCGRID 2002).Berlin:IEEE Computer Society Press,2002.404-405P
[29]Wayne Kelly,Paul Roe,Jiro Sumitomo.G2:A Grid Middleware for Cycle Donation Using.NET.In:Proceedings of the Interfiational Conference on Parallel and Distributed Processing Techniques and Applications.Las Vegas,Nevada,USA,2002,699-705P
[30]A.Duda.The effect of checkpointing on program excution time.Information Processing letters,1983,16(4):221-229P
[31]C.Germain,G.Fedak,V.Neri,er al.Global computing systems.Lecture Notes in Computer Science.2001,2179:218-227P
[32]李凯原,杨孝宗.减少检查点开销的一种方法.计算机工程与应用.2002(2):4-6页
[33]梅皓,沈志宇,廖湘科.基于Java的分布式并行计算关键技术.计算机工程与科学.2000,22(2):103-106页
[34]赵扬帆.基于遗传算法和蚁群算法的网格任务调度策略.中国海洋大学硕士学位论文.2006:35-36页
[35]Stutzle T,Hoos HH.MAX-MIN ant system and local search for the traveling salesman problem.In:IEEE Int'l Conf.on Evolutionary Computation.Indianapolis:IEEE Press,1997.309-314P