天文图像合成网格服务的负载平衡研究
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
网格计算在动态多机构的虚拟组织中协调资源共享和协同求解问题。负载平衡均衡所有结点上的负载,提高系统的资源利用率,减少任务平均响应时间。可分负载理论是进行并行分布式任务调度实现负载平衡的一个模型,有易于处理、可操作性强、扩展性好、适用于多数网络拓扑结构、易于与实际结合等优点,因此在数据密集型网格计算中进行可分负载理论的研究和应用深受研究人员的青睐。本文应用并行传输,计算通信可重叠的可分负载理论求解天文图像合成网格服务的任务分配和再分配问题,取得了一些成果。
     本文首先设计了天文图像合成网格服务的系统结构,分析总结了该网格服务四个主要工作流程,设计了该网格服务的服务流程。
     然后讨论了天文图像合成网格服务的任务分配问题。通过对基于蚂蚁算法的网格任务调度方法进行分析和改造,提出了基于蚂蚁算法的性能预测方法以进行简单高效的性能预测,为负载平衡提供支持。针对该网格服务四个工作流程建立了根结点提交二次叠加、根结点提交一次叠加、根结点处理二次叠加和根结点处理一次叠加四个可分负载模型。为了更好的处理存储限制问题,提出了一种比增量平衡策略算法更快适用范围更广的快速存储限制分配算法。基于已有求取次优解的整数逼近算法,提出了增一堆排序整数逼近算法以求取可分粒度问题的最优解。基于以上的模型和算法,设计了该网格服务的任务分配总体流程。通过仿真实验分析选取了基于蚂蚁算法的性能预测方法的算法参数。
     最后讨论的是天文图像合成网格服务的任务再分配问题。选取CPU就绪队列长度、磁盘请求队列长度和任务进度三个负载参数衡量资源性能,提出了基于连通分图的负载聚类方法以决定是否需要任务迁移及迁移任务的发送者和接收者。通过分析任务迁移后总完成时间变化规律建立了该网格服务任务再分配的可分负载模型。提出了该网格服务的任务再分配算法。
     本文对天文图像合成网格服务的任务分配和再分配模型和算法进行了深入的研究和设计,给出了多项式时间求取该网格服务负载平衡最优解的完整方案,提高网格系统性能。
Grid Computing implements resource sharing and coordinated problem solving in dynamic, multi-institutional virtual organizations. Load balance tries to balance the load of every computer node. Divisible load theory(DLT) is a methodology that implements load balance in data-intense parallel and distributed system scheduling. The paper uses the DLT to solve the task allocation and reallocation problem of astronomical image mosaicking grid service.
     Firstly, the system architecture of the grid service is designed. Four workflows are analysed and summarized. The service flow of the grid service is put forward.
     Secondly, the task allocation of the grid service is discussed. By making a deep research into the grid task scheduler based on the ant algorithm, the author brings forward a simple and effective performance prediction method based on the ant algorithm to support the load balance. Root submitting twice adding, root submitting once adding, root computing twice adding and root computing once adding divisible task models are proposed. To tackle the storage constraint and divisible granularity problems, a faster and wider used storage constraint allocation algorithm than IBS and increasing one heap sort optimum integer approximation algorithm based on a suboptimum integer approximation algorithm are designed. The task allocation flow based forenamed models and algorithms is found. A simulaton for choosing the parameters of the performance prediction method based on the ant algorithm is done.
     Finally, the task reallocation of the grid service is discussed. CPU queue length, disk request queue length and task progress rate are chosen to weigh the resource performance. A load cluster method based on connected subgraph is proposed to decide whether task migration is needed and the sender and receivers of it. By analysing the change of the complete time after the task migration, the author derives a task reallocation divisible task model. A task reallocation algorithm of the grid service is designed.
     The task allocation and reallocation models and algorithms of astronomical image mosaicking grid service are researched and designed. A integrated solution that can get the optimum allocation of the load balance of the grid service in polynomial time is presented to improve the grid system performance.
引文
[1] I Foster, C Kesselman. The Grid: Blueprint for a New Computing Infrastructure. San Francisco, USA: Morgan Kaufmann Publishers. 1999
    [2] I Foster, C Kesselman, and S Tuecke. The Anatomy of the Grid: Enabling Scalable Virtual Organization. International Journal of High Performance Computing Applications. 2001. 15(3):200~222
    [3] Rajkumar Buyya. Economic-based Distributed Resource Management and Scheduling for Grid Computing. Doctor of Philisophy, Monash University. 2002
    [4] I Foster, C Kesselman, J Nick, S Tuecke. The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration. January, 2002
    [5] Marty Humphrey, Glenn Wasson, Jarek Gawor, et al. State and Events for Web Services: A Comparison of Five WS-Resource Framework and WS-Notification Implementations. 14th IEEE International Symposium on High Performance Distributed Computing(HPDC-14), Research Triangle Park, NC. July 2005. 24~27
    [6] Maozhen Li, Mark Baker. The Grid Core Technologies. England: John Wiley& Sons, Ltd. 2005. 243~254
    [7]张飞,网格资源的性能预测方法研究:[硕士学位论文],上海:同济大学,2006
    [8]胡亮,车喜龙,唐阔,网格应用程序的性能预测策略,吉林大学学报(理学版),2005,43(6):794~798
    [9]戴光明,孟永良,网络并行计算中动态负载平衡的实现,计算机工程与应用,1998,(10):19~20
    [10]肖侬,黄金锋,卢宇彤,网络并行计算的动态负载平衡策略,计算机工程与科学,1998,20(3):13~17
    [11]王广芳,钟志伟,赵先武,分布式计算系统(DCS)负载平衡算法20年,计算机工程与设计,1994,16(5):57~63
    [12]李冬梅,施海虎,负载平衡调度问题的一般模型研究,计算机工程与应用,2007,43(8):121~125
    [13] Y.C. Cheng and T.G. Robertazzi. Distributed computation with communication delays. IEEE Transactions on Aerospace and Electronic Systems. November 1988. 24(6):700 ~712
    [14] Bharadwaj V., Ghose D., Robertazzi T.G. Divisible load theory: a new paradigm for load scheduling in distributed systems. Cluster Computing. 2003. 6(1):7 ~17
    [15] Robertazzi T.G. Ten reasons to use divisible load theory. Computer. May 2003. 36(5):63~68
    [16] Montage An Astronomical Image Mosaic Engine. http://montage.ipac.caltech.edu/
    [17] M Dorigo. Optimiztion, Learning and Natural Algorithma (in Italian). Ph.D. thesis, Dipartimento di Elettronica, Politecnico di Milano, IT. 1992
    [18] M Dorigo, V Maniezzo and A Colorni. The ant system: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics Part B. 1996. 26(1):29~41
    [19]许智宏,孙济洲,基于蚂蚁算法的网格计算任务调度方法设计,天津大学学报,2004,37(5):414~418
    [20] Zhihong Xu, Xiangdan Hou, Jizhou Sun. Ant algorithm based task scheduling in grid computing, CCECE. 2003:1107~1110
    [21] Beaumont O., Casanova H., Legrand A., et al. Scheduling divisible loads on star and tree networks: Results and open problems. IEEE Transaction on Parallel and Distributed Systems. 2005. 16(3):207~218
    [22] Sohn J., Robertazzi T.G. Optimal load sharing for a divisible job on a bus network. IEEE Transaction on Aerospace and Electronic Systems. 1996. 32(1):34~40
    [23] Batainch S., Robertazzi T.G., et al. Closed form solutions for bus and tree networks of processors load sharing a divible job. IEEE Transaction on Computers. 1994. 43(10):1184~1196
    [24] Chan S., Bharadwaj V., Ghose D. Large matrix-vectior products on distributed bus networks with communication delays using the divisible load paradigm:performance and simulation. Mathmatics and Computer in Simulation. 2001. 58:71~92
    [25] Bharadwaj V., Ghose D., Mani V. Optimal sequencing and arrangement in distributed single-level networks with communication delays. IEEE Transaction on Parallel and Distributed Systems. 1994. 5(9):968~976
    [26] Casanova, Henri. Modeling large-scale platforms for the analysis and the simulation of scheduling strategies. Proceedings. 18th International Parallel and Distributed Processing Symposium, IPDPS. 2004. 2391~2398
    [27] Marchal L., Yang Y., Casanova H., Robert Y. A realistic network/application model for scheduling divisible loads on large-scale platforms. 19th IEEE International Parallel and Distributed Processing Symposium. 2005. p 10 pp.
    [28] Dantong Yu, Robertazzi T.G. Divisible load scheduling for grid computing. Proceedings of the Fifteenth IASTED Internation Conference on Parallel and Distributed Computing and Systems. 2003. 1:1~6
    [29] Li X., Bharadwaj V., Ko C.C. Divisible load scheduling on single-level tree networks with buffer constraints. IEEE Transactions on Aerospace and Electronic Systems. Oct. 2000. 36(4):1298~1308
    [30] Drozdowski M., Wolniewicz P. Optimum divisible load scheduling on heterogeneous stars with limited memory. European Journal of Operational Research. July 2006. 172(2):545~559
    [31] Bharadwaj V., Yao J. Divisible load scheduling strategies on distributed multi-level tree networks with communication delays and buffer constraints. Computer Communications. Jan. 2004. 27(1):93~110
    [32] Veeravalli B., Xiaolin Li, Chi Chung Ko. On the influence of start-up costs in scheduling divisible loads on bus networks. IEEE Transactions on Parallel and Distributed Systems. Dec. 2000. 11(12):1288~1305
    [33] Veeravalli B., Viswanadham N. Suboptimal solutions using integer approximation techniques for scheduling divisible loads on distributed bus networks. IEEE Transactions on Systems, Man & Cybernetics, Part A (Systems & Humans). Nov. 2000. 30(6):680~691
    [34] Rajkumar Buyya, Manzur Murshed. GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing. Concurrency and Computation: Practice and Experirence. 2002. 14(3) :13~15
    [35] Anderson T.E., Culler D.E., Patterson D.A. A Case for NOW(Networks of Workstations). IEEE Micro. Feb. 1995. 15(1):54~64
    [36] Kunz T. The influence of different workload descriptions on a heuristic load balancing scheme. IEEE Transactions on Software Engineering. July 1991. 17(7):725~730
    [37] Banawan S.A., Zahorjan J. On comparing load indices using oracle simulation. New York, USA:1990 Winter Simulation Conference Proceedings(Cat. No.90CH2926-4). 1990. 851~856
    [38] Songnian Zhou. An experimental assessment of resource queue lengths as load indices. USENIX Assoc: USENIX Association Winter Conference Proceedings. 1987. 73~82
    [39] Ferrari D., Zhou S. A load index for dynamic load balancing. 1986 Proceedings of the Fall Joint Computer Conference (Cat. No.86CH2345-7). 1986. 684~690
    [40] Haddad E. Optimal dynamic redistribution of divisible load in distributed real-time systems. Proceedings of the IEEE Workshop on Real-Time Applications (Cat. No.94TH0663-5). 1994. 21~26
    [41] Haddad E. Real-time optimization of distributed load balancing. Proceedings of the Second Workshop on Parallel and Distributed Real-Time Systems. 1994. 52~57
    [42] Suda R., Tomi S. Task redistribution scheduling using multi-master divisible load model. Proceedings of the 18th IASTED International Conference on Parallel and Distributed Computing and Systems. 2006. 160~165
    [43] Haddad E. Runtime reallocation of divisible load under processor execution deadlines. Proceedings of the Third Workshop on Parallel and Distributed Real-Time Systems. 1995. 30~31
    [44] Marchal Loris, Rehn Veronika, Robert Yves, Vivien Frederic. Scheduling algorithms for data redistribution and load-balancing on master-slave platforms. Parallel Processing Letters. March 2007. 17(1):61~77
    [45]中国虚拟天文台,http://www.china-vo.org/
    [46] International Virtual Observatory Alliance. http://www.ivoa.net/
    [47]都志辉,网格计算,清华出版社,2002
    [48] Bataineh, Sameer M. Toward an analytical solution to task allocation, processor assignment, and performance evaluation of network processors. Journal of Parallel and Distributed Computing. January 2005. 65(1):29~47
    [49] Ghose D., Hyoung Joong Kim, Tae Hoon Kim. Adaptive divisible load scheduling strategies for workstation clusters with unknown network resources. IEEE Transactions on Parallel and Distributed Systems. Oct. 2005. 16(10):897~907
    [50] Sohn J., Robertazzi T.G. Optimal divisible job load sharing for bus networks. IEEE Transactions on Aerospace and Electronic Systems. Jan. 1996. 32(1):34~40
    [51] Barlas G.D. Collection-aware optimum sequencing of operations and closed-form solutions for the distribution of a divisible load on arbitrary processor trees Transactions on Parallel and Distributed Systems. May 1998. 9(5):429~441
    [52] Bharadwaj V., Li H.F., Radhakrishnan T. Scheduling divisible loads in bus networks with arbitrary processor release times. Computers & Mathematics with Applications. Oct. 1996. 32(7):57~77
    [53] Veeravalli B., Wong Han Min. Scheduling divisible loads on heterogeneous linear daisy chain networks with arbitrary processor release times. IEEE Transactions on Parallel and Distributed Systems. March 2004. 15(3):273~288
    [54] Bharadwaj V., Barlas G. Scheduling divisible loads with processor release times and finite size buffer capacity constraints in bus networks. Cluster Computing. 2003. 6(1):63~74

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700