基于Agent的集群负载均衡模型及其实验研究
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
随着网络技术的不断发展,越来越多的人通过网络来了解信息,由于用户群体的不断增加及网络应用的丰富,使得服务器接收到的请求数越来越多,有关于网络负载的问题越发突出。集群技术的出现很好地解决了这一问题。本文主要贡献及创新点包括:
     1、提出了一种基于Agent的动态自适应集群负载均衡模型。通过把Agent技术引入集群负载均衡系统中,创新性的构建了一个基于Agent的动态自适应集群负载均衡的普适模型。将Agent技术引入到传统集群负载均衡模型中,增加了集群负载系统的智能性和自适应能力;
     2、创新性的把请求任务识别与分类思想融合到了基于Agent的集群动态负载均衡模型中。通过引入请求任务识别与分类思想,结合Agent技术,实现了集群任务的自动识别及分类处理,提高了集群系统的运行效率;
     3、针对本文提出的基于Agent的集群动态负载均衡模型,对其具体功能设计了详细的算法,并针对模型多个指标做了性能分析。
     4、提出并实现了一种适用于流媒体业务的基于Agent的动态自适应负载均衡集群。在详细分析了流媒体业务存在的问题及特点的基础上,根据这些问题及特点做了针对性的优化,设计了针对于流媒体业务的集群模型并给出了具体实现。实验结果验证了本文模型的普适性和有效性。
     5、提出并实现了一种针对移动互联网业务的基于Agent的动态自适应负载均衡集群。针对移动互联网特点及新的挑战,设计了一种针对移动互联网的基于Agent的负载均衡集群,此集群结合本文提出的模型进行设计,同时又兼顾了移动互联网的特点。进一步验证了本文模型的普适性和有效性的同时。
With the rapid development of Internet technology, more and more people begin toretrieve news information and do shopping through the Internet. Many people would like tohave an e-account, even many enterprise make transactions through the Internet in order togain more ratification. But troubles are coming. With the increasing scale of Internet users,the load of the Internet is becoming heavier. Although enhancing the performance of theInternet hardware could somewhat relieve this problem,but the continuously increasingaccess request brings even more load to the servers. So, the development of hardware isimportant indeed, but there is also a need of corresponding load balancing strategy.Fortunately, the Computer Cluster technology preferably solved this problem.
     Cluster (also called Computer Cluster) accomplish computational task by a group ofloose combined software and hardware who cooperate highly tightly. One of the advantagesof cluster technology is that it could achieve high performance, high fault tolerance and highflexibility with a very low cost. Actually, a large scale cluster could be seen as a whole. Eachserver corresponds to a node, and server could connect to each other in number of ways.Reasonable scheduling among tasks is the core of the cluster technology.
     Nowadays dynamic balancing load technology is widely used in Internet clustertechnology. The load balancing techniques are often divided into two categories: Dynamicload balance and Static load balance. In order to achieve improved performance of the entirecluster system to enhance the overall network load capacity, improve network response timegoal, this paper, a more flexible intelligent Agent technology, this paper proposes a dynamicadaptive load balancing model based on Agent. And through the different request packetanalysis tasks, divided request task into N categories, each category corresponding to arelatively efficient load balancing schedule algorithm, which have enhanced the performanceof the cluster significantly.
     This paper mainly accomplished the following works:
     1.Built an Agent based dynamic adaptive cluster load balancing generalized model byintroducing the Agent technology, which could not only improve the intellectuality of the load balancing system, but also enhance the adaptive capability;
     2.By introducing the idea of recognition and classification of the request task andcombining the Agent technology, achieved the automatic recognition and classification of thecluster task, which further enhance the operational efficiency of the system;
     3. Based on the aforementioned model, designed a corresponding algorithm andimplement it. Our model is proven to be effective through the comparison with the originalLVS cluster on several performance parameters;
     4. Analyzed the special characteristics of the streaming media service and its demands onthe streaming media. On this basis, applied the Agent based dynamic adaptive load balancingmodel to the cluster environment of streaming media servers, and do specific optimizationsaiming at the special characteristics of the streaming media;
     5. Applied the Agent based dynamic adaptive load balancing model to the clusterenvironment of streaming media servers, and the model is proven to be effective in astreaming media server cluster through the comparison with the original cluster;
     6. Analyzed in depth the characteristics of the mobile Internet itself, including thedifferences of the user’s behavior, different network throughput, service scale, and servicerequirements. And new challenges brought by these were also analyzed;
     7. Aiming at the characteristics and new challenges, designed an Agent based dynamicadaptive load balancing cluster for the mobile Internet. The cluster is designed according tothe model we proposed in this paper, at the same time give considerations to thecharacteristics of mobile Internet;
     8. Constructed the model mentioned above in commercial environment, designed andimplemented4service types to access the cluster. The network throughput, service load andservice demands of these service types are different, which almost cover the common servicetypes of mobile Internet, simple and generalized, could validate our work convectively.
     In summary, this paper introduce the technology of Agent and the idea of recognitionand classification of the request task,presents an Agent based dynamic adaptive loadbalancing model and analyze the performance of this model; innovation used this model instreaming media, do specific optimizations aiming at the special characteristics and demandsof the streaming media; systematically analyses the characteristics and demands of mobileInternet and do specific optimizations on our model according to these characteristics,implement and validate it in commercial environment. The results showed that the theoreticalsignificance and application value of our Agent based dynamic adaptive load balancing modelcan provide a reference to the same kind of research and application.
引文
[1] Zhenguo Xuan. Load Balancing Technology Based On Consistent Hashing For DatabaseCluster Systems [C]. National Conference on Information Technology and ComputerScience (CITCS2012):396-400.
    [2]孟丹.集群高效通信系统的研究与实现[J].中国中国科学院计算研究所,2003:34-36.
    [3] Christen, P.. A parallel iterative linear system solver with dynamic load balancing [J].ANZIAM,2000,42(E):400-414.
    [4]马宁.动态反馈负载均衡算法的研究[J].华中科技大学学报,2010,02期:31-33.
    [5]张玉芳,魏钦磊,赵膺.基于负载权值的负载均衡算法[J].计算机应用研究,2012,12期:4712-4713.
    [6] Armbrust M, Fox A, Griffith R, et al. A view of cloud computing [J]. Communications ofthe ACM,2010,53(4):50-58.
    [7] R. Buyya, Y. S. Chee, S.Venugopal. Market-Oriented Cloud Computring: Vision, Hype,and Reality for Delivering IT Service as Computing Utilities [C]. Proc of the HPCC,2008:5-12.
    [8] RIMAL B P, JUKAN A, KATSAROS D, et al. Architectural requirements for cloudcomputing systems: An enterprise cloud approach [J]. Journal of Grid Computing,2011,9(1):3-26.
    [9]王红斌. Web服务器集群系统的自适应负载均衡调度策略研究[D].吉林大学,2013.
    [10]Jovanovic, Z., Maric, S.. A Heuristic algorithms for dynamic task scheduling in highlyparallel computing systems [J]. Future Comput. Syst.2001,17:721–732.
    [11]苏英如,王培军,高学松. WWW服务器虚拟镜像与url重定向[J]《.中国教育网络》,2009:5-8.
    [12]丁原,刘玉树,朱天焕.集群服务器技术综述[J].兵工自动化,2000,3:10-13.
    [13]季刚.集群技术及负载均衡调度算法研究[J].计算机时代,2012,8:37-43.
    [14]Zhang Xiaofang, Hu Guozheng, High-availability clustering technology research andapplication [J]. Computer Engineering,2003,29(4):26-37.
    [15]刘振英,方滨兴,胡铭曾等.一个有效的动态负载平衡方法[J].软件学报,2001,12(4):564-568.
    [16]XU C Z, LAU F C M. Iterative dynamic load balancing in multicomputers [J]. Journal ofthe Operational Research Society,1994,45(7):786-796.
    [17]Surdeanu M, Moldovan D I and Harabagiu S M. Performance analysis of a distributedquestion/answering system [J]. IEEE Transactions on Parallel and Distributed Systems,2002, vol.13:579-596.
    [18]Iqbal S and Carey G F. Performance analysis of dynamic load balancing algorithms withvariable number of processors [J]. Journal of parallel and distributed computing,2005,vol.65:934-948.
    [19]Koyama K, Shimizu K and Ashihara H, et al. Performance evaluation of adaptive loadbalancing policies in distributed systems[C]. Proceedings of IEEE SingaporeInternational Conference on. IEEE,1993, vol.2:606-611.
    [20]Shan Z, Lin C and Marinescu D C. Modeling and performance analysis of QoS-awareload balancing of web-server clusters [J]. Computer Networks,2002, vol.40:235-256.
    [21]Yang J, Jin D and Li Y. Modeling and simulation of performance analysis for acluster-based web server [J]. Simulation Modelling Practice and Theory,2006, vol.14:188-200.
    [22]Teo Y M and Ayani R. Comparison of load balancing strategies on cluster-based webservers [J]. Simulation,2001, vol.77:185-195.
    [23]陈全,邓倩妮.云计算及其关键技术[J].计算机应用,2009年9月,29卷9期:143-148.
    [24]陈康,郑纬民.云计算:系统实例与研究现状[J]. Journal of Software, Vol.20,No.5:1337-1348.
    [25]王萍,张际平.云计算与网络学习[J].现代教育技术,2008,18(11):81-84.
    [26]赵龙文,侯义斌.多Agent系统及其组织结构[J].计算机应用研究,2000,第7期:12-14.
    [27]王汝传,徐小龙,黄海平.智能Agent及其在信息网络中的应用[J].北京邮电大学出版社,2007:8-14.
    [28]Rajagopalan Ashok, Hariri Salim. An agent based dynamic load balancingsystem[C].Bangalore: India.2000:164-171.
    [29]Xinjun Mao, Zhiming Chang. Agent-Orient Software Design Patterns [J], ComputerEngineering and Science.2011,33:72-78.
    [30]维基百科.计算机集群[EB/oL].http://zh.wikipedia.org/wiki/%E8%AE%A1%E7%AE%97%E6%9C%BA%E9%9B%86%E7%BE%A4,2009-1.
    [31]林闯. Web服务器集群请求分配和选择的性能分析[J].计算机学报,2000.23(5):500-508.
    [32]Goscinski, A., Distributed Operating Systems [M], Addison-Wesley, Sydney,1991.
    [33]RahimAdatia, FaizArni, KyleGabhart等著[M]. EJB编程指南,北京:电子工业出版社,2001:34-38.
    [34]R.Mirchandaney, D.Towsley, and J.A.Stankovie. Analysis of the effeet of delays onload sharing. IEEE Trans[J].ComPuters,1989.38(11):1513-1525.
    [35]J.Hong, X.Tan, and D.Towsley. A performance analysis of minimum laxity and earliestdeadline scheduling in a real-time systems[J]. IEEE Trans. ComPuters,1989.38(12):1736-1744.
    [36]D.L.Eager, E.D.Lazowska, and J.Zahorjan, Adaptive Load Sharing in HomeogeneousDistributed Systems[J]. IEEE Trans.Software Eng,1986.12(5):662-675.
    [37]L.M.Ni, C.W.Xu, and T.B.Gendreau. A Distributed Drafting Algorithm for LoadBalancing[J]. IEEE Trans.SiftwareEng,1985.11(10):1153-1161.
    [38]Yagoubi, B., Slimani, Y.: Task load balancing strategy for grid computing [J]. Comput.Sci.2007,3(3):186-194.
    [39]Haddad E. Dynamic Optimization of Load Distribution in Heterogeneous Systems[J].IEEE,1994:29-34.
    [40]Antonis K., Garofalakis J., Mourtos I. and spirakis P. A Hierarchical Adaptive DistributeAlgorithm for Load Balancing [J]. Journal of Parallel and Distributed Computing,Elsevier Inc.2003:151-162.
    [41]Bernard G., Steve D. and Simatic M. A Survey of Load Sharing in Networks ofWorkstations [J]. The British Computerm Society, The Institute of Electrical Engineersand IOP Publishing Ltd,1993:75-86.
    [42]Dandamudi S.P. Sensitive Evaluation of Dynamic Load Sharing in Distributed System [J].IEEE,1998:3-12.
    [43]Lin H. C. and Raghavendra C.S. A Dynamic Load Balancing Policy with a Central JobDispatcher (LBC)[J]. IEEE,1992:148-158.
    [44]Xu C. and Lau F.C.M. Load Balancing in Parallel Computers: Theory and Practice [M].Kluwer Academic Press,1997.
    [45]武星燕. Linux集群的进程迁移技术研究[D].黑龙江:哈尔滨工业大学,2008,1.
    [46]Donaldson, V., Berman. F., and Paturi, R., Program Speedup in a HeterogeneousComputing Network [J], Journal of Parallel and Distributed Computing, June,1994,21,3:316-322.
    [47]Jain P. and Gupta D. An Algorithm for Dynamic Load Balancing in Distributed Systemswith Multiple Supporting Nodes by Exploiting the Interrupt Service [J]. AcademyPublisher,2009:232-236.
    [48]Kabalan K.Y., Smari W.W. and Hakimian J.Y. Adaptive load sharing in Heterogeneoussystem: Policies, Modifications and Simulation [J]. CiteSeerx,2008, Vol.3, No.2:89-97.
    [49]Chhabra A., Singh G., Waraich S.S., Sidhu B. and Kumar G. Qualitative ParametricComparison of Load Balancing Algorithms in parallel and Distributed ComputingEnvironment [J]. Word Academy of Science, Engineering and Technology,2006:39-42.
    [50]Md.Firoj Ali, Rafiqul Zaman Khan. THE STUDY ON LOAD BALANCINGSTRATEGIES IN DISTRIBUTED COMPUTING SYSTEM [J]. International Jouranl ofComputer Science&Engineering Survey (JICSES), April2012, Vol.3, No.2:19-30.
    [51]Eager, D.L., E.D.Lazowska, and J.Zahorijan. A comparison of receiver-initiated andsender-initiated adaptive load sharing [J].performance Evaluation,1986,1(3):53-68.
    [52]陈涛,陈启买.分布式计算机系统负载平衡研究[J].计算机技术与发展,2006,16(5):33-35.
    [53]Lin, F.C.H. and R.M.Keller. The gradient model load balancing method.IEEETransactions on Software Engineering,1987,13(1):28-32.
    [54]Bryant, R.M. and R.A.Finkel. A stable distributed scheduling algorithm [C].Proc. ofthe2ndint’1Conf. on Distributed Computing Systems,1981:314-323.
    [55]张帆,袁道华.基于Limix的服务器集群系统设计与实现[J].计算机工程与应用,2003,14(5):127-131.
    [56]郑灵翔,刘君尧,陈辉煌. Linux下的负载均衡集群LVS实现分析与测试[J].厦门大学学报(自然科学版),2002,41(6):726-730.
    [57]章文篙.内核中的连接调度算法[EB/OL]. http://Zh.linuxvirtualserVer.org/node/35.
    [58]AM. Colajanni, P.S. Yu, and D.M. Dias, Analysis of task assignment policies in scalabledistributed web-server systems [J], IEEE Trans. Parallel Distrib. Syst.,1998, vol.9,No.6:585-600.
    [59]Wooldridge M J, Jennings N R. Intelligent Agent: theory and practice[J]. KnowledgeEngineering Review,1995,10(2):115-152.
    [60]Choi E. Performance test and analysis for an adaptive load balancing mechanism ondistributed server cluster systems [J]. Future Generation Computer Systems.2004, vol.20:237-247.
    [61]Zaki, M.J, Li,W. Parthasarathy,S. Customized dynamic load balancing for a network ofworkstations[J]. Journal of Par-alleland Distribute Computing,1997,43(2):156-162.
    [62]Xu Lan. Load balancing technology research [J]. Agriculture&Technology.2009,vol.29:155-157.
    [63]Sun H, Chunguang T. Optimizing Multi-agent MicroGrid Resource Scheduling byCo-Evolutionary with Preference [J]. TELKOMNIKA Indonesian Journal of ElectricalEngineering,2013,11(12):7222-7229.
    [64]Jiang W, Zhang J, Li J, et al. A Resource Scheduling Strategy in Cloud Computing Basedon Multi-agent Genetic Algorithm [J]. TELKOMNIKA Indonesian Journal of ElectricalEngineering,2013,11(11):6563-6569.
    [65]Quan B, Li H, Le Z. Dynamic Routing and Resource Assignment Algorithm In slotedoptical Networks [J]. TELKOMNIKA Indonesian Journal of Electrical Engineering,2013,11(4):1813-1821.
    [66]Campos, L., Scherson, I.: Rate of change load balancing in distributed and parallelsystems [J]. Parallel Comput.2000,26:1213-1230..
    [67]Wang Hongbin, Fang Zhiyi and Cui Shuang. Dynamic adaptive feedback of loadbalancing strategy [J]. Source: Journal of Information and Computational Science,2011,10,vol.8:1901-1908.
    [68]Honghui Niu, Weijun Chen. A New Framework of Effective Loading-Balancing [J].Microcomputer Information,2009,25(1-3):184-186.
    [69]Chunyong Yang, Shaoping Cheng, Routing Optimization for Network Load BalanceBased on Improved Ant Colony Algorithm [J]. Computer Engineering,2010,36(8),4-6.
    [70]谢茂涛,宋中山. LVS集群系统负载均衡策略的研究[J].计算机工程与科学,2006(8):30-32.
    [71]Qin Liu, Julong Lan, Design and Implementation of Dynamic Load Balancing in LVS [J].Computer Technology and Its Applications,2007,9(1):116-119.
    [72]雷迎春,张松,李国杰. Web集群服务器的分离式调度策略[J].计算机研究与发展,2002,39(9):1093-1098.
    [73]Valeria C., Michele C., Philip S.Y.. Dynamic Load Balancing on Web-Server Systems.IEEE Internet Computing [J].1999, Vol3(3):28-39.
    [74]Ciardo,G.,Riska,A.,Smirni, E.EquiLoad. A load balancing Policy for clustered webservers [J]. Performance Evaluation,2001,46(2):198-203.
    [75]Wensong Zhang. Linux Virtual Server Clusters Build highly-scalable andhighly-available network services at low cost [J]. Linux Magazine,2003:1-14.
    [76]Xie Zuo-gui, Qi Xiao-ya. High-availability and Load-balance Cluster System withoutLoad balancer[J]. Computer Engineering,2007,vol.33(3):136-137.
    [77]Chu LK, Tang H, Yang T, Shen K. Optimizing data aggregation for cluster-basedInternet services[C]. In: Proc. of the ACM SIGPLAN Symp. on Principles and Practice ofParallel Programming. New York: ACM Press,2003:119-130.
    [78]赵亮.流媒体服务集群系统关键技术研究[D].西安电子科技大学,2009.
    [79]林光国,戴琼海,丁嵘.基于用户行为统计的流媒体集群负载均衡算法[J].清华大学学报,2005,45(4):525-528.
    [80]陈广东.流媒体服务器集群负载均衡算法研究[D].华中师范大学,2006.
    [81]RFC3605.Real Time Control Protocol (RTCP) attribute in Session Description Protocol (SDP)[EB/OL]. http://www.rfc-editor.org/,2003.10.
    [82]RFC3550.RTP: A Transport Protocol for Real-Time Applications[EB/OL].http://www.rfc-editor.org/,2003.07.
    [83]RFC1889.RTP: A Transport Protocol for Real-Time Applications[EB/OL].http://www.rfc-editor.org/,1996.01.
    [84]RFC2326,Real Time Streaming Protocol (RTSP)[EB/OL]. http://www.rfc-editor.org/,1998.04.
    [85]陶桂东,张占军.基于RTP协议H.264视频流传输QOS保证的研究[J].装甲兵工程学院学报,2006,20(5):58-60.
    [86]杜彬,王淑玲,杨海波.RTSP流媒体服务器性能测试工具[J].计算机系统应用,2011,20(3):74-79.
    [87]Qadeer M A, ZahM M, Iqbal A,et al. Network Traffic Analysis and Intrusion DetectionUsing Packet Sniffer [J]. Communication Software and Networks,2010.ICCSN '10.Second International Conference on.2010:313-317.
    [88]Scho X, Nwa X, Lder J, et aL. On the Inpact of Security Protocols on thePerformance ofSNMP[J]. Network and Service Management, IEEE Transactions oa.2011,8(1):52-64.
    [89]Yingying C,Jain S, Adhikari V K, et aL. A first look at inter-data centertrafficcharacteristics via Yahoo! Datasets [C]. INFOCOM,2011Proceedings IEEE.2011:1620-1628.
    [90]Kent A,Gavrilov E M, Fisk M R. Network Host Classification Using Statistical Analysisof Flow Data[J]. Network system research Los Alamos NationalLaboratory,2010.
    [91]Guo-feng Zhao, Lu Wang, Chuan Xu, Hong Tang. Portal User Behavior on MobileInternet [J]. Information: an International Interdisciplinary Journal,2011, Vol.14:4055-4060.

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700