流媒体集群系统复制存储策略研究
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
宽带化高速网络技术、动态影像压缩解码技术与大容量存储技术的成熟和日益增长的互联网多媒体需求,促成了流媒体的诞生和发展。流媒体就是应用流技术在网络上传输的多媒体文件,而流技术就是把连续的影像和声音信息经过压缩处理后放入网络服务器,让用户一边下载一边观看、收听,而不需要等整个压缩文件下载后才可观看的网络传输技术。集群是由一组网络互联且独立的分布式存储、流化服务器构成,和集中式结构相比具有高可扩展性、高可用性、高性价比三方面的优势。副本放置问题是指在分布式环境下,为了优化特定的系统性能指标而对目标数据(文件)进行副本生成、副本放置、副本替换等一系列操作的问题,通常分为静态和动态两种典型方式。本文研究复制存储方式下的流媒体服务器集群的副本放置/内容部署问题。
     首先研究了在固定节目流行度的情况下,如何进行内容优化部署以最小化流媒体集群系统拒绝率和降低复制存储消耗的问题。首先运用排队理论知识分析得出优化目标和服务器访问概率之间的数值联系,并且通过某些数值方法确定出系统最小拒绝率情况下的最优服务器访问概率。由于内容部署属于NP-Hard问题且完全决定每台服务器的访问概率,本文设计了初始放置、副本交换和对等副本访问概率调整三种启发式策略来进行内容部署,以满足在优化内容分布下每台服务器访问概率和最优值之间的差异最小,从而实现降低系统拒绝率和存储代价的目标。最后分别采用数值分析和离散事件仿真验证了模型的正确性和算法的有效性。
     其次研究了同构流媒体集群服务器环境下的动态副本放置问题,提出一种请求调度和动态内容部署相结合的新颖策略。首先采用滑动窗的方式预测各文件的点播率,以平衡服务器之间的实时累积访问概率为目标,在不进行内容部署的情况下,预分配相同文件不同副本的访问概率;同时进行零迁移代价的动态内容更新,从而能够提前反映出请求的实时倾向性,在降低系统请求拒绝率的同时,有效的减少了请求被分发到服务器后产生负载迁移的频度和代价。仿真分析的结果表明这种策略对于流媒体服务系统的研究和实践具有指导意义。
     最后结合离散事件和排队理论,独创性地搭建了一个普遍适用的服务器集群系统仿真平台。本文提出的流媒体仿真平台,符合实际运行系统特点,各组成单元均具有相对独立性,可以通过自由组合的方式,适应于各种不同参数要求、优化策略、运行环境的仿真。而且由于对每个功能严格细化,保证了不同策略下仿真结果的公平性、透明性。
Recent advances in high-speed network technology,dynamic video data compression and data decompression technology,bulk storage technology,and the increasing demand of network multimedia,accelerates the birth and development of streaming media.Streaming media is the multimedia transferred over network by stream technology. Stream technology,which compressing continuous audio or video program and placing them to the web server,can make the clints have no use for downloading the whole program from the server.The program can be listened and watched when it is being downloaded.The clustered server system,consisting a group of distributed streaming servers,is superior to the centralized structure in three aspects:good scalability, high availability,and competitive performance-to-price ratio.Replica Placement Problem(RPP),which involving how to produce the replicas of data or files,how to distribute the replicas to the servers and how to replace the old replicas with demanding replicas,et.,has the goal of enhancing the performance of the system.Typically, RPP formulations fall into two categories:static and dynamic.Our research focuses on replication policy or content distribution for clustered server system.
     Firstly,the optimizing problem of content distribution which minimizes the blocking probability and storage consumption on clustered streaming media system is discussed, in the case of knowing every program' s unchanged popularity.The queuing theory is adopted to analysis the relationship between the server' s access probability and the optimizing goal.The ideal access probability of every server can be obtained by some numerical methods,under the circumstance of minimal blocking probability. Content distribution determining each server' s access probability,has been proved to be NP-Hard.The whole content distribution process consists of three strategies,i.e. initial allocating,duplicate swapping and peer duplicate' s access probability adjusting. All the heuristic arithmetic is designed to perform the content distribution in order to minimize the distance between the result of optimization and the ideal one,minimize the storage consumption and reduce the blocking probability.Lastly the correctness of system modeling and the efficiency of proposed arithmetic are verified by numerical analysis and discrete event simulation.
     Secondly,the problem of dynamic content deployment for clustered streaming media system consisting of homogeneous servers is also discussed in this paper.We propose a dynamic storage balancing(DSB) based content updating strategy to lower the service blocking probability,to balance the load of media servers and to reduce the update cost.A relatively short time window is adopted to predict the changing popularity of all multimedia files.Unlike other exiting policies applying the predicted result to replicating and de-replicating,we assign it properly to each replica in the cluster for the purpose of unifying every server' s access probability(AP),in other words,making the system be in a condition of DSB.Then we gradually replicate the newly popular objects to enhance the ability of DSB.Meanwhile,we adopt a dereplication strategy with zero in-service request migration cost to avoid storage space being exhausted by replicas of previously popular objects.The simulation analysis shows that the proposed policy can achieve an outstanding system performance.
     Lastly,a novel simulation platform for server cluster is brought forward which is based on queueing theory.The component elements of the platform may be assembled easily to execute various simulation because of their relatively independence.Since the function of each sanction is subdivided strictly,the whole platform which is in accordance with the actual system can guarantee the clarity and fairness of different policies.
引文
昌玉芳,黄本雄.2007.一种基于集群的VOD媒体服务器方案设计[J].计算机与数字工程,35(2):172-175.
    倪云竹,吕光宏,黄彦辉2006.用遗传算法解决基于分条技术的磁盘负载均衡问题[J].计算机学报,29(11):1995-2001.
    代桂平.2005.半Markov控制过程的性能灵敏度分析、优化及其算法研究:[博士学位论文][D].中国科学技术大学.
    赵兵选.2008.IPTV系统中的负载均衡技术研究:[硕士学位论文][D].中国科学技术大学.
    吴松.2002.高性能集群流媒体服务器系统结构及存储系统研究:[博士学位论文][D].华中科技大学.
    高奎.2005.实时流媒体系统若干关键技术的研究:[博士学位论文][D].中国科学院计算技术研究所.
    李翠莲,马允胜,任久春.2004.一种基于网格的跨集群VOD系统及其仿真实现[J].复旦学报(自然科学版),43(1):103-109.
    林光国,戴琼海,丁嵘.2005.基于用户行为统计的流媒体集群负载均衡算法[J].清华大学学报(自然科学版),45(4):525-528.
    刘苏,易波.2006.一种以负载均衡为目标的分布式VOD系统视频文件存储方法[J].计算机工程与科学,28(7):43-46.
    刘威,程文青.2007.流媒体点播中用户交互式行为建模[J].电子与信息学报,29(9):2252-2256.
    王颖,顾铁成,张阳,陆桑璐,谢立.2005.一种基于分布式异构服务器机群VOD系统的数据分布策略[J].小型微型计算机系统,26(9):1611-1616.
    向哲,钟玉琢.2000.流调度算法验证平台的设计与实现[J].小型微型计算机系统,21(12):1237-1239.
    张丽.2001.流媒体技术大全[M].中国青年出版社.
    赵俊,金梅.Feb.2008.自适应的集群流媒体文件副本放置策略[J].计算机应用研究,25(2):594-596.
    周俊,李文中,陆桑璐,陈道蓄.2005.利用网格技术实现流媒体服务迁移[J].计算机科学,32(8):109-1 13.
    钟玉琢,向哲,沈洪.2003.流媒体和视频服务器[M].北京:清华大学出版社.
    Asit Dan,Martin Kienzl,Dinkar Sitaram,1995.Dynamic segment replication policy for load-balancing in video-on-demand servers[J].ACM Multimedia Systems,3(3):93-103.
    Choi C.C.Y,Hamdi M.,2003.A scalable video-on-demand system using multi-batch buffering techniques[J].IEEE Transactions on Broadcasting,49(2):178-191.
    C.Chou,L.Golubchik,J.Lui 2000.Striping doesn' t scale:How to achieve scalability for continuous media servers with replication[C].Proceedings of 20th International Conference on Distributed Computing Systems(ICDCS),Taipei,Taiwan,64-71.
    Chou C.,Golubchik L.,Lui J.2000.A performance study of dynamic replication technique in continuous media servers[C].Proc.the 8th International Symposium on Modeling,Analysis and Simulation of Computer and Telecommunication Systems,San Francisco California,64-71.
    Conklin G.J.,Greenbaum G.S.,Lillevold K.O.,et al.2001.Video coding for streaming media delivery on the Internet[J].IEEE Transactions on Circuits and Systems for Video Technology,11(3):269-281.
    M.Dakshayini,H.S.Guruprasad,H.D.Masheshappa,A.S.Manjunath 2007.Load Balancing in Distributed VoD using Local Proxy Server Group[LPSG][C].Proc.the 7th International Conference on Computational Intelligence and Multimedia Application,Sivakasi,India,162-166.
    Jonathan Dukes,Jeremy Jones.2002.Dynamic RePacking:A Content Replication Policy for Clustered Multimedia Servers[C].Proceedings of the Microsoft Research Summer Workshop,Cambridge,UK.
    M.H.Garey,D.S.Johnson.1979.Computers and Intractability:A Guide to the Theory of NP-Completeness[M].San Francisco,CA:Freeman.
    GK Zipf 1931.Selective studies and the principle of relative frequency in language [M].Massachusetts:Harvard University Press,Cambridge.
    Carsten Griwodz,Michael Bar,Lars C.Wolf,1997.Long-term Movie Popularity Models in Video-on-Demand Systems:or The Life of an on- Demand Movie[C]. Proceedings of the fifth ACM international conference on Multimedia,Washington,United States,349-357.
    D.Gross,and C.M.Harris.1985.Fundamentals of Queueing Theory[M].New York:Wiley.
    Jun Guo,E.W.M.Wong,S.Chan,Peter Taylor 2008.Combination Load Balancing for Video-on-Demand System[J].IEEE Transactions on Circuits and Systems for Video Technology,18(7):937-948.
    T.Ibarkai,N.Katoh 1988.Resource allocation problem-Algorithmic approaches[M].The MIT Press.
    D.L.Jagerman 1974.Some properties of the Erlang loss function[M].Bell Syst.Tech.J.
    Jianliang Xu,Bo Li,and Dik Lun Lee,Sep.2002.Placement Problems for Transparent Data Replication Proxy Services[J].IEEE Journal on Selectied Areas in Communications,20(7):1383-1398.
    Yiu-Wing Leung,Ricky Yuen-Tan Hou.2005.Assignment of Movies to Heterogeneous Video Servers[J].EEEE Transactions on Systems,Man,and Cybernetics-Part A:System and Human,35(5):665-681.
    K.-M.Ho,W.-F.Poon,K.-T.Lo 2007.Performance Study of Large-Scale Video Streaming Services in Highly Heterogeneous Environment[J].IEEE TRANSACTIONS ON BROADCASTING.
    S.S.Rao 1994.Optimization Theory and Applications[M].New York:Wiley,2nd.
    Sonia G.,Angeles Navarro,Juan L.,Emilio L.Zapata 2006.A Case Study of Load Sharing Based on Popularity in Distributed VoD Systems[J].IEEE TRANSACTIONS ON MULTIMEDIA,8(6):1299-1304.
    D.N.Serpanos,L.Georgiadis,and T.Bouloutas.Feb.1998.MMPacking:A load and storage balancing algorithm for distributed multimedia servers[J].IEEE Transactions on Circuits and Systems for Video Technology,8(1):13-17.
    K.S.Tang,K.T.Ko,S.Chan,E.Wong,Oct.2001.Optimal File Placement in VOD System Using Genetic Algorithm[J].IEEE Transactions on Industrial Electronics,48(5):891-897.
    Shiao-Li Tsao,Meng Chang Chen,Ming-Tat Ko,Jan-Ming Ho,Yueh-Min Huang,1999.Data Allocation and Dynamic Load Balancing for Distributed Video Storage Server[J].Journal of Visual Communication and Image Representation 10,197-218.
    Thanasis Loukopoulos,Petros Lampsas,Ishfaq Ahmad,2005.Continuous Replica Placement Schemes in Distributed Systems[C].Proceedings of the 19th Annual International Conference on Supercomputing,Boston,284-292.
    Li Wang,Aug.2005.Design and Implementation of TCPHA[D].http://dragon.linuxvs.org/~dragonfly/.
    Lian Shen,Wei Tu,Steinbach E,2007 A Flexible Starting Point Based Partial Caching Algorithm for Video on Demand[C].IEEE International Conference on Multimedia and Expo,76-79.
    J.L.Wolf,P.S.Yu,H.Shachinai.1997.Disk load balancing for video-on-demand systems[J].ACM/Springer Multimedia Systems Journal,5(6):358-370.
    Jian Yang,Hongsheng Xi,Li Jing,and Wei Xing.Aug.2008.A Time-Window Based Request Distribution Strategy for Media Server Cluster[C].International Conference on Computer Science and Information Technology,2008(ICCSIT'08),Singapore,152-156.
    Minglong Zhang,Boqin Feng.2008.A Novel Migration Algorithm Based-on“States-balancing”in a Distributed Multimedia Services System[C].Proc.the 2nd International Conference on Multimedia and Ubiquitous Engineering,Busan,South Korea,336-341.
    Zhao Y.Q.,Shi Z.,Kuo C.C.J.2002.Dynamic Load Balancing and Content Update for Media Storage Servers[C].Conference on Visual Information Processing XI,Orlando,Florida,201-212.
    Zhao Y.Q.,Kuo C.C.J.2004.Design issues on request migration for video-on-demand services[C].Proceedings of the 2004 International Symposium on Circuits and Systems,Orlando,Florida,2:49-52.
    Zhao Y.Q.,Shi Z.,Kuo C.C.J,2005.Scheduling Design for Distributed Videoon-Demand Servers[C].IEEE International Symposium on Circuit and Sys-tems(ISCAS' 2005),Kobe,Japan,1545-1548.
    Zhiqiang Bi,Christos Faloutsos,Flip Kron.2001.The“DGX”distribution for mining massive,skewed data[C].Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
    X.Zhou and C.Z.Xu.2002.Optimal Video Replication and Placement on a Cluster of Video-on-Demand Servers[C].Proc.the 31st International Conference on Parallel Processing(ICPP),Vancouver Canada,547-555.
    Songqing Chen,Haining Wang,Xiaodong Zhang,et al.2005.Segment-based proxy caching for Internet streaming media delivery[J].IEEE Multimedia,12(3):59-67.
    Tsang D,Ross K.1990.Algorithms to determine exact blocking probabilities for mul-tiratetree networks[J].IEEE Transactions on Communications,38(8):1266-1271.
    S.-H.Gary Chan,Fouad Tobagi,2001.Distributed Servers Architecture for Networked Video Services[J].IEEE/ACM TRANSACTIONS ON NETWORKING,9(2):125-136.
    Yu H,Zheng D,Zhao B,Zheng W.2006.Understanding user behavior in large-scale video-on-demand systems[C].Proceedings of the 1st ACM SIGOPS/EuroSys European Conference on Computer Systems 2006.ACM New York,NY,USA,333-344.
    Frederic Thouin 2006.Video-on-demand Equipment Allocation:[Dissertation for Master's Degree][M].McGill University Montreal,Canada
    Thanasis Loukopoulos,Petros Lampsas,Ishfaq Ahmad 2005.Continuous Replica Placement Schemes in Distributed Systems][C].Proceedings of the 19th Annual International Conference on Supercomputing,Boston,284-292

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700