时移电视集群系统缓存调度研究
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
随着计算机技术、网络技术的不断发展,电信网、广播电视网和互联网融合发展的逐步推进,流媒体应用获得了广泛关注。时移电视(Time-shifted TV)作为典型的流媒体业务,受到互联网运营商以及广播电视网运营商的青睐与关注,关于时移电视的研究具有学术意义与实用价值。时移电视是一种允许用户在观看电视直播的过程中进行暂停、快进、快退、跳转等VCR操作的服务,具有数据量大、实时性要求高等特点,时移电视对网络带宽、存储设备等资源的需求较高,而单台服务器的服务能力有限,在提供大规模并发服务时通常需要建造服务器集群系统。
     本文在国家863项目“上海高性能宽带信息网”、国家广电总局项目“海南双向HFC实验网络”等课题的基础上,围绕时移电视集群系统的缓存调度展开深入研究,分别针对分布式存储结构的集群系统、共享存储结构的集群系统、大规模时移电视集群研究对应的缓存调度方案,建立了缓存放置问题的数学模型,并根据系统特点和工程实践提出相应的缓存调度策略。
     本文的创新工作简要归纳如下:
     1.针对分布式存储的时移电视集群系统的缓存放置问题,以降低用户请求拒绝率为目标建立了优化模型,并提出了一种基于遗传算法的缓存调度策略,策略结合基于流行度的随机放置算法,并利用遗传技术寻找最优解,具有一定的自适应学习能力。该策略可有效地降低集群系统的用户请求拒绝率,对不同模式的用户点播行为具有良好的适应性。
     2.针对共享存储结构的时移电视集群系统存在的存储子系统磁盘带宽瓶颈,讨论如何在服务器上缓存热门节目内容的缓存放置问题,建立了时变流行度模型与多目标优化模型作为分析框架,提出了完整的缓存调度策略。该策略考虑了时移电视具有实时录制、用户点播行为随时间变化等特点,包括分组复制算法、代价感知的最小负载优先缓存放置算法和双阂值动态调整算法,策略在调度过程中充分利用之前已经缓存的节目副本来避免副本的创建和删除,在保证系统实现负载平衡的同时降低了调度的部署代价,使缓存调度策略更适合于时移电视系统的工程应用。
     3.时移电视集群系统服务规模的扩大给缓存调度带来了更大的难度,需要利用有限容量的内存来降低磁盘带宽消耗。针对大规模服务系统提出了一种协同缓存策略,以管理调度服务器为中心,结合接入控制、缓存管理等技术对集群内所有媒体服务器的缓冲区协作管理,当管理服务器收到一个用户请求后,它会综合考虑全局缓存情况来决定服务器任务,以此提高整个系统的缓存命中率。此外还针对时移电视支持VCR操作的特点,对协作缓存策略进行了扩展。试验表明协作缓存策略可以有效地提高磁盘I/O的利用率,降低系统的用户请求拒绝率,适合应用于大规模的时移电视集群系统。
With the rapid development of computing technology and broadband networking and convergence of telecommunications networks, broadcast networks and internet, streaming media applications obtained more attentions. As a typical streaming media business, Time-shifted TV service is introduced to the market by internet service provider and broadcast TV service provider. Thus, the research on time-shifted TV also has academic and practical significance. Time-shifted TV provide broadcast TV service to users, and enables users to watch TV programs with a time shift and VCR-like controls like pause, jump, stop, fast forward, fast rewind etc operation. Due to high volume of data and real-time storage, time-shifted TV requires high network bandwidth and disk I/O bandwidth. The capacity of a single server is quite limited, so clustered servers are needed to provide large scale streaming media services.
     The work in this dissertation is based on "China High Performance Broadband Information Network" (3T'Net) project and "Hainan bidirectional HFC experimental network" projects. This dissertation discusses cache scheduling strategy for time-shifted TV server clusters, including distributed storage cluster, shared storage cluster and large scale cluster. For each kind of clustered server system, this dissertation gives an analytical model to describe the problems and finally proposed a corresponding policy. The main research contents and contributions are summarized as follows:
     1. The designing of distributed storage time-shifted TV cluster faces a cache placement problem of deciding how to cache channels to multiple servers so that the blocking probability is minimized subject to memory capacity constraints. We propose a popularity-based random placement (PRP) scheme together with the genetic algorithm (GA) to find an optimal or approximate optimal solution of the problem. The experiment results reveal that our proposed algorithm is efficient on improving the performance of time-shifted TV cluster in terms of minimizing blocking probability.
     2. To alleviate disk I/O bandwidth bottleneck of a shared storage cluster, a classification replication algorithm and a novel cache placement algorithm, named as Cost-aware Least Load Proportion First (CALLF) were proposed. It was formulated as a multi-objective optimization problem based on the time-based popularity model. The algorithms focus on the time-varying nature of Time-shifted TV, and explore previous stored information to reduce the cost of redeploying. We also present a dual-threshold reconfigure scheme to further balance the load of servers dynamically. Simulation reveals that the proposed algorithms are efficient on improving the performance of Time-shifted TV system in terms of total outgoing bandwidth requirement, and allow the tradeoffs between the bandwidth utilization and the reconfiguration overhead.
     3. In large scale time-shifted TV server clusters, by adopting cooperative caching technique, the free cache memory of all the servers in the cluster can be used more efficiently. A cooperative cache buffer scheduling strategy is presented, which makes use of each server to perform the caching task by manager server. It will help to reduce disk accesses, resulting in the improvement of the overall throughput of cluster. The proposed method is also designed to handle VCR-like operations. Simulations are performed to investigate its performance, and the results show that it can reduce disk I/O overhead and provide high throughput in clustered servers.
引文
韦乐平.2007.电信网络技术发展趋势[R].
    诺达咨询.2006.2006年中国IPTV业务市场研究与策略分析报告[R]..
    钟玉琢,向哲,沈洪.2003.流媒体和视频服务器[M].北京:清华大学出版社.
    林凡.2001.集群的可扩展性及其分布式体系结构[R].
    黄伟,谢冬青.2006.视频数据的存储调度优化策略[J]。电脑与信息技术,1(14):55-59。
    王颖,顾铁成,张阳.2005.一种基于分布式异构服务器机群VOD系统的数据分布策略[J]。小型微型计算机系统,26(9):1611-1616。
    邬江兴.2002.中国高性能宽带信息网(3TNet)综述[J].通讯世界,8(12):37-39.
    许永明,谢质文,欧阳春.2006.IPTV—技术与应用实践[M].北京:电子工业出版社.
    佐思信息.2009.2009-2012年中国IPTV市场调研及与投资前景预测报告[R].
    中国互联网络信息中心.第24次中国互联网络发展状况统计报告[S].2009.12.
    中华人民共和国国务院办公厅.2008.关于鼓励数字电视产业发展的若干政策[R].2008.1.1.
    APPLE.2001. http:/ldeveloper.apple.com/documentation/Quicklime/QTFF/qtff.pdf.
    Almeida J M, Krueger J, Eager D L.2001. Analysis of educational media workloads. NOSSDAV'O1:Proceedings of the 1 lth international workshop on Network and operating systems support for digital audio and video, Port Jefferson, New York, United States,2001 [C]. New York, NY, USA:ACM
    Akimaru H and K. Kawashima,1999. Teletraffic:Theory and Applications,2nd ed. London, U.K.:Springer-Verlag.
    BARB Broadcasters'audience research board, http://www.barb.co.uk
    Bommaiah E, K. Guo, M. Hofmann et al.2000. Design and implementation of a caching system for streaming media over the internet[A]. In Proceedings of IEEE Real Time Technology and Applications Symposium. Washington, DC:2000, 111-121.
    Bektas T, O. Oguz, and I. Ouveysi,2006. A novel optimization algorithm for video placement and routing, Communications Letters, IEEE, vol.10, no.2,114-116.
    Bektas T, O. Oguz, and I. Ouveysi,2007. A Lagrangean relaxation and decomposition algorithm for the video placement and routing problem [J], European Journal of Operational Research, vol.182, no.1, pp.455-465.
    Cha Meeyoung, Rodriguez Pablo, Jon Crowcroft, Sue Moon, Xavier Amatriain, Watching television over an IP network, Internet Measurement Conference Proceedings of the 8th ACM SIGCOMM conference on Internet measurement, Vouliagmeni, Greece,71-84.
    Chesire M, Wolman A, Voelker Geoffrey M.2001. Measurement and analysis of a streaming media workload. USITS'O1:Proceedings of the 3rd conference on USENIX Symposium on Internet Technologies and Systems, San Francisco, California,2001 [C].
    Cherkasova L, Minaxi.2002. Characterizing locality, evolution, and life span of accesses in enterprise media server workloads. NOSSDAV'02:Proceedings of the 12th international workshop on Network and operating systems support for digital audio and video, Miami, Florida, USA,2002 [C]. New York, NY, USA:ACM
    Chakareski, J. and Frossard, P.2007. Adaptive Systems for Improved Media Streaming Experience, IEEE Communications Magazine, vol.45, no.1,77-83.
    Costa C. P., et al.,2004. Analyzing client interactivity in streaming media, Proc.13th Int. Conf.World Wide Web, New York,534-543.
    Chen H, Hai Jin, Jianhua Sun, Xiaofei Liao, Dafu Deng.2003. A new proxy caching scheme for parallel video servers [C]. In:IEEE 2003 International Conference on Computer Networks and Mobile Computing (ICCNMC 2003),438-441.
    Chen Song-qing, Hai-ning Wang, Bo Shen et al.2005. Segment-based Proxy Caching for Internet Streaming Media Delivery [J]. IEEE Multimedia Magazine, 2005,12(3):59-67.
    Chen Song-qing, Bo Shen, Susie Wee et al.2006. Segment-Based Streaming Media Proxy Modeling and Optimization [J]. IEEE TRANSACTIONS ON MULTIMEDIA,2006,8(2):243-256.
    Chen Songqing, Bo Shen, Yong Yan and Xiaodong Zhang,2005. Fast Proxy Delivery of Multiple Streaming Sessions in Shared Running Buffers [J]. IEEE TRANSACTIONS ON MULTIMEDIA, VOL.7, NO.6, DECEMBER 2005, 1157-1169.
    Chen S, B. Shen, S. Wee, and X. Zhang.2003. Adaptive and lazy segmentation based proxy caching for streaming media delivery [J], Proceedings of ACM Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV), June 2003.
    Chan C, Su T, Huang S, Wang J.2002. Cooperative proxy scheme for large-scale VoD systems [C]:404-409.
    Chou C, L. Golubchik and J. Lui,2000. A performance study of dynamic replication techniques in continuous media servers [J], Proceedings of 8th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, pp.256-263.
    Chervenak A.L, D.A. Patterson, and R.H. Katz,1995. Choosing the best storage system for video service [J], Proceedings of the third ACM international conference on Multimedia, pp.109-119.
    Dan A and D. Sitaram.1996. A generalized interval caching policy for mixed interactive and long video workloads [J]. Proceedings of Multimedia Computing and Networking.
    Dan A and D. Sitaram,1995. An online video placement policy based on bandwidth to space ratio (BSR), ACM SIGMOD Record, vol.24, no.2, pp.376-385.
    Dukes J, Jeremy Group.2002. Dynamic repacking:a content replication policy for clustered multimedia servers [R]. TCD-CS-2002-36, Ireland:Trinity College Dublin Computer Science Department.
    Dukes J, J Jones.2003. Dynamic replication of content in the hammerhead multimedia server [R]. TCD-CS-2003-9, Ireland:Trinity College Dublin Computer Science Department.
    Davis, L.1991. Handbook of Genetic Algorithms, Van Nostrand Reinhold, New York, U.S.A.
    Doganata Y, A. Tantawi.1994. A Cost/Performance Study of Video Servers with Hierarchical Storage [C],1st International Conference on Multimedia Computing and Systems, Boston Mass,1994.
    Guo L.2005. DISC:Dynamic interleaved segment caching for interactive streaming. Proc. Of IEEE ICDCS[C].
    Guo L, Tan E, Chen S Q, et al.2008. The stretched exponential distribution of Internet media access patterns. PODC'08:Proceedings of the twenty-seventh ACM symposium on Principles of distributed computing, Toronto, Canada,2008 [C]. New York, NY, USA:ACM.
    Goyal P. et al.1994. A statistical admission control algorithm for multimedia servers [C]. ACM Multimedia 94, San Francisco. U SA.
    Guo Y, et al.2006. Dynamic cache reconfiguration strategies for cluster-based streaming proxy, Computer Communications, vol.29, no.10,1710-1721.
    Guo J, et al.,2008. Performance Analysis of Resource Selection Schemes for a Large Scale Video-on-Demand System, IEEE Transactions on Multimedia, vol.10, no.1, 153-159.
    Gu T, Ye B, Guo M, Chen D.2004. Implementing Cooperative Caching in Distributed Streaming Media Server Clusters [J]. Lecture notes in computer science:807-817.
    Hwang R, Sun Y.1998. Optimal video placement for hierarchical video-on-demand systems [J]. IEEE transactions on broadcasting,44(4):392-401.
    ISMA.2004. ISMA Implementation Specification Version 1.0+Corrigenda [S].
    ISMA.2005. ISMA Implementation Specification Version 2.0 [S].
    ISMA.2006. Internet Streaming Conformance Profiles [EB/OLJ].2006, http://www.isma.tv/conformance/iscpdocs.html.
    ISO.2003. ISO/IEC 14496-10:Information Technology——Coding of Audio Visual Objects Part 10:Advanced Video Coding[S].
    ISO.2004. ISO/IEC 14496-14:Information Technology——Coding of Audio Visual Objects Part 14:MP4 File Format[S]
    Jensen J.2005. Interactive television:new genres, new format, new content [C]. Proceedings of the second Australasian conference on Interactive entertainment. Creativity& Cognition Studios Press Sydney, Australia, Australia,89-96.
    KyungOh Lee, Heon Y. Yeom.1999. An effective admission control mechanism for variable-bit-rate video streams [J]. Multimedia Systems.7(4):305-311.
    Kwon B, Heon Y Yeom.2000. An admission control scheme for continuous media servers using caching [J]. Proceedings of 19th IEEE International Perfo rmance, Computing, and Communications Conference (IPCCC), February,2000, Phoenix, USA.
    Kasenna.2006. Observed On-Demand Usage Patterns and its Implications for Large-Scale VOD System Design[R]. Kasenna(Com.)
    Kim S, Tran K.2007. Supporting Scalable and Cooperative Interval Caching in a Clustered Video Server [C]. Sixth IEEE International Symposium on Network Computing and Applications,2007. NCA 2007.283-286.
    Lee J, Leung R.2002. Design and analysis of a fault-tolerant mechanism for a serverless video-on-demand system [C]. Parallel and Distributed Systems,2002. Proceedings. Ninth International Conference on.489-494.
    Liu Jiang-chuan.2005. Streaming Media Caching[M]. US:Springer US.
    Ma G, Wu C, Yang M.1999. Media server for storage and retrieval of voluminous multimedia data [M]. Google Patents. US Patent 5,926,649.
    Man K. F, K. S. Tang, and S. Kwong,1999. Genetic Algorithms:Concepts and Designs. London, U.K.:Springer-Verlag.
    Padhye J.1999. Continuous-Media Courseware Server:A Study of Client Interactions [J]. IEEE INTERNET COMPUTING 1999.3(2):65-73.
    Saito Y 2001.Functionally homogeneous clustering:a framework for building scalable data-intensive Internet services [D].
    Sen S, J. Rexford, and D. Towsley.1999. Proxy prefix caching for multimedia streams [J], In Proceedings of IEEE INFOCOM, March 1999.
    Serpanos D. N., L. Georgiadis, and T. Bouloutas.1998. MMPacking:A load and storage balancing algorithm for distributed multimedia servers [J], IEEE Transactions on Circuits and Systems for Video Technology, vol.8, no.1, pp. 13-17, February,1998.
    Tantaoui M.A., K.A. Hua, and S. Sheu,2002. A scalable technique for VCR-like interactions in video-on-demand applications, Distributed Computing Systems Workshops. Proceedings.22nd International Conference on,246-251.
    Tang K.S et al,2001. Optimal file placement in VOD system using genetic algorithm, Industrial Electronics, IEEE Transactions on, vol.48, no.5,891-897.
    Tsao, S.L., et al.1999. Data Allocation and Dynamic Load Balancing for Distributed Video Storage Server [J], Journal of Visual Communication and Image Representation, vol.10, no.2, pp.197-218.
    Veloso E, Ahneida V, Meira W, et al.2002. A hierarchical characterization of a live streaming media workload. IMW'02:Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurement, Marseille, France,2002 [C]. New York, NY, USA:ACM.
    Venkatasubramanian N and S. Ramanathan,1997. Load Management in Distributed Video Servers," Proceedings of the 17th International Conference on Distributed Computing Systems (ICDCS'97), pp.528-535.
    Wang H.J., Bhaskaran Raman, Rahul Biswas.2000 ICEBERG:An Internet-core Network Architecture for Integrated Communications [J]. IEEE Personal Communications, vol.7 (4), August 2000:10-19.
    Wang Y, Claypool M, Zuo Z.2001.An empirical study of real video performance across the Internet [C]. Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement. ACM New York, NY, USA,295-309.
    Weon and J. H. Je.2005. Stretched exponential degradation of oxide cathodes, Applied Surface Science, vol.251,2005, pp.59-63.
    Wu Kun-lung, Philip S. Yu, Joel L. Wolf.2001. Segment Based Proxy Caching of Multimedia Streams [A]. ACM WWW10. Hong Kong:2001,36-44.
    Wauters T, et al.,2006. Co-operative Proxy Caching Algorithms for Time-Shifted IPTV Services, IEEE Computer Society Washington,379-386.
    Wolf J. L., P.S. Yu, and H. Shachnai,1997. Disk load balancing for video-on-demand systems [J], ACM/Springer Multimedia Systems Journal, vol.5, no.6, pp. 358-370.
    Xiang W, Wu G., Ling Q and Wang L.2007. Piecewise Patching for Time-shifted TV Over HFC Networks [J], IEEE Transactions on Consumer Electronics, vol.53, no.3,2007, pp.891-897.
    Xiao Ming-zhong, Li Xiao-ming, Liu Han-yu et al.2004. Proxy Cache Replacement Strategies Based on Bytes Benefit of Streaming Media File[J]. Chinese Journal of Computer,2004,27(12):1633-1641.
    Yu H, Zheng D, Zhao B, Zheng W.2006. Understanding user behavior in large-scale video-on-demand systems [C]. Proceedings of the 1 st ACM SIGOPS/EuroSys European Conference on Computer Systems 2006. ACM New York, NY, USA, 333-344.
    Yu J et.al,2006. A dynamic caching algorithm based on internal popularity distribution of streaming media [J], Multimedia Systems, pp.135-149.
    Zhao Y, Z. Shi, and C.-C. J. Kuo,2002. Dynamic load balancing and content update for media storage servers [J], in Visual Information Processing XI, Proceedings of SPIE, vol.4736, pp.201-212.
    Zhuo J.C, G. Wu, J. Li and S. Xu,2008. Efficient Cache Placement Scheme for Clustered Time-shifted TV Servers [J], IEEE Transactions on Consumer Electronics, vol.54, no.4,1947-1955.
    Zhuo JuChao, Jun Li, Gang Wu, Li-Yue Zhu,2008. A Novel Data Replication and Placement Scheme for Time-shifted TV Cluster [C], IEEE Conference on Computer Science and Software Engineering, Dec,2008,3:241-245.
    Zhuo JuChao, Jun Li, Gang Wu,2009. Study of Cache Placement for Time-shifted TV Cluster Using Genetic Algorithm [C], GEC Summit,2009,781-786.
    Zhou X.and C. Xu,2007. Efficient algorithms of video replication and placement on a cluster of streaming servers, Journal of Network and Computer Applications, vol.30, no.1,551-540.
    Zhou Xiao-bo, Cheng-zhong Xu.2000. Optimal video replication and placement on a cluster of video-on-demand servers[A]. In:Proceedings of the 2002 International Conference on Parallel Processing, in Vancouver, British Columbia, Canada:547-555.
    Zhou X and C.Z. Xu,2002. Optimal video replication and placement on a cluster of video-on-demand servers [C], Parallel Processing,2002. Proceedings. International Conference on, pp.547-555.
    Zimmermann R, K. Fu, and W.S. Ku,2003. Design of a Large Scale Data Stream Recorder, Proceedings of the 5th International Conference on Enterprise Information Systems (ICEIS 2003), pp.23-26.
    Zipf G. K.,1949. Human Behavior and the Principle of Least Effort:An Introduction to Human Ecology. Cambridge, MA:Addison-Wesley.

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700