面向大数据应用的自适应带宽分配策略
|
摘要
提出自适应带宽分配策略((Flexible Bandwidth Allocation for Big Data Transfer,FBA-BDT)。FBA-BDT给传输大数据的请求提供动态带宽分配,在满足数据有效期的条件下,最大化数据传输率。首先,构建优化规划目标函数,然后再用启发式算法求解目标函数,进而优化带宽的分配。实验数据表明,与最小带宽分配相比,提出的FBA-BDT算法的数据传输请求拒绝率下降40%、一天内传输的数据量提高至21 TB。
In this paper,we proposed a Flexible Bandwidth Allocation for Big Data Transfer that flexibly and adaptively allocates bandwidth to big data transfer requests with an objective to maximize the acceptance ratio of the requests while satisfying the deadline constraints.We first developed an optimization programming formulation and then proposed a heuristic algorithm to solve the problem.The proposed algorithm outperformed Minimum Bandwidth Allocation algorithms(Min BA) by reducing the rejection ratio by at least 40% and increasing the data transferred by at least 21 TB in a day.
In this paper,we proposed a Flexible Bandwidth Allocation for Big Data Transfer that flexibly and adaptively allocates bandwidth to big data transfer requests with an objective to maximize the acceptance ratio of the requests while satisfying the deadline constraints.We first developed an optimization programming formulation and then proposed a heuristic algorithm to solve the problem.The proposed algorithm outperformed Minimum Bandwidth Allocation algorithms(Min BA) by reducing the rejection ratio by at least 40% and increasing the data transferred by at least 21 TB in a day.
引文
[1]宋杰,孙宗哲,毛克明.MapReduce大数据处理平台与算法研究进展[J].软件学报,2017,28(3):514-543.
[2]严伟中.关于大数据下动态数据安全传输仿真[J].计算机仿真,2018,35(2):153-157.
[2]ZHENG K,YANG Z,ZHANG K,et al.Big Data-driven Optimization for Mobile Networks Toward 5G[J].IEEE Network,2016,30(1):44-51.
[3]WANG K,SHAO Y,SHU L,et al.Mobile Big Data Faulttolerant Processing for e Health Networks[J].IEEE Network,2016,30(1):36-42.
[4]FELDMAN D,SCHMIDT M,SOHLER C.Turning Big Data into Tiny Data:Constant-size Coresets for k-means,PCA and Projective Clustering[C]//SODA13,Louisiana,USA,2013:1434-1453.
[5]CHARBONNEAU N,VOKKARANE V M,GUOK C,MON-GA I.Advance Reservation Frameworks in Hybrid IP-WDMNetworks[J].IEEE Communications Magazine,2011,49(5):36-42.
[6]YASSINE A,SHIREHJINI A A N,SHIRMOHAMMADI S.Bandwidth On-demand for Multimedia Big Data Transfer across Geo-Distributed Cloud Data Centers[J].IEEETrans.Cloud Comput.,2016,5(7):56-63.
[7]ZUO L,ZHU M M.Concurrent Bandwidth Reservation Strategies for Big Data Transfers in High-Performance Networks[J].IEEE Trans.Netw.Service Manag.,2015,12(2):232-247.
[8]TUDORAN R,COSTAN A,ANTONIU G.Over Flow:MultiSite Aware Big Data Management for Scientific Workflows on Clouds[J].IEEE Trans.Cloud Comput.,2016,4(1):76-89.
[9]XIA Q,XU Z,LIANG W,ZOMAYA A Y.Collaboration-and Fairness-aware Big Data Management in Distributed Clouds[J].IEEE Trans.Parallel Distrib.Syst.,2016,27(7):1941-1953.
[10]LI D,ZHU J,WU J,GUAN J,ZHANG Y.Guaranteeing Hetero-geneous Bandwidth Demand in Multitenant Data Center Networks[J].IEEE/ACM Trans.Netw.,2015,23(5):1648-1660.
[11]LI D,CHEN C,GUAN J,ZHANG Y,ZHU J,YU R.DCloud:Deadline-aware Resource Allocation for Cloud Computing Jobs[J].IEEE Trans.Parallel Distrib.Syst.,2016,27(8):2248-2260.
[12]GUO J,LIU F,LIU J,JIN H.Fair Network Bandwidth Allocation in Iaa S Datacenters via a Cooperative Game Approach[J].IEEE/ACM Trans.Netw.,2016,24(2):873-886.
[13]DIVAKARAN D M,GURUSAMY M.Towards Flexible Guarantees in Clouds:Adaptive Bandwidth Allocation and Pricing[J].IEEE Trans.Parallel Distrib.Syst.,2015,26(6):1754-1764.
[14]MADANANTHA S,TRUONG-HUU T,GURUSAMY M.A-daptive Bandwidth Allocation for Virtual Network Embedding in Optical Data Center Networks[J].IEEE LCN,2016:316-324.
[2]严伟中.关于大数据下动态数据安全传输仿真[J].计算机仿真,2018,35(2):153-157.
[2]ZHENG K,YANG Z,ZHANG K,et al.Big Data-driven Optimization for Mobile Networks Toward 5G[J].IEEE Network,2016,30(1):44-51.
[3]WANG K,SHAO Y,SHU L,et al.Mobile Big Data Faulttolerant Processing for e Health Networks[J].IEEE Network,2016,30(1):36-42.
[4]FELDMAN D,SCHMIDT M,SOHLER C.Turning Big Data into Tiny Data:Constant-size Coresets for k-means,PCA and Projective Clustering[C]//SODA13,Louisiana,USA,2013:1434-1453.
[5]CHARBONNEAU N,VOKKARANE V M,GUOK C,MON-GA I.Advance Reservation Frameworks in Hybrid IP-WDMNetworks[J].IEEE Communications Magazine,2011,49(5):36-42.
[6]YASSINE A,SHIREHJINI A A N,SHIRMOHAMMADI S.Bandwidth On-demand for Multimedia Big Data Transfer across Geo-Distributed Cloud Data Centers[J].IEEETrans.Cloud Comput.,2016,5(7):56-63.
[7]ZUO L,ZHU M M.Concurrent Bandwidth Reservation Strategies for Big Data Transfers in High-Performance Networks[J].IEEE Trans.Netw.Service Manag.,2015,12(2):232-247.
[8]TUDORAN R,COSTAN A,ANTONIU G.Over Flow:MultiSite Aware Big Data Management for Scientific Workflows on Clouds[J].IEEE Trans.Cloud Comput.,2016,4(1):76-89.
[9]XIA Q,XU Z,LIANG W,ZOMAYA A Y.Collaboration-and Fairness-aware Big Data Management in Distributed Clouds[J].IEEE Trans.Parallel Distrib.Syst.,2016,27(7):1941-1953.
[10]LI D,ZHU J,WU J,GUAN J,ZHANG Y.Guaranteeing Hetero-geneous Bandwidth Demand in Multitenant Data Center Networks[J].IEEE/ACM Trans.Netw.,2015,23(5):1648-1660.
[11]LI D,CHEN C,GUAN J,ZHANG Y,ZHU J,YU R.DCloud:Deadline-aware Resource Allocation for Cloud Computing Jobs[J].IEEE Trans.Parallel Distrib.Syst.,2016,27(8):2248-2260.
[12]GUO J,LIU F,LIU J,JIN H.Fair Network Bandwidth Allocation in Iaa S Datacenters via a Cooperative Game Approach[J].IEEE/ACM Trans.Netw.,2016,24(2):873-886.
[13]DIVAKARAN D M,GURUSAMY M.Towards Flexible Guarantees in Clouds:Adaptive Bandwidth Allocation and Pricing[J].IEEE Trans.Parallel Distrib.Syst.,2015,26(6):1754-1764.
[14]MADANANTHA S,TRUONG-HUU T,GURUSAMY M.A-daptive Bandwidth Allocation for Virtual Network Embedding in Optical Data Center Networks[J].IEEE LCN,2016:316-324.