基于Bloom filter降低云数据库网络延时的影响
|
摘要
为了改善服务器端系统登录模块运行环境,采用云数据库解决方案,但该方式可能引发额外的网络延时消耗,导致系统性能下降。基于Bloom filter算法设计过滤器,提前判定数据是否在数据库中,能够减少数据库读取次数,进而降低网络延时带来的额外性能损耗。结合Redis良好的分布式性能及持久化方案对Bloom filter进行管理。实验结果表明,当查询非命中率达到0.5%时,可以有效降低系统整体网络延时及响应延时。得出结论:采用基于Bloom filter的过滤器对数据是否在数据库中进行判定,能够降低网络延时带来的影响,从而提高系统整体响应性能。
To reduce the performance loss of the login module caused by the network delay running on cloud database.,we use the filter which base on Bloom filter to judge if each data have been load in database or not to reduce the number of database read so that the performance loss caused by the network delay can be reduced.Bloom filter is managed by Redis distributed version which can promise high availability.Experiment results show that when the total query proportion of missing Sql query is more than 0.5%in the database base,the overall system network delay can be effectively reduced by using Bloom filter and the overall system response delay can be reduced.Thus we can get a conclusion that the filter based on Bloom filter can reduce the impact of network delay and improve the overall system response.
To reduce the performance loss of the login module caused by the network delay running on cloud database.,we use the filter which base on Bloom filter to judge if each data have been load in database or not to reduce the number of database read so that the performance loss caused by the network delay can be reduced.Bloom filter is managed by Redis distributed version which can promise high availability.Experiment results show that when the total query proportion of missing Sql query is more than 0.5%in the database base,the overall system network delay can be effectively reduced by using Bloom filter and the overall system response delay can be reduced.Thus we can get a conclusion that the filter based on Bloom filter can reduce the impact of network delay and improve the overall system response.
引文
[1]林子雨,赖永炫,林琛,等.云数据库研究[J].软件学报,2012,34(5):1148-1166.
[2]CORMODE G.Count-min sketch[J].Encyclopedia of Algorithms,2009,29(1):64-69.
[3]FAN B,KAMINSKY M,ANDERSEN D G.Cuckoo filter:better than bloom[J].The magazine of USENIX&SAGE,2013,38:36-40.
[4]吕健波,戴冠中,慕德俊.绝对延迟保证在Web应用服务器数据库连接池中的实现[J].计算机应用研究,2012,29(5):1838-1841.
[5]刘元珍.Bloom filter及其在网络中的应用综述[J].计算机应用与软件,2013(9):219-220.
[6]徐爱萍,王波,张煦.基于Hbase的时空大数据关联查询优化[J].计算机应用与软件,2017,34(6):37-42.
[7]罗军,陈席林,李文生.高效Key-Value持久化缓存系统的实现[J].计算机工程,2014,40(3):33-38.
[8]BHUSHAN M,BANERJEA S,YADAV S K.Bloom filter based optimization on Hbase with MapReduce[C].International Conference on Data Mining and Intelligent Computing.IEEE,2014:1-5.
[9]李文昊.基于确定性执行策略的分布式数据库中间件的设计与实现[D].太原:太原理工大学,2016.
[10]王韧,朱金连,周亮,等.中间件技术在移动应用数据库开发中的运用[J].电子设计工程,2015(2):170-172.
[11]MOSHARRAF N,JAYASUMANA A P,RAY I.Compacted bloom filter[C].International Conference on Collaboration and Internet Computing.IEEE,2017:304-311.
[12]冯锋,吴杰.基于Bloom filter的RFID中间件数据过滤算法研究[J].计算机应用研究,2015(5):1441-1444.
[13]张进,邬江兴,刘勤让.4种计数型Bloom filter的性能分析与比较[J].软件学报,2010,21(5):1098-1114.
[14]MOSHARRAF N,JAYASUMANA A P,RAY I.Compacted bloom filter[C].IEEE,International Conference on Collaboration and Internet Computing.IEEE,2017:304-311.
[2]CORMODE G.Count-min sketch[J].Encyclopedia of Algorithms,2009,29(1):64-69.
[3]FAN B,KAMINSKY M,ANDERSEN D G.Cuckoo filter:better than bloom[J].The magazine of USENIX&SAGE,2013,38:36-40.
[4]吕健波,戴冠中,慕德俊.绝对延迟保证在Web应用服务器数据库连接池中的实现[J].计算机应用研究,2012,29(5):1838-1841.
[5]刘元珍.Bloom filter及其在网络中的应用综述[J].计算机应用与软件,2013(9):219-220.
[6]徐爱萍,王波,张煦.基于Hbase的时空大数据关联查询优化[J].计算机应用与软件,2017,34(6):37-42.
[7]罗军,陈席林,李文生.高效Key-Value持久化缓存系统的实现[J].计算机工程,2014,40(3):33-38.
[8]BHUSHAN M,BANERJEA S,YADAV S K.Bloom filter based optimization on Hbase with MapReduce[C].International Conference on Data Mining and Intelligent Computing.IEEE,2014:1-5.
[9]李文昊.基于确定性执行策略的分布式数据库中间件的设计与实现[D].太原:太原理工大学,2016.
[10]王韧,朱金连,周亮,等.中间件技术在移动应用数据库开发中的运用[J].电子设计工程,2015(2):170-172.
[11]MOSHARRAF N,JAYASUMANA A P,RAY I.Compacted bloom filter[C].International Conference on Collaboration and Internet Computing.IEEE,2017:304-311.
[12]冯锋,吴杰.基于Bloom filter的RFID中间件数据过滤算法研究[J].计算机应用研究,2015(5):1441-1444.
[13]张进,邬江兴,刘勤让.4种计数型Bloom filter的性能分析与比较[J].软件学报,2010,21(5):1098-1114.
[14]MOSHARRAF N,JAYASUMANA A P,RAY I.Compacted bloom filter[C].IEEE,International Conference on Collaboration and Internet Computing.IEEE,2017:304-311.