P2P-MapReduce: Parallel data processing in dynamic Cloud environments

详细信息	查看全文 | 推荐本文 |

• 作者：Fabrizio Marozzo^a ; ^{fmarozzo@deis.unical.it} ; Domenico Talia^a ; ^b ; ^{talia@deis.unical.it} ; Paolo Trunfio^a ; ^{trunfio@deis.unical.it}
• 关键词：MapReduce ; Parallel processing ; Cloud computing ; Peer-to-peer computing ; P2P-MapReduce
• 刊名：Journal of Computer and System Sciences
• 出版年：2012
• 期刊代码：186_00220000
• 类别：et
• 出版时间：September, 2012
• 卷：78
• 期：5
• 页码：1382-1402
• 文件大小：986 K

摘要

MapReduce is a programming model for parallel data processing widely used in Cloud computing environments. Current MapReduce implementations are based on centralized master-slave architectures that do not cope well with dynamic Cloud infrastructures, like a Cloud of clouds, in which nodes may join and leave the network at high rates. We have designed an adaptive MapReduce framework, called P2P-MapReduce, which exploits a peer-to-peer model to manage node churn, master failures, and job recovery in a decentralized but effective way, so as to provide a more reliable MapReduce middleware that can be effectively exploited in dynamic Cloud infrastructures. This paper describes the P2P-MapReduce system providing a detailed description of its basic mechanisms, a prototype implementation, and an extensive performance evaluation in different network scenarios. The performance results confirm the good fault tolerance level provided by the P2P-MapReduce framework compared to a centralized implementation of MapReduce, as well as its limited impact in terms of network overhead.