Energy-aware processing of big data in homogeneous cluster

详细信息    查看全文

• 作者：Youwei Ding ; Xiaolin Qin ; Qian Zhou ; Liang Liu…
• 关键词：Energy saving ; Big data ; Homogeneous cluster ; Data placement strategy ; Workload balance
• 刊名：Signal, Image and Video Processing
• 出版年：2017
• 出版时间：February 2017
• 年：2017
• 卷：11
• 期：2
• 页码：371-379
• 全文大小：
• 刊物类别：Engineering
• 刊物主题：Signal,Image and Speech Processing; Image Processing and Computer Vision; Computer Imaging, Vision, Pattern Recognition and Graphics; Multimedia Information Systems;
• 出版者：Springer London
• ISSN：1863-1711
• 卷排序：11

文摘

The size of data centers is becoming larger to deal with the exponential data growth, and the energy consumption challenges the services providers and the environment. Various data placement strategies were developed to reduce the energy consumption of processing big data on the level of storage system, but they were typically developed for specific applications and storage medium. This paper proposes an energy-aware algorithm EABD of processing big data in homogeneous cluster with general data storage. We show that a variation of this optimization can be reduced to set cover problem, and a heuristic algorithm is proposed to reduce the energy consumption by selecting proper nodes and assigning balanced workload to each selected node. This algorithm will not be influenced by the data placement strategies and storage medium. Simulation results show that our algorithm significantly reduces energy consumption in different situations.