Performance evaluation of cloud-based log file analysis with Apache Hadoop and Apache Spark

详细信息    查看全文

• 作者：Ilias Mavridis ; ^{imavridis@csd.auth.gr} ; Helen Karatza ^{karatza@csd.auth.gr}
• 关键词：Log Analysis ; Cloud ; Apache Hadoop ; Apache Spark ; Performance Evaluation
• 刊名：Journal of Systems and Software
• 出版年：2017
• 出版时间：March 2017
• 年：2017
• 卷：125
• 期：Complete
• 页码：133-151
• 全文大小：4087 K
• 卷排序：125

文摘

We focus on three performance indicators, the execution time, resource utilization and scalability. We conducted realistic log file analysis experiments in both frameworks. We proposed a power consumption model and an utilization-based cost estimation. We experimentally confirmed Spark’s best performance.