Optimizing R with SparkR on a commodity cluster for biomedical research
详细信息 查看全文
- 作者：Martin Sedlmayra ; martin.sedlmayr@fau.de" class="auth_mail" title="E-mail the corresponding author ; Tobias Wü ; rfla ; Christian Maiera ; Lothar Hä ; berleb ; Peter Faschingb ; Hans-Ulrich Prokoscha ; Jan Christopha
- 关键词：Computing methodologies ; Genome-wide association study ; Big data ; Cluster computing ; SparkR
- 刊名：Computer Methods and Programs in Biomedicine
- 出版年：2016
- 出版时间：December 2016
- 年：2016
- 卷：137
- 期：Complete
- 页码：321-328
- 全文大小：513 K
文摘
- •
R is a popular environment for clinical data analysis. It does not directly support big data workloads.
- •
Both, the Message Passing Interface (MPI) and SparkR allow to parallelize computational demanding workloads on clusters.
- •
SparkR offers elastic resources even on non-dedicated hardware and tight integration with Hadoop distributed services.
- •
SparkR requires minimal changes to original code in R in order to utilize parallel execution.
- •
Computation in SparkR scales better than with the Message Passing Interface (MPI) due to optimized data communication.