Parallel meta-blocking for scaling entity resolution over big heterogeneous data

详细信息    查看全文

• 作者：Vasilis Efthymiou^a ; ^{vefthym@ics.forth.gr} ; George Papadakis^b ; ^{gpapadis@di.uoa.gr} ; George Papastefanatos^c ; ^{gpapas@imis.athena-innovation.gr} ; Kostas Stefanidis^d ; ^{kostas.stefanidis@uta.fi} ; Themis Palpanas^e ; ^{themis@mi.parisdescartes.fr}
• 关键词：Meta-blocking ; Map/Reduce model ; Parallelization
• 刊名：Information Systems
• 出版年：2017
• 出版时间：April 2017
• 年：2017
• 卷：65
• 期：Complete
• 页码：137-157
• 全文大小：7968 K
• 卷排序：65

文摘

We adapt Meta-blocking to the MapReduce paradigm through 3 alternative parallelization strategies: an edge-based strategy that explicitly builds the blocking graph, a comparison-based strategy that uses the blocking graph implicitly, as a conceptual model, and an entity-based strategy that is independent of the blocking graph. We also provide concrete implementations for all weighting schemes that are used in Meta-blocking. We present a load balancing technique that deals with skewness in the input block collection, splitting it into partitions of the same computational cost. We verify the scalability of our techniques through a thorough experimental evaluation over the four largest, real datasets that have been applied to Meta-blocking. The data and the implementation of our techniques are publicly available.