Duplicate Detection for Bayesian Network Structure Learning

详细信息    查看全文

• 作者：Niklas Jahnsson ; Brandon Malone ; Petri Myllymäki
• 关键词：Bayesian networks ; Structure learning ; State space search ; Delayed duplicate detection ; Structured duplicate detection
• 刊名：New Generation Computing
• 出版年：2017
• 出版时间：January 2017
• 年：2017
• 卷：35
• 期：1
• 页码：47-67
• 全文大小：
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence (incl. Robotics); Computer Hardware; Computer Systems Organization and Communication Networks; Software Engineering/Programming and Operating Systems; Computing Methodologies;
• 出版者：Ohmsha
• ISSN：1882-7055
• 卷排序：35

文摘

We address the well-known score-based Bayesian network structure learning problem. Breadth-first branch and bound (BFBnB) has been shown to be an effective approach for solving this problem. Duplicate detection is an important component of the BFBnB algorithm. Previously, an external sorting-based technique was used for delayed duplicate detection (DDD). We propose a hashing-based technique for DDD and a bin packing algorithm for minimizing the number of external memory files and operations. We also give a structured duplicate detection approach which completely eliminates DDD. Importantly, these techniques ensure the search algorithms respect any given memory bound. Empirically, we demonstrate that structured duplicate detection is significantly faster than the previous state of the art in limited-memory settings. Our results show that the bin packing algorithm incurs some overhead, but that the overhead is offset by reducing I/O when more memory is available.