Active learning: an empirical study of common baselines

详细信息    查看全文

• 作者：Maria E. Ramirez-Loaiza ; Manali Sharma ; Geet Kumar…
• 关键词：Active learning ; Query by committee ; Uncertainty sampling ; Empirical evaluation
• 刊名：Data Mining and Knowledge Discovery
• 出版年：2017
• 出版时间：March 2017
• 年：2017
• 卷：31
• 期：2
• 页码：287-313
• 全文大小：
• 刊物类别：Computer Science
• 刊物主题：Data Mining and Knowledge Discovery; Artificial Intelligence (incl. Robotics); Information Storage and Retrieval; Statistics for Engineering, Physics, Computer Science, Chemistry and Earth Sciences;
• 出版者：Springer US
• ISSN：1573-756X
• 卷排序：31

文摘

Most of the empirical evaluations of active learning approaches in the literature have focused on a single classifier and a single performance measure. We present an extensive empirical evaluation of common active learning baselines using two probabilistic classifiers and several performance measures on a number of large datasets. In addition to providing important practical advice, our findings highlight the importance of overlooked choices in active learning experiments in the literature. For example, one of our findings shows that model selection is as important as devising an active learning approach, and choosing one classifier and one performance measure can often lead to unexpected and unwarranted conclusions. Active learning should generally improve the model’s capability to distinguish between instances of different classes, but our findings show that the improvements provided by active learning for one performance measure often came at the expense of another measure. We present several such results, raise questions, guide users and researchers to better alternatives, caution against unforeseen side effects of active learning, and suggest future research directions.