Risk upper bounds for general ensemble methods with an application to multiclass classification

详细信息    查看全文

• 作者：Franç ; ois Laviolette^a ; ^{francois.laviolette@ift.ulaval.ca}Author Vitae ; Emilie Morvant^b ; ^{emilie.morvant@univ-st-etienne.fr}Author Vitae ; Liva Ralaivola^c ; ^{liva.ralaivola@lif.univ-mrs.fr}Author Vitae ; Jean-Francis Roy^a ; ^d ; ^{jfroy@coveo.com}Author Vitae
• 关键词：Majority vote ; Ensemble methods ; PAC-Bayesian Theory ; Multiclass classification ; Multilabel Prediction
• 刊名：Neurocomputing
• 出版年：2017
• 出版时间：5 January 2017
• 年：2017
• 卷：219
• 期：Complete
• 页码：15-25
• 全文大小：1104 K
• 卷排序：219

文摘

This paper generalizes a pivotal result from the PAC-Bayesian literature —the C-boundC-bound— primarily designed for binary classification to the general case of ensemble methods of voters with arbitrary outputs. We provide a generic version of the C-boundC-bound, an upper bound over the risk of models expressed as a weighted majority vote that is based on the first and second statistical moments of the vote's margin. On the one hand, this bound may advantageously be applied on more complex outputs than mere binary outputs, such as multiclass labels and multilabel, and on the other hand, it allows us to consider margin relaxations. We provide a specialization of the bound to multiclass classification together with empirical evidence that the presented theoretical result is tightly bound to the risk of the majority vote classifier. We also give insights as to how the proposed bound may be of use to characterize the risk of multilabel predictors.