Noise peeling methods to improve boosting algorithms

详细信息    查看全文

• 作者：Waldyn Martinez^a ; ^{martinwg@miamioh.edu" class="auth_mail" title="E-mail the corresponding author} ; J. Brian Gray^b ; ^{bgray@cba.ua.edu" class="auth_mail" title="E-mail the corresponding author}
• 关键词：AdaBoost ; Ensembles ; Generalization error ; Noise detection ; Noise filtering ; Outliers
• 刊名：Computational Statistics & Data Analysis
• 出版年：2016
• 出版时间：January 2016
• 年：2016
• 卷：93
• 期：Complete
• 页码：483-497
• 全文大小：1077 K

文摘

Boosting refers to a family of methods that combine sequences of individual classifiers into highly accurate ensemble models through weighted voting. AdaBoost, short for “Adaptive Boosting”, is the most well-known boosting algorithm. AdaBoost has many strengths. Among them, there is sufficient empirical evidence pointing to its performance being generally superior to that of individual classifiers. In addition, even when combining a large number of weak learners, AdaBoost can be very robust to overfitting usually with lower generalization error than other competing ensemble methodologies, such as bagging and random forests. However, AdaBoost, as most hard margin classifiers, tends to be sensitive to outliers and noisy data, since it assigns observations that have been misclassified a higher weight in subsequent iterations. It has recently been proven that for any booster with a potential convex loss function, and any nonzero random classification noise rate, there is a data set, which can be efficiently learnable by the booster if there is no noise, but cannot be learned with accuracy better than 1/2 with random classification noise present. Several techniques to identify and potentially delete (peel) noisy samples in binary classification are proposed in order to improve the performance of AdaBoost. It is found that peeling methods generally perform better than AdaBoost and other noise resistant boosters, especially when high levels of noise are present in the data.