半动态集成选择分类方法
|
摘要
在集成学习领域,传统的动态集成选择需要为每一个样本选择子分类器组成集成分类器,这极大地增加了计算复杂度。针对这一问题,提出一种新的半动态集成选择方法。该方法分为两阶段,第一阶段为所有的测试样本选择最好的个体分类器组成一个集成分类器,第二阶段从剩余的个体分类器集合中为当前测试样本动态地选择子分类器组成一个集成分类器。最终的分类结果通过融合两阶段得到集成分类器的结果得到。通过对UCI数据测试的结果表明,该算法不仅能取得较好的分类性能,而且能极大地降低计算复杂度。
Traditional Dynamic Ensemble Selection( DES) in ensemble learning needs to select individual classifiers for all the test samples. However,it leads to highly computational cost. Due to this issue,a new Semi Dynamic Ensemble Selection( SemiDES) strategy is proposed in this paper,which consists of two stages. Individual classifiers are selected for all the test samples in the first stage. In the second stage,the classifiers for each test sample are selected dynamically. The final result is obtained by integrating the output of the two stages. The experimental results on UCI data set demonstrate the proposed method can obtain a better classification performance. Moreover,Semi-DES can reduce the computational cost greatly.
Traditional Dynamic Ensemble Selection( DES) in ensemble learning needs to select individual classifiers for all the test samples. However,it leads to highly computational cost. Due to this issue,a new Semi Dynamic Ensemble Selection( SemiDES) strategy is proposed in this paper,which consists of two stages. Individual classifiers are selected for all the test samples in the first stage. In the second stage,the classifiers for each test sample are selected dynamically. The final result is obtained by integrating the output of the two stages. The experimental results on UCI data set demonstrate the proposed method can obtain a better classification performance. Moreover,Semi-DES can reduce the computational cost greatly.
引文
[1]Marques A I,Garcia V,Sanchez J S.Two-level classifier ensembles for credit risk assessment[J].Expert Systems with Applications,2012,39(12):10244-10250.
[2]Michelangelo Paci,Loris Nanni,Stefano Severi.An ensemble of classifier based on different texture descriptors for texture classification[J].Journal of King Saud UniversityScience,2013,25(3):235-244.
[3]Moumita Roy,Susmita Ghosh,Ashish Ghosh.A novel approach for change detection of remotely sensed images using semi-supervised multiple classifier system[J].Information Science,2014,269(10):35-47.
[4]慕昱,夏虹,刘永阔.基于集成学习的核电站故障诊断方法[J].原子能科学技术,2012,46(10):1254-1258.
[5]刘培,杜培军,谭琨.一种基于集成学习和特征融合的遥感影像分类新方法[J].红外与毫米波学报,2014,33(3):311-317.
[6]Mordelet F,Vert J P.A bagging SVM to learn from positive and unlabeled examples[J].Pattern Recognition Letters,2014,37(1),201-209.
[7]李明方,张化祥.针对不平衡数据集的Bagging改进算法[J].计算机工程与应用,2010,46(30):40-42.
[8]Nicolas Garcia-Pedrajas,Aida de Haro-Garcia.Boosting instance selection algorithm[J].Knowledge-based Systems,2014,67(9):342-360.
[9]于玲,吴铁军.集成学习:Boosting算法综述[J].模式识别与人工智能,2004,17(1):52-59.
[10]Le Zhang,Ponnuthrai Nagaratnam Suganthan.Random Forest with ensemble of feature spaces[J].Pattern Recognition,2014,47(10):3429-3437.
[11]杨明,王飞.一种基于局部随机子空间的分类集成算法[J].模式识别与人工智能,2012,25(4):595-603.
[12]Zhou Zhihua,Wu Jianxin,Tang Wei.Ensembling neural networks:Many could be better than all[J].Artificial Intelligence,2002,137(1-2):239-263.
[13]张春霞,张讲社.选择性集成学习算法综述[J].计算机学报,2011,34(8):1399-1410.
[14]Zhang Li,Zhou Weida.Sparse ensembles using weighted combination methods based on linear programming[J].Pattern Recognition,2011,44(1):97-106.
[15]Margineantu D D,Dietterich T G.Pruning adaptive boosting[C]//Proceedings of the Fourteenth International Conference on Machine Learning.1997:211-218.
[16]Partalas I,Tsoumakas G,Vlahavas I.Pruning an ensemble of classifiers via reinforcement learning[J].Neurocomputing,2009,72(7-9):1900-1909.
[17]Albert H R Koa,Robert Sabourina,Alceu Souza Britto Jr.From dynamic classifier selection to dynamic ensemble selection[J].Pattern Recognition,2008,41(5):1718-1731.
[18]De Souto M C P,Soares R G F,Santana A,et al.Empirical comparison of dynamic classifier selection methods based on diversity and accuracy for building ensembles[C]//Proceedings of the International Joint Conference on Neural Networks.2008:1480-1487.
[19]Tomasz Woloszynski,Marek Kurzynski,Pawel Podsiadlo,et al.A measure of competence based on random classification for dynamic ensemble selection[J].Information Fusion,2012,13(3):207-213.
[20]Newman D J,Hettich S,Blake C L,et al.UCI Repository of Machine Learning Databases[EB/OL].http://www.ics.uci.edu/~mlearn/MLRepository.html,2014-11-21.
[2]Michelangelo Paci,Loris Nanni,Stefano Severi.An ensemble of classifier based on different texture descriptors for texture classification[J].Journal of King Saud UniversityScience,2013,25(3):235-244.
[3]Moumita Roy,Susmita Ghosh,Ashish Ghosh.A novel approach for change detection of remotely sensed images using semi-supervised multiple classifier system[J].Information Science,2014,269(10):35-47.
[4]慕昱,夏虹,刘永阔.基于集成学习的核电站故障诊断方法[J].原子能科学技术,2012,46(10):1254-1258.
[5]刘培,杜培军,谭琨.一种基于集成学习和特征融合的遥感影像分类新方法[J].红外与毫米波学报,2014,33(3):311-317.
[6]Mordelet F,Vert J P.A bagging SVM to learn from positive and unlabeled examples[J].Pattern Recognition Letters,2014,37(1),201-209.
[7]李明方,张化祥.针对不平衡数据集的Bagging改进算法[J].计算机工程与应用,2010,46(30):40-42.
[8]Nicolas Garcia-Pedrajas,Aida de Haro-Garcia.Boosting instance selection algorithm[J].Knowledge-based Systems,2014,67(9):342-360.
[9]于玲,吴铁军.集成学习:Boosting算法综述[J].模式识别与人工智能,2004,17(1):52-59.
[10]Le Zhang,Ponnuthrai Nagaratnam Suganthan.Random Forest with ensemble of feature spaces[J].Pattern Recognition,2014,47(10):3429-3437.
[11]杨明,王飞.一种基于局部随机子空间的分类集成算法[J].模式识别与人工智能,2012,25(4):595-603.
[12]Zhou Zhihua,Wu Jianxin,Tang Wei.Ensembling neural networks:Many could be better than all[J].Artificial Intelligence,2002,137(1-2):239-263.
[13]张春霞,张讲社.选择性集成学习算法综述[J].计算机学报,2011,34(8):1399-1410.
[14]Zhang Li,Zhou Weida.Sparse ensembles using weighted combination methods based on linear programming[J].Pattern Recognition,2011,44(1):97-106.
[15]Margineantu D D,Dietterich T G.Pruning adaptive boosting[C]//Proceedings of the Fourteenth International Conference on Machine Learning.1997:211-218.
[16]Partalas I,Tsoumakas G,Vlahavas I.Pruning an ensemble of classifiers via reinforcement learning[J].Neurocomputing,2009,72(7-9):1900-1909.
[17]Albert H R Koa,Robert Sabourina,Alceu Souza Britto Jr.From dynamic classifier selection to dynamic ensemble selection[J].Pattern Recognition,2008,41(5):1718-1731.
[18]De Souto M C P,Soares R G F,Santana A,et al.Empirical comparison of dynamic classifier selection methods based on diversity and accuracy for building ensembles[C]//Proceedings of the International Joint Conference on Neural Networks.2008:1480-1487.
[19]Tomasz Woloszynski,Marek Kurzynski,Pawel Podsiadlo,et al.A measure of competence based on random classification for dynamic ensemble selection[J].Information Fusion,2012,13(3):207-213.
[20]Newman D J,Hettich S,Blake C L,et al.UCI Repository of Machine Learning Databases[EB/OL].http://www.ics.uci.edu/~mlearn/MLRepository.html,2014-11-21.