基于聚类与排序修剪的分类器集成方法
|
摘要
为了提高分类器集成性能,提出了一种基于聚类算法与排序修剪结合的分类器集成方法。将混淆矩阵作为量化基分类器间差异度的工具,通过聚类将分类器划分为若干子集,提出一种排序修剪算法,以距离聚类中心最近的分类器为起点,根据分类器的距离对差异度矩阵动态加权,以加权差异度作为排序标准对子集中的分类器进行按比例修剪,最后使用投票法对选出的基分类器进行集成。与多种集成方法在UCI数据库中的10组数据集上进行对比与分析,实验结果表明,基于聚类与排序修剪的分类器选择方法有效提升了集成系统的分类能力。
In order to enhance the performance of classifier ensemble,this paper proposed a selective ensemble method based on clustering and sorting pruning. Firstly,it used confusion matrix as a tool to quantify the difference between the base classifiers,and divided the classifiers into several subsets by clustering. Then it proposed the pruning algorithm,took the nearest cluster center classifier as a starting point,which gave the weights of the diversity matrix dynamically according to the distance of classifier. It scaled the subsets by weighted diversity which was the sorting standard. Finally,it used voting method to integrate the selective classifiers. Moreover this paper performed comparative experiments and analysis on 10 datasets from UCI database with other ensemble methods. The experimental results show that this approach can effectively improve the classification ability of the ensemble system.
In order to enhance the performance of classifier ensemble,this paper proposed a selective ensemble method based on clustering and sorting pruning. Firstly,it used confusion matrix as a tool to quantify the difference between the base classifiers,and divided the classifiers into several subsets by clustering. Then it proposed the pruning algorithm,took the nearest cluster center classifier as a starting point,which gave the weights of the diversity matrix dynamically according to the distance of classifier. It scaled the subsets by weighted diversity which was the sorting standard. Finally,it used voting method to integrate the selective classifiers. Moreover this paper performed comparative experiments and analysis on 10 datasets from UCI database with other ensemble methods. The experimental results show that this approach can effectively improve the classification ability of the ensemble system.
引文
[1]谢元澄,杨静宇.删除最差基学习器来层次修剪Bagging集成[J].计算机研究与发展,2009,46(2):261-266.
[2]毕凯,王晓丹,姚旭,等.一种基于Bagging和混淆矩阵的自适应选择性集成[J].电子学报,2014,42(4):711-716.
[3]Li Nan,Yu Yang,Zhou Zhihua.Diversity regularized ensemble pruning[C]//Proc of Joint European Conference on Machine Learning and Knowledge Discovery in Databases.Berlin:Springer-Verlag,2012:330-345.
[4]Giacinto G,Roli F,Fumera G.Design of effective multiple classifier systems by clustering of classifiers[C]//Proc of the 15th International Conference on Pattern Recognition.Piscataway,NJ:IEEE Press,2000:160-163.
[5]Beauxis-Aussalet E,Hardman L.Simplifying the visualization of confusion matrix[C]//Proc of Belgian-Dutch Conference on Artificial Intelligence.Amsterdom:Centrum Wiskunde&Informatioca,2014.
[6]Mantalos P,Kargrigoriou A.Bootstrapping the augmented DickeyFuller test for unit root using the MDIC[J].Journal of Statistical Computation&Simulation,2012,82(3):431-443.
[7]孔英会,景美丽.基于混淆矩阵和集成学习的分类方法研究[J].算机工程与科学,2012,34(6):111-117.
[8]Margineantu D D,Dietterich T G.Pruning adaptive boosting[C]//Proc of the 14th International Conference on Machine Learning.San Francisco:Morgan Kaufmann Publishers Inc,1997:211-218.
[9]Woz'niak M,Grana M,Corchado E,et al.A survey of multiple classifier systems as hybrid systems[J].Information Fusion,2014,16(1):3-17.
[10]Gong Yuejiao,Chen Weineng,Zhan Zhihui,et al.Distributed evolutionary algorithms and their models[J].Applied Soft Computing,2015,34(9):286-300.
[11]liobaitéI,Bifet A,Read J,et al.Evaluation methods and decision theory for classification of streaming data with temporal dependence[J].Machine Learning,2015,98(3):455-482.
[12]Pradhan B.A comparative study on the predictive ability of the decision tree,support vector machine and neuro-fuzzy models in landslide susceptibility mapping using GIS[J].Computers&Geosciences,2013,51(2):350-365.
[13]Godara A S,Verma A.Analysis of various clustering algorithms[J].International Journal of Innovative Technology&Exploring Engineering,2013,3(1):101-102
[14]Dai Qun,Zhang Ting,Liu Ningzhong.A new reverse reduce-error ensemble pruning algorithm[J].Applied Soft Computing,2015,28(1):237-249.
[2]毕凯,王晓丹,姚旭,等.一种基于Bagging和混淆矩阵的自适应选择性集成[J].电子学报,2014,42(4):711-716.
[3]Li Nan,Yu Yang,Zhou Zhihua.Diversity regularized ensemble pruning[C]//Proc of Joint European Conference on Machine Learning and Knowledge Discovery in Databases.Berlin:Springer-Verlag,2012:330-345.
[4]Giacinto G,Roli F,Fumera G.Design of effective multiple classifier systems by clustering of classifiers[C]//Proc of the 15th International Conference on Pattern Recognition.Piscataway,NJ:IEEE Press,2000:160-163.
[5]Beauxis-Aussalet E,Hardman L.Simplifying the visualization of confusion matrix[C]//Proc of Belgian-Dutch Conference on Artificial Intelligence.Amsterdom:Centrum Wiskunde&Informatioca,2014.
[6]Mantalos P,Kargrigoriou A.Bootstrapping the augmented DickeyFuller test for unit root using the MDIC[J].Journal of Statistical Computation&Simulation,2012,82(3):431-443.
[7]孔英会,景美丽.基于混淆矩阵和集成学习的分类方法研究[J].算机工程与科学,2012,34(6):111-117.
[8]Margineantu D D,Dietterich T G.Pruning adaptive boosting[C]//Proc of the 14th International Conference on Machine Learning.San Francisco:Morgan Kaufmann Publishers Inc,1997:211-218.
[9]Woz'niak M,Grana M,Corchado E,et al.A survey of multiple classifier systems as hybrid systems[J].Information Fusion,2014,16(1):3-17.
[10]Gong Yuejiao,Chen Weineng,Zhan Zhihui,et al.Distributed evolutionary algorithms and their models[J].Applied Soft Computing,2015,34(9):286-300.
[11]liobaitéI,Bifet A,Read J,et al.Evaluation methods and decision theory for classification of streaming data with temporal dependence[J].Machine Learning,2015,98(3):455-482.
[12]Pradhan B.A comparative study on the predictive ability of the decision tree,support vector machine and neuro-fuzzy models in landslide susceptibility mapping using GIS[J].Computers&Geosciences,2013,51(2):350-365.
[13]Godara A S,Verma A.Analysis of various clustering algorithms[J].International Journal of Innovative Technology&Exploring Engineering,2013,3(1):101-102
[14]Dai Qun,Zhang Ting,Liu Ningzhong.A new reverse reduce-error ensemble pruning algorithm[J].Applied Soft Computing,2015,28(1):237-249.