Constructing ECOC based on confusion matrix for multiclass learning problems

详细信息    查看全文

• 作者：Jindeng Zhou ; Yun Yang ; Mingjie Zhang ; Haibo Xing
• 关键词：machine learning ; multiclass classification ; error correcting output codes ; subclass partition ; confusion matrix
• 刊名：SCIENCE CHINA Information Sciences
• 出版年：2016
• 出版时间：January 2016
• 年：2016
• 卷：59
• 期：1
• 页码：1-14
• 全文大小：853 KB
• 参考文献：1.Dietterich T G, Bakiri G. Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res, 1995, 2: 263–286MATH
  2.Windeatt T, Ardeshir G. Boosted ECOC ensembles for face recognition. In: Proceedings of International Conference on Visual Information Engineering. California: AAAI Press, 2003. 165–168
  3.Windeatt T, Smith R S. Weighted decoding ECOC for facial action unit classification. In: Proceedings of International Conference on Artificial Intelligence. California: AAAI Press, 2008. 26–30
  4.Ghani R. Combining labeled and unlabeled data for text classification with a large number of categories. In: Proceedings of International Conference on Data Mining. California: AAAI Press, 2001. 597–598
  5.Escalera S, Pujol O, Radeva P. On the decoding process in ternary errorcorrecting output codes. IEEE Trans Patt Anal Mach Intell, 2010, 32: 120–134CrossRef
  6.Pujol O, Radeva P, Vitria J. Discriminant ECOC: a heuristic method for application dependent design of error correcting output codes. IEEE Trans Patt Anal Mach Intell, 2006, 28: 1001–1007CrossRef
  7.Alpaydin E, Mayoraz E. Learning error-correcting output codes from data. In: Proceedings of International Conference on Artificial Neural Networks. California: AAAI Press, 1999. 743–748
  8.Utschick W, Weichselberger W. Stochastic organization of output codes in multiclass learning problems. Neural Comput, 2001, 13: 1065–1102CrossRef MATH
  9.Escalera S, Tax D M J, Pujol O, et al. Subclass problem-dependent design of error-correcting output codes. IEEE Trans Patt Anal Mach Intell, 2008, 30: 1–14CrossRef
  10.Escalera S, Masip D, Puertas E, et al. Online error correcting output codes. Patt Recognit Lett, 2011, 32: 458–467CrossRef
  11.Simeone P, Marrocco C, Tortorella F. Design of reject rules for ECOC classification systems. Patt Recognit, 2012, 2: 863–875
  12.García-Pedrajas N, Ortiz-Boyer D. An empirical study of binary classifier fusion methods for multiclass classification. Inf Fusion, 2011, 2: 111–130CrossRef
  13.Zhou J, Suen C. Unconstrained numeral pair recognition using enhanced error correcting output coding: a holistic approach. In: Proceedings of International Conference on Document Analysis and Recognition. California: AAAI Press, 2005. 484–488
  14.Allwein E, Schapire R, Singer Y. Reducing multiclass to binary: a unifying approach for margin classifiers. Mach Learn Res, 2002, 1: 113–141MathSciNet MATH
  15.Ruda H, Snorrason M, Shue D. Framework for automatic target recognition optimization. Cambridge Charles River Analytics Technical Report No. R96451. 1997
  16.Jain A K, Duin R P W, Mao J C. Statistical pattern recognition: a review. IEEE Trans Patt Anal Mach Intell, 2000, 22: 4–37CrossRef
  17.Sun J X. Modern Pattern Recognition (in Chinese). Changsha: National Defense Industry Press, 2002. 112–118
  18.Song R, Zhang J, Xia S P, et al. An adaptive classification method of BP-NN group based classification system and its application (in Chinese). Acta Electron Sin, 2001, 29: 1950–1953
  19.Logan J D. Applied Mathematics. 2nd ed. Hoboken: Wiley-Interscience Ltd., 1996. 54–60
  20.UCI Machine Learning Repository. School of Information and Computer Sciences, University of California, Irvine, 2007
  21.Zhang Y X, Wang X D, Yao X, et al. HRRP recognition for polarization radar based on Bagging-SVM dynamic ensemble (in Chinese). Syst Eng Electron, 2012, 34: 1366–1372
  22.Hastie T, Tibshirani R. Classification by pairwise grouping. In: Jordan M I, Kearns M J, Solla S A, eds. Advances in Neural Information Processing Systems 10. Cambridge: MIT Press, 1998. 451–471
  23.Chapelle O, Vapnik V, Bousquet O, et al. Choosing multiple parameters for support machines. Mach Learn, 2002, 46: 131–159CrossRef MATH
  24.Demsar J. Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res, 2006, 2: 1–30MathSciNet MATH
  
• 作者单位：Jindeng Zhou (1)
  Yun Yang (1)
  Mingjie Zhang (1)
  Haibo Xing (1)
  
  1. Science and Technology on Complex Aviation, Systems Simulation Laboratory, Beijing, 100076, China
  
• 刊物类别：Computer Science
• 刊物主题：Chinese Library of Science
  Information Systems and Communication Service
  
• 出版者：Science China Press, co-published with Springer
• ISSN：1869-1919

文摘

In the pattern recognition field, error-correcting output codes (ECOC) are a powerful tool to fuse any number of binary classifiers to model multiclass problems, and the research of encoding based on data is attracting more and more attention. In this paper, we are going to propose a new encoding method for constructing subclass Error-Correcting Output Codes, which was first introduced by Escalera et al. To achieve this goal, we first obtain the correlation between each pair of classes with the help of confusion matrix. Then, we select the most easily separated subclasses for classification by following Fisher’s principle. At last, we were able to obtain binary partitions based on subclasses. After finishing this work, a new data-driven coding matrix-Subclass ECOC will be achieved. Experimental results on University of CaliforniaIrvine data sets and three kinds of high resolution range profile data sets with logistic linear classifier and support vector machine as the binary classifiers show that our approach can provide a better performance and the robustness of classification with a little longer but acceptable code length.