Constructing and Combining Orthogonal Projection Vectors for Ordinal Regression

详细信息    查看全文

• 作者：Bing-Yu Sun (1)
  Hai-Lei Wang (1) (2)
  Wen-Bo Li (1)
  Hui-Jing Wang (3)
  Jiuyong Li (4)
  Zhi-Qiang Du (5)
  
  1. Hefei Institute of Intelligent Machines ; Chinese Academy of Sciences ; Hefei ; Anhui ; People鈥檚 Republic of China
  2. The Department of Automation ; University of Science and Technology of China ; Hefei ; People鈥檚 Republic of China
  3. The Computer College ; Shenzhen Institute of Information Technology ; Shenzhen ; Guangdong ; People鈥檚 Republic of China
  4. The School of Computer and information Science ; University of South Australia ; Adelaide ; Australia
  5. The State Key Laboratory of Information Engineering in Surveying ; Mapping and Remote Sensing ; Wuhan University ; Wuhan ; People鈥檚 Republic of China
  
• 关键词：Ordinal regression ; Linear discriminant analysis ; Kernel discriminant analysis ; Multiple feature combination
• 刊名：Neural Processing Letters
• 出版年：2015
• 出版时间：February 2015
• 年：2015
• 卷：41
• 期：1
• 页码：139-155
• 全文大小：617 KB
• 参考文献：1. Kramer S, Widmer G, Pfahringer B, DeGroeve M (2001) Prediction of ordinal classes using regression trees. Fundamenta Informaticae 47(1鈥?):1鈥?3
  2. Herbrich R, Graepel T, Obermayer K (2000) Large margin rank boundaries for ordinal regression. In: Smola AJ, Bartlett PL, Sch枚lkopf B, Schuurmans D (eds) Advances in large margin classifiers. MIT Press, Cambridge, pp 115鈥?32
  3. Crammer K, Singer Y (2002) Pranking with ranking. In: Dietterich TG, Becker S, Ghahramani Z (eds) Advances in neural information processing systems. MIT Press, Cambridge, pp 641鈥?47
  4. Shashua A, Levin A (2003) Ranking with large margin principle: two approaches. In: Becker S, Thrun S, Obermayer K (eds) Advances in neural information processing systems. MIT Press, Cambridge, pp 961鈥?68
  5. Chu W, Keerthi SS (2005) New approaches to support vector ordinal regression. In: Proceedings of the 22nd international conference on machine learning (ICML 2005). Omnipress, pp 145鈥?52
  6. Lin L, Lin H-T (2007) Ordinal regression by extended binary classification. In: Advances in neural information processing systems 19: proceedings of the 2006 Conference (NIPS 2006). MIT Press, pp 865鈥?72
  7. Cardoso JS, Pinto da Costa JF (2007) Learning to classify ordinal data: the data replication method. J Mach Learn Res 8:1393鈥?429
  8. Liu Y, Liu Y, Chan KCC (2011) Ordinal regression via manifold learning. In: Proceedings of 25th AAAI conference on artificial Intelligence (AAAI11), pp 398鈥?03
  9. Baccianella S, Esuli A, SebastianiF F (2010) Feature selection for ordinal regression. In: Proceedings of the 2010 ACM symposium on applied computing (SAC 鈥?0). ACM, New York, pp 1748鈥?754
  10. Bishop CM (2006) Pattern recognition and machine learning. Springer, Heidelberg
  11. Duda RO, Hart PE, Stork D (2000) Pattern classification. Wiley, Chichester
  12. Li H, Jiang T, Zhang K (2006) Efficient and robust feature extraction by maximum margin criterion. IEEE Trans Neural Netw 17(1):157鈥?65 CrossRef
  13. Min W, Lu K, He X (2004) Locality pursuit embedding. Pattern Recognit 37(4):781鈥?88 CrossRef
  14. Zhang T, Huang K, Li X, Yang J, Tao D (2010) Generalized discriminant analysis: a matrix exponential approach. IEEE Trans Syst Man Cybern B 40(1):253鈥?63 CrossRef
  15. Xia F, Tao Q, Wang J, Zhang W (2007) Recursive feature extraction for ordinal regression. In: International joint conference on neural networks (IJCNN鈥?7), pp 78鈥?3
  16. Sun B-Y, Li J, Wu DD, Zhang X-M, Li W-B (2010) Kernel discriminant learning for ordinal regression. IEEE Trans Knowl Data Eng 22(6):906鈥?10 CrossRef
  17. Ye J (2005) Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems. J Mach Learn Res 6:4831502
  18. Ji S, Ye J (2008) Generalized linear discriminant analysis: a unified framework and efficient model selection. IEEE Trans Neural Netw 19(10):1768鈥?782 CrossRef
  19. Vapnik V (1998) The nature of statistical learning theory. Wiley, New York
  20. Muller K-R, Mika S, Ratsch G, Tsuda K, Scholkopf B (2001) An introduction to Kernel-based learning algorithms. IEEE Trans Neural Netw 12(2):181鈥?01 CrossRef
  21. Mika S (2002) Kernel fisher discriminants. PhD thesis, University of Technology, Berlin
  22. Guo Y, Hastie T, Tibshirani R (2007) Regularized linear discriminant analysis and its application in microarrays. Biostatistics 8(1):86鈥?00 CrossRef
  23. Kim H, Drake B, Park H (2006) Adaptive nonlinear discriminant analysis by regularized minimum squared errors. IEEE Trans Knowl Data Eng 18(5):603鈥?12 CrossRef
  24. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123鈥?40
  25. Rosenwald A, Wright G, Chan WC, Connors JM, Campo E, Fisher RI, Gascoyne RD, Muller-Hermelink HK, Smeland EB, Staudt LM (2002) The use of molecular profiling to predict survival after chemotherapy for diffuse large-B-cell lymphoma. N Engl J Med 346(25):1937鈥?947 CrossRef
  
• 刊物类别：Physics and Astronomy
• 刊物主题：Physics
  Complexity
  Artificial Intelligence and Robotics
  Electronic and Computer Engineering
  Operation Research and Decision Theory
  
• 出版者：Springer Netherlands
• ISSN：1573-773X

文摘

Ordinal regression is to predict categories of ordinal scale and it has wide applications in many domains where the human evaluation plays a major role. So far several algorithms have been proposed to tackle ordinal regression problems from a machine learning perspective. However, most of these algorithms only seek one direction where the projected samples are well ranked. So a common shortcoming of these algorithms is that only one dimension in the sample space is used, which would definitely lose some useful information in its orthogonal subspaces. In this paper, we propose a novel ordinal regression strategy which consists of two stages: firstly orthogonal feature vectors are extracted and then these projector vectors are combined to learn an ordinal regression rule. Compared with previous ordinal regression methods, the proposed strategy can extract multiple features from the original data space. So the performance of ordinal regression could be improved because more information of the data is used. The experimental results on both benchmark and real datasets proves the performance of the proposed method.