面向类别比例偏移的半监督支持向量机方法
|
摘要
当未标记数据与有标记数据类别比例偏移较大时,半监督支持向量机性能不佳.基于此情况,文中提出面向类别比例偏移的半监督支持向量机方法.首先估计未标记数据类中心,然后对多个类别比例下的类中心进行最坏情况集成,从而提升半监督支持向量机的性能保障.实验表明,文中方法有效提升半监督支持向量机在类别比例偏移时的性能保障.
When the label proportion of unlabeled data is far away from that of labeled data,direct supervised support vector machine( SVM) with only labeled data outperforms semi-supervised SVM( S3VM) with unlabeled data. Thus,a shifted label proportion aware S3VM( fair S3VM) is proposed. Specifically,the label mean of unlabeled data is firstly estimated. Then multiple label means corresponding to multiple label proportions are integrated under the worst-case scenario. Experimental results show that the performance guarantee of S3 VMs is effectively improved when the label proportion is shifted.
When the label proportion of unlabeled data is far away from that of labeled data,direct supervised support vector machine( SVM) with only labeled data outperforms semi-supervised SVM( S3VM) with unlabeled data. Thus,a shifted label proportion aware S3VM( fair S3VM) is proposed. Specifically,the label mean of unlabeled data is firstly estimated. Then multiple label means corresponding to multiple label proportions are integrated under the worst-case scenario. Experimental results show that the performance guarantee of S3 VMs is effectively improved when the label proportion is shifted.
引文
[1]CHAPELLE O,SCHLKOPF B,ZIEN A.Semi-supervised Learning.Cambridge,USA:MIT Press,2006.
[2]ZHOU Z H,LI M.Semi-supervised Learning by Disagreement.Knowledge and Information Systems,2010,24(3):415-439.
[3]ZHU X J.Semi-supervised Learning Literature Survey[J/OL].[2016-02-28].http://pages.cs.wisc.edu/~jerryzhu/pub/ssl_survey.pdf.
[4]LIU W,WANG J,CHANG S F.Robust and Scalable Graph-Based Semisupervised Learning.Proceeding of the IEEE.2012,100(9):2624-2638.
[5]周志华.基于分歧的半监督学习.自动化学报,2013,39(11):1871-1878.(ZHOU Z H.Disagreement-Based Semi-supervised Learning.Acta Automatica Sinica,2013,39(11):1871-1878.)
[6]JOACHIMS T.Transductive Inference for Text Classification Using Support Vector Machines//Proc of the 16th International Conference on Machine Learning.San Francisco,USA:Morgan Kaufmann Publishers,1999:200-209.
[7]WANG L,CHAN K L,ZHANG Z H.Bootstrapping SVM Active Learning by Incorporating Unlabelled Images for Image Retrieval//Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2003,I:629-634.
[8]KASABOV N,PANG S N.Transductive Support Vector Machines and Applications in Bioinformatics for Promoter Recognition//Proc of the International Conference on Neural Networks and Signal Processing.Nanjing,China:IEEE,2003,I:1-6.
[9]GOUTTE C,DJEAN H,GAUSSIER E,et al.Combining Labelled and Unlabelled Data:A Case Study on Fisher Kernels and Transductive Inference for Biological Entity Recognition//Proc of the 6th Conference on Natural Language Learning.Stroudsburg,USA:Association for Computational Linguistics,2002,20:1-7.
[10]BENNETT K P,DEMIRIZ A.Semi-supervised Support Vector Machines//KEARNS M J,SOLLA S A,COHN D A,eds.Advances in Neural Information Processing Systems 11.Cambridge,USA:MIT Press,1999:368-374.
[11]VAPNIK V N.Statistical Learning Theory.New York,USA:Wiely,1998.
[12]VAPNIK V N.The Nature of Statistical Learning Theory.New York,USA:Springer-Verlag,1995.
[13]VAPNIK V N,STERIN A M.On Structural Risk Minimization or Overall Risk in a Problem of Pattern Recognition.Automation and Remote Control,1977,10(3):1495-1503.
[14]CHAPELLE O,SINDHWANI V,KEERTHI S S.Optimization Techniques for Semi-supervised Support Vector Machines.Journal of Machine Learning Research,2008,9:203-233.
[15]CHAPELLE O,SINDHWANI V,KEERTHI S S.Branch and Bound for Semi-supervised Support Vector Machines[EB/OL].[2015-12-02].http://www.keerthis.com/bb_nips_chapelle_06.pdf.
[16]LI Y F,KWOK J T,ZHOU Z H.Semi-supervised Learning Using Label Mean//Proc of the 26th Annual International Conference on Machine Learning.New York,USA:ACM,2009:633-640.
[17]COLLOBERT R,SINZ F,WESTON J,et al.Large Scale Transductive SVMs.Journal of Machine Learning Research,2006,7:1687-1712.
[18]BIE T D,CRISTIANINI N.Convex Methods for Transduction//THRUN S,SAUL L K,SCHLKOPF B,eds.Advances in Neural Information Processing Systems 16.Cambridge,USA:MIT Press,2004:73-80.
[19]XU L L,SCHUURMANS D.Unsupervised and Semi-supervised Multi-class Support Vector Machines//Proc of the 20th National Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2005,II:904-910.
[20]CHAWLA N V,KARAKOULAS G.Learning from Labeled and Unlabeled Data:An Empirical Study across Techniques and Domains.Journal of Artificial Intelligence Research,2011,23:331-366.
[21]ZHANG T,OLES F J.A Probability Analysis on the Value of Unlabeled Data for Classification Problems//Proc of the 17th International Confe-rence on Machine Learning.New York,USA:ACM,2000:1191-1198
[22]LI Y F,TSANG I W,KWOK J T,et al.Convex and Scalable Weakly Labeled SVMs.Journal of Machine Learning Research,2013,14:2151-2188.
[23]NIGAM K,MCCALLUM A K,THRUN S,et al.Text Classification from Labeled and Unlabeled Documents Using EM.Machine Learning,2000,39:103-134.
[24]BLUM A,CHAWLA S.Learning from Labeled and Unlabeled Data Using Graph Mincuts//Proc of the 18th International Conference on Machine Learning.San Francisco,USA:Morgan Kaufmann Publishers,2001:19-26.
[25]CHEN K,WANG S H.Semi-supervised Learning via Regularized Boosting Working on Multiple Semi-supervised Assumptions.IEEE Trans on Pattern Analysis and Machine Intelligence,2011,33(1):129-143.
[26]LI Y F,ZHOU Z H.Towards Making Unlabeled Data Never Hurt.IEEE Trans on Pattern Analysis and Machine Intelligence,2015,37(1):175-188.
[27]FAN R E,CHANG K W,HSIEH C J,et al.LIBLINEAR:A Library for Large Linear Classification.Journal of Machine Learning Research,2008,9:1871-1874.
[2]ZHOU Z H,LI M.Semi-supervised Learning by Disagreement.Knowledge and Information Systems,2010,24(3):415-439.
[3]ZHU X J.Semi-supervised Learning Literature Survey[J/OL].[2016-02-28].http://pages.cs.wisc.edu/~jerryzhu/pub/ssl_survey.pdf.
[4]LIU W,WANG J,CHANG S F.Robust and Scalable Graph-Based Semisupervised Learning.Proceeding of the IEEE.2012,100(9):2624-2638.
[5]周志华.基于分歧的半监督学习.自动化学报,2013,39(11):1871-1878.(ZHOU Z H.Disagreement-Based Semi-supervised Learning.Acta Automatica Sinica,2013,39(11):1871-1878.)
[6]JOACHIMS T.Transductive Inference for Text Classification Using Support Vector Machines//Proc of the 16th International Conference on Machine Learning.San Francisco,USA:Morgan Kaufmann Publishers,1999:200-209.
[7]WANG L,CHAN K L,ZHANG Z H.Bootstrapping SVM Active Learning by Incorporating Unlabelled Images for Image Retrieval//Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2003,I:629-634.
[8]KASABOV N,PANG S N.Transductive Support Vector Machines and Applications in Bioinformatics for Promoter Recognition//Proc of the International Conference on Neural Networks and Signal Processing.Nanjing,China:IEEE,2003,I:1-6.
[9]GOUTTE C,DJEAN H,GAUSSIER E,et al.Combining Labelled and Unlabelled Data:A Case Study on Fisher Kernels and Transductive Inference for Biological Entity Recognition//Proc of the 6th Conference on Natural Language Learning.Stroudsburg,USA:Association for Computational Linguistics,2002,20:1-7.
[10]BENNETT K P,DEMIRIZ A.Semi-supervised Support Vector Machines//KEARNS M J,SOLLA S A,COHN D A,eds.Advances in Neural Information Processing Systems 11.Cambridge,USA:MIT Press,1999:368-374.
[11]VAPNIK V N.Statistical Learning Theory.New York,USA:Wiely,1998.
[12]VAPNIK V N.The Nature of Statistical Learning Theory.New York,USA:Springer-Verlag,1995.
[13]VAPNIK V N,STERIN A M.On Structural Risk Minimization or Overall Risk in a Problem of Pattern Recognition.Automation and Remote Control,1977,10(3):1495-1503.
[14]CHAPELLE O,SINDHWANI V,KEERTHI S S.Optimization Techniques for Semi-supervised Support Vector Machines.Journal of Machine Learning Research,2008,9:203-233.
[15]CHAPELLE O,SINDHWANI V,KEERTHI S S.Branch and Bound for Semi-supervised Support Vector Machines[EB/OL].[2015-12-02].http://www.keerthis.com/bb_nips_chapelle_06.pdf.
[16]LI Y F,KWOK J T,ZHOU Z H.Semi-supervised Learning Using Label Mean//Proc of the 26th Annual International Conference on Machine Learning.New York,USA:ACM,2009:633-640.
[17]COLLOBERT R,SINZ F,WESTON J,et al.Large Scale Transductive SVMs.Journal of Machine Learning Research,2006,7:1687-1712.
[18]BIE T D,CRISTIANINI N.Convex Methods for Transduction//THRUN S,SAUL L K,SCHLKOPF B,eds.Advances in Neural Information Processing Systems 16.Cambridge,USA:MIT Press,2004:73-80.
[19]XU L L,SCHUURMANS D.Unsupervised and Semi-supervised Multi-class Support Vector Machines//Proc of the 20th National Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2005,II:904-910.
[20]CHAWLA N V,KARAKOULAS G.Learning from Labeled and Unlabeled Data:An Empirical Study across Techniques and Domains.Journal of Artificial Intelligence Research,2011,23:331-366.
[21]ZHANG T,OLES F J.A Probability Analysis on the Value of Unlabeled Data for Classification Problems//Proc of the 17th International Confe-rence on Machine Learning.New York,USA:ACM,2000:1191-1198
[22]LI Y F,TSANG I W,KWOK J T,et al.Convex and Scalable Weakly Labeled SVMs.Journal of Machine Learning Research,2013,14:2151-2188.
[23]NIGAM K,MCCALLUM A K,THRUN S,et al.Text Classification from Labeled and Unlabeled Documents Using EM.Machine Learning,2000,39:103-134.
[24]BLUM A,CHAWLA S.Learning from Labeled and Unlabeled Data Using Graph Mincuts//Proc of the 18th International Conference on Machine Learning.San Francisco,USA:Morgan Kaufmann Publishers,2001:19-26.
[25]CHEN K,WANG S H.Semi-supervised Learning via Regularized Boosting Working on Multiple Semi-supervised Assumptions.IEEE Trans on Pattern Analysis and Machine Intelligence,2011,33(1):129-143.
[26]LI Y F,ZHOU Z H.Towards Making Unlabeled Data Never Hurt.IEEE Trans on Pattern Analysis and Machine Intelligence,2015,37(1):175-188.
[27]FAN R E,CHANG K W,HSIEH C J,et al.LIBLINEAR:A Library for Large Linear Classification.Journal of Machine Learning Research,2008,9:1871-1874.