基于三支决策的主动学习方法
|
摘要
主动学习是机器学习领域研究的热点之一,旨在解决样本无标签问题.将三支决策的思想应用到主动学习中,通过引入决策函数,并基于无标签样本的不确定性,将无标签样本划分为3个不同的域:正域、负域、边界域.针对不同区域的样本进行相应处理,提出一种基于三支决策理论的主动学习方法(TWD_Active方法).通过主动学习方法选出最有用的样本交给专家进行标记,扩大训练集,创建更有效的模型.与传统的被动学习相比,该方法可以选择信息量高、有代表性的样本进行打标,可避免样本的冗余添加.通过反复迭代的训练学习达到预设的迭代次数或期望的性能指标.实验结果表明,所提出的算法在F-value、AUC等评价指标上均可取得良好的效果,验证了该算法的有效性.
Active learning is one of the focuses in the field of machine learning, aiming to solve the unlabeled problem of samples. In this paper, a three-way decision model is applied to active learning. By introducing decision functions,the unlabeled samples are divided into three different parts: positive region, boundary region and negative region based on the uncertainty of unlabeled samples. Different solutions are adopted to process samples for each region. Then, an active learning method based on the three-way decision model, namely TWD_Active, is developed. The most useful samples are selected using the active learning method, and are labeled by experts, so more effective models can be trained by the expanded training set. Compared with traditional passive learning, this method can choose the informational and representative samples to label, avoiding the redundant addition of sample. The models are continuously trained until the expected number of iterations or performance indicators are achieved. Experimental results show that the proposed algorithm has a better performance in measures F-value, AUC and the effectiveness of the algorithm is verified.
Active learning is one of the focuses in the field of machine learning, aiming to solve the unlabeled problem of samples. In this paper, a three-way decision model is applied to active learning. By introducing decision functions,the unlabeled samples are divided into three different parts: positive region, boundary region and negative region based on the uncertainty of unlabeled samples. Different solutions are adopted to process samples for each region. Then, an active learning method based on the three-way decision model, namely TWD_Active, is developed. The most useful samples are selected using the active learning method, and are labeled by experts, so more effective models can be trained by the expanded training set. Compared with traditional passive learning, this method can choose the informational and representative samples to label, avoiding the redundant addition of sample. The models are continuously trained until the expected number of iterations or performance indicators are achieved. Experimental results show that the proposed algorithm has a better performance in measures F-value, AUC and the effectiveness of the algorithm is verified.
引文
[1]Gong X J,Sun J P,Shi Z Z.Active bayesian network classifier[J].Computer Research and Development,200239(5):574-579.
[2]Culotta A,McCallum A.Reducing labeling effort for structured prediction tasks[C].Proc of AAAI 2005Menlo Park:AAAI Press,2005:746-751.
[3]Lewis D,Catlett J.Heterogeneous uncertainty sampling for supervised learning[C].Proc of ICML 1994.San Francisco:Morgan Kaufmann,1994:148-156.
[4]Scheffer T,Decomain C,Wrobel S.Active hidden Markov models for information extraction[C].Int Conf on Advances in Intelligent Data Analysis.Berlin:Springer-Verlag,2001:309-318.
[5]Settles B.Active learning literature survey[J].University of Wisconsinmadison,2009,39(2):127-131.
[6]Tong S,Koller D.Support vector machine active learning with applications to text classification[J].J of Machine Learning Research,2001,2(1):45-66.
[7]Zhang C,Chen T.An active learni ng framework for content based information retrieval[J].IEEE Trans on Multimedia,2002,4(2):260-268.
[8]Xu Z,Yu K,Tresp V,et al.Representative sampling for text classification using support vector machines[C]European Conf on IR Research.Berlin:Springer-Verlag2003:393-407.
[9]Donmez P,Carbonell J G,Bennett P N.Dual strategy active learning[C].Proc of the 18th European Conf on Machine Learning.Berlin:Springer-Verlag,2008:208-215.
[10]Hoi S C H,Jin R,Zhu J,et al.Semi-supervised svm batch mode active learning for image retrieval[C].Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recongnition.Anchorage:IEEE Computer Society Press,2008:1-7.
[11]Huang S,Jin R,Zhou Z.Active learning by querying information and representative examples[C].Proc of NIPS 2010.Cambridge:MIT Press,2010:892-900.
[12]Settles B,Craven M.An analysis of active learning strategies for sequence labeling tasks[C].Proc EMNLPStroudsburg:ACL Press,2008:1069-1078.
[13]Seung H S,Opper M,Sompolinsky H.Query by committee[C].Proc of the 5th ACM Workshop on Computational Learning Theory.New York:ACM Press1992:287-294.
[14]Abe N,Mamitsuka H.Query learning strategies using boosting and bagging[C].Proc of ICML 1998.San Francisco:Morgan Kaufmann,1998:1-9.
[15]Cohn D,Ghahramani Z,Jordan M I.Active learning with statistical models[J].Artificial Intelligence Research1996,4(1):129-145.
[16]Yao Y Y.Three-way decisions with probabilistic rough sets[J].Information Sciences,2010,180(3):341-353.
[17]Yao Y Y.The superiority of three-way decision in probabilistic rough set models[J].Information Sciences2011,181(6):1080-1096.
[18]Yao Y Y.An outline of a theory of three-way decisions[C]Int Conf on Rough Sets and Current Trends in Computing Heidelberg:Springer,2012:1-17.
[19]Li H,Zhang L,Huang B,et al.Sequential three-way decision and granulation for cost-sensitive face recognition[J].Knowledge-Based Systems,2016,91(1):241-251.
[20]Liu D,Li T R,Liang D C.Incorporating logistic regression to decision-theoretic rough sets for classifications[J].Int J of Approximate Reasoning,201455(1):197-210.
[21]Liu D,Liang D,Wang C,et al.A novel three-way decision model based on incomplete information system[J]Knowledge-Based Systems,2016,91(1):32-45.
[22]Yu H,Wang Y,Jiao P.Detecting and refining overlapping regions in complex networks with three-way decisions[J]Information Sciences,2016,373:21-41.
[23]Yu H,Zhang C,Wang G Y.A tree-based incremental overlapping clustering method using the three-way decision theory[J].Knowledge-Based Systems,201691(1):189-203.
[24]Ma X A,Wang G Y,Yu H,et al.Decision region distribution preservation reduction in decision-theoretic rough set model[J].Information Sciences,2014(278):614-640.
[25]Chen Y,Zeng Z,Zhu Q,et al.Three-way decision reduction in neighborhood systems[J].Applied Soft Computing,2016,38(1):942-954.
[26]Lin T Y.Neighborhood systems and approximation in relational databases and knowledge bases[C].Proc of the 4th Int Symposium on Metho-dologies of Intelligent Systems.Charlotte:Oak Ridge National Laboratory1989:75-86.
[27]Hu Q,Yu D,Xie Z.Neighborhood classifiers[J].Expert Systems With Applications,2008,34(2):866-876.
[28]Stanfill C,Waltz D.Toward memory-based reasoning[J]Communications of the ACM,1986,29(12):1213-1228
[29]Liu D,Li T R,Miao D Q,et al.Three-way decision and granular computing[M].Beijing:Science Press,2013:12-30.
[30]Zhou Z H.Machine learning[M].Beijing:Tsinghua University Press,2016:150-154.
[31]Asuncion A,Newman D J.UCI machine learning repository[CP/OL].http://archive.ics.uci.edu/ml.
[32]Wikipedia Weka(machine learning)[CP/OL].http://en wikipedia.org/wiki/Weka.
[2]Culotta A,McCallum A.Reducing labeling effort for structured prediction tasks[C].Proc of AAAI 2005Menlo Park:AAAI Press,2005:746-751.
[3]Lewis D,Catlett J.Heterogeneous uncertainty sampling for supervised learning[C].Proc of ICML 1994.San Francisco:Morgan Kaufmann,1994:148-156.
[4]Scheffer T,Decomain C,Wrobel S.Active hidden Markov models for information extraction[C].Int Conf on Advances in Intelligent Data Analysis.Berlin:Springer-Verlag,2001:309-318.
[5]Settles B.Active learning literature survey[J].University of Wisconsinmadison,2009,39(2):127-131.
[6]Tong S,Koller D.Support vector machine active learning with applications to text classification[J].J of Machine Learning Research,2001,2(1):45-66.
[7]Zhang C,Chen T.An active learni ng framework for content based information retrieval[J].IEEE Trans on Multimedia,2002,4(2):260-268.
[8]Xu Z,Yu K,Tresp V,et al.Representative sampling for text classification using support vector machines[C]European Conf on IR Research.Berlin:Springer-Verlag2003:393-407.
[9]Donmez P,Carbonell J G,Bennett P N.Dual strategy active learning[C].Proc of the 18th European Conf on Machine Learning.Berlin:Springer-Verlag,2008:208-215.
[10]Hoi S C H,Jin R,Zhu J,et al.Semi-supervised svm batch mode active learning for image retrieval[C].Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recongnition.Anchorage:IEEE Computer Society Press,2008:1-7.
[11]Huang S,Jin R,Zhou Z.Active learning by querying information and representative examples[C].Proc of NIPS 2010.Cambridge:MIT Press,2010:892-900.
[12]Settles B,Craven M.An analysis of active learning strategies for sequence labeling tasks[C].Proc EMNLPStroudsburg:ACL Press,2008:1069-1078.
[13]Seung H S,Opper M,Sompolinsky H.Query by committee[C].Proc of the 5th ACM Workshop on Computational Learning Theory.New York:ACM Press1992:287-294.
[14]Abe N,Mamitsuka H.Query learning strategies using boosting and bagging[C].Proc of ICML 1998.San Francisco:Morgan Kaufmann,1998:1-9.
[15]Cohn D,Ghahramani Z,Jordan M I.Active learning with statistical models[J].Artificial Intelligence Research1996,4(1):129-145.
[16]Yao Y Y.Three-way decisions with probabilistic rough sets[J].Information Sciences,2010,180(3):341-353.
[17]Yao Y Y.The superiority of three-way decision in probabilistic rough set models[J].Information Sciences2011,181(6):1080-1096.
[18]Yao Y Y.An outline of a theory of three-way decisions[C]Int Conf on Rough Sets and Current Trends in Computing Heidelberg:Springer,2012:1-17.
[19]Li H,Zhang L,Huang B,et al.Sequential three-way decision and granulation for cost-sensitive face recognition[J].Knowledge-Based Systems,2016,91(1):241-251.
[20]Liu D,Li T R,Liang D C.Incorporating logistic regression to decision-theoretic rough sets for classifications[J].Int J of Approximate Reasoning,201455(1):197-210.
[21]Liu D,Liang D,Wang C,et al.A novel three-way decision model based on incomplete information system[J]Knowledge-Based Systems,2016,91(1):32-45.
[22]Yu H,Wang Y,Jiao P.Detecting and refining overlapping regions in complex networks with three-way decisions[J]Information Sciences,2016,373:21-41.
[23]Yu H,Zhang C,Wang G Y.A tree-based incremental overlapping clustering method using the three-way decision theory[J].Knowledge-Based Systems,201691(1):189-203.
[24]Ma X A,Wang G Y,Yu H,et al.Decision region distribution preservation reduction in decision-theoretic rough set model[J].Information Sciences,2014(278):614-640.
[25]Chen Y,Zeng Z,Zhu Q,et al.Three-way decision reduction in neighborhood systems[J].Applied Soft Computing,2016,38(1):942-954.
[26]Lin T Y.Neighborhood systems and approximation in relational databases and knowledge bases[C].Proc of the 4th Int Symposium on Metho-dologies of Intelligent Systems.Charlotte:Oak Ridge National Laboratory1989:75-86.
[27]Hu Q,Yu D,Xie Z.Neighborhood classifiers[J].Expert Systems With Applications,2008,34(2):866-876.
[28]Stanfill C,Waltz D.Toward memory-based reasoning[J]Communications of the ACM,1986,29(12):1213-1228
[29]Liu D,Li T R,Miao D Q,et al.Three-way decision and granular computing[M].Beijing:Science Press,2013:12-30.
[30]Zhou Z H.Machine learning[M].Beijing:Tsinghua University Press,2016:150-154.
[31]Asuncion A,Newman D J.UCI machine learning repository[CP/OL].http://archive.ics.uci.edu/ml.
[32]Wikipedia Weka(machine learning)[CP/OL].http://en wikipedia.org/wiki/Weka.