基于域与样例平衡的多源迁移学习方法
|
摘要
针对如何有效使用多源域的决策知识去预测目标域样例标签的问题,提出一种平衡域与样例信息的多源迁移学习算法.为实现上述目的,本文提出了一种基于域与样例平衡的多源迁移学习方法(Multi-source Transfer Learning by Balancing both Domains and Instances,MTL-BDI).该方法的基本思想是将域层面和样例层面的双加权平衡项嵌入到迁移学习的原始目标函数中,然后利用交替优化技术对提出的目标函数进行有效求解.在文本和图像数据集上的大量实验表明,该方法在分类精度方面确实优于现有的多源迁移学习方法MCC-SVM(Multiple Convex Combination of SVM)、A-SVM(Adaptive SVM)、Multi-KMM(Multiple Kernel Mean Matching)和DAM(Domain Adaptation Machine).
When transfer learning attempts to leverage the decision knowledge effectively from multiple source domains to predict the labels of instances accurately in target domain,it should consider how to well balance source and target domains,and their instances in both domains.In this paper,a novel multi-source transfer learning method called mlti-source transfer learning by balancing both domains and instances(MTL-BDI) is proposed to achieve the above goal.The basic idea of the proposed method is to embed the doubly weighted domain-level and instance-level balance term into the original objective function of transfer learning and then solve the proposed objective function effectively by using the alternating optimization technique.Extensive experiments on text and image datasets indicate that the proposed method indeed outperforms several existing multi-source transfer learning methods MCC-SVM(Multiple Convex Combination of SVM),A-SVM(Adaptive SVM),Multi-KMM(Multiple Kernel Mean Matching) and DAM(Domain Adaptation Machine) in the sense of classification accuracy on target domain.
When transfer learning attempts to leverage the decision knowledge effectively from multiple source domains to predict the labels of instances accurately in target domain,it should consider how to well balance source and target domains,and their instances in both domains.In this paper,a novel multi-source transfer learning method called mlti-source transfer learning by balancing both domains and instances(MTL-BDI) is proposed to achieve the above goal.The basic idea of the proposed method is to embed the doubly weighted domain-level and instance-level balance term into the original objective function of transfer learning and then solve the proposed objective function effectively by using the alternating optimization technique.Extensive experiments on text and image datasets indicate that the proposed method indeed outperforms several existing multi-source transfer learning methods MCC-SVM(Multiple Convex Combination of SVM),A-SVM(Adaptive SVM),Multi-KMM(Multiple Kernel Mean Matching) and DAM(Domain Adaptation Machine) in the sense of classification accuracy on target domain.
引文
[1] Luo P,Zhuang F Z,Xiong H,et al.Transfer learning from multiple source domains via consensus regularization[A].Proceedings of the ACM International Conference on Information and Knowledge Management[C].USA:ACM,2008.103-112.
[2] Duan L,Xu D,Chang S F.Exploiting web images for event recognition in consumer videos:A multiple source domain adaptation approach[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2012.1338-1345.
[3] Yao Y,Doretto G.Boosting for transfer learning with multiple sources[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2010.1855-1862.
[4] Sun S L,Shi H L,Wu Y B.A survey of multi-source domain adaptation[J].Information Fusion,2015,24(C):84-92.
[5] Schweikert G,Widmer C,Sch?lkopf B,et al.An empirical analysis of domain adaptation algorithms for genomic sequence analysis[A].Proceedings of the Neural Information Processing Systems Conference[C].USA:Curran Associates,2009.1433-1440.
[6] Yang J,Yan R,Hauptmann A G.Cross-domain video concept detection using adaptive SVMs[A].Proceedings of the ACM International Conference on Multimedia[C].USA:ACM,2007.188-197.
[7] Chattopadhyay R,Sun Q,Fan W,et al.Multi-source domain adaptation and its application to early detection of fatigue[A].ACM SIGKDD International Conference on Knowledge Discovery and Data Mining[C].USA:ACM,2011.717-725.
[8] Duan L,Xu D,Tsang I W.Domain adaptation from multiple sources:A domain-dependent regularization approach[J].IEEE Transaction on Neural Networks and Learning System,2012,23(3):504-518.
[9] Weston J,Collobert R,Sinz F,et al.Inference with the universum[A].Proceedings of the International Conference on Machine Learning[C].USA:ACM,2006.1009-1016.
[10] Bezdek J C.Pattern Recognition with Fuzzy Objective Function Algorithms[M].New York:Plenum,1981.43-93.
[11] Chen S,Zhang C S.Selecting informative universum sample for semi-supervised learning[A].Proceedings of the International Jont Conference on Artifical Intelligence[C].USA:Morgan Kaufmann,2009.1016-1021.
[12] Chang C C,Lin C J.LIBSVM:A library for support vector machines[J].ACM Transactions on Intelligent Systems and Technology,2011,2(3):1-27.
[13] Long M,Wang J,Ding G,et al.Adaptation regularization:a general framework for transfer learning[J].IEEE Transactions on Knowledge and Data Engineering,2014,26(5):1076-1089.
[14] Long M,Wang J,Ding G,et al.Transfer feature learning with joint distribution adaptation[A].Proceedings of the IEEE International Conference on Computer Vision[C].USA:IEEE,2013.2200-2207.
[15] Gong B,Shi Y,Sha F,et al.Geodesic flow kernel for unsupervised domain adaptation[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2012.2066-2073.
[16] Daumé H III.Frustratingly easy domain adaptation[A].Proceedings of the Annual Meeting of the Association for Computational Linguistics[C].USA:ACL,2007.256-263.
[17] Li S,Song S J,Huang G.Prediction reweighting for domain adaptation[J].IEEE Transactions on Neural Networks and Learning Systems,2017,28(7):1682-1695.
[18] 张景祥,王士同.基于共同决策方向矢量的多源迁移及其快速学习方法[J].电子学报,2015,43(7):1349-1355.Zhang Jing-xiang,Wang Shi-tong.Common-decision-vectorbased multiple source transfer learning classification and its fast learning method[J].Acta Electronica Sinica,2015,43(7):1349-1355.(in Chinese)
[19] 蒋亦樟,邓赵红,王士同.0阶L2型TSK迁移学习模糊系统[J].电子学报,2013,41(5):897-904.Jiang Yi-zhang,Deng Zhao-hong,Wang Shi-tong.0-order-l2-norm-takagi-sugeno-kang type transfer learning fuzzy system[J].Acta Electronica Sinica,2013,41(5):897-904.(in Chinese)
[20] 倪彤光,王士同,应文豪,邓赵红.迁移组概率学习机[J].电子学报,2013,41(11):2207-2215.Ni Tong-guang,Wang Shi-tong,Ying Wen-hao,Deng Zhao-hong.Transfer group probabilities based learning machine[J].Acta Electronica Sinica,2013,41(11):2207-2215.(in Chinese)
[2] Duan L,Xu D,Chang S F.Exploiting web images for event recognition in consumer videos:A multiple source domain adaptation approach[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2012.1338-1345.
[3] Yao Y,Doretto G.Boosting for transfer learning with multiple sources[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2010.1855-1862.
[4] Sun S L,Shi H L,Wu Y B.A survey of multi-source domain adaptation[J].Information Fusion,2015,24(C):84-92.
[5] Schweikert G,Widmer C,Sch?lkopf B,et al.An empirical analysis of domain adaptation algorithms for genomic sequence analysis[A].Proceedings of the Neural Information Processing Systems Conference[C].USA:Curran Associates,2009.1433-1440.
[6] Yang J,Yan R,Hauptmann A G.Cross-domain video concept detection using adaptive SVMs[A].Proceedings of the ACM International Conference on Multimedia[C].USA:ACM,2007.188-197.
[7] Chattopadhyay R,Sun Q,Fan W,et al.Multi-source domain adaptation and its application to early detection of fatigue[A].ACM SIGKDD International Conference on Knowledge Discovery and Data Mining[C].USA:ACM,2011.717-725.
[8] Duan L,Xu D,Tsang I W.Domain adaptation from multiple sources:A domain-dependent regularization approach[J].IEEE Transaction on Neural Networks and Learning System,2012,23(3):504-518.
[9] Weston J,Collobert R,Sinz F,et al.Inference with the universum[A].Proceedings of the International Conference on Machine Learning[C].USA:ACM,2006.1009-1016.
[10] Bezdek J C.Pattern Recognition with Fuzzy Objective Function Algorithms[M].New York:Plenum,1981.43-93.
[11] Chen S,Zhang C S.Selecting informative universum sample for semi-supervised learning[A].Proceedings of the International Jont Conference on Artifical Intelligence[C].USA:Morgan Kaufmann,2009.1016-1021.
[12] Chang C C,Lin C J.LIBSVM:A library for support vector machines[J].ACM Transactions on Intelligent Systems and Technology,2011,2(3):1-27.
[13] Long M,Wang J,Ding G,et al.Adaptation regularization:a general framework for transfer learning[J].IEEE Transactions on Knowledge and Data Engineering,2014,26(5):1076-1089.
[14] Long M,Wang J,Ding G,et al.Transfer feature learning with joint distribution adaptation[A].Proceedings of the IEEE International Conference on Computer Vision[C].USA:IEEE,2013.2200-2207.
[15] Gong B,Shi Y,Sha F,et al.Geodesic flow kernel for unsupervised domain adaptation[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2012.2066-2073.
[16] Daumé H III.Frustratingly easy domain adaptation[A].Proceedings of the Annual Meeting of the Association for Computational Linguistics[C].USA:ACL,2007.256-263.
[17] Li S,Song S J,Huang G.Prediction reweighting for domain adaptation[J].IEEE Transactions on Neural Networks and Learning Systems,2017,28(7):1682-1695.
[18] 张景祥,王士同.基于共同决策方向矢量的多源迁移及其快速学习方法[J].电子学报,2015,43(7):1349-1355.Zhang Jing-xiang,Wang Shi-tong.Common-decision-vectorbased multiple source transfer learning classification and its fast learning method[J].Acta Electronica Sinica,2015,43(7):1349-1355.(in Chinese)
[19] 蒋亦樟,邓赵红,王士同.0阶L2型TSK迁移学习模糊系统[J].电子学报,2013,41(5):897-904.Jiang Yi-zhang,Deng Zhao-hong,Wang Shi-tong.0-order-l2-norm-takagi-sugeno-kang type transfer learning fuzzy system[J].Acta Electronica Sinica,2013,41(5):897-904.(in Chinese)
[20] 倪彤光,王士同,应文豪,邓赵红.迁移组概率学习机[J].电子学报,2013,41(11):2207-2215.Ni Tong-guang,Wang Shi-tong,Ying Wen-hao,Deng Zhao-hong.Transfer group probabilities based learning machine[J].Acta Electronica Sinica,2013,41(11):2207-2215.(in Chinese)