Transfer Learning with Multiple Sources via Consensus Regularized Autoencoders
详细信息 查看全文
- 作者：Fuzhen Zhuang (23)
Xiaohu Cheng (23) (24)
Sinno Jialin Pan (25)
Wenchao Yu (23) (24)
Qing He (23)
Zhongzhi Shi (23) - 关键词：Transfer Learning ; Multiple Sources ; Consensus Regularization ; Feature Representation
- 刊名：Lecture Notes in Computer Science
- 出版年：2014
- 出版时间：2014
- 年：2014
- 卷：8726
- 期：1
- 页码：417-431
- 全文大小：480 KB
- 参考文献：1. Blitzer, J., Dredze, M., Pereira, F.: Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In: Proceedings of the 45th ACL (2007)
2. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the 5th AWCLT (1992)
3. Bourlard, H., Kamp, Y.: Auto-association by multilayer perceptrons and singular value decomposition. Biological Cybernetics (1988)
4. Chattopadhyay, R., Ye, J.P., Panchanathan, S., Fan, W., Davidson, I.: Multi-source domain adaptation and its application to early detection of fatigue. In: Proceedings of the 17th ACM SIGKDD, pp. 717鈥?25. ACM (2011)
5. Chen, M.M., Xu, Z.X., Weinberger, K., Sha, F.: Marginalized denoising autoencoders for domain adaptation. In: Proceedings of the 29th ICML (2012)
6. David, H., Stanley, L.: Applied Logistic Regression. Wiley, New York (2000)
7. Duan, L., Tsang, I.W., Xu, D., Chua, T.S.: Domain adaptation from multiple sources via auxiliary classifiers. In: Proceedings of the 26th ICML (2009)
8. Gao, J., Fan, W., Jiang, J., Han, J.W.: Knowledge transfer via multiple model local structure mapping. In: Proceedings of the 14th ACM SIGKDD (2008)
9. Ge, L., Gao, J., Zhang, A.D.: Oms-tl: a framework of online multiple source transfer learning. In: Proceedings of the 22nd ACM CIKM, pp. 2423鈥?428. ACM (2013)
10. Hinton, G.E., Zemel, R.S.: Autoencoders, minimum description length, and helmholtz free energy. In: Advances in NIPS (1994)
11. Hu, D.H., Zheng, V.W., Yang, Q.: Cross-domain activity recognition via transfer learning. Pervasive and Mobile Computing聽7(3), 344鈥?58 (2011) CrossRef
12. Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision Research (1997)
13. Pan, S.J., Kwok, J.T., Yang, Q.: Transfer learning via dimensionality reduction. In: Proceedings of the 23rd AAAI (2008)
14. Pan, S.J., Ni, X., Sun, J.T., Yang, Q., Chen, Z.: Cross-domain sentiment classification via spectral feature alignment. In: Proceedings of the 19th WWW (2010)
15. Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE TNN (2011)
16. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE TKDE (2010)
17. Poultney, C., Chopra, S., Cun, Y.L.: Efficient learning of sparse representations with an energy-based model. In: Advances in NIPS (2006)
18. Rifai, S., Vincent, P., Muller, X., Glorot, X., Bengio, Y.: Contractive auto-encoders: Explicit invariance during feature extraction. In: Proceedings of the 28th ICML (2011)
19. Shi, Z.P., Ye, F., He, Q., Shi, Z.Z.: Symmetrical invariant lbr texture descriptor and application for image retrieval. In: Congress on Image and Signal Processing (2008)
20. Si, S., Tao, D.C., Geng, B.: Bregman divergence-based regularization for transfer subspace learning. IEEE TKDE (2010)
21. Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th ICML (2008)
22. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. JMLR (2010)
23. Yu, W., Zeng, G., Luo, P., Zhuang, F., He, Q., Shi, Z.: Embedding with autoencoder regularization. In: Blockeel, H., Kersting, K., Nijssen, S., 沤elezn媒, F. (eds.) ECML PKDD 2013, Part III. LNCS, vol.聽8190, pp. 208鈥?23. Springer, Heidelberg (2013) CrossRef
24. Zhang, K., Zheng, V., Wang, Q.J., Kwok, J., Yang, Q., Marsic, I.: Covariate shift in hilbert space: A solution via sorrogate kernels. In: Proceedings of The 30th ICML, pp. 388鈥?95 (2013)
25. Zhang, L.: The Research on Human-computer Cooperation in Content-based Image Retrieval. Ph.D. thesis, Tsinghua University, Beijing (2001) (in Chinese)
26. Zhuang, F.Z., Luo, P., Xiong, H., He, Q., Xiong, Y.H., Shi, Z.Z.: Exploiting associations between word clusters and document classes for cross-domain text categorization. Statistical Analysis and Data Mining (2011)
27. Zhuang, F.Z., Luo, P., Xiong, H., Xiong, Y.H., He, Q., Shi, Z.Z.: Cross-domain learning from multiple sources: A consensus regularization perspective. IEEE TKDE (2010) - 作者单位：Fuzhen Zhuang (23)
Xiaohu Cheng (23) (24)
Sinno Jialin Pan (25)
Wenchao Yu (23) (24)
Qing He (23)
Zhongzhi Shi (23)
23. Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China
24. University of Chinese Academy of Sciences, Beijing, 100049, China
25. Institute for Infocomm Research, Singapore - ISSN：1611-3349
文摘
Knowledge transfer from multiple source domains to a target domain is crucial in transfer learning. Most existing methods are focused on learning weights for different domains based on the similarities between each source domain and the target domain or learning more precise classifiers from the source domain data jointly by maximizing their consensus of predictions on the target domain data. However, these methods only consider measuring similarities or building classifiers on the original data space, and fail to discover a more powerful feature representation of the data when transferring knowledge from multiple source domains to the target domain. In this paper, we propose a new framework for transfer learning with multiple source domains. Specifically, in the proposed framework, we adopt autoencoders to construct a feature mapping from an original instance to a hidden representation, and train multiple classifiers from the source domain data jointly by performing an entropy-based consensus regularizer on the predictions on the target domain. Based on the framework, a particular solution is proposed to learn the hidden representation and classifiers simultaneously. Experimental results on image and text real-world datasets demonstrate the effectiveness of our proposed method compared with state-of-the-art methods.