基于Wasserstein距离分层注意力模型的跨域情感分类
|
摘要
跨领域情感分类任务旨在利用已知情感标签的源域数据对缺乏标记数据的目标域进行情感倾向性分析.文中提出基于Wasserstein距离的分层注意力模型,结合Attention机制,采用分层模型进行特征提取,将Wasserstein距离作为域差异度量方式,通过对抗式训练自动捕获领域共享特征.进一步构造辅助任务捕获与共享特征共现的领域独有特征,结合两种特征表示完成跨域情感分类任务.在亚马逊评论等数据集上的实验表明,文中模型仅利用领域共享特征就达到较高的正确率,在不同的跨领域对之间具有较好的稳定性.
The task of cross-domain sentiment classification is to analyze the sentiment orientation of the target domain lacking labeled data using the source-domain data with sentiment labels. A hierarchical attention model based on Wasserstein distance is proposed in this paper. The hierarchical model is used for feature extraction by combining attention mechanism, and Wasserstein distance is used as the domain difference metric to automatically capture the domain-sharing features through adversarial training. Further auxiliary task is constructed to capture the domain-special features cooccurring with domain-sharing features. These two kinds of features are united to complete the cross-domain sentiment classification task. The experimental results on Amazon datasets demonstrate that the proposed model achieves a higher accuracy and a better stability on different cross-domain pairs.
The task of cross-domain sentiment classification is to analyze the sentiment orientation of the target domain lacking labeled data using the source-domain data with sentiment labels. A hierarchical attention model based on Wasserstein distance is proposed in this paper. The hierarchical model is used for feature extraction by combining attention mechanism, and Wasserstein distance is used as the domain difference metric to automatically capture the domain-sharing features through adversarial training. Further auxiliary task is constructed to capture the domain-special features cooccurring with domain-sharing features. These two kinds of features are united to complete the cross-domain sentiment classification task. The experimental results on Amazon datasets demonstrate that the proposed model achieves a higher accuracy and a better stability on different cross-domain pairs.
引文
[1] 赵传君,王素格,李德玉,等.基于分组提升集成的跨领域文本情感分类.计算机研究与发展,2015,52(3):629-638.(ZHAO C J,WANG S G,LI D Y,et al.Cross-Domain Text Sentiment Classification Based on Grouping-AdaBoost Ensemble.Journal of Computer Research and Development,2015,52(3):629-638.)
[2] 庄福振,罗平,何清,等.迁移学习研究进展.软件学报,2015,26(1):26-39.(ZHUANG F Z,LUO P,HE Q,et al.Survey on Transfer Learning Research.Journal of Software,2015,26(1):26-39.)
[3] PAN S J,YANG Q.A Survey on Transfer Learning.IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345-1359.
[4] 顾鑫,王士同,许敏.基于多源的跨领域数据分类快速新算法.自动化学报,2014,40(3):531-547.(GU X,WANG S T,XU M.A New Cross-multidomain Classification Algorithm and Its Fast Version for Large Datasets.Acta Automatica Sinica,2014,40(3):531-547.)
[5] BLITZER J,DREDZE M,PEREIA F.Biographies,Bollywood,Boom-Boxes and Blenders:Domain Adaption for Sentiment Classification // Proc of the 45th Annual Meeting of the Association for Computational Linguistics.Stroudsburg,USA:ACL,2007:440-447.
[6] PAN S J,NI X C,SUN J T,et al.Cross-Domain Sentiment Classification via Spectral Feature Alignment // Proc of the 19th International Conference on World Wide Web.New York,USA:ACM,2010:751-760.
[7] SHARMA R,BHATTACHARYYA P,DANDAPAT S,et al.Identifying Transferable Information across Domains for Cross-Domain Sentiment Classification // Proc of the 56th Annual Meeting of the Association for Computational Linguistics.Stroudsburg,USA:ACL,2018:968-978.
[8] WU F Z,HUANG Y F,YAN J.Active Sentiment Domain Adaptation // Proc of the 55th Annual Meeting of the Association for Computational Linguistics.Stroudsburg,USA:ACL,2017:1701-1711.
[9] GLOROT X,BORDES A,BENGIO Y.Domain Adaptation for Large-Scale Sentiment Classification:A Deep Learning Approach // Proc of the 28th International Conference on Machine Learning.Madison,USA:Omnipress,2011:513-520.
[10] CHEN M M,XU Z X,WEINBERGER K,et al.Marginalized Denoising Autoencoders for Domain Adaptation // Proc of the 29th International Conference on Machine Learning.Berlin,Germany:Springer,2012:767-774.
[11] ZISER Y,REICHART R.Neural Structural Correspondence Lear-ning for Domain Adaptation // Proc of the 21st Conference on Computational Natural Language Learning.Stroudsburg,USA:ACL,2017:400-410.
[12] ZISER Y,REICHART R.Pivot Based Language Modeling for Improved Neural Domain Adaption // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technologies.Stroudsburg,USA:ACL,2018:1241-1251.
[13] CUI X,AL-BAZZAZ N,BOLLEGALA D,et al.A Comparative Study of Pivot Selection Strategies for Unsupervised Cross-Domain Sentiment Classification.The Knowledge Engineering Review,2018,33.DOI:10.1017/S0269888918000085.
[14] GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-Adversarial Training of Neural Networks.Journal of Machine Learning Research,2015,17:2096-2030.
[15] YU J F,JIANG J.Leveraging Auxiliary Tasks for Document-Level Cross-Domain Sentiment Classification // Proc of the 8th International Joint Conference on Natural Language Processing.Stroudsburg,USA:ACL,2017:654-663.
[16] LI Z,ZHANG Y,WEI Y,et al.End-to-End Adversarial Memory Network for Cross-Domain Sentiment Classification // Proc of the 26th International Joint Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2017:2237-2243.
[17] LI Z,WEI Y,ZHANG Y,et al.Hierarchical Attention Transfer Network for Cross-Domain Sentiment Classification[C/OL].[2018-11-15].https://www.cse.ust.hk/～yuzhangcse/papers/Li_Wei_Zhang_Yang_AAAI18.pdf.
[18] TANG D Y,QIN B,LIU T.Document Modeling with Gated Recurrent Neural Network for Sentiment Classification // Proc of the Conference on Empirical Methods in Natural Language Processing.Stroudsburg,USA:ACL,2015:1422-1432.
[19] ZHOU P,SHI W,TIAN J,et al.Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification // Proc of the 54th Annual Meeting of the Association for Computational Linguistics.Stroudsburg,USA:ACL,2016:207-212.
[20] VASWANI A,SHAZEER N,PARMAR N,et al.Attention Is All You Need[C/OL].[2018-11-15].https://arxiv.org/pdf/1706.03762.pdf.
[21] MARTIN A,CHINTALA S,BOTTOU L.Wasserstein Gan[C/OL].[2018-11-15].https://arxiv.org/pdf/ 1701.07875.pdf.
[22] GULRAJANI I,AHMED F,ARJOVSKY M,et al.Improved Training of Wasserstein GANs[C/OL].[2018-11-15].https://arxiv.org/pdf/1704.00028.pdf.
[23] YU J F,JIANG J.Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification // Proc of the Conference on Empirical Methods in Natural Language Processing.Stroudsburg,USA:ACL,2016:236-246.
[2] 庄福振,罗平,何清,等.迁移学习研究进展.软件学报,2015,26(1):26-39.(ZHUANG F Z,LUO P,HE Q,et al.Survey on Transfer Learning Research.Journal of Software,2015,26(1):26-39.)
[3] PAN S J,YANG Q.A Survey on Transfer Learning.IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345-1359.
[4] 顾鑫,王士同,许敏.基于多源的跨领域数据分类快速新算法.自动化学报,2014,40(3):531-547.(GU X,WANG S T,XU M.A New Cross-multidomain Classification Algorithm and Its Fast Version for Large Datasets.Acta Automatica Sinica,2014,40(3):531-547.)
[5] BLITZER J,DREDZE M,PEREIA F.Biographies,Bollywood,Boom-Boxes and Blenders:Domain Adaption for Sentiment Classification // Proc of the 45th Annual Meeting of the Association for Computational Linguistics.Stroudsburg,USA:ACL,2007:440-447.
[6] PAN S J,NI X C,SUN J T,et al.Cross-Domain Sentiment Classification via Spectral Feature Alignment // Proc of the 19th International Conference on World Wide Web.New York,USA:ACM,2010:751-760.
[7] SHARMA R,BHATTACHARYYA P,DANDAPAT S,et al.Identifying Transferable Information across Domains for Cross-Domain Sentiment Classification // Proc of the 56th Annual Meeting of the Association for Computational Linguistics.Stroudsburg,USA:ACL,2018:968-978.
[8] WU F Z,HUANG Y F,YAN J.Active Sentiment Domain Adaptation // Proc of the 55th Annual Meeting of the Association for Computational Linguistics.Stroudsburg,USA:ACL,2017:1701-1711.
[9] GLOROT X,BORDES A,BENGIO Y.Domain Adaptation for Large-Scale Sentiment Classification:A Deep Learning Approach // Proc of the 28th International Conference on Machine Learning.Madison,USA:Omnipress,2011:513-520.
[10] CHEN M M,XU Z X,WEINBERGER K,et al.Marginalized Denoising Autoencoders for Domain Adaptation // Proc of the 29th International Conference on Machine Learning.Berlin,Germany:Springer,2012:767-774.
[11] ZISER Y,REICHART R.Neural Structural Correspondence Lear-ning for Domain Adaptation // Proc of the 21st Conference on Computational Natural Language Learning.Stroudsburg,USA:ACL,2017:400-410.
[12] ZISER Y,REICHART R.Pivot Based Language Modeling for Improved Neural Domain Adaption // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technologies.Stroudsburg,USA:ACL,2018:1241-1251.
[13] CUI X,AL-BAZZAZ N,BOLLEGALA D,et al.A Comparative Study of Pivot Selection Strategies for Unsupervised Cross-Domain Sentiment Classification.The Knowledge Engineering Review,2018,33.DOI:10.1017/S0269888918000085.
[14] GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-Adversarial Training of Neural Networks.Journal of Machine Learning Research,2015,17:2096-2030.
[15] YU J F,JIANG J.Leveraging Auxiliary Tasks for Document-Level Cross-Domain Sentiment Classification // Proc of the 8th International Joint Conference on Natural Language Processing.Stroudsburg,USA:ACL,2017:654-663.
[16] LI Z,ZHANG Y,WEI Y,et al.End-to-End Adversarial Memory Network for Cross-Domain Sentiment Classification // Proc of the 26th International Joint Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2017:2237-2243.
[17] LI Z,WEI Y,ZHANG Y,et al.Hierarchical Attention Transfer Network for Cross-Domain Sentiment Classification[C/OL].[2018-11-15].https://www.cse.ust.hk/～yuzhangcse/papers/Li_Wei_Zhang_Yang_AAAI18.pdf.
[18] TANG D Y,QIN B,LIU T.Document Modeling with Gated Recurrent Neural Network for Sentiment Classification // Proc of the Conference on Empirical Methods in Natural Language Processing.Stroudsburg,USA:ACL,2015:1422-1432.
[19] ZHOU P,SHI W,TIAN J,et al.Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification // Proc of the 54th Annual Meeting of the Association for Computational Linguistics.Stroudsburg,USA:ACL,2016:207-212.
[20] VASWANI A,SHAZEER N,PARMAR N,et al.Attention Is All You Need[C/OL].[2018-11-15].https://arxiv.org/pdf/1706.03762.pdf.
[21] MARTIN A,CHINTALA S,BOTTOU L.Wasserstein Gan[C/OL].[2018-11-15].https://arxiv.org/pdf/ 1701.07875.pdf.
[22] GULRAJANI I,AHMED F,ARJOVSKY M,et al.Improved Training of Wasserstein GANs[C/OL].[2018-11-15].https://arxiv.org/pdf/1704.00028.pdf.
[23] YU J F,JIANG J.Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification // Proc of the Conference on Empirical Methods in Natural Language Processing.Stroudsburg,USA:ACL,2016:236-246.