基于对抗学习的讽刺识别研究
|
摘要
为了避免现有讽刺识别方法的性能会受训练数据缺乏的影响,在使用有限标注数据训练的注意力卷积神经网络基础上,提出一种对抗学习框架,该框架包含两种互补的对抗学习方法。首先,提出一种基于对抗样本的学习方法,应用对抗生成的样本参与模型训练,以期提高分类器的鲁棒性和泛化能力。进而,研究基于领域迁移的对抗学习方法,以期利用跨领域讽刺表达数据,改善模型在目标领域上的识别性能。在3个讽刺数据集上的实验结果表明,两种对抗学习方法都能提高讽刺识别的性能,其中基于领域迁移方法的性能提升更显著;同时结合两种对抗学习方法能够进一步提高讽刺识别性能。
Existing sarcasm detection approaches suffer from lack of sufficient training data. To address this problem, the authors propose an adversarial learning framework built on convolutional neural network(CNN) and attention mechanism, which is trained from limited amounts of labeled data. Two complementary adversarial learning approaches are investigated. First, by training with generated adversarial examples, the authors attempt to enhance the robustness and generalization ability of the classifier. Then, a domain transfer based adversarial learning approach is proposed to leverage cross-domain sarcasm data for improving the performance of sarcasm detection in the target domain. Experimental results on three sarcasm datasets show that both adversarial learning approaches proposed improve the performance of sarcasm detection, but the domain transfer based approach achieves higher performance. Combining the two proposed approaches further improves the performance of sarcasm detection.
Existing sarcasm detection approaches suffer from lack of sufficient training data. To address this problem, the authors propose an adversarial learning framework built on convolutional neural network(CNN) and attention mechanism, which is trained from limited amounts of labeled data. Two complementary adversarial learning approaches are investigated. First, by training with generated adversarial examples, the authors attempt to enhance the robustness and generalization ability of the classifier. Then, a domain transfer based adversarial learning approach is proposed to leverage cross-domain sarcasm data for improving the performance of sarcasm detection in the target domain. Experimental results on three sarcasm datasets show that both adversarial learning approaches proposed improve the performance of sarcasm detection, but the domain transfer based approach achieves higher performance. Combining the two proposed approaches further improves the performance of sarcasm detection.
引文
[1]Kreuz R J,Caucci G M.Lexical influences on the Perception of Sarcasm//Procceings of the Workshop on Computational Approaches to Figurative Language.New York,2007:1-4
[2]Carvalho P,Sarmento L,Silva M,et al.Clues for detecting irony in user-generated contents:oh...!!it’s“so easy”;-)//Proceedings of the 1st CIKM Workshop on Topic-sentiment Analysis for Mass Opinion.HongKong,2009:53-56
[3]Bamman D,Smith A N.Contextualized sarcasm detection on twitter//Proceedings of the International Association for the Advancement of Artificial Intelligence Conference on Weblogs and Social Media.Austin,2015:574-577
[4]刘龙飞,杨亮,张绍武,等.基于卷积神经网络的微博情感倾向性分析.中文信息学报,2015,29(6):159-165
[5]Goodfellow I J,Shlens J,Szegedy C.Explaining and harnessing adversarial examples//Proceedings of International Conference on Learning Representations.San Diego,2015:1-10
[6]Mnih V,Heess N,Graves A,et al.Recurrent models of visual attention//Proceedings of Conference on Neural Information Processing Systems.Montreal,2014:2204-2212
[7]Lin Z,Feng M,Santos C N D,et al.A structured selfattentive sentence embedding[EB/OL].(2017-03-09)[2018-04-01].https://arxiv.org/abs/1703.03130
[8]Kim Y.Convolutional neural networks for sentence classification//Proceedings of Empirical Methods in Natural Language Processing.Doha,2014:1746-1751
[9]Szegedy C,Zaremba W,Sutskever I,et al.Intriguing properties of neural networks[EB/OL].(2014-02-19)[2018-04-01].https://arxiv.org/abs/1312.6199
[10]Abbott R,Ecker B,Anand P,et al.Internet argument corpus 2.0:an SQL schema for dialogic social media and the corpora to go with it//Proceedings of the Tenth International Conference on Language Resources and Evaluation.Portoro,2016:4445-4452
[11]Zhang M,Zhang Y,Fu G.Tweet sarcasm detection using deep neural network//Proceedings of International Conference on Computational Linguistics.Lisbon,2016:2449-2460
[12]Chen T,Xu R,He Y,et al.Learning user and product distributed representations using a sequence model for sentiment analysis.IEEE Computational Intelligence Magazine,2016,11(3):34-44
[13]Gui L,Zhou Y,Xu R,et al.Learning representations from heterogeneous network for sentiment classification of product reviews.Knowledge Based Systems,2017,124:34-45
[14]Liebrecht C,Kunneman F,Bosch V A.The perfect solution for detecting sarcasm in tweets#not//Proceedings of the 4th Workshop on Comutational Approaches to Subjectivity,Sentiment and Social Media Analysis.Atlanta,2013:29-37
[15]Jia R,Liang P.Adversarial examples for evaluating reading comprehension systems//Procceedings of Empirical Methods in Natural Language Processing.Copenhagen,2017:2021-2031
[16]Tramer F,Kurakin A,Papernot N,et al.Ensemble adversarial training:attacks and defenses[EB/OL].(2018-01-30)[2018-04-01].https://arxiv.org/abs/1705.07204
[17]Miyato T,Dai A M,Goodfellow I.Adversarial training methods for semi-supervised text classification[EB/OL].(2016-11-07)[2018-04-01].https://arxiv.org/abs/1605.07725
[18]Wu Y,Bamman D,Russell S.Adversarial training for relation extraction//Proceedings of Conference on Empirical Methods in Natural Language Processing.Copenhagen,2017:1778-1783
[19]Zhao Z,Dua D,Singh S,et al.Generating natural adversarial examples[EB/OL].(2018-02-23)[2018-04-01].https://arxiv.org/abs/1710.11342
[20]Glorot X,Bordes A,Bengio Y.Domain adaptation for large-scale sentiment classification:a deep learning approach//Proceedings of International Conference on Machine Learning.Lille,2011:513-520
[21]Tzeng E,Hoffman J,Zhang N,et al.Deep domain confusion:maximizing for domain invariance[EB/OL].(2014-12-10)[2018-04-01].https://arxiv.org/abs/1412.3474
[22]Tzeng E,Hoffman J,Saenko K,et al,Adversarial discriminative domain adaptation//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,2017:2962-2971
[23]Ganin Y,Lempitsky V.Unsupervised domain adaptation by backpropagation//Proceedings of International Conference on Machine Learning.Lille,2015:1180-1189
[24]Gui L,Xu R,Lu Q,et al.Negative transfer detection in transductive transfer learning.International Journal of Machine Learning and Cybernetics,2018,9(2):185-197
[25]魏晓聪,林鸿飞.面向迁移学习的文本特征对齐算法.计算机工程,2017,43(2):215-219
[26]Pascanu R,Mikolov T,Bengio Y,et al.On the difficulty of training recurrent neural networks//Proceedings of International Conference on Machine Learning.Atlanta,2013:1310-1318
[27]Oraby S,Harrison V,Reed L,et al.Creating and characterizing a diverse corpus of sarcasm in dialogue//Proceedings of SIG dial Workshop on Discourse and Dialog.Los Angeles,2016:31-41
[28]Felbo B,Mislove A.Using millions of emoji occurrences to learn any-domain representations for detecting sentiment,emotion and sarcasm//Proceedings of Empirical Methods in Natural Language Processing.Copenhagen,2017:1615-1625
[2]Carvalho P,Sarmento L,Silva M,et al.Clues for detecting irony in user-generated contents:oh...!!it’s“so easy”;-)//Proceedings of the 1st CIKM Workshop on Topic-sentiment Analysis for Mass Opinion.HongKong,2009:53-56
[3]Bamman D,Smith A N.Contextualized sarcasm detection on twitter//Proceedings of the International Association for the Advancement of Artificial Intelligence Conference on Weblogs and Social Media.Austin,2015:574-577
[4]刘龙飞,杨亮,张绍武,等.基于卷积神经网络的微博情感倾向性分析.中文信息学报,2015,29(6):159-165
[5]Goodfellow I J,Shlens J,Szegedy C.Explaining and harnessing adversarial examples//Proceedings of International Conference on Learning Representations.San Diego,2015:1-10
[6]Mnih V,Heess N,Graves A,et al.Recurrent models of visual attention//Proceedings of Conference on Neural Information Processing Systems.Montreal,2014:2204-2212
[7]Lin Z,Feng M,Santos C N D,et al.A structured selfattentive sentence embedding[EB/OL].(2017-03-09)[2018-04-01].https://arxiv.org/abs/1703.03130
[8]Kim Y.Convolutional neural networks for sentence classification//Proceedings of Empirical Methods in Natural Language Processing.Doha,2014:1746-1751
[9]Szegedy C,Zaremba W,Sutskever I,et al.Intriguing properties of neural networks[EB/OL].(2014-02-19)[2018-04-01].https://arxiv.org/abs/1312.6199
[10]Abbott R,Ecker B,Anand P,et al.Internet argument corpus 2.0:an SQL schema for dialogic social media and the corpora to go with it//Proceedings of the Tenth International Conference on Language Resources and Evaluation.Portoro,2016:4445-4452
[11]Zhang M,Zhang Y,Fu G.Tweet sarcasm detection using deep neural network//Proceedings of International Conference on Computational Linguistics.Lisbon,2016:2449-2460
[12]Chen T,Xu R,He Y,et al.Learning user and product distributed representations using a sequence model for sentiment analysis.IEEE Computational Intelligence Magazine,2016,11(3):34-44
[13]Gui L,Zhou Y,Xu R,et al.Learning representations from heterogeneous network for sentiment classification of product reviews.Knowledge Based Systems,2017,124:34-45
[14]Liebrecht C,Kunneman F,Bosch V A.The perfect solution for detecting sarcasm in tweets#not//Proceedings of the 4th Workshop on Comutational Approaches to Subjectivity,Sentiment and Social Media Analysis.Atlanta,2013:29-37
[15]Jia R,Liang P.Adversarial examples for evaluating reading comprehension systems//Procceedings of Empirical Methods in Natural Language Processing.Copenhagen,2017:2021-2031
[16]Tramer F,Kurakin A,Papernot N,et al.Ensemble adversarial training:attacks and defenses[EB/OL].(2018-01-30)[2018-04-01].https://arxiv.org/abs/1705.07204
[17]Miyato T,Dai A M,Goodfellow I.Adversarial training methods for semi-supervised text classification[EB/OL].(2016-11-07)[2018-04-01].https://arxiv.org/abs/1605.07725
[18]Wu Y,Bamman D,Russell S.Adversarial training for relation extraction//Proceedings of Conference on Empirical Methods in Natural Language Processing.Copenhagen,2017:1778-1783
[19]Zhao Z,Dua D,Singh S,et al.Generating natural adversarial examples[EB/OL].(2018-02-23)[2018-04-01].https://arxiv.org/abs/1710.11342
[20]Glorot X,Bordes A,Bengio Y.Domain adaptation for large-scale sentiment classification:a deep learning approach//Proceedings of International Conference on Machine Learning.Lille,2011:513-520
[21]Tzeng E,Hoffman J,Zhang N,et al.Deep domain confusion:maximizing for domain invariance[EB/OL].(2014-12-10)[2018-04-01].https://arxiv.org/abs/1412.3474
[22]Tzeng E,Hoffman J,Saenko K,et al,Adversarial discriminative domain adaptation//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,2017:2962-2971
[23]Ganin Y,Lempitsky V.Unsupervised domain adaptation by backpropagation//Proceedings of International Conference on Machine Learning.Lille,2015:1180-1189
[24]Gui L,Xu R,Lu Q,et al.Negative transfer detection in transductive transfer learning.International Journal of Machine Learning and Cybernetics,2018,9(2):185-197
[25]魏晓聪,林鸿飞.面向迁移学习的文本特征对齐算法.计算机工程,2017,43(2):215-219
[26]Pascanu R,Mikolov T,Bengio Y,et al.On the difficulty of training recurrent neural networks//Proceedings of International Conference on Machine Learning.Atlanta,2013:1310-1318
[27]Oraby S,Harrison V,Reed L,et al.Creating and characterizing a diverse corpus of sarcasm in dialogue//Proceedings of SIG dial Workshop on Discourse and Dialog.Los Angeles,2016:31-41
[28]Felbo B,Mislove A.Using millions of emoji occurrences to learn any-domain representations for detecting sentiment,emotion and sarcasm//Proceedings of Empirical Methods in Natural Language Processing.Copenhagen,2017:1615-1625