基于语言学扰动的事件检测数据增强方法
|
摘要
近年来,深度学习在事件检测领域取得了长足进展。但是,现有方法通常受制于事件检测标注数据的规模和训练阶段的不稳定性。针对上述问题,本文提出了基于语言学扰动的事件检测数据增强方法,从语法和语义两个角度生成伪数据来提升事件检测的性能。为了有效的利用生成的伪数据,该文探索了数据增加和多实例学习两个训练策略。在KBP 2017事件检测数据集上的实验验证了我们方法的有效性。此外,在人工构造的少量ACE2005数据集上的实验结果证明该文方法可以大幅度提升小数据情况下的模型学习性能。
Deep learning recently applied in the event detection task is limited by the scarcity of the annotated data and the instability during the training phase.This paper proposes a data augmentation method based on linguistic perturbation for event detection,which generates pseudo data from both syntactic and semantic perspectives to improve the performance of event detection systems.In order to effectively exploit generated pseudo data,this paper explores two training strategies:data addition and multi-instance learning.Experiments on the KBP 2017 event detection dataset demonstrate the effectiveness of our approach.Furthermore,the empirical results on a manual constructed portion of ACE2005 dataset show that the proposed method can significantly improve the model performance on small training data.
Deep learning recently applied in the event detection task is limited by the scarcity of the annotated data and the instability during the training phase.This paper proposes a data augmentation method based on linguistic perturbation for event detection,which generates pseudo data from both syntactic and semantic perspectives to improve the performance of event detection systems.In order to effectively exploit generated pseudo data,this paper explores two training strategies:data addition and multi-instance learning.Experiments on the KBP 2017 event detection dataset demonstrate the effectiveness of our approach.Furthermore,the empirical results on a manual constructed portion of ACE2005 dataset show that the proposed method can significantly improve the model performance on small training data.
引文
[1]Chen Y,Xu L,Liu K,et al.Event extraction via dynamic multipooling convolutional neural networks[C]//Proceedings of ACL 2015.Beijing, China:ACL,2015:167-176.
[2]Nguyen H,Cho K,Grishman R.Joint event extraction via recurrent neural networks[C]//Proceedings of NAACL-HLT 2016.San Diego, USA:ACL,2016:300-309.
[3]Li Q,Ji H,Huang L.Joint event extraction via structured prediction with global features[C]//Proceedings of ACL 2013.Soa,Bulgaria:ACL,2013:73-82.
[4]Chen Y,Liu S,Zhang X,et al.Automatically labeled data generation for large scale event extraction[C]//Proceedings of ACL 2017.Vancouver,Canada:ACL,2017:409-419.
[5]Liu S,Chen Y,He S,et al.Leveraging framenet to improve automatic event detection[C]//Proceedings of ACL 2016.Berlin,Germany:ACL,2016:2134-2143.
[6]Jiang Y,Zur M,Pesce L,et al.A study of the eect of noise injection on the training of artificial neural networks[C]//Proceedings of IJCNN 2013.Atlanta,USA:IEEE,2009:1428-1432.
[7]Goodfellow I,Shlens J,Szegedy C.Explaining and Harnessing Adversarial Examples[C]//Proceedings of ICLR 2015.Vancouver,Canada:2016.
[8]Zhao Z,Dua D,Singh S.Generating natural adversarial examples[C]//Proceedings of ICLR 2018.Vancouver,Canada:2018.
[9]Li Y,Cohn T,Baldwin T.Robust training under linguistic adversity[C]//Proceedings of EACL 2017Valencia,Spain:ACL,2017:21-27.
[10]Copestake A,Flickinger D.An open source grammar development environment and broad-coverage english grammar using hpsg[C]//Proceedings of LREC2000.Athens,Greece:2000:167-176.
[11]Copestake A,Flickinger D,Pollard C,et al.Minimal recursion semantics:An introduction[J].Research on Language and Computation 2015,3(2),281-332.
[12]Setiawan H,Dyer C,Resnik P.Discriminative word alignment with a function word reordering model[C]//Proceedings of EMNLP 2010.Massachusetts,USA:ACL,2010:534-544.
[13]Nguyen H,Grishman R.Event detection and domain adaptation with convolutional neural networks[C]//Proceedings of ACL 2015.Beijing,China:ACL,2015:365-371.
[14]Mrksi'cN,Séaghdha DO,Thomson B,et al.Counter-fitting word vectors to linguistic constraints[C]//Proceedings of NAACL-HLT 2016. San Diego,USA:ACL,2016:142-148.
[15]Mikolov T,Yih W,Zweig G.Linguistic regularities in continuous space word representations[C]//Proceedings of NAACL 2013.Atlanta,USA:ACL,2013:746-751.
[16]Heafield K,Pouzyrevsky I,Clark H,et al.Scalable modified kneserney language model estimation[C]//Proceedings of ACL 2013.Sofia,Bulgaria:ACL,2013:690-696.
[17]Zeng D,Liu K,Chen Y,et al.Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of EMNLP 2015.Lisbon,Portugal:ACL,2015:1753-1762.
[18]Zeiler D.Adadelta:An adaptive learning rate method[J].arXiv preprint arXiv:1212.5701,2012.
[19]Srivastava N,Hinton G,Krizhevsky A,et al.Dropout:A simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research,2014,15,1929-1958.
[20]Lin H,Lu Y,Han X,et al.Adaptive scaling for sparse detection in information extraction[C]//Proceedings of ACL 2018.Melbourne,Australia:ACL,2018:1033-1043.
[21]Yang B,Mitchell T.Leveraging knowledge bases inlstms for improving machine reading[C]//Proceedings of ACL 2017.Vancouver,Canada:ACL,2017:1436-1446.
[22]Lin H,Lu Y,Han X,et al.Nugget proposal networks for chinese event detection[C]//Proceedings of ACL 2018.Melbourne,Australia:ACL,2018:1565-1574.
(1)https://tac.nist.gov/2017/KBP/data.html
(2)http://sweaglesw.org/linguistics/ace
(1)http://jamesclarke.net/research/resources
(2)https://wordnet.princeton.edu
(3)https://catalog.ldc.upenn.edu/ldc2009t13
(1)https://code.google.com/archive/p/word2vec
(1)https://tac.nist.gov/2017/KBP
(2)https://stanfordnlp.github.io/CoreNLP
(3)https://github.com/hunterhector/EvmEval
(1)https://catalog.ldc.upenn.edu/ldc2006t06
[2]Nguyen H,Cho K,Grishman R.Joint event extraction via recurrent neural networks[C]//Proceedings of NAACL-HLT 2016.San Diego, USA:ACL,2016:300-309.
[3]Li Q,Ji H,Huang L.Joint event extraction via structured prediction with global features[C]//Proceedings of ACL 2013.Soa,Bulgaria:ACL,2013:73-82.
[4]Chen Y,Liu S,Zhang X,et al.Automatically labeled data generation for large scale event extraction[C]//Proceedings of ACL 2017.Vancouver,Canada:ACL,2017:409-419.
[5]Liu S,Chen Y,He S,et al.Leveraging framenet to improve automatic event detection[C]//Proceedings of ACL 2016.Berlin,Germany:ACL,2016:2134-2143.
[6]Jiang Y,Zur M,Pesce L,et al.A study of the eect of noise injection on the training of artificial neural networks[C]//Proceedings of IJCNN 2013.Atlanta,USA:IEEE,2009:1428-1432.
[7]Goodfellow I,Shlens J,Szegedy C.Explaining and Harnessing Adversarial Examples[C]//Proceedings of ICLR 2015.Vancouver,Canada:2016.
[8]Zhao Z,Dua D,Singh S.Generating natural adversarial examples[C]//Proceedings of ICLR 2018.Vancouver,Canada:2018.
[9]Li Y,Cohn T,Baldwin T.Robust training under linguistic adversity[C]//Proceedings of EACL 2017Valencia,Spain:ACL,2017:21-27.
[10]Copestake A,Flickinger D.An open source grammar development environment and broad-coverage english grammar using hpsg[C]//Proceedings of LREC2000.Athens,Greece:2000:167-176.
[11]Copestake A,Flickinger D,Pollard C,et al.Minimal recursion semantics:An introduction[J].Research on Language and Computation 2015,3(2),281-332.
[12]Setiawan H,Dyer C,Resnik P.Discriminative word alignment with a function word reordering model[C]//Proceedings of EMNLP 2010.Massachusetts,USA:ACL,2010:534-544.
[13]Nguyen H,Grishman R.Event detection and domain adaptation with convolutional neural networks[C]//Proceedings of ACL 2015.Beijing,China:ACL,2015:365-371.
[14]Mrksi'cN,Séaghdha DO,Thomson B,et al.Counter-fitting word vectors to linguistic constraints[C]//Proceedings of NAACL-HLT 2016. San Diego,USA:ACL,2016:142-148.
[15]Mikolov T,Yih W,Zweig G.Linguistic regularities in continuous space word representations[C]//Proceedings of NAACL 2013.Atlanta,USA:ACL,2013:746-751.
[16]Heafield K,Pouzyrevsky I,Clark H,et al.Scalable modified kneserney language model estimation[C]//Proceedings of ACL 2013.Sofia,Bulgaria:ACL,2013:690-696.
[17]Zeng D,Liu K,Chen Y,et al.Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of EMNLP 2015.Lisbon,Portugal:ACL,2015:1753-1762.
[18]Zeiler D.Adadelta:An adaptive learning rate method[J].arXiv preprint arXiv:1212.5701,2012.
[19]Srivastava N,Hinton G,Krizhevsky A,et al.Dropout:A simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research,2014,15,1929-1958.
[20]Lin H,Lu Y,Han X,et al.Adaptive scaling for sparse detection in information extraction[C]//Proceedings of ACL 2018.Melbourne,Australia:ACL,2018:1033-1043.
[21]Yang B,Mitchell T.Leveraging knowledge bases inlstms for improving machine reading[C]//Proceedings of ACL 2017.Vancouver,Canada:ACL,2017:1436-1446.
[22]Lin H,Lu Y,Han X,et al.Nugget proposal networks for chinese event detection[C]//Proceedings of ACL 2018.Melbourne,Australia:ACL,2018:1565-1574.
(1)https://tac.nist.gov/2017/KBP/data.html
(2)http://sweaglesw.org/linguistics/ace
(1)http://jamesclarke.net/research/resources
(2)https://wordnet.princeton.edu
(3)https://catalog.ldc.upenn.edu/ldc2009t13
(1)https://code.google.com/archive/p/word2vec
(1)https://tac.nist.gov/2017/KBP
(2)https://stanfordnlp.github.io/CoreNLP
(3)https://github.com/hunterhector/EvmEval
(1)https://catalog.ldc.upenn.edu/ldc2006t06