基于多任务双向长短时记忆网络的隐式句间关系分析
|
摘要
隐式句间关系识别是篇章句间关系识别任务中一个重要的问题。由于隐式句间关系的语料没有较好的特征,目前该任务的识别仍不能达到很好的效果。隐式句间关系的语句和显式句间关系的语句在语义等方面有着一定的联系,为了充分利用这两个任务之间的联系,该论文使用多任务学习的方法,并使用双向长短时记忆(BiLSTM)网络学习语句的相关特征;同时,为充分利用文本的特征,采用融合词嵌入的方法并引入先验知识。与其他基于哈工大的中文篇章级语义关系语料库的实验结果表明,该文方法的平均F1值为53%,提升约13%;平均召回率(Recall)为51%,提升约9%。
Implicit discourse relation recognition is an important issue in the task of discourse relationship recognition.Nowadays,the corpus of implicit discourse relationship does not provide enough information for good results.To make full use of the fact that the sentences of implicit discourse and explicit discourse have some contact in semantic or some other aspects,this paper adopts multi-task learning method to handle the recognition task.The bidirectional long short term memory(Bi-LSTM)network is applied to learn the related features of the sentences.At the same time,the method of merging word vector has been adopted together with prior knowledge.Compared with other results,experiment results on the HIT-CDTB show that the average F1 score of this paper reaches 53%(about13%relative improvement),and the average recall score reaches 51%(about 9%relative improvement).
Implicit discourse relation recognition is an important issue in the task of discourse relationship recognition.Nowadays,the corpus of implicit discourse relationship does not provide enough information for good results.To make full use of the fact that the sentences of implicit discourse and explicit discourse have some contact in semantic or some other aspects,this paper adopts multi-task learning method to handle the recognition task.The bidirectional long short term memory(Bi-LSTM)network is applied to learn the related features of the sentences.At the same time,the method of merging word vector has been adopted together with prior knowledge.Compared with other results,experiment results on the HIT-CDTB show that the average F1 score of this paper reaches 53%(about13%relative improvement),and the average recall score reaches 51%(about 9%relative improvement).
引文
[1]张牧宇,宋原,秦兵,等.中文篇章级句间语义关系识别[J].中文信息学报,2013,27(6):51-58.
[2]张牧宇,秦兵,刘挺.中文篇章级句间语义关系体系及标注[J].中文信息学报,2014,28(2):28-36.
[3]徐凡,朱巧明,周国栋.基于树核的隐式篇章关系识别[J].软件学报,2013(5):1022-1035.
[4]王智强,李茹,梁吉业,等.基于汉语篇章框架语义分析的阅读理解问答研究[J].计算机学报,2016,39(4):795-807.
[5]姬建辉,张牧宇,秦兵,等.中文篇章级句间关系自动分析[J].江西师范大学学报(自然版),2015(2):124-131.
[6]Rutherford A T,Xue N.Discovering implicit discourse relations through Brown cluster pair representation and coreference patterns[C]//Proceedings of EACL,2014:645-654.
[7]Liu Y,Li S,Zhang X,et al.Implicit discourse relation classification via multi-task neural networks[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence,2016:2750-2756.
[8]R9nnqvist S,Schenk N,Chiarcos C.A recurrent neural model with attention for the recognition of Chinese implicit discourse relations[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics,2017:256-262.
[9]朱珊珊,洪宇,丁思远,等.面向不平衡数据的隐式篇章关系分类方法研究[J].中文信息学报,2015,29(6):110-118.
[10]Graves A.Supervised sequence labelling with recurrent neural networks[M].Springer Berlin Heidelberg,2012.
[11]Goodfellow I,Bengio Y,Courville A.Deep learning[M].Cambridge:MIT press,2016.
[12]Schuster M,Paliwal K K.Bidirectional recurrent neural networks[M].IEEE Press,1997.
[13]Bengio Y,Ducharme R,Vincent P,Jauvin C.A neural probabilistic language model[M].Innovations in Machine Learning.Springer Berlin Heidelberg,2006:137-186.
[14]Mikolov T,Sutskever I,Chen K,et al.Distributed representations of words and phrases and their compositionality[C]//Proceedings of International Conference on Neural Information Processing Systems.Curran Associates Inc,2013:3111-3119.
[15]Mikolov T,Chen K,Corrado G,et al.Efficient estimation of word representations in vector space[J].arXiv preprint arXiv:1301.3781,2013.
[16]Liu P,Qiu X,Huang X.Recurrent neural network for text classification with multi-task learning[C]//Proceedings of International Joint Conference on Artificial Intelligence.AAAI Press,2016:2873-2879.
[17]Liu P,Qiu X,Huang X.Adversarial multi-task learning for text classification[C]//Proceedings of ACL,2017.
[18]Qi Y,Collobert R,Kuksa P,et al.Combining labeled and unlabeled data with word-class distribution learning[C]//Proceedings of the 18th ACM conference on Information and Knowledge Management,2009:1737-1740.
[2]张牧宇,秦兵,刘挺.中文篇章级句间语义关系体系及标注[J].中文信息学报,2014,28(2):28-36.
[3]徐凡,朱巧明,周国栋.基于树核的隐式篇章关系识别[J].软件学报,2013(5):1022-1035.
[4]王智强,李茹,梁吉业,等.基于汉语篇章框架语义分析的阅读理解问答研究[J].计算机学报,2016,39(4):795-807.
[5]姬建辉,张牧宇,秦兵,等.中文篇章级句间关系自动分析[J].江西师范大学学报(自然版),2015(2):124-131.
[6]Rutherford A T,Xue N.Discovering implicit discourse relations through Brown cluster pair representation and coreference patterns[C]//Proceedings of EACL,2014:645-654.
[7]Liu Y,Li S,Zhang X,et al.Implicit discourse relation classification via multi-task neural networks[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence,2016:2750-2756.
[8]R9nnqvist S,Schenk N,Chiarcos C.A recurrent neural model with attention for the recognition of Chinese implicit discourse relations[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics,2017:256-262.
[9]朱珊珊,洪宇,丁思远,等.面向不平衡数据的隐式篇章关系分类方法研究[J].中文信息学报,2015,29(6):110-118.
[10]Graves A.Supervised sequence labelling with recurrent neural networks[M].Springer Berlin Heidelberg,2012.
[11]Goodfellow I,Bengio Y,Courville A.Deep learning[M].Cambridge:MIT press,2016.
[12]Schuster M,Paliwal K K.Bidirectional recurrent neural networks[M].IEEE Press,1997.
[13]Bengio Y,Ducharme R,Vincent P,Jauvin C.A neural probabilistic language model[M].Innovations in Machine Learning.Springer Berlin Heidelberg,2006:137-186.
[14]Mikolov T,Sutskever I,Chen K,et al.Distributed representations of words and phrases and their compositionality[C]//Proceedings of International Conference on Neural Information Processing Systems.Curran Associates Inc,2013:3111-3119.
[15]Mikolov T,Chen K,Corrado G,et al.Efficient estimation of word representations in vector space[J].arXiv preprint arXiv:1301.3781,2013.
[16]Liu P,Qiu X,Huang X.Recurrent neural network for text classification with multi-task learning[C]//Proceedings of International Joint Conference on Artificial Intelligence.AAAI Press,2016:2873-2879.
[17]Liu P,Qiu X,Huang X.Adversarial multi-task learning for text classification[C]//Proceedings of ACL,2017.
[18]Qi Y,Collobert R,Kuksa P,et al.Combining labeled and unlabeled data with word-class distribution learning[C]//Proceedings of the 18th ACM conference on Information and Knowledge Management,2009:1737-1740.