基于触发词语义选择的Twitter事件共指消解研究
|
摘要
随着社交媒体的发展与普及,如何识别短文本中事件描述的共指关系已成为一个亟待解决的问题。在传统的事件共指消解研究中,需要从NLP工具和知识库中获得丰富的语义特征,这种方式不仅限制了领域的扩展性,而且还导致了误差传播。为了打破上述局限,提出了一种新颖的基于事件触发词来选择性表达句子语义的方法,以判断短文本中事件的共指关系。首先,利用双向长短记忆模型(Bi-LSTM)提取短文本的句子级语义特征和事件描述级语义特征;其次,通过在句子级特征上应用一个基于事件触发词的选择门来选择性表达句子级语义,以产生潜在语义特征;然后,设计了触发词重叠词数和时间间隔两个辅助特征;最后,通过融合以上特征形成一个分类器来预测共指关系。为评估上述方法,基于Twitter数据标注了一个新的数据集EventCoreOnTweets(ECT)。实验结果表明,与两个基准模型相比,提出的选择性表达模型显著提升了短文本共指消解的性能。
With the development and popularization of social media,how to recognize the coreference relation between two event mention in short texts is an urgent issue.In traditional researches about event coreference resolution,a rich set of linguistic features derived from pre-existing NLP tools and various knowledge bases is required,which restricts domain scalability and leads to the propagation of errors.To overcome these limitations,this paper proposed a novel selective expression approach based on event trigger to explore the coreference relationship on Twitter.Firstly,a bi-direction long short term memory(Bi-LSTM)is exploited to extract the features at sentence level and at mention level.Then,the latent features are generated by applying agate on sentence level features to make it selectively express.Next,two auxiliary features named the overlapped words of trigger and time interval are designed.Finally,all these features are concatenated and fed into a simple classifier to predict the coreference relationship.In order to evaluate this method,this paper annotated a new dataset EventCoreOnTweet(ECT).The experimental results demonstrate that the selective expression approach significantly improves the performance of coreference resolution of short texts.
With the development and popularization of social media,how to recognize the coreference relation between two event mention in short texts is an urgent issue.In traditional researches about event coreference resolution,a rich set of linguistic features derived from pre-existing NLP tools and various knowledge bases is required,which restricts domain scalability and leads to the propagation of errors.To overcome these limitations,this paper proposed a novel selective expression approach based on event trigger to explore the coreference relationship on Twitter.Firstly,a bi-direction long short term memory(Bi-LSTM)is exploited to extract the features at sentence level and at mention level.Then,the latent features are generated by applying agate on sentence level features to make it selectively express.Next,two auxiliary features named the overlapped words of trigger and time interval are designed.Finally,all these features are concatenated and fed into a simple classifier to predict the coreference relationship.In order to evaluate this method,this paper annotated a new dataset EventCoreOnTweet(ECT).The experimental results demonstrate that the selective expression approach significantly improves the performance of coreference resolution of short texts.
引文
[1]BEJAN C A,HARABAGIU S.Unsupervised event coreference resolution with rich linguistic features[C]∥Meeting of the Association for Computational Linguistics.Association for Computational Linguistics,2010:1412-1422.
[2]HOVY E,MITAMURA T,VERDEJO F,et al.Events are not simple:Identity,non-identity,and quasi-identity[OL].http://aclweb.org/anthology/w13-1203.
[3]ALLAN J.Topic Detection and Tracking Pilot Study:Final Report[C]∥Proceedings of DARPA Broadcast News Transcription and Understanding Workshop.1998:194-218.
[4]HUMPHREYS K,GAIZAUSKAS R,AZZAM S.Event coreference for information extraction[C]∥A Workshop on Operational Factors in Practical,Robust Anaphora Resolution for Unrestricted Texts.Association for Computational Linguistics,1997:75-81.
[5]TELLEX S,KATZ B,LIN J,et al.Quantitative evaluation of passage retrieval algorithms for question answering[C]∥International ACM SIGIR Conference on Research and Development in Informaion Retrieval.ACM,2003:41-47.
[6]MCCARTHY D,CARROLL J.Disambiguating Nouns,Verbs,and Adjectives Using Automatically Acquired Selectional Preferences[J].Computational Linguistics,2003,29(4):639-654.
[7]ZENG D,LIU K,LAI S,et al.Relation classification via convolutional deep neural network[C]∥Proceedings of COLING2014,the 25th International Conference on Computational Linguistics:Technical Papers.2014:2335-2344.
[8]NGUYEN T H,GRISHMAN R.Event Detection and Domain Adaptation with Convolutional Neural Networks[C]∥Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing(Volume 2:Short Papers).2015:365-371.
[9]CHEN Y,XU L,LIU K,et al.Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks[C]∥The Meeting of the Association for Computational Linguistics.2015.
[10]KRAUSE S,XU F,USZKOREIT H,et al.Event Linking with Sentential Features from Convolutional Neural Networks[C]∥Signll Conference on Computational Natural Language Learning.2016:239-249.
[11]HAGHIGHI A,DAN K.Coreference resolution in a modular,entity-centered model[C]∥Human Language Technologies:the2010Conference of the North American Chapter of the Association for Computational Linguistics.Association for Computational Linguistics,2010:385-393.
[12]RAHMAN A,NG V.Coreference Resolution with World Knowledge[C]∥The Meeting of the Association for Computational Linguistics:Human Language Technologies.2011:814-824.
[13]RAO D,MCNAMEE P,DREDZE M.Streaming Cross Document Entity Coreference Resolution[C]∥International Conference on Coling 2010.2010:1050-1058.
[14]MNIH V,HEESS N,Graves A.Recurrent models of visual attention[C]∥Advances in neural information processing systems.2014:2204-2212.
[15]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv preprint arXiv:1409.0473,2014.
[16]BAGGA A,BALDWIN B.Cross-document coreference:Annotations,Experiments,and Observations[C]∥Proceedings of ACL-99 Workshop on Coreference and Its Applications.1999:1-8.
[17]CHEN Z,JI H,HARALICK R.A pairwise event coreference model,feature impact and evaluation for event coreference resolution[C]∥The Workshop on Events in Emerging Text Types.Association for Computational Linguistics,2009:17-22.
[18]CHEN Z,JI H.Graph-based event coreference resolution[C]∥The Workshop on Graph-Based Methods for Natural Language Processing.Association for Computational Linguistics,2009:54-57.
[19]LIU Z,ARAKI J,HOVY E H,et al.Supervised Within-Document Event Coreference using Information Propagation[OL].http://www.lrec-conf.org/proceedings/lrec 2014/pdf/646_paper.pdf.
[20]PENG H,SONG Y,DAN R.Event Detection and Co-reference with Minimal Supervision[C]∥Conference on Empirical Methods in Natural Language Processing.2016:392-402.
[21]TEH Y W,JORDAN M I,BEAL M J,et al.Hierarchical Dirichlet Processes[J].Publications of the American Statistical Association,2006,101(476):1566-1581.
[22]GAEL J V,TEH Y W,GHAHRAMANI Z.The infinite factorial hidden Markov model[C]∥International Conference on Neural Information Processing Systems.Curran Associates Inc.,2008:1697-1704.
[23]YANG B,CARDIE C,FRAZIER P.A Hierarchical Distance-dependent Bayesian Model for Event Coreference Resolution[J].arXiv:1504.05929,2015.
[24]BLEI D M,FRAZIER P I.Distance Dependent Chinese Restaurant Processes[J].Journal of Machine Learning Research,2011,12(1):2461-2488.
[25]LEE H,RECASENS M,CHANG A,et al.Joint entity and event coreference resolution across documents[C]∥Proceedings of the2012Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.Association for Computational Linguistics,2012:489-500.
[26]PRADHAN S S,RAMSHAW L,WEISCHEDEL R,et al.Unrestricted coreference:Identifying entities and events in OntoNotes[C]∥International Conference on Semantic Computing.IEEEComputer Society,2007:446-453.
[27]ARAKI J,LIU Z,HOVY E H,et al.Detecting Subevent Structure for Event Coreference Resolution[C]∥International Conference on Language Resource and Evaluation.2014:4553-4558.
[28]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[C]∥International Conference on Neural Information Processing Systems.Curran Associates Inc.,2013:3111-3119.
[29]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[30]SUTSKEVER I,VINYALS O,LE Q V.Sequence to sequence learning with neural networks[C]∥Advances in neural information processing systems.2014:3104-3112.
[31]WU Y,SCHUSTER M,CHEN Z,et al.Google’s neural machine translation system:Bridging the gap between human and machine translation[J].arXiv preprint arXiv:1609.08144,2016.
[32]KINGMA D P,BA J.A method for stochastic optimization[J].arXiv preprint arXiv:1412.6980.2014.
[33]COHEN J.A coefficient of agreement for nominal scales[J].Educational&Psychological Measurement,2016,20(1):37-46.
[34]VILAIN M,BURGER J,ABERDEEN J,et al.A Model-Theoretic Coreferenc e Scoring Scheme[C]∥Conference on Message Understanding,Muc 1995,Columbia,Maryland,Usa,November.DBLP,1995:45-52.
[35]BAGGA A,BALDWIN B.Algorithms for scoring coreference chains[C]∥The First International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference.1998:563-566.
[36]RECASENS M,HOVY E.BLANC:Implementing the Rand index for coreference evaluation[J].Natural Language Engineering,2011,17(4):485-510.
[37]LUO X.On coreference resolution performance metrics[C]∥HLT/EMNLP 2005,Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing,Proceedings of the Conference,Vancouver,British Columbia,Canada.DBLP,2005:25-32.
[38]PRADHAN S,LUO X,RECASENS M,et al.Scoring Coreference Partitions of Predicted Mentions:A Reference Implementation[C]∥Meeting of the Association for Computational Linguistics.2014:30.
1)http://projects.ldc.upenn.edu/ace/
1)http://www.nist.gov/speech/tests/ace/
1)https://dev.twitter.com/docs/streaming-api
2 )https://github.com/conll/reference-coreference-scorers
3 )https://github.com/tensorflow/tensorflow
[2]HOVY E,MITAMURA T,VERDEJO F,et al.Events are not simple:Identity,non-identity,and quasi-identity[OL].http://aclweb.org/anthology/w13-1203.
[3]ALLAN J.Topic Detection and Tracking Pilot Study:Final Report[C]∥Proceedings of DARPA Broadcast News Transcription and Understanding Workshop.1998:194-218.
[4]HUMPHREYS K,GAIZAUSKAS R,AZZAM S.Event coreference for information extraction[C]∥A Workshop on Operational Factors in Practical,Robust Anaphora Resolution for Unrestricted Texts.Association for Computational Linguistics,1997:75-81.
[5]TELLEX S,KATZ B,LIN J,et al.Quantitative evaluation of passage retrieval algorithms for question answering[C]∥International ACM SIGIR Conference on Research and Development in Informaion Retrieval.ACM,2003:41-47.
[6]MCCARTHY D,CARROLL J.Disambiguating Nouns,Verbs,and Adjectives Using Automatically Acquired Selectional Preferences[J].Computational Linguistics,2003,29(4):639-654.
[7]ZENG D,LIU K,LAI S,et al.Relation classification via convolutional deep neural network[C]∥Proceedings of COLING2014,the 25th International Conference on Computational Linguistics:Technical Papers.2014:2335-2344.
[8]NGUYEN T H,GRISHMAN R.Event Detection and Domain Adaptation with Convolutional Neural Networks[C]∥Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing(Volume 2:Short Papers).2015:365-371.
[9]CHEN Y,XU L,LIU K,et al.Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks[C]∥The Meeting of the Association for Computational Linguistics.2015.
[10]KRAUSE S,XU F,USZKOREIT H,et al.Event Linking with Sentential Features from Convolutional Neural Networks[C]∥Signll Conference on Computational Natural Language Learning.2016:239-249.
[11]HAGHIGHI A,DAN K.Coreference resolution in a modular,entity-centered model[C]∥Human Language Technologies:the2010Conference of the North American Chapter of the Association for Computational Linguistics.Association for Computational Linguistics,2010:385-393.
[12]RAHMAN A,NG V.Coreference Resolution with World Knowledge[C]∥The Meeting of the Association for Computational Linguistics:Human Language Technologies.2011:814-824.
[13]RAO D,MCNAMEE P,DREDZE M.Streaming Cross Document Entity Coreference Resolution[C]∥International Conference on Coling 2010.2010:1050-1058.
[14]MNIH V,HEESS N,Graves A.Recurrent models of visual attention[C]∥Advances in neural information processing systems.2014:2204-2212.
[15]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv preprint arXiv:1409.0473,2014.
[16]BAGGA A,BALDWIN B.Cross-document coreference:Annotations,Experiments,and Observations[C]∥Proceedings of ACL-99 Workshop on Coreference and Its Applications.1999:1-8.
[17]CHEN Z,JI H,HARALICK R.A pairwise event coreference model,feature impact and evaluation for event coreference resolution[C]∥The Workshop on Events in Emerging Text Types.Association for Computational Linguistics,2009:17-22.
[18]CHEN Z,JI H.Graph-based event coreference resolution[C]∥The Workshop on Graph-Based Methods for Natural Language Processing.Association for Computational Linguistics,2009:54-57.
[19]LIU Z,ARAKI J,HOVY E H,et al.Supervised Within-Document Event Coreference using Information Propagation[OL].http://www.lrec-conf.org/proceedings/lrec 2014/pdf/646_paper.pdf.
[20]PENG H,SONG Y,DAN R.Event Detection and Co-reference with Minimal Supervision[C]∥Conference on Empirical Methods in Natural Language Processing.2016:392-402.
[21]TEH Y W,JORDAN M I,BEAL M J,et al.Hierarchical Dirichlet Processes[J].Publications of the American Statistical Association,2006,101(476):1566-1581.
[22]GAEL J V,TEH Y W,GHAHRAMANI Z.The infinite factorial hidden Markov model[C]∥International Conference on Neural Information Processing Systems.Curran Associates Inc.,2008:1697-1704.
[23]YANG B,CARDIE C,FRAZIER P.A Hierarchical Distance-dependent Bayesian Model for Event Coreference Resolution[J].arXiv:1504.05929,2015.
[24]BLEI D M,FRAZIER P I.Distance Dependent Chinese Restaurant Processes[J].Journal of Machine Learning Research,2011,12(1):2461-2488.
[25]LEE H,RECASENS M,CHANG A,et al.Joint entity and event coreference resolution across documents[C]∥Proceedings of the2012Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.Association for Computational Linguistics,2012:489-500.
[26]PRADHAN S S,RAMSHAW L,WEISCHEDEL R,et al.Unrestricted coreference:Identifying entities and events in OntoNotes[C]∥International Conference on Semantic Computing.IEEEComputer Society,2007:446-453.
[27]ARAKI J,LIU Z,HOVY E H,et al.Detecting Subevent Structure for Event Coreference Resolution[C]∥International Conference on Language Resource and Evaluation.2014:4553-4558.
[28]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[C]∥International Conference on Neural Information Processing Systems.Curran Associates Inc.,2013:3111-3119.
[29]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[30]SUTSKEVER I,VINYALS O,LE Q V.Sequence to sequence learning with neural networks[C]∥Advances in neural information processing systems.2014:3104-3112.
[31]WU Y,SCHUSTER M,CHEN Z,et al.Google’s neural machine translation system:Bridging the gap between human and machine translation[J].arXiv preprint arXiv:1609.08144,2016.
[32]KINGMA D P,BA J.A method for stochastic optimization[J].arXiv preprint arXiv:1412.6980.2014.
[33]COHEN J.A coefficient of agreement for nominal scales[J].Educational&Psychological Measurement,2016,20(1):37-46.
[34]VILAIN M,BURGER J,ABERDEEN J,et al.A Model-Theoretic Coreferenc e Scoring Scheme[C]∥Conference on Message Understanding,Muc 1995,Columbia,Maryland,Usa,November.DBLP,1995:45-52.
[35]BAGGA A,BALDWIN B.Algorithms for scoring coreference chains[C]∥The First International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference.1998:563-566.
[36]RECASENS M,HOVY E.BLANC:Implementing the Rand index for coreference evaluation[J].Natural Language Engineering,2011,17(4):485-510.
[37]LUO X.On coreference resolution performance metrics[C]∥HLT/EMNLP 2005,Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing,Proceedings of the Conference,Vancouver,British Columbia,Canada.DBLP,2005:25-32.
[38]PRADHAN S,LUO X,RECASENS M,et al.Scoring Coreference Partitions of Predicted Mentions:A Reference Implementation[C]∥Meeting of the Association for Computational Linguistics.2014:30.
1)http://projects.ldc.upenn.edu/ace/
1)http://www.nist.gov/speech/tests/ace/
1)https://dev.twitter.com/docs/streaming-api
2 )https://github.com/conll/reference-coreference-scorers
3 )https://github.com/tensorflow/tensorflow