基于注意力机制的问句实体链接
|
摘要
问句实体链接不仅需要大量的数据处理和特征选择工作,而且容易形成错误累积,降低链接效果.针对这种情况,文中提出基于注意力机制的编码器-解码器问句实体链接模型.模型使用双向的长短期记忆网络编码问句,经过注意力机制解码,生成对应的实体指称和消歧信息输出,最后链接到知识库实体.在有关汽车领域车系产品问句和实体数据集上的实验表明,文中模型仅利用较少的上下文信息便可取得良好效果.
In question entity linking,a large amount of work in data processing and feature selection is required,cumulative errors are caused easily and the linking effect is reduced. To address the issues,an attention mechanism based encoder-decoder model for entity linking( AMEDEL) is proposed. In this model,long short-term memory network is utilized to encode the questions. Then,entity mentions and disambiguation information are generated as outputs through the decoder process by attention mechanism.Finally,these outputs are linked to the entities in knowledge base. The experiments are conducted on a dataset of questions and entities about products in automotive field. The results show that the proposed model obtains satisfactory results by only employing rare contextual information.
In question entity linking,a large amount of work in data processing and feature selection is required,cumulative errors are caused easily and the linking effect is reduced. To address the issues,an attention mechanism based encoder-decoder model for entity linking( AMEDEL) is proposed. In this model,long short-term memory network is utilized to encode the questions. Then,entity mentions and disambiguation information are generated as outputs through the decoder process by attention mechanism.Finally,these outputs are linked to the entities in knowledge base. The experiments are conducted on a dataset of questions and entities about products in automotive field. The results show that the proposed model obtains satisfactory results by only employing rare contextual information.
引文
[1] NADEAU D,SEKINE S. A Survey of Named Entity Recognition and Classification. Lingvisticae Investigationes,2007,30(1):3-26.
[2]BIKEL D M,SCHWARTZ R,WEISCHEDEL R M. An Algorithm That Learns What's in a Name. Machine Learning,1999,34(1/2/3):211-231.
[3]FRESKO M,ROSENFELD B,FELDMAN R. A Hybrid Approach to NER by MEMM and Manual Rules//Proc of the 14th ACM CIKM International Conference on Information and Knowledge Management. New York,USA:ACM,2005:361-362.
[4]MCCALLUM A,LI W. Early Results for Named Entity Recognition with Conditional Random Fields,Feature Induction and Web-Enhanced Lexicons//Proc of the 7th Conference on Natural Language Learning at HLT-NAACL. Stroudsburg,USA:ACL,2003,IV:188-191.
[5]ISOZAKI H,KAZAWA H. Efficient Support Vector Classifiers for Named Entity Recognition//Proc of the 19th International Conference on Computational linguistics. Stroudsburg, USA:ACL,2002,I:1-7.
[6]CARRERAS X,MARQUEZ L,PADROL. Named Entity Extraction Using Adaboost//Proc of the 6th Conference on Natural Language Learning. Stroudsburg,USA:ACL,2002:167-170.
[7]LAMPLE G,BALLESTEROS M,SUBRAMANIAN S,et al. Neural Architectures for Named Entity Recognition[C/OL].[2018-04-20]. https://arxiv. org/pdf/1603. 01360. pdf.
[8]XU J,LU Q,LIU J,et al. NLPComp in TAC 2012 Entity Linking and Slot-Filling[C/OL].[2018-04-20]. https://tac. nist. gov/publications/2012/participant. papers/NLPComp. proceedings. pdf.
[9]ZHANG Y Y,DAI H J,KOZAREVA Z,et al. Variational Reasoning for Question Answering with Knowledge Graph[C/OL].[2018-04-20]. https://arxiv. org/pdf/1709. 04071. pdf.
[10]怀宝兴,宝腾飞,祝恒书,等.一种基于概率主题模型的命名实体链接方法.软件学报,2014,25(9):2076-2087.(HUAI B X,BAO T F,ZHU H S,et al. Topic Modeling Approach to Named Entity Linking. Journal of Software,2014,25(9):2076-2087.)
[11] HAN X P,SUN L,ZHAO J. Collective Entity Linking in Web Text:A Graph-Based Method//Proc of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York,USA:ACM,2011:765-774.
[12] BAGGA A,BALDWIN B. Entity-Based Cross-Document Coreferencing Using the Vector Space Model//Proc of the 36th Annual Meeting of the Association for Computational Linguistics and the17th International Conference on Computational Linguistics.Stroudsburg,USA:ACL,1998,I:79-85.
[13] FRANCIS-LANDAU M,DURRETT G,KLEIN D. Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks//Proc of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg,USA:ACL,2016:1256-1261.
[14]齐爱芹,徐蔚然.基于词向量的实体链接方法.数据采集与处理,2017,32(3):604-611.(QI A Q,XU W R. Method of Entity Linking Based on Word Embedding. Journal of Data Acquisition and Processing,2017,32(3):604-611.)
[15]毛二松,王波,唐永旺,等.基于词向量的中文微博实体链接方法.计算机应用与软件,2017,34(4):11-15.(MAO E S,WANG B,TANG Y W,et al. Entity Linking Method of Chinese Micro-blog Based on Word Vector. Computer Applications and Software,2017,34(4):11-15.)
[16]冯冲,石戈,郭宇航,等.基于词向量语义分类的微博实体链接方法.自动化学报,2016,42(6):915-922.(FENG C,SHI G,GUO Y H,et al. An Entity Linking Method for Microblog Based on Semantic Categorization by Word Embeddings. Acta Automatica Sinica,2016,42(6):915-922.)
[17]BAHDANAU D,CHO K,BENGIO Y. Neural Machine Translation by Jointly Learning to Align and Translate[C/OL].[2018-04-20]. https://arxiv. org/pdf/1409. 0473. pdf.
[18]LUONG M T,PHAM H,MANNING C D. Effective Approaches to Attention-Based Neural Machine Translation//Proc of the Conference on Empirical Methods in Natural Language Processing.Stroudsburg,USA:ACL,2015:1412-1421.
[19]LIN Z H,FENG M W,DOS SANTOS C N,et al. A Structured Self-attentive Sentence Embedding[C/OL].[2018-04-20].https://arxiv. org/pdf/1703. 03130. pdf.
[20]DANILUK M,ROCKTASCHEL T,WELBL J,et al. Frustratingly Short Attention Spans in Neural Language Modeling[C/OL].[2018-04-20]. https://arxiv. org/pdf/1702. 04521. pdf.
[21]KADLEC R,SCHMID M,BAJGAR O,et al. Text Understanding with the Attention Sum Reader Network//Proc of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg,USA:ACL,2016,I:908-918
[22] PARIKH A,TACKSTROM O,DAS D,et al. A Decomposable Attention Model for Natural Language Inference//Proc of the Conference on Empirical Methods in Natural Language Processing.Stroudsburg,USA:ACL,2016:2249-2255.
[23] PAULUS R,XIONG C,SOCHER R. A Deep Reinforced Model for Abstractive Summarization[C/OL].[2018-04-20].https://arxiv. org/pdf/1705. 04304. pdf.
[24] SUTSKEVER I,VINYALS O,LE Q V. Sequence to Sequence Learning with Neural Networks//GHAHRAMANI Z,WELLING M,CORTES C,et al.,eds. Advances in Neural Information Processing Systems 27. Cambridge,USA:The MIT Press,2014:3104-3112.
[2]BIKEL D M,SCHWARTZ R,WEISCHEDEL R M. An Algorithm That Learns What's in a Name. Machine Learning,1999,34(1/2/3):211-231.
[3]FRESKO M,ROSENFELD B,FELDMAN R. A Hybrid Approach to NER by MEMM and Manual Rules//Proc of the 14th ACM CIKM International Conference on Information and Knowledge Management. New York,USA:ACM,2005:361-362.
[4]MCCALLUM A,LI W. Early Results for Named Entity Recognition with Conditional Random Fields,Feature Induction and Web-Enhanced Lexicons//Proc of the 7th Conference on Natural Language Learning at HLT-NAACL. Stroudsburg,USA:ACL,2003,IV:188-191.
[5]ISOZAKI H,KAZAWA H. Efficient Support Vector Classifiers for Named Entity Recognition//Proc of the 19th International Conference on Computational linguistics. Stroudsburg, USA:ACL,2002,I:1-7.
[6]CARRERAS X,MARQUEZ L,PADROL. Named Entity Extraction Using Adaboost//Proc of the 6th Conference on Natural Language Learning. Stroudsburg,USA:ACL,2002:167-170.
[7]LAMPLE G,BALLESTEROS M,SUBRAMANIAN S,et al. Neural Architectures for Named Entity Recognition[C/OL].[2018-04-20]. https://arxiv. org/pdf/1603. 01360. pdf.
[8]XU J,LU Q,LIU J,et al. NLPComp in TAC 2012 Entity Linking and Slot-Filling[C/OL].[2018-04-20]. https://tac. nist. gov/publications/2012/participant. papers/NLPComp. proceedings. pdf.
[9]ZHANG Y Y,DAI H J,KOZAREVA Z,et al. Variational Reasoning for Question Answering with Knowledge Graph[C/OL].[2018-04-20]. https://arxiv. org/pdf/1709. 04071. pdf.
[10]怀宝兴,宝腾飞,祝恒书,等.一种基于概率主题模型的命名实体链接方法.软件学报,2014,25(9):2076-2087.(HUAI B X,BAO T F,ZHU H S,et al. Topic Modeling Approach to Named Entity Linking. Journal of Software,2014,25(9):2076-2087.)
[11] HAN X P,SUN L,ZHAO J. Collective Entity Linking in Web Text:A Graph-Based Method//Proc of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York,USA:ACM,2011:765-774.
[12] BAGGA A,BALDWIN B. Entity-Based Cross-Document Coreferencing Using the Vector Space Model//Proc of the 36th Annual Meeting of the Association for Computational Linguistics and the17th International Conference on Computational Linguistics.Stroudsburg,USA:ACL,1998,I:79-85.
[13] FRANCIS-LANDAU M,DURRETT G,KLEIN D. Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks//Proc of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg,USA:ACL,2016:1256-1261.
[14]齐爱芹,徐蔚然.基于词向量的实体链接方法.数据采集与处理,2017,32(3):604-611.(QI A Q,XU W R. Method of Entity Linking Based on Word Embedding. Journal of Data Acquisition and Processing,2017,32(3):604-611.)
[15]毛二松,王波,唐永旺,等.基于词向量的中文微博实体链接方法.计算机应用与软件,2017,34(4):11-15.(MAO E S,WANG B,TANG Y W,et al. Entity Linking Method of Chinese Micro-blog Based on Word Vector. Computer Applications and Software,2017,34(4):11-15.)
[16]冯冲,石戈,郭宇航,等.基于词向量语义分类的微博实体链接方法.自动化学报,2016,42(6):915-922.(FENG C,SHI G,GUO Y H,et al. An Entity Linking Method for Microblog Based on Semantic Categorization by Word Embeddings. Acta Automatica Sinica,2016,42(6):915-922.)
[17]BAHDANAU D,CHO K,BENGIO Y. Neural Machine Translation by Jointly Learning to Align and Translate[C/OL].[2018-04-20]. https://arxiv. org/pdf/1409. 0473. pdf.
[18]LUONG M T,PHAM H,MANNING C D. Effective Approaches to Attention-Based Neural Machine Translation//Proc of the Conference on Empirical Methods in Natural Language Processing.Stroudsburg,USA:ACL,2015:1412-1421.
[19]LIN Z H,FENG M W,DOS SANTOS C N,et al. A Structured Self-attentive Sentence Embedding[C/OL].[2018-04-20].https://arxiv. org/pdf/1703. 03130. pdf.
[20]DANILUK M,ROCKTASCHEL T,WELBL J,et al. Frustratingly Short Attention Spans in Neural Language Modeling[C/OL].[2018-04-20]. https://arxiv. org/pdf/1702. 04521. pdf.
[21]KADLEC R,SCHMID M,BAJGAR O,et al. Text Understanding with the Attention Sum Reader Network//Proc of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg,USA:ACL,2016,I:908-918
[22] PARIKH A,TACKSTROM O,DAS D,et al. A Decomposable Attention Model for Natural Language Inference//Proc of the Conference on Empirical Methods in Natural Language Processing.Stroudsburg,USA:ACL,2016:2249-2255.
[23] PAULUS R,XIONG C,SOCHER R. A Deep Reinforced Model for Abstractive Summarization[C/OL].[2018-04-20].https://arxiv. org/pdf/1705. 04304. pdf.
[24] SUTSKEVER I,VINYALS O,LE Q V. Sequence to Sequence Learning with Neural Networks//GHAHRAMANI Z,WELLING M,CORTES C,et al.,eds. Advances in Neural Information Processing Systems 27. Cambridge,USA:The MIT Press,2014:3104-3112.