融合多层注意力机制与双向LSTM的语义关系抽取
|
摘要
关系抽取是构建如知识图谱等上层自然语言处理应用的基础。针对目前大多数关系抽取模型中忽略部分文本局部特征的问题,设计一种结合实体位置特征与多层注意力机制的双向LSTM网络结构。首先根据位置特征扩充字向量特征,并将文本信息向量化,然后将文本向量化信息输入双向LSTM模型,通过多层注意力机制,提高LSTM模型输入与输出之间的相关性,最后通过分类器输出关系获取结果。使用人工标注的百科类语料进行语义关系获取实验,结果表明,改进方法优于传统基于模式匹配的关系获取方法。
Relational extraction is the basis for constructing upper natural language processing applications such as knowledge graph.Because most of state-of-the-art systems ignore the importance of the local feature,in this paper,we design a bidirectional LSTM network structure which combines position eigenvector and multi-level attention mechanism. Firstly,the model embedded the text information by extending word vector feature which was based on positional features. Secondly,the information was introduced into the bidirectional LSTM model,and multi-level attention was used to improve the probability between the input and output of LSTM model. Finally,it obtained the result by classifier. The proposed method in this paper achieves an better result than the tradition method.
Relational extraction is the basis for constructing upper natural language processing applications such as knowledge graph.Because most of state-of-the-art systems ignore the importance of the local feature,in this paper,we design a bidirectional LSTM network structure which combines position eigenvector and multi-level attention mechanism. Firstly,the model embedded the text information by extending word vector feature which was based on positional features. Secondly,the information was introduced into the bidirectional LSTM model,and multi-level attention was used to improve the probability between the input and output of LSTM model. Finally,it obtained the result by classifier. The proposed method in this paper achieves an better result than the tradition method.
引文
[1]郭喜跃,何婷婷.信息抽取研究综述[J].计算机科学,2015,42(2):14-17,38.
[2]郭喜跃,何婷婷,胡小华,等.基于句法语义特征的中文实体关系抽取[J].中文信息学报,2014,28(6):183-189.
[3]台丽婷.基于半监督机器学习的实体关系抽取算法研究[D].北京:北京邮电大学,2018.
[4]陈立玮,冯岩松,赵东岩.基于弱监督学习的海量网络数据关系抽取[J].计算机研究与发展,2013,50(9):1825-1835.
[5]贾真,何大可,杨燕,等.基于弱监督学习的中文网络百科关系抽取[J].智能系统学报,2015(1):113-119.
[6]徐健,张智雄,吴振新.实体关系抽取的技术方法综述[J].现代图书情报技术,2008(8):18-23.
[7]李枫林,柯佳.基于深度学习框架的实体关系抽取研究进展[J].情报科学,2018,36(3):169-176.
[8]刘绍毓,李弼程,郭志刚,等.实体关系抽取研究综述[J].信息工程大学学报,2016,17(5):541-547.
[9]郭丽,刘磊.基于关系相似度计算的实体关系分类研究[J].软件导刊,2013,12(4):130-131.
[10]SOCHER R,HUVAL B,MANNING C D,et al.Semantic compositionality through recursive matrix-vector spaces[C].Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning,2012:1201-1211.
[11]LIU C Y,SUN W B,CHAO W H,et al.Convolution neural network for relation extraction[C].International Conference on Advanced Data Mining and Applications,2013:231-242.
[12]SUNDERMEYER M,SCHLüTER R,NEY H.LSTM neural networks for language modeling[EB/OL].http://www-i6.informatik.rwthaachen.de/publications/download/820/Sundermeyer-2012.pdf.
[13]GERS F A,SCHMIDHUBER J,CUMMINS F.Learning to forget:continual prediction with LSTM[J].Neural Computation,2014,12(10):2451-2471.
[14]YAN X,MOU L,LI G,et al.Classifying relations via long short term memory networks along shortest dependency path[J].Computer Science,2015,42(1):56-61.
[15]刘燊.面向《大词林》的中文实体关系挖掘[D].哈尔滨:哈尔滨工业大学,2016.
[16]TREISMAN A,SYKES M,GELADE G.Selective attention and stimulus integration[J].Attention and performance VI,1977,333:97-110.
[17]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].Computer Science,2014.
[18]HOCHREITER S.The vanishing gradient problem during learning recurrent neural nets and problem solutions[J].International Journal of Uncertainty,Fuzziness and Knowledge-Based,Systems,1998,6(2):107-116.
[19]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[C].International Conference on Neural Information Processing Systems.Curran Associates Inc,2013:3111-3119.
[20]BRIN S.Extracting patterns and relations from the World Wide Web[M].Berlin:Springer Berlin Heidelberg,1998.
[21]李梦瑶,向卓元.基于语音识别技术的移动全能秘书平台设计[J].软件导刊,2015,14(8):127-129.
[2]郭喜跃,何婷婷,胡小华,等.基于句法语义特征的中文实体关系抽取[J].中文信息学报,2014,28(6):183-189.
[3]台丽婷.基于半监督机器学习的实体关系抽取算法研究[D].北京:北京邮电大学,2018.
[4]陈立玮,冯岩松,赵东岩.基于弱监督学习的海量网络数据关系抽取[J].计算机研究与发展,2013,50(9):1825-1835.
[5]贾真,何大可,杨燕,等.基于弱监督学习的中文网络百科关系抽取[J].智能系统学报,2015(1):113-119.
[6]徐健,张智雄,吴振新.实体关系抽取的技术方法综述[J].现代图书情报技术,2008(8):18-23.
[7]李枫林,柯佳.基于深度学习框架的实体关系抽取研究进展[J].情报科学,2018,36(3):169-176.
[8]刘绍毓,李弼程,郭志刚,等.实体关系抽取研究综述[J].信息工程大学学报,2016,17(5):541-547.
[9]郭丽,刘磊.基于关系相似度计算的实体关系分类研究[J].软件导刊,2013,12(4):130-131.
[10]SOCHER R,HUVAL B,MANNING C D,et al.Semantic compositionality through recursive matrix-vector spaces[C].Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning,2012:1201-1211.
[11]LIU C Y,SUN W B,CHAO W H,et al.Convolution neural network for relation extraction[C].International Conference on Advanced Data Mining and Applications,2013:231-242.
[12]SUNDERMEYER M,SCHLüTER R,NEY H.LSTM neural networks for language modeling[EB/OL].http://www-i6.informatik.rwthaachen.de/publications/download/820/Sundermeyer-2012.pdf.
[13]GERS F A,SCHMIDHUBER J,CUMMINS F.Learning to forget:continual prediction with LSTM[J].Neural Computation,2014,12(10):2451-2471.
[14]YAN X,MOU L,LI G,et al.Classifying relations via long short term memory networks along shortest dependency path[J].Computer Science,2015,42(1):56-61.
[15]刘燊.面向《大词林》的中文实体关系挖掘[D].哈尔滨:哈尔滨工业大学,2016.
[16]TREISMAN A,SYKES M,GELADE G.Selective attention and stimulus integration[J].Attention and performance VI,1977,333:97-110.
[17]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].Computer Science,2014.
[18]HOCHREITER S.The vanishing gradient problem during learning recurrent neural nets and problem solutions[J].International Journal of Uncertainty,Fuzziness and Knowledge-Based,Systems,1998,6(2):107-116.
[19]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[C].International Conference on Neural Information Processing Systems.Curran Associates Inc,2013:3111-3119.
[20]BRIN S.Extracting patterns and relations from the World Wide Web[M].Berlin:Springer Berlin Heidelberg,1998.
[21]李梦瑶,向卓元.基于语音识别技术的移动全能秘书平台设计[J].软件导刊,2015,14(8):127-129.