基于注意力机制的命名实体识别模型研究——以军事文本为例
|
摘要
针对双向长短时记忆网络模型提取特征不充分的特点,将字向量和词向量同时作为双向长短时记忆网络的输入,并利用注意力机制分别提取两者对当前输出有用的特征,用维特比算法约束最终输出的标签序列,构建一种新的命名实体识别模型。实验结果表明,在军事文本的命名实体识别中,该模型取得了较优的识别率。
Due to the insufficiency of extracting features by bi-directional long-short term memory network model,the character vector and the word vector are used as the input and the attention mechanism is used to extract the features that are useful for the current output.In this paper,a new named entity recognition model was constructed by constraining the final output tag sequence with the Viterbi algorithm.The experimental results show that the model has achieved a better recognition rate in the identification of military texts.
Due to the insufficiency of extracting features by bi-directional long-short term memory network model,the character vector and the word vector are used as the input and the attention mechanism is used to extract the features that are useful for the current output.In this paper,a new named entity recognition model was constructed by constraining the final output tag sequence with the Viterbi algorithm.The experimental results show that the model has achieved a better recognition rate in the identification of military texts.
引文
[1] 俞鸿魁,张华平,刘群,等.基于层叠隐马尔可夫模型的中文命名实体识别[J].通信学报,2006(2):87-94.
[2] 胡文博,都云程,吕学强,等.基于多层条件随机场的中文命名实体识别[J].计算机工程与应用,2009,45(1):163-165,227.
[3] PASSOS A,KUMAR V,MCCALLUM A.Lexicon Infused Phrase Embeddings for Named Entity Resolution[C]//Proceeding of the Eighteenth Conference on Computational Language Learning,2014:78-86.
[4] CHIU J P C,NICHOLS E.Named Entity Recognition with Bidirectional LSTM-CNNs[J].ArXiv:1511.08308.
[5] COLLOBERT R,WESTON J,KARLEN M,et al.Natural Language Processing(Almost) from Scratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[6] 冯艳红,于红,孙庚,等.基于BLSTM的命名实体识别方法[J].计算机科学,2018,45(2):261-268.
[7] 王蕾.基于神经网络的中文命名实体识别研究[D].南京:南京师范大学,2017.
[8] MNIH V,HEESS N,GRAVES A,et al.Recurrent models of visual attention[C]//Proceedings of the 27th International Conference on Neural Information Processing System.2014:2204-2212.
[9] LUONG M T,PHAM H,MANNING C D.Effective Approa- ches to Attention-based Neural Machine Translation[J].ArXiv:1508.04025.
[10] VASWANI A,SHAZEER N,PARMAR N,et al.Attention Is All You Need[J].arXiv:1706.03762.
[11] TAN Z,WANG M,XIE J,et al.Deep Semantic Role Labeling with Self-Attention[J].ArXiv:1712.01586.
[12] 谢志宁.中文命名实体识别算法研究[D].杭州:浙江大学,2017.
[13] GUL K S Q,尹继泽,潘丽敏,等.基于深度神经网络的命名实体识别方法研究[J].信息网络安全,2017(10):29-35.
[2] 胡文博,都云程,吕学强,等.基于多层条件随机场的中文命名实体识别[J].计算机工程与应用,2009,45(1):163-165,227.
[3] PASSOS A,KUMAR V,MCCALLUM A.Lexicon Infused Phrase Embeddings for Named Entity Resolution[C]//Proceeding of the Eighteenth Conference on Computational Language Learning,2014:78-86.
[4] CHIU J P C,NICHOLS E.Named Entity Recognition with Bidirectional LSTM-CNNs[J].ArXiv:1511.08308.
[5] COLLOBERT R,WESTON J,KARLEN M,et al.Natural Language Processing(Almost) from Scratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[6] 冯艳红,于红,孙庚,等.基于BLSTM的命名实体识别方法[J].计算机科学,2018,45(2):261-268.
[7] 王蕾.基于神经网络的中文命名实体识别研究[D].南京:南京师范大学,2017.
[8] MNIH V,HEESS N,GRAVES A,et al.Recurrent models of visual attention[C]//Proceedings of the 27th International Conference on Neural Information Processing System.2014:2204-2212.
[9] LUONG M T,PHAM H,MANNING C D.Effective Approa- ches to Attention-based Neural Machine Translation[J].ArXiv:1508.04025.
[10] VASWANI A,SHAZEER N,PARMAR N,et al.Attention Is All You Need[J].arXiv:1706.03762.
[11] TAN Z,WANG M,XIE J,et al.Deep Semantic Role Labeling with Self-Attention[J].ArXiv:1712.01586.
[12] 谢志宁.中文命名实体识别算法研究[D].杭州:浙江大学,2017.
[13] GUL K S Q,尹继泽,潘丽敏,等.基于深度神经网络的命名实体识别方法研究[J].信息网络安全,2017(10):29-35.