反馈式K近邻语义迁移学习的领域命名实体识别
|
摘要
领域命名实体识别是构建领域知识图谱的重要基础。针对专业领域语料匮乏的特点,构建基于深度学习的BiLSTM-CNN-CRFs网络模型,并提出一种反馈式K近邻语义迁移学习的领域命名实体识别方法。首先,对专业领域语料和通用领域语料分别训练得到语料文档向量,使用马哈拉诺比斯距离计算领域语料与通用语料的语义相似性,针对每个专业领域样本分别取K个语义最相似的通用领域样本进行语义迁移学习,构建多个迁移语料集。然后,使用BiLSTM-CNN-CRFs网络模型对迁移语料集进行领域命名实体识别,并对识别结果进行评估和前馈,根据反馈结果选取合适的K值,作为语义迁移学习的最佳阈值。以包装领域和医疗领域为例进行实验验证,结果表明:本文方法取得了很好的识别效果,可以有效解决专业领域语料匮乏问题。
Domain-named entity recognition is an important foundation in constructing domain knowledge maps. In view of the scarcity of such recognition, this paper constructs a BiLSTM-CNN-CRFs network model based on deep learning as well as proposes a domain-named entity recognition method based on feedback K-nearest-neighbor semantic transfer learning. First, the corpus of the professional field and the general field were trained to obtain the corpus document vector, and the semantic similarity between the corpus of a domain and the common corpus was calculated using the Mahalanobis distance calculation. For each specialized domain sample, K common domain samples with the most similar semantics were taken for semantic transfer learning, and several transfer corpus sets were constructed. Then, the BiLSTM-CNN-CRFs network model was used to identify domain-named entities in N migration corpuses and evaluate and feedforward the recognition results. An appropriate K value was selected as the best threshold for semantic transfer learning according to the feedback results. The packaging and medical fields were taken as examples for experimental verification. The results showed that the method proposed in this paper has a good recognition effect and can effectively solve the problem of lack of corpus in the field of specialization.
Domain-named entity recognition is an important foundation in constructing domain knowledge maps. In view of the scarcity of such recognition, this paper constructs a BiLSTM-CNN-CRFs network model based on deep learning as well as proposes a domain-named entity recognition method based on feedback K-nearest-neighbor semantic transfer learning. First, the corpus of the professional field and the general field were trained to obtain the corpus document vector, and the semantic similarity between the corpus of a domain and the common corpus was calculated using the Mahalanobis distance calculation. For each specialized domain sample, K common domain samples with the most similar semantics were taken for semantic transfer learning, and several transfer corpus sets were constructed. Then, the BiLSTM-CNN-CRFs network model was used to identify domain-named entities in N migration corpuses and evaluate and feedforward the recognition results. An appropriate K value was selected as the best threshold for semantic transfer learning according to the feedback results. The packaging and medical fields were taken as examples for experimental verification. The results showed that the method proposed in this paper has a good recognition effect and can effectively solve the problem of lack of corpus in the field of specialization.
引文
[1]SOCHER R,HUVAL B,MANNING C D,et al.Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.Jeju Island,Korea:ACM,2012:1201-1211.
[2]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural computation,1997,9(8):1735-1780.
[3]COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing(almost)from scratch[J].Journal of machine learning research,2011,12:2493-2537.
[4]张海楠,伍大勇,刘悦,等.基于深度神经网络的中文命名实体识别[J].中文信息学报,2017,31(4):28-35.ZHANG Hainan,WU Dayong,LIU Yue,et al.Chinese named entity recognition based on deep neural network[J].Journal of Chinese information processing,2017,31(4):28-35.ternational Conference on International Conference on Machine Learning.Beijing,China,2014:Ⅱ-1188-Ⅱ-1196.
[5]MA Xuezhe, HOVY E. End-to-end sequence labeling via Bi-directional LSTM-CNNs-CRF[C]//Proceedings of the54th Annual Meeting of the Association for Computational Linguistics(ACL)2016. Berlin, Germany:ACL, 2016:1064-1074.
[6]CHIU J P C, NICHOLS E. Named entity recognition with bidirectional LSTM-CNNs[J].Computer science, 2016:357-370.
[7]姚霖,刘轶,李鑫鑫,等.词边界字向量的中文命名实体识别[J].智能系统学报, 2017, 11(1):37–42.YAO Lin, LIU Yi, LI Xinxin, et al. Chinese named entity recognition via word boundary based character embedding[J]. CAAI transactions on intelligent systems,2017, 11(1):37–42.
[8]PAN S J, YANG Qiang. A survey on transfer learning[J].IEEE transactions on knowledge and data engineering,2010, 22(10):1345–1359.
[9]PAN S J, TSANG I W, KWOK J T, et al. Domain adaptation via transfer component analysis[J]. IEEE transactions on neural networks, 2011, 22(2):199–210.
[10]LONG Mingsheng, WANG Jianmin, DING Guiguang, et al. Transfer feature learning with joint distribution adaptation[C]//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney, NSW:IEEE, 2013:2200-2207.
[11]卞则康,王士同.基于相似度学习的多源迁移算法[J].控制与决策, 2017, 32(11):1941–1948.BIAN Zekang, WANG Shitong. Similarity-learning based multi-source transfer learning algorithm[J]. Control and decision, 2017, 32(11):1941–1948.
[12]庄福振,罗平,何清,等.迁移学习研究进展[J].软件学报, 2015, 26(1):26–39.ZHUANG Fuzhen, LUO Ping, HE Qing, et al. Survey on transfer learning research[J]. Journal of software, 2015,26(1):26–39.
[13]MIKOLOV T, CHEN K, CORRADO G, etal. Efficient estimation of word representations in space[J]. Computer science, 2013:1-12.
[14]LE Q V, MIKOLOV T. Distributed representations of sentences and documents[C]//Proceedings of the 31st International Conference on International Conference on Machine Learning. Beijing, China, 2014:II-1188-II-1196.
[15]MAHALANOBIS P C.On the generalized distance in statistics[J].Proceedings of national institute of sciences,1936,2(1):49-55.
[16]杨绪兵,王一雄,陈斌.马氏度量学习中的几个关键问题研究及几何解释[J].南京大学学报(自然科学),2013,49(2):133-141.YANG Xubing,WANG Yixiong,CHEN Bin.Research on several key problems of Mahalanobis metric learning and corresponding geometrical interpretaions[J].Journal of Nanjing University(natural sciences),2013,49(2):133-141.
[17]中国包装新闻数据.中国包装网[EB/OL].(2017.12)[2017.12.25],http://news.pack.cn/China Packaging Network.Chinese packaging news data[EB/OL].(2017-12).[201 7.12.25].http://news.pack.cn/.
[18]搜狐新闻数据.搜狗实验室[EB/OL].(2012.08.16)[2017.12.20].http://www.sogou.com/labs/resource/cs.php.Sougou Labs.The whole network news data set[EB/OL].(2012.08.16)[2017.12.20].http://www.sogou.com/labs/resource/ca.php.
[19]电子病历命名实体识别数据集.全国知识图谱与语义计算大会[EB\OL].(2017.05.20)[2017.12.20].http://www.ccks2018.cn/?page_id=16Clinical named entity recognition dataSet.China conference on knowledge graph and semantic computing[EB\OL].(2017.05.20)[2017.12.20].http://www.ccks2018.cn/en/)
[20]医疗资讯网.医疗新闻数据[EB/OL].(2013.01.01)[2017.12.20].http://www.120news.org/.Medical information network,medic al news data[EB/OL].(2013.01.01)[2017.12.20].http://www.120news.org/.
[21]好大夫在线.医疗新闻数据[EB/OL].(2006.01.01)[2017.12.20].https://www.haodf.com/article/.Good doctor online.Medical news data[EB/OL].(2006.01.01)[2017.12.20].https://www.haodf.com/article/.
[2]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural computation,1997,9(8):1735-1780.
[3]COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing(almost)from scratch[J].Journal of machine learning research,2011,12:2493-2537.
[4]张海楠,伍大勇,刘悦,等.基于深度神经网络的中文命名实体识别[J].中文信息学报,2017,31(4):28-35.ZHANG Hainan,WU Dayong,LIU Yue,et al.Chinese named entity recognition based on deep neural network[J].Journal of Chinese information processing,2017,31(4):28-35.ternational Conference on International Conference on Machine Learning.Beijing,China,2014:Ⅱ-1188-Ⅱ-1196.
[5]MA Xuezhe, HOVY E. End-to-end sequence labeling via Bi-directional LSTM-CNNs-CRF[C]//Proceedings of the54th Annual Meeting of the Association for Computational Linguistics(ACL)2016. Berlin, Germany:ACL, 2016:1064-1074.
[6]CHIU J P C, NICHOLS E. Named entity recognition with bidirectional LSTM-CNNs[J].Computer science, 2016:357-370.
[7]姚霖,刘轶,李鑫鑫,等.词边界字向量的中文命名实体识别[J].智能系统学报, 2017, 11(1):37–42.YAO Lin, LIU Yi, LI Xinxin, et al. Chinese named entity recognition via word boundary based character embedding[J]. CAAI transactions on intelligent systems,2017, 11(1):37–42.
[8]PAN S J, YANG Qiang. A survey on transfer learning[J].IEEE transactions on knowledge and data engineering,2010, 22(10):1345–1359.
[9]PAN S J, TSANG I W, KWOK J T, et al. Domain adaptation via transfer component analysis[J]. IEEE transactions on neural networks, 2011, 22(2):199–210.
[10]LONG Mingsheng, WANG Jianmin, DING Guiguang, et al. Transfer feature learning with joint distribution adaptation[C]//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney, NSW:IEEE, 2013:2200-2207.
[11]卞则康,王士同.基于相似度学习的多源迁移算法[J].控制与决策, 2017, 32(11):1941–1948.BIAN Zekang, WANG Shitong. Similarity-learning based multi-source transfer learning algorithm[J]. Control and decision, 2017, 32(11):1941–1948.
[12]庄福振,罗平,何清,等.迁移学习研究进展[J].软件学报, 2015, 26(1):26–39.ZHUANG Fuzhen, LUO Ping, HE Qing, et al. Survey on transfer learning research[J]. Journal of software, 2015,26(1):26–39.
[13]MIKOLOV T, CHEN K, CORRADO G, etal. Efficient estimation of word representations in space[J]. Computer science, 2013:1-12.
[14]LE Q V, MIKOLOV T. Distributed representations of sentences and documents[C]//Proceedings of the 31st International Conference on International Conference on Machine Learning. Beijing, China, 2014:II-1188-II-1196.
[15]MAHALANOBIS P C.On the generalized distance in statistics[J].Proceedings of national institute of sciences,1936,2(1):49-55.
[16]杨绪兵,王一雄,陈斌.马氏度量学习中的几个关键问题研究及几何解释[J].南京大学学报(自然科学),2013,49(2):133-141.YANG Xubing,WANG Yixiong,CHEN Bin.Research on several key problems of Mahalanobis metric learning and corresponding geometrical interpretaions[J].Journal of Nanjing University(natural sciences),2013,49(2):133-141.
[17]中国包装新闻数据.中国包装网[EB/OL].(2017.12)[2017.12.25],http://news.pack.cn/China Packaging Network.Chinese packaging news data[EB/OL].(2017-12).[201 7.12.25].http://news.pack.cn/.
[18]搜狐新闻数据.搜狗实验室[EB/OL].(2012.08.16)[2017.12.20].http://www.sogou.com/labs/resource/cs.php.Sougou Labs.The whole network news data set[EB/OL].(2012.08.16)[2017.12.20].http://www.sogou.com/labs/resource/ca.php.
[19]电子病历命名实体识别数据集.全国知识图谱与语义计算大会[EB\OL].(2017.05.20)[2017.12.20].http://www.ccks2018.cn/?page_id=16Clinical named entity recognition dataSet.China conference on knowledge graph and semantic computing[EB\OL].(2017.05.20)[2017.12.20].http://www.ccks2018.cn/en/)
[20]医疗资讯网.医疗新闻数据[EB/OL].(2013.01.01)[2017.12.20].http://www.120news.org/.Medical information network,medic al news data[EB/OL].(2013.01.01)[2017.12.20].http://www.120news.org/.
[21]好大夫在线.医疗新闻数据[EB/OL].(2006.01.01)[2017.12.20].https://www.haodf.com/article/.Good doctor online.Medical news data[EB/OL].(2006.01.01)[2017.12.20].https://www.haodf.com/article/.