基于循环胶囊网络的临床语义关系识别研究
|
摘要
得益于医疗信息化的不断推进,医院已积累了大量的电子病历记录。然而,这些病历记录大多以自然语言的形式存在,无法为计算机所"理解",也就无法对其做进一步的处理与挖掘。由此,对病历文本进行结构化研究,识别出病历实体间的语义关系,便显得尤为重要。本文针对临床语义关系识别任务,提出循环胶囊网络模型,使用分段循环神经网络来捕捉两实体及其上下文信息,并使用胶囊网络来进行最终的关系分类。实验表明,本文提出的方法较现有监督学习方法取得了更好的识别效果(F1-score为96.51%),证明了本文方法的优越性。
A large number of electronic health records(EHRs)have been accumulated since the wide adoption of medical information systems in China.However,most of these records are written in natural language,which cannot be processed by computers directly.Thus,it is important to transform unstructured EHRs into structured ones.In this paper,a recurrent capsule network is proposed for clinical relation extraction in EHRs,where entity pairs and their contexts are captured by piece-wise recurrent neural network layers,and capsule layers are finally employed for relation classification.Experimental results show that this model performs better than the existing supervised methods,achieving aF1-score of 96.51%.
A large number of electronic health records(EHRs)have been accumulated since the wide adoption of medical information systems in China.However,most of these records are written in natural language,which cannot be processed by computers directly.Thus,it is important to transform unstructured EHRs into structured ones.In this paper,a recurrent capsule network is proposed for clinical relation extraction in EHRs,where entity pairs and their contexts are captured by piece-wise recurrent neural network layers,and capsule layers are finally employed for relation classification.Experimental results show that this model performs better than the existing supervised methods,achieving aF1-score of 96.51%.
引文
[1] GUNTER T D,TERRY N P.The emergence of national electronic health record architectures in the United States and Australia:models,costs,and questions[J].Journal of Medical Internet Research,2005,7(1):e3.DOI:10.2196/jmir.7.1.e3.
[2] ZELENKO D,AONE C,RICHARDELLA A.Kernel methods for relation extraction[C]//Proceedings of the ACL-02Conference on Empirical Methods in Natural Language Processing:Volume 10.Stroudsburg,PA:Association for Computational Linguistics,2002:71-78.DOI:10.3115/1118693.1118703.
[3] ZHAO Shubin,GRISHMAN R.Extracting relations with integrated information using kernel methods[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2005:419-426.DOI:10.3115/1219840.1219892.
[4] SOCHER R,HUVAL B,MANNING C D,et al.Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2012:1201-1211.
[5] ZENG Daojian,LIU Kang,LAI Siwei,et al.Relation classification via convolutional deep neural network[C]//Proceedings of COLING 2014,the 25th International Conference on Computational Linguistics:Technical Papers.Stroudsburg,PA:Association for Computational Linguistics,2014:2335-2344.
[6] NGUYEN T H,GRISHMAN R.Relation extraction:perspective from convolutional neural networks[C]//Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2015:39-48.DOI:10.3115/v1/W15-1506.
[7] ZHANG Dongxu,WANG Dong.Relation classification via recurrent neural network[EB/OL].(2015-08-05)[2018-09-27].https://arxiv.org/abs/1508.01006.
[8] ZHOU Peng,SHI Wei,TIAN Jun,et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Vol.2Short Papers.Stroudsburg,PA:Association for Computational Linguistics,2016:207-212.DOI:10.18653/v1/P16-2034.
[9] GUPTA S,MANNING C D.Improved pattern learning for bootstrapped entity extraction[C]//Proceedings of the Eighteenth Conference on Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2014:98-108.DOI:10.3115/v1/W14-1611.
[10] LIN Yankai,SHEN Shiqi,LIU Zhiyuan,et al.Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Vol.1 Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2016:2124-2133.DOI:10.18653/v1/P16-1200.
[11] JI Guoliang,LIU Kang,HE Shizhu,et al.Distant supervision for relation extraction with sentence-level attention and entity descriptions[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence.Palo Alto,CA:AAAI Press,2017:3060-3066.
[12] SABOUR S,FROSST N,HINTON G E.Dynamic routing between capsules[C]//Advances in Neural Information Processing Systems 30.Red Hook,NY:Curran Associates Inc,2017:3856-3866.
[13] MILLER S,FOX H,RAMSHAW L,et al.A novel use of statistical parsing to extract information from text[C]//Proceedings of the 1st North American Chapter of the Association for Computational Linguistics Conference.Stroudsburg,PA:Association for Computational Linguistics,2000:226-233.
[14] KAMBHATLA N.Combining lexical,syntactic,and semantic features with maximum entropy models for extracting relations[C]//Proceedings of the ACL 2004on Interactive Poster and Demonstration Sessions.Stroudsburg,PA:Association for Computational Linguistics,2004:22.DOI:10.3115/1219044.1219066.
[15] CULOTTA A,McCALLUM A,BETZ J.Integrating probabilistic extraction models and data mining to discover relations and patterns in text[C]//Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2006:296-303.DOI:10.3115/1220835.1220873.
[16] XU Yan,MOU Lili,LI Ge,et al.Classifying relations via long short term memory networks along shortest dependency paths[C]//Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2015:1785-1794.DOI:10.18653/v1/D15-1206.
[17] CARLSON A,BETTERIDGE J,KISIEL B,et al.Toward an architecture for never-ending language learning[C]//Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence.Palo Alto,CA:AAAI Press,2010:1306-1313.
[18] RIEDEL S,YAO Limin,McCALLUM A.Modeling relations and their mentions without labeled text[C]//Joint European Conference on Machine Learning and Knowledge Discovery in Databases:LNCS Volume 6323.Berlin:Springer,2010:148-163.DOI:10.1007/978-3-642-15939-8_10.
[19] HOFFMANNR,ZHANG Congle,LING Xiao,et al.Knowledge-based weak supervision for information extraction of overlapping relations[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies:Volume 1.Stroudsburg,PA:Association for Computational Linguistics,2011:541-550.
[20] SURDEANU M,TIBSHIRANI J,NALLAPATI R,et al.Multi-instance multi-label learning for relation extraction[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2012:455-465.
[21] ZENG Daojian,LIU Kang,CHEN Yubo,et al.Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2015:1753-1762.DOI:10.18653/v1/D15-1203.
[22] SINGH S,RIEDEL S,MARTIN B,et al.Joint inference of entities,relations,and coreference[C]//Proceedings of the2013Workshop on Automated Knowledge Base Construction.New York,NY:ACM Press,2013:1-6.DOI:10.1145/2509558.2509559.
[23] MIWA M,SASAKI Y.Modeling joint entity and relation extraction with table representation[C]//Proceedings of the2014 Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2014:1858-1869.DOI:10.3115/v1/D14-1200.
[24] LI Qi,JI Heng.Incremental joint extraction of entity mentions and relations[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics:Volume 1Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2014:402-412.DOI:10.3115/v1/P14-1038.
[25] MIWA M,BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Volume 1Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2016:1105-1116.
[26] ZHENG Suncong,WANG Feng,BAO Hongyun,et al.Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics:Volume 1Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2017:1227-1236.
[27] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.DOI:10.1162/neco.1997.9.8.1735.
[28] MIKOLOV T,CHEN Kai,CORRADO G,et al.Efficient estimation of word representations in vector space[EB/OL].(2013-01-16)[2018-09-27].https://arxiv.org/abs/1301.3781.
[29] KINGMA D P,BA J L.ADAM:a method for stochastic optimization[EB/OL].(2014-12-22)[2018-09-27].https://arxiv.org/abs/1412.6980.
[2] ZELENKO D,AONE C,RICHARDELLA A.Kernel methods for relation extraction[C]//Proceedings of the ACL-02Conference on Empirical Methods in Natural Language Processing:Volume 10.Stroudsburg,PA:Association for Computational Linguistics,2002:71-78.DOI:10.3115/1118693.1118703.
[3] ZHAO Shubin,GRISHMAN R.Extracting relations with integrated information using kernel methods[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2005:419-426.DOI:10.3115/1219840.1219892.
[4] SOCHER R,HUVAL B,MANNING C D,et al.Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2012:1201-1211.
[5] ZENG Daojian,LIU Kang,LAI Siwei,et al.Relation classification via convolutional deep neural network[C]//Proceedings of COLING 2014,the 25th International Conference on Computational Linguistics:Technical Papers.Stroudsburg,PA:Association for Computational Linguistics,2014:2335-2344.
[6] NGUYEN T H,GRISHMAN R.Relation extraction:perspective from convolutional neural networks[C]//Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2015:39-48.DOI:10.3115/v1/W15-1506.
[7] ZHANG Dongxu,WANG Dong.Relation classification via recurrent neural network[EB/OL].(2015-08-05)[2018-09-27].https://arxiv.org/abs/1508.01006.
[8] ZHOU Peng,SHI Wei,TIAN Jun,et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Vol.2Short Papers.Stroudsburg,PA:Association for Computational Linguistics,2016:207-212.DOI:10.18653/v1/P16-2034.
[9] GUPTA S,MANNING C D.Improved pattern learning for bootstrapped entity extraction[C]//Proceedings of the Eighteenth Conference on Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2014:98-108.DOI:10.3115/v1/W14-1611.
[10] LIN Yankai,SHEN Shiqi,LIU Zhiyuan,et al.Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Vol.1 Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2016:2124-2133.DOI:10.18653/v1/P16-1200.
[11] JI Guoliang,LIU Kang,HE Shizhu,et al.Distant supervision for relation extraction with sentence-level attention and entity descriptions[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence.Palo Alto,CA:AAAI Press,2017:3060-3066.
[12] SABOUR S,FROSST N,HINTON G E.Dynamic routing between capsules[C]//Advances in Neural Information Processing Systems 30.Red Hook,NY:Curran Associates Inc,2017:3856-3866.
[13] MILLER S,FOX H,RAMSHAW L,et al.A novel use of statistical parsing to extract information from text[C]//Proceedings of the 1st North American Chapter of the Association for Computational Linguistics Conference.Stroudsburg,PA:Association for Computational Linguistics,2000:226-233.
[14] KAMBHATLA N.Combining lexical,syntactic,and semantic features with maximum entropy models for extracting relations[C]//Proceedings of the ACL 2004on Interactive Poster and Demonstration Sessions.Stroudsburg,PA:Association for Computational Linguistics,2004:22.DOI:10.3115/1219044.1219066.
[15] CULOTTA A,McCALLUM A,BETZ J.Integrating probabilistic extraction models and data mining to discover relations and patterns in text[C]//Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2006:296-303.DOI:10.3115/1220835.1220873.
[16] XU Yan,MOU Lili,LI Ge,et al.Classifying relations via long short term memory networks along shortest dependency paths[C]//Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2015:1785-1794.DOI:10.18653/v1/D15-1206.
[17] CARLSON A,BETTERIDGE J,KISIEL B,et al.Toward an architecture for never-ending language learning[C]//Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence.Palo Alto,CA:AAAI Press,2010:1306-1313.
[18] RIEDEL S,YAO Limin,McCALLUM A.Modeling relations and their mentions without labeled text[C]//Joint European Conference on Machine Learning and Knowledge Discovery in Databases:LNCS Volume 6323.Berlin:Springer,2010:148-163.DOI:10.1007/978-3-642-15939-8_10.
[19] HOFFMANNR,ZHANG Congle,LING Xiao,et al.Knowledge-based weak supervision for information extraction of overlapping relations[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies:Volume 1.Stroudsburg,PA:Association for Computational Linguistics,2011:541-550.
[20] SURDEANU M,TIBSHIRANI J,NALLAPATI R,et al.Multi-instance multi-label learning for relation extraction[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2012:455-465.
[21] ZENG Daojian,LIU Kang,CHEN Yubo,et al.Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2015:1753-1762.DOI:10.18653/v1/D15-1203.
[22] SINGH S,RIEDEL S,MARTIN B,et al.Joint inference of entities,relations,and coreference[C]//Proceedings of the2013Workshop on Automated Knowledge Base Construction.New York,NY:ACM Press,2013:1-6.DOI:10.1145/2509558.2509559.
[23] MIWA M,SASAKI Y.Modeling joint entity and relation extraction with table representation[C]//Proceedings of the2014 Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2014:1858-1869.DOI:10.3115/v1/D14-1200.
[24] LI Qi,JI Heng.Incremental joint extraction of entity mentions and relations[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics:Volume 1Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2014:402-412.DOI:10.3115/v1/P14-1038.
[25] MIWA M,BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Volume 1Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2016:1105-1116.
[26] ZHENG Suncong,WANG Feng,BAO Hongyun,et al.Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics:Volume 1Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2017:1227-1236.
[27] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.DOI:10.1162/neco.1997.9.8.1735.
[28] MIKOLOV T,CHEN Kai,CORRADO G,et al.Efficient estimation of word representations in vector space[EB/OL].(2013-01-16)[2018-09-27].https://arxiv.org/abs/1301.3781.
[29] KINGMA D P,BA J L.ADAM:a method for stochastic optimization[EB/OL].(2014-12-22)[2018-09-27].https://arxiv.org/abs/1412.6980.