《同义词词林》的嵌入表示与应用评估
|
摘要
在自然语言处理中,嵌入表示是表达语言知识的重要途径和手段,以《同义词词林》为例,提出基于知识库训练嵌入表示的伪句式构造方法,并在多项任务上测试新方法的有效性.根据《同义词词林》词义编码反映的层级结构,将这些编码扩展为多种伪句式,并据此生成不同的伪语料库,采用word2vec模型在伪语料库上训练义素向量及词向量,得到CiLin2Vec资源,并应用于词义合成、类比推理和词义相似度计算等任务.在词义合成、类比推理任务上的准确率达到90%以上,超过了以往在语料库上训得的结果.证明该方法可以有效地将知识库中的理性知识注入嵌入表示中,也显示了CiLin2Vec嵌入表示资源在应用上的巨大潜力.
In natural language processing(NLP),to learn embedded representation is an effective approach of capturing semantics from language resources.At present,however,this approach has been much limited to using large-scale corpora,with little attention to extracting rational knowledge from knowledge bases.In this paper,based on " Tongyici Cilin",a famous Chinese thesaurus,we present a method for implanting rational knowledge into embedded representation,then evaluate it in terms of different NLP tasks.According to the hierarchical encodings for morphemic and lexical meanings in " Tongyici Cilin",we design multiple templates to create instances as pseudo-sentences from these pieces of knowledge,and apply word2 vec to obtain CiLin2 Vec,the sememe and word embeddings of new kinds as for " Tongyici Cilin".For evaluation,tasks of semantic compositionality,analogical reasoning and word similarity measurement are taken into consideration.We make progress and breakthrough on the tasks,reaching an accuracy of over90%for both semantic compositionality and analogical reasoning,demonstrating that the pieces of rational knowledge have been appropriately implanted,with very promising prospects for adoption of the knowledge bases.
In natural language processing(NLP),to learn embedded representation is an effective approach of capturing semantics from language resources.At present,however,this approach has been much limited to using large-scale corpora,with little attention to extracting rational knowledge from knowledge bases.In this paper,based on " Tongyici Cilin",a famous Chinese thesaurus,we present a method for implanting rational knowledge into embedded representation,then evaluate it in terms of different NLP tasks.According to the hierarchical encodings for morphemic and lexical meanings in " Tongyici Cilin",we design multiple templates to create instances as pseudo-sentences from these pieces of knowledge,and apply word2 vec to obtain CiLin2 Vec,the sememe and word embeddings of new kinds as for " Tongyici Cilin".For evaluation,tasks of semantic compositionality,analogical reasoning and word similarity measurement are taken into consideration.We make progress and breakthrough on the tasks,reaching an accuracy of over90%for both semantic compositionality and analogical reasoning,demonstrating that the pieces of rational knowledge have been appropriately implanted,with very promising prospects for adoption of the knowledge bases.
引文
[1]田久乐,赵蔚.基于同义词词林的词相似度计算方法[J].吉林大学学报(信息科学版),2010,28(6):602-608.
[2]吕立辉,梁维薇,冉蜀阳.基于《词林》的词相似度的度量[J].现代计算机(专业版),2013,1:3-6.
[3]朱新华,马润聪,孙柳,等.基于知网与《词林》的词语义相似度计算[J].中文信息学报,2016,30(4):29-36.
[4]刘丹丹,彭成,钱龙华,等.《同义词词林》在中文实体关系抽取中的作用[J].中文信息学报,2014,28(2):91-99.
[5]徐庆,段利国,李爱萍,等.基于实体词义相似度的中文实体关系抽取[J].山东大学学报(工学版),2015,45(6):7-15.
[6]李国臣,吕雷,王瑞波,等.基于同义词词林信息特征的语义角色自动标注[J].中文信息学报,2016,30(1):101-108.
[7]王东,熊世桓.基于同义词词林扩展的短文本分类[J].兰州理工大学学报,2015,4:104-108.
[8] DEERWESTER S,DUMAIS S T,FURNAS G W et al.Indexing by latent semantic analysis[J].Journal of the American Society for Information Science,1990,41(6):391-407.
[9] SCHUTZE H.Dimensions of meaning[C]∥Proceedings of the 1992ACM/IEEE Conference on Supercomputing.California:IEEE,1992:787-796.
[10] LUND K,BURGESS C.Producing high-dimensional semantic spaces from lexical co-occurrence[J].Behavior Research Methods,Instruments,&Computers,1996,28(2):203-208.
[11] COLLOBERT R,WESTON J.A unified architecture for natural language processing:deep neural networks with multitask learning[C]∥International Conference on Machine Learning.Helsinki:ACM,2008:160-167.
[12] COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing(almost)from scratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[13] TURNEY P D.Domain and function:a dual-space model of semantic relations and compositions[J].Journal of Artificial Intelligence Research,2012,44:533-585.
[14] PENNINGTON J,SOCHER R,MANNING C D.Glove:global vectors for word representation[C]∥Conference on Empirical Methods on Natural Language Processing. Doha:AssociationforComputational Linguistics,2014:1532-1543.
[15] BARTUSIAK R,AUGUSTYNIAK L,KAJDANOWICZ T,et al.WordNet2Vec:corpora agnostic word vectorization method[J].Neurocomputing,2017.doi:10.1016/j.neucom.2017.01.121.
[16] TISSIER J,GRAVIER C,HABRARD A.Dict2vec:learning word embeddings using lexical dictionaries[C]∥Conference on Empirical Methods in Natural Language Processing.Copenhagen:Association for Computational Linguistics,2017:254-263.
[17] ROTHE S,SCHUTZE H.AutoExtend:extending word embeddings to embeddings for synsets and lexemes[EB/OL].[2018-04-20].http:∥arxiv.org/pdf/1507.0112701.pdf.
[18] PANCHENKO A.Best of both worlds:making word sense embeddings interpretable[C]∥Edition of the Language Resources and Evaluation Conference.Portoro6:ELRA,2016:2649-2655.
[19] YANG L,SUN M.Improved learning of Chinese word embeddings with semantic knowledge[M]∥Chinese computationallinguisticsandnaturallanguage processing based on naturally annotated big data.Switzerland:Springer,2015:15-25.
[20] GOIKOETXEA J,SOROA,AGIRRE E.Random walks and neural network language models on knowledge bases[C]∥Proceedings of the 2015Annual Conference of the North American Chapter of the ACL.San Diego:ACL,2015:1434-1439.
[21]梅家驹,竺一鸣,高蕴奇,等.同义词词林[M].上海:上海辞书出版社,1983:1-362.
[22] HARRIS Z.Distributional structure[J].Word,1954,10(2):146-162.
[23] HINTON G E,MCCELLAND J L,RUMELHART D E.Distributed respresentations[M]∥RUMELHART D E,MCCLELLAND J L.Parallel distributed processing:explorations in the microstructure of cognition(volume 1).Cambridge:MIT,1986:77-109.
[24]孙飞,郭嘉丰,兰艳艳,等.分布式单词表示综述[J].计算机学报,2016,39:1-22.
[25] CHOMSKY N.Three models for the description of language[J].IRE Transactions on Information Theory,1956,2(3):113-124.
[26] YESSENALINA A,CARDIE C.Compositional matrixspace models for sentiment analysis[C]∥Conference on Empirical Methods on Natural Language Processing.Edinburgh:Association for Computational Linguistics,2011:172-182.
[27] SOCHER R,HUVAL B,MANNING C D,et al.Semantic compositionality through recursive matrixvector spaces[C]∥Conference on Empirical Methods on Natural Language Processing.Jeju Island:Association for Computational Linguistics,2012:1201-1211.
[28] GREFENSTETTE E,DINU G,ZHANG Y Z,et al.Multi-stepregressionlearningforcompositional distributional semantics[EB/OL].(2013-01-29)[2018-04-01].http:∥cn.arXiv.org/abs/:1301.6939.
[29] FODOR J A,PYLYSHYN Z W.Connectionism and cognitive architecture:a critical analysis[J].Cognition,1988,28(1/2):3-71.
[30] GERSHMAN S,TENENBAUM J B.Phrase similarity in humans and machines[C]∥Proceedings of the 37th Annual Conference of the Cognitive Science Society.Cambridge:MIT,2015:776-781.
[31] VAKULENKO S.The notion of sememe in the work of Adolf Noreen[J].Henry Sweet Society for the History of Linguistic Ideas Bulletin,2005(44):19-35.
[32] LYONS J.Linguistic semantics[M].Cambridge:Cambridge University Press,1996.
[33] MIKOLOV T,YIH W T,ZWEIG G.Linguistic regularities in continuous space word representations[C]∥Proceeding of the 2013Conference of the North American Chapter of the ACL.Atlanta:Association for Computational Linguistics,2013:746-751.
[34] CHEN X,XU L,LIU Z,et al.Joint learning of character and word embeddings[C]∥Proceedings of IJCAI.Buenos Aires:AAAI,2015:1236-1242.
[35]葛斌,李芳芳,郭丝路,等.基于知网的词汇语义相似度计算方法研究[J].计算机应用研究,2010,27(9):3329-3333.
[36]石静,吴云芳,邱立坤,等.基于大规模语料库的汉语词义相似度计算方法[J].中文信息学报,2013,27(1):1-6.
[37] LI Y,BANDAR Z A,MCLEAN D.An approach for measuring semantic similarity between words using multiple information sources[J].IEEE Transactions on Knowledge and Data Engineering,2003,15(4):871-882.
[38]梅立军,周强,臧路,等.知网与同义词词林的信息融合研究[J].中文信息学报,2005,19(1):64-71.
[39] TAIEB M A H,AOUICHA M B,HAMADOU A B.Ontology-based approach for measuring semantic similarity[J].Engineering Applications of Artificial Intelligence,2014,36:238-261.
[2]吕立辉,梁维薇,冉蜀阳.基于《词林》的词相似度的度量[J].现代计算机(专业版),2013,1:3-6.
[3]朱新华,马润聪,孙柳,等.基于知网与《词林》的词语义相似度计算[J].中文信息学报,2016,30(4):29-36.
[4]刘丹丹,彭成,钱龙华,等.《同义词词林》在中文实体关系抽取中的作用[J].中文信息学报,2014,28(2):91-99.
[5]徐庆,段利国,李爱萍,等.基于实体词义相似度的中文实体关系抽取[J].山东大学学报(工学版),2015,45(6):7-15.
[6]李国臣,吕雷,王瑞波,等.基于同义词词林信息特征的语义角色自动标注[J].中文信息学报,2016,30(1):101-108.
[7]王东,熊世桓.基于同义词词林扩展的短文本分类[J].兰州理工大学学报,2015,4:104-108.
[8] DEERWESTER S,DUMAIS S T,FURNAS G W et al.Indexing by latent semantic analysis[J].Journal of the American Society for Information Science,1990,41(6):391-407.
[9] SCHUTZE H.Dimensions of meaning[C]∥Proceedings of the 1992ACM/IEEE Conference on Supercomputing.California:IEEE,1992:787-796.
[10] LUND K,BURGESS C.Producing high-dimensional semantic spaces from lexical co-occurrence[J].Behavior Research Methods,Instruments,&Computers,1996,28(2):203-208.
[11] COLLOBERT R,WESTON J.A unified architecture for natural language processing:deep neural networks with multitask learning[C]∥International Conference on Machine Learning.Helsinki:ACM,2008:160-167.
[12] COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing(almost)from scratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[13] TURNEY P D.Domain and function:a dual-space model of semantic relations and compositions[J].Journal of Artificial Intelligence Research,2012,44:533-585.
[14] PENNINGTON J,SOCHER R,MANNING C D.Glove:global vectors for word representation[C]∥Conference on Empirical Methods on Natural Language Processing. Doha:AssociationforComputational Linguistics,2014:1532-1543.
[15] BARTUSIAK R,AUGUSTYNIAK L,KAJDANOWICZ T,et al.WordNet2Vec:corpora agnostic word vectorization method[J].Neurocomputing,2017.doi:10.1016/j.neucom.2017.01.121.
[16] TISSIER J,GRAVIER C,HABRARD A.Dict2vec:learning word embeddings using lexical dictionaries[C]∥Conference on Empirical Methods in Natural Language Processing.Copenhagen:Association for Computational Linguistics,2017:254-263.
[17] ROTHE S,SCHUTZE H.AutoExtend:extending word embeddings to embeddings for synsets and lexemes[EB/OL].[2018-04-20].http:∥arxiv.org/pdf/1507.0112701.pdf.
[18] PANCHENKO A.Best of both worlds:making word sense embeddings interpretable[C]∥Edition of the Language Resources and Evaluation Conference.Portoro6:ELRA,2016:2649-2655.
[19] YANG L,SUN M.Improved learning of Chinese word embeddings with semantic knowledge[M]∥Chinese computationallinguisticsandnaturallanguage processing based on naturally annotated big data.Switzerland:Springer,2015:15-25.
[20] GOIKOETXEA J,SOROA,AGIRRE E.Random walks and neural network language models on knowledge bases[C]∥Proceedings of the 2015Annual Conference of the North American Chapter of the ACL.San Diego:ACL,2015:1434-1439.
[21]梅家驹,竺一鸣,高蕴奇,等.同义词词林[M].上海:上海辞书出版社,1983:1-362.
[22] HARRIS Z.Distributional structure[J].Word,1954,10(2):146-162.
[23] HINTON G E,MCCELLAND J L,RUMELHART D E.Distributed respresentations[M]∥RUMELHART D E,MCCLELLAND J L.Parallel distributed processing:explorations in the microstructure of cognition(volume 1).Cambridge:MIT,1986:77-109.
[24]孙飞,郭嘉丰,兰艳艳,等.分布式单词表示综述[J].计算机学报,2016,39:1-22.
[25] CHOMSKY N.Three models for the description of language[J].IRE Transactions on Information Theory,1956,2(3):113-124.
[26] YESSENALINA A,CARDIE C.Compositional matrixspace models for sentiment analysis[C]∥Conference on Empirical Methods on Natural Language Processing.Edinburgh:Association for Computational Linguistics,2011:172-182.
[27] SOCHER R,HUVAL B,MANNING C D,et al.Semantic compositionality through recursive matrixvector spaces[C]∥Conference on Empirical Methods on Natural Language Processing.Jeju Island:Association for Computational Linguistics,2012:1201-1211.
[28] GREFENSTETTE E,DINU G,ZHANG Y Z,et al.Multi-stepregressionlearningforcompositional distributional semantics[EB/OL].(2013-01-29)[2018-04-01].http:∥cn.arXiv.org/abs/:1301.6939.
[29] FODOR J A,PYLYSHYN Z W.Connectionism and cognitive architecture:a critical analysis[J].Cognition,1988,28(1/2):3-71.
[30] GERSHMAN S,TENENBAUM J B.Phrase similarity in humans and machines[C]∥Proceedings of the 37th Annual Conference of the Cognitive Science Society.Cambridge:MIT,2015:776-781.
[31] VAKULENKO S.The notion of sememe in the work of Adolf Noreen[J].Henry Sweet Society for the History of Linguistic Ideas Bulletin,2005(44):19-35.
[32] LYONS J.Linguistic semantics[M].Cambridge:Cambridge University Press,1996.
[33] MIKOLOV T,YIH W T,ZWEIG G.Linguistic regularities in continuous space word representations[C]∥Proceeding of the 2013Conference of the North American Chapter of the ACL.Atlanta:Association for Computational Linguistics,2013:746-751.
[34] CHEN X,XU L,LIU Z,et al.Joint learning of character and word embeddings[C]∥Proceedings of IJCAI.Buenos Aires:AAAI,2015:1236-1242.
[35]葛斌,李芳芳,郭丝路,等.基于知网的词汇语义相似度计算方法研究[J].计算机应用研究,2010,27(9):3329-3333.
[36]石静,吴云芳,邱立坤,等.基于大规模语料库的汉语词义相似度计算方法[J].中文信息学报,2013,27(1):1-6.
[37] LI Y,BANDAR Z A,MCLEAN D.An approach for measuring semantic similarity between words using multiple information sources[J].IEEE Transactions on Knowledge and Data Engineering,2003,15(4):871-882.
[38]梅立军,周强,臧路,等.知网与同义词词林的信息融合研究[J].中文信息学报,2005,19(1):64-71.
[39] TAIEB M A H,AOUICHA M B,HAMADOU A B.Ontology-based approach for measuring semantic similarity[J].Engineering Applications of Artificial Intelligence,2014,36:238-261.