基于双通道卷积神经网络的问句意图分类研究
|
摘要
人机对话技术近年来受到学术界和工业界的广泛关注。人机对话系统的一个关键任务就是如何让聊天机器人理解用户的问句意图并将用户的输入正确地分类到相应领域中,其性能直接影响到特定领域的人机对话质量。该文针对对话问句具有句子长度短、局部特征明显等特点,单通道卷积神经网络(Convolutional Neural Network,CNN)视角单一,不能充分学习到问句的特征信息和语义信息。该文在研究和分析了CNN算法的基础上,提出了意图分类双通道卷积神经网(Intent Classification Dual-channel Convolutional Neural Networks,ICDCNN)算法。该方法首先采用Word2Vec工具和Embedding层进行训练词向量提取问句中的语义信息特征;然后采用两个不同的通道进行卷积运算,一个通道传入字级别的词向量,另一个通道传入词级别的词向量,使用细粒度的字级别词向量协助词级别的词向量捕获自然语言问句中更深层次的语义信息;最后通过设置不同尺寸的卷积核,学习问句内部更深层次的抽象特征。通过对比实验结果表明,该算法在选用的中文实验数据集上取得了较高的准确率,较其他算法具有一定的优势。
Human-machine conversation technology has received extensive attention from the academic and industrial fields in recent years.The users question intention classification is an important key issues with direct effect on the quality of human-machine dialogue.In this paper,we propose an intent classification dual-channel Convolutional Neural Networks(ICDCNN):we first extract semantic features by using Word2 vec and Embedding layer to train the word vector;then,two different channels are used for convolution,one for character level word vector,the other for word level word vector;thirdly,the character level word vectors(fine-grained)are combined with word level word vectors to mine deeper semantic information of natural language question;finally,with convolution kernels of different sizes,deeper abstract features inside the questions are learnt.Experimental results show that the algorithm achieves high accuracy on Chinese datasets,which has certain advantages compared to other methods.
Human-machine conversation technology has received extensive attention from the academic and industrial fields in recent years.The users question intention classification is an important key issues with direct effect on the quality of human-machine dialogue.In this paper,we propose an intent classification dual-channel Convolutional Neural Networks(ICDCNN):we first extract semantic features by using Word2 vec and Embedding layer to train the word vector;then,two different channels are used for convolution,one for character level word vector,the other for word level word vector;thirdly,the character level word vectors(fine-grained)are combined with word level word vectors to mine deeper semantic information of natural language question;finally,with convolution kernels of different sizes,deeper abstract features inside the questions are learnt.Experimental results show that the algorithm achieves high accuracy on Chinese datasets,which has certain advantages compared to other methods.
引文
[1]冯志伟.自然语言问答系统的发展与现状[J].外国语(上海外国语大学学报),2012(6):2-16.
[2]周鑫鹏.基于深度学习的问题分类的研究[D].哈尔滨:哈尔滨工业大学硕士论文,2016.
[3]冶忠林,贾真,尹红风.多领域自然语言问句理解研究[J].计算机科学,2017,44(6):216-221.
[4]霍帅,张敏,刘奕群,等.基于微博内容的新词发现方法[J].模式识别与人工智能,2014,27(2):141-145.
[5]许坤,冯岩松,赵东岩,等.面向知识库的中文自然语言问句的语义理解[J].北京大学学报(自然科学版),2014,50(1):85-92.
[6]Wang X J,Zhang L,Ma W Y.Answer ranking in community question-answering sites:US,US8346701[P].2013.
[7]Angelino,Elaine,Larus-Stone,et al.Learning certifiably optimal rule lists for categorical data[C]//Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,2017:35-44.
[8]Surdeanu M,Surdeanu M.Performanceissues and error analysis in an open-domain question answering system[J].Acm Transactions on Information Systems,2003,21(2):133-154.
[9]李平,戴月明,吴定会.双通道卷积神经网络在文本情感分析中的应用[J].计算机应用,2018,38,334(06):22-26.
[10]Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[C]//Proceedings of International Conference on Neural Information Processing Systems.Curran Associates Inc.2012:1097-1105.
[11]Graves A,Mohamed A,Hinton G.Speech recognition with deep recurrent neural networks[C]//Proceedings of the ICASSP,2013,38(2003):6645-6649.
[12]Yih W.Semantic Parsing for Single-Relation Question Answering[C]//Proceedings of Acl,2014:643-648.
[13]Shen Y,He X,Gao J,et al.Learning semantic representations using convolutional neural networks for web search[C]//Proceedings of International Conference on World Wide Web.ACM,2014:373-374.
[14]Kalchbrenner N,Grefenstette E,Blunsom P.A Convolutional Neural Network for Modelling Sentences[J].Eprint Arxiv,2014,1.
[15]Collobert R,Weston J,Karlen M,et al.Natural Language Processing(Almost)from Scratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[16]Kim Y.Convolutional Neural Networks for Sentence Classification[J].Eprint Arxiv,2014.
[17]Prager J,Radev D,Brown E,et al.The Use of Predictive Annotation for Question Answering in TREC8[C]//Proceedings of Conference of the 8th Text Retrieval,1999:399--411.
[18]Xin L,Dan R.Learning question classifiers[C]//Proceedings of International Conference on Computational Linguistics.Association for Computational Linguistics(ACL2002),2002,:1-7.
[19]Li X,Dan R.Learning question classifiers:the role of semantic information.[J].Natural Language Engineering,2015,12(3):229-249.
[20]Loni B.A Survey of State-of-the-Art Methods on Question Classification[J].Electrical Engineering Mathematics&Computer Science,2011,8(1):23-40.
[21]Hinton G E,Salakhutdinov R.Reducing the dimensionality of data with neural networks[J].Science.2006,313(5786):504-507.
[22]Komninos A,Manandhar S.Dependency Based Embeddings for Sentence Classification Tasks[C]//Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:1490-1500.
[23]Li X,Huang X,Wu L.Combined multiple classifiers based on TBL algorithm and their application in question classification[J].Journal of Computer Research&Development,2008,45(3):535-541.
[24]Li F,Zhang X,Yuan J,et al.Classifying What-Type Questions by Head Noun Tagging.[C]//Proceedings of International Conference on Computational Linguistics,DBLP,2008:481-488.
[25]Mikolov T,Chen K,Corrado G,et al.Efficient Estimation of Word Representations in Vector Space[J].arXiv preprint arXiv:1301.3781,2013.
[26]刘龙飞,杨亮,张绍武,等.基于卷积神经网络的微博情感倾向性分析[J].中文信息学报,2015,29(6):159-165.
[27]Lécun Y,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[C]//Proceedings of the IEEE,1998,86(11):2278-2324.
[2]周鑫鹏.基于深度学习的问题分类的研究[D].哈尔滨:哈尔滨工业大学硕士论文,2016.
[3]冶忠林,贾真,尹红风.多领域自然语言问句理解研究[J].计算机科学,2017,44(6):216-221.
[4]霍帅,张敏,刘奕群,等.基于微博内容的新词发现方法[J].模式识别与人工智能,2014,27(2):141-145.
[5]许坤,冯岩松,赵东岩,等.面向知识库的中文自然语言问句的语义理解[J].北京大学学报(自然科学版),2014,50(1):85-92.
[6]Wang X J,Zhang L,Ma W Y.Answer ranking in community question-answering sites:US,US8346701[P].2013.
[7]Angelino,Elaine,Larus-Stone,et al.Learning certifiably optimal rule lists for categorical data[C]//Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,2017:35-44.
[8]Surdeanu M,Surdeanu M.Performanceissues and error analysis in an open-domain question answering system[J].Acm Transactions on Information Systems,2003,21(2):133-154.
[9]李平,戴月明,吴定会.双通道卷积神经网络在文本情感分析中的应用[J].计算机应用,2018,38,334(06):22-26.
[10]Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[C]//Proceedings of International Conference on Neural Information Processing Systems.Curran Associates Inc.2012:1097-1105.
[11]Graves A,Mohamed A,Hinton G.Speech recognition with deep recurrent neural networks[C]//Proceedings of the ICASSP,2013,38(2003):6645-6649.
[12]Yih W.Semantic Parsing for Single-Relation Question Answering[C]//Proceedings of Acl,2014:643-648.
[13]Shen Y,He X,Gao J,et al.Learning semantic representations using convolutional neural networks for web search[C]//Proceedings of International Conference on World Wide Web.ACM,2014:373-374.
[14]Kalchbrenner N,Grefenstette E,Blunsom P.A Convolutional Neural Network for Modelling Sentences[J].Eprint Arxiv,2014,1.
[15]Collobert R,Weston J,Karlen M,et al.Natural Language Processing(Almost)from Scratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[16]Kim Y.Convolutional Neural Networks for Sentence Classification[J].Eprint Arxiv,2014.
[17]Prager J,Radev D,Brown E,et al.The Use of Predictive Annotation for Question Answering in TREC8[C]//Proceedings of Conference of the 8th Text Retrieval,1999:399--411.
[18]Xin L,Dan R.Learning question classifiers[C]//Proceedings of International Conference on Computational Linguistics.Association for Computational Linguistics(ACL2002),2002,:1-7.
[19]Li X,Dan R.Learning question classifiers:the role of semantic information.[J].Natural Language Engineering,2015,12(3):229-249.
[20]Loni B.A Survey of State-of-the-Art Methods on Question Classification[J].Electrical Engineering Mathematics&Computer Science,2011,8(1):23-40.
[21]Hinton G E,Salakhutdinov R.Reducing the dimensionality of data with neural networks[J].Science.2006,313(5786):504-507.
[22]Komninos A,Manandhar S.Dependency Based Embeddings for Sentence Classification Tasks[C]//Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:1490-1500.
[23]Li X,Huang X,Wu L.Combined multiple classifiers based on TBL algorithm and their application in question classification[J].Journal of Computer Research&Development,2008,45(3):535-541.
[24]Li F,Zhang X,Yuan J,et al.Classifying What-Type Questions by Head Noun Tagging.[C]//Proceedings of International Conference on Computational Linguistics,DBLP,2008:481-488.
[25]Mikolov T,Chen K,Corrado G,et al.Efficient Estimation of Word Representations in Vector Space[J].arXiv preprint arXiv:1301.3781,2013.
[26]刘龙飞,杨亮,张绍武,等.基于卷积神经网络的微博情感倾向性分析[J].中文信息学报,2015,29(6):159-165.
[27]Lécun Y,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[C]//Proceedings of the IEEE,1998,86(11):2278-2324.