基于Bi-LSTM和CNN并包含注意力机制的社区问答问句分类方法
|
摘要
问句分类的目标是将用户提出的自然语言问句分到预先设定的类别.在社区问答中,如何准确高效的对问句进行分类是一项重要任务.本文提出了一种基于深度神经网络的问句分类方法,该方法首先将问句用词向量进行表示,然后用融合双向长短时记忆网络(Bi-LSTM)和卷积神经网络(CNN)结构并包含注意力机制的深度学习模型提取问句特征进行分类.该方法的特色在于利用Bi-LSTM和CNN在句子级文本表示的优点,充分捕捉问句特征,并结合问句的对应答案来表示问句,丰富了问句信息.实验表明,该问句分类方法准确率较高,在多个数据集上取得不错结果.
The goal of question categorization is to classify natural language questions that user raised into predefined categories. How to classify question sentences accurately and efficiently is an important task in community question answering. In this study, we propose a question categorization method based on deep neural network. Firstly, the words of the question are transformed to vectors. Then, we use a novel Bidirectional Long Short-Term Memory(Bi-LSTM) based Convolutional Neural Network(CNN) model with attention mechanism to capture the most important features in a question. Finally, the features are fed into the classifier to predict the category of the question. We use the Bi-LSTM and CNN to capture the features of question because of their benefits in representing sentence level documents. We also use the answer set to enrich the information of the question. The experimental results on several datasets demonstrate the effectiveness of the proposed approach.
The goal of question categorization is to classify natural language questions that user raised into predefined categories. How to classify question sentences accurately and efficiently is an important task in community question answering. In this study, we propose a question categorization method based on deep neural network. Firstly, the words of the question are transformed to vectors. Then, we use a novel Bidirectional Long Short-Term Memory(Bi-LSTM) based Convolutional Neural Network(CNN) model with attention mechanism to capture the most important features in a question. Finally, the features are fed into the classifier to predict the category of the question. We use the Bi-LSTM and CNN to capture the features of question because of their benefits in representing sentence level documents. We also use the answer set to enrich the information of the question. The experimental results on several datasets demonstrate the effectiveness of the proposed approach.
引文
1Li X, Roth D. Learning question classifiers. Proceedings ofthe 19th International Conference on ComputationalLinguistics. Taipei, China. 2002. 1–7. [doi: 10.3115/1072228.1072378]
2镇丽华, 王小林, 杨思春. 自动问答系统中问句分类研究综述. 安徽工业大学学报 (自然科学版), 2015, 32(1): 48–54,66. [doi: 10.3969/j.issn.1671-7872.2015.01.010]
3Shen D, Pan R, Sun JT, et al. Query enrichment for web-query classification. ACM Transactions on InformationSystems, 2006, 24(3): 320–352. [doi: 10.1145/1165774]
4Broder A, Fontoura M, Gabrilovich E, et al. Robustclassification of rare queries using Web knowledge.Proceedings of the 30th Annual International ACM SIGIRConference on Research and Development in InformationRetrieval (SIGIR’07). Amsterdam, Holland. 2007. 231–238.[doi: 10.1145/1277741.1277783]
5Hui ZJ, Liu J, Ouyang LM. Question classifiaction based onan extend class sequential rule model. Proceedings of the 5thInternational Joint Conference on Natural LanguageProcessing. Chiang Mai, Thailand. 2011. 938–946.
6Mishra M, Mishra VK, Sharma HR. Question classificationusing semantic, syntactic and lexical features. InternationalJournal of Web & Semantic Technology, 2013, 4(3): 39–47.
7Aikawa N, Sakai T, Yamana H. Community QA questionclassification: Is the asker looking for subjective answers ornot? IPSJ Online Transactions, 2011, (4): 160–168. [doi:10.2197/ipsjtrans.4.160]
8Liu L, Yu ZT, Guo JY, et al. Chinese question classificationbased on question property kernel. International Journal ofMachine Learning and Cybernetics, 2014, 5(5): 713–720.[doi: 10.1007/s13042-013-0216-y]
9杨思春, 高超, 秦锋, 等. 融合基本特征和词袋绑定特征的问句特征模型. 中文信息学报, 2012, 26(5): 46–52. [doi:10.3969/j.issn.1003-0077.2012.05.008]
10王艳娜, 孙丙宇. 基于卷积神经网络的烟瘾渴求脑电分类.计算机系统应用, 2017, 26(6): 254–258.
11Wei YC, Zhao Y, Lu CY, et al. Cross-modal retrieval withCNN Visual features: A new baseline. IEEE Transactions onCybernetics, 2017, 47(2): 449–460. [doi: 10.1109/TCYB.2016.2519449]
12Hao YC, Zhang YZ, Liu K, et al. An end-to-end model forquestion answering over knowledge base with cross-attentioncombining global knowledge. Proceedings of the 55thAnnual Meeting of the Association for ComputationalLinguistics. Vancouver, Canada. 2017. 221–231. [doi:10.18653/v1/P17-1021]
13Kim Y. Convolutional neural networks for sentenceclassification. ar Xiv eprint ar Xiv: 1408.5882.
14Shi YY, Yao KS, Tian L, et al. Deep LSTM based featuremapping for query classification. Proceedings of the 2016Conference of the North American Chapter of theAssociation for Computational Linguistics: Human LanguageTechnologies. San Diego, CA. 2016. 1501–1511. [doi: 10.18653/v1/N16-1176]
15Graves A, Mohamed AR, Hinton G. Speech recognition withdeep recurrent neural networks. Proceedings of 2013 IEEEInternational Conference on Acoustics, Speech and SignalProcessing. Vancouver, Canada. 2013. 6645–6649. [doi:10.1109/ICASSP.2013.6638947]
16Bahdanau D, Cho K, Bengio Y. Neural machine translationby jointly learning to align and translate. ar Xiv eprintar Xiv:1409.0473, 2014.
2镇丽华, 王小林, 杨思春. 自动问答系统中问句分类研究综述. 安徽工业大学学报 (自然科学版), 2015, 32(1): 48–54,66. [doi: 10.3969/j.issn.1671-7872.2015.01.010]
3Shen D, Pan R, Sun JT, et al. Query enrichment for web-query classification. ACM Transactions on InformationSystems, 2006, 24(3): 320–352. [doi: 10.1145/1165774]
4Broder A, Fontoura M, Gabrilovich E, et al. Robustclassification of rare queries using Web knowledge.Proceedings of the 30th Annual International ACM SIGIRConference on Research and Development in InformationRetrieval (SIGIR’07). Amsterdam, Holland. 2007. 231–238.[doi: 10.1145/1277741.1277783]
5Hui ZJ, Liu J, Ouyang LM. Question classifiaction based onan extend class sequential rule model. Proceedings of the 5thInternational Joint Conference on Natural LanguageProcessing. Chiang Mai, Thailand. 2011. 938–946.
6Mishra M, Mishra VK, Sharma HR. Question classificationusing semantic, syntactic and lexical features. InternationalJournal of Web & Semantic Technology, 2013, 4(3): 39–47.
7Aikawa N, Sakai T, Yamana H. Community QA questionclassification: Is the asker looking for subjective answers ornot? IPSJ Online Transactions, 2011, (4): 160–168. [doi:10.2197/ipsjtrans.4.160]
8Liu L, Yu ZT, Guo JY, et al. Chinese question classificationbased on question property kernel. International Journal ofMachine Learning and Cybernetics, 2014, 5(5): 713–720.[doi: 10.1007/s13042-013-0216-y]
9杨思春, 高超, 秦锋, 等. 融合基本特征和词袋绑定特征的问句特征模型. 中文信息学报, 2012, 26(5): 46–52. [doi:10.3969/j.issn.1003-0077.2012.05.008]
10王艳娜, 孙丙宇. 基于卷积神经网络的烟瘾渴求脑电分类.计算机系统应用, 2017, 26(6): 254–258.
11Wei YC, Zhao Y, Lu CY, et al. Cross-modal retrieval withCNN Visual features: A new baseline. IEEE Transactions onCybernetics, 2017, 47(2): 449–460. [doi: 10.1109/TCYB.2016.2519449]
12Hao YC, Zhang YZ, Liu K, et al. An end-to-end model forquestion answering over knowledge base with cross-attentioncombining global knowledge. Proceedings of the 55thAnnual Meeting of the Association for ComputationalLinguistics. Vancouver, Canada. 2017. 221–231. [doi:10.18653/v1/P17-1021]
13Kim Y. Convolutional neural networks for sentenceclassification. ar Xiv eprint ar Xiv: 1408.5882.
14Shi YY, Yao KS, Tian L, et al. Deep LSTM based featuremapping for query classification. Proceedings of the 2016Conference of the North American Chapter of theAssociation for Computational Linguistics: Human LanguageTechnologies. San Diego, CA. 2016. 1501–1511. [doi: 10.18653/v1/N16-1176]
15Graves A, Mohamed AR, Hinton G. Speech recognition withdeep recurrent neural networks. Proceedings of 2013 IEEEInternational Conference on Acoustics, Speech and SignalProcessing. Vancouver, Canada. 2013. 6645–6649. [doi:10.1109/ICASSP.2013.6638947]
16Bahdanau D, Cho K, Bengio Y. Neural machine translationby jointly learning to align and translate. ar Xiv eprintar Xiv:1409.0473, 2014.