人机对话系统中意图识别方法综述
|
摘要
口语理解是人机对话系统的重要组成部分,而意图识别是口语理解中的一个子任务,而且至关重要。意图识别的准确性直接关系到语义槽填充的性能并且有助于后续对话系统的研究。考虑到人机对话系统中意图识别的困难,传统的机器学习方法无法理解用户话语的深层语义信息,主要对近些年应用在意图识别研究方面的深度学习方法进行分析、比较和总结,进一步思考如何将深度学习模型应用到多意图识别任务中,从而推动基于深度神经网络的多意图识别方法的研究。
Spoken Language Understanding(SLU)is a vital part of the human-machine dialogue system, which includes an important sub-task called intent detection. The accuracy of intent detection is directly related to the performance of semantic slot filling, and it is helpful to the following research of the dialogue system. Considering the difficulty of intent detection in human-machine dialogue system, the traditional machine learning methods cannot understand the deep semantic information of user's discourse. This paper mainly analyzes, compares and summarizes the deep learning methods applied in the research of intent detection in recent years, and further considers how to apply deep learning model to multi-intent detection task, so as to promote the research of multi-intent detection methods based on deep neural network.
Spoken Language Understanding(SLU)is a vital part of the human-machine dialogue system, which includes an important sub-task called intent detection. The accuracy of intent detection is directly related to the performance of semantic slot filling, and it is helpful to the following research of the dialogue system. Considering the difficulty of intent detection in human-machine dialogue system, the traditional machine learning methods cannot understand the deep semantic information of user's discourse. This paper mainly analyzes, compares and summarizes the deep learning methods applied in the research of intent detection in recent years, and further considers how to apply deep learning model to multi-intent detection task, so as to promote the research of multi-intent detection methods based on deep neural network.
引文
[1]Chen Hongshen,Liu Xiaorui,Yin Dawei,et al.A survey on dialogue systems:recent advances and new frontiers[J].SIGKDD Explorations,2017,19(2):25-35.
[2]Tur G.Spoken language understanding:systems for extracting semantic information from speech[M].New York,NY:John Wiley and Sons,2011.
[3]Austin J A.How to do things with words[M].Cambridge:Harvard University Press,1962.
[4]Celikyilmaz A,Hakkani-Tur D,Tur G,et al.Exploiting distance based similarity in topic models for user intent detection[C]//IEEE Workshop on Automatic Speech Recognition&Understanding,2011:425-430.
[5]Luo Bingfeng,Feng Yansong,Wang Zheng,et al.Marrying up regular expressions with neural networks:a case study for spoken language understanding[C]//Proc of the 56th Annual Meeting of the Association for Computational Linguistics,2018:2083-2093.
[6]李艳玲,颜永红.中文口语理解弱监督训练方法[J].计算机应用,2015,35(7):1965-1968.
[7]Chen Zhiyuan,Liu Bing,Hsu Meichun,et al.Identifying intention posts in discussion forums[C]//Proc of Conference of the North American Chapter of the Association for Computational Linguistics-Human Language Technologies,Atlanta,2013:1041-1050.
[8]钱岳.聊天机器人中用户出行消费意图识别方法研究[D].哈尔滨:哈尔滨工业大学,2017.
[9]Dauphin Y N,Tur G,Hakkani-Tur D,et al.Zero-shot learning for semantic utterance classification[EB/OL].(2013)[2018-12-25].https://arxiv.org/abs/1401.0509v3.
[10]Appelt D,Bear J,Cherny L,et al.GEMINI:a natural language system for spoken-language understanding[C]//Proc Meeting of the Association for Computational Linguistics,1993:54-61.
[11]燕鹏举.对话系统中的自然语言理解研究[D].北京:清华大学,2002.
[12]Ahmad A S,Hassan M Y,Abdullah M P,et al.A review on applications of ANN and SVM for building electrical energy consumption forecasting[J].Renewable&Sustainable Energy Reviews,2014,33(2):102-109.
[13]Prager J,Radev D,Brown E,et al.The use of predictive annotation for question answering in TREC8[C]//Conference of the Eighth Text Retrieval,1999:399-411.
[14]Ramanand J,Bhavsa R K,Pedaneka R N.Wishful thinking:finding suggestions and‘buy’wishes from product reviews[C]//Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text.Stroudsburg,PA:Association for Computational Linguistics,2010:54-61.
[15]Li X,Dan R.Learning question classifiers:the role of semantic information[J].Natural Language Engineering,2015,12(3):229-249.
[16]McCallum A,Nigam K.A comparison of event models for naive bayes text classification[C]//AAAI-98 Workshop on Learning for Text Categorization,1998:41-48.
[17]Schapire R E,Singer Y.BoosTexter:a boosting-based system for text categorization[J].Machine Learning,2000,39(2/3):135-168.
[18]Haffner P,Tur G,Wright J H.Optimizing SVMs for complex call classification[C]//IEEE International Conference on Acoustics,2003:632-635.
[19]Genkin A,Lewis D D,Madigan D.Large-scale bayesian logistic regression for text categorization[J].Technometrics,2007,49(3):291-304.
[20]陈浩辰.基于微博的消费意图挖掘[D].哈尔滨:哈尔滨工业大学,2014.
[21]贾俊华.一种基于AdaBoost和SVM的短文本分类模型[D].天津:河北工业大学,2016.
[22]Kim B,Ryu S,Gary G L.Two-stage multi-intent detection for spoken language understanding[J].Multimedia Tools and Applications,2017,76(9):1-14.
[23]Ratnaparkhi A.Maximum entropy models for natural language ambiguity resolution[D].University of Pennsylvania,1998.
[24]Lafferty J,Mccallum A,Pereira F C N.Conditional Random fields:probabilistic models for segmenting and labeling sequence data[J].Proceedings of ICML,2001,3(2):282-289.
[25]Bengio Y,Ducharme R,Vincent P,et al.A neural probabilistic language model[J].Journal of Machine Learning Research,2003,3(2):1137-1155.
[26]Kim D,Lee Y,Zhang J,et al.Lexical feature embedding for classifying dialogue acts on Korean conversations[C]//Proc of 42nd Winter Conference on Korean Institute of Information Scientists and Engineers,2015:575-577.
[27]Kim J K,Tur G,Celikyilmaz A,et al.Intent detection using semantically enriched word embeddings[C]//Spoken Language Technology Workshop,2016:414-419.
[28]Fellbaum C,Miller G.Word Net:an electronic lexical database[J].Library Quarterly Information Community Policy,1998,25(2):292-296.
[29]Pavlick E,Rastogi P,Ganitkevitch J,et al.PPDB 2.0:better paraphrase ranking,fine-grained entailment relations,word embeddings,and style classification[C]//Meeting of the Association for Computational Linguistics&the International Joint Conference on Natural Language Processing,2015:425-430.
[30]Lecun Y L,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[31]Wang Peng,Xu Jiaming,Xu Bo,et al.Semantic clustering and convolutional neural network for short text categorization[C]//Proceedings ACL,2015:352-357.
[32]Kim Y.Convolutional neural networks for sentence classification[C]//Proc of the 2014 Conference on Empirical Methods in Natural Language Processing,2014:1746-1751.
[33]Hashemi H B,Asiaee A,Kraft R.Query intent detection using convolutional neural networks[C]//International Conference on Web Search and Data Mining,Workshop on Query Understanding,2016.
[34]Bhargava A,Celikyilmaz A,Hakkanitur D,et al.Easy contextual intent prediction and slot detection[C]//IEEEInternational Conference on Acoustics,2013:8337-8341.
[35]Hochreiter S,Schmidhuber J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[36]Ravuri S V,Stolcke A.Recurrent neural network and LSTMmodels for lexical utterance classification[C]//16th Annual Conference of the International Speech Communication Association,2015:135-139.
[37]Dey R,Salemt F M.Gate-variants of gated recurrent unit(GRU)neural networks[C]//IEEE 60th International Midwest Symposium on Circuits and Systems,2017:1597-1600.
[38]Chung J,Gulcehre C,Cho K H,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[EB/OL].(2014)[2018-12-25].https://arxiv.org/abs/1412.3555.
[39]Ravuri S,Stolcke A.A comparative study of recurrent neural network models for lexical domain classification[C]//Proc of the 41st IEEE International Conference on Acoustics,Speech,and Signal Processing,2016:6075-6079.
[40]余慧,冯旭鹏,刘利军,等.聊天机器人中用户就医意图识别方法[J].计算机应用,2018,38(8):2170-2174.
[41]黄佳伟.人机对话系统中用户意图分类方法研究[D].武汉:华中师范大学,2018.
[42]Lin Zhouhan,Feng Minwei,Santos C N D,et al.A structured self-attentive sentence embedding[EB/OL].(2017)[2018-12-25].https://arxiv.org/pdf/1703.03130.pdf.
[43]Hinton G E,Krizhevsky A,Wang S D.Transforming autoencoders[C]//International Conference on Artificial Neural Networks,2011:44-51.
[44]Sabour S,Frosst N,Hinton G E.Dynamic routing between capsules[C]//Advances in Neural Information Processing Systems,2017:3859-3869.
[45]Zhao Wei,Ye Jianbo,Yang Min,et al.Investigating capsule networks with dynamic routing for text classification[C]//Proc of the 2018 Conference on Empirical Methods in Natural Language Processing,2018:3110-3119.
[46]Xia Congying,Zhang Chenwei,Yan Chenwei,et al.Zeroshot user intent detection via capsule neural networks[C]//Proc of the 2018 Conference on Empirical Methods in Natural Language Processing,2018:3090-3099.
[47]李艳玲,颜永红.统计中文口语理解执行策略的研究[J].计算机科学与探索,2017,11(6):980-987.
[48]Liu B,Lane I.Attention-based recurrent neural network models for joint intent detection and slot filling[C]//17th Annual Conference of the International Speech Communication Association,2016:685-689.
[49]杨春妮,冯朝胜.结合句法特征和卷积神经网络的多意图识别模型[J].计算机应用,2018,38(7):1839-1845.
[2]Tur G.Spoken language understanding:systems for extracting semantic information from speech[M].New York,NY:John Wiley and Sons,2011.
[3]Austin J A.How to do things with words[M].Cambridge:Harvard University Press,1962.
[4]Celikyilmaz A,Hakkani-Tur D,Tur G,et al.Exploiting distance based similarity in topic models for user intent detection[C]//IEEE Workshop on Automatic Speech Recognition&Understanding,2011:425-430.
[5]Luo Bingfeng,Feng Yansong,Wang Zheng,et al.Marrying up regular expressions with neural networks:a case study for spoken language understanding[C]//Proc of the 56th Annual Meeting of the Association for Computational Linguistics,2018:2083-2093.
[6]李艳玲,颜永红.中文口语理解弱监督训练方法[J].计算机应用,2015,35(7):1965-1968.
[7]Chen Zhiyuan,Liu Bing,Hsu Meichun,et al.Identifying intention posts in discussion forums[C]//Proc of Conference of the North American Chapter of the Association for Computational Linguistics-Human Language Technologies,Atlanta,2013:1041-1050.
[8]钱岳.聊天机器人中用户出行消费意图识别方法研究[D].哈尔滨:哈尔滨工业大学,2017.
[9]Dauphin Y N,Tur G,Hakkani-Tur D,et al.Zero-shot learning for semantic utterance classification[EB/OL].(2013)[2018-12-25].https://arxiv.org/abs/1401.0509v3.
[10]Appelt D,Bear J,Cherny L,et al.GEMINI:a natural language system for spoken-language understanding[C]//Proc Meeting of the Association for Computational Linguistics,1993:54-61.
[11]燕鹏举.对话系统中的自然语言理解研究[D].北京:清华大学,2002.
[12]Ahmad A S,Hassan M Y,Abdullah M P,et al.A review on applications of ANN and SVM for building electrical energy consumption forecasting[J].Renewable&Sustainable Energy Reviews,2014,33(2):102-109.
[13]Prager J,Radev D,Brown E,et al.The use of predictive annotation for question answering in TREC8[C]//Conference of the Eighth Text Retrieval,1999:399-411.
[14]Ramanand J,Bhavsa R K,Pedaneka R N.Wishful thinking:finding suggestions and‘buy’wishes from product reviews[C]//Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text.Stroudsburg,PA:Association for Computational Linguistics,2010:54-61.
[15]Li X,Dan R.Learning question classifiers:the role of semantic information[J].Natural Language Engineering,2015,12(3):229-249.
[16]McCallum A,Nigam K.A comparison of event models for naive bayes text classification[C]//AAAI-98 Workshop on Learning for Text Categorization,1998:41-48.
[17]Schapire R E,Singer Y.BoosTexter:a boosting-based system for text categorization[J].Machine Learning,2000,39(2/3):135-168.
[18]Haffner P,Tur G,Wright J H.Optimizing SVMs for complex call classification[C]//IEEE International Conference on Acoustics,2003:632-635.
[19]Genkin A,Lewis D D,Madigan D.Large-scale bayesian logistic regression for text categorization[J].Technometrics,2007,49(3):291-304.
[20]陈浩辰.基于微博的消费意图挖掘[D].哈尔滨:哈尔滨工业大学,2014.
[21]贾俊华.一种基于AdaBoost和SVM的短文本分类模型[D].天津:河北工业大学,2016.
[22]Kim B,Ryu S,Gary G L.Two-stage multi-intent detection for spoken language understanding[J].Multimedia Tools and Applications,2017,76(9):1-14.
[23]Ratnaparkhi A.Maximum entropy models for natural language ambiguity resolution[D].University of Pennsylvania,1998.
[24]Lafferty J,Mccallum A,Pereira F C N.Conditional Random fields:probabilistic models for segmenting and labeling sequence data[J].Proceedings of ICML,2001,3(2):282-289.
[25]Bengio Y,Ducharme R,Vincent P,et al.A neural probabilistic language model[J].Journal of Machine Learning Research,2003,3(2):1137-1155.
[26]Kim D,Lee Y,Zhang J,et al.Lexical feature embedding for classifying dialogue acts on Korean conversations[C]//Proc of 42nd Winter Conference on Korean Institute of Information Scientists and Engineers,2015:575-577.
[27]Kim J K,Tur G,Celikyilmaz A,et al.Intent detection using semantically enriched word embeddings[C]//Spoken Language Technology Workshop,2016:414-419.
[28]Fellbaum C,Miller G.Word Net:an electronic lexical database[J].Library Quarterly Information Community Policy,1998,25(2):292-296.
[29]Pavlick E,Rastogi P,Ganitkevitch J,et al.PPDB 2.0:better paraphrase ranking,fine-grained entailment relations,word embeddings,and style classification[C]//Meeting of the Association for Computational Linguistics&the International Joint Conference on Natural Language Processing,2015:425-430.
[30]Lecun Y L,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[31]Wang Peng,Xu Jiaming,Xu Bo,et al.Semantic clustering and convolutional neural network for short text categorization[C]//Proceedings ACL,2015:352-357.
[32]Kim Y.Convolutional neural networks for sentence classification[C]//Proc of the 2014 Conference on Empirical Methods in Natural Language Processing,2014:1746-1751.
[33]Hashemi H B,Asiaee A,Kraft R.Query intent detection using convolutional neural networks[C]//International Conference on Web Search and Data Mining,Workshop on Query Understanding,2016.
[34]Bhargava A,Celikyilmaz A,Hakkanitur D,et al.Easy contextual intent prediction and slot detection[C]//IEEEInternational Conference on Acoustics,2013:8337-8341.
[35]Hochreiter S,Schmidhuber J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[36]Ravuri S V,Stolcke A.Recurrent neural network and LSTMmodels for lexical utterance classification[C]//16th Annual Conference of the International Speech Communication Association,2015:135-139.
[37]Dey R,Salemt F M.Gate-variants of gated recurrent unit(GRU)neural networks[C]//IEEE 60th International Midwest Symposium on Circuits and Systems,2017:1597-1600.
[38]Chung J,Gulcehre C,Cho K H,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[EB/OL].(2014)[2018-12-25].https://arxiv.org/abs/1412.3555.
[39]Ravuri S,Stolcke A.A comparative study of recurrent neural network models for lexical domain classification[C]//Proc of the 41st IEEE International Conference on Acoustics,Speech,and Signal Processing,2016:6075-6079.
[40]余慧,冯旭鹏,刘利军,等.聊天机器人中用户就医意图识别方法[J].计算机应用,2018,38(8):2170-2174.
[41]黄佳伟.人机对话系统中用户意图分类方法研究[D].武汉:华中师范大学,2018.
[42]Lin Zhouhan,Feng Minwei,Santos C N D,et al.A structured self-attentive sentence embedding[EB/OL].(2017)[2018-12-25].https://arxiv.org/pdf/1703.03130.pdf.
[43]Hinton G E,Krizhevsky A,Wang S D.Transforming autoencoders[C]//International Conference on Artificial Neural Networks,2011:44-51.
[44]Sabour S,Frosst N,Hinton G E.Dynamic routing between capsules[C]//Advances in Neural Information Processing Systems,2017:3859-3869.
[45]Zhao Wei,Ye Jianbo,Yang Min,et al.Investigating capsule networks with dynamic routing for text classification[C]//Proc of the 2018 Conference on Empirical Methods in Natural Language Processing,2018:3110-3119.
[46]Xia Congying,Zhang Chenwei,Yan Chenwei,et al.Zeroshot user intent detection via capsule neural networks[C]//Proc of the 2018 Conference on Empirical Methods in Natural Language Processing,2018:3090-3099.
[47]李艳玲,颜永红.统计中文口语理解执行策略的研究[J].计算机科学与探索,2017,11(6):980-987.
[48]Liu B,Lane I.Attention-based recurrent neural network models for joint intent detection and slot filling[C]//17th Annual Conference of the International Speech Communication Association,2016:685-689.
[49]杨春妮,冯朝胜.结合句法特征和卷积神经网络的多意图识别模型[J].计算机应用,2018,38(7):1839-1845.