融合卷积神经网络与层次化注意力网络的中文文本情感倾向性分析
|
摘要
文本情感倾向性分析是自然语言处理研究领域的一个基础问题。基于深度学习的模型是处理此问题的常用模型。而当前的多数深度学习模型在中文文本情感倾向性分析方面的应用存在两个问题:一是未能充分考虑到文本的层次化结构对情感倾向性判定的重要作用,二是传统的分词技术在处理文本时会产生歧义。该文针对这些问题基于卷积神经网络与层次化注意力网络的优点提出了一种深度学习模型C-HAN(Convolutional Neural Network-based and Hierarchical Attention Network-based Chinese Sentiment Classification Model),先用并行化卷积层学习词向量间的联系与组合形式,再将其结果输入到基本单元为双向循环神经网络的层次化注意力网络中判定情感倾向。实验表明:模型在中文评论数据集上倾向性分类准确率达到92.34%,和现有多个情感分析模型相比有所提升;此外,对于中文文本,选择使用字级别词向量作为原始特征会优于词级别词向量作为原始特征。
Text sentiment orientation analysis is a fundamental problem in natural language processing.To further improve the deep learning based models used in this issue,this paper proposes a new model named C-HAN,i.e.Convolutional Neural Network-based and Hierarchical Attention Network-based Chinese Sentiment Classification Model.It utilizes a convolution layer to extract a sequence of higher-level phrase representations,which are then fed into a hierarchical attention network to obtain the final representations.On the Chinese sentiment analysis corpus,the character level C-HAN achieves a sentiment prediction accuracy of 92.34%,slightly better than the word level C-HAN yielding 91.96% accuracy.
Text sentiment orientation analysis is a fundamental problem in natural language processing.To further improve the deep learning based models used in this issue,this paper proposes a new model named C-HAN,i.e.Convolutional Neural Network-based and Hierarchical Attention Network-based Chinese Sentiment Classification Model.It utilizes a convolution layer to extract a sequence of higher-level phrase representations,which are then fed into a hierarchical attention network to obtain the final representations.On the Chinese sentiment analysis corpus,the character level C-HAN achieves a sentiment prediction accuracy of 92.34%,slightly better than the word level C-HAN yielding 91.96% accuracy.
引文
[1] Liu B.Sentiment analysis:Mining opinions,sentiments,and emotions[J].Computational Linguistics,2015,42(3):1-4.
[2] Ortony A,Clore G L,Collins A.The cognitive structure of emotions[M].Cambridge University Press,1990.
[3] Pang B,Lillian L,Vaithyanathan S.Thumbsup?:sentiment classification using machine learning techniques[C]//Proceedings of ACL,2002:79-86.
[4] Hu M,Liu B.Mining and summarizing customer reviews[C]//Proceedings of SIGKDD,2004:168-177.
[5]梁军,等.基于深度学习的微博情感分析[J].中文信息学报,2014,28(5):155-161.
[6] Zhang X,Zhao J,Lecun Y.Character-level convolutional networks for text classification[C]//Proceedings of NIPS,2015:645-657.
[7] Kim Y,et al.Character-aware neural language models[C]//Proceedings of AAAI,2016:2741-2749.
[8] Kamps J,Marx M.Words with attitude[C]//Proceedings of International Conference on Global WordNet,2002:332-341.
[9] Esuli A,Sebastiani F.Pageranking wordnet synsets:An application to opinion mining[C]//Proceedings of Annual Conference of the Association for Computational Linguistics,2007:442-431.
[10] Zhen D D,Qiang D.HowNet and the computation of meaning(With Cd-rom)[M].World Scientific,2006.
[11] Pang B,Lee L.A Sentimental education:Sentiment analysis using subjectivity summarization based on minimum cuts[C]//Proceedings of ACL,2004:271-278.
[12] Collobert R,et al.Natural language processing(almost)from scratch[J].Journal of Machine Learning Research,2011,12(8):2493-2537.
[13] Kim Y.Convolutional neural networks for sentence classification[C]//Proceedings of EMNLP,2014.
[14] Kalchbrenner N,Grefenstette E,Blunsom P.A convolutional neural network for modelling sentences[J].arXiv preprint arXiv:1404.2188,2014.
[15] Conneau,et al.Very deep convolutional networks for text classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics,2017.
[16] Tang D,Qin B,Liu T.Document modeling with gated recurrent neural network for sentiment classification[C]//Proceedings of EMNLP,2015:1422-1432.
[17] Wang B.Disconnected recurrent neural networks for text categorization[C]//Proceedings of ACL,2018:2311-2320.
[18] Lai S,et al.Recurrent convolutional neural networks for text classification[C]//Proceedings of AAAI,2015:2267-2273.
[19] Zhou C T,et al.A C-LSTM neural network for text classification[J].Computer Science,2015,1(4):39-44.
[20] Yang Z,et al.Hierarchical attention networks for document classification[C]//Proceedings of the 2016Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2016:1480-1489.
[21]刘龙飞,等.基于卷积神经网络的微博情感倾向性分析[J].中文信息学报,2015,29(6):159-165.
[22] Cho K,et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[J].arXiv preprint arXiv:1406.1078,2014.
[23] Bahdanau D,Cho K,Bengio Y.Neural machine translation by jointly learning to align and translate[J].arXiv preprint arXiv:1409.0473,2014.
[24] Yin W,et al.Abcnn:Attention-based convolutional neural network for modeling sentence pairs[J].arXiv preprint arXiv:1512.05193,2015.
[25]栾克鑫,等.基于注意力机制的句子排序方法[J].中文信息学报,2018,32(1):123-130.
[26] Zhang Y,Wallace B.A sensitivity analysis of(and practitioners'guide to)convolutional neural networks for sentence classification[J].arXiv preprint arXiv:1510.03820,2015.
[27] Mikolov T,et al.Distributed representations of words and phrases and their compositionality[C]//Proceedings of Advances in Neural Information Processing Systems,2013:3111-3119.
[28] Chollet F,Keras[CP/OL].[2015].https://github.com/fchollet/keras.
[29] Kingma D,Jimmy B.Adam:A method for stochastic optimization[J].arXiv preprint arXiv:1412.6980,2014.
[30] Bojanowski P,et al.Enriching word vectors with subword information[J].arXiv preprint arXiv:1607.04606,2016.
[2] Ortony A,Clore G L,Collins A.The cognitive structure of emotions[M].Cambridge University Press,1990.
[3] Pang B,Lillian L,Vaithyanathan S.Thumbsup?:sentiment classification using machine learning techniques[C]//Proceedings of ACL,2002:79-86.
[4] Hu M,Liu B.Mining and summarizing customer reviews[C]//Proceedings of SIGKDD,2004:168-177.
[5]梁军,等.基于深度学习的微博情感分析[J].中文信息学报,2014,28(5):155-161.
[6] Zhang X,Zhao J,Lecun Y.Character-level convolutional networks for text classification[C]//Proceedings of NIPS,2015:645-657.
[7] Kim Y,et al.Character-aware neural language models[C]//Proceedings of AAAI,2016:2741-2749.
[8] Kamps J,Marx M.Words with attitude[C]//Proceedings of International Conference on Global WordNet,2002:332-341.
[9] Esuli A,Sebastiani F.Pageranking wordnet synsets:An application to opinion mining[C]//Proceedings of Annual Conference of the Association for Computational Linguistics,2007:442-431.
[10] Zhen D D,Qiang D.HowNet and the computation of meaning(With Cd-rom)[M].World Scientific,2006.
[11] Pang B,Lee L.A Sentimental education:Sentiment analysis using subjectivity summarization based on minimum cuts[C]//Proceedings of ACL,2004:271-278.
[12] Collobert R,et al.Natural language processing(almost)from scratch[J].Journal of Machine Learning Research,2011,12(8):2493-2537.
[13] Kim Y.Convolutional neural networks for sentence classification[C]//Proceedings of EMNLP,2014.
[14] Kalchbrenner N,Grefenstette E,Blunsom P.A convolutional neural network for modelling sentences[J].arXiv preprint arXiv:1404.2188,2014.
[15] Conneau,et al.Very deep convolutional networks for text classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics,2017.
[16] Tang D,Qin B,Liu T.Document modeling with gated recurrent neural network for sentiment classification[C]//Proceedings of EMNLP,2015:1422-1432.
[17] Wang B.Disconnected recurrent neural networks for text categorization[C]//Proceedings of ACL,2018:2311-2320.
[18] Lai S,et al.Recurrent convolutional neural networks for text classification[C]//Proceedings of AAAI,2015:2267-2273.
[19] Zhou C T,et al.A C-LSTM neural network for text classification[J].Computer Science,2015,1(4):39-44.
[20] Yang Z,et al.Hierarchical attention networks for document classification[C]//Proceedings of the 2016Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2016:1480-1489.
[21]刘龙飞,等.基于卷积神经网络的微博情感倾向性分析[J].中文信息学报,2015,29(6):159-165.
[22] Cho K,et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[J].arXiv preprint arXiv:1406.1078,2014.
[23] Bahdanau D,Cho K,Bengio Y.Neural machine translation by jointly learning to align and translate[J].arXiv preprint arXiv:1409.0473,2014.
[24] Yin W,et al.Abcnn:Attention-based convolutional neural network for modeling sentence pairs[J].arXiv preprint arXiv:1512.05193,2015.
[25]栾克鑫,等.基于注意力机制的句子排序方法[J].中文信息学报,2018,32(1):123-130.
[26] Zhang Y,Wallace B.A sensitivity analysis of(and practitioners'guide to)convolutional neural networks for sentence classification[J].arXiv preprint arXiv:1510.03820,2015.
[27] Mikolov T,et al.Distributed representations of words and phrases and their compositionality[C]//Proceedings of Advances in Neural Information Processing Systems,2013:3111-3119.
[28] Chollet F,Keras[CP/OL].[2015].https://github.com/fchollet/keras.
[29] Kingma D,Jimmy B.Adam:A method for stochastic optimization[J].arXiv preprint arXiv:1412.6980,2014.
[30] Bojanowski P,et al.Enriching word vectors with subword information[J].arXiv preprint arXiv:1607.04606,2016.