图文融合的微博情感分析方法
|
摘要
为解决单独基于文本或图片的情感分析方法不能充分挖掘微博用户情感的问题,提出一种图文融合的微博情感分析方法。对大规模图片数据集上预训练的CNN模型参数进行迁移,以微调的方式训练图片情感分类模型FCNN;通过训练词向量模型将文本表示为包含丰富语义信息的词向量,并输入可提取文本语义单元之间上下文特征的双向LSTM中,训练文本情感分类模型WBLSTM;根据late fusion的模型融合思想,设计模型融合公式融合FCNN模型与WBLSTM模型,进行图文融合的微博情感分析。实验结果表明,该方法比单独基于文本或图片的微博情感分析方法具有更好的情感分类效果。
To solve the problem that sentiment analysis approach that merely based on texts or images cannot fully discover the sentiment of microblog users,ajoint visual-textual approach was proposed for microblog sentiment analysis.Parameters of the CNN model pre-trained on a large-scale image datasets were transferred,and the image sentiment classification model named FCNN was trained in a fine-tuned way.Texts were represented as word vectors that contained rich semantic information by training word vector model,and word vectors were input into the bidirectional LSTM that extracted the contextual feature between text semantic units to train text sentiment classification model named WBLSTM.According to the idea of late fusion,a model fusion formula was designed to fuse FCNN model and WBLSTM model,to carry out joint visual-textual microblog sentiment analysis.The experimental results show that sentiment classification effects of the proposed approach outperform the approach merely based on texts or images.
To solve the problem that sentiment analysis approach that merely based on texts or images cannot fully discover the sentiment of microblog users,ajoint visual-textual approach was proposed for microblog sentiment analysis.Parameters of the CNN model pre-trained on a large-scale image datasets were transferred,and the image sentiment classification model named FCNN was trained in a fine-tuned way.Texts were represented as word vectors that contained rich semantic information by training word vector model,and word vectors were input into the bidirectional LSTM that extracted the contextual feature between text semantic units to train text sentiment classification model named WBLSTM.According to the idea of late fusion,a model fusion formula was designed to fuse FCNN model and WBLSTM model,to carry out joint visual-textual microblog sentiment analysis.The experimental results show that sentiment classification effects of the proposed approach outperform the approach merely based on texts or images.
引文
[1]HE Yanxiang,SUN Songtao,NIU Feifei,et al. A deep learning model enhanced with emotion semantics for microblog sentiment analysis[J].Chinese Journal of Computers,2017,40(4):773-790(in Chinese).[何炎祥,孙松涛,牛菲菲,等.用于微博情感分析的一种情感语义增强的深度学习模型[J].计算机学报,2017,40(4):773-790.]
[2]Sun X,Li C,Xu W,et al.Chinese microblog sentiment classification based on deep belief nets with extended multi-modality features[C]//IEEE International Conference on Data Mining Workshop,2014:928-935.
[3]Tang D,Wei F,Qin B,et al.Coooolll:A deep learning system for twitter sentiment classification[C]//Proceedings of the 8th International Workshop on Semantic Evaluation,2014:208-212.
[4]HUANG Faliang,FENG Shi,WANG Daling,et al.Mining topic sentiment in microblogging based on multi-feature fusion[J].Chinese Journal of Computers,2017,40(4):872-888(in Chinese).[黄发良,冯时,王大玲,等.基于多特征融合的微博主题情感挖掘[J].计算机学报,2017,40(4):872-888.]
[5]Cao D,Ji R,Lin D,et al.Visual sentiment topic model based microblog image sentiment analysis[J].Multimedia Tools&Applications,2016,75(15):8955-8968.
[6]You Q,Yang J,Yang J,et al.Robust image sentiment analysis using progressively trained and domain transferred deep networks[C]//29th AAAI Conference on Artificial Intelligence,2015:381-388.
[7]Islam J,Zhang Y.Visual sentiment analysis for social images using transfer learning approach[C]//IEEE International Conferences on Big Data and Cloud Computing.IEEE Computer Society,2016:124-130.
[8]Wang M,Cao D,Li L,et al.Microblog sentiment analysis based on cross-media bag-of-words model[C]//Proceedings of International Conference on Internet Multimedia Computing and Service.ACM,2014:76.
[9]Borth D,Ji R,Chen T,et al.Large-scale visual sentiment ontology and detectors using adjective noun pairs[C]//Proceedings of the 21st ACM International Conference on Multimedia,2013:223-232.
[10]You Q,Luo J,Jin H,et al.Joint visual-textual sentiment analysis with deep neural networks[C]//Proceedings of the23rd ACM International Conference on Multimedia,2015:1071-1074.
[11]You Q,Yang J,Yang J,et al.Robust image sentiment analysis using progressively trained and domain transferred deep networks[C]//29th AAAI Conference on Artificial Intelligence,2015:381-388.
[12]Krizhevsky A,Sutskever I,Hinton GE.ImageNet classification with deep convolutional neural networks[C]//International Conference on Neural Information Processing Systems.Curran Associates Inc,2012:1097-1105.
[13]Graves A,Jaitly N,Mohamed AR.Hybrid speech recognition with deep bidirectional LSTM[C]//Automatic Speech Recognition and Understanding.IEEE,2014:273-278.
[14]You Q,Luo J,Jin H,et al.Building a large scale dataset for image emotion recognition:The fine print and the benchmark[C]//AAAI,2016:308-314.
[2]Sun X,Li C,Xu W,et al.Chinese microblog sentiment classification based on deep belief nets with extended multi-modality features[C]//IEEE International Conference on Data Mining Workshop,2014:928-935.
[3]Tang D,Wei F,Qin B,et al.Coooolll:A deep learning system for twitter sentiment classification[C]//Proceedings of the 8th International Workshop on Semantic Evaluation,2014:208-212.
[4]HUANG Faliang,FENG Shi,WANG Daling,et al.Mining topic sentiment in microblogging based on multi-feature fusion[J].Chinese Journal of Computers,2017,40(4):872-888(in Chinese).[黄发良,冯时,王大玲,等.基于多特征融合的微博主题情感挖掘[J].计算机学报,2017,40(4):872-888.]
[5]Cao D,Ji R,Lin D,et al.Visual sentiment topic model based microblog image sentiment analysis[J].Multimedia Tools&Applications,2016,75(15):8955-8968.
[6]You Q,Yang J,Yang J,et al.Robust image sentiment analysis using progressively trained and domain transferred deep networks[C]//29th AAAI Conference on Artificial Intelligence,2015:381-388.
[7]Islam J,Zhang Y.Visual sentiment analysis for social images using transfer learning approach[C]//IEEE International Conferences on Big Data and Cloud Computing.IEEE Computer Society,2016:124-130.
[8]Wang M,Cao D,Li L,et al.Microblog sentiment analysis based on cross-media bag-of-words model[C]//Proceedings of International Conference on Internet Multimedia Computing and Service.ACM,2014:76.
[9]Borth D,Ji R,Chen T,et al.Large-scale visual sentiment ontology and detectors using adjective noun pairs[C]//Proceedings of the 21st ACM International Conference on Multimedia,2013:223-232.
[10]You Q,Luo J,Jin H,et al.Joint visual-textual sentiment analysis with deep neural networks[C]//Proceedings of the23rd ACM International Conference on Multimedia,2015:1071-1074.
[11]You Q,Yang J,Yang J,et al.Robust image sentiment analysis using progressively trained and domain transferred deep networks[C]//29th AAAI Conference on Artificial Intelligence,2015:381-388.
[12]Krizhevsky A,Sutskever I,Hinton GE.ImageNet classification with deep convolutional neural networks[C]//International Conference on Neural Information Processing Systems.Curran Associates Inc,2012:1097-1105.
[13]Graves A,Jaitly N,Mohamed AR.Hybrid speech recognition with deep bidirectional LSTM[C]//Automatic Speech Recognition and Understanding.IEEE,2014:273-278.
[14]You Q,Luo J,Jin H,et al.Building a large scale dataset for image emotion recognition:The fine print and the benchmark[C]//AAAI,2016:308-314.