基于公共特征空间的自适应情感分类
|
摘要
针对情感分类这一项从文章或句子中得到观点态度的任务,常规情感分类模型大多需要耗费大量人力获取标注数据.为解决某些领域缺乏标注数据,且其他领域分类器无法在目标领域直接使用的现状,设计了一种新颖的基于构建公共特征空间方法,使分类模型可从有标注领域向无标注领域进行迁移适应,减少人工标注的成本开销,实现情感分类的领域自适应.该方法以大规模语料下预训练的词向量信息作为以词为元素的特征,在同种语言中表达情感所采用的句法结构相似这一假设前提下,通过对领域内特有的领域特征词进行替换的方式构建有标注数据集与无标注数据集基本共有的公共特征空间,使有标注数据集与无标注数据集实现信息共享.以此为基础借助深度学习中卷积神经网络采用不同尺寸卷积核对词语不同范围的上下文特征进行抽取学习,进而采用半监督学习与微调学习相结合的方式从有标注数据集向未标注数据集开展领域自适应.在来自京东与携程共5个领域的真实电商数据集上进行实验,分别研究了领域特征词选择方法及其词性约束对领域间适应能力的影响,结果表明:相较于不采用领域适应的模型,可提升平均2.7%的准确率;且在来自亚马逊电商的公开数据集实验中,通过与现有方法进行对比,验证了该方法的有效性.
Sentiment classification, which extracts the opinions from sentences/documents,has been extensively studied. Most of the conventional sentiment classification models require a lot of cost to obtain the labeled data. In order to solve the problem that a trained classifier from other domain cannot be used directly on the target domain which lack labeled data,we proposed a novel domain adaptation model with reconstructing a common feature representation. This model makes the classifier from the labeled domain adapt to the unlabeled domain,reduces the cost of manual labeling and achieves the domain adaptation of sentiment classification. This model utilizes the pre-trained word vectors as the feature of the words. With the premise that the syntactic structure used to express sentiment in the same language is similar,a common feature space shared by the labeled and unlabeled data set is reconstructed by replacing the special domain words that unique to the domain. Therefore,the information sharing between the labeled and unlabeled data sets is realized. Based on this,the convolutional neural network in the model uses different size of convolution kernels to extract the context features of different range of words. With semi-supervised learning and finetuning learning,the model can be domain adapted from the labeled domain to the unlabeled domain. In experiments based on real data from Jingdong and Xiecheng, we separately compared the influence of different domain words selection and different POS constraints on the performance of our model,and found our model can improve the accuracy by about 2.7% compared to our baseline. In addition,we compared our model with related works on the public data from Amazon,and verified the effectiveness of our model.
Sentiment classification, which extracts the opinions from sentences/documents,has been extensively studied. Most of the conventional sentiment classification models require a lot of cost to obtain the labeled data. In order to solve the problem that a trained classifier from other domain cannot be used directly on the target domain which lack labeled data,we proposed a novel domain adaptation model with reconstructing a common feature representation. This model makes the classifier from the labeled domain adapt to the unlabeled domain,reduces the cost of manual labeling and achieves the domain adaptation of sentiment classification. This model utilizes the pre-trained word vectors as the feature of the words. With the premise that the syntactic structure used to express sentiment in the same language is similar,a common feature space shared by the labeled and unlabeled data set is reconstructed by replacing the special domain words that unique to the domain. Therefore,the information sharing between the labeled and unlabeled data sets is realized. Based on this,the convolutional neural network in the model uses different size of convolution kernels to extract the context features of different range of words. With semi-supervised learning and finetuning learning,the model can be domain adapted from the labeled domain to the unlabeled domain. In experiments based on real data from Jingdong and Xiecheng, we separately compared the influence of different domain words selection and different POS constraints on the performance of our model,and found our model can improve the accuracy by about 2.7% compared to our baseline. In addition,we compared our model with related works on the public data from Amazon,and verified the effectiveness of our model.
引文
[1]Pang B,Lee L,Vaithyanathan S.Thumbs up?:Sentiment classification using machine learning techniques[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing.Philadelphia,USA,2002:79-86.
[2]Pang B,Lee L.Seeing stars:Exploiting class relationships for sentiment categorization with respect to rating scales[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics.Ann Arbor,USA,2005:115-124.
[3]Yang Changhua,Lin K,Chen H.Emotion classification using web blog corpora[C]//IEEE/WIC/ACM International Conference on Web Intelligence.California,USA,2007:275-278.
[4]Read J.Using emoticons to reduce dependency in machine learning techniques for sentiment classification[C]//Proceedings of the ACL Student Research Workshop.Ann Arbor,USA,2005:43-48.
[5]Hassan A,Abbasi A,Zeng D.Twitter sentiment analysis:A bootstrap ensemble framework[C]//2013International Conference on Social Computing.Washington,DC,USA,2013:357-364.
[6]Jiang Jing,Zhai Chengxiang.Instance weighting for domain adaptation in nlp[C]//Proceedings of the 45th Annual Meeting on Association for Computational Linguistics.Prague,Czech Republic,2007:264-271.
[7]Dai Wenyuan,Yang Qiang,Xue Guirong,et al.Boosting for transfer learning[C]//Proceedings of the24th International Conference on Machine Learning.Corvallis,USA,2007:193-200.
[8]Huang Jiayuan,Gretton A,Borgwardt K M,et al.Correcting sample selection bias by unlabeled data[C]//Advances in Neural Information Processing Systems.Vancouver,Canada,2007:601-608.
[9]Sugiyama M,Nakajima S,Kashima H,et al.Direct importance estimation with model selection and its application to covariate shift adaptation[C]//Advances in Neural Information Processing Systems.Vancouver,Canada,2008:1433-1440.
[10]Bendavid S,Blitzer J,Crammer K,et al.Analysis of representations for domain adaptation[C]//Advances in Neural Information Processing Systems.Vancouver,Canada,2007:137-144.
[11]Blitzer J,Dredze M,Pereira F.Biographies,bollywood,boom-boxes and blenders:Domain adaptation for sentiment classification[C]//Proceedings of the 45th Annual Meeting on Association for Computational Linguistics.Prague,Czech Republic,2007:440-447.
[12]Ando R K,Zhang T.A framework for learning predictive structures from multiple tasks and unlabeled data[J].Journal of Machine Learning Research,2005(6):1817-1853.
[13]Argyriou A,Evgeniou T,Pontil M.Multi-task feature learning[C]//Advances in Neural Information Processing Systems.Vancouver,Canada,2007:41-48.
[14]Raina R,Battle A,Lee H,et al.Self-taught learning:Transfer learning from unlabeled data[C]//Proceedings of the 24th International Conference on Machine Learning.Corvallis,USA,2007:759-766.
[15]Pan S J,Kwok J T,Yang Q.Transfer learning via dimensionality reduction[C]//AAAI Conference on Artificial Intelligence.Chicago,USA,2008:677-682.
[16]Bollegala D,Weir D,Carroll J.Cross-domain sentiment classification using a sentiment sensitive thesaurus[J].IEEE Transactions on Knowledge and Data Engineering,2013,25(8):1719-1731.
[17]Pan S J,Ni Xiaochuan,Sun J T,et al.Cross-domain sentiment classification via spectral feature alignment[C]//Proceedings of the 19th International Conference on World Wide Web.Montreal,Canada,2010:751-760.
[18]Kalchbrenner N,Grefenstette E,Blunsom P.Aconvolutional neural network for modelling sentences[C]//the 52nd Annual Meeting of the Association for Computational Linguistics.Baltimore,USA,2014:655-665.
[19]Hu Baotian,Lu Zhengdong,Li Hang,et al.Convolutional neural network architectures for matching natural language sentences[C]//Advances in Neural Information Processing Systems.Montreal,Canada,2014:2042-2050.
[20]Kim Y.Convolutional neural networks for sentence classification[J].Empirical Methods in Natural Language Processing,2014:1746-1751.
[21]Collobert R,Weston J,Bottou L,et al.,Natural language processing(almost)from scratch[J].Journal of Machine Learning Research,2011,12(8):2493-2537.
[22]Glorot X,Bordes A,Bengio Y.Domain adaptation for large-scale sentiment classification:A deep learning approach[C]//Proceedings of the 28th International Conference on Machine Learning.Bellevue,USA,2011:513-520.
[23]Blum A,Mitchell T M.Combining labeled and unlabeled data with co-training[C]//Proceedings of the11th Annual Conference on Computational Learning Theory.Madison,USA,1998:92-100.
[24]Joachims T.Transductive inference for text classification using support vector machines[C]//16th International Conference on Machine Learning.San Francisco,USA,1999:200-209.
[25]Nigam K,Mccallum A,Thrun S,et al.Text classification from labeled and unlabeled documents using em[J].Machine Learning,2000,39:103-134.
[26]Tajbakhsh N,Shin J Y,Gurudu S R,et al.Convolutional neural networks for medical image analysis:Full training or fine tuning?[J].IEEETransactions on Medical Imaging,2016,35(5):1299-1312.
[27]Van der Maaten L,Hinton G.Visualizing data using tsne[J].Journal of Machine Learning Research,2008,9(11):2579-2605.
[28]Srivastava N,Hinton G E,Krizhevsky A,et al.Dropout:A simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research,2014,15(1):1929-1958.
[29]Mikolov T,Sutskever I,Chen K,et al.Distributed representations of words and phrases and their compositionality[C]//Advances in Neural Information Processing Systems.Lake Tahoe,USA,2013:3111-3119.
[2]Pang B,Lee L.Seeing stars:Exploiting class relationships for sentiment categorization with respect to rating scales[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics.Ann Arbor,USA,2005:115-124.
[3]Yang Changhua,Lin K,Chen H.Emotion classification using web blog corpora[C]//IEEE/WIC/ACM International Conference on Web Intelligence.California,USA,2007:275-278.
[4]Read J.Using emoticons to reduce dependency in machine learning techniques for sentiment classification[C]//Proceedings of the ACL Student Research Workshop.Ann Arbor,USA,2005:43-48.
[5]Hassan A,Abbasi A,Zeng D.Twitter sentiment analysis:A bootstrap ensemble framework[C]//2013International Conference on Social Computing.Washington,DC,USA,2013:357-364.
[6]Jiang Jing,Zhai Chengxiang.Instance weighting for domain adaptation in nlp[C]//Proceedings of the 45th Annual Meeting on Association for Computational Linguistics.Prague,Czech Republic,2007:264-271.
[7]Dai Wenyuan,Yang Qiang,Xue Guirong,et al.Boosting for transfer learning[C]//Proceedings of the24th International Conference on Machine Learning.Corvallis,USA,2007:193-200.
[8]Huang Jiayuan,Gretton A,Borgwardt K M,et al.Correcting sample selection bias by unlabeled data[C]//Advances in Neural Information Processing Systems.Vancouver,Canada,2007:601-608.
[9]Sugiyama M,Nakajima S,Kashima H,et al.Direct importance estimation with model selection and its application to covariate shift adaptation[C]//Advances in Neural Information Processing Systems.Vancouver,Canada,2008:1433-1440.
[10]Bendavid S,Blitzer J,Crammer K,et al.Analysis of representations for domain adaptation[C]//Advances in Neural Information Processing Systems.Vancouver,Canada,2007:137-144.
[11]Blitzer J,Dredze M,Pereira F.Biographies,bollywood,boom-boxes and blenders:Domain adaptation for sentiment classification[C]//Proceedings of the 45th Annual Meeting on Association for Computational Linguistics.Prague,Czech Republic,2007:440-447.
[12]Ando R K,Zhang T.A framework for learning predictive structures from multiple tasks and unlabeled data[J].Journal of Machine Learning Research,2005(6):1817-1853.
[13]Argyriou A,Evgeniou T,Pontil M.Multi-task feature learning[C]//Advances in Neural Information Processing Systems.Vancouver,Canada,2007:41-48.
[14]Raina R,Battle A,Lee H,et al.Self-taught learning:Transfer learning from unlabeled data[C]//Proceedings of the 24th International Conference on Machine Learning.Corvallis,USA,2007:759-766.
[15]Pan S J,Kwok J T,Yang Q.Transfer learning via dimensionality reduction[C]//AAAI Conference on Artificial Intelligence.Chicago,USA,2008:677-682.
[16]Bollegala D,Weir D,Carroll J.Cross-domain sentiment classification using a sentiment sensitive thesaurus[J].IEEE Transactions on Knowledge and Data Engineering,2013,25(8):1719-1731.
[17]Pan S J,Ni Xiaochuan,Sun J T,et al.Cross-domain sentiment classification via spectral feature alignment[C]//Proceedings of the 19th International Conference on World Wide Web.Montreal,Canada,2010:751-760.
[18]Kalchbrenner N,Grefenstette E,Blunsom P.Aconvolutional neural network for modelling sentences[C]//the 52nd Annual Meeting of the Association for Computational Linguistics.Baltimore,USA,2014:655-665.
[19]Hu Baotian,Lu Zhengdong,Li Hang,et al.Convolutional neural network architectures for matching natural language sentences[C]//Advances in Neural Information Processing Systems.Montreal,Canada,2014:2042-2050.
[20]Kim Y.Convolutional neural networks for sentence classification[J].Empirical Methods in Natural Language Processing,2014:1746-1751.
[21]Collobert R,Weston J,Bottou L,et al.,Natural language processing(almost)from scratch[J].Journal of Machine Learning Research,2011,12(8):2493-2537.
[22]Glorot X,Bordes A,Bengio Y.Domain adaptation for large-scale sentiment classification:A deep learning approach[C]//Proceedings of the 28th International Conference on Machine Learning.Bellevue,USA,2011:513-520.
[23]Blum A,Mitchell T M.Combining labeled and unlabeled data with co-training[C]//Proceedings of the11th Annual Conference on Computational Learning Theory.Madison,USA,1998:92-100.
[24]Joachims T.Transductive inference for text classification using support vector machines[C]//16th International Conference on Machine Learning.San Francisco,USA,1999:200-209.
[25]Nigam K,Mccallum A,Thrun S,et al.Text classification from labeled and unlabeled documents using em[J].Machine Learning,2000,39:103-134.
[26]Tajbakhsh N,Shin J Y,Gurudu S R,et al.Convolutional neural networks for medical image analysis:Full training or fine tuning?[J].IEEETransactions on Medical Imaging,2016,35(5):1299-1312.
[27]Van der Maaten L,Hinton G.Visualizing data using tsne[J].Journal of Machine Learning Research,2008,9(11):2579-2605.
[28]Srivastava N,Hinton G E,Krizhevsky A,et al.Dropout:A simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research,2014,15(1):1929-1958.
[29]Mikolov T,Sutskever I,Chen K,et al.Distributed representations of words and phrases and their compositionality[C]//Advances in Neural Information Processing Systems.Lake Tahoe,USA,2013:3111-3119.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |