结合改进主动学习的SVD-CNN弹幕文本分类算法
|
摘要
为解决传统卷积神经网络(CNN)模型使用池化层进行文本特征降维会损失较多文本语义信息的问题,提出一种基于奇异值分解(SVD)算法的卷积神经网络模型(SVD-CNN)。首先,采用改进的基于密度中心点采样的主动学习算法(DBC-AL)选择对分类模型贡献率较高的样本进行标注,以低标注代价获得高质量模型训练集;然后,结合SVD算法建立SVD-CNN弹幕文本分类模型,使用奇异值分解的方法代替传统CNN模型池化层进行特征提取和降维,并在此基础上完成弹幕文本分类任务;最后,使用改进的梯度下降算法(PSGD)对模型参数进行优化。为了验证改进算法的有效性,使用多种弹幕数据样本集,对提出的模型与常用的文本分类模型进行对比实验。实验结果表明,改进的算法能够更好地保留文本语义特征,保证训练过程的稳定性并提高了模型的收敛速度,在不同的弹幕文本上较传统算法具有更好的分类性能。
For the loss of much semantic information in dimension reduction of text features when using pooling layer of the traditional Convolutional Network(CNN) model, a Convolutional Neural Network model based on Singular Value Decomposition algorithm(SVD-CNN) was proposed. Firstly, an improved Active Learning algorithm based on Density Center point sampling(DC-AL) was used to tag samples contributing a lot to the classification model, obtaining a high-quality model training set at a low tagging cost. Secondly, an SVD-CNN barrage text classification model was established by combining SVD algorithm, and SVD was used to replace the traditional CNN model pooling layer for feature extraction and dimension reduction, then the barrage text classification task was completed on these bases. Finally, the model parameters were optimized by using Partial Sampling Gradient Descent algorithm(PSGD). In order to verify the effectiveness of the improved algorithm, multiple barrage data sample sets were used in the comparison experiments between the proposed model and the common text classification model. The experimental results show that the improved algorithm can better preserve semantic features of the text, ensure the stability of training process and improve the convergence speed of the model. In summary, the proposed algorithm has better classification performance than traditional algorithms on multiple barrage texts.
For the loss of much semantic information in dimension reduction of text features when using pooling layer of the traditional Convolutional Network(CNN) model, a Convolutional Neural Network model based on Singular Value Decomposition algorithm(SVD-CNN) was proposed. Firstly, an improved Active Learning algorithm based on Density Center point sampling(DC-AL) was used to tag samples contributing a lot to the classification model, obtaining a high-quality model training set at a low tagging cost. Secondly, an SVD-CNN barrage text classification model was established by combining SVD algorithm, and SVD was used to replace the traditional CNN model pooling layer for feature extraction and dimension reduction, then the barrage text classification task was completed on these bases. Finally, the model parameters were optimized by using Partial Sampling Gradient Descent algorithm(PSGD). In order to verify the effectiveness of the improved algorithm, multiple barrage data sample sets were used in the comparison experiments between the proposed model and the common text classification model. The experimental results show that the improved algorithm can better preserve semantic features of the text, ensure the stability of training process and improve the convergence speed of the model. In summary, the proposed algorithm has better classification performance than traditional algorithms on multiple barrage texts.
引文
[1]谭侃,高旻,李文涛,等.基于双层采样主动学习的社交网络虚假用户检测方法[J].自动化学报,2017,43(3):448-461.(TANK,GAO M,LI W T,et al.Two-layer sampling active learning algorithm for social spammer detection[J].Acta Automatica Sinica,2017,43(3):448-461.)
[2]徐海龙,别晓峰,冯卉,等.一种基于QBC的SVM主动学习算法[J].系统工程与电子技术,2015,37(12):2865-2871.(XU H L,BIE X F,FENG H,et al.Active learning algorithm for SVM based on QBC[J].Systems Engineering and Electronics,2015,37(12):2865-2871.)
[3]姚拓中,安鹏,宋加涛.基于历史分类加权和分级竞争采样的多视角主动学习[J].电子学报,2017,45(1):46-53.(YAO T Z,AN P,SONG J T.Multi-view active learning based on weighted hypothesis boosting and hierarchical competition sampling[J].Atca Electronica Sinica,2017,45(1):46-53.)
[4]LI M,WANG R,TANG K.Combining semi-supervised and active learning for hyperspectral image classification[C]//Proceedings of the 2013 IEEE Symposium on Computational Intelligence and Data Mining.Piscataway,NJ:IEEE,2013:89-94.
[5]WAN L,TANG K,LI M,et al.Collaborative active and semisupervised learning for hyperspectral remote sensing image classification[J].IEEE Transactions on Geoscience and Remote Sensing,2015,53(5):2384-2396.
[6]WANG Z,DU B,ZHANG L,et al.A novel semisupervised activelearning algorithm for hyperspectral image classification[J].IEEETransactions on Geoscience and Remote Sensing,2017,55(6):3071-3083.
[7]SAMIAPPAN S,MOORHEAD R J.Semi-supervised co-training and active learning framework for hyperspectral image classification[C]//Proceedings of the 2015 IEEE International Geoscience and Remote Sensing Symposium.Piscataway,NJ:IEEE,2015:401-404.
[8]魏超,罗森林,张竞,等.自编码网络短文本流形表示方法[J].浙江大学学报(工学版),2015,49(8):1591-1599.(WEI C,LUO SL,ZHANG J,et al.Short text manifold representation based on autoencoder network[J].Journal of Zhejiang University(Engineering Science),2015,49(8):1591-1599.)
[9]谢金宝,侯永进,康守强,等.基于语义理解注意力神经网络的多元特征融合中文文本分类[J].电子与信息学报,2018,40(5):1258-1265.(XIE J B,HOU Y J,KANG S Q,et al.Multi-feature fusion based on semantic understanding attention neural network for Chinese text categorization[J].Journal of Electronics and Information Technology,2018,40(5):1258-1265.)
[10]孙松涛,何炎祥.基于CNN特征空间的微博多标签情感分类[J].四川大学学报(工程科学版),2017,49(3):162-169.(SUNS T,HE Y X.Multi-label emotion classification for microblog based on CNN feature space[J].Journal of Sichuan University(Engineering Science Edition),2017,49(3):162-169.)
[11]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.A convolutional neural network for modelling sentences[EB/OL].[2018-07-11].https://arxiv.org/pdf/1404.2188v1.pdf.
[12]KIM Y.Convolutional neural networks for sentence classification[EB/OL].[2018-07-11].https://arxiv.org/pdf/1408.5882.pdf.
[13]郑啸,王义真,袁志祥,等.基于卷积记忆神经网络的微博短文本情感分析[J].电子测量与仪器学报,2018(3):195-200.(ZHENG X,WANG Y Z,YUAN Z X,et al.Sentiment analysis of micro-blog short text based on convolutional memory neural network[J].Journal of Electronic Meatruement and Instrumentation,2018(3):195-200.)
[14]HSU S T,MOON C,JONES P,et al.A hybrid CNN-RNN alignment model for phrase-aware sentence classification[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver:[s.n.],2017,2:433-449.
[15]YIN W,SCHUTZE H,XIANG B,et al.ABCNN:attention-based convolutional neural network for modeling sentence pairs[EB/OL].[2018-07-11].https://arxiv.org/pdf/1512.05193.pdf.
[16]陈荣,曹永锋,孙洪.基于主动学习和半监督学习的多类图像分类[J].自动化学报,2011,37(8):954-962.(CHEN R,CAO YF,SUN H.Multi-class image classification with active learning and semi-supervised learning[J].Acta Automatica Sinica,2011,37(8):954-962.)
[17]陈珂,梁斌,柯文德,等.基于多通道卷积神经网络的中文微博情感分析[J].计算机研究与发展,2018,55(5):945-957.(CHEN K,LIANG B,KE W D,et al.Chinese micro-blog sentiment analysis based on multi-channels convolutional neural networks[J].Journal of Computer Research and Development,2018,55(5):945-957.)
[2]徐海龙,别晓峰,冯卉,等.一种基于QBC的SVM主动学习算法[J].系统工程与电子技术,2015,37(12):2865-2871.(XU H L,BIE X F,FENG H,et al.Active learning algorithm for SVM based on QBC[J].Systems Engineering and Electronics,2015,37(12):2865-2871.)
[3]姚拓中,安鹏,宋加涛.基于历史分类加权和分级竞争采样的多视角主动学习[J].电子学报,2017,45(1):46-53.(YAO T Z,AN P,SONG J T.Multi-view active learning based on weighted hypothesis boosting and hierarchical competition sampling[J].Atca Electronica Sinica,2017,45(1):46-53.)
[4]LI M,WANG R,TANG K.Combining semi-supervised and active learning for hyperspectral image classification[C]//Proceedings of the 2013 IEEE Symposium on Computational Intelligence and Data Mining.Piscataway,NJ:IEEE,2013:89-94.
[5]WAN L,TANG K,LI M,et al.Collaborative active and semisupervised learning for hyperspectral remote sensing image classification[J].IEEE Transactions on Geoscience and Remote Sensing,2015,53(5):2384-2396.
[6]WANG Z,DU B,ZHANG L,et al.A novel semisupervised activelearning algorithm for hyperspectral image classification[J].IEEETransactions on Geoscience and Remote Sensing,2017,55(6):3071-3083.
[7]SAMIAPPAN S,MOORHEAD R J.Semi-supervised co-training and active learning framework for hyperspectral image classification[C]//Proceedings of the 2015 IEEE International Geoscience and Remote Sensing Symposium.Piscataway,NJ:IEEE,2015:401-404.
[8]魏超,罗森林,张竞,等.自编码网络短文本流形表示方法[J].浙江大学学报(工学版),2015,49(8):1591-1599.(WEI C,LUO SL,ZHANG J,et al.Short text manifold representation based on autoencoder network[J].Journal of Zhejiang University(Engineering Science),2015,49(8):1591-1599.)
[9]谢金宝,侯永进,康守强,等.基于语义理解注意力神经网络的多元特征融合中文文本分类[J].电子与信息学报,2018,40(5):1258-1265.(XIE J B,HOU Y J,KANG S Q,et al.Multi-feature fusion based on semantic understanding attention neural network for Chinese text categorization[J].Journal of Electronics and Information Technology,2018,40(5):1258-1265.)
[10]孙松涛,何炎祥.基于CNN特征空间的微博多标签情感分类[J].四川大学学报(工程科学版),2017,49(3):162-169.(SUNS T,HE Y X.Multi-label emotion classification for microblog based on CNN feature space[J].Journal of Sichuan University(Engineering Science Edition),2017,49(3):162-169.)
[11]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.A convolutional neural network for modelling sentences[EB/OL].[2018-07-11].https://arxiv.org/pdf/1404.2188v1.pdf.
[12]KIM Y.Convolutional neural networks for sentence classification[EB/OL].[2018-07-11].https://arxiv.org/pdf/1408.5882.pdf.
[13]郑啸,王义真,袁志祥,等.基于卷积记忆神经网络的微博短文本情感分析[J].电子测量与仪器学报,2018(3):195-200.(ZHENG X,WANG Y Z,YUAN Z X,et al.Sentiment analysis of micro-blog short text based on convolutional memory neural network[J].Journal of Electronic Meatruement and Instrumentation,2018(3):195-200.)
[14]HSU S T,MOON C,JONES P,et al.A hybrid CNN-RNN alignment model for phrase-aware sentence classification[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver:[s.n.],2017,2:433-449.
[15]YIN W,SCHUTZE H,XIANG B,et al.ABCNN:attention-based convolutional neural network for modeling sentence pairs[EB/OL].[2018-07-11].https://arxiv.org/pdf/1512.05193.pdf.
[16]陈荣,曹永锋,孙洪.基于主动学习和半监督学习的多类图像分类[J].自动化学报,2011,37(8):954-962.(CHEN R,CAO YF,SUN H.Multi-class image classification with active learning and semi-supervised learning[J].Acta Automatica Sinica,2011,37(8):954-962.)
[17]陈珂,梁斌,柯文德,等.基于多通道卷积神经网络的中文微博情感分析[J].计算机研究与发展,2018,55(5):945-957.(CHEN K,LIANG B,KE W D,et al.Chinese micro-blog sentiment analysis based on multi-channels convolutional neural networks[J].Journal of Computer Research and Development,2018,55(5):945-957.)