融合群稀疏与排他性稀疏正则项的神经网络压缩情感分析方法
|
摘要
文本情感分析是目前网络环境下舆情监控、服务评价及满意度分析等领域的重要任务,一些基于深度神经网络的方法已被用于此类任务。规模庞大的深度神经网络模型结构赋予了深度学习模型强大的非线性拟合的能力,大规模的数据资源为训练这样大规模的模型并保证其泛化能力提供了可能性。然而,在实际应用中,深度模型的时间和空间开销仍然制约着这些方法的落地。针对上述问题,提出一种融合群稀疏与排他性稀疏正则项的神经网络压缩情感分析方法,首先分别构建循环-卷积神经网路与卷积-循环神经网络,通过门控单元融合两种网络组成的分析模型,在模型中引入群稀疏与排他性稀疏正则项,剪除冗余神经元或链接,压缩模型规模。在不同数据集上的实验结果验证了本文方法的有效性。
In the current internet environment, text sentiment analysis is one of the most important tasks in the field of public opinion monitoring and analysis of customer service satisfaction. Deep network models offer a great improvement over the traditional methods in sentiment analysis tasks, since large-scale networks enable a deep model to fit nonlinear data effectively and large-scale data resources provide the possibility to train such a large-scale model and guarantee its generalization ability. However, the time and space costs of the deep model still restrict its application. This paper proposes a neural network compressed sentiment analysis method that combines group spar-sity and exclusive sparsity regularization. We first construct an analysis model composed of RCNN and C-RNN which is integrated by a gate unit. Group sparsity and exclusive sparsity regularization are then introduced in order to make the model compressed. Experiments on different data sets verify the effectiveness of our method.
In the current internet environment, text sentiment analysis is one of the most important tasks in the field of public opinion monitoring and analysis of customer service satisfaction. Deep network models offer a great improvement over the traditional methods in sentiment analysis tasks, since large-scale networks enable a deep model to fit nonlinear data effectively and large-scale data resources provide the possibility to train such a large-scale model and guarantee its generalization ability. However, the time and space costs of the deep model still restrict its application. This paper proposes a neural network compressed sentiment analysis method that combines group spar-sity and exclusive sparsity regularization. We first construct an analysis model composed of RCNN and C-RNN which is integrated by a gate unit. Group sparsity and exclusive sparsity regularization are then introduced in order to make the model compressed. Experiments on different data sets verify the effectiveness of our method.
引文
[1] SOCHER R. Recursive deep learning for natural lan-guage processing and computer vision[D]. Palo Alto: Stanford University, 2014.
[2] SOCHER R, CHEN D Q, MANNING C D, et al. Reasoning with neural tensor networks for knowledge base completion[C]//Advances in Neural Information Processing Systems. Lake Tahoe, 2013: 926- 934.
[3] SOCHER R, PERELYGIN A, WU J Y, et al. Recur-sive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle, 2013: 1631- 1642.
[4] LUONG M T, SOCHER R, MANNING C D. Better word representations with recursive neural networks for morphology[C]//Proceedings of the Seventeenth Conference on Computational Natural Language Learning. Sofia, 2013: 104- 113.
[5] SOCHER R, HUVAL B, MANNING C D, et al. Se-mantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Jeju Island, 2012: 1201- 1211.
[6] LE Q V, NGIAM J, COATES A, et al. On optimization methods for deep learning[C]//Proceedings of the 28th International Conference on Machine Learning. Bellevue: Omni Press, 2011: 265- 272.
[7] COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011, 12: 2493- 2537.
[8] KALCHBRENNER N, BLUNSOM P. Recurrent convolutional neural networks for discourse compositionality[J]. arXiv preprint arXiv: 1306. 3584, 2013.
[9] CLICHE M. BB_twtr at SemEval-2017 Task 4: Twitter sentiment analysis with CNNs and LSTMs[J]. arXiv preprint arXiv: 1704. 06125, 2017.
[10] SHI B G, BAI X, YAO C. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(11): 2298- 2304.
[11] KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences[J]. arXiv preprint arXiv: 1404. 2188, 2014.
[12] LAI S W, XU L H, LIU K, et al. Recurrent convolu-tional neural networks for text classification[C]//Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. Austin, 2015: 2267- 2273.
[13] WEN W, WU C P, WANG Y D, et al. Learning struc-tured sparsity in deep neural networks[C]//30th Conference on Neural Information Processing Systems. Barcelona, 2016: 2074- 2082.
[14] ALVAREZ J M, SALZMANN M. Learning the number of neurons in deep networks[C]// 30th Conference on Neural Information Processing Systems. Barcelona, 2016: 2270- 2278.
[15] ZHOU Y, JIN R, HOI S C H. Exclusive lasso for multi-task feature selection[C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Sardinia, 2010: 988- 995.
[16] Stanford Press Release. Stanford sentiment tree-bank[EB/OL]. [2018- 11- 01]. https: //nlp. stanford. edu/sentiment/index. html.
[17] KINGMA D P, BA L J. Adam: a method for stochastic optimization[J]. arXiv preprint arXiv: 1412. 6980, 2014.
[2] SOCHER R, CHEN D Q, MANNING C D, et al. Reasoning with neural tensor networks for knowledge base completion[C]//Advances in Neural Information Processing Systems. Lake Tahoe, 2013: 926- 934.
[3] SOCHER R, PERELYGIN A, WU J Y, et al. Recur-sive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle, 2013: 1631- 1642.
[4] LUONG M T, SOCHER R, MANNING C D. Better word representations with recursive neural networks for morphology[C]//Proceedings of the Seventeenth Conference on Computational Natural Language Learning. Sofia, 2013: 104- 113.
[5] SOCHER R, HUVAL B, MANNING C D, et al. Se-mantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Jeju Island, 2012: 1201- 1211.
[6] LE Q V, NGIAM J, COATES A, et al. On optimization methods for deep learning[C]//Proceedings of the 28th International Conference on Machine Learning. Bellevue: Omni Press, 2011: 265- 272.
[7] COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011, 12: 2493- 2537.
[8] KALCHBRENNER N, BLUNSOM P. Recurrent convolutional neural networks for discourse compositionality[J]. arXiv preprint arXiv: 1306. 3584, 2013.
[9] CLICHE M. BB_twtr at SemEval-2017 Task 4: Twitter sentiment analysis with CNNs and LSTMs[J]. arXiv preprint arXiv: 1704. 06125, 2017.
[10] SHI B G, BAI X, YAO C. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(11): 2298- 2304.
[11] KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences[J]. arXiv preprint arXiv: 1404. 2188, 2014.
[12] LAI S W, XU L H, LIU K, et al. Recurrent convolu-tional neural networks for text classification[C]//Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. Austin, 2015: 2267- 2273.
[13] WEN W, WU C P, WANG Y D, et al. Learning struc-tured sparsity in deep neural networks[C]//30th Conference on Neural Information Processing Systems. Barcelona, 2016: 2074- 2082.
[14] ALVAREZ J M, SALZMANN M. Learning the number of neurons in deep networks[C]// 30th Conference on Neural Information Processing Systems. Barcelona, 2016: 2270- 2278.
[15] ZHOU Y, JIN R, HOI S C H. Exclusive lasso for multi-task feature selection[C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Sardinia, 2010: 988- 995.
[16] Stanford Press Release. Stanford sentiment tree-bank[EB/OL]. [2018- 11- 01]. https: //nlp. stanford. edu/sentiment/index. html.
[17] KINGMA D P, BA L J. Adam: a method for stochastic optimization[J]. arXiv preprint arXiv: 1412. 6980, 2014.