融合词性的维吾尔语文本分类研究
|
摘要
目前维吾尔语文本分类仍存在特征空间的高维度问题以及文本向量表示的高稀疏性问题,为了更好地解决这一问题。首先根据词性特征过滤掉对文本分类无意义词或者包含少量分类特性的词汇,然后分别在KNN、决策树、SVM、CNN、RNN、CNN-BLSTM等分类器上进行实验。实验结果表明,与传统语料相比,根据词性过滤过的语料对多数分类器实验的准确率均有所提高,并且大大缩短训练时间。
At present, Uyghur language text classification still has problems with high dimension of feature space and high sparsity of text vector expression, in order to solve this problem better, firstly, according to the part-of-speech feature, filters the non-meaningful words or the words containing a small amount of classification characteristics for text categorization, then carries out experiments on KNN, decision tree, SVM, CNN, RNN, CNN-BLSTM and so on. The results show that according to experiment, compared with the training without wordof-speech filtering, the accuracy rate has been improved on most classifiers, and training time has been greatly shortened.
At present, Uyghur language text classification still has problems with high dimension of feature space and high sparsity of text vector expression, in order to solve this problem better, firstly, according to the part-of-speech feature, filters the non-meaningful words or the words containing a small amount of classification characteristics for text categorization, then carries out experiments on KNN, decision tree, SVM, CNN, RNN, CNN-BLSTM and so on. The results show that according to experiment, compared with the training without wordof-speech filtering, the accuracy rate has been improved on most classifiers, and training time has been greatly shortened.
引文
[1]张勇.基于词性与LDA主题模型的文本分类技术研究[D].安徽大学,2016.
[2]艾海麦提江·阿布来提,吐尔地·托合提,艾斯卡尔·艾木都拉.基于Naive Bayes的维吾尔文文本分类算法及其性能分析[J].计算机应用与软件,2012,29(12):27-29.
[3]阿力木江·艾沙,吐尔根·依布拉音,库尔班·吾布力,等.基于短语的维吾尔文文本分类[J].计算机应用,2012,32(10):2923-2926.
[4]买买提依明·哈斯木,吾守尔·斯拉木,维尼拉·木沙江,等.基于N元模型的维吾尔文文本分类技术研究[J].计算机应用研究,2015,32(7):1986-1988.
[5]吐尔地·托合提,维尼拉·木沙江,艾斯卡尔·艾木都拉.基于语义串抽取及主题相似度度量的维吾尔文文本分类[J].中文信息学报,2017,31(4):100-107.
[6]胡燕,吴虎子,钟珞.中文文本分类中基于词性的特征提取方法研究[J].武汉理工大学学报,2007,29(4):132-135.
[7]路永和,王鸿滨.文本分类中受词性影响的特征权重计算方法[J].现代图书情报技术,2015,31(4):18-25.
[8]黄贤英,李沁东,刘英涛.结合词性的短文本相似度算法及其在文本分类中的应用[J].电讯技术,2017,57(1):78-82.
[9]阿力甫·阿不都克里木,李晓.基于TextRank算法和互信息相似度的维吾尔文关键词提取及文本分类[J].计算机科学,2016,43(12):36-40.
[10]Maimaiti M,Wumaier A,Abiderexiti K,et al. Bidirectional Long Short-Term Memory Network with a Conditional Random Field Layer for Uyghur Part-Of-Speech Tagging[J]. Information,2017,8(4):157.
[2]艾海麦提江·阿布来提,吐尔地·托合提,艾斯卡尔·艾木都拉.基于Naive Bayes的维吾尔文文本分类算法及其性能分析[J].计算机应用与软件,2012,29(12):27-29.
[3]阿力木江·艾沙,吐尔根·依布拉音,库尔班·吾布力,等.基于短语的维吾尔文文本分类[J].计算机应用,2012,32(10):2923-2926.
[4]买买提依明·哈斯木,吾守尔·斯拉木,维尼拉·木沙江,等.基于N元模型的维吾尔文文本分类技术研究[J].计算机应用研究,2015,32(7):1986-1988.
[5]吐尔地·托合提,维尼拉·木沙江,艾斯卡尔·艾木都拉.基于语义串抽取及主题相似度度量的维吾尔文文本分类[J].中文信息学报,2017,31(4):100-107.
[6]胡燕,吴虎子,钟珞.中文文本分类中基于词性的特征提取方法研究[J].武汉理工大学学报,2007,29(4):132-135.
[7]路永和,王鸿滨.文本分类中受词性影响的特征权重计算方法[J].现代图书情报技术,2015,31(4):18-25.
[8]黄贤英,李沁东,刘英涛.结合词性的短文本相似度算法及其在文本分类中的应用[J].电讯技术,2017,57(1):78-82.
[9]阿力甫·阿不都克里木,李晓.基于TextRank算法和互信息相似度的维吾尔文关键词提取及文本分类[J].计算机科学,2016,43(12):36-40.
[10]Maimaiti M,Wumaier A,Abiderexiti K,et al. Bidirectional Long Short-Term Memory Network with a Conditional Random Field Layer for Uyghur Part-Of-Speech Tagging[J]. Information,2017,8(4):157.