基于谷歌翻译及Doc2vec的中英句子相似度计算
|
摘要
句子相似度计算在统计机器翻译、基于实例的机器翻译与语料对齐领域有着巨大的研究价值。本文借助于谷歌翻译作为媒介,针对由Word2vec改进而来的表示句子向量的方法——Doc2vec模型,对中英句子相似度计算进行了研究。分别利用谷歌翻译将中文译文进行中文句向量训练及计算与英文译文的英文句向量训练及计算对比分析。实验结果表明,基于Doc2vec的方法在计算句子相似度方面,无论是中文句向量还是英文句向量,其作为句向量表达句子含义的准确性都还有待商榷。
Sentence similarity calculation has great research value in the fields of statistical machine translation, instance-based machine translation and corpus alignment. This paper uses Google Translate as a favorable medium to study the similarity between Chinese and English sentences based on the Doc2 vec model, which is improved by Word2 vec, which represents the sentence vector. Using Google Translate to use Chinese translation to carry out Chinese sentence vector training and calculation and English sentence vector training and calculation comparison analysis. The experimental results show that the Doc2 vec-based method in terms of calculating sentence similarity, whether it is Chinese sentence vector or English sentence vector, its accuracy as a sentence vector to express sentence meaning is still open to question.
Sentence similarity calculation has great research value in the fields of statistical machine translation, instance-based machine translation and corpus alignment. This paper uses Google Translate as a favorable medium to study the similarity between Chinese and English sentences based on the Doc2 vec model, which is improved by Word2 vec, which represents the sentence vector. Using Google Translate to use Chinese translation to carry out Chinese sentence vector training and calculation and English sentence vector training and calculation comparison analysis. The experimental results show that the Doc2 vec-based method in terms of calculating sentence similarity, whether it is Chinese sentence vector or English sentence vector, its accuracy as a sentence vector to express sentence meaning is still open to question.
引文
[1]熊富林,邓怡豪,唐晓晟.Word2vec的核心架构及其应用[J].南京师范大学学报(工程技术版),2015,15(1):43-48.
[2]王明文,徐雄飞,徐凡,等.基于word2vec的大中华区词对齐库的构建[J].中文信息学报,2015,29(5):76-83.
[3]唐明,朱磊,邹显春.基于Word2Vec的一种文档向量表示[J].计算机科学,2016,43(06):214-217+269.
[4]唐亮,席耀一,彭波,等.基于词向量的越汉跨语言事件检索研究[J].中文信息学报,2018,32(3):64-70.
[5]张剑,屈丹,李真.基于词向量特征的循环神经网络语言模型[J].模式识别与人工智能,2015,28(4):299-305.
[6]杨阳,刘龙飞,魏现辉,等.基于词向量的情感新词发现方法[J].山东大学学报(理学版),2014,49(11):51-58.
[7]杨宇婷,王名扬,田宪允,等.基于文档分布式表达的新浪微博情感分类研究[J].情报杂志,2016,35(2):151-156.
[8]贾晓婷,王名扬,曹宇.结合Doc2Vec与改进聚类算法的中文单文档自动摘要方法研究[J].数据分析与知识发现,2018,2(2):86-95.
[9]姜天文,秦兵,刘挺.基于表示学习的开放域中文知识推理[J].中文信息学报,2018,32(3):34-41.
[10]刘知远,孙茂松,林衍凯,等.知识表示学习研究进展[J].计算机研究与发展,2016,53(2):247-261.
[2]王明文,徐雄飞,徐凡,等.基于word2vec的大中华区词对齐库的构建[J].中文信息学报,2015,29(5):76-83.
[3]唐明,朱磊,邹显春.基于Word2Vec的一种文档向量表示[J].计算机科学,2016,43(06):214-217+269.
[4]唐亮,席耀一,彭波,等.基于词向量的越汉跨语言事件检索研究[J].中文信息学报,2018,32(3):64-70.
[5]张剑,屈丹,李真.基于词向量特征的循环神经网络语言模型[J].模式识别与人工智能,2015,28(4):299-305.
[6]杨阳,刘龙飞,魏现辉,等.基于词向量的情感新词发现方法[J].山东大学学报(理学版),2014,49(11):51-58.
[7]杨宇婷,王名扬,田宪允,等.基于文档分布式表达的新浪微博情感分类研究[J].情报杂志,2016,35(2):151-156.
[8]贾晓婷,王名扬,曹宇.结合Doc2Vec与改进聚类算法的中文单文档自动摘要方法研究[J].数据分析与知识发现,2018,2(2):86-95.
[9]姜天文,秦兵,刘挺.基于表示学习的开放域中文知识推理[J].中文信息学报,2018,32(3):34-41.
[10]刘知远,孙茂松,林衍凯,等.知识表示学习研究进展[J].计算机研究与发展,2016,53(2):247-261.