融合单词翻译的神经机器翻译
详细信息    查看全文 | 推荐本文 |
  • 英文篇名:Modeling Word Translation to Neural Machine Translation
  • 作者:韩冬 ; 李军辉 ; 周国栋
  • 英文作者:HAN Dong;LI Junhui;ZHOU Guodong;School of Computer Science and Technology,Soochow University;
  • 关键词:单词翻译 ; Transformer ; 神经机器翻译
  • 英文关键词:word translation;;Transformer;;neural machine translation
  • 中文刊名:MESS
  • 英文刊名:Journal of Chinese Information Processing
  • 机构:苏州大学计算机科学与技术学院;
  • 出版日期:2019-07-15
  • 出版单位:中文信息学报
  • 年:2019
  • 期:v.33
  • 基金:国家自然科学基金(61502149,61401295)
  • 语种:中文;
  • 页:MESS201907006
  • 页数:6
  • CN:07
  • ISSN:11-2325/N
  • 分类号:45-50
摘要
神经机器翻译由于无法完全学习源端单词语义信息,往往造成翻译结果中存在着大量的单词翻译错误。该文提出了一种融入单词翻译用以增强源端信息的神经机器翻译方法。首先使用字典方法找到每个源端单词对应的目标端翻译,然后提出并比较两种不同的方式,用以融合源端单词及其翻译信息:①Factored编码器:单词及其翻译信息直接相加;②Gated编码器:通过门机制控制单词翻译信息的输入。基于目前性能最优的基于自注意力机制的神经机器翻译框架Transformer,在中英翻译任务的实验结果表明,与基准系统相比,该文提出的两种融合源端单词译文的方式均能显著提高翻译性能,BLEU值获得了0.81个点的提升。
        Due to incapability of fully learning the semantic details of source words,neural machine translation(NMT)tends to have a large number of wrong word translations in translation output.This paper proposes to explicitly incorporate word translation into NMT encoder.Firstly,the dictionary method is used to find the corresponding word translation for each source word.Then two different ways are proposed to fuse the source word and its translation information:(1)Factored Encoder:words and their translation information are added directly;(2)Gated Encoder:controls the input of word translation information through gate mechanism.Based on the state-ofthe-art NMT framework of transformer with self-attention mechanism,experimental results on Chinese-English translation task show that the proposed encoders can significantly improve the performance,especially the Gated Encoder method achieves 0.81 BLEU scores improvement over the baseline system.
引文
[1]Dzmitry Bahdanau,Kyunghyun Cho,Yoshua Bengio.Neural Machine Translation by Jointly Learning to Align and Translate[J].arXiv preprint arXiv:1409.0473v7,2016.
    [2]Minh-Thang Luong,Hieu Pham,Christopher D Manning.Effective Approaches to Attention-based Neural Machine Translation[C]//Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing,Lisbon,Portugal,2015.
    [3]Jonas Gehring,Michael Auli,David Grangier,et al.A convolutional encoder model for neural machine translation[J].arXiv preprint arXiv:1611.02344,2016.
    [4]Yonghui Wu,Mike Schuster,Zhifeng Chen,et al.Google’s Neural Machine Translation System:Bridging the Gap between Human and Machine Translation[J].arXiv preprint arXiv:1609.08144,2016.
    [5]Denny Britz,Anna Goldie,Thang Luong,et al.Massive Exploration of Neural Machine Translation Architectures[J].arXiv preprint arXiv:1703.03906,2017.
    [6]Ashish Vaswani,Noam Shazeer,Niki Parmar,et al.Attention Is All You Need[J].arXiv preprint arXiv:1706.03762,2017.
    [7]Martin Sundermeyer,Ralf Schlüter,Hermann Ney.LSTM Neural Networks for Language Modeling[C]//Proceedings of Interspeech.2012:601-608.
    [8]Jonas Gehring,Michael Auli,David Grangier,,et al.Convolutional Sequence to Sequence Learning[J].arXiv preprint arXiv:1705.03122,2017.
    [9]Frederick Liu,Han Lu,Graham Neubig.Handling Homographs in Neural Machine Translation[J].arXiv preprint arXiv:1708.06510,2017.
    [10]Lemao Liu,Masao Utiyama,Andrew Finch,et al.Neural Machine Translation with Supervised Attention[C]//Proceedings of COLING2016,2016.
    [11]Haitao Mi,Zhiguo Wang,Abe Ittycheriah.Supervised Attentions for Neural Machine Translation[C]//Proceedings of Conference on Empirical Methods in Natural Language Processing,Austin,Texas,2016:2283-2288.
    [12]Chris Dyer,Victor Chahuneau,Noah A Smith.A simple,fast,and effective reparameterization of IBM model 2[C]//Proceedings of the 2013Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Atlanta,Georgia,2013.
    [13]Junhui Li,Deyi Xiong,Zhaopeng Tu,et al.Modeling Source Syntax for Neural Machine Translation[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics,Vancouver,Canada,2017:688-697.
    [14]Rico Sennrich,Barry Haddow.Linguistic Input Features Improve Neural Machine Translation[C]//Proceedings of the 1st Conference on Machine Translation.2016:83-91.
    [15]Diederik Kingma,Jimmy Ba.Adam:A Method for Stochastic Optimization[C]//Proceedings of the 3rd International Conference on Learning Representations,2015.
    [16]Kishore Papineni,Salim Roukos,Todd Ward,et al.BLEU:A method for automatic evaluation of MT,Computer Science RC22176(W0109-022)[R].New York:IBM T.J.Watson Research Center.2002.
    (1)其中包括了:LDC2002E18,LDC2003E07,LDC2003E14,LDC2004T07,LDC2004T08和LDC2005T06。
NGLC 2004-2010.National Geological Library of China All Rights Reserved.
Add:29 Xueyuan Rd,Haidian District,Beijing,PRC. Mail Add: 8324 mailbox 100083
For exchange or info please contact us via email.