基于领域特征的神经机器翻译领域适应方法
|
摘要
神经机器翻译在资源丰富领域上训练的翻译模型往往在其他资源稀缺领域中表现较差,领域适应是利用资源丰富的领域帮助资源稀少的领域提升翻译质量的一种方法。该文提出基于领域特征的领域适应方法以提升资源稀缺领域的神经机器翻译质量。具体而言,该文尝试构建领域敏感网络以获得领域特有特征,构建领域不敏感网络以获得领域间的共有特征。一个领域判别器被用于区分领域。该文通过训练领域敏感网络使得该领域判别器更易做出准确判断,同时引入对抗机制,使得领域不敏感网络欺骗该领域判别器。最后,提出一种系统集成机制,融合基准神经翻译网络、领域敏感网络、领域不敏感网络以完成神经机器翻译的领域适应。实验结果显示,该方法在中英广播对话领域上和英德口语领域上的翻译效果均有显著提升。
Translation models trained by neural machine translation system in resource rich areas tend to perform poorly in resource poor areas.This paper proposes domain adaptation based on domain features to improve the quality of neural machine translation with poor resource.Specifically,this paper establishes domain sensitive networks to obtain domain specific features,as well as to build domain insensitive networks to obtain common features between domains.A domain discriminator is used to distinguish the domain.This paper trained domain sensitive network to make it easier for the domain discriminator to make accurate judgements.At the same time,the adversarial mechanism is used so that the domain insensitive network can deceive the domain discriminator.Finally,a system combination mechanism is proposed by combining the base neural translation network,the domain sensitive network,and the domain insensitive network for the domain adaptation task.The experimental results show that this method achieves significant improvement in Chinese-English Broadcast Conversation translation task and English-German Spoken Language translation task.
Translation models trained by neural machine translation system in resource rich areas tend to perform poorly in resource poor areas.This paper proposes domain adaptation based on domain features to improve the quality of neural machine translation with poor resource.Specifically,this paper establishes domain sensitive networks to obtain domain specific features,as well as to build domain insensitive networks to obtain common features between domains.A domain discriminator is used to distinguish the domain.This paper trained domain sensitive network to make it easier for the domain discriminator to make accurate judgements.At the same time,the adversarial mechanism is used so that the domain insensitive network can deceive the domain discriminator.Finally,a system combination mechanism is proposed by combining the base neural translation network,the domain sensitive network,and the domain insensitive network for the domain adaptation task.The experimental results show that this method achieves significant improvement in Chinese-English Broadcast Conversation translation task and English-German Spoken Language translation task.
引文
[1]Dzmitry Bahdanau,Kyunghyun Cho,Yoshua Bengio.Neural machine translation by jointly learning to align and translate[J].arXiv preprint arXiv:1409.0473,2014.
[2]Minh-Thang Luong,Hieu Pham,Christopher D Manning.Effective approaches to attention-based neural machine translation[C]//Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal,2015:1412-1421.
[3]李亚超,熊德意,张民.神经机器翻译综述[J].计算机学报,2018,41(12):100-121.
[4]Rui Wang,Hai Zhao,Bao-Liang Lu,et al.Connecting phrase based statistical machine translation adaptation[C]//Proceedings of the 26th International Conference on Computational Linguistics.Osaka,Japan,2016:3135-3145.
[5]Shafiq Joty,Hassan Sajjad,Nadir Durrani,et al.How to avoid unwanted pregnancies:Domain adaptation using neural network models[C]//Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal,2015:1259-1270.
[6]Rui Wang,Andrew Finch,Masao Utiyama,et al.Sentence embedding for neural machine translation domain adaptation[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada,2017:560-566.
[7]Marlies van der Wees, Arianna Bisazza,Christof Monz.Dynamic data selection for neural machine translation[J].arXiv preprint arXiv:1708.00712,2017.
[8]Rui Wang,Masao Utiyama,Lemao Liu,et al.Instance weighting for neural machine translation domain adaptation[C]//Proceedings of the 2017Conference on Empirical Methods in Natural Language Processing.Copenhagen,Denmark,2017:1482-1488.
[9]Catherine Kobus,Josep Crego,Jean Senellart.Domain control for neural machine translation[J].arXiv preprint arXiv:1612.06140,2016.
[10]Minh-Thang Luong,Christopher D Manning.Stanford neural machine translation systems for spoken language domains[C]//Proceedings of the International Workshop on Spoken Language Translation.Da Nang,Vietnam,2015:76-79.
[11]Ian J Goodfellow, Jonathon Shlens, Christian Szegedy.Generative adversarial nets[C]//Proceedings of the Advances in Neural Information Processing Systems,2014:2672-2680.
[12]Lijun Wu,Yingce Xia,Li Zhao,et al.Adversarial neural machine translation[J].arXiv preprint arXiv:1704.06933,2017.
[13]Zhen Yang,Wei Chen,Feng Wang,et al.Improving neural machine translation with conditional sequence generative adversarial nets[J].arXiv preprint arXiv:1703.04887,2017.
[14]Sebastien Jean,Kyunghyun Cho,Roland Memisevic.On using very large target vocabulary for neural machine translation[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics(ACL).Beijing,China,2015:1-10.
[15]Ekaterina Garmash,Christof Monz.Ensemble learning for multi-source neural machine translation[C]//Proceedings of the 26th International Conference on Computational Linguistics. Osaka,Japan,2016:1409-1418.
[16]Mauro Cettolo,Jan Niehues,et al.Report on the11th IWSLT evaluation campaign[C]//Proceedings of the International Workshop on Spoken Language Translation.Hanoi,Vietnam,2014:2-17.
[17]Sennrich R,Haddow B,Birch A.Neural machine translation of rare words with subword units[J].arXiv preprint arXiv:1508.07909,2015.
[18]Kishore Papineni,Salim Roukos,Todd Ward.et al.BLEU:A method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting on Association for Computational Linguistics.Philadelphia,Pennsylvania,2002:311-318.
(1)https://github.com/pytorch/fairseq/tree/v0.4.0
[2]Minh-Thang Luong,Hieu Pham,Christopher D Manning.Effective approaches to attention-based neural machine translation[C]//Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal,2015:1412-1421.
[3]李亚超,熊德意,张民.神经机器翻译综述[J].计算机学报,2018,41(12):100-121.
[4]Rui Wang,Hai Zhao,Bao-Liang Lu,et al.Connecting phrase based statistical machine translation adaptation[C]//Proceedings of the 26th International Conference on Computational Linguistics.Osaka,Japan,2016:3135-3145.
[5]Shafiq Joty,Hassan Sajjad,Nadir Durrani,et al.How to avoid unwanted pregnancies:Domain adaptation using neural network models[C]//Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal,2015:1259-1270.
[6]Rui Wang,Andrew Finch,Masao Utiyama,et al.Sentence embedding for neural machine translation domain adaptation[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada,2017:560-566.
[7]Marlies van der Wees, Arianna Bisazza,Christof Monz.Dynamic data selection for neural machine translation[J].arXiv preprint arXiv:1708.00712,2017.
[8]Rui Wang,Masao Utiyama,Lemao Liu,et al.Instance weighting for neural machine translation domain adaptation[C]//Proceedings of the 2017Conference on Empirical Methods in Natural Language Processing.Copenhagen,Denmark,2017:1482-1488.
[9]Catherine Kobus,Josep Crego,Jean Senellart.Domain control for neural machine translation[J].arXiv preprint arXiv:1612.06140,2016.
[10]Minh-Thang Luong,Christopher D Manning.Stanford neural machine translation systems for spoken language domains[C]//Proceedings of the International Workshop on Spoken Language Translation.Da Nang,Vietnam,2015:76-79.
[11]Ian J Goodfellow, Jonathon Shlens, Christian Szegedy.Generative adversarial nets[C]//Proceedings of the Advances in Neural Information Processing Systems,2014:2672-2680.
[12]Lijun Wu,Yingce Xia,Li Zhao,et al.Adversarial neural machine translation[J].arXiv preprint arXiv:1704.06933,2017.
[13]Zhen Yang,Wei Chen,Feng Wang,et al.Improving neural machine translation with conditional sequence generative adversarial nets[J].arXiv preprint arXiv:1703.04887,2017.
[14]Sebastien Jean,Kyunghyun Cho,Roland Memisevic.On using very large target vocabulary for neural machine translation[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics(ACL).Beijing,China,2015:1-10.
[15]Ekaterina Garmash,Christof Monz.Ensemble learning for multi-source neural machine translation[C]//Proceedings of the 26th International Conference on Computational Linguistics. Osaka,Japan,2016:1409-1418.
[16]Mauro Cettolo,Jan Niehues,et al.Report on the11th IWSLT evaluation campaign[C]//Proceedings of the International Workshop on Spoken Language Translation.Hanoi,Vietnam,2014:2-17.
[17]Sennrich R,Haddow B,Birch A.Neural machine translation of rare words with subword units[J].arXiv preprint arXiv:1508.07909,2015.
[18]Kishore Papineni,Salim Roukos,Todd Ward.et al.BLEU:A method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting on Association for Computational Linguistics.Philadelphia,Pennsylvania,2002:311-318.
(1)https://github.com/pytorch/fairseq/tree/v0.4.0
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |