基于混淆网络的机器翻译系统融合研究
|
摘要
近年来,系统融合成为机器翻译领域的一个研究热点。系统融合研究如何充分利用各种不同机器翻译系统的优势来提高翻译的性能。目前系统融合方法可以分为句子级别和词级别两大类。句子级别系统融合主要是基于最小贝叶风险解码,词级别系统融合则是以混淆网络的形式融合。词级别的系统融合因为能够稳定、显著地提高翻译性能,已成为系统融合的主要方法。
基于最小贝叶斯风险解码的句子级别系统融合需要对最小贝叶斯风险定义损失函数。损失函数对句子级别系统融合有影响。词级别系统融合需要先对多个机器翻译结果进行词对齐,再依据词对齐结果构建混淆网络,最终从混淆网络中解码输出融合结果。词对齐质量和混淆网络的解码直接关系到词级别系统融合的效果。
本文对目前句子级别和词级别的系统融合方法进行了较为详细的分析与总结。针对句子级别系统融合中的损失函数问题,分别设计了三种基于不同机器翻译评测标准的损失函数。针对词级别系统融合中词对齐没有考虑语言信息、词对齐方式单一的问题,提出了融入语言信息、融合一致性词对齐结果的两种改进词对齐质量的方法,以通过改善词对齐质量来提高词级别系统融合的效果;针对单个混淆网络系统融合过于依赖参考句子、调序能力有限的问题,提出了多个混淆网络的重评分和最小贝叶斯风险的解码方法,以通过多个混淆网络搜索到更好的融合结果。本文的主要工作表现如下:
1、句子级别和词级别系统融合的研究与实现
句子级别系统融合基于最小贝叶斯风险解码。为了对最小贝叶斯风险解码的有效性进行全面测试,本文设计了基于BLEU、TER(翻译错误率)和WER(词错误率)的三种损失函数。经典的词级别系统融合方法主要是基于TER对齐、GIZA++对齐。将多个翻译结果对齐后,词级别系统融合构建出混淆网络,以对数模型融合了语言模型、词后验概率、词惩罚因子等多特征,最终以柱搜索算法从混淆网络中找到最佳融合结果。实验证明了句子级别系统融合可以提高0.5个BLEU,词级别系统融合可以提高1个BLEU。
2、改进词对齐质量的系统融合研究
词对齐是词级别系统融合中非常关键的一步。为改进词对齐质量,在词对齐过程中融入语言信息、融合一致性词对齐结果。本文主要在词对齐过程中采用了词干和同义词两种信息;通过一致性对齐方法获得一致性词对齐结果,并将一致性词对齐与GIZA++词对齐以并集、交集的方式进行融合。语言信息的加入可以缓解词对齐中的数据稀疏问题,一致性词对齐的加入可以提高词对齐的准确率。实验证明改进词对齐质量的系统融合可以比经典的词级别系统融合提高0.1-0.5个BLEU。
3、改进混淆网络解码方法的系统融合研究
改进混淆网络解码方法是针对单个混淆网络融合过分依赖参考句子、调序能力有限的改进,以多个混淆网络的形式融合,并提出以重评分和最小贝叶斯风险解码两种方法对多个混淆网络进行一致性解码。实验证明改进混淆网络解码方法的系统融合可以比没有改进的融合方法提高0.5个左右的BLEU。
With the development of machine translation, many different kinds of machine translation systems are invented. System combination is a kind of technology to make use of different kinds of machine translation systems to improve translation quality. Recently, system combination has become one of the research hotspot in machine translation. The present system combination methods can be categorized into two major types: sentence level system combination and word level system combination. Sentence level system combination is based on minimum bayes-risk decoding and word level system combination is achieved in form of confusion network. Owing to the advantage of improving translation quality stably and remarkably, word level system combination has become the popular method in system combination. Word level system combination first needs to align many different machine translation system’s outputs and then builds a confusion network based on outputs’word alignment. Then the best combination result is extracted from the confusion network. Word alignment quality is vital to word level system combination’s effect.
The current sentence and word level system combination methods are analyzed and summarized comprehensively in this thesis. Considering the word alignment of the current word level system combination dose not take language information into account, a new kind of word level system combination method which aims to improve the word alignment quality is proposed. And in order to solve the problem of over-reliance on reference sentences and weak reordering ability of system combination based on a single confusion network, multiple confusion networks decoding is proposed. Rescore and minimum bayes-risk decoding methods are used to decode multiple confusion networks. The main contributions of this thesis are listed below:
1. Research and realization of sentence level and word level system combination
Sentence level system combination is based on minimum bayes-risk decoding and loss function is needed to be defined in this method. Three kinds of loss functions based on BLEU, TER (Translation Error Rate) and WER (Word Error Rate) respectively are defined to test the effect of sentence level system combination. Classical word level system combination uses TER or incremental TER or GIZA++ to align different machine translations. The confusion network is built using the word alignment of different machine translations. Language model, word posterior probability and word penalty are integrated in log-linear model into confusion network decoding. The final combination result is extracted by the beam search algorithm in the confusion network. Experiment results prove that 0.5 BLEU improvement of translation quality can be achieved by sentence level system combination methods and 1.0 BLEU improvement of translation quality can be achieved by word level system combination.
2. Research of methods to improve word alignment quality
As word alignment quality is critical to word level system combination, two methods are used in word aligning to improve word alignment quality. One method adds language information into word alignment and the other method integrates other source of word alignment information. Stem and synonym are the two language information used. The other source of alignment information is got through alignment by agreement and the alignment results of alignment by agreement and GIZA++ are integrated by intersection or union. Language information can relieve the data sparse problem of word alignment. And integrating word alignment by agreement result can improve word alignment precision. Experiment results confirm that better word alignment quality leads to better system combination result.
3. Research and realization of improved word level system combination
Improved word level system combination is realized by improved word alignment quality and improved confusion network decoding. Word level system combination is built on improved word alignment quality. Improved confusion network decoding, using multiple confusion networks, is used to solve the shortcomings of single confusion network decoding of over-reliance on reference sentences and limited word reordering ability. Rescoring and minimum bayes-risk decoding methods are used to extract best result in improved confusion network decoding. Experiments show that improved word level system combination methods can improve translation quality of about 0.5 BLEU than the methods are not improved.
基于最小贝叶斯风险解码的句子级别系统融合需要对最小贝叶斯风险定义损失函数。损失函数对句子级别系统融合有影响。词级别系统融合需要先对多个机器翻译结果进行词对齐,再依据词对齐结果构建混淆网络,最终从混淆网络中解码输出融合结果。词对齐质量和混淆网络的解码直接关系到词级别系统融合的效果。
本文对目前句子级别和词级别的系统融合方法进行了较为详细的分析与总结。针对句子级别系统融合中的损失函数问题,分别设计了三种基于不同机器翻译评测标准的损失函数。针对词级别系统融合中词对齐没有考虑语言信息、词对齐方式单一的问题,提出了融入语言信息、融合一致性词对齐结果的两种改进词对齐质量的方法,以通过改善词对齐质量来提高词级别系统融合的效果;针对单个混淆网络系统融合过于依赖参考句子、调序能力有限的问题,提出了多个混淆网络的重评分和最小贝叶斯风险的解码方法,以通过多个混淆网络搜索到更好的融合结果。本文的主要工作表现如下:
1、句子级别和词级别系统融合的研究与实现
句子级别系统融合基于最小贝叶斯风险解码。为了对最小贝叶斯风险解码的有效性进行全面测试,本文设计了基于BLEU、TER(翻译错误率)和WER(词错误率)的三种损失函数。经典的词级别系统融合方法主要是基于TER对齐、GIZA++对齐。将多个翻译结果对齐后,词级别系统融合构建出混淆网络,以对数模型融合了语言模型、词后验概率、词惩罚因子等多特征,最终以柱搜索算法从混淆网络中找到最佳融合结果。实验证明了句子级别系统融合可以提高0.5个BLEU,词级别系统融合可以提高1个BLEU。
2、改进词对齐质量的系统融合研究
词对齐是词级别系统融合中非常关键的一步。为改进词对齐质量,在词对齐过程中融入语言信息、融合一致性词对齐结果。本文主要在词对齐过程中采用了词干和同义词两种信息;通过一致性对齐方法获得一致性词对齐结果,并将一致性词对齐与GIZA++词对齐以并集、交集的方式进行融合。语言信息的加入可以缓解词对齐中的数据稀疏问题,一致性词对齐的加入可以提高词对齐的准确率。实验证明改进词对齐质量的系统融合可以比经典的词级别系统融合提高0.1-0.5个BLEU。
3、改进混淆网络解码方法的系统融合研究
改进混淆网络解码方法是针对单个混淆网络融合过分依赖参考句子、调序能力有限的改进,以多个混淆网络的形式融合,并提出以重评分和最小贝叶斯风险解码两种方法对多个混淆网络进行一致性解码。实验证明改进混淆网络解码方法的系统融合可以比没有改进的融合方法提高0.5个左右的BLEU。
With the development of machine translation, many different kinds of machine translation systems are invented. System combination is a kind of technology to make use of different kinds of machine translation systems to improve translation quality. Recently, system combination has become one of the research hotspot in machine translation. The present system combination methods can be categorized into two major types: sentence level system combination and word level system combination. Sentence level system combination is based on minimum bayes-risk decoding and word level system combination is achieved in form of confusion network. Owing to the advantage of improving translation quality stably and remarkably, word level system combination has become the popular method in system combination. Word level system combination first needs to align many different machine translation system’s outputs and then builds a confusion network based on outputs’word alignment. Then the best combination result is extracted from the confusion network. Word alignment quality is vital to word level system combination’s effect.
The current sentence and word level system combination methods are analyzed and summarized comprehensively in this thesis. Considering the word alignment of the current word level system combination dose not take language information into account, a new kind of word level system combination method which aims to improve the word alignment quality is proposed. And in order to solve the problem of over-reliance on reference sentences and weak reordering ability of system combination based on a single confusion network, multiple confusion networks decoding is proposed. Rescore and minimum bayes-risk decoding methods are used to decode multiple confusion networks. The main contributions of this thesis are listed below:
1. Research and realization of sentence level and word level system combination
Sentence level system combination is based on minimum bayes-risk decoding and loss function is needed to be defined in this method. Three kinds of loss functions based on BLEU, TER (Translation Error Rate) and WER (Word Error Rate) respectively are defined to test the effect of sentence level system combination. Classical word level system combination uses TER or incremental TER or GIZA++ to align different machine translations. The confusion network is built using the word alignment of different machine translations. Language model, word posterior probability and word penalty are integrated in log-linear model into confusion network decoding. The final combination result is extracted by the beam search algorithm in the confusion network. Experiment results prove that 0.5 BLEU improvement of translation quality can be achieved by sentence level system combination methods and 1.0 BLEU improvement of translation quality can be achieved by word level system combination.
2. Research of methods to improve word alignment quality
As word alignment quality is critical to word level system combination, two methods are used in word aligning to improve word alignment quality. One method adds language information into word alignment and the other method integrates other source of word alignment information. Stem and synonym are the two language information used. The other source of alignment information is got through alignment by agreement and the alignment results of alignment by agreement and GIZA++ are integrated by intersection or union. Language information can relieve the data sparse problem of word alignment. And integrating word alignment by agreement result can improve word alignment precision. Experiment results confirm that better word alignment quality leads to better system combination result.
3. Research and realization of improved word level system combination
Improved word level system combination is realized by improved word alignment quality and improved confusion network decoding. Word level system combination is built on improved word alignment quality. Improved confusion network decoding, using multiple confusion networks, is used to solve the shortcomings of single confusion network decoding of over-reliance on reference sentences and limited word reordering ability. Rescoring and minimum bayes-risk decoding methods are used to extract best result in improved confusion network decoding. Experiments show that improved word level system combination methods can improve translation quality of about 0.5 BLEU than the methods are not improved.
引文
[1]冯志伟.机器翻译的现状和问题,科学出版社, 2003年
[2] Peter F. Brown, Stephan A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. The mathematics of statistical machine translation: Parameter Estimation. Computational Linguistics, 1993(2):263-311.
[3] Franz J. Och, Christoph Tillman, and Hermann Ney. Improved alignment models for statistical machine translation. In Proceedings of EMNLP 1999, pages 20-28, College Park, Maryland, June.
[4] Franz J. Och and Hermann Ney. Discriminative training and maximum entropy models for statistical machine translation. In Proceedings of ACL 2002, pages 295-302, Philadelphia, PA, July.
[5] Philipp Koehn, Franz J. Och, and Daniel Marcu. Statistical phrase-based translation. In Proceedings of HLT/NAACL 2003, pages 127-133.
[6] Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen,W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., & Herbst, E. Moses: Open source toolkit for statistical machine translation. In AnnualMeeting of ACL2007, demonstration session, pages 177–180, Czech Republic, June.
[7] Philipp Koehn and Hieu Hoang. Factored Translation Models. In Proceedings of the EMNLP 2007, pp. 868–876, Prague, Czech Republic, June.
[8] David Chiang. A hierarchical phrase-based model for statistical machine translation. In Proceedings of ACL 2005, pages 263-270, Ann Arbor, Michigan, June.
[9] Dekai Wu. Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. Computational Linguistics, 1997, 23:377-404.
[10] Chris Quirk, Arul Menezes, and Colin Cherry. Dependency treelet translation: Syntactically informed phrasal SMT. In Proceedings of ACL 2005, pages 271-279, Ann Arbor, Michigan, June.
[11] Yang Liu, Qun Liu, and Shouxun Lin. Tree-to-string alignment template for statistical machine translation. In Proceedings of COLING/ACL 2006, pages 609-616, Sydney, Australia, July.
[12] Liang Huang, Kevin Knight, and Aravind Joshi. Statistical syntax-directed translation with extended domain of locality. In Proceedings of AMTA2006, Boston, MA, August.
[13] Kenji Yamada and Kevin Knight. A syntax-based statistical translation model. In Proceedings of ACL 2001, pages 523-530.
[14] Michel Galley, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve DeNeefe, Wei Wang and Ignacio Thayer. Scalable inference and training of context-rich syntactic translation models. In Proceedings of COLING/ACL 2006, pages 961-968, Sydney, Australia, July.
[15] Daniel Marcu, Wei Wang, Abdessamad Echihabi, and Kevin Knight. SPMT: Statistical machine translation with syntactified target language phrases. In Proceedings of EMNLP 2006, pages 44-52.
[16] Yuan Ding and Martha Palmer. Machine translation using probabilistic synchronous dependency insertion grammars. In Proceedings of ACL 2005, pages 541-548, Ann Arbor, Michigan, June.
[17] Min Zhang, Hongfei Jiang, Aiti Aw, Haizhou Li, Chew Lim Tan, Sheng Li. A Tree Sequence Alignment-based Tree-to-Tree Translation Model. In Proceedings of HLT:ACL 2008.
[18] Deyi Xiong, Min Zhang, Ai Ti Aw, Haitao Mi, Qun Liu and Shouxun Lin. Refinements in BTG-based Statistical Machine Translation. In Proceedings of IJCNLP 2008.
[19] Haitao Mi, Liang Huang and Qun Liu. Forest-Based Translation. In Proceedings of ACL2008, Ohio, U.S.A., June 15-20, 2008
[20] Haitao Mi and Liang Huang. Forest-based Translation Rule Extraction. In Proceedings of EMNLP 2008, Hawaii.
[21]米海涛.容错统计机器翻译关键技术研究,博士论文, 2009
[22] Chris Callison-Burch, Philipp Koehn, et al. Findings of the 2009 Workshop on Statistical Machine Translation. In Proceedings of the 4th EACL Workshop on Statistical Machine Translation, 2009, Greece:1-28.
[23] http://www.itl.nist.gov/iad/mig//tests/mt/
[24]孙乐.第五届全国机器翻译研讨会论文集(CWMT2009).第五届全国机器翻译研讨会, 2009,南京.
[25] J . G. Fiscus. A Post-Processing System to Yield Reduced Word Error Rates: Recognizer Output Voting Error Reduction (ROVER). IEEE Workshop on Automatic Speech Recognition and Understanding ,1997.
[26] Srinivas Bangalore, German Bordel, Giuseppe Riccardi. Computing Consensus Translation from Multiple Machine Translation Systems. In Proceedings of ASRU, Madonna di Campiglio, Italy, Dec.2001, CD-ROM.
[27] Shankar Kumar and William Byrne. Minimum Bayes-Risk Decoding for Statistical Machine Translation. In Proceedings of HLT-NAACL 2004, Bostaon, MA.
[28] Sanjika Hewavitharana, Bing Zhao, Almut Silja Hildebrand,et al. The CMU Statistical Machine Translation System for IWSLT 2005. In Proceedings of IWSLT 2005,
[29] Evgeny Matusov, Nicola Ueffing, Hermann Ney. Computing Consensus Translation from Multiple Machine Translation Systems Using Enhanced Hypotheses Alignment. ACL,2006.
[30] F. J. Och and H. Ney. A systematic comparison of various statistical alignment models. Computing Linguistics, vol. 29, no.19, pages 51–60.
[31] K.C. Sim, W.J. Byrne, M.J.F. Gales, et al. Consensus Network Decoding for Statistical Machine Translation System Combination. ICASSP,2007.
[32] Antti Veikko I. Rosti , Necip Fazil Ayan , Bing Xiang, et al. Combing Outputs from Multiple Machine Translation Systems. In Proceedings of NAACL HLT 2007, pages 228-235.
[33] Antti-Veikko I. Rosti , Bing Zhang , Spyros Matsoukas, et al. Incremental Hypothesis Alignment for Building Confusion Networks with Application to Machine Translation System Combination. In Proceedings of the Third Workshop on Statistical Machine Translation, 2008, pages 183–186.
[34] Xiaodong He, Mei Yang, Jianfeng Gao, et al. Indirect-HMM-based Hypothesis Alignment for Combining Outputs from Machine Translation Systems. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 98–107.
[35] Chi-Ho Li, Xiaodong He, Yupeng Liu, Ning Xi. Incremental HMM Alignment for MT System Combination. In Proceedings of ACL and AFNLP 2009, Suntec, Singapore.
[36] Richard Zens,Franz Josef Och,and Hermann Ney.Phrase-Based Statistical Machine Translation[A]. In:25th Annual German Conferenceon Artificial Intelligence(KI2002), volume 2479 of Lecture Notes in Artificial Intelligence (LNAI), 2002, springer Verlag. Pages 18-22.
[37] Franz J. Och. 2002. Statistical machine translation: From single-word models to alignment templates. Ph.D. thesis, Computer Science Department, RWTH Aachen, Germany, October.
[38] P. Koehn, Pharaoh: a beam search decoder for phrase-based statistical machine translation models, Proc. of the 6th Conf. of the Association for Machine Translation in the Americas, pags. 115~124, October 2004.
[39] Liang Huang, David Chiang. Forest Rescoring: Faster Decoding with Integrated Language Models[A]. In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL’07). 2007,144-151.
[40]熊德意,刘群,林守勋.基于句法的统计机器翻译综述.中文信息学报, 2008年, 22(2): 28-39.
[41] Kishore Papieni , Salim Roukos ,Todd Ward , et al. BLEU : a method for automatic evaluation of machine translation . ACL 2002 :311-318 .
[42] Abhaya Agarwal,Alon Lavie. METEOR,M-BLEU and M-TER: Evaluaiton Metrics forHigh-Correlation with Human Rankings of Machine Translation Output.ACL 2008:115-118.
[43] Maththew Snover,Bonnie Dorr,Richard Schwartz,et al. A Study of Translation Edit Rate with Targeted Human Annotation. AMTA 2006,USA,223-231.
[44] Ming Zhou,Bo Wang,Shujie Liu, et al. Diagnostic Evaluation of Machine Translation Systems Using Automatically Constructed Linguistic Check-Points.In Proceedings of the 22nd International Conference on Computational Linguistics ,Manchester,2008:1121-1128.
[45] Chris Callison-Burch, Philipp Koehn, et al. Findings of the 2009 Workshop on Statistical Machine Translation. In Proceedings of the 4th EACL Workshop on Statistical Machine Translation, 2009, Greece:1-28.
[46] http://mastarpj.nict.go.jp/IWSLT2009/
[47] http://www.statmt.org/
[48] http://www.863data.org.cn/index.php
[49]赵红梅,谢军,吕雅娟等.第四届全国机器翻译研讨会(CWMT2008)评测报告.第四届全国机器翻译研讨会, 2008,北京.
[50]杨攀,张建,李淼等.汉蒙统计机器翻译的形态学方法研究.中文信息学报,2009,23(1):50-57.
[51]杨攀,李淼,张建等.第一届汉民机器翻译评测结果报告.第二届全国少数民族青年自然语言处理学术研讨会, 2008,合肥.
[52] Franz Josef Och. Minimum Error Rate Training in Statistical Machine Translation. ACL, 2003.
[53] Nicola Bertoldi, Barry Haddow, Jean-Baptiste Fouet. Improved Minimum Error Rate Traning in Moses. In Proceedings of PBML 2009.
[54] Philipp Koehn, Hieu Hoang, Alexandra Birch, et al. Moses: Open source toolkit for statistical machine translation. ACL, 2007.
[55] Zhifei Li, Chris Callison-Burch, Chris Dyery, et al. Joshua: An Open Source Toolkit for Parsing-based Machine Translation. WMT, 2009.
[56] Ashish Venugopal , Andreas Zollmann. Syntax Augmented Machine Translation via Chart Parsing with Integrated Language Modeling. NAACL, 2006.
[57] Percy Liang, Ben Taskar, Dan Klein. Alignment by Agreement. NAACL, 2006.
[58] M.F. Porter. An algorithm for suffix stripping. Program, 1980, 14(3):130-137.
[59] http://wordnet.princeton.edu/wordnet/
[60]罗毅.基于短语统计机器翻译的柱搜索解码器的优化实现,硕士论文, 2007
[61]强静.基于短语的统计翻译模型的研究与应用,硕士论文, 2008
[62]骆凯.基于形态度句法信息的短语翻译模型研究,硕士论文, 2009
[2] Peter F. Brown, Stephan A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. The mathematics of statistical machine translation: Parameter Estimation. Computational Linguistics, 1993(2):263-311.
[3] Franz J. Och, Christoph Tillman, and Hermann Ney. Improved alignment models for statistical machine translation. In Proceedings of EMNLP 1999, pages 20-28, College Park, Maryland, June.
[4] Franz J. Och and Hermann Ney. Discriminative training and maximum entropy models for statistical machine translation. In Proceedings of ACL 2002, pages 295-302, Philadelphia, PA, July.
[5] Philipp Koehn, Franz J. Och, and Daniel Marcu. Statistical phrase-based translation. In Proceedings of HLT/NAACL 2003, pages 127-133.
[6] Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen,W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., & Herbst, E. Moses: Open source toolkit for statistical machine translation. In AnnualMeeting of ACL2007, demonstration session, pages 177–180, Czech Republic, June.
[7] Philipp Koehn and Hieu Hoang. Factored Translation Models. In Proceedings of the EMNLP 2007, pp. 868–876, Prague, Czech Republic, June.
[8] David Chiang. A hierarchical phrase-based model for statistical machine translation. In Proceedings of ACL 2005, pages 263-270, Ann Arbor, Michigan, June.
[9] Dekai Wu. Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. Computational Linguistics, 1997, 23:377-404.
[10] Chris Quirk, Arul Menezes, and Colin Cherry. Dependency treelet translation: Syntactically informed phrasal SMT. In Proceedings of ACL 2005, pages 271-279, Ann Arbor, Michigan, June.
[11] Yang Liu, Qun Liu, and Shouxun Lin. Tree-to-string alignment template for statistical machine translation. In Proceedings of COLING/ACL 2006, pages 609-616, Sydney, Australia, July.
[12] Liang Huang, Kevin Knight, and Aravind Joshi. Statistical syntax-directed translation with extended domain of locality. In Proceedings of AMTA2006, Boston, MA, August.
[13] Kenji Yamada and Kevin Knight. A syntax-based statistical translation model. In Proceedings of ACL 2001, pages 523-530.
[14] Michel Galley, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve DeNeefe, Wei Wang and Ignacio Thayer. Scalable inference and training of context-rich syntactic translation models. In Proceedings of COLING/ACL 2006, pages 961-968, Sydney, Australia, July.
[15] Daniel Marcu, Wei Wang, Abdessamad Echihabi, and Kevin Knight. SPMT: Statistical machine translation with syntactified target language phrases. In Proceedings of EMNLP 2006, pages 44-52.
[16] Yuan Ding and Martha Palmer. Machine translation using probabilistic synchronous dependency insertion grammars. In Proceedings of ACL 2005, pages 541-548, Ann Arbor, Michigan, June.
[17] Min Zhang, Hongfei Jiang, Aiti Aw, Haizhou Li, Chew Lim Tan, Sheng Li. A Tree Sequence Alignment-based Tree-to-Tree Translation Model. In Proceedings of HLT:ACL 2008.
[18] Deyi Xiong, Min Zhang, Ai Ti Aw, Haitao Mi, Qun Liu and Shouxun Lin. Refinements in BTG-based Statistical Machine Translation. In Proceedings of IJCNLP 2008.
[19] Haitao Mi, Liang Huang and Qun Liu. Forest-Based Translation. In Proceedings of ACL2008, Ohio, U.S.A., June 15-20, 2008
[20] Haitao Mi and Liang Huang. Forest-based Translation Rule Extraction. In Proceedings of EMNLP 2008, Hawaii.
[21]米海涛.容错统计机器翻译关键技术研究,博士论文, 2009
[22] Chris Callison-Burch, Philipp Koehn, et al. Findings of the 2009 Workshop on Statistical Machine Translation. In Proceedings of the 4th EACL Workshop on Statistical Machine Translation, 2009, Greece:1-28.
[23] http://www.itl.nist.gov/iad/mig//tests/mt/
[24]孙乐.第五届全国机器翻译研讨会论文集(CWMT2009).第五届全国机器翻译研讨会, 2009,南京.
[25] J . G. Fiscus. A Post-Processing System to Yield Reduced Word Error Rates: Recognizer Output Voting Error Reduction (ROVER). IEEE Workshop on Automatic Speech Recognition and Understanding ,1997.
[26] Srinivas Bangalore, German Bordel, Giuseppe Riccardi. Computing Consensus Translation from Multiple Machine Translation Systems. In Proceedings of ASRU, Madonna di Campiglio, Italy, Dec.2001, CD-ROM.
[27] Shankar Kumar and William Byrne. Minimum Bayes-Risk Decoding for Statistical Machine Translation. In Proceedings of HLT-NAACL 2004, Bostaon, MA.
[28] Sanjika Hewavitharana, Bing Zhao, Almut Silja Hildebrand,et al. The CMU Statistical Machine Translation System for IWSLT 2005. In Proceedings of IWSLT 2005,
[29] Evgeny Matusov, Nicola Ueffing, Hermann Ney. Computing Consensus Translation from Multiple Machine Translation Systems Using Enhanced Hypotheses Alignment. ACL,2006.
[30] F. J. Och and H. Ney. A systematic comparison of various statistical alignment models. Computing Linguistics, vol. 29, no.19, pages 51–60.
[31] K.C. Sim, W.J. Byrne, M.J.F. Gales, et al. Consensus Network Decoding for Statistical Machine Translation System Combination. ICASSP,2007.
[32] Antti Veikko I. Rosti , Necip Fazil Ayan , Bing Xiang, et al. Combing Outputs from Multiple Machine Translation Systems. In Proceedings of NAACL HLT 2007, pages 228-235.
[33] Antti-Veikko I. Rosti , Bing Zhang , Spyros Matsoukas, et al. Incremental Hypothesis Alignment for Building Confusion Networks with Application to Machine Translation System Combination. In Proceedings of the Third Workshop on Statistical Machine Translation, 2008, pages 183–186.
[34] Xiaodong He, Mei Yang, Jianfeng Gao, et al. Indirect-HMM-based Hypothesis Alignment for Combining Outputs from Machine Translation Systems. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 98–107.
[35] Chi-Ho Li, Xiaodong He, Yupeng Liu, Ning Xi. Incremental HMM Alignment for MT System Combination. In Proceedings of ACL and AFNLP 2009, Suntec, Singapore.
[36] Richard Zens,Franz Josef Och,and Hermann Ney.Phrase-Based Statistical Machine Translation[A]. In:25th Annual German Conferenceon Artificial Intelligence(KI2002), volume 2479 of Lecture Notes in Artificial Intelligence (LNAI), 2002, springer Verlag. Pages 18-22.
[37] Franz J. Och. 2002. Statistical machine translation: From single-word models to alignment templates. Ph.D. thesis, Computer Science Department, RWTH Aachen, Germany, October.
[38] P. Koehn, Pharaoh: a beam search decoder for phrase-based statistical machine translation models, Proc. of the 6th Conf. of the Association for Machine Translation in the Americas, pags. 115~124, October 2004.
[39] Liang Huang, David Chiang. Forest Rescoring: Faster Decoding with Integrated Language Models[A]. In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL’07). 2007,144-151.
[40]熊德意,刘群,林守勋.基于句法的统计机器翻译综述.中文信息学报, 2008年, 22(2): 28-39.
[41] Kishore Papieni , Salim Roukos ,Todd Ward , et al. BLEU : a method for automatic evaluation of machine translation . ACL 2002 :311-318 .
[42] Abhaya Agarwal,Alon Lavie. METEOR,M-BLEU and M-TER: Evaluaiton Metrics forHigh-Correlation with Human Rankings of Machine Translation Output.ACL 2008:115-118.
[43] Maththew Snover,Bonnie Dorr,Richard Schwartz,et al. A Study of Translation Edit Rate with Targeted Human Annotation. AMTA 2006,USA,223-231.
[44] Ming Zhou,Bo Wang,Shujie Liu, et al. Diagnostic Evaluation of Machine Translation Systems Using Automatically Constructed Linguistic Check-Points.In Proceedings of the 22nd International Conference on Computational Linguistics ,Manchester,2008:1121-1128.
[45] Chris Callison-Burch, Philipp Koehn, et al. Findings of the 2009 Workshop on Statistical Machine Translation. In Proceedings of the 4th EACL Workshop on Statistical Machine Translation, 2009, Greece:1-28.
[46] http://mastarpj.nict.go.jp/IWSLT2009/
[47] http://www.statmt.org/
[48] http://www.863data.org.cn/index.php
[49]赵红梅,谢军,吕雅娟等.第四届全国机器翻译研讨会(CWMT2008)评测报告.第四届全国机器翻译研讨会, 2008,北京.
[50]杨攀,张建,李淼等.汉蒙统计机器翻译的形态学方法研究.中文信息学报,2009,23(1):50-57.
[51]杨攀,李淼,张建等.第一届汉民机器翻译评测结果报告.第二届全国少数民族青年自然语言处理学术研讨会, 2008,合肥.
[52] Franz Josef Och. Minimum Error Rate Training in Statistical Machine Translation. ACL, 2003.
[53] Nicola Bertoldi, Barry Haddow, Jean-Baptiste Fouet. Improved Minimum Error Rate Traning in Moses. In Proceedings of PBML 2009.
[54] Philipp Koehn, Hieu Hoang, Alexandra Birch, et al. Moses: Open source toolkit for statistical machine translation. ACL, 2007.
[55] Zhifei Li, Chris Callison-Burch, Chris Dyery, et al. Joshua: An Open Source Toolkit for Parsing-based Machine Translation. WMT, 2009.
[56] Ashish Venugopal , Andreas Zollmann. Syntax Augmented Machine Translation via Chart Parsing with Integrated Language Modeling. NAACL, 2006.
[57] Percy Liang, Ben Taskar, Dan Klein. Alignment by Agreement. NAACL, 2006.
[58] M.F. Porter. An algorithm for suffix stripping. Program, 1980, 14(3):130-137.
[59] http://wordnet.princeton.edu/wordnet/
[60]罗毅.基于短语统计机器翻译的柱搜索解码器的优化实现,硕士论文, 2007
[61]强静.基于短语的统计翻译模型的研究与应用,硕士论文, 2008
[62]骆凯.基于形态度句法信息的短语翻译模型研究,硕士论文, 2009