Pronoun-Targeted Fine-tuning for NMT with Hybrid Losses

作者: Prathyusha Jwalapuram , Shafiq Joty , Youlin Shen

DOI: 10.18653/V1/2020.EMNLP-MAIN.177

关键词: Machine translationBLEUArtificial intelligenceBenchmark (computing)Fine-tuningNatural language processingClass (biology)PronounComputer science

摘要: Popular Neural Machine Translation model training uses strategies like backtranslation to improve BLEU scores, requiring large amounts of additional data and training. We introduce a class conditional generative-discriminative hybrid losses that we use fine-tune trained machine translation model. Through combination targeted fine-tuning objectives intuitive re-use the has failed adequately learn from, performance both sentence-level contextual without using any data. target improvement pronoun translations through our evaluate models on benchmark testset. Our shows 0.5 WMT14 IWSLT13 De-En testsets, while achieves best results, improving from 31.81 32 testset, 32.10 33.13 with corresponding improvements in translation. further show generalizability method by reproducing two language pairs, Fr-En Cs-En.

参考文章(38)
Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, Yang Liu, Improving the Transformer Translation Model with Document-Level Context empirical methods in natural language processing. pp. 533- 542 ,(2018) , 10.18653/V1/D18-1049
Rico Sennrich, Barry Haddow, Alexandra Birch, Neural Machine Translation of Rare Words with Subword Units Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). ,vol. 1, pp. 1715- 1725 ,(2016) , 10.18653/V1/P16-1162
Jörg Tiedemann, Yves Scherrer, Neural Machine Translation with Extended Context empirical methods in natural language processing. pp. 82- 92 ,(2017) , 10.18653/V1/W17-4811
Rico Sennrich, Barry Haddow, Alexandra Birch, Improving Neural Machine Translation Models with Monolingual Data meeting of the association for computational linguistics. ,vol. 1, pp. 86- 96 ,(2016) , 10.18653/V1/P16-1009
Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin, None, Attention is All You Need neural information processing systems. ,vol. 30, pp. 5998- 6008 ,(2017)
Philipp Koehn, Rebecca Knowles, Six Challenges for Neural Machine Translation. meeting of the association for computational linguistics. pp. 28- 39 ,(2017) , 10.18653/V1/W17-3204
Andy Way, Peyman Passban, Dimitar Sht. Shterionov, Alberto Poncelas, Gideon Maillette de Buy Wenniger, Investigating Backtranslation in Neural Machine Translation Poncelas, Alberto ORCID: 0000-0002-5089-1687 <https://orcid.org/0000-0002-5089-1687>, Shterionov, Dimitar ORCID: 0000-0001-6300-797X <https://orcid.org/0000-0001-6300-797X>, Way, Andy ORCID: 0000-0001-5736-5930 <https://orcid.org/0000-0001-5736-5930>, Maillette de Buy Wenniger, Gideon and Passban, Peyman (2018) Investigating backtranslation in neural machine translation. In: 21st Annual Conference of The European Association for Machine Translation, 28-30 May 2018, Alicante, Spain.. pp. 249- 258 ,(2018)
Yoshua Bengio, Kyunghyun Cho, Dzmitry Bahdanau, Neural Machine Translation by Jointly Learning to Align and Translate international conference on learning representations. ,(2015)
Isaac Caswell, Ciprian Chelba, David Grangier, Tagged Back-Translation Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers). pp. 53- 63 ,(2019) , 10.18653/V1/W19-5206
Prathyusha Jwalapuram, Shafiq Joty, Irina Temnikova, Preslav Nakov, Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suite empirical methods in natural language processing. pp. 2964- 2973 ,(2019) , 10.18653/V1/D19-1294