D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension
作者： Hongyu Li , Xiyuan Zhang , Yibing Liu , Yiming Zhang , Quan Wang
DOI: 10.18653/V1/D19-5828
关键词:
摘要: In this paper, we introduce a simple system Baidu submitted for MRQA (Machine Reading Question Answering) 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models. Our is built framework pretraining and fine-tuning, namely D-NET. The techniques pre-trained language models multi-task learning are explored to improve the MRC conduct experiments examine effectiveness these strategies. ranked at top 1 all participants in terms averaged F1 score. codes will be released PaddleNLP.
aclweb.org参考文章(18)
Hannaneh Hajishirzi, Ali Farhadi, Aniruddha Kembhavi, Min Joon Seo, Bidirectional Attention Flow for Machine Comprehension international conference on learning representations. ,(2016)
Matthew Dunn, Mike Higgins, Volkan Cirik, V. Ugur Guney, Kyunghyun Cho, Levent Sagun, SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine arXiv: Computation and Language. ,(2017)
Weifeng Lv, Ming Zhou, Furu Wei, Chuanqi Tan, Bowen Du, Nan Yang, S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension arXiv: Computation and Language. ,(2017)
Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, Ming Zhou, Gated Self-Matching Networks for Reading Comprehension and Question Answering meeting of the association for computational linguistics. ,vol. 1, pp. 189- 198 ,(2017) , 10.18653/V1/P17-1018
Adams Wei Yu, Kai Chen, Minh-Thang Luong, Mohammad Norouzi, David Dohan, Rui Zhao, Quoc V. Le, QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension arXiv: Computation and Language. ,(2018)
Alexandra Chronopoulou, Christos Baziotis, Alexandros Potamianos, An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models north american chapter of the association for computational linguistics. pp. 2089- 2095 ,(2019) , 10.18653/V1/N19-1213
Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, Kristina Toutanova, BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions north american chapter of the association for computational linguistics. pp. 2924- 2936 ,(2019) , 10.18653/V1/N19-1300
Chris Alberti, Daniel Andor, Emily Pitler, Jacob Devlin, Michael Collins, Synthetic QA Corpora Generation with Roundtrip Consistency meeting of the association for computational linguistics. pp. 6168- 6173 ,(2019) , 10.18653/V1/P19-1620
Ruslan Salakhutdinov, Jaime Carbonell, Quoc V. Le, Yiming Yang, Zhilin Yang, Zihang Dai, XLNet: Generalized Autoregressive Pretraining for Language Understanding arXiv: Computation and Language. ,(2019)
Alon Talmor, Jonathan Berant, MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension meeting of the association for computational linguistics. pp. 4911- 4921 ,(2019) , 10.18653/V1/P19-1485