EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systems
作者： Mohammad H. Mahoor , Rohola Zandie
DOI:
关键词: Dialog system 、 Dialog box 、 Perplexity 、 Human–computer interaction 、 Architecture 、 Transformer (machine learning model) 、 Computer science
摘要: Understanding emotions and responding accordingly is one of the biggest challenges dialog systems. This paper presents EmpTransfo, a multi-head Transformer architecture for creating an empathetic system. EmpTransfo utilizes state-of-the-art pre-trained models (e.g., OpenAI-GPT) language generation, though with different sizes can be used. We show that utilizing history other metadata improve quality generated conversations by Our experimental results using challenging corpus proposed approach outperforms in terms Hit@1 PPL (Perplexity).
arxiv.org
harvard.edu参考文章(18)
Thomas Wolf, Julien Chaumond, Clement Delangue, Victor Sanh, TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents arXiv: Computation and Language. ,(2019)
Pierre Colombo, Wojciech Witon, Ashutosh Modi, James Kennedy, Mubbasir Kapadia, Affect-driven dialog generation north american chapter of the association for computational linguistics. pp. 3734- 3743 ,(2019) , 10.18653/V1/N19-1374
Hannah Rashkin, Eric Michael Smith, Margaret Li, Y-Lan Boureau, Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pp. 5370- 5381 ,(2019) , 10.18653/V1/P19-1534
Zhenqiao Song, Xiaoqing Zheng, Lu Liu, Mu Xu, Xuanjing Huang, Generating Responses with a Specific Emotion in Dialog meeting of the association for computational linguistics. pp. 3685- 3695 ,(2019) , 10.18653/V1/P19-1359
Xianda Zhou, William Yang Wang, MojiTalk: Generating Emotional Responses at Scale meeting of the association for computational linguistics. ,vol. 1, pp. 1128- 1137 ,(2018) , 10.18653/V1/P18-1104
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding north american chapter of the association for computational linguistics. pp. 4171- 4186 ,(2018) , 10.18653/V1/N19-1423
Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin, None, Attention is All You Need neural information processing systems. ,vol. 30, pp. 5998- 6008 ,(2017)
Jekaterina Novikova, Ondřej Dušek, Amanda Cercas Curry, Verena Rieser, Why We Need New Evaluation Metrics for NLG empirical methods in natural language processing. pp. 2241- 2252 ,(2017) , 10.18653/V1/D17-1238
Vera Demberg, Xiaoyu Shen, Shuzi Niu, Hui Su, Improving Variational Encoder-Decoders in Dialogue Generation national conference on artificial intelligence. pp. 5456- 5463 ,(2018)
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi, The Curious Case of Neural Text Degeneration international conference on learning representations. ,(2020)