Evaluation of NLG in an end-to-end Spoken dialogue system- is it worth it?
作者： Nina Dethlefs , Simon Keizer , Xingkun Liu , Helen Hastie , Heriberto Cuayahuitl
DOI:
关键词:
摘要: In the past 10 years, only around 15% of published conference papers include some kind extrinsic evaluation an NLG component in end-to-end system. These types evaluations are costly to set-up and run, so is it worth it? Is there anything be gained over above intrinsic quality measures obtained off-line experiments? this paper, we describe a case study evaluating two variants surface realiser show that significant differences both measures. would need factored into future iterations therefore, conclude worthwhile.
hw.ac.uk参考文章(10)
Pirros Tsiakoulis, Milica Gasic, Steve J. Young, Matthew Henderson, Catherine Breslin, Dongho Kim, Dialogue context sensitive speech synthesis using factorized decision trees. conference of the international speech communication association. pp. 2937- 2941 ,(2014)
Heriberto Cuayahuitl, Nina Dethlefs, Helen Hastie, Xingkun Liu, Training a statistical surface realiser from automatic slot labelling spoken language technology workshop. pp. 112- 117 ,(2014) , 10.1109/SLT.2014.7078559
Verena Rieser, Oliver Lemon, Simon Keizer, Natural language generation as incremental planning under uncertainty: adaptive information presentation for statistical dialogue systems IEEE Transactions on Audio, Speech, and Language Processing. ,vol. 22, pp. 979- 994 ,(2014) , 10.1109/TASL.2014.2315271
Heriberto Cuayahuitl, Nina Dethlefs, Helen Hastie, A Semi-supervised Clustering Approach for Semantic Slot Labelling international conference on machine learning and applications. pp. 500- 505 ,(2014) , 10.1109/ICMLA.2014.87
Oliver Lemon, Nina Dethlefs, Helen Hastie, Heriberto Cuayáhuitl, Conditional Random Fields for Responsive Surface Realisation using Global Features meeting of the association for computational linguistics. pp. 1254- 1263 ,(2013)
Yves Vanrompay, Oliver Lemon, Marie-Aude Aufaure, Nina Dethlefs, Pirros Tsiakoulis, Milica Gasic, Blaise Thomson, Nesrine Ben Mustapha, Panos Alexopoulos, Xingkun Liu, Verena Rieser, Peter Mika, James Henderson, Helen Hastie, Heriberto Cuayáhuitl, Demonstration of the PARLANCE system: a data-driven incremental, spoken dialogue system for interactive search annual meeting of the special interest group on discourse and dialogue. pp. 154- 156 ,(2013)
Dimitra Gkatzia, Saad Mahamood, A Snapshot of NLG Evaluation Practices 2005 - 2014 natural language generation. pp. 57- 60 ,(2015) , 10.18653/V1/W15-4708
Martin Szummer, Pirros Tsiakoulis, Milica Gasic, Steve Young, Matthew Henderson, Catherine Breslin, Dongho Kim, Blaise Thomson, POMDP-based dialogue manager adaptation to extended domains annual meeting of the special interest group on discourse and dialogue. pp. 214- 222 ,(2013)
Anja Belz, Helen Hastie, Comparative Evaluation and Shared Tasks for NLG in Interactive Systems natural language generation. pp. 302- 350 ,(2014) , 10.1017/CBO9780511844492.013
Amy Isard, Athanasios Karasimos, Multi-lingual Evaluation of a Natural Language Generation System language resources and evaluation. pp. 829- 832 ,(2004)