Interaction Quality

摘要: We present Interaction Quality (IQ) paradigm to model quality in ongoing spoken HMI.Model can be used determine objective score at arbitrary dialog points.Strong correlations of IQ with real User Satisfaction (US).IQ predicts better than US and is easier obtain.Only a minority users show visible anger when they are dissatisfied. This study presents novel expert-based approach assess the Spoken Dialog System (SDS) interactions. call this "Interaction Quality" (IQ). It an measure which relies on statistical classification Support Vector Machines (SVMs). compare expert annotations SDS interactions subjective (US) ratings that correlate ( ? = . 66 ). Expert obviously mirror user impression great extent while are, above all, much obtain. The quantifies interaction generated using median exchange several experts. tracked 38 interacting SDS. A large, comprehensive set domain-independent, automatic parameters introduced quantify exchanges. Furthermore, manually annotated negative emotion feature added parameter order evaluate contribution emotions US. For evaluation we use CMU Let's Go bus information system. yields correlation 80 classifying scores field data from achieves 74 for predicting lab data, 89 data. presented outperforms related studies field. Only marginal performance observed, implying not influenced by emotions. analyze causalities between target variables US/IQ identify relevant predictors. With paradigm, critical dialogs found; once deployed as online monitoring technique, could render SDSs more friendly improve acceptance.