Co-expressivity of Speech and Gesture: Lessons for Models of Aligned Speech and Gesture Production

作者: Stefan Kopp , Kirsten Bergmann

DOI:

关键词:

摘要: When people combine language and gesture to convey their intended information, both modalities are characterized by an intriguing degree of coherence consistency. For developing account how speech aligned each other, one question major importance is meaning distributed across the two channels. In this paper, we start from recent empirical findings indicating a flexible interaction between systems show that psycholinguistic models production in literature cannot for interplay equally well. Based on discussion these theories as well current computational approaches, point out conclusions model must be designed order simulate aligned, human-like multimodal behavior virtual expressive agents.

参考文章(19)
Stefan Kopp, Kirsten Bergmann, Verbal or Visual? How Information is Distributed across Speech and Gesture in Spatial Dialog brandial 06. Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue (SemDial-10). ,(2006)
Robert M. Krauss, Yihsiu Chen, Rebecca F. Gottesman, Language and Gesture: Lexical gestures and lexical access: a process model Cambridge University Press. pp. 261- 283 ,(2000) , 10.1017/CBO9780511620850.017
Jan Peter de Ruiter, The production of gesture and speech Language and Gesture: Window into Thought and Action. pp. 284- 311 ,(2000) , 10.1017/CBO9780511620850.018
Björn Hartmann, Maurizio Mancini, Catherine Pelachaud, Implementing Expressive Gesture Synthesis for Embodied Conversational Agents Lecture Notes in Computer Science. pp. 188- 199 ,(2006) , 10.1007/11678816_22
M. Denis, The description of routes : A cognitive approach to the production of spatial discourse Cahiers de psychologie cognitive. ,vol. 16, pp. 409- 458 ,(1997)
Stefan Kopp, Paul A. Tepper, Kimberley Ferriman, Kristina Striegnitz, Justine Cassell, Trading Spaces: How Humans and Humanoids Use Speech and Gesture to Give Directions Conversational Informatics: an Engineering Approach. pp. 133- 160 ,(2007) , 10.1002/9780470512470.CH8
Justine Cassell, Matthew Stone, Hao Yan, Coordination and context-dependence in the generation of embodied conversation Proceedings of the first international conference on Natural language generation - INLG '00. pp. 171- 178 ,(2000) , 10.3115/1118253.1118277