iSign: An Architecture for Humanoid Assisted Sign Language Tutoring

作者: Hatice Kose , Neziha Akalin , Rabia Yorganci , Bekir S. Ertugrul , Hasan Kivrak

DOI: 10.1007/978-3-319-12922-8_6

关键词:

摘要: This paper investigates the role of interaction and communication kinesics in human-robot interaction. It is based on a project Sign Language (SL) tutoring through games with humanoid robots. The aim study to design computational framework, which enables motivate children problems (i.e., ASD hearing impairments) understand imitate signs implemented by robot using basic upper torso gestures sound turn-taking manner. framework consists modular components endow capability perceiving actions children, carrying out game or storytelling task any desired mode, i.e., supervised semi-supervised. Visual (colored cards), vocal (storytelling, music), touch (using tactile sensors communicate), motion (recognition implementation including signs) cues are proposed be used for multimodal between robot, child therapist/parent. We present an empirical exploratory investigating effect non-verbal consisting hand movements, body face expressed having comprehended word, will give relevant feedback SL visually according context game.

参考文章(35)
Amy Bruckman, Zahoor Zafrulla, Thad Starner, Kimberly A. Weaver, Peter Presti, Harley Hamilton, Helene Brashear, Improving the language ability of deaf signing children through an interactive American sign language-based video game international conference of learning sciences. pp. 306- 307 ,(2010)
Michael Kipp, Alexis Heloir, Quan Nguyen, Sign language avatars: animation and comprehensibility intelligent virtual agents. pp. 113- 126 ,(2011) , 10.1007/978-3-642-23974-8_13
Hideki Kozima, Cocoro Nakagawa, Yuriko Yasuda, Children-robot interaction: a pilot study in autism therapy. Progress in Brain Research. ,vol. 164, pp. 385- 400 ,(2007) , 10.1016/S0079-6123(07)64021-7
Nicoletta Adamo-Villani, A Virtual Learning Environment for Deaf Children: Design and Evaluation World Academy of Science, Engineering and Technology, International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering. ,vol. 1, pp. 700- 705 ,(2007)
Cem Keskin, Lale Akarun, Oya Aran, Sign Language Tutoring tool european signal processing conference. pp. 1- 4 ,(2005)
Neziha Akalin, Pinar Uluer, Hatice Kose, Gokhan Ince, Humanoid robots communication with participants using sign language: An interaction based sign language game advanced robotics and its social impacts. pp. 181- 186 ,(2013) , 10.1109/ARSO.2013.6705526
Bekir Sıtkı Ertuğrul, Cemal Gurpinar, Hasan Kivrak, Hatice Kose, None, Gesture recognition for humanoid assisted interactive sign language tutoring 2013 21st Signal Processing and Communications Applications Conference (SIU). pp. 1- 4 ,(2013) , 10.1109/SIU.2013.6531542
Hatice Kose-Bagci, Kerstin Dautenhahn, Dag S. Syrdal, Chrystopher L. Nehaniv, Drum-mate: interaction dynamics and gestures in human-humanoid drumming experiments Connection Science. ,vol. 22, pp. 103- 134 ,(2010) , 10.1080/09540090903383189